Research Snapshots

GRAND integrates leading digital media laboratories and research centres at universities across Canada – our “Centres of Excellence.”

Research Snapshots - GRAND

GRAND’s highly interdisciplinary research explores digital games, interactive media, user interface design, human-computer interaction, and new media technologies for use by consumers, researchers, and industry. We also study the impact of digital media on education and learning, intellectual property, privacy and security, business and commerce, sustainability, arts and culture, health and wellness, and the workplace.

In the following video we present snapshots of some of the many diverse and innovative projects going on in GRAND. More information about the research can be found via the links below.


Personal Cockpit
Advanced head-mounted mobile display

B. Ens, R. Finnegan, P. Irani (University of Manitoba)

The Personal Cockpit is a head-mounted, mobile display that is projected onto the world around us. Using head and body motions, the device allows users to navigate through a configurable dashboard of virtual displays that surround the immediate space – similar to a cockpit’s instrument panel. Data and visual information - for example, a street map on Google Maps or a personal calendar - can be conveniently placed anywhere within the projection and accessed when needed.



OrMiS (Orchestrating Military Simulations)
Tabletop interface for simulation-based training

C. Bortolaso, E. Ingle, M. Oskamp, N. Graham (Queen's University)

The Canadian military makes extensive use of simulations to train officers in making strategic decisions. Working with the Army Simulation Centre in Kingston, Ontario, researchers at Queen's University have developed a multi-touch tabletop simulation environment that is efficient, easy to use, and supports a shared coordination of military units. OrMiS gives detailed, interactive views of aerial maps with zoom capabilities, while maintaining an overview of the battlefield. Beyond military applications, other industry partners are exploring uses of the mapping software for next-generation simulation tools.



Radiant Soil
Living architecture and responsive spaces

P. Beesley, R. Gorbet (University of Waterloo), et al.

Radiant Soil is a striking biomimetic sculpture by artist and architect Philip Beesley (University of Waterloo). Composed of tens of thousands of lightweight digitally fabricated components that are fitted with microprocessors, the work recreates the architecture of living systems. Approaching visitors set off bursts of light, triggering chains of motion – flexing, breathing, swallowing – that ripple throughout the environment. Scent-emitting “glands” attract viewers and encourage interaction. Radiant Soil was presented as part of the 2013 ALIVE / EN VIE exhibition in Paris, France.



CBC NewsWorld Holodeck
Immersive augmented reality display

M. Ladly, David G. (OCAD University), P. Zhou, C. Chen, R. Tanyag, P. Chintraruck (OCAD University), G. Penn (University of Toronto), Rouzbeh Farahmand

For over seventy years, the CBC has given voice to our unique Canadian perspective on the world, producing a phenomenally rich multimedia record of our social, political and cultural heritage. The CBC NewsWorld Holodeck brings this record to life in an immersive “day-by-day” experience featuring video and audio recordings from the NewsWorld corpus dating back to 1989. Through voice- and screen-activated queries, users can search and browse video and audio segments of national and international news, human-interest stories, political events, arts and culture, sports, weather, and advertising.



Bicycle navigation through tactile feedback

B. Huxtable, H. Lai, J. Zhu, P. Lam, Y. Choi, C. Neustaedter, G. Corness (School of Interactive Arts + Technology Simon Fraser University)

Ziklo is a touch interface for wayfinding devices designed for cyclists. Two vibrating wristbands wirelessly connected to the cyclist's mobile device signal left and right turns based on directions given on a mobile app. The wristbands allow for different vibration patterns and strengths for sending different notifications. The hands-free design minimizes interference during riding.



Kinect sensor modified for wheelchair gaming 

K. Gerling, M. Kalyn, R. Mandryk. (University of Saskatchewan)

By modifying a Microsoft Kinect sensor, researchers at the University of Saskatchewan have shown how gamers in wheelchairs can play motion-based games. Called KINECTWheels, the software library gives game developers the functionality to design games with wheelchair gesture input, making even off-the-shelf PC games wheelchair-accessible. As well as children and adults, the research is aimed at introducing older adults to wheelchair gaming as a form of exercise. KINECTWheels was presented at CHI 2013 in Paris, France.



Exer-gaming for children with Cerebral Palsy   

H. Hernandez, N. Graham, L. Switzer, Z. Ye, M. Hamza, C. Savery, T. Stach (Queen's University), D. Fehlings (Holland Bloorview Kids Rehabilitation Hospital), et al.

Liberi is a multiplayer exer-game therapy for children with Cerebral Palsy (CP). As children with CP reach adolescence, their muscle strength fails to increase in proportion to their body growth; some lose the ability to walk with a walker and transition to using a wheelchair, which contributes to poor physical fitness and social isolation. Regular play sessions with the cycle-based Liberi give teenagers with CP a vigorous cardiovascular and muscle strengthening workout that also encourages lively social interaction. Liberi began as a joint project between GRAND and the NeuroDevNet NCE.



Tangible programming game for kids

C. Neustaedter, P. Mun-Yee Lam, C. Ka-Ho Lai, Y. Choi, B. Huxtable, J. Castro, A. Hawryshkewich. (School of Interactive Arts + Technology, Simon Fraser University)

Loopo helps introduce computer programming to primary school children. Part tangible device and part computer interface, this fun interactive game is specially designed to engage young learners. Various learning activities have players arrange physical blocks in a computer-linked template to program changes and actions that appear on a screen. The physical interaction encourages students to collaborate and learn together, while teaching them the fundamentals of computer programming in a relatable,and interesting way. 



Collaborative, multi-touch sustainability game

A. Antle (School of Interactive Arts + Technology, Simon Fraser University), A. Wise (Faculty of Education, SFU), A. Hall (Faculty of Education, SFU), et al. 

YouTopia is a collaborative two-player tabletop game designed to introduce young children to sustainable development. A hybrid of digital and physical board games, players work collaboratively to develop natural resources for building dams, farms and housing. Physical stamps are used to mark different land uses on a large tabletop map. As the game progresses, the impact of their activities on the environment, and in meeting population needs, is shown. Designed to have no winners or losers, the game instead encourages discussion and shared decision-making to build a world both players can live in.


Shape-changing smartphone  

A. Gomes, A. Nesbitt, R. Vertegaal. (Queen's University)  

MorePhone is not your typical smartphone. Made with a thin, flexible electrophoretic display manufactured by Plastic Logic, the flat phone can alter its shape to give users silent but visual cues of incoming phone calls, text messages, or emails. Beneath the phone's display lie shape memory alloy wires that contract when activated allowing the phone to curl its entire body, or up to three individual corners. Each corner can be tailored to convey a particular message – for example, the top right corner may bend when receiving a text message, and the bottom right corner when receiving an email.


Vibrotactile music for the hearing-impaired 

D. Fels, G. Smith, F. Russo, et al. (Ryerson University)   

Emoti-chair offers a multi-sensory way for the deaf and hearing-impaired to experience sounds and music. Designed to emulate the human cochlea – the auditory portion of the inner ear – each chair is equipped with hardware and software that work together to transmit varying levels of frequency to the user. Through vibrations in voice coils embedded throughout the chair, sounds of different instruments or musical notes stimulate the user's senses. Additional rocking motions and blasts of air enhance the experience.



Embodied Gameplay   

L. Hughes, B. Simon, J. Tingley (Concordia University), et al.

Propinquity (meaning “proximity”) is a two-player sensor-based gestural game that combines the rhythms and movements of dancing and fighting games. Focused on the actions of the body rather than the screen, Propinquity uses sound and game mechanics to produce an intensely social and physical experience. Players wear soft patches with proximity sensors and a glove that provides tactile feedback. As players move to music, each comes as close as possible to his or her partner's patches to score points. The game has been displayed in Montreal, San Francisco, Boston, Brussels, and Paris, and at IndieCade 2013 in Los Angeles.



Context-aware communication aid   

R. Baecker, K. Fenwick, M. Massimi, S. Black, D. Ryan (University of Toronto), et al.   

With most smartphones equipped with sensors to detect location, researchers at the University of Toronto's TagLab have developed a context-aware application called “MyVoice” (later branded "Talk Rocket Go).” Based on the location of the user, the app suggests a vocabulary (words, phrases, and sentences) appropriate for possible encounters (such as cup sizes at a coffee shop). The software can assist those who experience difficulties in recalling names or places – for example, individuals with anomic aphasia, which often results from strokes, certain types of dementia, and brain injuries.



Visual Text
Descriptive non-photorealistic rendering    

M. Chang, C. Collins (University of Ontario Institute of Technology)  

Descriptive non-photorealistic rendering combines 3-D renderings with text and data visualization. This type of visualization emphasizes both the relative significance of words in the text as well as their physical real-world relationships. Interactive widgets and direct multi-touch interaction with the 3-D models also allow for a deeper data analysis. In this example, users can explore vehicle complaint reports from National Highway Traffic Safety Administration (NHTSA) to compare the reliability of different makes and models, find interesting facts, and reveal possible causal relations between car parts.



Interactive Surfaces
Multi-touch information visualization

S. Carpendale, U. Hinrichs (University of Calgary), et al.  

University of Calgary's InnoVis lab is developing tools and techniques to transform the flood of data we encounter daily into something accessible and understandable for everyone. Directed by Dr. Sheelagh Carpendale, an internationally renowned and award-winning innovator in information visualization and interactive displays, InnoVis is examining approaches to successfully designing more natural, accessible, and people-centred technologies. Lab projects encompass applications in learning, financial management, healthcare, and sustainability. It is also one of the few labs in the world developing interactive tabletop displays, helping to revitalize tabletop research and put Alberta on the map in the field of human-computer interaction.



Full-body interactive Tetris 

F. Chevalier, D. Freeman (University of Toronto), Kyle Duffield (Artist), K. Hartman, D. Reilly, E. Westecott (OCAD University), et al.    

Tweetris is a Kinect-based two-player game involving the whole body. Players contort their bodies to match Tetris brick shapes or “tetrominos” displayed on a screen. Snapshots of the players in position are then tweeted to become tetrominos for another game of Tetris played in real-time at a nearby kiosk, or on mobile devices. The collaborative art project involved researchers at the Digital Futures Initiative (OCAD University), computer scientists at the Dynamic Graphics Project Lab (University of Toronto), and independent artists. Launched at Nuit Blanche as part of Toronto’s 2011 LEITMOTIF exhibition, Tweetris has entertained participants at Digifest, TEI 2012 as well as Halifax's 2012 Nocturne: Art at Night festival, featured in the video clip.



Paper-thin flexible tablet     

R. Vertegaal, J. Burstyn, D. Holman, K. Kim, P. Strohmeier, A. Tarun, P. Wang, D. Wightman (Queen’s University), A. Girouard (Carleton University), D. Reilly (OCAD University and Dalhousie University)  

PaperTab is a thin flexible tablet that looks and feels like sheets of paper. Developed at the Queen’s University Human Media Lab in partnership with Intel Labs and Plastic Logic, the invention offers a robust, shatterproof alternative to conventional glass-based displays. Its touchscreen interface, powered by the Intel i5 processor, allows users to navigate through a document by bending the corners of the tablet, similar to the turning of pages in a magazine. It includes several interactive features: by tapping one tablet to another, a user can instantly transfer data, or by placing two tablets side by side, enlarge an image across both screens. The device was officially introduced at the 2013 International Consumer Electronics Show (CES) in Las Vegas.



3-D Stereoscropic Cinema
Exploring variable high-frame rates (HFR) 

S. Arden, A. Goldman, M. Lantin, D. Quesnel, R. Overington (Emily Carr University of Art + Design)

At Emily Carr University's Stereoscopic 3D (S3D) Centre, researchers are exploring variable frame rates with the production of L'âme Soeur (Soul Mate) 3D.  The short film incorporates scenes at 24, 48, and 60 fps that demonstrate the creative potential of variable frame rates within for both 3DTV and cinema. By showing fine movement detail, HFR productions can create a different viewing experience: instead of more resolution in the image there is more resolution in time. For the creative community, the research evaluates how the technology can be used to tell compelling stories.



Implicit Skinning
Real-time surface deformation   

B. Wyvill (University of Victoria), R. Vaillant (Université de Toulouse, University of Victoria), L. Barthe, O. Gourmel, M. Paulin (Université de Toulouse), G. Guennebaud (Inria Bordeaux), M. Cani (Grenoble Universités - Inria), D. Rhomer (CPE Lyon - Inria)

A challenge when animating a virtual character is the production of realistic skin contact and muscular bulges at the joints in a process known as 'skinning.' Canadian and French researchers have developed the first purely geometric method for handling these effects in real-time: a geometric mesh is first approximated by a set of “implicit surfaces” that are combined in real-time without any loss of detail and without intensive computation. The result is a seamless handling of contacts between skin parts that fits into the standard animation pipeline.



FORK-1S (First Order Reduced Compliant System)
Simulation of multi-body mechanisms

S. Andrews, P.G. Kry (McGill University), M. Teichmann (CM Labs Simulations)

Real-time physics simulation is now a fundamental component of immersive virtual environments. These environments have important applications, such as in the training of heavy equipment operators and the simulation of virtual humans in video games. Computational techniques developed at McGill University have improved the speed and behaviour of simulations for complex multi-body mechanisms involving numerous joints and points of contact, for example, the simulation of human hands during grasping.



Wrinkling Simulation
High resolution surface wrinkling

O. Remillard, P.G. Kry (McGill University)   

Wrinkles are important visual details that appear on the surface of deformable objects, such as cushions or elastic materials. In computer graphics, there exists a range of approaches to modeling and animating wrinkling. At McGill University, researchers are pioneering a technique of embedding thin shells to simulate highly detailed wrinkling of different kinds of objects with soft interiors and hard skins. Possible applications include the wrinkles on animated characters, or imprints on upholstered furniture.



Immersive virtual environment to manage stress

J. Mamisao, J. Giordano, S. Derochie, M. Karamnejad, D. Gromala, C. Shaw (School of Interactive Arts + Technology, Simon Fraser University), B. Ludlow (OCAD University)

Sensorium is an immersive virtual reality system designed to help users learn how to manage stress. Using Galvanic Skin Response (GSR) technology that measures skin conductance (an indicator of stress), biofeedback controls change immersive visuals and sounds based on the relaxation or anxiety of the user. Developed by a team of student designers, the project is part of the Transforming Pain Research Group (TPRG) at Simon Fraser University’s School of Interactive Arts and Technology (SIAT). TPRG intersects pain medicine, computer science, communications, and design in the development of innovative treatments for chronic pain.