If computers could understand their context, how would we choose to communicate with them? If mixed reality could help us interact with light, what interactions would most easily indicate common commands? Our group has answered these questions through a case study in controlling audio content.

The UI elements may be made of light, but they behave like real objects. By extending the metaphor of physical objects to the user interface, we intuitively understand how to interact. A UI based on objects is native to the medium and superior in use - no more clumsy air taps in an abstracted 2D interface.

We built a prototype to test our interaction model and found that a UI placed lower in the field of vision was more comfortable than a traditional 2D interface floating at eye level.

We collectively worked on concept development for 8 weeks, then split into individual specialties for our final two weeks. I took on video responsibilities, including planning, equipment, shooting, editing, and post effects to integrate Derek’s 3D animations.

The technology powering mixed reality experiences is progressing rapidly, yet no sufficient interaction models have emerged in this field. Current models are ill suited to the medium, having primarily mimicked the 2-dimensional interfaces of the desktop world, simply projecting them in 3-dimensional space instead of creating interactions more suitable to the medium. The lack of practical interaction methods in this medium is a barrier to mainstream adoption.

The metaphors of the last generation of technology are an inevitable default condition in any new technological medium. The better option is to establish direct, natural metaphors early in this new medium’s development, ensuring the highest level of communication in the interaction of humans and their technology.

For a more academic study of metaphors in mixed reality, see my research paper on Metaphors of Physicality in Interaction Design