Sublimate: State-Changing Virtual and Physical Rendering to Augment Interaction with Shape Displays

Recent research in 3D user interfaces pushes towards immersive graphics and actuated shape displays. Our work explores the hybrid of these directions, and we introduce sublimation and deposition, as metaphors for the transitions between physical and virtual states. We discuss how digital models, han...

Full description

Bibliographic Details
Main Authors: Leithinger, Daniel, Olwal, Alex, Luescher, Samuel, Lee, Jinha, Ishii, Hiroshi, Follmer, Sean Weston, Hogge, Akimitsu G.
Other Authors: Massachusetts Institute of Technology. Center for Transportation & Logistics
Format: Article
Language:en_US
Published: Association for Computing Machinery (ACM) 2013
Online Access:http://hdl.handle.net/1721.1/80421
https://orcid.org/0000-0003-3407-8722
https://orcid.org/0000-0003-4918-8908
Description
Summary:Recent research in 3D user interfaces pushes towards immersive graphics and actuated shape displays. Our work explores the hybrid of these directions, and we introduce sublimation and deposition, as metaphors for the transitions between physical and virtual states. We discuss how digital models, handles and controls can be interacted with as virtual 3D graphics or dynamic physical shapes, and how user interfaces can rapidly and fluidly switch between those representations. To explore this space, we developed two systems that integrate actuated shape displays and augmented reality (AR) for co-located physical shapes and 3D graphics. Our spatial optical see-through display provides a single user with head-tracked stereoscopic augmentation, whereas our handheld devices enable multi-user interaction through video seethrough AR. We describe interaction techniques and applications that explore 3D interaction for these new modalities. We conclude by discussing the results from a user study that show how freehand interaction with physical shape displays and co-located graphics can outperform wand-based interaction with virtual 3D graphics.