Envision a future in which the shape of interactive devices is not fixed but can dynamically reconfigure. Imagine downloading a game on your mobile phone and watching the device automatically reshape into a console by curling to facilitate grasping or by producing pop-up physical joysticks on its surface. Although one might expect to encounter such a future in science fiction, recent trends in research have shown that it is close to becoming science fact. The days of smartphones, and other interactive devices, looking like bricks is coming to an end and we are now entering an age where they are malleable and able to transform into arbitrary shapes on demand.
Shape-changing devices are not just for entertainment, they also enhance the interaction between the physical and digital world. Consider, for example, a shape-changing map in the classroom of the future. When learning about the formation of continents, mountain ranges or river systems, students will be able to physically interact with and mold surfaces in order to experience the effect of geological and environmental forces. At the heart of these capabilities is the emergent affordances and analytical reasoning that shape-changing devices provide. In everyday life we interact with objects of various forms (water bottles, pens, door handles and so on) whose specific shapes give us cues on where and how to grip, operate and interact with them. This is a notion that is often referred to as ‘affordance’ [1], meaning the properties of an object that tell us how it wants to be used or helps us to use it. Shape-changing devices provide emergent affordances by dynamically adapting the shape of an interactive device to generate appropriate physical cues for the user.
Currently there are several researchers within the Human Computer Interaction (HCI) community that are interested in designing, prototyping and evaluating such shape-changing devices to make this vision a reality. The Interaction and Graphics group of the University of Bristol has been exploring this through prototypes such as Morphees [2] or the TiltDisplay [3] that have been created as part of an on-going EU funded FET-Open project [4]. Realizing this vision, however, requires tackling lots of research challenges in materials science, engineering and HCI. For example, on the materials science side, breakthroughs in stable and accessible materials are required to create novel proof-of-concept devices; while on the HCI side, we need to have a deep appreciation for the material's properties, and thoroughly consider how those properties can provide affordances that unleash the human interaction potential. While these challenges are interesting for the respective research communities we believe that the true power of shape-changing devices can be magnified many-fold by bringing together the two communities. Through communication, collaboration and coordination of research, we can reach further and faster resulting in an acceleration of research that proves that both together can be much greater than the sum of their parts.
For example, smart memory alloys and electro-active polymer research in Europe have been developing apace, particularly in laboratory-based experiments that develop our understanding of the underlying fundamentals of such materials and showcase their abilities in proof-of-principle experiments. But most of the requirements for materials properties are derived from industrial requirements (like automation and transportation), which are very different from interactive device requirements. Typical proof-of-principle experiments from materials science often fail to consider the rich plethora of interaction possibilities afforded by the user. For instance, electro-active polymers have been shown to have high strain when actuated but typically suffer from low strength. While this may be a limitation in some industrial requirements, with good design they can be harnessed to create prototypes of interactive devices. But creating these designs requires the use of a user-centered design process (UCD), a research skill that is typical of HCI groups.
UCD involves end-users during the entire design, implementation and evaluation process and gathers device and material requirements not through a single requirement-gathering phase but through an iterative design process. This allows us to expose end-users to novel materials and elicit design suggestions by not only drawing on the benefits of new materials but also exposing the user to its potential limitations. These material properties which are seen as limitations can then be turned into strong design benefits of the device. Such is the case of the famous Post-it® Notes. The adhesive, created by Dr. Spence Silver at 3M, was too weak for any industrial applications, but a UCD process resulted in the creation of an application that turned the adhesive's limitation into a benefit.
In summary, starting from the vision of the ultimate display technology by Sutherland in 1965 [5], there has been a growing amount of literature on shape changing objects. The practical implementation of shape-changing interfaces is still far off (estimates range from 15 to 20 years) due to a lack of fundamental knowledge in creating interactive systems and in understanding the basic science of supporting interactions with such systems. We strongly believe that we have reached a point in the design of these devices where we need a real synergy between research teams from multiple fields, all involved to accelerate the realization of this vision. In other words our message is ‘let's talk and work together!’.
Further reading
[1] Don A. Norman, The Psychology of Everyday Things. Basic Books, New York, 1988. In paperback as The Design of Everyday Things. Doubleday, New York, 1990.
[2] A. Roudaut, A. Karnik, M. Löchtefeld, S. Subramanian, Morphees: toward high “shape resolution” in self-actuated flexible mobile devices, Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI’13), ACM, New York, NY, USA (2013), pp. 593–602
[3] J. Alexander, A. Lucero, S. Subramanian, Tilt display demonstration: a display surface with multi-axis tilt & actuation, Proceedings of the 14th International Conference on Human–Computer Interaction with Mobile Devices and Services Companion (MobileHCI’12), ACM, New York, NY, USA (2012), pp. 213–214
[4] http://www.ghost-fet.com/
[5] Ivan E. Sutherland, The ultimate display, Proceedings of IFIP Congress (1965), pp. 506–508 (Retrieved 22.09.11)