Publications

Grasping is one of the most important factors of human evolution and even young children start to explore the world by dominantly using touch. However, current interaction technologies suffer from an imbalance between visual and haptic stimulation.

Shape-changing interfaces are one line of research addressing this issue. Unfortunately, these systems are often very complex and difficult to replicate. This is not the case for Elastic Displays that have low mechanical complexity and utilize consumer hardware. Albeit their simple setup, neither off-the-shelf hardware for research purposes or instructions on how to assemble such systems are available. This hampers open science and the replication of research results.

In this paper, we describe our experiences on constructing an Elastic Display. We present lessons learned from several field tests and the final requirements of the setup. Moreover, we introduce an open source documentation of the hardware assembly instructions, which integrates with a previously published open source software framework. With both hardware and software now being readily available, novel application concepts for Elastic Displays can be explored by a broad community.

Elastic Displays are shape-changing interfaces, typically made from elastic fabric. Surface deformation creates an additional dimension of interaction and enhances the haptic experience for the user compared to rigid touch interfaces. Other forms of shape-changing interfaces such as actuated displays introduce significant mechanical complexity with the benefit of providing active haptic feedback and self-deformation. We investigate ways to make Elastic Displays more 'active' while maintaining their simple setup. Our approach adds vibrotactile feedback to the elastic surface, which add explicit haptic feedback when pushing and pulling the surface.

In this contribution, we describe the technical setup and introduce concepts for exploring this active haptic feedback. We realized a case study, in which a layered information space is explored. Finally, we report lessons learned from user feedback, showing that solely vibrotactile feedback may have distracting effects on the user, but in combination with visual feedback can improve the user experience.

Elastic Displays as a specific type of Shape-Changing Interfaces provide haptic feedback by using the deformation of the screen for interaction. There have been several prototypes showcasing different use cases, visualization techniques and interaction metaphors.

In this paper, we present an overview over our efforts to create a modular and extendable software framework to support research in the context of this specific type of interface. The main goal is to provide a common and easy starting point to streamline research on this technology.

The ReFlex framework is based on a service-oriented client-server architecture and uses modern web technologies for platform-independent development. Furthermore, it provides tools for configuration and calibration to simplify the hardware setup. Development tools as well as plugins and templates for common client technologies provide a starting point for rapid prototyping concepts for Elastic Displays.

Finally, we provide insights on our experiences with framework as part of student projects and describe our ideas for future research on possible framework extensions and technological enhancements in the context of Elastic Displays.

Within the last years, several use-cases for Elastic Displays have been explored, showing the potential for this novel technology. However, one of the shortcomings of this type of interface is that it is not possible to provide active haptic feedback to the user. This feature is limited to actuated displays, which in turn are much more complex to build, more expensive, less robust and portable.

In this paper we describe another approach to provide basic active haptic feedback for Elastic Displays by using vibration motors. We explore different options for vibrotactile feedback and describe the results of a student project in which some of the concepts were realized as part of a small game prototype.

Creating innovative applications for novel interface technologies poses several challenges for both designers and developers. Rapid prototyping helps to examine concepts, check hypotheses, and acquire early user feedback. In this paper, we present ReFlex, a framework that provides rapid prototyping and development tools for Elastic Displays, and its application in the context of a student project developing a spatial controller for remote 3D navigation.

Elastic displays afford a natural stacking of information layers in their haptic interaction space. So far, research has focused on technical issues and interaction concepts by pushing and pulling the surface of such displays.

We report a study with 24 participants that investigates the feasibility and limitations of interaction with 6 to 21 layers. We measured completion times and error rates for reaching and holding specific target layers and observed performed gestures. Our findings show that solution time increases with more layers while precision is maintained with a mean success rate of holding a layer of 70 %, even when using up to 21 layers. User experience measurements show that hedonic qualities of the interaction were rated higher than pragmatic qualities.

Our investigation proves the general feasibility of using layer-based interaction with an elastic display and provides guidelines on the limitations.

Any design endeavor requires practitioners to engage in divergent thinking. Numerous creativity techniques and methods exist to support the design process and give structure in order to converge a wide variety of ideas into suitable candidates for prototyping and testing. Designing for specific, tangible interaction technologies requires a careful framing of the selected methods.

Over the course of 10 years, our research group has gained experience in designing and developing applications for elastic displays. In this pictorial paper, we illustrate methods and intermediate results for a concrete application design. Due to the Covid19 pandemic, we made extensive use of the collaborative whiteboard application miro. We designed a way to visualize, explore, and evaluate clustering algorithms in the domain of machine learning using an elastic display. Finally, we present follow-up techniques for prototyping the concepts and ideas.

Tangible Interaction in collaborative scenarios represents a natural and intuitive way to explore and discuss complex issues in a playful way, supporting communication, negotiation and teamwork. We present a concept for Elastic Displays to support decision-making processes in the context of civil engineering construction planning, related to the research context of Building Information modelling techniques. This involves the exploration of spatial issues, as well as identifying changes to the project schedule and cost plan.

We use the power of Elastic Displays to quickly explore semantic and time layers in complex maps and use them in conjunction with an integrated overview of the project timeline and cost overview. Furthermore, we discuss some gestures used for interacting with the Zoomable User Interface and navigate Layered Visualizations with magic lenses. Finally, we discuss our findings from the implemented prototype and share some thoughts on future work for Zoomable UIs on Elastic Displays.

Shape-changing interfaces are addressed by the research community with more scientific vigor in recent years. Elastic displays afford novel ways to interact with data and have also been investigated for almost a decade now. Our goal is to move this interaction technology out of its experimental stage and into more sophisticated real-world scenarios.

In this work-in-progress paper, we describe our research agenda and preliminary results. On the one hand, we aim to consolidate and extend fundamental knowledge about the interaction and visualization techniques that are suitable and effective for elastic displays. Our main goals in this respect are the development and evaluation of a gesture language and the introduction of an online knowledge base. On the other hand, we investigate specific use cases that leverage the unique properties of elastic displays. For this leg of our research, we carry out a long-term observation of a museum installation of an elastic display and devise new applications in a research project with industrial partners.

Building and maintaining knowledge about specific interface technologies is a challenge. Current solutions include standard file-based document repositories, wikis, and other online tools. However, these solutions are often only available in intranets, become outdated and do not support the acquisition of knowledge in an efficient manner. The effort to gain an overview and detailed knowledge about novel interface technologies can be overwhelming and requires to research and read many technical reports and scientific publications. We propose to implement open source online knowledge bases that include building blocks for creating custom workshops to understand and apply the contained knowledge.

We demonstrate this concept with a knowledge base for shape-changing interfaces (SCI-KB). The SCI-KB is hosted online at GitHub and fosters collaborative creation of knowledge elements accompanied by practical exercises and workshop elements that can be combined and adapted by individuals or groups of people new to the topic of shape-changing interfaces.

Novel shape-changing interfaces promise to provide a rich haptic experience for human-computer interaction. As a specific instance of shape-changing interfaces, Elastic Displays provide large interaction surfaces that can be temporally deformed using force-touch. The unique property of these displays is that they automatically return to their initial flat state. Recently, several review and position papers have stimulated a discussion towards consolidating the knowledge about shape-changing interfaces. The knowledge about Elastic Displays is similarly scattered across multiple publications from recent years.

This paper contributes a task taxonomy based on productive uses of Elastic Displays found in literature, on the web, and in our interaction lab. This taxonomy emphasizes tasks, but also encompasses general aspects regarding content types, visualization technology, and interaction styles. All aspects of the taxonomy are illustrated using case studies from literature.

Im Rahmen des vorliegenden Papers soll ein kurzer Abriss über verschiedene Prototypen für Zoomable User Interfaces (ZUI) auf Elastic Displays erfolgen, welche in den letzten Jahren realisiert wurden. Auf Basis dieser verschiedenen Systeme soll der Versuch einer Kategorisierung erfolgen, um die Bandbreite der Möglichkeiten unter Berücksichtigung der Spezifika von Elastic Displays aufzuzeigen. Abschließend werden als Ausblick auf zukünftige Arbeiten die Chancen und Herausforderungen in der Umsetzung solcher Systeme dargelegt.

In this paper, we present an interaction and visualization concept for elastic displays. The interaction concept was inspired by the search process of a rummage table to explore a large set of product data. The basic approach uses a similarity-based search pattern — based on a small set of items, the user refines the search result by examining similar items and exchanging them with items from the current result. A physically-based approach is used to interact with the data by deforming the surface of the elastic display. The presented visualization concept uses glyphs to directly compare items at a glance. Zoomable UI techniques controlled by the deformation of the elastic surface allow to display different levels of detail for each item.

In this paper, we propose a concept to help data analysts to quickly assess parameters and results of cluster algorithms. The presentation and interaction on a flexible display makes it possible to grasp the functioning of algorithms and focus on the data itself. Two interaction concepts are presented, which demonstrate the strength of elastic displays: a layer concept that allows the recognition of differences between various parameter settings of cluster algorithms, and a Zoomable User Interface, which encourages the in-depth analysis of clusters.

Interaktive Bilder stellen eine besondere Klasse der Lösungsbilder dar. Sie werden nicht nur betrachtet und interpretiert, sondern auch verändert. Sie verkörpern Zustände eines Prozesses der Lösungsfindung. Für die Interaktion stehen heute zahlreiche Technologien bereit. Das Feld der Möglichkeiten reicht von Blicksteuerung bis zur berührungsbasierten Interaktion. Anders ausgedrückt: das Spektrum der Technologien ist grundsätzlich zwischen den Wahrnehmungsmodalitäten Visualität und Haptik einordenbar. Die polaren Begriffe Echtzeit und Modellzeit spannen eine weitere Dimension auf, um Interaktionsformen zu ordnen. Während Echtzeit-Interaktion auf Immersion beruht, erzeugt Interaktion in einem von Echtzeit unabhängigen Zeitregime ein emersives Verhältnis zum Interaktionsgegenstand.

In den letzten Jahren hat sich die Forschung im Bereich der Mensch-Computer-Interaktion zu alternativen Displays weiter verstärkt. Nachdem die Multitouch-Technik als etabliert anzusehen ist, erweitern Forscher durch Konzepte und entsprechende Prototypen die Möglichkeiten zur direkten Interaktion mit digitalen Inhalten immer weiter. Elastische Displays nehmen dabei eine prominente Rolle ein. Bisher fehlen jedoch erprobte Richtlinien und Gestaltungsempfehlungen sowie Beispiele für produktive Anwendungen. In diesem Beitrag werden zwei Anwendungsfälle und entsprechende Interaktions- und Visualisierungskonzepte für elastische Displays vorgestellt.

Elastic displays empower users to interact naturally through push- ing and pulling, folding and twisting. While this kind of interaction is not as precise as on other devices, it utilizes interaction metaphors which are easy to learn and understand. We present a system that uses physically based interaction and visualization metaphors to gain a deeper comprehension of the underlying data and its structure. By applying pressure on specific interface elements, associated items are attracted and repelled, the exerted force on the items itself translates into a semantic zoom behavior to display more in-depth information about the specific entity. We present the core concepts of the system, explain the decisions made during the design process and discuss the advantages and disadvantages of the proposed system as well as a short view on further improvements and open research questions.

Der vorliegende Beitrag betrachtet den Einsatz des Digitalen Engineerings mittels mobiler, elastischer und interaktiver Oberflächen als Interface. Dabei wird die Evolution der Forschungsprototypen, beginnend von einem Tabletop mit verformbarer Oberfläche über eine elastische Projektionswand hinzu einem mobilen, Tablet-ähnlichem Prototypen mit interaktivem, deformierbaren Display aufgezeigt. Jeder dieser Gerätetypen verfügt aufgrund seiner Bauweise über spezifische Eignungskriterien abhängig vom entsprechenden Interaktionsszenario. Zudem sind die Verformung und deren Übertragung auf die Datenvisualisierung abhängig von den zugrundeliegenden Daten- und Interaktionsmodellen. Im Kontext des durchgängigen Digitalen Engineerings zum Planen, Testen und Betreiben technischer Systeme bedeutet dies die Möglichkeit, die Interaktion mit komplexen Daten zu vereinfachen, die Visualisierung großer Datenmengen auf intuitiv navigierbare semantische Ebenen aufzuteilen und neue Formen der Interaktion mit dreidimensionalen Objekten zu erschließen. 3D-Modelle und komplexe Zusammenhänge werden damit für den Nutzer erfahrbarer und leichter begreifbar.

By their deformable screen-materials elastic displays and projection screens provide physical three- dimensional interaction modalities like push, pull or bend. Compared with conventional Multi-Touch displays they offer an additional interaction dimension which can be used to explore data. In this article we describe the FlexiWall, a large elastic display, and several example applications using layered data sets. The exploration of several layers and correlations between these is not common to traditional user interfaces, where the interaction is often constrained to two dimensions. Therefore, new forms of interaction are introduced. We furthermore propose additional techniques and tools to explore layered data sets, e.g. utilizing transparent objects when interacting with elastic displays.

Elastic displays offer new ways to interact with multi-dimensional data by using the deformation of the surface as a tool to explore, filter, structure, or manipulate data. While a large number of prototypes exist, a general concept for using this promising technology in real-world application domains has not been established. In this paper, we introduce a framework about elastic displays and their applications with reference to the interaction techniques they provide. We investigate the data applicable to elastic displays and the appropriate interaction techniques. Using this approach, it is possible to identify strengths and weaknesses of this technology regarding specific scenarios, to find commonalities to traditional user interfaces and to explore novel concepts for interaction.

Der technische Fortschritt und die gegenwärtigen Neuerungen, insbesondere im Bereich der computergrafisch generierten Schnittstelle zwischen Mensch und Maschine, unter anderem vor dem Hintergrund relevanter Fragestellungen an der „Schwelle zur vierten industriellen Revolution“ (Industrie 4.0), erfordern die Integration der interdisziplinären Grundlagenforschung in die Betrachtungen zu konkreten Anwendungsgebieten. Entsprechend relevanten Forschungsfragen geht unter anderen die Nachwuchsforschergruppe Cognitive Interface Technologies (CogITo) nach, bestehend aus Wissenschaftlern der Fachrichtungen Maschinenbau, Psychologie, Informatik, Design und Architektur. Gegenstand der Forschung sind verschiedene Aspekte von Mensch-Maschine-Schnittstellen, bezogen auf Fragestellungen im Kontext des „Digital Engineering“. Dabei werden unter anderem technische Systeme fokussiert, die dynamisch auf die Wahrnehmungsmöglichkeiten des menschlichen Nutzers reagieren, vielmehr seine kognitiven Fähigkeiten aber auch Limitationen berücksichtigen. Die Grenzen der Wahrnehmung stellen dabei ein besonderes Potenzial für interaktive Systeme dar, insbesondere in Hinblick auf Aspekte wie Subtilität, Immersion, Emotion, etc. Ziel ist es, die Effizienz und die Nutzbarkeit vorhandener Systeme durch Einbeziehung und Übertragung wahrnehmungspsychologischer Erkenntnisse zu verbessern.

In this article the idea of music composing with an elastic tabletop including tangible objects embedded into its surface is described. An elastic display dynamically creates several physical shapes representing different music samples for interaction. The shapes can be activated and manipulated by the user to arrange the associated samples. The key benefits of this form of interaction are the intuitive use, the ability to playfully explore music and the expressiveness of the physical representation of sound.

This position paper describes our view on the current state of research in the domain of elastic displays. Elastic displays exhibit a number of unique properties: they enable varied haptic feedback, maintain consistency of shape, and afford natural physics. The main contribution of this paper is a systematic review of current technological and conceptual issues for enabling elastic displays. We discuss shortcomings that hamper a readily adoption of elastic displays and propose three directions for future research: elastic gestures, hooks, and gravibles.

In this paper we describe DepthTouch, an installation which explores future interactive surfaces and features elastic feedback, allowing the user to go deeper than with regular multi-touch surfaces. DepthTouch's elastic display allows the user to create valleys and ascending slopes by depressing or grabbing its textile surface. We describe the experimental approach for eliciting appropriate interaction metaphors from interaction with real materials and the resulting digital prototype.