Extending museum exhibits by embedded media content for an embodied interaction experience

Investigation topic

Nowadays, museums not only collect, categorize, preserve and present; a museum must also educate and entertain, all the while following market principles to attract visitors. To satisfy this mission, they started to introduce interactive technologies in the 1990s, such as multimedia terminals and audio guides, which have since become standard for delivering contextual information. More recently there has been a shift towards the creation of personalized sensorial experiences by applying user tracking and adaptive user modeling based on location-sensitive and context-aware sensor systems with mobile information retrieval devices. However, the technological gadgets and complex graphical user interfaces (GUIs) generate a separate information layer and detach visitors from the physical exhibits. The attention is drawn to the screen and the interactive technology becomes a competing element with the environment and the exhibited collection [Stille 2003, Goulding 2000, Wakkary 2007]. Furthermore, the vast majority of visitors comes in groups and the social setting gets interrupted by the digital information extension [Petrelli 2016].

First studies about museum visitor behavior were carried out at the end of the 19th and during the 20th Century [Robinson 1928, Melton 1972]. More recently, a significant body of ethnographic research about visitor experience of single persons and groups has contributed studies about technologically extended and interactive installations. Publications about visitor motivation, circulation and orientation, engagement, learning processes, as well as cognitive and affective relationship to the exhibits are of interest for our research approach [Bitgood 2006, Vom Lehn 2007, Dudley 2010, Falk 2011]. Most relevant are studies of the Human Computer Interaction (HCI) researcher community in the fields of Ubiquitous Computing, Tangible User Interfaces and Augmented Reality, investigating hybrid exhibition spaces and the bridging of the material and physical with the technologically mediated and virtual [Hornecker 2006, Wakkary 2007, Benford 2009, Petrelli 2016].

Approach

At the Institute of Experimental Design and Media Cultures (IXDM) we have conducted several design research projects applying AR for cultural applications but got increasingly frustrated with disturbing GUIs and physical interfaces such as mobile phones and Head Mounted Displays. We therefore started to experiment with Ubiquitous Computing, the Internet of Things and physical computing technologies that became increasingly accessible for the design community during the last twelve years because of shrinking size and price of sensors, actuators and controllers. In the presented research project, we therefore examine the extension of museum exhibits by physically embedded media technologies for an embodied interaction experience. We intend to overcome problems of distraction, isolation and stifled learning processes with artificial GUIs by interweaving mediated information directly into the context of the exhibits and by triggering events according to visitor behavior.

Our research approach was interdisciplinary and praxis-based including the observation of concept, content and design development and technological implementation processes before the final evaluations. The team was composed of two research partners, three commercial/engineering partners and three museums, closely working together on three tracks: technology, design and museology. The engineering partners developed and implemented a scalable distributed hardware node system and a Linux-based content management system. It is able to detect user behavior and accordingly process and display contextual information. The content design team worked on three case studies following a scenario-driven prototyping approach. They first elaborated criteria catalogues, suitable content and scenarios to define the requirement profiles for the distributed technological environment. Subsequently, they carried out usability studies in the Critical Media Lab of the IXDM and finally set up and evaluated three case studies with test persons. The three museums involved, the Swiss Open-Air Museum Ballenberg, the Roman City of Augusta Raurica and the Museum der Kulturen Basel, all have in common that they exhibit objects or rooms that function as staged knowledge containers and can therefore be extended by means of ubiComp technologies. The three case studies were thematically distinct and offered specific exhibition situations:

• Case study 1: Roman City of Augusta Raurica: “The Roman trade center Schmidmatt“. The primary imparting concept was “oral history”, and documentary film served as a related model: An archaeologist present during the excavations acted as a virtual guide, giving visitors information about the excavation and research methods, findings, hypotheses and reconstructions.

• Case study 2: Open-Air Museum Ballenberg: “Farmhouse from Uesslingen“. The main design investigation was “narratives” about the former inhabitants and the main theme “alcohol”: Its use for cooking, medical application, religious rituals and abuse.

• Case study 3: Museum der Kulturen Basel: “Meditation box“. The main design investigation was “visitor participation” with biofeedback technologies.

Technological development

This project entailed the development of a prototype for a commercial hardware and software toolkit for exhibition designers and museums. Our technology partners elaborated a distributed system that can be composed and scaled according to the specific requirements of an exhibition. The system consists of two main parts:

• A centralized database with an online content management system (CMS) to setup and control the main software, node scripts, media content and hardware configuration. After the technical installation it also allows the museums to edit, update, monitor and maintain their exhibitions.

• Different types of hardware nodes that can be extended by specific types of sensors and actuators. Each node, sensor and actuator has its own separate ID; they are all networked together and are therefore individually accessible via the CMS. A node can run on a Raspberry Pi, for example, an FPGA based on Cyclone V or any desktop computer and can thus be adapted to the required performance.

The modular architecture allows for technological adaption or extension according to specific needs. First modules were developed for the project and then implemented according to the case study scenarios.

Evaluation methods

Through a participatory design process, we developed a scenario for each case study, suitable for walkthrough with several test persons. Comparable and complementary case study scenarios allowed us to identify risks and opportunities for exhibition design and knowledge transfer and define the tasks and challenges for technical implementation. For the visitor evaluation, we selected end-users, experts and in-house museum personnel. The test persons were of various genders and ages (including families with children), had varying levels of technical understanding and little or no knowledge about the project. For each case study we asked about 12 persons or groups of persons to explore the setting as long as they wanted (normally 10–15 minutes). They agreed to be observed and video recorded during the walkthrough and to participate in a semi-structured interview afterwards. We also asked the supervisory staff about their observations and mingled with regular visitors to gain insight into their primary reactions, comments and general behavior. The evaluation was followed by a heuristic qualitative content analysis of the recorded audio and video files and the notes we took during the interviews. Shortly after each evaluation we presented and discussed the results in team workshops.

Findings and Conclusions

The field work lead to many detailed insights about interweaving interactive mediated information directly into the context of physical exhibits. The findings are relevant for museums, design researchers and practitioners, the HCI community and technology developers. We organized the results along five main investigation topics:

1. Discovery-based information retrieval

Unexpected ambient events generate surprise and strong experiences but also contain the risk of information loss if visitors do not trigger or understand the media aids. The concept of unfolding the big picture by gathering distributed, hidden information fragments requires visitor attentiveness. Teasing, timing and the choice of location are therefore crucial to generate flowing trajectories.

2. Embodied interaction

The ambient events are surprising but visitors are not always aware of their interactions. The unconscious mode of interaction lacks of an obvious interaction feedback. But introducing indicated hotspots or modes of interactions destroys the essence of the project’s approach. The fact that visitors do not have to interact with technical devices or learn how to operate graphical user interfaces means that no user groups are excluded from the experience and information retrieval.

3. Non-linear contextual information accumulation

When deploying this project’s approach as a central exhibition concept, information needs to be structured hierarchically. Text boards or info screens are still a good solution for introducing visitors to the ways they can navigate the exhibition. The better the basic topics and situations are initially introduced, the more freedom emerges for selective and memorable knowledge staged in close context to the exhibits.

4. Contextually extended physical exhibits

A crucial investigation topic was the correlation between the exhibit and the media extension. We therefore declined concepts that would overshadow the exhibition and would use it merely as a stage for storytelling with well-established characters or as an extensive media show. The museums requested that media content fade in only shortly when someone approached a hotspot and that there were no technical interfaces or screens for projections that challenged the authenticity of the exhibits. We also discussed to what extend the physical exhibit should be staged to bridge the gap to the media extension.

5. Invisibly embedded technology

The problem of integrating sensors, actuators and controllers into cultural heritage collections was a further investigation topic. We used no visible displays to leave the exhibition space as pure as possible and investigated the applicability of different types of media technologies.

Final conclusion

Our museum partners agreed that the approach should not be implemented as a central concept and dense setting for an exhibition. As often propagated by exhibition makers, first comes the well-researched and elaborated content and carefully constructed story line, and only then the selection of the accurate design approach, medium and form of implementation. This rule also seems to apply to ubiComp concepts and technologies for knowledge transfer. The approach should be applied as a discreet additional information layer or just as a tool to be used when it makes sense to explain something contextually or involve visitors emotionally.

References

Steve Benford et al. 2009. From Interaction to Trajectories: Designing Coherent Journeys Through User Experiences. Proc. CHI ’09, ACM Press. 709–718.

Stephen Bitgood. 2006. An Analysis of Visitor Circulation: Movement Patterns and the General Value Principle. Curator the museum journal, Volume 49, Issue 4,463–475.

John Falk. 2011. Contextualizing Falk’s Identity-Related Visitor Motivational Model. Visitors Studies. 14, 2, 141-157.

Sandra Dudley. 2010. Museum materialities: Objects, sense and feeling. In Dudley, S. (ed.) Museum Materialities: Objects, Engagements, Interpretations. Routledge, UK, 1-18.

Christina Goulding. 2000. The museum environment and the visitor experience. European Journal of marketing 34, no. 3/4, pp. 261-278.

Eva Hornecker and Jacob Buur. 2006. Getting a Grip on Tangible Interaction: A Framework on Physical Space and Social Interaction. CHI, Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 437-446.

Dirk vom Lehn, Jon Hindmarsh, Paul Luff, Christian Heath. 2007. Engaging Constable: Revealing art with new technology. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '07), 1485-1494.

Arthur W. Melton. 1972. Visitor behavior in museums: Some early research in environmental design. In Human Factors. 14(5): 393-403.

Edward S. Robinson. 1928. The behavior of the museum visitor. Publications of the American Association of Museums, New Series, Nr. 5. Washington D.C.

Daniela Petrelli, Nick Dulake, Mark T. Marshall, Anna Pisetti, Elena Not. 2016. Voices from the War: Design as a Means of Understanding the Experience of Visiting Heritage. Proceedings Human-Computer Interaction, San Jose, CA, USA.

Alexander Stille. 2003. The future of the past. Macmillan. Pan Books Limited.

Ron Wakkary and Marek Hatala. 2007. Situated play in a tangible interface and adaptive audio museum guide. Published online: 4 November 2006. Springer-Verlag London Limited.

The spaces we inhabit and interact with on a daily basis are made up of layers of cultural activity that are, quite literally, built up over time. While museum exhibits, archaeological narratives, and public programs communicate this heritage, they often don’t allow for the public to experience interactive, place-based, and individually driven exploration of content and spaces. Further, designers of public heritage and archaeology programs rarely explore the binary nature of both the presented content and the scholarly process by which the understanding of that content was reached. In short, the scholarly narrative of material culture, heritage, and archaeology is often hidden from public exploration, engagement, and understanding. Additionally, many traditional public heritage and archaeology programs often find it challenging to negotiate the balance between the voice and goals of the institution and those of communities and groups. In recent years, the maturation of mobile and augmented reality technology has provided heritage institutions, sites of memory and memorialization, cultural landscapes, and archaeological projects with interesting new avenues to present research and engage the public. We are also beginning to see exemplar projects that suggest fruitful models for moving the domain of mobile heritage forward considerably.

University campuses provide a particularly interesting venue for leveraging mobile technology in the pursuit of engaging, place-based heritage and archaeology experiences. University campuses are usually already well traveled public spaces, and therefore don’t elicit the same level of concern that you might find in other contexts for publicly providing the location of archaeological and heritage sites and resources. They have a built in audience of alumni and students eager to better understand the history and heritage of their home campus. Finally, many university campuses are starting to seriously think of themselves as places of heritage and memory, and are developing strategies for researching, preserving, and presenting their own cultural heritage and archaeology.

It is within this context that this paper will explore a deeply collaborative effort at Michigan State University that leverages mobile technology to build an interactive and place-based interpretive layer for campus heritage and archaeology. Driven by the work of the Michigan State University Campus Archaeology Program, an internationally recognized initiative that is unique in its approach to campus heritage, these efforts have unfolded across a number of years and evolved to meet the ever changing need to present the rich and well studied heritage and archaeology of Michigan State University's historic campus.

Ultimately, the goal of this paper is not only to present and discuss the efforts at Michigan State University, but to provide a potential model for other university campuses interested in leveraging mobile technology to produce engaging digital heritage and archaeology experiences.

In this paper I discuss the multisensory digital interface and art installation Zelige Door on Golborne Road as part of the wider research project ‘Mapping Memory Routes: Eliciting Culturally Diverse Collective Memories for Digital Archives’. The interface is conceived as a tool for capturing and displaying the living heritage of members of Moroccan migrant communities, shared through an artwork composed of a digital interactive sensorial map of Golborne Road (also known as Little Morocco), which includes physical objects related to various aspects of Moroccan culture, each requiring a different sense to be experienced (smell, taste, sight, hearing, touch). Augmented Reality (AR) and olfactory technologies have been used in the interface to superimpose pre-recorded video material and smells to the objects. As a result, the neighbourhood is represented as a living museum of cultural memories expressed in the form of artefacts, sensory stimulation and narratives of citizens living, working or visiting the area. Based on a model I developed for the multisensory installation ‘Streets of...7 cities in 7 minutes’, the interface was designed with Dr Mariza Dima (HCI designer), and Prof. Monica Bordegoni and Dr Marina Carulli (olfactory technology designers) to explore new methods able to elicit cultural Collective Memories through the use of multi-sensory technologies. The tool is also aimed at stimulating collective curatorial practices and democratise decision-making processes in urban planning and cultural heritage.