Interactive Narrative Wall – Letters to José
Letters to José is a non-linear interactive narrative wall based on 27 letters, written and sent by a medical student between 1948 and 1957, to his younger brother in the air force. These letters are not only a biographic narrative account of the personal, professional and filial changes of the sender’s world, but a brief record of the social, cultural, and economic shifts of Colombia during the first half of the 20th century.
In a physical/digital hybrid form of a playable story, the events described in Letters to José are represented in a series of interactive, multimodal, unfolding story worlds that combine distinctive paper mechanisms with embodied, visual, and auditory modes. The interplay between these modes, as part of the interface or the narrative, as well as the dual role of the interactor-performer, have diverse implications on the performative experience of the playable story itself due to its multimodal nature where the performer is both a spectator and protagonist.
PhD Designer Candidate from the School of Design, The Hong Kong Polytechnic University Kowloon, Daniel Echeverri says that his idea was that people get to touch, place, and manipulate different aspects of story world to move forward in a narrative. It was all inspired by some letters his grandpa sent to his brothers. Although right now they are developing the first prototype, in reality, the whole project is made from 3 of these paper worlds, the biggest one being 1.5 m wide and 42 cm tall.
For this interactive narrative wall, they’re using the Touch Board Pro Kit, as it has all these tools, plus ready to use screen printed sensors with Electric Paint, which were great for early prototyping and experimentation.
From a technical point of view, they are using the Touch Board to drive touch, sound, and visuals. They have a Pro micro 5v16M Leonardo-based board connected to the Touch Board that drives a PN532 NFC RFID Module tag reader. The 5v16M converts each RFID code embedded inside a tag into a ”fake” secondary touch board pin, giving them the chance to add more than 12 tracks of sound.
Daniel and Huaxin also connected a PAM8403 amplifier that outputs sound to two separate iPhone speakers, allowing them to have really good sound quality. Each speaker outputs sound to one side of the paper world. Everything is powered by a 1800mAh 3.7 V Li-ion battery that is charged using inductive charging through a Qi PCB module board, also connected to the Touch Board.
Daniel is using Bare Conductive Electric Paint to screen print some words over the illustrations that people can touch to activate sound and hear certain parts of the story. All the electrodes are connected with copper tape, combined with conductive fabric ”hinges” in parts where the paper world folds, to avoid breaking the copper.
The challenges they faced were mainly around the technical part and process. Finding a way to create an ecosystem around the Touch Board (using the RFID reader, the amplifier, the additional Leonardo, and so on) took a lot of time, especially sorting out the flat form factor of the paper world.
But more than technical, the real challenges are in terms of communicating with the users. Since people are not used to interacting through many modes of interaction at the same time, there is a very steep learning curve; people assume things like touching as if it was a screen, or not paying attention to the text, or the voice. Daniel spent most of the time, building the right script, so people could hear the instructions and follow them to play and interact with the story.
Daniel shared the first results of the interactive narrative wall with us and they seem very interesting! People engage with these kinds of stories better, it becomes a lot more immersive and fun, while giving them a lot of space to design better interactive experiences.
Images & Video: Daniel Echeverri, Huaxin Wei
If you’d like to see your project on our blog, email us at: email@example.com