Method and toolkit for designing digital musical instruments: generating ideas and prototypes

Abstract

Last decade witnessed a considerable rise in physical, programmable, interactive artifacts. Sensors, devices, platforms and frameworks have become more accessible and more people are programming the physical world beyond the screen. Interactive devices for artistic expression present challenges that are worth investigating because the interaction often needs a high level of skill that is hard to be obtained. Therefore, interactive artistic approaches can teach valuable lessons applicable to other levels of interaction design and human-computer interaction. One class of artistic, physical interactive objects is the digital musical instrument (DMI). DMIs are artifacts in which gestural control and sound production are physically decoupled but digitally mapped. It provides freedom for a DMI designer, since several combinations are possible, but increases the complexity of the design space. Besides, structured methods and guidelines that would help the design have not yet been established. To address this issue, prototyping seems to be a promising approach as they are not only a tool for testing and communicating ideas, but also for generating them. As a DMI is a means to produce music, its prototype should provide real-time sound feedback for control gestures. For that reason, in DMI context, non-functional prototypes are not entirely suitable. On the other hand, the development of functional prototypes demands more time and effort, and consequently, can be a bottleneck of iterative design. How to provide structured and exploratory paths to generate DMI ideas? How to decrease time and effort of building functional DMI prototypes? To deal with those questions, we propose the concept of instrumental inheritance, that is the application of gestural and/or structural components of existing instruments to generate ideas of new instruments. As support for analysis and combination, we leverage a traditional design method, the morphological chart, in which existing artifacts are split into parts, presented in a visual form and then recombined to produce new ideas. Finally, integrating the concept and the method in a concrete object, we developed a physical prototyping toolkit for building functional DMI prototypes: Probatio, a modular system of blocks and supports to prototype instruments based on certain ways of holding and gestural controls for musical interaction. The evaluation of the toolkit showed that it contributed to reducing the time to achieve a functional prototype, and also influencing the increase in the number of cycles of idea exploration. Besides, the users reported more musical engagement with Probatio in comparison to a generic sensor toolkit.