About the project
This interdisciplinary project, combining scientific, technological and artistic methods, proposes three crucial topics for future development of multimodal devices:
- the importance of action-sound couplings for both performance and perception of music
- the control potential of human bodily movement
- a move towards an active music experience
Research questions and methods
The basic research question that underlies this project is:
-
How can action-sound couplings be exploited in human-computer interaction?
A number of sub-questions are linked to this:
- Which sensor technologies can be used to capture complex body movement?
- Which machine-learning techniques can be used to extract semantic actions from a continuous stream of movement data?
- How are actions and sounds coupled in everyday life?
- How can such theories and technologies be used to control sound and music?
- How can we create musical structures that can be controlled by the user?
Aims and objectives
Principal objective:
- Exploring action-sound couplings in human-computer interaction.
Sub-goals:
- Develop sensor technologies for capturing complex body movement.
- Develop machine-learning techniques and segmentation methods for extracting semantic actions from a continuous stream of sensor data.
- Develop theories of action-sound couplings in everyday life.
- Develop prototypes of enactive media devices that allow for continuously controlling music based on the actions of the user.
- Develop hypermusic structures that can be used in the enactive media devices.
Cooperation and financing
Sensing Music-related Actions is a joint research project of the departments of Musicology and Informatics, and has received external funding through the VERDIKT program of the The Research Council of Norway. The project runs from July 2008 until July 2011.