Turning Objects into AR Musical Instruments
An object-centred augmented reality musical interaction system that transforms everyday physical objects into expressive sound controls for live performance, embodied improvisation and playful musical discovery.
Overview
This project investigates how familiar objects can become musical interfaces inside an augmented reality environment. Physical objects sit at the centre of the interaction: a cup, box or other everyday item can be recognised, augmented and performed as an instrument with its own spatial behaviour and sonic identity.
The experience was developed as a research-driven creative technology project for SMCCLab. It combines augmented reality visual feedback, embodied gesture, object affordances and real-time sound synthesis, giving musical control clear visual relationships, tactile cues and strong live readability.
Design and technical route
The system was structured around three layers: a Unity-based AR scene for spatial interaction and visual feedback, an object-centred interaction model that maps physical movement to musical behaviour, and a sound layer built around real-time synthesis and parameter control. The design prioritised clear relationships between action and sound, so that visual movement, hand gesture and audio response could be understood as one continuous performance gesture.
The live presentation translated the prototype into a stage context, positioning the system as an interface that can be watched, understood and performed. The work connects extended reality, HCI, creative coding and sound interaction through one coherent system of interface design, technical implementation and performance thinking.
Gallery
Outcome
- Built a working object-centred augmented reality music system for live performance and embodied interaction.
- Connected spatial movement, object affordance and real-time sound mapping into a readable performance interface.
- Presented the system through research and performance material at ANU SMCCLab.