songyun-tao.jpg

Physical therapy (PT) is a crucial and highly beneficial activity that supports recovery from acute injury and generally promotes mobility and functioning throughout the lifespan. However, adherence to at-home PT is a major challenge given boredom, motivation, and difficulties tracking progress. Considering over half the population could benefit from PT, enhancing these experiences is an important design challenge. Our paper introduces an approach that leverages the engaging, therapeutic power of music to provide enjoyable, real-time feedback and adaptive guidance during PT. Specifically, informed by the music therapy literature and a highly iterative user-centered design process, our system tracks and sonifies limb movements using state-of-the-art computer vision and music generation. Our lab-based evaluation shows this musical approach improves exercise performance and user engagement compared to traditional practices. These findings offer a blueprint for music-mediated PT and suggest broader design directions for interactive musical technologies that enhance health management experiences.

On this page

Songyun Tao

Projects

MusicalPT