Cymatics
The Deaf and Hard of Hearing communities are perhaps the most marginalized in the world of music. Cymaspace is a Portland, Oregon non-profit that facilitates musical performances and concerts and focuses on making technologies that translate sound into lights and vibration for non-hearing musicians and audiences. I collaborate with Oregon State senior capstone students from Music, Mechanical, Electrical, and Computer Science disciplines, alongside Cymaspace staff, to help create new multi-sensory interfaces.
Cymatics is the translation of sound into other sensations. Synesthesia is a natural phenomenon where human senses are effectively crossed, causing some people to see color when they hear sound, or to taste or smell things when presented with a color. We hope to produce technologies that enable translating sound into color, shape, and brightness; and sound into vibration. Below are some projects and initiatives we’ve built together.
AudioLux
AudioLux is a device that represents music and audio signals on an LED strip in real-time. It can be plugged into a 3.5mm headphone jack, or use an onboard microphone as input. The audio signal is analyzed for qualities like dominant frequency, present frequencies, and volume. Based on these parameters, the LEDs’ position, brightness, and colors are changed in real-time. Preset patterns can be cycled by the user, allowing them to find ones that fit the song well. Additionally, new patterns could be programmed onto the device. An accompanying phone app allows the user to have control over the system, without being physically near the device AudioLux is to increase accessibility for the people with hearing impairment in live music and other artistic performances and to have the potential for use in education for encouraging kids’ interest in STEM and having DIY expandability for at-home builders.
VibroSonics
One of the most exhilarating aspects of experiencing live music events is “feeling” the sound pump against your skin. What if you could experience this sensation from your own playlist anywhere you go? Vibrosonics translates audio from the spectrum of human hearing into the tactile sub-audio range to drive vibrations through the body. This is accomplished through a unique audio encoding process.
What would be some of the benefits of this technology? Ludwig Van Beethoven was famous for being a deaf composer who overcame the condition of his ears by using touch on the piano. This technology could enable Deaf and Hard of Hearing individuals to produce, dance with, and experience music in new ways; gamers in VR-AR to feel and react to more aspects of their environment, and workers in high-noise situations feel critical sonic cues to help them avoid accidents.
Oregon State Senior Capstone teams from electrical engineering and computer science we working to produce the first prototype in Fall 2022 under my mentorship and with Cymaspace.
