Signal and Noise
User-generated installation: 16mm + depth-sensing camera | sound | 2022
Signal and Noise is an interactive installation that translates motion into sound and studies the movement patterns of an individual as they react to stimuli.
Custom software was created to track hand movements across an XYZ plane as a participant moves in front of the projection surface. Different hand positioning determines the sound piece playing and its volume. If there is no participant, there is no noise. Empty space is made reactive as reach participant moves and reacts in singular patterns.
The piece began with research into how to translate visual music into physical movement and position hand-painted 16mm film within an experiential environment.
The depth-sensing camera splits the projection surface into nine zones that become three-dimensional areas, existing from the surface of the screen 12 inches forward. The program is given eighteen (this could be any number) individual phrases of sound and randomly cycles through them at a provided interval. Sound pieces don't belong to any specific zone and shift between all of them. Each time a piece is played, the program will "destroy" it before it is played again through any combination of manipulating the speed, pitch, or static content. As the 16mm film continually loops through a projector and loses parts of its structure, the sound pieces do as well.
Work is still be done with PoseNet, a machine learning software, to study and identify the patterns created by a body's movement in reaction to user-generated sound.