Projects
Stories are Flowing Trees | Nov 2011
Working at the Srishti School of Design‘s Center for Experimental Media, I have developed and delivered a workshop on creative coding to get students familiar with the openFrameworks and maximilian programming libraries for audio-visual DSP during their interim semester. The culmination of the project will be exhibited in the BAR1 gallery in central Bangalore and developed and curated in collaboration with artist and researcher, Prayas Abhinav.
Real-time Auditory Memory Mosaicing | Jun 2011
Memory Mosaicing is a new type of Augmented Sonic Reality that resynthesizes your sonic world using recorded segments of sound from the microphone. You can also add a song from your iTunes Library to the app’s memory, creating a mashup of sounds in your sonic environment based on your favorite music, techno-fying or hiphop-i-fying your world.
Oramics | Jun 2011
this project focused on an iPhone and Desktop (for the Science Museum in London) emulator which would try to bring the sound of Daphne Oram’s “Oramics Machine” to life, through the Oramics drawn sound technique. the interactive desktop app goes live in the science museum of london on july 29th, with the iPhone app being released soon after.
ECHOES | Jun 2011
ECHOES is a technology-enhanced learning environment where 5-to-7-year-old children on the Autism Spectrum and their typically developing peers can explore and improve social and communicative skills through interacting and collaborating with virtual characters (agents) and digital objects. ECHOES provides developmentally appropriate goals and methods of intervention that are meaningful to the individual child, and prioritises communicative skills such as joint attention.
Real-time Source Separation | Mar 2011
Separating foreground from background can also elicit a model of auditory saliency, or a model of what is likely to be important in the auditory stream of information. First a chunk of audio is trained as a background in real-time. Next, the audio is discretized into matrix factors through a number of maximum likelihood iterations (Expectation-Maximization) into 3 variables representing basis components in the 2D spectrum, their weights, and the impulses of where they occur. Foreground is a reprojection of this data onto additional components. This project works in real-time on an iPhone 4.
Real-time Binauralization | Feb 2011
this works extends the IRCAM Listen database for real-time cluster-based binauralization on the iPhone, allowing up to 30 sound sources to be spatialized in 3D in real-time on an iPhone 4, using the GPS, compass, and altitude information.
Bronze Format | Feb 2011
Bronze is a new non-interactive music format in which recorded material is transformed in real time, generating a unique and constantly evolving interpretation of a song on each listen.
Sound-seeer | Nov 2010
In collaboration with R. Beau Lotto of lottolab studios, and Mick Grierson of Goldsmiths, University of London’s Department of Computing, this project sought to allow children to design visual search experiments investigating the relationship of sound and vision. Setup during the november Science Museum of London LATES exhibition, and the i,Scientist 2011 program, participants were blindfolded and navigated a maze using the sound from this iPod app, which converted the camera image into spatialized sounds.
Sonic Graffiti | Nov 2010
sounds are placed in the city as graffiti using the iPhone 4′s GPS and microphone. the result is a sonification of the graffiti around you as a spatialized orchestra in 3D sound.
Geodesic Dome Projection Mapping | May 2010
Custom built software for interactive projection mapping of a geodesic dome. Dome design by Tom Clowney for the artist Cardboard.
Dynamic Images and Eye-Movements | 2008-2010
The DIEM Project (Prof John Henderson, Dr Robin Hill, Dr Tim Smith, Parag Mital) are developing new visualisation tools for eye movements in dynamic images, as well as new data analysis tools and techniques based on dynamic-regions-of-interest (DROIs) for use in film and video. We applied these new methods to investigate how people see and understand the visual world as depicted in film and video in developing a stronger theory of active visual cognition.
Attention: The Experimental Film | 2008
A collaboration between stefanie tan, dave stewart, and myself, “attention” explored how eye-tracking could be used to algorithmically edit new films, using sound and video databases. a pov 2×2 film shot in the style of michael figgis’s timecode was created alongside of additional wide/mid/and close-shot videos. sound bytes and narratives were also collected. a final installation of dual projection and quadrophonic audio was algorithmically edited in real-time based on viewers eye-tracking information.
Interactive Light Field Renderer | 2006
As part of a post-doctoral seminar I attended at the University of Delaware, I implemented bespoke software for a Light Field Renderer with support for aperture size, synthetic focal length, and translational motion of the virtual viewing camera. This project was under the direction of Dr. Jingyi Yu.
Gradient Domain Context Enhancement Using Poisson Integration | 2006
As part of a post-doctoral seminar I attended at the University of Delaware, I built bespoke software to correct either a highly saturated day-time video using video material from the night, or vice-versa correcting a overly dark night time video using material from the day time. This project was under the direction of Dr. Jingyi Yu.