de-kAn-'str&kt

Dear Sony ATV Publishing, UMPG Publishing, and other concerned parties,

I ask you to please withdraw your copyright violation notice on my video, “PSY – GANGNAM STYLE (?????) M/V (YouTube SmashUp)” as I believe my use of any copyrighted material is protected under Fair Use or Fair Dealing. This video was created by an automated process as part of an art project developed during my PhD at Goldsmiths, University of London: http://pkmital.com/home/projects/visual-smash-up/ and http://pkmital.com/home/projects/youtube-smash-up/

The process which creates the audio and video is entirely automated meaning the accused video is created by an algorithm. This algorithm begins by first creating a large database of tiny fragments of audio and video (less than 1 second of audio per fragment) using 9 videos from YouTube’s top 10 list. From this database, the tiny fragments of video and audio are stored as unrelated pieces of information and described only by a short series of 10-15 numbers. These numbers represent low-level features describing the texture and shape of the fragment of audio or video. These tiny fragments are then matched to the tiny fragments of audio and video detected within the target for resynthesis, in this case the number one YouTube video at the time, “PSY – GANGNAM STYLE (?????) M/V”.

To reiterate, the content from the target video, “PSY – GANGNAM STYLE (?????) M/V”, is not used in the resulting synthesis. That is, the process is creating a new video by not merely copying the target video, but attempting to re-create it out of entirely different material, the remaining 9 top 10 YouTube videos. Abstractly, there may appear to be a similar form or structure due to the collection of many fragments organized in a similar way as the target for resynthesis. These fragments however are from a very large collection of very different material to the original content’s own material. The content used in the resynthesis itself is only from the large database of tiny fragments of audio and video segmented from 9 other videos. As a result, I would argue the use of any content within this video is only through Fair Use or Fair Dealing of the content.

This art project’s purpose is towards highlighting an important aspect of how computers and humans perceive and how copyright itself may be dealt with within a computational arts practice which by its nature has to make use of existing content. The nature of this work further seeks to transform existing material into something entirely different such that experiencing a resynthesized video reveals a new understanding of one’s own perception. The amount of the content used is fragmented in nature and assembled using a coarse idea of audiovisual scene understanding with no notion of semantics. As a result, the video itself is very abstract and at times incomprehensible. Further, its effect on the publisher’s marker as noted by the very low view rate on YouTube is marginal at best. I therefore ask you to please withdraw your copyright claim.

Sincerely,
Parag K. Mital

Related: Copyright Violation Notice from “Rightster”, Intention in Copyright, EFF Wins Renewal of Smartphone Jailbreaking Rights Plus New Legal Protections for Video Remixing, YouTube’s Copyright School

[UPDATE Dec 8, 2012: All copyright violation notices have been dropped and the video is publicly accessible.]

[...]

I’ve been interested in exploring ways of navigating media archives. Typically, you may use iTunes and go from artist to artist, or have managed to tediously classify your collection into genres. Some may still even browse their music through a file browser, perhaps making sure the folders and filenames of their collection are descriptive of the artist, album, year, etc… Though what about how the content actually sounds?

Wouldn’t it be nice to hear all music which shares similar sounds, or similar phrases of sounds? Research in the last 10-15 years have developed methods precisely to solve this problem and fall under the umbrella term content-based information retrieval (CBIR) algorithms, or uncovering the relationships of an archive through the information within the content. For images, Google’s Search by Image is a great example which only recently became public. For audio, audioDB and ShaZam are good examples of discovering music through the way it sounds, or the content-based relationships of the audio itself. Though, each of these interfaces present a list of matches to a image or audio query, making exploring the content-based relationships of a specific set of material difficult.

The video above demonstrates interaction with a novel 3D browser of a collection of music by one artist, Daphne Oram. The sounds are grouped in 3D space based on the way they sound, clustering together similar sounding material. Each of the 3 axes describes a grouping of sound frequencies. So a timbre, or a texture of sound. The position of the sound along one of these axes means there is a lot of that group of frequencies present in the sound file.

Exploring her work in this browser really demonstrates the variety of sounds she achieved. It also makes exploring the collection really fun to use, as there is a fun visual form, and you also get to hear stuff right away.

The browser has also been built to be a real-time tool for creating new sounds. Mousing over any of the tiny boxes (representing parts of audio files in the collection) triggers the clip to play. Since similar sounding clips are grouped closer together, one can “perform” the collection along perceptually coherent axes by moving the mouse along any of the axes.

[...]

From November 21st to the 2nd of December, I’ll have the pleasure to lead a course and workshop with Prayas Abhinav at the Center for Experimental Media Arts in the Srishti School of Design in Banaglore, IN.  Many thanks to Meena Vari for all her help in organizing the project.

Stories are flowing trees

Key words:  3D, interactive projects, data, histories, urban, creative coding, technology, sculpture, projection mapping

Project Brief:

Urban realities are more like fictions, constructed through folklore, media and policy. Compressing these constructions across time would offer some possibilities for the emergence of complexity and new discourse. Using video projections adapted for 3D surfaces, urban histories will become data and information – supple, malleable, and material.

The project will begin with a one week workshop by Parag Mital on “Creative Coding” using the openFrameworks platform for C/C++ coding”.

About the Artists:

Prayas Abhinav

Presently he teaches at the Srishti School of Art, Design and Technology and is a researcher at the Center for Experimental Media Arts (CEMA). He has taught in the past at Dutch Art Institute (DAI) and Center for Environmental Planning and Technology (CEPT).
He has been supported by fellowships by Openspace India (2009), TED (2009), Center for Media Studies (CMS) (2006), Public Service Broadcasting Trust (PSBT) (2006), Sarai/CSDS (2005). He has presented his projects and proposals in the last few years at Periferry, Guwahati (2010), Exit Art, New York (2010), Futuresonic, Manchester (2009), Wintercamp, Amsterdam (2009), 48c: Public Art Ecology (2008), Khoj (2008), Urban Climate Camp, ISEA (2008), Sensory Urbanism, Glasgow (2008), First Monday, Chicago (2006), The Paris Accord (2006) and PSBT/Prasar Bharti (2006).
He has also participated in the exhibitions Myth ?? Reality (2011) at The Guild, Mumbai, Continuum Transfunctioner (2010) at exhibit 320 in Delhi, Contested Space – Incursions (2010) at Gallery Seven Arts in Delhi and Astonishment of Being (2009) at the Birla Academy of Art and Culture in Kolkatta (2009).

http://prayas.in

Parag K Mital (London)

Parag K Mital is an American-born London-based PhD-student in Arts and Computational Technology at Goldsmiths, University of London working on augmented realities and audiovisual resynthesis. As an audiovisual installation artist, his work encourages the audience to directly question the processes surrounding perception through introspection and curiosity from experiencing real-time models of audiovisual perception. His work has traveled extensively in London, Athens, and Moscow, including the London Science Museum and the British Film Institute. As an educator he has taught at Edinburgh University, Goldsmiths, University of London, and is due to deliver a course on Audiovisual Processing for iPhone/iPad at the Victoria & Albert Museum in London.

http://pkmital.com

Workshop in “Creative Coding” using the openFrameworks platform

The workshop will cover the basics of openFrameworks, a c/c++ creative coding platform. This course will also introduce students to digital signal processing techniques in synthesis and analysis of audio and visual signals for interactive techniques using custom made libraries developed at Goldsmiths, University of London. Depending on interest, participants will also receive tutorial on developing for the iPhone/iPad in order to create real-time audiovisual apps.

http://www.openframeworks.cc
http://pkmital.com/home/code/
http://maximilian.strangeloop.co.uk/

[...]

Sometime last year I gave an interview for the ECHOES II project.

“ECHOES II aims to develop an adventurous technology- enhanced learning environment in which both typically developing children and children with Aspergers Syndrome at Key Stage 1 (ages 5-7) can explore and improve social interaction and collaboration skills. The environment will also serve as a tool for researchers, teachers, parents, and practitioners to investigate problems that children may encounter in specific social contexts and the ways in which those problems may be addressed.

The proposed technology-enhanced learning environment will combine existing technologies in new ways. With the active participation of user groups, we will combine interactive multitouch screens, gesture and gaze tracking, and intelligent agent-based context-sensitive interfaces to create a novel interactive multi-modal environment that can be adapted to the needs of specific individuals, and that can provide new ways of investigating and supporting the development of social skills in children.”

Needless to say, I didn’t get the position. However, I did get to play with creating an interactive interview using Flash in my stubborn attempt to never use Powerpoint (which I have failed in). Anyway, somehow I believed it would fit in the whole idea of using all possible modes of communication. Rather than a simple projection of the data, you can explore the presentation by moving the mouse around. And since I was 6000 miles away on the phone communicating to the interviewers, it seemed like they could have some fun while conversing.

Check it out here.

After you’ve played with it, check out this, and imagine how much sexier my next interview presentation will be.

[...]