Robert-Depot - User contributions [en]

WiiSound - Leilani Martin

2010-06-10T23:11:42Z

Leilanim14: /* Documentation */

== Title ==
WiiSound

== Description ==
*Motivation
I have a strong interest in music and manipulating sound. I wanted to create a different way to control sound by using a Wiimote as the main controller. It creates a more interesting interaction with the interface instead of just buttons or mousing everything.

*Interactive paradigm
Using Pure Data or Pd, I created a patch that controls different parameters of a playback loop. Using the Wiimote the user can control the volume, location, and different effects of the loop.

*Technical Description
Using a software called OSCulator, I connected the Wiimote to my computer through its routing parameters and bluetooth. Pd-extended contains a library called "mrpeach" that communicates the information gathered from OSCulator. Through several parsing methods I was able to connect different buttons and motions with specific controllers within my patch.

== Documentation ==
[[Image:main.png]]
[[Image:filters.png]]
[[Image:OSCcontrols.png]]
[[Image:panner.png]]
[[Image:OSCulator.png]]

== Video Demonstration ==
link: http://www.youtube.com/watch?v=CZF72EM_zLI

File:OSCulator.png

2010-06-10T23:02:57Z

Leilanim14:

File:Panner.png

2010-06-10T23:02:35Z

Leilanim14:

File:OSCcontrols.png

2010-06-10T23:02:17Z

Leilanim14:

File:Filters.png

2010-06-10T23:01:45Z

Leilanim14:

File:Main.png

2010-06-10T23:01:25Z

Leilanim14:

WiiSound - Leilani Martin

2010-06-10T23:00:58Z

Leilanim14: /* Description */

WiiSound - Leilani Martin

2010-06-10T22:58:45Z

Leilanim14: /* Title */

WiiSound - Leilani Martin

2010-06-10T22:54:29Z

Leilanim14: New page: == Title == MIDI Capture

== Title ==
MIDI Capture

Classes/2010/VIS145B

2010-06-10T22:53:31Z

Leilanim14: /* Final Projects */

==Related Links:==

[http://www.bodybuildingrevealed.com/'''body building''']

== Time and Process Based Digital Media II ==
Time: Thursdays 3:30-6:20pm, VAF 228

This class is an advanced study and portfolio project course centered on the use of hardware and software to create interactive and time-based art. These projects can take many forms—interactive installations, dynamic visualizations/sonifications, printed renderings—chosen by the students. This will not be a course of technical instruction—rather we will consider technical and conceptual issues in tandem, supplementing discussions and activities with specific technical instruction where necessary. There is a strong emphasis on the development and articulation of personal directions of research by the students in the course.

I would like to split the reading/homework responsibility for two parts of the class. In the first half of the term I will present a series of works and readings covering my particular interests--the intersections of social performance, embodied experience, and cognition. In the latter half of the class (after the midterm) you all will do the presentations on topics of your choosing. Working individually or in small groups, you will provide us with some conceptual provocation (reading material) covering topics you intend to engage with your final, and you will lead a discussion on technical and conceptual issues. Reading and critical writing, in response to text and works you present and those I present, are integral to this course.

The schedule is a living document and will be revised over the period of the course.

== Instructor ==
Robert Twomey

rtwomey@ucsd.edu
*http://roberttwomey.com
*http://experimentalgamelab.net
*http://crca.ucsd.edu

Office Hours: Wednesday 3-4pm, Atkinson Hall Rm 1601 (CRCA research neighborhood). Please e-mail me if you plan to attend.

== Grading ==
*Midterm Project - 30%
*Final Project - 40%
*Presentations - 10%
*Readings/Assignments/Homework - 10%
*Participation - 10%

=== Presentations ===
(1) Short presentation on your work in the second week of class. This should be a statement of your interests, direction, goals with media art. Present examples from your own work which you feel strongly about, and which best represent your interests and trajectory. Present examples of other artist's work that serve as models for the kind of work you would like to make. (5-10 minutes each)

(2) Medium presentation on final projects in the second semester of the course (weeks 7-9). This is the portion of the class where you dictate the reading and the discussion. If you are presenting on a given week, you need to provide us with a reading 1 week in advance. We will sign up for those time slots in week 6, just after the midterm. (10-15 minutes)

=== Reading Responses ===
These are written summaries and critical responses to materials assigned for out of class viewing. Things to consider: What points does the author make? Do you buy their assumptions or agree with their conclusions? Reading responses will be printed and turned in to the instructor at the beginning of class. Generally these should be 1 page long.

=== Projects ===
Midterm and final projects will be graded on concept, effort, and realization. Formal proposals are a necessary component of the process so take them seriously. Make the effort to get started early and seek the help you need--we want to see finished, well-considered pieces for the midterm and final. Additionally, you will need to submit documentation of the project after completion which includes images, video, and source code where applicable. These materials (proposals and documentation) will all be posted to the wiki.
=== Documentation Policy ===
*section on your project
*source code
*image/video documentation. 5 images or 5 videos.
*descriptive writing (on intent, motivation, context)

=== Attendance ===
Attendance is mandatory. Each unexcused absence will drop your final grade one letter. There are only 10 weeks of class, please come to them all.

== Schedule ==
=== Week 1 - Intro ===
*Introductions
*Scope of course, interests, technical possibilities.
*My work.
*Watch: We Live In Public. 2009. (excerpts)
*In class: personal page on wiki. [http://www.trsp.net/teaching/gamemod/ game-mod exercise]. [http://www.trsp.net/teaching/gamemod/gamemod_breakout_source_en.zip download link]
*Read: [http://www.nyu.edu/projects/xdesign/mainmenu/archive_tangible.html Against Virtualized Information], [http://www.nyu.edu/projects/xdesign/mainmenu/archive_analtictech.html Novel Analytic Techniques], and [http://www.nyu.edu/projects/xdesign/mainmenu/archive_infocounts.html What Information Counts?] by [http://www.environmentalhealthclinic.net/people/natalie-jeremijenko/ Natalie Jeremijenko].
*Read: [http://www.yalealumnimagazine.com/issues/2004_03/jeremijenko.html An Engineer for the Avante Garde]
*Read: [http://www.worldchanging.com/archives/001450.html Natalie Jeremijenko The WorldChanging Interview]
*Read: [http://tech90s.walkerart.org/nj/transcript/nj_01.html Database Politics and Social Simulations], good background on her earlier artwork.

=== Week 2 - Student Research Interests ===
*Due: 1 page on Jeremijenko.
*Presentations on your work.
*Read: [http://www.flong.com/texts/essays/essay_cvad/ Computer Vision for Artists and Designers: Pedagogic Tools and Techniques for Novice Programmers] Golan Levin. ''pay particular attention to part II. ELEMENTARY COMPUTER VISION TECHNIQUES. we are going to try these in class next week.''

=== Week 3 - Computer Vision / Human Perception ===
*Due: Nothing. Read the Golan Levin piece, but no written response.
*Discuss:
**Myron Kreuger. Video Place. 1989 [http://www.youtube.com/watch?v=dqZyZrN3Pl0]
**Text Rain. Camille Utterback & Romy Achituv. 1999. [http://www.youtube.com/watch?v=toWFvXHghDk] [http://www.camilleutterback.com/]
**Very Nervous System. 1982-1991. [http://vimeo.com/8120954]
**Suicide Box. Bureau of Inverse Technology. 1996. (13:00)
**Marie Sester. ACCESS. 2003. [http://accessproject.net]
**Messa di Voce. Golan Levin and Zach Lieberman with Jaap Blonk and Joan La Barbara. 2003. [http://www.flong.com/projects/messa/] [http://www.tmema.org/messa/messa.html]
**Seen. David Rokeby. 2002. [http://vimeo.com/6012986]
**Sorting Daemon. David Rokeby. 2003. [http://homepage.mac.com/davidrokeby/sorting.html]
**Cheese. Christian Moller. 2003. [http://www.christian-moeller.com/display.php?project_id=36] made in collaboration with UCSD [http://mplab.ucsd.edu/wordpress/ Machine Perception Lab]
**Eyewriter. 2009 [http://www.eyewriter.org/]
**Saccade. 2010 [http://roberttwomey.com/saccade] (in progress)
*Discuss:
**thresholding
**frame difference
**OpenCV - [http://ubaa.net/shared/processing/opencv/ download] [http://www.cs.unc.edu/Research/stc/FAQs/OpenCV/OpenCVReferenceManual.pdf reference manual]. If you are getting this for your computer, be sure to get OpenCV, the OpenCV Processing Library, and the OpenCV Processing Examples (three separate downloads).
**face recognition
*In Class:
**Working alone or in small groups, do experiments with video processing and computer vision.

=== Week 4 - Computer Vision Work ===
* In Class:
** Work on computer vision projects
** Talk about midterm projects.

=== Week 5 - Midterm Workshop ===
*Due: Midterm project proposal.
**Working individually or in small groups (2-3 people), produce an interactive piece that bridges the gap between screen space and physical space. There are many ways to do this--using image-based computer vision techniques, game controllers, audio input, or other physical hardware (Arduino?). Think about the parameters of interaction--are you documenting viewer's behavior (unknown to them), are you taking a familiar form (such as a video game) and tweaking it in some way, are you intervening in social space? Think about what form the output will take. In your one page proposal, describe the input(s), output(s), and dynamic of interaction, as well as some statement of your motivation. Why is this a valuable or interesting project? In addition to the written description, produce supporting visual materials. These should be two functional diagram images and two visual/aesthetic images. The functional diagrams should show the necessary software and hardware components and explain how the interaction will occur. The aesthetic diagrams will give us a sense of what it will look like, how the output will appear. Make a page for your project (including a title) in the Midterm Projects section at the bottom of this page, upload the necessary materials and embed them in that page. This proposal is due in class next week where we will critique and workshop the ideas.
*In class:
**Workshop midterm project ideas. (45 minutes)
**Work on midterm projects.
*NOTE: Best of ICAM from Candy Harris. There will be an install in the annex here at Mandeville and presentations at the Experimental Theater in the CPMC (music building). They should come see what they are going to have to live up to for their final projects. Plus the keynote speakers (ICAM alumns) always have great info about career paths after graduation.

=== Week 6 ===
In class work on midterms.

=== Week 7 - Midterm Critiques ===
In class critique of midterms.

=== Week 8 ===

Due: Written response (1 page) to one of your classmate's projects.

In Class: Draft final project proposal and post to wiki by the end of class. In class discussion as needed.

=== Week 9 ===
work on finals

=== Week 10 - Final Critiques ===
In-class critiques of finals.

=== Finals Week ===
Final documentation due.

== Topics ==
To Be Scheduled

'''Performance for the camera, for the web'''
*Discuss Chatroulette. Facebook, Twitter, Youtube. Attention in the social net.
*ManyCam [http://www.manycam.com/]
*PS3 eye
*jennicam [http://www.wired.com/thisdayintech/2010/04/0414jennicam-launches wired]
*Lonelygirl15 [http://www.youtube.com/watch?v=-goXKtd6cPo youtube] [http://www.wired.com/wired/archive/14.12/lonelygirl.html article]
*Discuss telematic perfromance.
* Justin.tv [http://www.justin.tv/#r=s7RVqBU~]
*Read: The Presentation of Self in Everyday Life (excerpt). Erving Goffman. 1959.
*Read: Performance: A Critical Introduction (excerpt). Richard Carlson. 2004.
*Do: Intervention in social circuits. Chatroulette/Facebook/Youtube exercise.

'''Social Networks/Web 2.0'''
*Read: Protocol, Control, and Networks by Alexander Galloway and Eugene Thacker. Grey Room 17, Fall 2004 p 6-29.
*Read: DIGITAL MAOISM: The Hazards of the New Online Collectivism. Jaron Lanier. 2006.
*Watch: MediatedCultures @ Kansas State http://mediatedcultures.net/mediatedculture.htm
*Datamining/Complex Networks, node-edge graphing.

'''Digital Memory/Personal Media: Where do we exist and how do we remember?'''
*Read: Mediated Memories in the Digital Age (excerpt). Jose van Dijck. 2007.
*Read: Are you sure you want to do this? Matthias Fuchs 1994.
*Read: Delete: The Virtue of Forgetting in the Digital Age (excerpt). Viktor Mayer-Schonberger. 2009.
*Flickr.com, Facebook
*Discuss: My Pocket. Burak Arikan. 2008.

'''Cognition + Creativity'''
*Generative Art vs. Computational Creativity
*Casy Reas
*Processing.org
*Tom Shannon. [http://www.wired.com/magazine/2010/03/pl_arts_pendulum/all/1]
*Read: Triumph of the Cyborg Composer.
*Read: How to draw three people in a garden. 1988.
*Read: Shades of Computational Evocation and Meaning: The GRIOT System and Improvisational Poetry Generation. 2006.

'''Artificial Intelligence'''
*Read: Expressive Processing (excerpt), Noah Wardrip Fruin, 2009.
*Read: Elephants Don't Play Chess, Rodney Brooks, 1990.

'''Appropriation and Remix'''
*Read: The Fiction of Memory. New York Times, March 12, 2010. Luc Sante
*Read: Jonatham Lethem. The Ecstasy of Influence. Harpers Magazine. 2007.
*Remix Culture. Lev.
*God's Little Toys: Confessions of a cut & paste artist. William Gibson. 2005. *http://www.wired.com/wired/archive/13.07/gibson.html
*Reality Hunger: A Manifesto. David Shields. 2010.

'''Materiality in the information age.'''
*Tangible interfaces, haptic feedback.
*Read: Evocative Objects: Things We Think With (excerpt). Sherry Turkle, 2007.
*Read: New Media and the Forensic Imagination (excerpt). Matthew Kirschenbaum. 2008.
*View: BIT Plane.
*View: Garbage Cubes
*Discuss techniques of markerless tracking, augmented reality, QR codes, etc. *Online/Offline Space.

'''Embodiment'''
*Computing with bodies, engineered bodies
*tactile media, haptic interface
*embodied perception
*Read: Stelarc.

'''Self-Image'''
*Read: Self/Image: Technology, Representation, and the Contemporary Subject (excerpt). Amelia Jones, 2006.
*Do: Forensic Photoshop Exercise.
*http://www.flickr.com/photos/dryponder/sets/72157623726710218/
*http://nymag.com/daily/intel/2010/02/obama_being_forced_to_look_at.html#photo=1
*http://niccageaseveryone.blogspot.com/
*http://bubleraptor.tumblr.com/
*photoshop free Marie Claire issue: http://jezebel.com/5511507/so-long-as-your-face-looks-alright-everything-else-can-be-photoshopped

== Places to Find Art ==
* http://we-make-money-not-art.com/
* http://www.isea-web.org/, http://www.isea2010ruhr.org/
* http://www.transmediale.de/en
* http://01sj.org/
* http://www.file.org.br/
* http://www.aec.at/festival_about_en.php
* http://www.sciencegallery.com/lightwave09
* Institutions that Sponsor/Show Media Art
** Eyebeam New York City
** New Museum/Rhizome.org http://rhizome.org
** HarvestWorks
** Machine Project, Los Angeles.

== Midterm Projects ==
Make pages here.
* [[DummyProject | Dummy Project]]
* [[MidtermProject| MotionDJ - Leilani Martin]]
* [[What's For Lunch, Kids? by Kelley Kim| ''What's For Lunch, Kids?'' - Kelley Kim]]
* [[Virtual Walk? - Joeny Thipsidakhom]]
* [[Untitled Midterm| Untitled - Jezreel Callejas]]
* [[Midterm Project - Tony Lu | Virtual Maze - Tony Lu]]
* [[Midterm Project | SayCHEESE - Joel and Jenny Chang]]
* [[Carnival Ride| Carnival Ride - Christina Sanchez and Jennifer Sunga]]
* [[Hunted - Anna Lin, Jenny Wang, and Ellen Huang]]
* [[Eye Motion Experiment - Javi Lee]]
* [[Social Creature | Boo (formerly Social Creature) - Jet Antonio]]
* [[Sound Sketch - Emilio Marcelino, Greg Parsons, and Ben Brickley]]

== Final Projects ==
* [[Aquarium| supermarioslaughter - Jezreel Callejas]]
* [[Dance in the Dark - Anna Lin, Jenny Wang and Ellen Huang]]
* [[Inkling One | Inkling One - Jet Antonio]]
* [[Camera Keyboard - Joeny Thipsidakhom]]
* [[uMV | uMV - Tony Lu]]
* [[Stone Face | Stone Face - Jennifer Sunga]]
* [[This Music: Prohibited | This Music: Prohibited - Kelley Kim]]
* [[Censored | Censored: Too Vulgar - Javier Lee]]
* [[What do you see? | What do you see? - Christina Sanchez]]
* [[Happy Days - Gregory Parsons]]
* [[Adios Chancellor! - Ben Brickley]]
* [[FRSynth - Emilio Marcelino]]
* [[WiiSound - Leilani Martin]]

== Student Pages ==
Click "edit" on the right to add your own page below.
* [[Students/RobertTwomey | RobertTwomey]]
* [[Students/Javier Lee | Javier Lee]]
* [[Students/Jenny Wang | Jenny Wang]]
* [[Students/Joeny Thipsidakhom | Joeny Thipsidakhom]]
* [[Students/Kuan-Ting Lu | Tony Lu]]
* [[Students/Jezreel Callejas| Jezreel Callejas]]
* [[Students/ChristinaSanchez| Christina Sanchez]]
* [[Students/BenBrickley | BenBrickley]]
* [[Students/Ellen Huang | Ellen Huang]]
* [[Students/Kelley Kim | Kelley Kim]]
* [[Students/EmilioMarcelino | EmilioMarcelino]]
* [[Students/Anna Lin | Anna Lin]]
* [[Student/Jenny Chang | Jenny Chang]]
* [[Student/Jet Antonio | Jet Antonio]]
* [[Students/GregoryParsons | Gregory Parsons]]
* [[Students/Jennifer Sunga | Jennifer Sunga]]
* [[Students/LeilaniMartin | Leilani Martin]]

MidtermProject

2010-05-21T00:08:11Z

Leilanim14: /* Documentation */

== Title ==
MIDI Capture

== Description ==
*Motivation
I have a strong interest in music and manipulating sound. I wanted to do a project that would spin off those interests and combine that with something interactive involving video capture methods in Processing.

*Interactive paradigm
This program uses the brightness tracking in the Video library of Processing to track the brightest pixel. Using a flashlight, the user controls the frequency and amplitude of drum synthesized MIDI notes. Simultaneously, a predetermined array of notes plays at random and creates random ellipses that appears on the screen. This creates an improvisational live "duet" between the participant and the program. The colors of the ellipses both controlled by the user and the program are generated at random to add a dynamic visual experience.

*Technical Description
The flashlight held by the user will be tracked by the brightness tracking method in Processing's video library. The top half of the screen is designated to the area where the user can control the MIDI notes. Left to right controls the amplitude, while Up and Down control the frequency. While the bottom half of the screen is designated to the random ellipses that create a sort of rhythmic dance on the screen while playing an improvised melody for the user to interact with. I used the Soundcipher library to control the MIDI sounds. The speed of each drawing of the ellipses as well as the playing of the notes is determined by the frame rate is by default is set at 6. The user can change the frame rate to faster or slower by pressing the up or down button on the keyboard.

The flow in of information will go as follows: As the user moves the flashlight across the screen, it will change the MIDI note's parameters (amplitude and frequency) while the random ellipses play a melody on the bottom half of the screen. The ending result is an interactive MIDI sound dialogue between the user and the program.

== Documentation ==
*Code
/**
*MIDI Capture
* by Leilani Martin
*
* Tracks the brightest pixel in a live video signal and plays
a musical note at that pixel, a live duet between a predetermined/random MIDI
melody and the drum notes controlled by the user.
*/

import processing.video.*;
import arb.soundcipher.*;

//soundcipher data
SoundCipher sc = new SoundCipher(this);

//MIDI scale for square melody
int[] scale = {57, 60, 60, 62, 64, 67, 67, 69, 72};
int rate = 7;

Capture video;
void setup() {
size(640, 480);
//160
// Uses the default video input, see the reference if this causes an error
video = new Capture(this, width, height, 30);
noStroke();
smooth();
frameRate(6);

}

//Handles Frame Rate
void keyPressed(){
if (key == CODED){
if (keyCode == UP){
frameRate(rate * 2);
}
else if(keyCode == DOWN){
frameRate(rate - 1);
}
}
}

//Captures the video and finds the brightest pixel
void draw() {
if (video.available()) {
video.read();
image(video, 0, 0, width, height); // Draw the webcam video onto the screen
int brightestX = 0; // X-coordinate of the brightest video pixel
int brightestY = 0; // Y-coordinate of the brightest video pixel
float brightestValue = 0; // Brightness of the brightest video pixel
// Search for the brightest pixel: For each row of pixels in the video image and
// for each pixel in the yth row, compute each pixel's index in the video
video.loadPixels();
int index = 0;
for (int y = 0; y < video.height; y++) {
for (int x = 0; x < video.width; x++) {
// Get the color stored in the pixel
int pixelValue = video.pixels[index];
// Determine the brightness of the pixel
float pixelBrightness = brightness(pixelValue);
// If that value is brighter than any previous, then store the
// brightness of that pixel, as well as its (x,y) location
if (pixelBrightness > brightestValue) {
brightestValue = pixelBrightness;
brightestY = y;
brightestX = x;
}
index++;
}
}

//Capture to MIDI note (Drum sounds)
ellipse(brightestX, brightestY, 25, 25);
fill(random(0, 255), random(0, 255), random(0, 255));
stroke(random(0, 255));
strokeWeight(3);
sc.instrument(sc.DRUM);
sc.playNote(brightestY, brightestX, 3000);

//Line dividing the screen
line(0, 160, 700, 160);
stroke(200);

//Random ellipse melody
if (random(1) < 0.8) {
sc.instrument(2);
sc.playNote(scale[(int)random(scale.length)], (int)random(80)+40, 0.2);
stroke(color(random(256), random (256), random(256)));
ellipse(random(0, 1000), random(300, 500), 100, 100);
}
}
}

*Screen Captures

[[Image:shot1.png]]

[[Image:shot2.png]]

*Live Screen Capture Video Link
-View Live video recording: http://s102.photobucket.com/albums/m110/alohahoopstar/?action=view&current=ISUHD.flv

MidtermProject

2010-05-20T23:25:36Z

Leilanim14: /* Documentation */

== Title ==
MIDI Capture

== Description ==
*Motivation
I have a strong interest in music and manipulating sound. I wanted to do a project that would spin off those interests and combine that with something interactive involving video capture methods in Processing.

*Interactive paradigm
This program uses the brightness tracking in the Video library of Processing to track the brightest pixel. Using a flashlight, the user controls the frequency and amplitude of drum synthesized MIDI notes. Simultaneously, a predetermined array of notes plays at random and creates random ellipses that appears on the screen. This creates an improvisational live "duet" between the participant and the program. The colors of the ellipses both controlled by the user and the program are generated at random to add a dynamic visual experience.

*Technical Description
The flashlight held by the user will be tracked by the brightness tracking method in Processing's video library. The top half of the screen is designated to the area where the user can control the MIDI notes. Left to right controls the amplitude, while Up and Down control the frequency. While the bottom half of the screen is designated to the random ellipses that create a sort of rhythmic dance on the screen while playing an improvised melody for the user to interact with. I used the Soundcipher library to control the MIDI sounds. The speed of each drawing of the ellipses as well as the playing of the notes is determined by the frame rate is by default is set at 6. The user can change the frame rate to faster or slower by pressing the up or down button on the keyboard.

The flow in of information will go as follows: As the user moves the flashlight across the screen, it will change the MIDI note's parameters (amplitude and frequency) while the random ellipses play a melody on the bottom half of the screen. The ending result is an interactive MIDI sound dialogue between the user and the program.

== Documentation ==
*Code
/**
*MIDI Capture
* by Leilani Martin
*
* Tracks the brightest pixel in a live video signal and plays
a musical note at that pixel, a live duet between a predetermined/random MIDI
melody and the drum notes controlled by the user.
*/

import processing.video.*;
import arb.soundcipher.*;

//soundcipher data
SoundCipher sc = new SoundCipher(this);

//MIDI scale for square melody
int[] scale = {57, 60, 60, 62, 64, 67, 67, 69, 72};
int rate = 7;

Capture video;
void setup() {
size(640, 480);
//160
// Uses the default video input, see the reference if this causes an error
video = new Capture(this, width, height, 30);
noStroke();
smooth();
frameRate(6);

}

//Handles Frame Rate
void keyPressed(){
if (key == CODED){
if (keyCode == UP){
frameRate(rate * 2);
}
else if(keyCode == DOWN){
frameRate(rate - 1);
}
}
}

//Captures the video and finds the brightest pixel
void draw() {
if (video.available()) {
video.read();
image(video, 0, 0, width, height); // Draw the webcam video onto the screen
int brightestX = 0; // X-coordinate of the brightest video pixel
int brightestY = 0; // Y-coordinate of the brightest video pixel
float brightestValue = 0; // Brightness of the brightest video pixel
// Search for the brightest pixel: For each row of pixels in the video image and
// for each pixel in the yth row, compute each pixel's index in the video
video.loadPixels();
int index = 0;
for (int y = 0; y < video.height; y++) {
for (int x = 0; x < video.width; x++) {
// Get the color stored in the pixel
int pixelValue = video.pixels[index];
// Determine the brightness of the pixel
float pixelBrightness = brightness(pixelValue);
// If that value is brighter than any previous, then store the
// brightness of that pixel, as well as its (x,y) location
if (pixelBrightness > brightestValue) {
brightestValue = pixelBrightness;
brightestY = y;
brightestX = x;
}
index++;
}
}

//Capture to MIDI note (Drum sounds)
ellipse(brightestX, brightestY, 25, 25);
fill(random(0, 255), random(0, 255), random(0, 255));
stroke(random(0, 255));
strokeWeight(3);
sc.instrument(sc.DRUM);
sc.playNote(brightestY, brightestX, 3000);

//Line dividing the screen
line(0, 160, 700, 160);
stroke(200);

//Random ellipse melody
if (random(1) < 0.8) {
sc.instrument(2);
sc.playNote(scale[(int)random(scale.length)], (int)random(80)+40, 0.2);
stroke(color(random(256), random (256), random(256)));
ellipse(random(0, 1000), random(300, 500), 100, 100);
}
}
}

*Photo Documentation

[[Image:shot1.png]]

[[Image:shot2.png]]

File:Shot2.png

2010-05-20T23:24:20Z

Leilanim14:

File:Shot1.png

2010-05-20T23:23:51Z

Leilanim14:

MidtermProject

2010-05-20T23:23:23Z

Leilanim14: /* Documentation */

== Title ==
MIDI Capture

== Description ==
*Motivation
I have a strong interest in music and manipulating sound. I wanted to do a project that would spin off those interests and combine that with something interactive involving video capture methods in Processing.

*Interactive paradigm
This program uses the brightness tracking in the Video library of Processing to track the brightest pixel. Using a flashlight, the user controls the frequency and amplitude of drum synthesized MIDI notes. Simultaneously, a predetermined array of notes plays at random and creates random ellipses that appears on the screen. This creates an improvisational live "duet" between the participant and the program. The colors of the ellipses both controlled by the user and the program are generated at random to add a dynamic visual experience.

*Technical Description
The flashlight held by the user will be tracked by the brightness tracking method in Processing's video library. The top half of the screen is designated to the area where the user can control the MIDI notes. Left to right controls the amplitude, while Up and Down control the frequency. While the bottom half of the screen is designated to the random ellipses that create a sort of rhythmic dance on the screen while playing an improvised melody for the user to interact with. I used the Soundcipher library to control the MIDI sounds. The speed of each drawing of the ellipses as well as the playing of the notes is determined by the frame rate is by default is set at 6. The user can change the frame rate to faster or slower by pressing the up or down button on the keyboard.

The flow in of information will go as follows: As the user moves the flashlight across the screen, it will change the MIDI note's parameters (amplitude and frequency) while the random ellipses play a melody on the bottom half of the screen. The ending result is an interactive MIDI sound dialogue between the user and the program.

== Documentation ==
*Code
/**
*MIDI Capture
* by Leilani Martin
*
* Tracks the brightest pixel in a live video signal and plays
a musical note at that pixel, a live duet between a predetermined/random MIDI
melody and the drum notes controlled by the user.
*/

import processing.video.*;
import arb.soundcipher.*;

//soundcipher data
SoundCipher sc = new SoundCipher(this);

//MIDI scale for square melody
int[] scale = {57, 60, 60, 62, 64, 67, 67, 69, 72};
int rate = 7;

Capture video;
void setup() {
size(640, 480);
//160
// Uses the default video input, see the reference if this causes an error
video = new Capture(this, width, height, 30);
noStroke();
smooth();
frameRate(6);

}

//Handles Frame Rate
void keyPressed(){
if (key == CODED){
if (keyCode == UP){
frameRate(rate * 2);
}
else if(keyCode == DOWN){
frameRate(rate - 1);
}
}
}

//Captures the video and finds the brightest pixel
void draw() {
if (video.available()) {
video.read();
image(video, 0, 0, width, height); // Draw the webcam video onto the screen
int brightestX = 0; // X-coordinate of the brightest video pixel
int brightestY = 0; // Y-coordinate of the brightest video pixel
float brightestValue = 0; // Brightness of the brightest video pixel
// Search for the brightest pixel: For each row of pixels in the video image and
// for each pixel in the yth row, compute each pixel's index in the video
video.loadPixels();
int index = 0;
for (int y = 0; y < video.height; y++) {
for (int x = 0; x < video.width; x++) {
// Get the color stored in the pixel
int pixelValue = video.pixels[index];
// Determine the brightness of the pixel
float pixelBrightness = brightness(pixelValue);
// If that value is brighter than any previous, then store the
// brightness of that pixel, as well as its (x,y) location
if (pixelBrightness > brightestValue) {
brightestValue = pixelBrightness;
brightestY = y;
brightestX = x;
}
index++;
}
}

//Capture to MIDI note (Drum sounds)
ellipse(brightestX, brightestY, 25, 25);
fill(random(0, 255), random(0, 255), random(0, 255));
stroke(random(0, 255));
strokeWeight(3);
sc.instrument(sc.DRUM);
sc.playNote(brightestY, brightestX, 3000);

//Line dividing the screen
line(0, 160, 700, 160);
stroke(200);

//Random ellipse melody
if (random(1) < 0.8) {
sc.instrument(2);
sc.playNote(scale[(int)random(scale.length)], (int)random(80)+40, 0.2);
stroke(color(random(256), random (256), random(256)));
ellipse(random(0, 1000), random(300, 500), 100, 100);
}
}
}

*Photo Documentation

[[Image:shot1.jpg]]

[[Image:shot2.jpg]]

MidtermProject

2010-05-20T23:19:31Z

Leilanim14: /* Description */

MidtermProject

2010-05-20T23:12:45Z

Leilanim14: /* Description */

== Title ==
MIDI Capture

== Description ==
*Motivation
I have a strong interest in music and manipulating sound. I wanted to do a project that would spin off those interests and combine that with something interactive involving video capture methods in Processing.

*Interactive paradigm
This program uses the brightness tracking in the Video library of Processing to track the brightest pixel. Using a flashlight, the user controls the frequency and amplitude of piano synthesized MIDI notes. Simultaneously, a predetermined array of notes plays at random and creates random ellipses that appears on the screen. This creates an improvisational live "duet" between the participant and the program. The colors of the ellipses both controlled by the user and the program are generated at random to add a dynamic visual experience.

*Technical Description
The flashlight held by the user will be tracked by the brightness tracking method in Processing's video library. The top half of the screen is designated to the area where the user can control the MIDI notes. Left to right controls the amplitude, while Up and Down control the frequency. While the bottom half of the screen is designated to the random ellipses that create a sort of rhythmic dance on the screen while playing an improvised melody for the user to interact with. I used the Soundcipher library to control the MIDI sounds.

The flow in of information will go as follows: As the user moves the flashlight across the screen, it will change the MIDI note's parameters (amplitude and frequency) while the random ellipses play a melody on the bottom half of the screen. The ending result is an interactive MIDI sound dialogue between the user and the program.

== Documentation ==
*Code
/**
*MIDI Capture
* by Leilani Martin
*
* Tracks the brightest pixel in a live video signal and plays
a musical note at that pixel, a live duet between a predetermined/random MIDI
melody and the drum notes controlled by the user.
*/

import processing.video.*;
import arb.soundcipher.*;

//soundcipher data
SoundCipher sc = new SoundCipher(this);

//MIDI scale for square melody
int[] scale = {57, 60, 60, 62, 64, 67, 67, 69, 72};
int rate = 7;

Capture video;
void setup() {
size(640, 480);
//160
// Uses the default video input, see the reference if this causes an error
video = new Capture(this, width, height, 30);
noStroke();
smooth();
frameRate(6);

}

//Handles Frame Rate
void keyPressed(){
if (key == CODED){
if (keyCode == UP){
frameRate(rate * 2);
}
else if(keyCode == DOWN){
frameRate(rate - 1);
}
}
}

//Captures the video and finds the brightest pixel
void draw() {
if (video.available()) {
video.read();
image(video, 0, 0, width, height); // Draw the webcam video onto the screen
int brightestX = 0; // X-coordinate of the brightest video pixel
int brightestY = 0; // Y-coordinate of the brightest video pixel
float brightestValue = 0; // Brightness of the brightest video pixel
// Search for the brightest pixel: For each row of pixels in the video image and
// for each pixel in the yth row, compute each pixel's index in the video
video.loadPixels();
int index = 0;
for (int y = 0; y < video.height; y++) {
for (int x = 0; x < video.width; x++) {
// Get the color stored in the pixel
int pixelValue = video.pixels[index];
// Determine the brightness of the pixel
float pixelBrightness = brightness(pixelValue);
// If that value is brighter than any previous, then store the
// brightness of that pixel, as well as its (x,y) location
if (pixelBrightness > brightestValue) {
brightestValue = pixelBrightness;
brightestY = y;
brightestX = x;
}
index++;
}
}

//Capture to MIDI note (Drum sounds)
ellipse(brightestX, brightestY, 25, 25);
fill(random(0, 255), random(0, 255), random(0, 255));
stroke(random(0, 255));
strokeWeight(3);
sc.instrument(sc.DRUM);
sc.playNote(brightestY, brightestX, 3000);

//Line dividing the screen
line(0, 160, 700, 160);
stroke(200);

//Random ellipse melody
if (random(1) < 0.8) {
sc.instrument(2);
sc.playNote(scale[(int)random(scale.length)], (int)random(80)+40, 0.2);
stroke(color(random(256), random (256), random(256)));
ellipse(random(0, 1000), random(300, 500), 100, 100);
}
}
}

MidtermProject

2010-05-20T23:10:49Z

Leilanim14: /* Documentation */

== Title ==
MIDI Capture

== Description ==
*Motivation
I have a strong interest in music and manipulating sound. I wanted to do a project that would spin off those interests and combine that with something interactive involving video capture methods in Processing.

*Interactive paradigm
This program uses the brightness tracking in the Video library of Processing to track the brightest pixel. Using a flashlight, the user controls the frequency and amplitude of piano synthesized MIDI notes. Simultaneously, a predetermined array of notes plays at random and creates random squares that appears on the screen. This creates an improvisational live "duet" between the participant and the program.

*Technical Description
The flashlight held by the user will be tracked by the brightness tracking method in Processing's video library. The top half of the screen is designated to the area where the user can control the MIDI notes. Left to right controls the amplitude, while Up and Down control the frequency. While the bottom half of the screen is designated to the random squares that create a sort of rhythmic dance on the screen while playing an improvised melody for the user to interact with. I used the Soundcipher library to control the MIDI sounds.

The flow in of information will go as follows: As the user moves the flashlight across the screen, it will change the MIDI note's parameters (amplitude and frequency) while the random squares play a melody on the bottom half of the screen. The ending result is an interactive MIDI sound dialogue between the user and the program.

== Documentation ==
*Code
/**
*MIDI Capture
* by Leilani Martin
*
* Tracks the brightest pixel in a live video signal and plays
a musical note at that pixel, a live duet between a predetermined/random MIDI
melody and the drum notes controlled by the user.
*/

import processing.video.*;
import arb.soundcipher.*;

//soundcipher data
SoundCipher sc = new SoundCipher(this);

//MIDI scale for square melody
int[] scale = {57, 60, 60, 62, 64, 67, 67, 69, 72};
int rate = 7;

Capture video;
void setup() {
size(640, 480);
//160
// Uses the default video input, see the reference if this causes an error
video = new Capture(this, width, height, 30);
noStroke();
smooth();
frameRate(6);

}

//Handles Frame Rate
void keyPressed(){
if (key == CODED){
if (keyCode == UP){
frameRate(rate * 2);
}
else if(keyCode == DOWN){
frameRate(rate - 1);
}
}
}

//Captures the video and finds the brightest pixel
void draw() {
if (video.available()) {
video.read();
image(video, 0, 0, width, height); // Draw the webcam video onto the screen
int brightestX = 0; // X-coordinate of the brightest video pixel
int brightestY = 0; // Y-coordinate of the brightest video pixel
float brightestValue = 0; // Brightness of the brightest video pixel
// Search for the brightest pixel: For each row of pixels in the video image and
// for each pixel in the yth row, compute each pixel's index in the video
video.loadPixels();
int index = 0;
for (int y = 0; y < video.height; y++) {
for (int x = 0; x < video.width; x++) {
// Get the color stored in the pixel
int pixelValue = video.pixels[index];
// Determine the brightness of the pixel
float pixelBrightness = brightness(pixelValue);
// If that value is brighter than any previous, then store the
// brightness of that pixel, as well as its (x,y) location
if (pixelBrightness > brightestValue) {
brightestValue = pixelBrightness;
brightestY = y;
brightestX = x;
}
index++;
}
}

//Capture to MIDI note (Drum sounds)
ellipse(brightestX, brightestY, 25, 25);
fill(random(0, 255), random(0, 255), random(0, 255));
stroke(random(0, 255));
strokeWeight(3);
sc.instrument(sc.DRUM);
sc.playNote(brightestY, brightestX, 3000);

//Line dividing the screen
line(0, 160, 700, 160);
stroke(200);

//Random ellipse melody
if (random(1) < 0.8) {
sc.instrument(2);
sc.playNote(scale[(int)random(scale.length)], (int)random(80)+40, 0.2);
stroke(color(random(256), random (256), random(256)));
ellipse(random(0, 1000), random(300, 500), 100, 100);
}
}
}

MidtermProject

2010-05-20T23:08:10Z

Leilanim14: /* Visualization */

== Title ==
MIDI Capture

== Description ==
*Motivation
I have a strong interest in music and manipulating sound. I wanted to do a project that would spin off those interests and combine that with something interactive involving video capture methods in Processing.

*Interactive paradigm
This program uses the brightness tracking in the Video library of Processing to track the brightest pixel. Using a flashlight, the user controls the frequency and amplitude of piano synthesized MIDI notes. Simultaneously, a predetermined array of notes plays at random and creates random squares that appears on the screen. This creates an improvisational live "duet" between the participant and the program.

*Technical Description
The flashlight held by the user will be tracked by the brightness tracking method in Processing's video library. The top half of the screen is designated to the area where the user can control the MIDI notes. Left to right controls the amplitude, while Up and Down control the frequency. While the bottom half of the screen is designated to the random squares that create a sort of rhythmic dance on the screen while playing an improvised melody for the user to interact with. I used the Soundcipher library to control the MIDI sounds.

The flow in of information will go as follows: As the user moves the flashlight across the screen, it will change the MIDI note's parameters (amplitude and frequency) while the random squares play a melody on the bottom half of the screen. The ending result is an interactive MIDI sound dialogue between the user and the program.

== Documentation ==
*Code
/**
*MIDI Capture
* by Leilani Martin
*
* Tracks the brightest pixel in a live video signal and plays
a musical note at that pixel, a live duet between a predetermined/random MIDI
melody and the drum notes controlled by the user.
*/

import processing.video.*;
import arb.soundcipher.*;

//soundcipher data
SoundCipher sc = new SoundCipher(this);

//MIDI scale for square melody
int[] scale = {57, 60, 60, 62, 64, 67, 67, 69, 72};
int rate = 7;

Capture video;
void setup() {
size(640, 480);
//160
// Uses the default video input, see the reference if this causes an error
video = new Capture(this, width, height, 30);
noStroke();
smooth();
frameRate(6);

}

//Handles Frame Rate
void keyPressed(){
if (key == CODED){
if (keyCode == UP){
frameRate(rate * 2);
}
else if(keyCode == DOWN){
frameRate(rate - 1);
}
}
}

//Captures the video and finds the brightest pixel
void draw() {
if (video.available()) {
video.read();
image(video, 0, 0, width, height); // Draw the webcam video onto the screen
int brightestX = 0; // X-coordinate of the brightest video pixel
int brightestY = 0; // Y-coordinate of the brightest video pixel
float brightestValue = 0; // Brightness of the brightest video pixel
// Search for the brightest pixel: For each row of pixels in the video image and
// for each pixel in the yth row, compute each pixel's index in the video
video.loadPixels();
int index = 0;
for (int y = 0; y < video.height; y++) {
for (int x = 0; x < video.width; x++) {
// Get the color stored in the pixel
int pixelValue = video.pixels[index];
// Determine the brightness of the pixel
float pixelBrightness = brightness(pixelValue);
// If that value is brighter than any previous, then store the
// brightness of that pixel, as well as its (x,y) location
if (pixelBrightness > brightestValue) {
brightestValue = pixelBrightness;
brightestY = y;
brightestX = x;
}
index++;
}
}

//Capture to MIDI note (Drum sounds)
ellipse(brightestX, brightestY, 25, 25);
fill(random(0, 255), random(0, 255), random(0, 255));
stroke(random(0, 255));
strokeWeight(3);
sc.instrument(sc.DRUM);
sc.playNote(brightestY, brightestX, 3000);

//Line dividing the screen
line(0, 160, 700, 160);
stroke(200);

//Random square melody
if (random(1) < 0.8) {
sc.instrument(2);
sc.playNote(scale[(int)random(scale.length)], (int)random(80)+40, 0.2);
stroke(color(random(256), random (256), random(256)));
ellipse(random(0, 1000), random(300, 500), 100, 100);
}
}
}

MidtermProject

2010-05-20T23:07:20Z

Leilanim14: /* Description */

MidtermProject

2010-05-20T22:58:36Z

Leilanim14: /* Title */

MidtermProject

2010-04-22T22:17:52Z

Leilanim14:

== Title ==
MotionDJ

== Description ==
*Motivation
I have a strong interest in music and manipulating sound. I wanted to do a project that would spin off those interests and combine that with something interactive involving video capture methods in Processing.

*Interactive paradigm
It uses the viewers movements to control different sounds a song. I would like to use the frame differencing method and utilize the way it quantifies the amount of movement in the video frame to control different parameters/filters of a song. Some of the control parameters are going to be the speed, treble, key, and volume. I’m hoping that this project will create an interaction between what the see on the screen and how their movement affects the song that is playing. It will give way to a unique experience with a music that does not involve electrical controls, turntables, dance pads or even an actual instrument. But rather, it gives your body some control over the sounds you hear through your own motions.

*Technical Description
Uses the frame differencing methods to capture the amount of movement in a frame and then send those numbers to Logic Pro to control different filters that would then change the sound of the song. The input would be the amount of movement in the video frame and the output would be a change in the sound of the songs. I would split the frame into quadrants, one for a different filter. An amount of movement also has to be specified to control the amount change. For example, the speed at which the participant is moving in the quadrant that controls the amplitude of the song determines how loud of soft the song is playing.

The flow in of information will go as follows: A song would begin to play and the user would move and interact with the screen. Processing will then take the amount of movement in the designated quadrants of the screen and send those numbers to Logic Pro. Logic Pro has an environment that is able to process numbers and use them as controls to different filters and parameters. The changes in the parameters would then be transferred to the audio speakers, and finally the user can hear how their movements changed the sound of the song.

== Visualization ==
*Functional Diagrams
Flow Diagram
[[Image:flowdiagram.jpg]]

Sample Screenshots (What the user will see)
[[Image:screenshots.jpg]]

File:Screenshots.jpg

2010-04-22T22:17:37Z

Leilanim14: uploaded a new version of "Image:Screenshots.jpg"

File:Flowdiagram.jpg

2010-04-22T22:17:14Z

Leilanim14: uploaded a new version of "Image:Flowdiagram.jpg"

File:Screenshots.jpg

2010-04-22T22:13:58Z

Leilanim14:

File:Flowdiagram.jpg

2010-04-22T22:13:36Z

Leilanim14:

MidtermProject

2010-04-22T22:10:58Z

Leilanim14:

MidtermProject

2010-04-22T22:06:52Z

Leilanim14: New page: == Title == MotionDJ

== Title ==
MotionDJ

Classes/2010/VIS145B

2010-04-22T22:05:45Z

Leilanim14: /* Midterm Projects */

== Time and Process Based Digital Media II ==
Time: Thursdays 3:30-6:20pm, VAF 228

This class is an advanced study and portfolio project course centered on the use of hardware and software to create interactive and time-based art. These projects can take many forms—interactive installations, dynamic visualizations/sonifications, printed renderings—chosen by the students. This will not be a course of technical instruction—rather we will consider technical and conceptual issues in tandem, supplementing discussions and activities with specific technical instruction where necessary. There is a strong emphasis on the development and articulation of personal directions of research by the students in the course.

I would like to split the reading/homework responsibility for two parts of the class. In the first half of the term I will present a series of works and readings covering my particular interests--the intersections of social performance, embodied experience, and cognition. In the latter half of the class (after the midterm) you all will do the presentations on topics of your choosing. Working individually or in small groups, you will provide us with some conceptual provocation (reading material) covering topics you intend to engage with your final, and you will lead a discussion on technical and conceptual issues. Reading and critical writing, in response to text and works you present and those I present, are integral to this course.

The schedule is a living document and will be revised over the period of the course.

== Instructor ==
Robert Twomey

rtwomey@ucsd.edu
*http://roberttwomey.com
*http://experimentalgamelab.net
*http://crca.ucsd.edu

Office Hours: Wednesday 3-4pm, Atkinson Hall Rm 1601 (CRCA research neighborhood). Please e-mail me if you plan to attend.

== Grading ==
*Midterm Project - 30%
*Final Project - 40%
*Presentations - 10%
*Readings - 10%
*Participation - 10%

=== Presentations ===
(1) Short presentation on your work in the second week of class. This should be a statement of your interests, direction, goals with media art. Present examples from your own work which you feel strongly about, and which best represent your interests and trajectory. Present examples of other artist's work that serve as models for the kind of work you would like to make. (5-10 minutes each)

(2) Medium presentation on final projects in the second semester of the course (weeks 7-9). This is the portion of the class where you dictate the reading and the discussion. If you are presenting on a given week, you need to provide us with a reading 1 week in advance. We will sign up for those time slots in week 6, just after the midterm. (10-15 minutes)

=== Reading Responses ===
These are written summaries and critical responses to materials assigned for out of class viewing. Things to consider: What points does the author make? Do you buy their assumptions or agree with their conclusions? Reading responses will be printed and turned in to the instructor at the beginning of class. Generally these should be 1 page long.

=== Projects ===
Midterm and final projects will be graded on concept, effort, and realization. Formal proposals are a necessary component of the process so take them seriously. Make the effort to get started early and seek the help you need--we want to see finished, well-considered pieces for the midterm and final. Additionally, you will need to submit documentation of the project after completion which includes images, video, and source code where applicable. These materials (proposals and documentation) will all be posted to the wiki.
=== Documentation Policy ===
*personal wiki page
*source code on wiki
*image/video documentation as appropriate.
*explanatory writing (on intent, motivation, context)

=== Attendance ===
Attendance is mandatory. Each unexcused absence will drop your final grade one letter. There are only 10 weeks of class, please come to them all.

== Schedule ==
=== Week 1 - Intro ===
*Introductions
*Scope of course, interests, technical possibilities.
*My work.
*Watch: We Live In Public. 2009. (excerpts)
*In class: personal page on wiki. [http://www.trsp.net/teaching/gamemod/ game-mod exercise]. [http://www.trsp.net/teaching/gamemod/gamemod_breakout_source_en.zip download link]
*Read: [http://www.nyu.edu/projects/xdesign/mainmenu/archive_tangible.html Against Virtualized Information], [http://www.nyu.edu/projects/xdesign/mainmenu/archive_analtictech.html Novel Analytic Techniques], and [http://www.nyu.edu/projects/xdesign/mainmenu/archive_infocounts.html What Information Counts?] by [http://www.environmentalhealthclinic.net/people/natalie-jeremijenko/ Natalie Jeremijenko].
*Read: [http://www.yalealumnimagazine.com/issues/2004_03/jeremijenko.html An Engineer for the Avante Garde]
*Read: [http://www.worldchanging.com/archives/001450.html Natalie Jeremijenko The WorldChanging Interview]
*Read: [http://tech90s.walkerart.org/nj/transcript/nj_01.html Database Politics and Social Simulations], good background on her earlier artwork.

=== Week 2 - Student Research Interests ===
*Due: 1 page on Jeremijenko.
*Presentations on your work.
*Read: [http://www.flong.com/texts/essays/essay_cvad/ Computer Vision for Artists and Designers: Pedagogic Tools and Techniques for Novice Programmers] Golan Levin. ''pay particular attention to part II. ELEMENTARY COMPUTER VISION TECHNIQUES. we are going to try these in class next week.''

=== Week 3 - Computer Vision / Human Perception ===
*Due: Nothing. Read the Golan Levin piece, but no written response.
*Discuss:
**Myron Kreuger. Video Place. 1989 [http://www.youtube.com/watch?v=dqZyZrN3Pl0]
**Text Rain. Camille Utterback & Romy Achituv. 1999. [http://www.youtube.com/watch?v=toWFvXHghDk] [http://www.camilleutterback.com/]
**Very Nervous System. 1982-1991. [http://vimeo.com/8120954]
**Suicide Box. Bureau of Inverse Technology. 1996. (13:00)
**Marie Sester. ACCESS. 2003. [http://accessproject.net]
**Messa di Voce. Golan Levin and Zach Lieberman with Jaap Blonk and Joan La Barbara. 2003. [http://www.flong.com/projects/messa/] [http://www.tmema.org/messa/messa.html]
**Seen. David Rokeby. 2002. [http://vimeo.com/6012986]
**Sorting Daemon. David Rokeby. 2003. [http://homepage.mac.com/davidrokeby/sorting.html]
**Cheese. Christian Moller. 2003. [http://www.christian-moeller.com/display.php?project_id=36] made in collaboration with UCSD [http://mplab.ucsd.edu/wordpress/ Machine Perception Lab]
**Eyewriter. 2009 [http://www.eyewriter.org/]
**Saccade. 2010 [http://roberttwomey.com/saccade] (in progress)
*Discuss:
**thresholding
**frame difference
**OpenCV - [http://ubaa.net/shared/processing/opencv/ download] [http://www.cs.unc.edu/Research/stc/FAQs/OpenCV/OpenCVReferenceManual.pdf reference manual]. If you are getting this for your computer, be sure to get OpenCV, the OpenCV Processing Library, and the OpenCV Processing Examples (three separate downloads).
**face recognition
*In Class:
**Working alone or in small groups, do experiments with video processing and computer vision.

=== Week 4 - Midterm Workshop ===
*Due: Midterm project proposal.
**Working individually or in small groups (2-3 people), produce an interactive piece that bridges the gap between screen space and physical space. There are many ways to do this--using image-based computer vision techniques, game controllers, audio input, or other physical hardware (Arduino?). Think about the parameters of interaction--are you documenting viewer's behavior (unknown to them), are you taking a familiar form (such as a video game) and tweaking it in some way, are you intervening in social space? Think about what form the output will take. In your one page proposal, describe the input(s), output(s), and dynamic of interaction, as well as some statement of your motivation. Why is this a valuable or interesting project? In addition to the written description, produce supporting visual materials. These should be two functional diagram images and two visual/aesthetic images. The functional diagrams should show the necessary software and hardware components and explain how the interaction will occur. The aesthetic diagrams will give us a sense of what it will look like, how the output will appear. Make a page for your project (including a title) in the Midterm Projects section at the bottom of this page, upload the necessary materials and embed them in that page. This proposal is due in class next week where we will critique and workshop the ideas.
*Review computer vision experiments from last class.
*Work on midterm projects.

=== Week 5 - Midterm Critiques ===
In-class critiques of midterm projects.

=== Week 6 ===
=== Week 7 ===
Student presentations
=== Week 8 ===
Student presentations
=== Week 9 ===
Student presentations
=== Week 10 - Final Critiques ===
In-class Critiques.

=== Finals Week ===
Final documentation due.

== Topics ==
To Be Scheduled

'''Performance for the camera, for the web'''
*Discuss Chatroulette. Facebook, Twitter, Youtube. Attention in the social net.
*ManyCam [http://www.manycam.com/]
*PS3 eye
*jennicam [http://www.wired.com/thisdayintech/2010/04/0414jennicam-launches wired]
*Lonelygirl15 [http://www.youtube.com/watch?v=-goXKtd6cPo youtube] [http://www.wired.com/wired/archive/14.12/lonelygirl.html article]
*Discuss telematic perfromance.
* Justin.tv [http://www.justin.tv/#r=s7RVqBU~]
*Read: The Presentation of Self in Everyday Life (excerpt). Erving Goffman. 1959.
*Read: Performance: A Critical Introduction (excerpt). Richard Carlson. 2004.
*Do: Intervention in social circuits. Chatroulette/Facebook/Youtube exercise.

'''Social Networks/Web 2.0'''
*Read: Protocol, Control, and Networks by Alexander Galloway and Eugene Thacker. Grey Room 17, Fall 2004 p 6-29.
*Read: DIGITAL MAOISM: The Hazards of the New Online Collectivism. Jaron Lanier. 2006.
*Watch: MediatedCultures @ Kansas State http://mediatedcultures.net/mediatedculture.htm
*Datamining/Complex Networks, node-edge graphing.

'''Digital Memory/Personal Media: Where do we exist and how do we remember?'''
*Read: Mediated Memories in the Digital Age (excerpt). Jose van Dijck. 2007.
*Read: Are you sure you want to do this? Matthias Fuchs 1994.
*Read: Delete: The Virtue of Forgetting in the Digital Age (excerpt). Viktor Mayer-Schonberger. 2009.
*Flickr.com, Facebook
*Discuss: My Pocket. Burak Arikan. 2008.

'''Cognition + Creativity'''
*Generative Art vs. Computational Creativity
*Casy Reas
*Processing.org
*Tom Shannon. [http://www.wired.com/magazine/2010/03/pl_arts_pendulum/all/1]
*Read: Triumph of the Cyborg Composer.
*Read: How to draw three people in a garden. 1988.
*Read: Shades of Computational Evocation and Meaning: The GRIOT System and Improvisational Poetry Generation. 2006.

'''Artificial Intelligence'''
*Read: Expressive Processing (excerpt), Noah Wardrip Fruin, 2009.
*Read: Elephants Don't Play Chess, Rodney Brooks, 1990.

'''Appropriation and Remix'''
*Read: The Fiction of Memory. New York Times, March 12, 2010. Luc Sante
*Read: Jonatham Lethem. The Ecstasy of Influence. Harpers Magazine. 2007.
*Remix Culture. Lev.
*God's Little Toys: Confessions of a cut & paste artist. William Gibson. 2005. *http://www.wired.com/wired/archive/13.07/gibson.html
*Reality Hunger: A Manifesto. David Shields. 2010.

'''Materiality in the information age.'''
*Tangible interfaces, haptic feedback.
*Read: Evocative Objects: Things We Think With (excerpt). Sherry Turkle, 2007.
*Read: New Media and the Forensic Imagination (excerpt). Matthew Kirschenbaum. 2008.
*View: BIT Plane.
*View: Garbage Cubes
*Discuss techniques of markerless tracking, augmented reality, QR codes, etc. *Online/Offline Space.

'''Embodiment'''
*Computing with bodies, engineered bodies
*tactile media, haptic interface
*embodied perception
*Read: Stelarc.

'''Self-Image'''
*Read: Self/Image: Technology, Representation, and the Contemporary Subject (excerpt). Amelia Jones, 2006.
*Do: Forensic Photoshop Exercise.
*http://www.flickr.com/photos/dryponder/sets/72157623726710218/
*http://nymag.com/daily/intel/2010/02/obama_being_forced_to_look_at.html#photo=1
*http://bubleraptor.tumblr.com/
*photoshop free Marie Claire issue: http://jezebel.com/5511507/so-long-as-your-face-looks-alright-everything-else-can-be-photoshopped

== Places to Find Art ==
* http://we-make-money-not-art.com/
* http://www.isea-web.org/, http://www.isea2010ruhr.org/
* http://www.transmediale.de/en
* http://01sj.org/
* http://www.file.org.br/
* http://www.aec.at/festival_about_en.php
* http://www.sciencegallery.com/lightwave09
* Institutions that Sponsor/Show Media Art
** Eyebeam New York City
** New Museum/Rhizome.org http://rhizome.org
** HarvestWorks
** Machine Project, Los Angeles.

== Midterm Projects ==
Make pages here.
* [[DummyProject | Dummy Project]]
* [[MidtermProject| MotionDJ - Leilani Martin]]

== Student Pages ==
Click "edit" on the right to add your own page below.
* [[Students/RobertTwomey | RobertTwomey]]
* [[Students/Javier Lee | Javier Lee]]
* [[Students/Jenny Wang | Jenny Wang]]
* [[Students/Joeny Thipsidakhom | Joeny Thipsidakhom]]
* [[Students/Kuan-Ting Lu | Tony Lu]]
* [[Students/Jezreel Callejas| Jezreel Callejas]]
* [[Students/ChristinaSanchez| Christina Sanchez]]
* [[Students/BenBrickley | BenBrickley]]
* [[Students/Ellen Huang | Ellen Huang]]
* [[Students/Kelley Kim | Kelley Kim]]
* [[Students/EmilioMarcelino | EmilioMarcelino]]
* [[Students/Anna Lin | Anna Lin]]
* [[Student/Jenny Chang | Jenny Chang]]
* [[Student/Jet Antonio | Jet Antonio]]
* [[Students/GregoryParsons | Gregory Parsons]]
* [[Students/Jennifer Sunga | Jennifer Sunga]]
* [[Students/LeilaniMartin | Leilani Martin]]

Classes/2010/VIS145B

2010-04-15T22:14:27Z

Leilanim14: student page

== Time and Process Based Digital Media II ==
Time: Thursdays 3:30-6:20pm, VAF 228

This class is an advanced study and portfolio project course centered on the use of hardware and software to create interactive and time-based art. These projects can take many forms—interactive installations, dynamic visualizations/sonifications, printed renderings—chosen by the students. This will not be a course of technical instruction—rather we will consider technical and conceptual issues in tandem, supplementing discussions and activities with specific technical instruction where necessary. There is a strong emphasis on the development and articulation of personal directions of research by the students in the course.

I would like to split the reading/homework responsibility for two parts of the class. In the first half of the term I will present a series of works and readings covering my particular interests--the intersections of social performance, embodied experience, and cognition. In the latter half of the class (after the midterm) you all will do the presentations on topics of your choosing. Working individually or in small groups, you will provide us with some conceptual provocation (reading material) covering topics you intend to engage with your final, and you will lead a discussion on technical and conceptual issues. Reading and critical writing, in response to text and works you present and those I present, are integral to this course.

The schedule is a living document and will be revised over the period of the course.

== Instructor ==
Robert Twomey

rtwomey@ucsd.edu
*http://roberttwomey.com
*http://experimentalgamelab.net
*http://crca.ucsd.edu

Office Hours: Wednesday 3-4pm, Atkinson Hall Rm 1601 (CRCA research neighborhood). Please e-mail me if you plan to attend.

== Grading ==
*Midterm Project - 30%
*Final Project - 40%
*Presentations - 10%
*Readings - 10%
*Participation - 10%

=== Presentations ===
(1) Short presentation on your work in the second week of class. This should be a statement of your interests, direction, goals with media art. Present examples from your own work which you feel strongly about, and which best represent your interests and trajectory. Present examples of other artist's work that serve as models for the kind of work you would like to make. (5-10 minutes each)

(2) Medium presentation on final projects in the second semester of the course (weeks 7-9). This is the portion of the class where you dictate the reading and the discussion. If you are presenting on a given week, you need to provide us with a reading 1 week in advance. We will sign up for those time slots in week 6, just after the midterm. (10-15 minutes)

=== Reading Responses ===
These are written summaries and critical responses to materials assigned for out of class viewing. Things to consider: What points does the author make? Do you buy their assumptions or agree with their conclusions? Reading responses will be printed and turned in to the instructor at the beginning of class. Generally these should be 1 page long.

=== Projects ===
Midterm and final projects will be graded on concept, effort, and realization. Formal proposals are a necessary component of the process so take them seriously. Make the effort to get started early and seek the help you need--we want to see finished, well-considered pieces for the midterm and final. Additionally, you will need to submit documentation of the project after completion which includes images, video, and source code where applicable. These materials (proposals and documentation) will all be posted to the wiki.
=== Documentation Policy ===
*personal wiki page
*source code on wiki
*image/video documentation as appropriate.
*explanatory writing (on intent, motivation, context)

=== Attendance ===
Attendance is mandatory. Each unexcused absence will drop your final grade one letter. There are only 10 weeks of class, please come to them all.

== Schedule ==
=== Week 1 - Intro ===
*Introductions
*Scope of course, interests, technical possibilities.
*My work.
*Watch: We Live In Public. 2009. (excerpts)
*In class: personal page on wiki. [http://www.trsp.net/teaching/gamemod/ game-mod exercise]. [http://www.trsp.net/teaching/gamemod/gamemod_breakout_source_en.zip download link]
*Read: [http://www.nyu.edu/projects/xdesign/mainmenu/archive_tangible.html Against Virtualized Information], [http://www.nyu.edu/projects/xdesign/mainmenu/archive_analtictech.html Novel Analytic Techniques], and [http://www.nyu.edu/projects/xdesign/mainmenu/archive_infocounts.html What Information Counts?] by [http://www.environmentalhealthclinic.net/people/natalie-jeremijenko/ Natalie Jeremijenko].
*Read: [http://www.yalealumnimagazine.com/issues/2004_03/jeremijenko.html An Engineer for the Avante Garde]
*Read: [http://www.worldchanging.com/archives/001450.html Natalie Jeremijenko The WorldChanging Interview]
*Read: [http://tech90s.walkerart.org/nj/transcript/nj_01.html Database Politics and Social Simulations], good background on her earlier artwork.

=== Week 2 - Student Research Interests ===
*Due: 1 page on Jeremijenko.
*Presentations on your work.
*Read: [http://www.flong.com/texts/essays/essay_cvad/ Computer Vision for Artists and Designers: Pedagogic Tools and Techniques for Novice Programmers] Golan Levin. ''pay particular attention to part II. ELEMENTARY COMPUTER VISION TECHNIQUES. we are going to try these in class next week.''

=== Week 3 - Computer Vision / Human Perception ===
*Due: Nothing. Read the Golan Levin piece, but no written response.
*Discuss:
**Myron Kreuger. Video Place. 1989 [http://www.youtube.com/watch?v=dqZyZrN3Pl0]
**Text Rain. Camille Utterback & Romy Achituv. 1999. [http://www.youtube.com/watch?v=toWFvXHghDk] [http://www.camilleutterback.com/]
**Very Nervous System. 1982-1991. [http://vimeo.com/8120954]
**Suicide Box. Bureau of Inverse Technology. 1996. (13:00)
**Marie Sester. ACCESS. 2003. [http://accessproject.net]
**Messa di Voce. Golan Levin and Zach Lieberman with Jaap Blonk and Joan La Barbara. 2003. [http://www.flong.com/projects/messa/] [http://www.tmema.org/messa/messa.html]
**Seen. David Rokeby. 2002. [http://vimeo.com/6012986]
**Sorting Daemon. David Rokeby. 2003. [http://homepage.mac.com/davidrokeby/sorting.html]
**Cheese. Christian Moller. 2003. [http://www.christian-moeller.com/display.php?project_id=36] made in collaboration with UCSD [http://mplab.ucsd.edu/wordpress/ Machine Perception Lab]
**Eyewriter. 2009 [http://www.eyewriter.org/]
**Saccade. 2010 [http://roberttwomey.com/saccade] (in progress)
*Discuss:
**thresholding
**frame difference
**OpenCV - [http://ubaa.net/shared/processing/opencv/ download] [http://www.cs.unc.edu/Research/stc/FAQs/OpenCV/OpenCVReferenceManual.pdf reference manual]. If you are getting this for your computer, be sure to get OpenCV, the OpenCV Processing Library, and the OpenCV Processing Examples (three separate downloads).
**face recognition
*In Class:
**Working alone or in small groups, do experiments with video processing and computer vision.

=== Week 4 - Midterm Workshop ===
*Due: Midterm project proposal.
**Working individually or in small groups (2-3 people), produce an interactive piece that bridges the gap between screen space and physical space. There are many ways to do this--using image-based computer vision techniques, game controllers, audio input, or other physical hardware (Arduino?). Think about the parameters of interaction--are you documenting viewer's behavior (unknown to them), are you taking a familiar form (such as a video game) and tweaking it in some way, are you intervening in social space? Think about what form the output will take. In your one page proposal, describe the input(s), output(s), and dynamic of interaction, as well as some statement of your motivation. Why is this a valuable or interesting project? In addition to the written description, produce supporting visual materials. These should be two functional diagram images and two visual/aesthetic images. The functional diagrams should show the necessary software and hardware components and explain how the interaction will occur. The aesthetic diagrams will give us a sense of what it will look like, how the output will appear. Make a page for your project (including a title) in the Midterm Projects section at the bottom of this page, upload the necessary materials and embed them in that page. This proposal is due in class next week where we will critique and workshop the ideas.
*Review computer vision experiments from last class.
*Work on midterm projects.

=== Week 5 - Midterm Critiques ===
In-class critiques of midterm projects.

=== Week 6 ===
=== Week 7 ===
Student presentations
=== Week 8 ===
Student presentations
=== Week 9 ===
Student presentations
=== Week 10 - Final Critiques ===
In-class Critiques.

=== Finals Week ===
Final documentation due.

== Topics ==
To Be Scheduled

'''Performance for the camera, for the web'''
*Discuss Chatroulette. Facebook, Twitter, Youtube. Attention in the social net.
*ManyCam [http://www.manycam.com/]
*PS3 eye
*jennicam [http://www.wired.com/thisdayintech/2010/04/0414jennicam-launches wired]
*Lonelygirl15 [http://www.youtube.com/watch?v=-goXKtd6cPo youtube] [http://www.wired.com/wired/archive/14.12/lonelygirl.html article]
*Discuss telematic perfromance.
* Justin.tv [http://www.justin.tv/#r=s7RVqBU~]
*Read: The Presentation of Self in Everyday Life (excerpt). Erving Goffman. 1959.
*Read: Performance: A Critical Introduction (excerpt). Richard Carlson. 2004.
*Do: Intervention in social circuits. Chatroulette/Facebook/Youtube exercise.

'''Social Networks/Web 2.0'''
*Read: Protocol, Control, and Networks by Alexander Galloway and Eugene Thacker. Grey Room 17, Fall 2004 p 6-29.
*Read: DIGITAL MAOISM: The Hazards of the New Online Collectivism. Jaron Lanier. 2006.
*Watch: MediatedCultures @ Kansas State http://mediatedcultures.net/mediatedculture.htm
*Datamining/Complex Networks, node-edge graphing.

'''Digital Memory/Personal Media: Where do we exist and how do we remember?'''
*Read: Mediated Memories in the Digital Age (excerpt). Jose van Dijck. 2007.
*Read: Are you sure you want to do this? Matthias Fuchs 1994.
*Read: Delete: The Virtue of Forgetting in the Digital Age (excerpt). Viktor Mayer-Schonberger. 2009.
*Flickr.com, Facebook
*Discuss: My Pocket. Burak Arikan. 2008.

'''Cognition + Creativity'''
*Generative Art vs. Computational Creativity
*Casy Reas
*Processing.org
*Tom Shannon. [http://www.wired.com/magazine/2010/03/pl_arts_pendulum/all/1]
*Read: Triumph of the Cyborg Composer.
*Read: How to draw three people in a garden. 1988.
*Read: Shades of Computational Evocation and Meaning: The GRIOT System and Improvisational Poetry Generation. 2006.

'''Artificial Intelligence'''
*Read: Expressive Processing (excerpt), Noah Wardrip Fruin, 2009.
*Read: Elephants Don't Play Chess, Rodney Brooks, 1990.

'''Appropriation and Remix'''
*Read: The Fiction of Memory. New York Times, March 12, 2010. Luc Sante
*Read: Jonatham Lethem. The Ecstasy of Influence. Harpers Magazine. 2007.
*Remix Culture. Lev.
*God's Little Toys: Confessions of a cut & paste artist. William Gibson. 2005. *http://www.wired.com/wired/archive/13.07/gibson.html
*Reality Hunger: A Manifesto. David Shields. 2010.

'''Materiality in the information age.'''
*Tangible interfaces, haptic feedback.
*Read: Evocative Objects: Things We Think With (excerpt). Sherry Turkle, 2007.
*Read: New Media and the Forensic Imagination (excerpt). Matthew Kirschenbaum. 2008.
*View: BIT Plane.
*View: Garbage Cubes
*Discuss techniques of markerless tracking, augmented reality, QR codes, etc. *Online/Offline Space.

'''Embodiment'''
*Computing with bodies, engineered bodies
*tactile media, haptic interface
*embodied perception
*Read: Stelarc.

'''Self-Image'''
*Read: Self/Image: Technology, Representation, and the Contemporary Subject (excerpt). Amelia Jones, 2006.
*Do: Forensic Photoshop Exercise.
*http://www.flickr.com/photos/dryponder/sets/72157623726710218/
*http://nymag.com/daily/intel/2010/02/obama_being_forced_to_look_at.html#photo=1
*http://bubleraptor.tumblr.com/
*photoshop free Marie Claire issue: http://jezebel.com/5511507/so-long-as-your-face-looks-alright-everything-else-can-be-photoshopped

== Places to Find Art ==
* http://we-make-money-not-art.com/
* http://www.isea-web.org/, http://www.isea2010ruhr.org/
* http://www.transmediale.de/en
* http://01sj.org/
* http://www.file.org.br/
* http://www.aec.at/festival_about_en.php
* http://www.sciencegallery.com/lightwave09
* Institutions that Sponsor/Show Media Art
** Eyebeam New York City
** New Museum/Rhizome.org http://rhizome.org
** HarvestWorks
** Machine Project, Los Angeles.

== Midterm Projects ==
Make pages here.
* [[DummyProject | Dummy Project]]

== Student Pages ==
Click "edit" on the right to add your own page below.
* [[Students/RobertTwomey | RobertTwomey]]
* [[Students/ Javier Lee | Javier Lee]]
* [[Students/Jenny Wang | Jenny Wang]]
* [[Students/Joeny Thipsidakhom | Joeny Thipsidakhom]]
* [[Students/Kuan-Ting Lu | Tony Lu]]
* [[Students/Jezreel Callejas| Jezreel Callejas]]
* [[Students/ChristinaSanchez| Christina Sanchez]]
* [[Students/BenBrickley | BenBrickley]]
* [[Students/Ellen Huang | Ellen Huang]]
* [[Students/Kelley Kim | Kelley Kim]]
* [[Students/EmilioMarcelino | EmilioMarcelino]]
* [[Students/Anna Lin | Anna Lin]]
* [[Student/Jenny Chang | Jenny Chang]]
* [[Student/Jet Antonio | Jet Antonio]]
* [[Students/GregoryParsons | Gregory Parsons]]
* [[Students/Jennifer Sunga | Jennifer Sunga]]

Leilani Martin

ICAM music major, sports fanatic, and collegiate athlete. I've just began my studies of computer music and mostly interested in generating/manipulating sounds.