UMV

Title

You Music Video (uMV)

Description

• Motivation

I've had interest in rotoscoping and animation in videos and dance for a while - and I'm interesting in mixing them, live.

• Interactive paradigm

As the person dances/moves, animation or other features will react to their movement, enhancing it on video, and basically creating a music video on the spot.

• Technical Description

The person would be caught and tracked on camera, and we could use frame differencing to track the magnitude of his or her movements and the pre-coded/drawn animations would react accordingly. Background images can be added and changed over time to improve the effect.

Visual Concept

• Reference:

http://www.motiontheory.com/work/nike_presto-04

View video 'Urban Canvas'.

http://www.motiontheory.com/work/nike_flow

View video 'Flow'.

Documentation

• Source Code:

-coding consists of initialization of camera, bitmap manipulating to create motion-detecting/color/background editing effects, and some animating code. -camera code and motion detecting code is adapted from [[1]], with some changes for color and background editing effects.

CAMERA: -sets up webcam and shows security panel

import flash.media.Camera; import flash.system.Security; import flash.system.SecurityPanel; import flash.events.*;

var cam:Camera = Camera.getCamera(); if(cam != null){ cam.setMode(video1.width, video1.height, 12, true); video1.attachCamera(cam); cam.addEventListener(ActivityEvent.ACTIVITY, activityHandler); }else{ Security.showSettings(SecurityPanel.CAMERA) } var inited:Boolean=false; function activityHandler(event:ActivityEvent):void { if(cam.muted){ Security.showSettings(SecurityPanel.CAMERA) }else if(!inited){ initialize(); inited=true; }

    //trace("activityHandler: " + event);

}

BITMAP: import flash.display.Bitmap; import flash.display.BitmapData; import flash.geom.Matrix; import flash.utils.Timer; import flash.events.TimerEvent;

var now:BitmapData; var out:BitmapData; var before:BitmapData; var bitmap:Bitmap; var timer:Timer; var m:Matrix; var max:int = 7; function initialize(){ //create an array that will be the fading image cue list=new Array() //create an identity matrix m=new Matrix() //scale the transformation matrix to match the scale of the Video object on stage m.scale((video1.scaleX),(video1.scaleY)) //create a bitmap object in memory that is the same size as the Video object on stage now=new BitmapData(video1.width, video1.height, true, 0x00000000); //create a bitmap object in memory to hold a layered bitmap from the fading image cue out=new BitmapData(video1.width, video1.height, true, 0x00000000); //out.draw(video1,m); before = out.clone(); bitmap=new Bitmap(); //create a bitmap to display the bitmapdata this.addChild(bitmap); //position it to the right of the live Webcam feed //bitmap.x=video1.x+video1.width; bitmap.x=video1.x; bitmap.y=video1.y; //call the snap function every 1/10th of a second timer = new Timer(100);

   timer.addEventListener(TimerEvent.TIMER, snap);
   timer.start();

} //capture a freezeframe from the webcam and copy it, called regularly do detect movement function snap(event:TimerEvent){

bitmap.bitmapData=out; //draw the video into the bitmapdata object object with the scaled transformation matrix now.draw(video1,m) //copy the current freezeframe done=now.clone() //now draw the previous freezeframe ("BEFORE"!) on top of the current frameframe and apply the 'difference' blendmode done.draw(before,m,null,"difference")

       //filter out all pixels in the differenced bitmap that are greater than just over black (0xFF111111) and make them green (0xFF00FF00)

//over-write the previous contents of the bitmap done.threshold(done,done.rect,done.rect.topLeft,">",0xFF707070,0xFF00FF00,0xFFFFFFFF,true) //important line, used to set threshold of colors and change background to black while movement/people to green

//add the finished bitmap to a fading image cue list.push(done.clone()) //if the cue is now greater than the maximum length we specified if(list.length > max) { //then remove the bitmap at the very front of the cue //and free the memory it was using list.shift().dispose() } //how many items in the fading cue var l=list.length //how much should the green channel deteriate on each degradation var each=255/l //declare local var to speed up the loop var g //var changed:uint //clear the out bitmap object so we can start a fresh out.fillRect(out.rect,0xFF000000) //copy the current bitmap and save it for the next time this function is called before=now.clone() //loop through each bitmap in the fading image cue for(var i=0;i<l;++i) { //determine the current degradation g=each*i

out.threshold(list[i], out.rect, out.rect.topLeft, "<=", 0xFF707070, 0x00000000, 0xFF000000, true); //this line was added to make unimportant pixels transparent - used for background changing

               //copy all the pixels that are green in the current item in the fading image cue and make them into the degradated green color

//add those pixels on top of the bitmap that will displayed to the user out.threshold(list[i],out.rect,out.rect.topLeft,"==",0xFF00FF00,(255<<24 | 0<<16 | g<<8 | 0),0x00FFFFFF,false)

}