EyeMusic: Performing Live Music and Multimedia Compositions with Eye Movements

eye-controlled piano

fingerplay is basically a sample player. webcam is used to track movement (of the actors hands) and analyse these movements. the resulting parameters are converted to suitable control parameters for sampling engine to achieve a touchless audio control system in the tradition of the theremin.

fingerplay

More tutorials:

Max/MSP Tutorial files
Max Tutorials, PDF file (2.5mb)

General Resources, Max Patches and Objects, Max-friendly Hardware, Max Learning Materials, Selected Projects Using Max/MSP/Jitter

http://www.cuttlefish.org/maxmspjitter.html

Cv.jit

Pelletier's toolkit, which is the most feature-rich of the three vision plug-ins(David Rokeby's SoftVNS, Eric Singer's Cyclops, and Jean-Marc Pelletier's CV.Jit) is also the only which is freeware.

Cv.jit is a collection of max/msp/jitter tools for computer vision applications. The goals of this project are to provide externals and abstractions to assist users in tasks such as image segmentation, shape and gesture recognition, motion tracking, etc. as well as to provide educational tools that outline the basics of computer vision techniques.

http://www.iamas.ac.jp/~jovan02/cv/

Cyclops Realtime Video Analyzer:

Through a simple interface, you can track performers and users with a video camera and analyze greyscale and color information from the live image. You can use the resulting Cyclops data to control MIDI, audio, video and anything else that you can do in Max.

How Cyclops Works

Cyclops receives live video from a QuickTime input source and analyzes each frame of captured video in real-time. It divides the image area into a grid of rectangular zones and analyzes the zones for greyscale, threshold, difference (motion) and color. Cyclops allows you to specify the grid resolution, target zones for analysis and indicate the type of analysis to be performed in each zone. Cyclops outputs messages for each analyzed video frame that can be used to trigger any Max processes or control any patch parameters.

http://www.cycling74.com/products/cyclops

About computer vision and interactive art :

http://www.flong.com/writings/texts/essay_cvad.html

The various computer vision plug-ins for Max/MSP/Jitter, such as David Rokeby's SoftVNS, Eric Singer's Cyclops, and Jean-Marc Pelletier's CV.Jit, can be used to trigger any Max processes or control any system parameters. Pelletier's toolkit, which is the most feature-rich of the three, is also the only which is freeware.
CV.Jit provides abstractions to assist users in tasks such as image segmentation, shape and gesture recognition, motion tracking, etc. as well as educational tools that outline the basics of computer vision techniques [Pelletier].

Project Description

GraphBeat project is about creating a digital space where users can make their own graphics and rhythms by live video tracking.User will form a visual and aural composition.

Objectives

The main objective of this project is developing a platform where users can create their own unique graphics and musics.

Goals

Max/Msp CV.Jit, tracking live video input
Forming specific rhythms
Creating and programming an interactive platform

Challanges

Project will be done using Max/MSP/Jitter. I have to learn Max.

Scope

Objects each having unique rhythms can be added and with the interaction between user and the objects can form a new composition.

Project Schedule

10 - 15.11 ~ Max/Msp CV.Jit, tracking live video input
15 - 25.11 ~ organizing visuals, rhythms, programming
25 - 30.11 ~ simple demostration of the system
01 - 10.12 ~ simple working demostration of the system

Project pdf and Gantt chart http://160.75.53.220/ozhararb/GraphBeat.pdf