A Mouse-driven Graphic Equalizer

How to sample the speed of your mouse

The Concept

Robert O'Rourke was talking about making graphic equalizers with Processing.js in the Google Group, which I thought was a pretty sweet idea. Coming from a video-performance background, I have experimented with 3D graphic equalizers in programs like Blender 3D a few times before: example; so I thought it would be fun to play around with this idea myself.

Wanting to keep the process closely paralleled with audio techniques, I figured a an audio-buffer equivalent would be a good place to start. The mouse-buffer would store the mouse's speed at frame intervals and then paint the buffer to the canvas in a beautiful curveVertex() array.

The script you see running reads the mouse position on every void draw() frame and checks it against the mouse's XY values from the previous frame. This allows us to calculate the speed that the user has moved the mouse in one frame and that value is stored in the lastDistXY[array] (the mouse-buffer).

In the first version of the code, I was drawing the curve after the buffer had been filled with samples. This made the curve appear to 'stutter' as the curve would not move again until the buffer had been re-filled. In the audio domain, this is not such a problem as there are 1.4 million samples per second with CD audio, so the display can be updated very quickly before the buffer needs to be filled again. If you have a decent graphic equalizer to hand... try changing the sample size up to 2048 and see how the refresh begins to 'stutter' with CD quality audio.

When we measure the movement of a mouse in a 320 x 240 sized canvas over the duration of a single frame, we have drastically reduced our sampling possibilities... or 'dynamic range'.

320 x 240 x 2 (X & Y) = 153,600

So to create the illusion of a greater sample rate and higher dynamic range, I built a very simple algorithm to morph between the last curve position and the next, using the variable called rate to control the morph speed. I then added the curve drawing routine into the main draw() loop, so that the line is constantly bending. I also added a couple of graphic features to inform the user of the sample speed, position, and used the background color to visually demonstrate the delay in playback.

Things you should try
  1. Try to create a constant deep blue background colour using steady mouse movement.
  2. Use the HTML form to update the sample rate to 32 and try to create a regular blue flash.
  3. Update the sample rate to 64 and move the mouse suddenly creating a rhythm. Watch how the background flashes blue as the transport indicator reaches the recorded samples in the previous buffer, playing back your rhythm.
The Code
// Mouse-Motion to Frequency
// By F1LT3R - http://groups.google.com/group/processingjs

void setup(){
  size(320, 240);
  strokeWeight(2);
  stroke(0);
  noFill();  
}

// Generic Looping Variables
int n, i;

// Controls the morph speed of the curve
int rate=10;

// Set Last Mouse X & Y for dist check
int lastMouseX, lastMouseY;
int distX, distY, lastDistXY;

// Set sample rate and buffer arrays
int sampleRate = 16, sampleLoc = 0;
int lastSampleRate = sampleRate;
int[] distXY = new int[sampleRate];
int[] nextDistXY = new int[sampleRate]; 

// Fill original target buffer
for (i=0; i < sampleRate; i++) { 
  distXY[i] = height / 2;
} 


void draw(){ 
  
  // If the samplerate is set in the window...
  if (window.Processing.data.sampleRate){        
    // Get the sample rate from the cwindow
    sampleRate = window.Processing.data.sampleRate;    
    // If the sample rate has changed since last draw...    
    if (sampleRate!=lastSampleRate){      
      // Re-dimension buffers and transport indcator
      lastSampleRate = sampleRate;
      distXY = new int[sampleRate];
      nextDistXY = new int[sampleRate];
      sampleLoc=0;      
    }    
  }
  
  // Set background color to to amplitude of sampleLoc   
  colorDelay = (255/sampleRate) * distXY[sampleLoc];  
  background(0, colorDelay/2, colorDelay, 100);
  
  // Get mouse inputs
  readMouse();
  
  // Loop through buffer for length of sample rate
  sampleLoc ++;
  
  // Draw transport indicator for pos in sample buffer
  fill(#0088aa);
  stroke(#006688);
  rect((((width-1)/sampleRate)*sampleLoc)-10, height-(height/20), 20, (height/20)-1);
  
  // Reset playback location when end of buffer reached 
  if (sampleLoc==sampleRate){sampleLoc = 0;}           
  
  // Loop through samples
  for (i=0; i < sampleRate; i++){      
    
    // Composite sample rate ruler over playback indicator 
    stroke(255, 255, 255, 100);
    noFill();
    strokeWeight(1);
    rect(((width-1)/sampleRate)*i, height-(height/20), ((width-1)/sampleRate)-3, (height/20)-1);
    
    // Morph old curve into new curve
    if (nextDistXY[i] < distXY[i]) {    
        distXY[i] = distXY[i] - ((distXY[i] - nextDistXY[i]) / rate);
    } else if (nextDistXY[i] > distXY[i]){
        distXY[i] = distXY[i] + ((nextDistXY[i] - distXY[i]) / rate);
    }
  }
  
  // Draw curve
  noFill();
  stroke(255);
  strokeWeight(4);
  beginShape();
  curveVertex( (width/(sampleRate-1))*0, height/2 - distXY[0] );
  for (n=0; n < sampleRate; n++){    
    curveVertex( (width/(sampleRate-1))*n, height/2 - distXY[n] );    
  }
  curveVertex( (width/(sampleRate-1))*n-1, height/2 - distXY[n-1] );
  endShape();
    
}

void readMouse(){  
  // Check lastMouseX has value to stop initial jump   
  if (!isNaN(lastMouseX)){               
    
    // Calculate MouseX Movement Distance
    if (mouseX > lastMouseX){distX = mouseX - lastMouseX;}
    else {distX = lastMouseX - mouseX;}
    
    // Calculate MouseY Movement Distance
    if (mouseY > lastMouseY){distY = mouseY - lastMouseY;}
    else {distY = lastMouseY - mouseY;}
    
    //Combine total distance traveled
    nextDistXY[sampleLoc] = distX + distY;
    lastDistXY = distX + distY;    
  }
  
  // Setup lastMouseX & Y for next movement
  lastMouseX = mouseX;
  lastMouseY = mouseY;
}

Code for the example above.