Final Project and Final Thoughts

Well, I never quite knew what to expect when I signed on for PhysComp. Several of the shorter assignments were helpful for grasping a very baseline introduction to the materials and methods that ITP artists use in their work. But, like following recipes, until you have the confidence to try out your own variations, you're not really thinking for yourself. I think there could have been some additional intermediary steps taken to make the experience seem less like being "thrown to the wolves". I can't speak for everyone, but I definitely could have used a simple assignment that requires building a specific style enclosure out of wood, a hands-on primer in sensor wiring, securing and strain relief, but most of this was left to schematic abstractions on the board and cold readings. But ultimately, a lot depends on defining clearly what it is you'd like to do in the class, and aside from knowing that I wanted to focus on sound, I had no real game plan and not much prior exposure to tech innovations in the arts.

In a final project I went for something modular, a system that could be flexibly adapted for the express purpose of allowing for a multiplicity of choices in making music. In retrospect, this was probably too large in scope and vague. Most successful projects seemed to have one or two clearly-defined goals in what the user was supposed to achieve. In other projects, much emphasis was on making an object that was fun for a user to interact with. Music, by contrast, isn't necessarily fun, and certainly by tradition it isn't interactive with audiences in a physical sense. But I wanted a way to interact with myself so I could eventually perform as a wind soloist. No impressive visuals, just sound, and to make something impressive in this way requires a huge learning curve that for me goes well beyond the time frame of one semester. At least the exposure turned me on to a new realm of possibilities.

I began with the premise of wanting to make sonic use for the extra body motion I was using to play sax, an answer of sorts to earlier critics of my highly gestural playing. I didn't necessarily want an electronic wind controller, but I wanted my pure tone to be processible, and for other gestures to trigger some algorithmic possibilities in the music. I also wanted to trigger accompanying sound files in interesting, algorithmic ways - shape my own amplitude and rhythmic envelopes, add and vary delay lines, and play around with spatialization somewhat. I came from the standpoint of learning first some basic sound sculpting procedures in MAX/MSP, and then when I had what I wanted, figuring out a way to control them hands-free so I could also focus on playing the horn. Trouble was, the MAX platform itself is a steep learning curve, but having struggled with it once before, I wanted this time around for my progress to be real and palpable.

I finished - with the help of Peter McCullough - a 4-channel DJ rig based on rhythmic buffers (reading from slices of those sound files with lookup tables set to multislider presets, so the effect is a loop, but also a varying probability of "slice" playback), and added some sax processing capabilities including sample-and-hold record/loop functions (with random step walking), spatially-driven delay and pitchshifting effects, and harmonizing. My comfort zone in providing a serial link to these functions from the outside world lay in MIDI controller messages, and I became interested particularly in a Behringer foot module and Eric Singer's MIDItron as my primary interfaces, but only when previous attempts at designing my own failed. Some of these failures included flex sensing on my elbow pit (the sensor snapped off in class and, even after subsequent reinforcement - through heat shrinking, hot glue, stitching, velcro adhesive, the works - lost its responsiveness), FSRs (which worked well with foot pressure but which limited my mobility because I was wearing the MIDItron transmitter), and accelerometers (I soldered these incorrectly and ran out of time before I could reverse my failure). I did, however, manage to make a stretch sensor part of my foot module, and this was able to provide me with varying feedback in one delay line. I also discovered the joys of the Wii, and with a translation program called the Osculator could send MIDI cc messages in response to tilt and roll with it mounted to my arm. This became effective in controlling the speed of the delay on my sax and also the steps in pitchshifting taken in that delay. There was no ulterior motive in choosing my methods, just simply to choose the different components that worked best for what I wanted to accomplish.

The MIDI lab did provide some additional insight and, time permitting, I might have chosen to do one or more retrofitted footswitches that communicate MIDI messages via the Arduino. None of the ground triggers would have to be wireless, and in fact one issue I couldn't resolve was how to make the MIDItron truly wireless (there is no connection from transmitter to receiver, but every sensor is connected to the transmitter), or at least less confining. If I had an FSR affixed to the ground I couldn't easily wear the MIDItron and also have it connected to an arm sensor without risk of yanking wires free, and certainly any walking mobility would be compromised. So in a modular performance rig with many inputs there may need to be a separation between ground inputs and body ones (this is essentially the route I took, but with the Behringer/stretch sensor/MIDItron (not on my body) and Wii (on my body)). The Wii works so well I could see attaching one to every limb. Since I understand it to have a gyroscope sensor, with some long-term work a smaller, more stealthy version of it could be developed. I also think it's reasonable to develop a simpler wearable project that is designed to do just 1 or 2 things sonically, and incorporates soft circuits (an Arduino Lilypad, conductive thread stitching) and a small wireless transmitter (XBee system) to a speaker or some other output. This is more aligned with the spirit of PhysComp, an object-oriented interface (the one wearable item) that anyone can use and have fun with. That said, in future iterations of my modular music rig I could see giving the audience some control by handing out the Wiis.

The toe-raising on an FSR during a climactic note was part of my live demo in class, fairly effective in terms of making my body movement work for me, and I'd like one day to find a larger, more rugged FSR and make a larger square that with foot pressure can apply the needed mechanical pressure to send out usable values (this vision is a dreamy abstraction, I have no idea how practically to implement this or I might have done so sooner). But these results too can be accomplished with the Wii affixed to a leg calf, so it all comes down to: how much time do I want to put into the physical interface vs. the programming vs. actually performing. If the point was to have a sellable product to mass-market, the physical interface becomes more important, along with cost issues. For me, I'm happy with something that just I can use.

In my video demo I kept the trigger types to a minimum so it's clearer what I'm doing with what. I used mostly one channel of playback (Behringer switch triggered), a sample-and-hold loop of a recorded sax phrase being retriggered to random pitches (the record and loop functions are Behringer switch triggered), a sax delay line that varies in speed according to Wii tilt and varies in pitch of the echoes according to Wii roll, a delay line on the playback channel that increases feedback proportionately to the stretch of the sensor affixed to the Behringer's continuous controller pedal (this stretch sensor is routed to the MIDItron, also laying on the floor). Had I set up two stereo speakers there would be audible a ping-pong effect on the panning of the sax delay (which in turn would be influenced by some other aspect of Wii motion such as yaw). The CC pedal is itself multi-functional - depending on bank select, it can fade in/out a channel or adjust the overall tempo.

There is a moment where I'm not sure what to do next, and it will require more actual practice for me to become fluent in performance. That said, the channel playback being continuous is a great feature that will enable me to take "breathers" while I plan my next sax move, switch to flute, load other sound buffers, or make some other transition.

Note: Sorry, at this time you will have to view my video and pics from my ftp site - type in mattsteckler dot com slash remoteuser slash PhysCompFinalDocumentation

http://www.mattsteckler.com/remoteuser/PhysCompFinalDocumentation

MIDI Lab - Extra Credit

Having already tinkered on my own with MIDI in the past - and utilized it in my final project's first iteration - I understood the logic behind the schematics and the code in this lab far quicker than in my previous labs. Above all, I can immediately jump to conclude several applications for this circuit. For my 4 Channel DJ + Sax rig, for example, I can send notein messages into MAX to toggle on/off states for the muting of each channel, in lieu of the laptop key commands I had used prior.

Most everything worked as indicated, save a couple hiccups on the way. One was making sure my correct Arduino serial port and board were selected (they'd been changed in the application from when I borrowed someone's Diecimila). Another was that I connected my MIDI-to-USB cable to the MIDI port of the Arduino with its MIDI In plug rather than Out. I lapsed thinking "going IN to the computer", but with MIDI you have to align like directions, so it's OUT going to OUT, then the USB side goes IN to the computer.


I knew ahead of time to configure a new setup in my Audio/MIDI Setup control panel so it would listen for my new device, and I also knew that I needed a MIDI-triggerable sound source, so I opened Garage Band - which automatically opens to a MIDI piano track. The first program played an ascending chromatic scale at 2/10 of a second per note (1/10 for note on, 1/10 for note off) across 2 octaves, then repeated the sequence indefinitely. I recorded the results in Garage Band with a notation view and also filmed this result:



When I moved to the second program, I first tried to use an FSR, but I only very faintly heard a sound upon pressuring the sensor and then toggle switching on, and it was a pretty low one. Perhaps if I scaled the lowest analog state to a higher pitch value I might have corrected this, but I knew the given code would work with a potentiometer since it had a direct scaling function of the pot's natural 0 to 1023 range, so I switched to that and the results were instantly clean. Most notes were within reach of a treble clef staff.


For a real music application, I imagine the separate variable resitor-pitch vs. toggle switch-note on/off scenario would get quite cumbersome. To make a more keyboard-like setup, I assume one would use pushbutton type switches, each "tuned" in the code to a different pitch in a 12 half-step octave range or something like that. To send variable velocity messages with one keystroke, the pushbutton switches would have to somehow mechanically apply pressure to some FSRs located beneath them, then those values could be scaled to an appropriate MIDI velocity range. But I'm not one to advocate reinventing the wheel, I think it's better to send messages that trigger something not so 1-to-1 as far as pitch goes, like my channel muting idea where each channel reads from a playback audio buffer. I may try 4 switches to this end.

DC Motor Lab

The added components required to complete this lab are a bit intimidating, so I need to slow down to understand every step along the circuit. I completed a preliminary lab in testing and adjusting the speed in one direction of a DC Motor using a potentiometer (videos here). One reason is that the H-bridge lab doesn't appear to explain everything sequentially, so I need to take a step back. This first lab makes sense and the motor responds well - and my cats reacted to it. I wonder if it's normal for a slight burning smell to occur; I kept it running a while and nothing burned out, but hopefully I'm not hurting the situation.

The real H-bridge lab seems to miss something in the explaining. In fact, in the photos the switches change and the LED is left out. I've set up the board exactly as instructed despite the lack of labeling on the H-bridge, and even after checking for proper component directionality, missing pieces and/or code glitches, the motor still does not turn on. I am setting up a help appointment.

After my help from my savior Jenny, it appears the H-bridge was the culprit. We thought maybe either my Arduino had burned out or something wire-related got caught in the breadboard, but just a faulty bridge and the fact that - for whatever reason (the lab online didn't seem to need this) - my motor needed a 9V external power source because it was drawing too much power off my 12V DC adaptor. She helped me get it running so fast that the only gear that I could use was a snug one that is hard to see changing direction, but you can still hear the result on video.

Simple breakthroughs happened for me on this lab at last too. For one, I'm finally starting to read through schematics with a little more precision and understanding, as well as understanding how this translate to more than one viable layout option on the breadboard. I know that's small potatoes for some, but it's important for me. I also learned which way is "up" on the H-bridge, which wasn't explained in the lab. And, headers, which were touched upon briefly in readings but never stressed online, seem to make the wiring more secure when troubleshooting. Finally, I'm just barely getting into tweaking existing code for my own purposes (I'm still not comfortable writing it out from scratch) - I was able to insert an AnalogWrite function to vary the speed of the motor, similar to the prep lab.

Lab 5 - Serial Output/Processing

So the good news is my camera's new, and even without knowing much about it it still takes better shots. Things kind of go downhill from there.

Unlike in the example, I used a photocell and pot as my two analog ins with my digital in switch. I'm starting to understand more fluidly the reason for scaling/ranging, it sometimes outputs a more stable or controllable range of data. My first send through pin 0 (the pot) emitted DEC values of around 227-228, while the 2nd time I tried a BYTE modifier in the code and my values were more á and "beta".
















I was ready to try sending serial communication using multiple modifiers. My results? In tabbed columns:
Aplha-Beta (BYTE/ASCII) 11100010-11 (BIN) 226-227 (DEC) E2-E3 (HEX) 342-343 (OCT)



Clearly, Arduino and serial i/o are communicating. When I sent BIN values for all 3 sensors, I got my continuous feed of A's until I entered a character, and in the serial monitor I received a 3-ASCII char response. Then I tried it with DEC variables and it spit back 0-255 values in response to the pot, plus values of 100 (off) and 255 (on) when I clicked the digital switch.



So why wouldn't Processing run my program properly when all other communications were verified? I can tell you I checked many factors over and over again, and checked with many people including my professor, students and PhysComp grads on the floor. I could have perhaps used a multimeter to check voltages across the circuit, but other tests ascertained it was not a hardware problem but a code one in nature. Having never seen the Processing environment before (I am not in ITP and haven't taken ICM), I can tell you the code was different enough to make things difficult to debug alone, but I picked up a few tips from others along the way, including writing in println statements at each step to ensure messages were being received. We did deduce that somehow color messages (supposedly controlled by the digital switch) were not being sent - or were sent only as pure black, rendering visibility impossible on a black background - so we changed that part of the code to a constant white value. Good, at least it's on the screen.



And it stayed mostly upper left thereafter, at first with a lot of jostling back and forth which meant control with the pot wasn't smooth and obvious, until some extra tweaks like ridding the code of tabs in the printout, which slowed the ball down considerably. But at least some tangible result was reached. I owe my newest ITP floor buddy Kristen a lot in helping me understand Processing better, so I will take this as a "whatever doesn't kill you makes you stronger" moment. Thanks, Kristen!

Lab 4 - Servo; Analog Output

I'm looking heavily into upgrading my documentation capabilities with a better digital camera, having enlisted advice from Rob, ITP students, my Dad, and consumer reports online. I can't decide whether to go really high-end or just find a better portable camera (money may make this choice for me). Anyway, for kicks I checked out the worst camera from the ER for this lab and it still worked better than mine, plus I tried a couple short movies with it which you'll also find below. Today I'll go to B&H to try out some cameras.



By now I understand the basics behind preparing a simple circuit on a breadboard, and following the photos and schematics is never that hard. Still, it took me a minute to realize the servo outputs can't go directly into the Arduino but need a bypass first on the breadboard - this involves tampering with the 3 header pins with needle-nose pliers to get them long enough to seat. I color coded the analog output leads to match the servo's leads, and used one blue lead in series with my photoresistor, since I didn't own an FSR with 2 entwined leads (as shown in the lab demo picture). I followed the code exactly and it did produce physical results, a 180 degree turn with every pinch or light variation on the resistor, however the serial monitor did not display any data. Also, the motor did not respond well to a thermistor, so that and the serial issue together leave some unanswered questions.

But look at my movie!



For kicks I tried the $15 stretch sensor I'd bought a few days back - I didn't bother to solder leads to it but if I use this in a project down the road then I probably should. For lab purposes, the servo responded fine to my stretch sensor.



The main issue I'm having, and with all the labs is, what can I do with all this? I'm not conjuring anything original like some of the students have shared, and wondering when my stroke of inspiration will begin. I'm not even quite sure how to attach other blades to the servo and what kinds of things it can power, so I'll have to read some forums online I suppose. Not being an ITP student, I'm not as "immersed" in the culture here as much as some, so I took it upon myself to check out some of the advanced projects currently on the floor... knowing now how much effort goes into one little lab, I'm amazed by the things I've seen: The inkjet cartridge guitar string machine, the scantron test robot, the Torrini magician documentary box, and a seesaw with an Arduino on it that I didn't get to see work but seems really interesting too.

When it gets to project time, I would like to incorporate materials for both this class and my Adv MAX/MSP class. On the Arduino site I found an arduino2MAX object, so once I get more comfortable with MAX I should start thinking of a creative sensor to trigger a patch with.

Lab 3 - Analog Input

I'm killing time waiting to get some help... the first stage is already giving me trouble because the Arduino program on upload reads "programmer is not responding". Kind of surprised as this one looked fairly straightforward, and the code was already written for me?? Wait, I unplugged and replugged the USB cable, and remembered to hold down reset before uploading, and now the program loads. But, no light on the LED. Tried another LED, no luck. OMG, would you believe that I switched the leads in the power and ground pins on the Arduino? Now it lights, and the POT dims the LED at will.



On my second task, looks like I'll have to write the program myself. I'm still very much at the stage of "following recipes". Yes, I understand it later if it all works and I check as to why, but I'm not thinking ahead toward other possibilities at this point. I've just read that the 2 variable resistors I'll use will need to have their voltage divided prior to analog input, so I've set up the breadboard accordingly and will now have to go out and buy some flex or some other sensors. Back shortly.



Now I've come armed with photocells, thermistors and a stretch sensor (and bought the basic took kit I never had before), and have decided to start with comparing one each of the first two choices. I varied the previous code by doubling certain commands to address a second variable, renamed all my variables and... voila! The lights are pretty much completely on, and there's no visible indication that my hand pressure or the flash on my camera is affecting the LEDs.

I sought Jeff's assistance and he showed me how to run the serial monitor to see the printed values, and edit the code so the printouts read in clear columns and compute a little more slowly, for easier reading. Some variations were noticeable, but he said that in order to really see the difference in the LEDs the resultant values would have to be scaled to a range appropriate for outputting voltage, namely 0-255. That required making sense of the math and writing a good formula:

OUT = 255/(INmax - INmin) * (IN - INmin)

... where OUT is the desired voltage value to send to the LEDs, IN is the initial reading, INmin is the initial minimum value and INmax the initial maximum value. I re-wrote the code as follows and it helped my variable resistors' sensitivity much better:

int photoPin = 0; // Analog input pin that the photocell is attached to
int thermoPin = 1; // Analog input pin that the thermistor is attached to
int photoValue = 0; // value read from the photocell
int thermoValue = 0; // value read from the thermistor
int ledA = 9; // PWM pin that the LEDa is on. n.b. PWM 0 is on digital pin 9
int ledB = 10; // PWM pin that the LEDb is on. n.b. PWM 0 is on digital pin 10
int P,T;

void setup() {
// initialize serial communications at 9600 bps:
Serial.begin(9600);
pinMode(photoPin, OUTPUT);
pinMode(thermoPin, OUTPUT);
}

void loop() {
photoValue = analogRead(photoPin); // read the photocell value
thermoValue = analogRead(thermoPin); // read the thermistor value

Serial.print("photovalue; "); // print the photocell value back to the debugger pane
Serial.print(photoValue); // print the photocell value back to the debugger pane
Serial.print(" thermovalue; "); // print the thermistor value back to the debugger pane
Serial.println(thermoValue); // print the thermistor value back to the debugger pane

P = 5*(photoValue-100)/6;
T = 8*(thermoValue-90)/1;

analogWrite(ledA, P); // PWM the LED with the photocell value (divided by 4 to fit in a byte)
analogWrite(ledB, T); // PWM the LED with the thermistor value (divided by 4 to fit in a byte)

delay(100); // wait 100 milliseconds before the next loop
delay(100); // wait 100 milliseconds before the next loop
}

Scaling to a range of values is something I'll need for MAX/MSP and other applications so it was good to get clear with that in my head and in the code. To make a truly user-friendly luv-o-meter, a step I'm not yet prepared to take, I would want many LEDs arranged in an aesthetically pleasing design and also a video display that outputs the scaled values as a dial reading with a meter hand, and each chunk out of the total 255 represents a different level of Don Juan-ness with some silly appellation and graphic for it. I wonder if this is what Jitter is for - in that case it would probably not require an extra microcontroller and out connection. But if it's for market sale and needs to be portable, then maybe it would require a second microcontroller, or some other step that would take the place of needing a whole PC to do the work.



I think that of the two, the thermistor makes the most sense for a luv-o-meter. For a test of strength, some giant FSR or stretch sensor may be appropriate.

PhysComp Lab 2 - The Arduino

Today this handheld device, in one fell swoop, was to help me create a blinker, a switchlight and a combo lock-driven light, and it supplied USB power to my breadboard, rendering the DC power unnecessary for this lab.

Still, I kept the LED on the far end of my board to indicate that the board was indeed powered when I plugged in the Arduino. I also put an LED in the digital output and ground of my Arduino to make sure power worked in that as well.



Once power was determined on all fronts, I created an LED circuit much like in the first lab, but also wrote a simple program to create a blinker with .5 sec delay.





void setup() {
pinMode(13, OUTPUT);
}

void loop() {
digitalWrite(13, HIGH);
delay(500);
digitalWrite(13, LOW);
delay(500);
}


I was now ready to try a toggle switch light with the Arduino. This was done in the 1st lab without code, but it's more satisfying to make something work with it.



void setup() {
pinMode(13, OUTPUT);
pinMode(2, INPUT);
}

void loop() {
// check the digial input. If it's high, do something:
if (digitalRead(2) == HIGH) {
// set the digital output HIGH (turn on the LED):
digitalWrite(13, HIGH);
}
else {
// turn off the LED:
digitalWrite(13, LOW);
}
}


I furthered the code in a 3rd application by adding one LED and allowing for the switch between one light being on while the other was off, and vice versa:


void setup() {
pinMode(13, OUTPUT);
pinMode(12, OUTPUT);
pinMode(2, INPUT);
}

void loop() {
// check the digial input. If it's high, do something:
if (digitalRead(2) == HIGH) {
// set the digital output HIGH (turn on the LED):
digitalWrite(13, HIGH);
}
else {
// turn off the LED:
digitalWrite(13, LOW);
}
// check the digial input. If it's high, do something:
if (digitalRead(2) == HIGH) {
// set the digital output HIGH (turn on the LED):
digitalWrite(12, LOW);
}
else {
// turn off the LED:
digitalWrite(12, HIGH);
}
}




Finally, my combo application was a simple recursion in the code - an "if" statement inside an "if" statement - that allowed the 1st switch pressed followed by the 2nd to cause the LED to light up.

void setup() {
pinMode(13, OUTPUT);
pinMode(12, OUTPUT);
pinMode(2, INPUT);
pinMode(3, INPUT);
}

void loop() {
// check the digial input. If it's high, do something:
if (digitalRead(2) == HIGH) {
if (digitalRead(3) == HIGH)
{
// set the digital output HIGH (turn on the LED):
digitalWrite(13, HIGH);}
}
else {
// turn off the LED:
digitalWrite(13, LOW);
}
}




The venture was a success, and I was pleased to realize this could imply much more than powering lights. Motors and sound cues could be the stuff that dreams are made of from here on out.