Rhapsody in CS Major (11/22/2023)

Signal

Motivation

HackUMass XI, the annual hackathon hosted by students at UMass Amherst, happened this year on the weekend of November 10th. One of the UMass CyberSecurity Club members, Larry, told me about a project he wanted to work on: a synthesizer. I decided to help out since I am a really huge fan of music, and I thought it would be a fun challenge for all of us. Eventually, we ended up building a working synthesizer from a Raspberry Pi Pico, which should be replicable without spending much money. I'll throw some photos up when I find them.

Getting Started

As soon as the hackathon started, we immediately got to work on trying to generate sound that we could hear. My suggestion was to simulate an analog signal using pulse-width modulation (PWM), and use that to generate a basic analog waveform which we could pipe into a speaker.

The reason this works is rather simple. In a digital signal, there are a finite number of values the voltage can represent (for example, many signals use a voltage of 0V as a logical LOW, and a voltage of VCC as a logical HIGH). However, an analog signal is a continuous function, driven by the voltage at a given time t.

By utilizing the concept of pulse-width modulation, we can rapidly toggle a digital signal on and off at a rate representing a given analog value at some time t, and emulate an analog waveform. I'm kinda handwaving the explanation, but here is a better one.

Anyways, within an hour of the hackathon starting, we were able to generate an audible square wave using this technique. But, it was very quiet. Of course, if we wanted this to sound good, we needed an amplifier. And, thus began my research into amplification circuits, while the other members of my team (Minh Do, Larry Liu, and Theo Krueger) worked on generating different waveforms.

Signal Processing Hell

Now, as a computer science major, we are already at a disadvantage compared to electrical engineers; the only hardware-ish courses we have at UMass Amherst in the Computer Science department are Physical Computing (CICS 256), and Inside the Box: How Computers Work (COMPSCI 335). We don't learn that much about signal processing in these courses, let alone audio processing. As such, I spent a bit of time reading about how amplification circuits worked.

Essentially, our problem is pretty simple: the analog waveform we are generating does not take full advantage of our speaker's capabilities. So, naturally, we tried boosting the logic level of the circuit; this should increase the average value of our pulse-width modulation, leading to an amplified signal. But, no matter how much we tried, this did not work out well for us.

I started by trying the following single-stage amplification circuit. I didn't have any BC547 transistors at the moment, so I subbed them out for some other transistors of different specs.

What would happen is our peak-to-peak voltage would differ in some variations -- being pulled up to our target logic level of 9 volts -- but the signal we output still sounded exactly the same.

This problem plagued me for the next 24 hours. Permutation after permutation, the circuits I tried simply did not work. But, it didn't really matter at this point: we ended up being able to generate various waveforms: sine waves, triangle waves, square waves, and sawtooth waves. They all sounded great, albeit at a lower volume.

Theo and I had to go give a talk about Xbox 360 security, so I let this stay in the back of my mind during it.

I decided to take a break from the amplification circuit, and build the body of the synthesizer. It'll eventually hit me.

Constructing the Keytar

Larry and I decided to pay a visit to my workplace: the CICS Makerspace.

With Larry's help, I built the body of the keytar out of some spare MDF panel. All I did was cut some of it in half, then construct a guitar-like frame with it, some screws, and a bit of patience. That was when we decided to make it look like a keytar.

Then, we used some spare acrylic, some screws, and a drill to create a pickup guard which we mounted our twelve buttons on. Larry marked them out in a piano form, and I drilled away.

From there, we mounted it onto the frame of the keytar, and we were ready to roll.

I then used one of the soldering irons to solder 24 wires, two for each button, leaving a lot of room for slack (in case we needed to debug/wire some stuff). One wire was to go to the input voltage, 3V3, whereas the other one was to go to our pulldown resistor circuits, which were connected to our Pi Pico.

After finishing the job, we had a pretty decent looking body. I was also able to cop some speakers we could directly wire into. We brought this all back to the ILC, where the Pi Pico was, and decided to mount our circuit onto the board.

I took two breadboards and mounted them onto the neck of the keytar. One breadboard contained the pulldown resistor circuits (and the wires to go to our buttons), and the other the Pi Pico. After wiring everything up, we were ready to go.

Programming the Keytar

We needed to map each button to a frequency, like an actual keytar. Additionally, we also needed to implement wave-type switching, and we needed an algorithm that doubled or halved our base frequency based on the joystick position. This is where Minh shined.

I wrote the initial button mapping code (which just mapped each button to a position in some field), and Minh took over. He proceeded to pull an all nighter writing all of the code needed to fully glue this together, while I tidied up the hardware with Larry.

Basically, all we did here was have a main loop which updated the internal state of all of the buttons and joysticks. If a given button was pressed, then the action corresponding to it happened. In the case of the twelve main buttons, we calculated the frequency dynamically based a few factors:

The joystick X position.
The button pressed.
The waveform type.

Then, we had to represent the PWM signal as a function of time. So we calculated the PWM signal at a given interval differently based on the waveform we wanted to emulate. The exact function (in the C++ rewrite) is as follows:

double Wave::CalculateLevel(double new_frequency, double modulation) {
    double c_EffectiveTime = (new_frequency * this->m_Multiplier * modulation * (this->m_WaveTime - this->m_WaveStart) / 1000000);
    c_EffectiveTime -= (int) c_EffectiveTime;
    switch (this->m_WaveType) {
        case SineWave: {
            double c = cos(M_PI * c_EffectiveTime);
            return c * c;
        }
        case SquareWave:
            return c_EffectiveTime < 0.5f;
        case SawWave:
            return c_EffectiveTime;
        case TriangleWave:
            return 2 * fabs(0.5f - c_EffectiveTime);
        default:
            break;
    };
    return 0;
}

The full code can be found here.

Slightly off topic, but a funny thing that happened: at around 3 AM the day the hackathon was supposed to end, we were trying to map notes. Whenever Minh would play a note, there would be a disagreement about what the note actually was. We started wondering if Larry or I had defective ears, and I started wondering if I was being gas lit at 4 AM.

Eventually we got the keytar in tune, and it was just a matter of fixing any weird oddities and trying to build the amplifier. For some reason, the LidaCane group had everything an electrical engineering student would ever need to build literally anything (including a bunch of transistors), so we asked their group for advice on amplification. One of their team members actually gave us a decent idea: try to pull up our signal with a high voltage on the drain, while driving the gate with our original signal.

Unfortunately, this didn't work either, even though we saw a bunch of stronger waveforms on the oscilloscope. But, they were always digital waveforms.

And then, it hit me: we were trying to amplify a DC signal with an AC amplifier! So this would clearly never work. I decided to try and sleep on the idea, and see if there was anything we could do. But our audio output was relatively fine, regardless.

After this, we made our Devpost submission (with the bumass name "Rhapsody in CS Major"), Larry and I went and got some sleep (for about 3 hours), and Minh put the finishing touches on the software.

By the time we woke up, we had chorded note support, and the keytar was ready to go.

Results

We demoed to 3 judges, and I played some notes (especially chords) to demonstrate that it was actually a working keytar and not some nonsense. I probably annoyed the hell of out anyone who was in that room, though. Not as much as Larry trying to play that damn lick...

Minh implemented some features while the judges came and went, so it got more impressive every time a new judge walked by. Good man.

After a while, we cleaned up, and brought the keytar to the Cybersecurity Club room (since we wanted to keep working on it at some point). I took a nap in the room while cleanup happened, and went to the awards ceremony when it started with the rest of our team.

Our room, ILC S110, had a few groups working in there. There was LidaCane, Bubble Up, RISC-V-JS, and No Strings Attached, to name a few. Turns out every single one of those teams had won a prize; RISC-V-JS won Best Security Hack and Best Software Hack, No Strings Attached won third place overall and Best Game Hack, Bubble Up won Best Idea, and LidaCane won Best Healthcare Hack. What was absolutely wild was when we saw our team's name on the screen.

Turns out we won Best Hardware Hack, and that came with a Raspberry Pi 4 for each of us, and a HackUMass XI Teddy Bear.

You can see us (sans Theo) running down the stairs with smiles on our face.

GVNG SHIT

We took a photo with some of the organizers; from left to right: Jasmine (Co-Director), Samridh (Hardware Director), Minh, Larry, Ibrahima, Anushree (Hardware Co-Director)

GVNG SHIT 2

GVNG SHIT!