Works in Progress

I teach music workshops at an afterschool program where kids build their own sample libraries from scratch. They find the default audio editor (Audacity) really frustrating, and I’m with them! So I decided to build an audio editor that does all the stuff that kids like to do (like reverse sounds, change the speed, and add reverb). I wanted an easy way to make good sounds, and learn about sound in the process.Even though it was designed for an audience who has never worked with digital sound before, the Web Audio Editor can be of use to anyone who wants effects or just a quick edit. I wanted to do this in the browser, rather than as a standalone application, after researching music education software’s transition to the web through my Technology for Music Education course (at Steinhardt). I also drew inspiration from Pixlr, the online photo editor that I choose over Photoshop for quick little edits. Web audio is still in its early stages, so the pieces necessary to build this weren’t available until very recently. But HTML5, the WebAudio API, and Canvas open up a lot of potential for audio playback and visualizations.

Much of the Web Audio Editor’s functionality comes from SoX, a command line utility that has been around for over 2 decades. Shawn Van Every introduced me to SoX in CommLab Web, and helped me set it up on my server so that I can call it with PHP. The frontend is built off of Wavesurfer, a JavaScript library that visualizes the waveform in a JavaScript Canvas. I also added a Canvas to visualize the frequency spectrum, inspired by ICM and Marius Watz’ Sound as Data workshop. I am continuing to develop this project and would be thrilled to present it at the ITP show.

My primary inspiration was seeing kids struggle to use Audacity. I wanted to see if I could build something simpler that functions in the browser. I developed the idea in a course at Steinhardt called ‘Technological Trends in Music Education: Designing Technologies and Experiences for Making, Learning and Engagement.’ I made it a project in CommLab Web (just to get the PHP / SoX running) and ICM (to visualize the sound data and build a user interface).

My course at Steinhardt included readings from MIT’s Lifelong Kindergarten group, and I became very interested in Scratch, a visual programming environment for kids. Scratch very effectively teaches programming concepts by giving a space to create and readymade pieces, like lego blocks. Scratch has an audio editor that has been inspirational for this project.On a more technical level, I’ve found several attempts to build a Web Audio Editor, including plucked.js, a thesis project called Audiee (http://www.stud.fit.vutbr.cz/~xmyler00/audiee/#), and a thesis project by Ryan Viglizzo here at ITP. I’ve been in touch with Ryan. I have dug deep into PHP, SoX, CSS, HTML5 / JavaScript / Web Audio / Canvas to figure out how to do a lot of things I had never done before.

The Web Audio Editor was designed for kids in my after-school workshop, so 2nd-8th graders who have short attention spans and haven’t necessarily worked with digital audio before. But I think it’s of use to anybody who needs to make a quick audio edit, or add some fun effects.

I’ve learned a ton! I had never heard of SoX and was new to PHP. Initially I thought I’d be able to use Processing.js and Minim to visualize and scrub through the audio. But I quickly learned that Minim uses JavaSound, and Java is going out of fashion for web browsers. I also learned more about how Processing.JS works and why it won’t really be able to help me in this case. So I dug into HTML5/JavaScript, where I discovered that much of what works in Processing can be recreated in a Canvas, and the WebAudio API works a bit like Minim. I’ve also learned a lot about user testing with different audiences, including the kids in my after-school program as well as my classmates at ITP.

Built with SoX, and Waveform.JS a library by Katspaugh. Other libraries include Bootstrap and jquery. The frequency spectrum visualization draws from Boris Smus’ Web Audio API ebook (O’Reilly 2013). Many thanks to Shawn Van Every, Daniel Shiffman, Alex Ruthmann, Sam Brenner, Sam Lavigne, Luisa Pereira, Ryan Viglizzo, Danne Woo, Beam Center and everyone else who has helped me figure out how to do this!