Where I’m At – App Idea

For Always On, Always Connected, I want to explore location-based audio and develop a mobile app around that idea.

First, I need to learn more about how people approach location + audio. What could make location-based audio interesting enough to create? To share privately, or publicly? Is there data about audio that might be interesting, or should I focus just on the audio itself?

I have a few ideas already. The first one is called Where I’m At. It’s basically a mashup of Shazam (or, more likely, the Echo Nest’s musical fingerprinting service), and FourSquare.

Shazam has a social component. But most people don’t use it. That’s because if you are shazamming something, the assumption is that you didn’t know what song was playing. You had to use Shazam to find out. That’s not a cool thing to share.

But the idea of listening to a specific song in a specific place at a specific time changes what it means to tag a song. For example, I could tag that I’m at Brad’s, and they’re playing “Freaks Come Out At Night” by Whodini, and then share that with friends. The geolocation isn’t interesting, the song isn’t that interesting, but maybe the combination is.

Tagging the song becomes an easy way to share the vibe of where I’m at. Where I’m At could be the name of the app.

Continue reading →

Fable of the User-Centered Designer

The lessons of David Travis’ Fable of the User-Centered Designer seem obvious: Start by getting to know who you are designing for, test your assumptions, and iterate.

But the approach is not quite so obvious because, as the fable’s protagonist discovers, there is no manual for user-centered design. You can’t just go by the book. You have to engage with your users at every step of the process.

I wish I knew this back in 2008. I was leading beta tests for a social music website. When I came to the project in February of that year, there was already a spec outlining all of the features the site was to offer. All that was left was to build the site (we hired a boutique company who gave us a non-profit rate), and fill it with great Creative Commons music (that was my job).

We had imagined that the site would appeal to many different types of users. But we had not done any user testing outside the world of our small nonprofit.

As the beta testing began, I discovered that the red routes were not as clear as we had imagined. I had to outline step-by-step instructions for how to do key things like upload a song, or create a playlist. My first round of beta tests were conducted by remote, without the ability to observe. I learned much more by actually watching people use the site, and talking through the process.

Some of these lessons wouldn’t really hit home until the site was opened to the public. At that point, I started to identify who our users really were. The categories we had initially envisioned didn’t fit the reality of our userbase. I fielded help emails describing issues I never imagined anyone might ask, mostly because I’d been living with the site for so long I had never thought to question. Additionally, many of the features from the original spec went unused. The design looked nice, but in reality it was too cluttered. A user-centered design process could have helped prioritize the information.

At the time, I was more concerned with testing functionality than with testing the user experience. The site was developed with a grant, and we had a limited timeframe to do what we said we would do, and we said we would do a lot of things. In hindsight, we should have allocated resources to learn more about our potential users as part of a more iterative design process.

Instead, I learned more about our users after the site launched. Many of the users who we’d imagined showed no interest in the site. For example, we thought remix artists would use the site to find legal samples. The original grant proposal even described an area of the site that would be dedicated to remix artists. But I vividly remember my first on-the-job conversation with an actual remix artist, because I found that she didn’t care about “legal music” because—the idea was actually quite insulting to her because she believed that, as an artist, her sampling is protected under fair use. Instead, it turned out that online video producers would become the primary audience. With more user research, we might have been able to predict this and either focus the site’s features, or reprioritize the goals of the site to cater to the other users we had hoped to attract. As we began to focus more on content for video producers, this audience continued to grow, but it also raised new issues that would have been better served by an iterative, user-centered design process.

And that’s why I came to ITP!

Updated Thesis Summary

Title: OLoS: Online Open Source Music

Synopsis: A modular web audio environment where everything—both the music and the underlying code—can be remixed. I will design interfaces that show the underlying processes and invite play. My goal is to foster an open source community for shared learning.

Description: OLoS is an environment to play with code and music. The web makes both more accessible and shareable. I want to provide a forum for the community that is forming around the Web Audio API to share resources through modules that can be combined and remixed. Through user research into the community’s best practices, I’ll develop an API spec for modules that will enable integration into outside projects as well as the OLoS node-based visual interface. My research might lead me to extend a pre-existing project. Nodes will include instruments, effects and algorithms, and UI/control nodes. They can be chained together to design custom environments for music production and interaction. Users can also share/remix environments & compositions.

Continue reading →

Always On, Always Connected

When I leave the house in the morning, if I’m planning to take the subway, I often bring a magazine in case I get bored. But I rarely wind up reading that magazine, because I rarely get bored, because I always have my phone.

When I get home, I’m excited to open the mailbox to see if there are any messages. Usually the messages are bills or spam. But, at least once a week, there is a new magazine. This is exciting, but also troubling, because I rarely feel finished with last week’s New Yorker, and now a week has gone by, and I put the old news aside in favor of the new. I’ll to keep the old issues around for at least a few weeks, but I can’t archive them in my apartment like I archive old emails.

One of my favorite apps is Pocket, formerly Read It Later. Whenever I find an interesting article or PDF that I don’t have time to read right now, I save it to my pocket. It literally shows up in my pocket, so I can read it when I’m on the subway. But instead, I usually check my email or update the news on my nytimes app.

When I take the subway, I have a choice between the G train and the J/M. Whenever possible, even if it takes a bit longer, I take the J/M, because it’s an above ground train, so I can check the internet on my phone.

I like listening to music on my phone. But I don’t like to listen to the same thing multiple times. Every once in a while, I’ll load up a big playlist of music that is completely new to me so that I can listen to it later on my phone. Often, I’ll listen to the radio instead so that I don’t have to choose what to play. Or I’ll listen to podcasts, because there are always new podcasts to check out each with new episodes to follow.

Sometimes, I use my phone to make music. There are so many cool apps.

Often, I’ll download an app because it sounds cool, play it a few times, and never come back to it.

I use my phone to take notes. I used to send myself emails, then I set up an IFTTT hashtag to send those emails to the appropriate place in Google Drive, and now I mostly use Evernote. I use it for text, as well as for audio notes. I like to hum musical ideas into my phone, and the best ideas tend to arrive when I’m outside walking around, doing something. The best ideas rarely come to me when I’m sitting down, on a computer, or using my phone. But I’m always glad that I have my phone on me when the ideas show up!

My phone buzzes when it receives incoming messages. I have a Pavlovian response. I get the phantom buzz sometimes, too. My phone gives me the sense that wherever I am, whatever I’m doing, whatever I’m reading or listening to, there is always something better, and it’s the portal to that better world of infinite potential. Same thing with people, since our phones can connect us to anybody in the world instantly. So when hanging out with other people, whether it’s dinner or just a casual conversation, I try not to use my phone because this really signals that they are less interesting than the people who exist in the phone world.

I just got a new phone. My old one was 4.5 years old, and it was very slow. It took over a minute to connect to facebook, which is almost forever in phone time. The new one is fascinating. It has a better camera, and the camera loads really fast. It’s been one month and I’ve already taken over 500 photos. I can talk to it. It does this predictive text thing where it guesses three words that you might want to say after every word you type. I wrote about that when I first got the phone. I use it all the time.

Fun with wget

For Storage Wars and Data Dumps, I spent some time playing with

wget

to download websites.

I found some good instructions programminghistorian.org.

The command

wget -r --no-parent -w 2 --limit-rate=20k <website>

set my terminal plowing through every link it could find.

-r

stands for recursive retrieval, meaning it’ll follow all links on the page, but

--no-parent

prevents wget from downloading stuff from other websites. It waits 2 seconds before downloading more stuff, and limits bandwidth so as not to cause problems for the server as it makes these recursive requests.

First, I tried wget without any arguments. This downloaded individual html files. Loading the files in a browser, none of the links work, including any referenced CSS or JS. The menus or elements that didn’t seem obvious when styled properly are now as much at the forefront as the main content.

Then, I went after as much of the site as I could nab.

I’m interested in creating an archive / repository of local events. So I looked at a few sites that list upcoming events. I tried timeoutny, which was a bit of a mess. I tried the New Yorker, which is great because it just picks a handful of events and the format would be easy to parse. But the best was nononsensenyc, which hosts archives of its plain text emails. Finally, I also tried wget’ing wfmu.org, which has so many recursive links, it’s still going…so far it’s downloaded 1842+ items that make up 100+ megabytes consisting of JS libraries, images, php files, favicons, audio files, and more. One interesting components is the playlist pages, which document music played in almost every program dating back to 1999.

Next I downloaded the Terminal Boredom forum, for which I had to ignore the robots using the -erobots=off command.

Computer Graphics

The first assignment for Ken Perlin’s Computer Graphics course is to explain why I’m interested in computer graphics and what I want to get out of the course. My assignments will be posted to my github.

I’m interested in Computer Graphics as a creative window into more advanced topics in computer science and math. I took computer science as an undergrad, but, without any graphics, the topic seemed very dry. It took weeks before we could see any results from our code. Even though I had always been interested in using computers to create things, starting with HyperCard when i was very young, I wound up gravitating towards Computer Music, where I could hear the results of my work instantly. Similarly, I became less interested in math until I had concrete applications, such as Digital Signal Theory, where I could see or hear the results.

JavaScript has rekindled my love of programming as a means of creative expression. In particular, libraries like Processing’s p5.js make it easy to get started sketching out your creative ideas. Libraries like three.js make it relatively easy to get up and running with 3D using WebGL, and I’ve enjoyed playing with this in some recent projects. Now that I have some basic understanding and use cases, I’m eager to learn more about what is happening under the hood.

I’m especially interested in learning about computer graphics from Ken Perlin, who is a very inspiring person. I started contributing to his Chalktalk project last semester, and already I’ve learned a lot from him. I want to learn more so that I can make more meaningful contributions to Chalktalk. A better understanding of computer graphics will serve me well in life, even if my focus leans toward the audio side of things. In the long run, I’m interested in sound design for video games and virtual/augmented reality. In the short run, I want to keep contributing to Chalktalk, and also somebody needs to bring WebGL rendering to p5.js for a “3D Mode”—maybe I could help make that happen this summer! Now that I’ve scratched the surface of computer graphics, I’m excited to learn more.

Research for Thesis (excerpt from my private thesis blog)

TITLE: OnLine Open Source DAW.

 

SYNOPSIS: OLOS-DAW is a browser-based music production environment where everything—both the music and the underlying code—can be remixed. It is an open source, educational resource for communities to form around the intersection of music, code and the web.

DESCRIPTION: OLOS is a digital audio workstation, similar to Garage Band, but with a more modular architecture inspired by Max/MSP.

Web browsers now ship with many of the building blocks and optimizations of these advanced music production environments.

A community is forming around web audio and could benefit from a centralized place to share code. OLOS would provide that resource, and a spec for Web Audio modules. Modules could include synthesizers, effects, samplers, and algorithms, each with one or many UI modules.

Anybody who wants to make music can play with OLOS’ many different instruments and interfaces, or chain them together in a DAW. Creations can be saved, shared, and remixed.

Modules and songs be public/private, rated, curated.

 

RESEARCH: I think that a lot of the components have been done before, but I don’t think it’s been put together in quite this way.

Examples of online DAW’s:

 

Examples of online music/coding environments:

Attempts to unify Web Audio community with a spec:

  • Web Audio Components (why did this fizzle out in 2013? Will the project be revived? Follow up with everyone involved and figure out how I could help revive)
  • Open Music

Research into Web Audio community:

  • What would be most useful for Web Audio Community?
  • Who do I want to contribute the first modules?

Research into potential music creators:

  • How could this be used in the classroom?

Examples of other online code/music communities – what makes them work (or not)?

  • Csound, Faust, Max/MSP, Pure Data
  • Scratch

Open Source UI Components

Researching Potential Tools and Technical Challenges

  • Learn Angular.js because that’s what all the other DAW’s seem to be using
  • Google Blockly (similar to Scratch, a way to code using blocks)
  • What is the best approach for editable code in the browser?
  • How to handle the database?
  • Many more questions…

 

PERSONAL STATEMENT: Before coming to ITP, I directed an online music library inspired by the open software movement. It was inspiring to see communities form around shared music.

Now the web can be a platform for music creation, as well as distribution. I think this will excite a lot of people for a lot of different reasons, but most don’t realize it at this moment. I want to bring music creators and coders together to see what happens when these lines are blurred in a community forum where everything is open source / remixable.

I’m inspired by teaching after school workshops to young people with Scratch. Scratch uses code blocks and sprites to teach computational thinking, but for the millions of kids who use it, it’s just about fun making stories and games. Scratch was once just a desktop application, but now it’s online, and every project can be shared. There are millions of them. Every project has a “see inside” button and clicking that is similar to a github fork. I believe that remixing is a key part of the educational experience, and I want to bring the Scratch approach to music.

i started working with web audio in my first semester at ITP. I wanted to make a Web Audio Editor, and realized it couldn’t be done in Processing because any audio stuff I did would not be translatable by ‘javascript mode.’ This was a bummer because the whole point is that it would live in the browser, where people spend a lot of their time these days anyway, and where it could easily be shared. I studied Computer Music as an undergrad, and at that time it was hard to share interactive experiences—nobody outside of the department cared to check out my Max patches. But the Web Audio API is accessible to all—just open up your Javascript console and create a new AudioContext. This past summer I created p5.sound, an addon for p5.js that brings the Processing approach to Web Audio. In the process, I’ve developed a new appreciation for API specs.

I’ve been thinking about this idea for a few months, I’ve done some research and I’m excited to start prototyping so that I can trim down my goals to something that is actually feasible in 3 months.

Themes from the 1st WAC (outline)

This is just an outline off the top of my head of all the cool stuff that happened at the 1st WAC at IRCAM and Mozilla Paris.
  • Distributed performance
    • talk / demo
    • Lissijous –> distributed
    • Technical questions
    • Compositional questsion—maybe latency is something to be embraced by the composer.
      • how much control to give the audience, when they are performers, how much room to allow for improvisation?
    • Design questions:
      • phone as an interface – do you want people to interact with each other?
  • Port DSP to Web
    • 3D audio custom HRTF (IRCAM)
    • Meyda – audio feature extraction library – inspired by the Yaffa library (Python / C++)
      • using jsfft library to do fft themselves
    • various environments for running / optimizing C++ code
      • asm.js (Jari Kleimola DAW plugins for web browsers)
    • AudioWorker will make this all much more accessible and standardized.
  • Port frameworks to the web
    • Csound and Faust with emscripten and potable data client
    • Tone.JS inspired by Ableton, Max/MSP etc but built with the web in mind
    • WebPD – two separate projects
    • Recently converted to Web Audio:
      • Earsketch – more powerful than Reaper, transitioned 1 year ago
        • reference DAW paradigm while learning Python and JS
        • MOOC
        • Why is web better? No installation hurdles. Essential for education. Plus, accessible from anywhere!
      • Sonoport
    • Cool Hacks to get things to work:
      • Sonoport
      • Tone.js
    • Chris Lowis’ keynote on the original of computer music at Bell Labs half a century ago, really hasn’t evolved that much. Simple principles:
      • flexible building blocks
      • allow extension
      • separate orchestra from score
    • Q: What about the web makes it different?
      • challenges described by Paul Adenot
  • Modular resources
    • IRCAM’s WAVES – common.js
    • BRAID – Web Audio UI builder, uses canvas.
      • idea: share instruments via central database
      • modular with npm
    • Open Music – not talked about