Social Data Visualization

landscape3
Here’s a visualization of ITP-related twitter data, made with Gephi. It’s an open ord with lots of expansion and a low edge cut, colored according to modularity, with large text for betweenness centrality. I think there is also some Avg Weighted Degree…I’m not sure what all of that means exactly, but I wanted to highlight some unexpected ITP tweeps within sub-groups, and I think that’s sort of what happened. The purple seems to be a cluster of recent grads / current ITP students, green/pink has organizations like ITP and Engadget, green and yellow both highlight people who I don’t know. I haven’t found myself on here yet actually but I will try later!

Here’s a visualization from evolver.fm that caught my eye of how new music introduced on the (now defunct) Turntable.FM would sometimes would go viral:

Stupid Hackathon: Is it ok to dance?

For this weekend’s Stupid Hackathon, I made Ok2Dance.

Ok2Dance is kind of like Shazam except instead of telling you what music is playing, it analyzes the sound and tells you whether it’s ok for dancing.

Ok2Dance accesses your computer’s microphone to record 20 seconds of audio. It saves the recording (temporarily) as a .wav file. Then it sends that file to the Echo Nest for an in-depth musical analysis. From there, Ok2D will ping Echo Nest continuously for the results of the analysis. Once the analysis is ready, Ok2D converts Echo Nest’s floating point number for danceability to a simpler Yes or No answer. Then it loads a corresponding animated GIF. Then it shows the result, accompanied by the animated GIF.

At this point, the song might be over. But at least you’ll know whether it would have been ok to have danced to that.

Why not try Ok2Dance next time you’re at a club or party and wondering if it’s ok to start dancing? Simply log onto your desktop computer, and point its web browser to jasonsigal.cc/ok2dance/. (**NOTE: Mobile devices do not support getUserMedia, so you must bring a desktop computer!!). If you’re looking for test audio, disco usually is danceable, but the sound of staring blankly into a computer screen usually is not.

You can record anything you want, as many times as you want. And don’t worry about copyright or privacy, because the files are not stored on my server. They’re deleted immediately after the danceability analysis is complete.

Source Code on Github

Kandinskify: Kandinsky-inspired generative music visualizations

At the Music Visualization Hackathon, I collaborated with Michelle Chandra, Pam Liou and Ziv Schneider to create Kandinskify. You can check out the work in progress here.

Kandinskify prototype

For inspiration, we looked to the history of music visualization. Ziv brought up Wassily Kandinsky, the Russian painter and art theorist.

kandinsky
Kandinsky was fascinated by the relationship between music and color. He even developed a code for representing music through color: Kandinsky's Music/Color Theory We set about adapting Kandinsky’s code into a generative visualization. Kandinskify analyzes musical attributes of a song as it plays to create a digital painting in the style of Kandinsky.

Kandinsky’s musical attributes, like “Melody seems the dawn of another world,” are not easy to decipher with the human ear, let alone with a computer program. But we used the Echo Nest Track Analysis to figure out some useful attributes for a song, like ‘acousticness’ and ‘energy.’ We can use this information to determine the color palette of a song. The Echo Nest also offers a timestamped array of every beat in a song, which we used to determine when shapes should be drawn.

Kandinsky’s instrumentation like “Middle range church bell” can be hard to find in contemporary music. But with digital audio files, we are able to analyze the amplitude and frequency spectrum of a song as it plays, and it is possible to use this information to isolate certain instruments. We started out working with Zo0o0o0p!!! by Kidkanevil & Oddisee, one of my favorites from the Free Music Archive (though it may not have been Kandinsky’s choice). We were able to isolate the frequency ranges of certain instruments and map them to shapes. For example, the bell makes a spike at 7.5kHz and we mapped that to the oval shape. In the long run, we’d like to let users upload any song.

We used p5.js to create visuals on an HTML5 canvas. I’m developing an audio library for p5 as part of Google Summer of Code, so this was a fun opportunity to map audio to visuals with p5Sound.

The visual composition spirals out from the center in a radial pattern, inspired by this clip of Kandinsky drawing.


We’re planning to allow users to upload any song (or sound). We’re developing different color and shape palettes that will change based on the musical /sonic attributes. Kandinskify will analyze the music to create a unique generative visualization that will be different for every song. We’ll update the source code on github and demo here.

Interactive Klein Bottle

Luke DuBois introduced us to Paul Bourke, a researcher from Australia who shares an amazing online repository of math equations for geometry, textures, shapes and fractals. I tinkered with some of these equations through a JavaScript file in Max/MSP/Jitter.

I created the shapes using little cubes at each point, one point at a time. You can watch the shape take form:

klein1

klein2

I settled on the Klein Bottle shape. Here’s a patch that forms the shape with a fun visualization and sonifies it, allowing you to tinker with parameters like # of points and size of the cube at each point.

github

zip download

 

The Phrases and Pronunciation

My final poetry-generating Python program is based on noun phrases, a markov chain, a recursive function, and a loose poetic structure. The vocabulary was sourced from three public domain source texts:

The Sailor’s Word Book, An Alphabetical Digest Of Nautical Terms, Including Some More Especially Military And Scientific, But Useful To Seamen; As Well As Archaisms Of Early Voyagers, Etc. By The Late Admiral W. H. Smyth (1867)

Emily Post’s Etiquette: In Society, In Business, In Politics And At Home (1922)

Fyodor Dostoevsky’s Notes From Underground (1864)

These texts make for interesting juxtapositions. For one thing, Post describes good manners for society, while Dostoevsky’s narrator is an outcast from society. The Sailor’s Word Book is a dictionary, but its definitions of sailor’s words are the most poetic of my three source texts—especially when taken out of context. The admiral who wrote this book in 1867 refers to ships as female (i.e. “her head”) and officers as male (i.e. “steersman”) which makes everything sound like an analogy. Likewise, Post and Dostoevsky use plenty of gendered pronouns. They also use proper nouns like “Mrs. Worldly” and “Simonov.” These texts refer to people—whether they are objects (such as a ship), individuals or society as a whole. As a result, when I feed these three texts into my program, they often seem to be talking about the same thing.

 

MARKOV CHAIN

My Markov object creates a dictionary of probabilities for transitioning from one word to the next, but I added some special features. Its “generate” function takes a starting word as an argument. If the starting word exists in the source text (Sailor’s Words + Dostoevsky), it will generate text based on its dictionary of probable transitions from that word to any other word.

  # Generate a text from the information in self.ngrams
  def generate(self, start_word):
    from random import choice

    # If the start_word is in the text, generate a markov chain
    if start_word in self.all_words:
      current = choice([item for item in self.ngrams if item[0] == start_word])
      output = list(current)

      for i in range(self.max):
        if current in self.ngrams:
          possible_next = self.ngrams[current]
          next = choice(possible_next)
          output.append(next)
          current = tuple(output[-self.n:])
        else:
          break

      output_str = self.concatenate(output)
      return output_str
    else:
      return ''

This allowed me to create a generate_question() function that starts a Markov chain with What When Where Why or How. The questions are part of my poetic structure.

def generate_question():
    next_line = underground.generate(random.choice(['What', 'How', 'Why', 'When', 'Where']))
    return next_line

I felt it was important to feed sentences into the Markov chain, rather than lines. So I made an array of Strings and used TextBlob to parse the sentences which I fed into my Markov chain like this:

underground = MarkovGenerator(5, 6)
utext = []
for line in open('../texts/notesfromunderground.txt', 'r'):
    utext.append(line + ' ')
utext_sentences = TextBlob(" ".join(utext).decode('ascii', errors='replace')).sentences
for sentence in utext_sentences:
    underground.feed(sentence)

 

NOUN PHRASES

I used TextBlob to generate a list of noun phrases from Etiquette and Sailor’s Words.

Noun phrases are a very interesting way to parse a text. In fact, I could have stopped there and made a poem out of noun phrases like these:

dead men 's dust
vulgar salutation
long lewd lither loiterer
moral elevation ensues
float of a fishing-line
conical tops of mountains

This is a series of noun phrases from Sailor’s Words:

barrel of small arms.
barrel of the wheel.
barrier of ice. ice
olive oil. _yellow basilicon_
battening the hatches. securing
battens of the hatches.
beam of the anchor. synonymous
land 's end bore
before or abaft the beam.
bend on the tack.
berthing of the head.
beset in ice. surrounded

This is a series of noun phrases from Etiquette:

politeness implies
gentlemanly character
social intercourse
whole detail
person shows respect
time implies
reciprocal respect
gentlemanly character
womanhood combined.

Originally, I created a Noun Phrase List object, but this was time consuming to generate every time, so I just saved them as a text file using the NPLC object I created in my nplc module.

post = NPLC('../texts/sailorswordbook.txt', 4)
npz = open("sailor_nouns.txt", "wb")
for i in post.noun_phrases:
    npz.write(i.encode('utf-8', errors='replace').strip() + '\n')
npz.close()

 

I thought that these noun phrases could give my poems some rhythmic structure, especially if repeated. I sorted them by length:

# generate three noun phrases
nOne = random.choice(postnouns).strip()
nTwo = random.choice(postnouns).strip()
nThree = random.choice(sailornouns).strip()

print "nOne: " + nOne + ', nTwo: ' + nTwo + ', and Three: ' + nThree

# sort the noun phrases by length
if len(nOne) > len(nTwo):
    nX = nOne
    nOne = nTwo
    nTwo = nX

if len(nTwo) > len(nThree):
    nX = nTwo
    nTwo = nThree
    nThree = nX

I also created a function to prepend an appropriate preposition to a noun phrase

# This function tries to figures out what preposition to put before
# a Noun Phrase based on whether it is plural and based on its starting letter.
def preposition(line):
    first_letter = line[0]
    for word in line.split():
        tb = TextBlob(word)
        for w, t in tb.tags:
            if t == 'NN':
                b = Word(word)
                if word == str(b.singularize()):
                    # print word + " is probably singular like " + b.singularize()
                    if not_a_vowel(first_letter):
                        return random.choice(['The ', 'A ', '']) + line
                    else:
                        return random.choice(['The ', 'An ', '']) + line
                elif word == str(b.pluralize()):
                    return random.choice(['The ', 'Some ', 'Many ', 'Of ', 'For all of the ', '']) + line
    ## if it gets to this point, we dont know if it is plural, so just figure out if 'a' or 'an'
    if not_a_vowel(first_letter):
        return random.choice(['A ', 'The ', '']) + line
    else:
        return random.choice(['An ', 'The ', '']) + line

That uses my not_a_vowel function:

def not_a_vowel(letter):
    for char_to_check in ["a", "e", "i", "o", "u"]:
        if letter == char_to_check:
            return False
    return True

Which I originally wrote in order to approximate the number of syllables in a gibberish word. I did not deal with the complexities of “Y.”

RECURSION

The noun phrases, questions and markov chains help give my poems rhythm and space. I pick up the pace with a recursive function called redo_line:

## This function takes a word an changes it to another word that rhymes or has the same # of syllables.
## Sometimes, it also appends a random nounphrase
## rhyming_word is in a separate file, the rhymebot module.
def changeWord(word):
    new_word = ''
    #   nounize(word)
    if word.upper() in all_words:
        new_word = rhyming_word(word, 1)
        if new_word:
            return new_word.lower()
        else:
            new_word = str(random.choice(syl_lookup[syl_bible[word.upper()]]))
            # new_line = new_line + " " + new_word.lower()
    else:
        gibSyl = gib_syls(word)
        new_word = str(random.choice(syl_lookup[gibSyl]))
        # new_line = new_line + " " +
    return new_word.lower() + " " + random.choice(postnouns + sailornouns).strip()


## This function iterates recursively through a line, three words at a time.
## It transforms the last word each time by calling the changeWord function.
def redo_line(line):
    words = line.split()
    if len(words) > 0:
        w = words.pop()
        y = changeWord(w)
        if len(words) > 3:  #crucial variable
            new_line = ' '.join(words)
            try:
                print ' '.join(new_line.split()[-3:]) + " " +y.lower()
            except:
                print ' '.join(new_line.split()[-2:])
            redo_line(new_line)
        if len(words) == 3:
            new_line = ' '.join(words)
            try:
                print ' '.join(new_line.split()[-2:]) + " " +y.lower()
            except:
                print ' '.join(new_line.split()[-1:])
            redo_line(new_line)
        if len(words) == 2:
            new_line = ' '.join(words)
            try:
                print ' '.join(new_line.split()[-1:]) + " " +y.lower()
            except:
                print ' '.join(new_line.split()[-1:])
            redo_line(new_line)
        if len(words) == 1:
            print words[0] + ' ' + y
            print underground.generate(words[0])
            print underground.generate(words[0])
            print words[0] + w
            return

I originally developed this function for my midterm project. Here, I use it towards the end of the poem to iterate through the poem’s three noun phrases and the poem’s opening line.

You can view the full code on my github and the two sample poems from my reading are excerpted below:

PHRASES & PRONUNCIATION

  ABBLAST.
young people
How is enjoyment in this to be
  explained?
 
An elderly guest
phrases and pronunciation
phrases and pronunciation
pronunciation phrases and gillon
and pronunciation phrases remodeled
phrases and pronunciation Townsend's
guest phrases and catamaran
guest phrases liquefied
guest amdahl's
elderly simonette
elderly
 
As far as
  my personal opinion is concerned, to care only for well-being seems to
  me positively ill-bred.
time by his sighed perfect little house
that time by kerins
at that time bly
attention at that duhaime
my attention at cooperate
distracted my attention introduced
Apollon distracted my ritalin
that Apollon distractedly
fact, that Apollon hausfeld
in fact, that 'm chapter xxv
thing, in fact, inhibit
good thing, in aux telegraph pole
a good thing, doron
was a good er luncheon menu=
was a wood
was japonica
It chides
It would be to the interests of humanity and courtesy were it made
  indispensable.
It was first devised for
  the service of mortars, and named after the inventor, Gomer, in the late
  wars.
It
The elderly guest
The phrases and pronunciation
The phrases and pronunciation

 

TRUE PURPOSE, SOCIAL WORLD

A true purpose
A social world
A senior officer 's signature


A fore-and-aft sail, setting with a boom and gaff, in
  ships, synonymous with the spencers of brigs and schooners, and the
  spanker or driver of ships.

The true purpose
The social world
The senior officer 's signature

In size, as a ship's boat, smaller than
  the barge, and, like it, carvel-built.
true purpose
d��butante wears
The senior officer 's signature

true one--arising from lateral pressure and the effect of sea when
  close-hauled.
purpose of firing signals,
  as the officer who commands her is particularly ordered to carry no
  cargo, ammunition, or implements of war.

Why are we perverse and ask for something
  else?
ask for something fu dining-room service.
and ask for fudging
perverse and ask reyer
we perverse and hunk
are we perverse duguid?
are we britz
are indri
Why moondreamer?
Why did I invite myself to this dinner
  yesterday?
Why are you so
  cross?
Why?
 
A true purpose
social world
A senior officer 's signature

Phrases and Pronunciation

ABBLAST.
young people
How is enjoyment in this to be
explained?

An elderly guest
phrases and pronunciation
phrases and pronunciation
pronunciation phrases and gillon
and pronunciation phrases remodeled
phrases and pronunciation Townsend’s
guest phrases and catamaran
guest phrases liquefied
guest amdahl’s
elderly simonette
elderly

As far as
my personal opinion is concerned, to care only for well-being seems to
me positively ill-bred.
time by his sighed perfect little house
that time by kerins
at that time bly
attention at that duhaime
my attention at cooperate
distracted my attention introduced
Apollon distracted my ritalin
that Apollon distractedly
fact, that Apollon hausfeld
in fact, that ‘m chapter xxv
thing, in fact, inhibit
good thing, in aux telegraph pole
a good thing, doron
was a good er luncheon menu=
was a wood
was japonica

It chides
It would be to the interests of humanity and courtesy were it made
indispensable.
It was first devised for
the service of mortars, and named after the inventor, Gomer, in the late
wars.
It

The elderly guest
The phrases and pronunciation
The phrases and pronunciation

True Purpose, Social World, Senior Officer’s Signature

A true purpose
A social world
A senior officer ‘s signature

A fore-and-aft sail, setting with a boom and gaff, in
ships, synonymous with the spencers of brigs and schooners, and the
spanker or driver of ships.

The true purpose
The social world
The senior officer ‘s signature

In size, as a ship’s boat, smaller than
the barge, and, like it, carvel-built.
true purpose
debutante wears
The senior officer ‘s signature

true one–arising from lateral pressure and the effect of sea when
close-hauled.
purpose of firing signals,
as the officer who commands her is particularly ordered to carry no
cargo, ammunition, or implements of war.

Why are we perverse and ask for something
else?
ask for something fu dining-room service.
and ask for fudging
perverse and ask reyer
we perverse and hunk
are we perverse duguid?
are we britz
are indri
Why moondreamer?
Why did I invite myself to this dinner
yesterday?
Why are you so
cross?
Why?

A true purpose
social world
A senior officer ‘s signature

DWD Project Description

I’m building a collaborative discussion forum for Dynamic Web Development. This is also a project for Hacking Higher Ed where I’ve been researching the social conditions that facilitate learning, and what types of discussion forums might lead a community to adopt these social norms.

These are the features I’m considering (though I will probably need to trim ’em down!)

  • all visitors can view and search Q&A’s
  • visitor becomes a ‘user’ by signing in and creating account –> profile w/ image, bio, url’s
  • users can create questions, but first must search to see if there are already similar results
  • users can answer questions
  • users can edit questions to better frame the concept
  • version control: track history of edits to questions & answers (Wiki-style)
  • users can award each other points
  • users can award each other badges for Good Question and Good Answer
  • users can mark questions ‘resolved’ or add follow-up questions

Recursive One Word Generative Poetry

Input any word. My program will turn it into another word that has the same number of phonemes. Then it will find an example of that word in use via the Wordnik API. Then it will recursively run through four bit chunks, starting from the end, modifying the last word in each line using the CMU phoneme dictionary. I got a bit carried away parsing the CMU pronouncing dictionary and this includes some functions that I didn’t wind up using, but would like to use in the future.

Children

So, yeah, a freakin ‘HUGO nominee recommeded ME for ArmadilloCon – yeah, good thing I accepted that invitation.
accepted that
thing I accepted dees
good thing I rodeheaver
yeah, good thing err(1)
– yeah, good wang
ArmadilloCon – yeah, king
ArmadilloCon –
for ArmadilloCon
ME for
nominee recommeded ME ville
‘HUGO nominee recommeded leh
‘HUGO nominee
a freakin ‘HUGO emmerson
a freakin
yeah so

hello

JOSEPH BAST, president of the Heartland Institute, whose unshaven face beards a very similar resemblance to the president of another country on this Earth — no, not the Czech Republic, but go head and guess!
head and
but go head zoch
Republic, but go peart
Czech Republic, but uwe
the Czech Republic, bohr
the Czech
no, not the yeung
— no, not so
Earth — no, pitt
Earth —
this Earth
country on this pay
another country on fors
of another country o.s
president of another smolen
the president of clemo
to the president of(1)
resemblance to the tumbleson
similar resemblance to h
very similar resemblance auth
a very similar creditworthy
beards a very zimpfer
face beards a knapke
unshaven face beards owe
whose unshaven face wesat
Institute, whose unshaven prow
Heartland Institute, whose sicilia
the Heartland Institute, luau
the Heartland
president of the mastodon
BAST, president of thao

At one point, I was including the fill line rather than the last four words:

midterm2.py
172-29-18-92:midterm jasonsigal$ python midterm2.py c458d5c93b0a018f000070bc23407600a2332144b9ac156e1
hello
PEIFER: My wife is the one who always wanted to go to Africa.
PEIFER: My wife is the one who always wanted to go to
PEIFER: My wife is the one who always wanted to go jha
PEIFER: My wife is the one who always wanted to jha
PEIFER: My wife is the one who always wanted our
PEIFER: My wife is the one who always yearlings
PEIFER: My wife is the one who schiltz
PEIFER: My wife is the one o.s’
PEIFER: My wife is the noon
PEIFER: My wife is hour(1)
PEIFER: My wife u.
PEIFER: My fifth(1)
PEIFER: herb

Continue reading