Works in Progress

I thought it might be interesting to plot two very different trending topics over time using Twitter Trending Topic data. I imagined the twitterverse would be caught up in a pop culture vortex anticipating Taylor Swift’s new album, but in fact, all it took was one Ebola case in the US to turn it into a massively trending topic. Ebola has trended 78709 times this month, while Taylor Swift has trended only 3253 times. Obviously this is not a fair or appropriate juxtaposition, but I went ahead with it anyway.

Area. Red = Ebola, Green = Taylor Swift

Stacked Bar

For the Taylor Swift plot, a key date is Oct 21st when Taylor Swift accidentally released 8 seconds of white noise on iTunes in Canada, topping the Canadian charts. Oct 12th she made TV appearances and announced that she would release a new single on Oct 14th.

For the Ebola plot, a key date is the first death from the virus in the US on Oct 8th—leading to a huge spike in twitterverse. On Oct 13th, a nurse who became infected was identified and revealed to be on the road to recovery, and there is another spike.

I used the top locations where Taylor Swift was trending for both plots. Below is the full list of top 10 for each topic. Interestingly, Ottawa is the most on top of both of these trends.

# of times “Ebola” was in the Trending Topics this month:
Ottawa 2520
Toronto 1745
Portsmouth 1666
Bristol 1603
Austin 1542
Denver 1482
Birmingham 1471
New Orleans 1464
Cincinnati 1461
Winnipeg 1429
Norfolk 1409
Indianapolis 1308
San Antonio 1274
Atlanta 1269
Detroit 1268
Orlando 1248
Dublin 1245
Dallas-Ft. Worth 1236
Providence 1231
Phoenix 1222

# of times “Taylor Swift” was in the Trending Topics this month:
Ottawa 184
Brisbane 146
Birmingham 141
Kuala Lumpur 140
Dublin 140
Manchester 132
Brighton 106
Calgary 96
Leicester 88
Sheffield 87
Glasgow 82
Salt Lake City 75
Klang 73
San Diego 71
Bekasi 69
Nottingham 69
Cardiff 68
Charlotte 67
Belfast 66
Perth 63

It might be more interesting to try to plot the tone of the phrases people use in regards to these two topics. I’ll save that for another day.

The first thing I think of when I hear recommendation system, is a music recommendation system. So I tried to make one using Instagram data, which is probably the wrong form of social media / tag, but I was curious to see what might happen.

I set up a remote server and ran this script as a cronjob every 15 minutes for a few days, gathering a dataset of 61974 Instagram media items that contained the hashtag “music.”

Then I loaded all of the 300+ pickle files into an ipython notebook. I made a hashtable of users to tags, where each tag had a number that represents how often that user used that tag in connection with music.

I found that only 165 users produced all 61974 instagram posts. Obviously I screwed up the way I calculated the max_tag_id. As it turns out, I only got 198 unique media items. I’ll look into this later. In the meantime, I’ll use the Pearson Correlation Coefficient to smooth out the weight of each user’s tag.

I decided to run this again 200 times on a more specific tag where I might get multiple instagram posts from the same user. So I used the Morrissey with a dataset of 200 items. This time I got 4097 unique users and 8741 on 6633 media items.

I used like count to weight each tag, with each instance of the tag adding 1 and each tag adding 0.25.

The most popular tags:

‘morrissey’, 6623
‘thesmiths’, 2435
‘moz’, 1635
‘music’, 500
‘mozarmy’, 493
‘concert’, 385
‘live’, 351
‘love’, 298
‘smiths’, 267
‘truetoyou’, 233

User-Based Collaborative Filtering / Cosine Similarity & K-nearest neighbor

For a random user, olivia.lord, here are 10 nearest neighbors:

I also tried recommending tags to users with Manhattan. That gave far fewer results. For example, it did not return any results for olivia.lord. melchiano, it gave two results: (‘meatismurder’, 4), (‘pescara’, 4).
Item/Tag-Based Collaborative Filtering

Here are the 20 top tags for ‘Morrissey’:

for ‘music’:

for ‘mozfest’:

for ‘vegan’:

for ‘meat’: