Duke Listens!: On the stairway to the Celestial Jukebox - a blog by Paul Lamere

Here's a Freakomendation from iLike. I was listening to the progressive rock masterpiece Karn Evil 9 by Emerson Lake and Palmer, when iLike suggested that I might like the song Grandma got run over by a reindeer performd by Elmo and Patsy. The rest of the recommendations were other Christmas schlock.

ilike apparently thought that ELPs only distinctive song was the Christmas chestnut I Believe in Father Christmas and connected it up with other well worn Christmas music.

One Llama is a music recommender that uses "acoustic analysis, cultural analysis and collaborative filtering tools for music navigation, discovery and search. On their website they say One Llama uses a combination of Collaborative Filtering and Audio Similarity modeling to generate recommendations. Our model harvests cultural references and social networking data about each track, and listens to the audio using an advanced "virtual ear." The result is a stronger combined logic for all our recommendations. The One Llama method has the advantage of being able to give intelligent recommendations for new audio tracks immediately while becoming increasingly smarter as additional information is collected about the tracks from playlists, downloads, user feedback, etc.

So with all that advanced mojo, one would expect some pretty good recommendations. Here's a recommendation based on the seed song 'Hey Jude' (I chose the Elvis version because they didn't seem to have the Beatles version in their catalog).

There's no doubt that these songs are "like 'Hey Jude'", but somehow the recommendation lacks subtlety and novelty of a real recommendation. Clearly the songs are not acoustically similar (Arthur Fielder vs. Tiny Tim?), and I can't imagine any set of users that would be listening to this set of song, so this is not being driven by a collaborative filtering algorithm. It seems that, at least for this recommendation, the primary driving force is metadata similarity. It is almost as if they just grabbed the Musicbrainz track data, tossed it all into a text similarity engine and turned the crank to get these similarities.

Zac points out another case where One Llama seems to be relying mostly on metadata. Here's a playlist that One LLama generates for songs similar to "Let Go" by Frou Frou. The set seems mostly reasonable from an acoustic point of view - the playlist could have been constructed by an expert - and in fact it was. The songs (with one exception) can all be found on the Garden State soundtrack.

This is probably what one could expect from a collaborative filtering system. Lots of music listeners have bought the soundtrack. Any good CF algorithm will notice this and tie the items together. However, I don't think that is what is going on here. Looking at the One LLama playlist, there is one song that is not on the Garden State album. One Llama has added The Postal Service's 'Such Great Heights' to the playlist, while the Garden State has the cover of 'Such Great Heights' by Iron & Wine - although this is a cover, they sound very different; one is electronic-noise-pop, while the other is strictly acoustic. I suspect that, as with the Hey Jude example, One Llama is relying mostly on metadata similarity to determine similarity

Here's the track list for the Garden State:

Using metadata to generate track similarity is not inherently bad. It makes sense to use what works best. A young recommender company like One Llama doesn't have the deep user data necessary to generate good CF recommendations. Creating recommendations based on automatic acoustic analysis is really hard, acoustic-based recommendations are frequently prone to making mistakes that no human would make. I suspect that One Llama has adjusted the dials on their recommender to give more weight to the metadata until they get more user data and their automated analysis is up to par.

Last.fm has a similar artist feature. When you are looking at the page for an artist they will show you artists that are similar based upon the wisdom of the crowds. Last.fm can tell you for instance, that people who listen to Emerson, Lake and Palmer also listen to Yes.

If you go the Hillary Rodham Clinton page at Last.fm and take a look at her 'similar artists' you'll find a motley crew that includes Joseph Stalin and Adolf Hitler. Perhaps not the types of world leaders that Hillary would want to be associated with.

It is even worse if you go to Joseph Stalin's page, where you'll find similar artists such as Michael Savage, Ann Coulter and Rush Limbaugh. Now this was clearly engineered as a prank. Someone (or a group of someones), must have created a playlist with Hillary, Adolf, Coulter, Limbaugh and Stalin and just played them over and over again, feeding their play data into the Last.fm audioscrobbler until Last.fm noticed the correlation and declared that they were similar artists. This is one of the first instances of I've seen where a music recommender has been noticeably manipulated to produce a dishonest recommendation. It certainly demonstrates how these types of systems can be vulnerable to attack.

Luckily, there are some smart people working to protect us from this hacking. Bamshad Mobasher has some good papers on the topic that are worth reading.

As more people seek out long tail content, recommenders will become increasingly important, which means that the folks who are spamming and splogging and seo-ing, will be trying to hack our recommenders to get their remedies for hair loss treatment at the top of the list. (Thanks Elias)

I was looking at the Keith Fullerton Whitman Google Music page when I noticed that there was a Google Sponsored Link on the side for Pandora. KFW is not exactly a mainstream artist, so it seemed odd that Pandora would be purchasing Sponsored Links for his page. Well, I clicked on the link and much to my surprise I was brought to a Pandora Radio station for Goldenboy. Now, lets be clear, there's absolutely no similarity between Goldenboy and KFW. Looking at the google page, I can't figure out the reason behind the Pandora ad placement. Something went awry somewhere - or perhaps there's some connection between KFW and Goldenboy that I don't know about. Perhaps Pandora or the Echonest can answer the question.

I hope people don't think I am picking on Amazon. Amazon clearly has one of the best recommenders in the world. The last time I went to Amazon, I intended to buy 1 book, I ended up with 5 all because of their recommender. 99% of the recommendations from the Amazon recommender are spot on - but there's a small number of recommendations that are surprising, funny or just plain crazy. Now this doesn't always mean that these are bad recommendations. For example, here's one that was sent to me by Anita Lillie. She says:

I noticed you are posting freakomendations on your blog, and it reminded me of how I was looking for flowers for a mother-in-law-type person for Christmas last year, and I got a recommendation for the video game Halo. I went back today to try to find the same recommendation, but I couldn't find it. Instead, I see a "recommendation" ("other customers who bought... also bought...") for the movie "Hot Fuzz" fairly frequently within the "flowering indoor plants" product category. Anyway, it was particularly funny with the Halo, and I'm guessing it's all those 20-something guys who go online to order something for their moms.

As Anita suggests, the demographic of Amazon flower purchases probably skews to 20-somthing guys getting something for their moms, so throwing in Halo or Hot Fuzz, may not be a bad way for Amazon to make an extra sale or two.

Zac sends along this freakomendation: I don't know if you're a Kinky Friedman fan, but his books are detective stories -- kind of a foul-mouthed cross between Phillip Marlowe, Hunter Thompson and Groucho Marx. He happens to have a cat.

Steve points to a freakomendation thread on John Scalzi's blog: "Today Amazon suggested The Last Colony to me for purchase. Yeah, you know, I’ve read that. But it’s nice to know Amazon’s algorithm thinks I might like my own stuff." One interesting comment: Amazon’s algorithm also has an annoying (well, it was funny the first time, since it happened on April 1st. But then it kept on happening, and I realized they were serious) habit of treating writer’s names, without bothering to check if it’s the same writer or not. I bought a few of Sharon Lee’s and Steve Miller’s Liaden Universe books through Amazon. So they started to give me recommendation for other books by Steve Miller. Which would have been fine, except this new Steve Miller is a completely different Steve Miller and Amazon apparently thinks I would really like illustration advice books.

This is what happened with yesterday's Steve Martin freakomendation, where Amazon recommended a book by the wrong Steve Martin. LibraryThing, another book recommender, at least understands that there are two Steve Martin's that write books, but they still can't tell them apart. At the LibraryThing author page for Steve Martin there is this notice: Steve Martin is actually two authors, Steve Martin the comedian and author of Cruel Shoes, Kindly Lent By Their Owner: The Private Collection of Steve Martin, Shopgirl, Pure Drivel, WASP, The Pleasure of My Company, and Born Standing Up; and Steve Martin the author of Britain and the Slave Trade. In the future LibraryThing will be able to split authors with identical names. At present, it cannot.

This makes be appreciate MusicBrainz so much more. MusicBrainz knows all about the various ambiguous artist names and can tell them apart. I guess there's no such thing as BookBrainz yet.

Here are a few more freakomendations from Netflix.

No Country for Old Ogres

From "The Killers" to "Looney Tunes"

A Death Wish for Moses

Here's a recommendation from Netflix. Because you enjoyed Trekkies, Netflix suggests that you'll enjoy Frontier House.

Are these really Better Together?

Some people are bird watchers, some collect cars. I like to collect unusual recommendations. I'm calling these 'Freakomendations'. There's almost always a story behind the recommendation - but sometimes it is hard to track them down.

This recommendation, Amazon suggests that since Aaron Hurly has purchased books by Steve Martin, he may be interested in the (no doubt) hilarious Public Services Inspection in the Uk: Research Highlights in Social Work

It looks like Amazon decided that the Steve Martin who edited this book was that same wild and crazy guy.

Via aaronhurley.org