Remix the News: what news can learn from Last.fm and Pandora
As Paul Bradshaw and many others have pointed out, there is a natural synergy between music and news. Both are content-driven industries that are struggling to adapt to new forms of distribution.
But the similarities don’t end there.
Music is finally being recognized as an art that depends largely on social influences. We tend to listen to what our friends listen to, and it is one of the primary ways we discover new music.
News, however, has not caught up to music in this way. Links are passed around on Twitter, Facebook, in emails, and via instant messaging. Each time we pass along a link to a story or blog post, that data is not recorded in any way, forcing readers to start from scratch when they want to discover new stuff to read (or constantly go back to the same site).
Sure, there is Digg, Tweetmeme and other aggregators, but there is no service that adequately customizes content to my tastes based on previous reading. With my music, I can use applications like Pandora and Last.fm and can punch in my favorite artist and explore playlists of other fans as well as listen to similar bands. For news, we’re out of luck. This despite that both media can be tagged, linked, retweeted, downloaded and consumed in nearly the same way.
So lets build it.
Below is how a news discovery app might work, along with the music service that inspired the feature. Ready?
Normalize Data
Before we build our news remixer application we have to get our data in order.
The organization structure of music is similar to that of news. For instance what does the song “Here I Come” by The Roots and this news article have in common? Data structure (see right).
Music is mostly tagged using the ID3 standard so users don’t not have to constantly rename files when they play them in iTunes and then Windows Media Player. News, on the other hand, has no standard way of semantically tagging stories. Most XML feeds contain limited amounts of data.
To have any sort of news remixer someone will need to normalize news metadata much like SimpleGeo has normalized geolocation data.
After the data is all normalized (no small feat) we can then borrow some features from existing music services for our news remixer.
Shuffle: iTunes
As we’ve written before, The Guardian and Wordpress blogs already allow you to read a story chosen at random.
As any music junkie knows, hitting those crisscrossing arrows in iTunes can pull a random song from more than one artist, so where is the shuffle feature that allows you to read a random article from any publication?
StumbleUpon fills most of that void but it’s not truly random as sites are submitted by users and the content isn’t just news, it’s any page on the web. That can be a little too random for our news remixer app.
Most popular stories: The Hype Machine
For music junkies looking to discover the bleeding edge latest releases, they usually go to The Hype Machine. The service posts find the hottest mp3 files posted to blogs. Instead of going to blog after blog and hitting play, the user can just have The Hype Machine create a playlist automatically.
Our news remixer application should automatically be able to create a reading list based on the most blogged about news stories of the day.
Find stories like this one: Pandora
There’s a reason that seems like every office jockey loves Pandora. The service allows you to enter in one artist and it will automatically create a playlist of similar artists, driving the discovery of new music. With just one search you can have music all day.
News, however, is a bit more complicated. Would I want to read stories by the same author? Same newspaper? Same subject matter? I think our news remixer should take into account four factors when finding a similar story: Length, tone, timeliness and subject matter.
For example if I read a great 3000-word investigative piece on the local court system, the related articles could be similar lengthy investigative pieces.
Social Graph and history: Last.fm
Last.fm is a service where users can log or “scrobble” the music the listen to. Last.fm then learns from your listening habits and can recommend new music. Last.fm also is able to see what your friends are listening to and recommend new music (see above, a random user that last.fm has identified as my musical "neighbor").
The newsremixer should able to use Facebook connect, Twitter connect and Gmail contacts to find who my friends are and what they are reading. For example, many of friends, like me, are Philadelphia Phillies fans. Our app could see that I typically read stories about the Phillies and recommend the Phillies stories that my friends are reading that I may have missed.
Create a news playlist: mflow
Again, I’ll leave most of the mlfow comparisons to Paul Brashaw, who helped inspire this post. The short version of his article, however, is that mflow creates a list of recommended songs based on the artists and friends you “follow.”
Can’t we do the same for news?
Things we can’t learn from music
Micropayments - While music and news have a lot of overlapping characteristics there are some things that are better left to music, especially the illusion of an “iTunes for news.” Better arguments against micropayments have been made elsewhere, but suffice to say I’d be hardpressed to find anyone who would pay a dollar to read a news story.
Merchandise - Many small bands rely on merchandise for revenue. I don't think anyone is buying any items from the drastically overpriced New York Times store.
Join the discussion
Last Saturday in between sessions at BCNI Philly (thanks Sean!) some of us were talking about something similar to this for non-news content. I think I called it "intelligent serendipity". As a big Pandora fan, I'm a believer.
I believe a song is a more like a book than a news article, in that its shelf life is measured in decades, not days. In addition, each song is a fixed, discrete item, with some minor exceptions (remixes and samples). The singer, song title and genre are excellent, meaningful data points. The music archive, combined with the preference data of millions of listeners, is therefore a huge dataset that can be used to pretty reliably predict new music that you'll like.
How can one news story, reported by dozens of reporters, rewritten, curated, annotated and mashed up by thousands of bloggers, and aggregated by millions of sites, be categorized as easily?
For a "Pandora for news" to deliver intelligent serendipity (and not just an echo chamber of the same ol' same ol'), I imagine it would have to process reader preferences ("likes") in pretty much real time. There's enough news being consumed to make this theoretically possible, but where's the ROI for building such a system? It's probable that this massive amount of data handling will soon be trivial enough to make it cheap enough for someone to try it, but I'd advise them to figure out how they're going to make it pay first.