April 6, 2008
by Hutch Carpenter
Toluu has entered the ever-growing recommendation space with something different: blog recommendations. And the service does a good job of finding blogs you’ll like.
I love the RSS experience of reading various blogs, loading up my reader with a lot of them and checking updates several times a day. So I was happy to have the chance to try this out. The service is new, launching in mid or late March. Louis Gray has a good post detailing its initial launch. Here’s a description of how it works from the Toluu site:
- After joining, you will be prompted to import your feeds. We have many methods of importing your feeds such as OPML import, URL input, and a nifty bookmarklet.
- Toluu will do some crazy math to find others in the system who have similar tastes as you.
One thing founder Caleb stressed on his blog: “Toluu is not another social network. I repeat Toluu is not another social network.”
So with that intro, let’s look at the user experience and how Toluu rates versus competitors. First, a brief discussion of recommendations.
Quick Note on Recommendations
The recommendations space is a hot area right now. For instance, Loomia, which recommends web content based on what your friends read, just raised $5 million. Amazon.com has been a real pioneer here with its “customers who bought this item also bought…” recommendations.
Ideally, recommendations are exactly matched to your interests. That’s pretty much impossible, but recommendations engines will employ proxies to get a bunch of recommendations that are close to your interests. And hopefully one or more click with you.
There are myriad ways to approximate your interests, and the world of recommendation engines is full of different methodologies. The key thing for most of them is (i) the amount and quality of information about your preferences, and (ii) the amount of population data available to build out recommendations. Toluu uses your OPML file of feeds, which is a very good source of data about your preferences. And Toluu improves as more people participate.
Finally, I’d want a recommendation service to mix highly popular items that I may be missing, as well as less popular items that are relevant to me. That latter category is the real jewel of a recommendation engine, and its the hardest to get right.
Toluu’s Organizing Principle: Match Percentage
Toluu’s primary organizing basis is its Match %. As Caleb mentioned above, this is their “crazy math” secret sauce. After you log in, you click on matches. A list of 5 people are displayed, sorted according to the Match %. The first 5 people you see are your highest matches. Each subsequent page shows the next 5 highest rated people. Each person has 5 feeds listed beside them. These “feeds you might like” are the top 5 recommendations per person.
I had 60 people in my list of matches. My highest match was at 91%. The bottom of the list was guy with whom I matched at 31%.
As I looked through the people that I matched, I noticed a trend. The best Match %’s were with people who had fewer blogs. The lower Match %’s seemed to be with people that had large numbers of blogs. I pulled together some numbers for 30 people to see if this was true. My top 10 matches, 10 people that fell just below the 50% Match %, and my bottom 10 matches. I then graphed it:
Sure enough, the higher the number of feeds for a given user (red line), the lower the Match % (blue line). I’m not quite sure what to make of that. It may be an outcome of the math – the match percentage is lower just because a user has so many feeds there’s no way to match. Or maybe I don’t match up well with the hard-core RSS addicts. I dunno.
One effect is that people who go deeper in their blog interests will fall lower in my matches. Assuming users don’t go too far down in viewing their matches, this could reduce the chance for finding those golden nuggets of less popular, but valuable blogs.
Top Toluu Recommendations Can Be Limited
I cruised through my people matches, and read the 5 “feeds you might like” for each one. There is a high degree of commonality on the recommendations. The 5 recommendations seem to use popularity as an primary input. And that makes sense. You’re providing a service, and popularity means somethings been deemed worthy by the public at large. Start with that!
Again, I looked at the top 5 recommendations for the 30 people I analyzed above. That meant I was looking at my top matches, my mid-tier matches, and my lowest matches.
There wasn’t a lot of variation in the top 5 recommendations for people in the different groups. Micro Persuasion, Engadget, Lifehacker, a couple Google company blogs and Boing Boing consistently showed up, regardless of the Match %.
This narrowness in the recommendations was something that Allen Stern at CenterNetworks wrote about. If you see a recommendation once, you’ll tend to see it repeatedly.
The Rubber Meets the Road: Toluu vs. Google Reader vs. NewsGator
So all that’s well and good. But how does the service perform? I decided to see how Toluu worked relative to two big established market players: Google Reader and NewsGator.
Google Reader has a Discover function. Here’s how it’s described: “Recommendations for new feeds are generated by comparing your interests with the feeds of users similar to you.” Sounds like Toluu, doesn’t it?
NewsGator has a Recommended for Me function: “NewsGator has analyzed your current subscriptions and post ratings, and recommended these new feeds for you.” Doesn’t say how that’s done.
I compared the top dozen recommendations for each of the three services. To assemble my top 12 for Toluu, I calculated the number of times the different blogs appeared in the 30 people I analyzed above. For instance, the blog Micro Persuasion appeared in 19 of the 30 matched users, making it #1. The table below shows those top 12 for each service:
One thing that immediately was apparent. No blog appeared more than once! Three different sets of recommendations and no overlap among Toluu, Google and NewsGator. Incredible!
I then checked out the 36 different sites. After a quick scan of each one, I decided whether it was one I would add to my RSS feeds. Those are highlighted in yellow above. NewsGator’s recommendations fell flat with me. They were too hard-core tech. Several had blog posts with lines of code on them.
Google Reader’s recommendations were the most relevant for me, with 5 that I liked. I subscribe to a number of Enterprise 2.0 blogs, so blogs like Intranet Benchmarking Forum and Portals and KM were good.
But Toluu did well here. The crowd was right – I like Micro Persuasion. Webware.com and Web Worker Daily are also interesting. There are a lot of Google blogs that show up in the recommendations. Maybe a bunch of Google employees are trying out the service?
More people joining Toluu will probably improve this some. At least push the Google blogs off the top recommendations. But there will be some reinforcing behavior as people join. Sites like Engadget and Lifehacker have large followings, and I’d expect a number of new folks joining Toluu to have those already.
Serendipity: Looking at My Top Matches’ Other Blogs
For each person in your match list, you see all the blogs they have that you don’t. It’s here where some of those golden nuggets, and even better known blogs, can be found. It takes work. You need to click each person, and then click each blog. There’s a limit to how much of this I wanted to do.
So I only looked at the feeds of my top 3 matches. And, I did find more blogs I’m going to add to my Google Reader:
- Marshall Kirkpatrick
- Adam Ostrow
- BubbleGeneration
- SocialTimes.com
- mathewingram.com/work
Toluu Assessment = These Guys Are Doing It Right
I picked up 8 new blogs to follow courtesy of Toluu. That’s no small accomplishment. And considering they’re just getting underway and don’t have a ton of users yet, they compete quite well against Google.
I haven’t touched on other features of Toluu has. You can favorite a blog in your collection. I assume this helps the matching algorithm? You can track the activities of others to see what blogs and contacts they’re adding. But remember…this is not a social network!!!
Things I’d Like to See
I’d like to have an easier experience seeing the feeds for my top matches. Since there’s such a commonality in the top 5 for each of them, it would help me discover other blogs if I could see several of my matches’ unique blogs at once.
Show the top ten blogs recommended for me based on my top 10 matches. Criteria = frequency of a blog’s recommendations, with overall popularity as a tie breaker.
I’d like to get a little more info about some of these blogs in a summary fashion, without having to click each one. Maybe the headlines for the most recent 3 posts, or top tags of the blog?
But all in all, a very nice start for Toluu. Thumbs up here. Now I’ve got to go scan my RSS feeds.
I'm Not Actually a Geek 