Social Media “Glue” and Gnip’s Co-opetition with FriendFeed

We believe that enabling web technologies are going through a similar development cycle as enterprise application integration technology did 10+ years ago. Companies are creating tools, applications and platforms to enable more productive and automated uses of resources that have become ubiquitous parts of the online ecosystem. We think about these enabling technologies as the glue that will increasingly hold together that ecosystem.

Seth Levine, Foundry Group

Venture capital firm Foundry Group, which includes partner Brad Feld, described an important investment theme for their firm. Titled Theme: Glue, the thesis is that the growing number of web services and content-generating sites are causing increased complexity, and that there is a need for an infrastructure to handle all this.

This can seem a bit dry (“I know this back end plumbing stuff is boring to most of you”, as Michael Arrington says), but its relevance is can be considered in the context of:

Foundry’s thesis extends beyond server-load management. But its initial investment in Gnip starts on that part of the “Glue” story.

The Problem Gnip Solves

The rise of user generated content has made this problem particularly acute. We’re creating so much content, all over the place. Flickr, Del.icio.us, Digg, YouTube, Twitter, WordPress.com, Google Reader, etc. I mean, there’s a lot of stuff!

It turns out a lot of other sites want to consume this stuff – FriendFeed, Plaxo Pulse, Strands, SocialMedian and many, many other sites. And the direction for production and consumption of all this content is only going one way –> up.

The problem that arises is the way consuming services, such as FriendFeed, have had to find out if you’ve got a new tweet, blog post, Digg, etc. The consuming services need to ping every individual user’s account on some site, such as Flickr,  with the query, “got anything new?” For most people, the answer is no. But that query is the cause of a lot of traffic and latency. Imagine all these new web services pinging en masse all the UGC sites. It can be quite a load to handle. In Twitter’s case, it was too much to handle.

Gnip addresses this issue, standing between UGC sites and consuming sites:

gnip-flowThe UGC sites (aka producers) push updates for their various users to Gnip. These updates are either change notifications, or full content for each user. So Gnip becomes the reference layer for anything occurring for a UGC site’s users.

The consuming sites then look to Gnip as a “single throat to choke” in terms of updates. Gnip handles the updates, and gets them out to consuming sites in real-time. Gnip also removes the burden on consuming sites to write and maintain polling scripts for all the various UGC sites of users.

The idea is a good one, as it offloads a lot of the burden for both producers and consumers. Of course, it shines a light on Gnip’s scalability and uptime stats.

Initial consuming sites using Gnip include:

Gnip is off to a nice start. But what about FriendFeed?

FriendFeed’s Ex-Googlers Roll Their Own

FriendFeed is one of those consuming sites. But they’ve not signed on for Gnip so far. Not surprising, considering their Google background. Lots of good knowledge about scalability to be learned from Google.

Rather than sign on to Gnip’s service, Friendfeed has proposed the simple update protocol (SUP). What’s SUP?

SUP (Simple Update Protocol) is a simple and compact “ping feed” that web services can produce in order to alert the consumers of their feeds when a feed has been updated.

The idea is that the UGC sites provide a single point for posting notifications of new user activities. Rather than the consuming sites running the “got anything new?” query for every single user on their platform, they go to a single place to see what’s new. They have a list of the user IDs they want to check, which they run against the SUP location. Much more efficient.

Which does sound a little like Gnip, doesn’t it? Here’s a Q&A between Marshall Kirkpatrick of ReadWriteWeb and FriendFeed co-founder Paul Buchheit:

RWW: [Where is this relative to] Gnip? (See our coverage of Gnip, a startup that appears to be aiming to do what SUP will do and more.)

Buchheit: We’re talking with several companies about supporting SUP, but aren’t ready to announce anything.

On the TechCrunch post about Gnip 2.0, commenter Nikolay Kolev writes:

Even if I like Gnip’s concept a lot at this moment, I think it’s just a temporary solution of the real problem. It solves deficiencies of the vast majority of the data producers nowadays, yes, but if more implement XMPP PubSub, FriendFeed SUP and other similar technologies, there will be less incentives for data consumers to make their business rely on a single provider that supposedly aggregates and replicates all of the Web…

On FriendFeed, user Dani Radu writes this in response to Gnip:

pretty interesting, I mean making all this data handling drop dead simple is great – but this means they want to cache and route the direction of interest. Own the process and sell it. Again, perfectly sane – but I’d rather go for the SUP (simple update protocol) which in a way – if adopted widely – does the same but keeps the handling free (as free as the services are anyway) We shall see what future brings tho…

The Gnip vs. SUP question came up on Hacker News, which included this exchange with FriendFeed’s Paul Buchheit:

ryanwaggoner: Isn’t this what Gnip is doing, except that Gnip’s solution is readily available to anyone who wants it? In fact, I believe Gnip uses XMPP to push notifications to data consumers, which seems even more efficient. Am I missing something?

paul: No, Gnip is a complementary service and will likely consume SUP. SUP is intended to make it easier for feed publishers to expose information about which feeds have been updated. Without this information, Gnip can’t know when feeds have updated except by polling all of them. SUP allows them to poll a single URL instead.

ryanwaggoner: Got it. So this is designed to be the piece that allows publishers to easily integrate with intermediate services like Gnip, or with aggregation services like FF, SocialThing, etc.

paul: Exactly

Paul’s right, but the earlier comments are also right. Gnip may want to get updates for its UGC producing sites using SUP. But there’s truth to the idea that if producers offer SUP, some of the value proposition of Gnip is eroded.

Gnip: More Than Real-Time Updates

But Gnip appears to provide a range of services above and beyond simple update notifications. My guess is that those extra services Gnip provides above and beyond providing a single place to get notifications will be their secret sauce.

On the Gnip blog, product head Shane Pearson writes the four use cases on which Gnip is focused:

  1. Eliminate the need for developers to write dozens of pollers for UGC sites, all of which must be maintained and updated
  2. Target business specific applications that need this data. There may be interesting functional or vertical application that SUP won’t cover.
  3. Offload the overhead on UGC producers’ sites (which sounds like SUP). But beyond that, create an alternative channel for their content, provide analytics on the data consumed through Gnip ,and add filters an d target endpoints.
  4. Use Gnip as a source of market research and brand analysis for what consumers are saying about companies.

So what you see here is that the developer world sees SUP competition in the infrastructure part of Gnip. Gnip is looking beyond the developer world in terms of where it delivers value.

I wouldn’t be surprised if other companies enter the mix. “Glue” is an early, interesting space right now.

*****

See this post on FriendFeed: http://friendfeed.com/search?q=%22Social+Media+%E2%80%9CGlue%E2%80%9D+and+Gnip%E2%80%99s+Co-opetition+with+FriendFeed%22&who=everyone

Would You Apply a ‘Dislike’ to Your Co-Workers’ Content?

On Digg, you can apply the “bury” rating to stories. On Amazon.com, you can apply a single star to rate something negatively.

Would you ever do that to the work of your colleagues?

I’m not talking the annual HR exercise of 360 reviews. I’m talking Enterprise 2.0 apps, which incorporate the features we see out in Web 2.0. The ability of people to rate the content they see.

A few social media sites have taken a binary approach to ratings: (1) positive rating, or (2) abstain:

While some others are incorporating the notion of negative ratings:

Out on the Web, where you’re interacting on platforms with thousands of anonymous or unknown people, negative ratings make sense and help bring some order to the scrum of content and products.

See Louis Gray’s post for a good perspective on this whole rating thing in social media.

Inside companies, things are a little different. There’s a vetting of other Enterprise 2.0 users, in the form of the hiring and annual review process. This automatically raises the average quality of contributions.

And there’s this….Enterprise 2.0 apps are used by people you know and work with. People you’re going to see in meetings, on projects and who have common connections. A negative rating to someone’s Yammer or wiki entry or social bookmark is a big deal. You’re essentially saying:

“Dude, this is bad. I mean really bad. So bad that I had to ‘dis you and let the rest of the organization know how bad it is.”

Personally, I’d have a hard time with this. In the most egregious cases, I’d apply the negative rating. But I’d strongly prefer to “work it out” in the comments to the original content.

My concern is that a negative rating turns into a basis for internal feuding and chills open discussion about ideas, information and observations. But in a large organization with a heavy flow of content, maybe the negative rating is the most efficient way to handle the value if information.

Perhaps I’m in the minority here. How about you? Would you give a thumbs down to your co-workers’ content via Enterprise 2.0 apps?

*****

See this post on DFriendFeed: http://friendfeed.com/search?q=%22Would+You+Apply+a+%E2%80%98Dislike%E2%80%99+to+Your+Co-Workers%E2%80%99+Content%3F%22&who=everyone

Tag Clouds for Our Lifestreams

We are marching down the lifestreaming road. There are a proliferation of lifestream apps, such as FriendFeed, SocialThing, Strands, Swurl and others. Lifestreaming is getting hotter, and there’s some thought that lifestreaming will be the new blogging:

Sites and social tools like these and many others encourage more participation on the social web than ever before. Although the social participants on these sites are often more active in socializing than they are in blogging, there’s still that need to stake out your own piece of real estate on the web. But we wonder: does that really need to be a blog anymore? Perhaps not.

It’s a great concept, one that Mark Krynsky has been chronicling for a while at the Lifestream Blog.

An area I think that is ripe for inn ovation here is the ability to find the meta data from one’s lifestream. On FriendFeed, people will have multiple services that fill up their lifestreams. A couple issues that crop up on FriendFeed are:

  • Figuring out whether to subscribe to someone
  • Catching up on what particular individuals have been streaming

Because there is one thing that has been noticed with all this lifestreaming – there’s a lot of information generated (or “noise” as some might say).

So here’s my idea:

Create tag clouds for our lifestreams

What do I mean? Read on.

FriendFeed Lifestream

I’ll use the lifestream service with which I’m most familiar, FriendFeed. Here are the tag clouds I’d like to see for each user’s lifestream:

  • Blog
  • Music
  • Google Reader shares
  • Bookmarks
  • Twitter
  • YouTube
  • Flickr
  • Digg
  • etc…

And I’d like to see tag clouds for what users Like and Comment on. Because on FriendFeed, Likes and Comments have the same effect as a direct feed of someone’s lifestream: they put the content into the feed of all their followers.

So via the tag cloud, I’m can quickly come up to speed on what someone is interested in.

Let’s Make Tagging Easy

I don’t propose that users suddenly tag their own streams. Rather, let’s leverage the work of others.

It’s de rigueur for Web 2.0 apps to include tagging. Bloggers tag. Social bookmarkers tag. Music lovers tag. Why not pull the tags applied to the source content into the lifestream?

Here’s what I mean. My blog has plenty of tags. These tags are included in the RSS feed of my blog. So any feed that includes my blog should include these tags. Let’s leverage:

  1. The tags that people apply to their own Web 2.0 content
  2. RSS/Atom feeds that include tags

For some background on this, click here for a page on Technorati that talks about tags in feeds.

By leveraging the tagging work already resident in user-generated content, one can quickly build up a tag cloud for lifestreams.

An Example: Google Reader Shares

Google Reader is a good example. People ‘share’ blog posts they read via their Google Readers. Sharing lets others see the articles that someone finds interesting and useful. And of course, those blog posts that someone is sharing have tags.

Here’s what the tag cloud of my recent Google Reader shares looks like. I’ve simulated the tag cloud by using Wordle for the tags.

You can see my interests lately: Enterprise 2.0, FriendFeed, social media. If someone wanted to get a quick sense of the things they’ll see by subscribing to me, this tag cloud answers that. And if someone is curious about the specific posts I’ve been sharing that relate to a subject, they could click on one of the tags and get a list of my Google Reader shares.

Curious, I ran the same analysis on the Google Reader shares of four people I follow on FriendFeed: Robert Scoble, Louis Gray, Sarah Perez, Mike Fruchter. Here are the topics they’ve been sharing lately:

Robert Scoble clearly is following the iPhone and Google. Louis Gray was following the happenings at Gnomedex. Sarah Perez is pretty even in her interests, with FireFox, social bookmarking, FriendFeed, Twitter, search and photos among her favorite topics. Mike Fruchter has been reading up on Twitter and social media.

Just like that, I’ve gotten a sense for their interests right now. And if those were true tag clouds, I could click the tag and see the Google Reader shares. Robert Scoble is really good at tracking useful relevant things. Clicking the ‘iPhone’ tag and reading his shares would be a quick way to understand what’s goin.

Tags + Wordles

As I said, most user generated content comes with tags these days. So pulling these into the feeds and representing them in a tag cloud would be a fantastic move forward in creating lifestream tag clouds.

But what about Twitter? There are no tags on tweets. Not a problem. FriendFeed and other lifestream services could do a Wordle-like tag cloud. Tally the most common words in someone’s tweets, represent it as a tag cloud. And make the tag cloud clickable, which would essentially run a Summize Twitter search of the user’s tweets for a given tag.

Use Existing Metadata to Solve Two Problems

The key here is to not make it onerous on the end user. Tag once, re-use everywhere. If desired, users could be given the option to add tags to their own lifestreams. But the core idea is to eliminate double tagging work for users.

If this could be done, you’ve got a visual representation of people’s lifestreams. And an easy way to find the specific entries in a lifestream that relate to a topic.

Lifestreamers – would you want something like this

I’m @bhc3 on Twitter.