Advertisements

Social Media “Glue” and Gnip’s Co-opetition with FriendFeed

We believe that enabling web technologies are going through a similar development cycle as enterprise application integration technology did 10+ years ago. Companies are creating tools, applications and platforms to enable more productive and automated uses of resources that have become ubiquitous parts of the online ecosystem. We think about these enabling technologies as the glue that will increasingly hold together that ecosystem.

Seth Levine, Foundry Group

Venture capital firm Foundry Group, which includes partner Brad Feld, described an important investment theme for their firm. Titled Theme: Glue, the thesis is that the growing number of web services and content-generating sites are causing increased complexity, and that there is a need for an infrastructure to handle all this.

This can seem a bit dry (“I know this back end plumbing stuff is boring to most of you”, as Michael Arrington says), but its relevance is can be considered in the context of:

Foundry’s thesis extends beyond server-load management. But its initial investment in Gnip starts on that part of the “Glue” story.

The Problem Gnip Solves

The rise of user generated content has made this problem particularly acute. We’re creating so much content, all over the place. Flickr, Del.icio.us, Digg, YouTube, Twitter, WordPress.com, Google Reader, etc. I mean, there’s a lot of stuff!

It turns out a lot of other sites want to consume this stuff – FriendFeed, Plaxo Pulse, Strands, SocialMedian and many, many other sites. And the direction for production and consumption of all this content is only going one way –> up.

The problem that arises is the way consuming services, such as FriendFeed, have had to find out if you’ve got a new tweet, blog post, Digg, etc. The consuming services need to ping every individual user’s account on some site, such as Flickr,  with the query, “got anything new?” For most people, the answer is no. But that query is the cause of a lot of traffic and latency. Imagine all these new web services pinging en masse all the UGC sites. It can be quite a load to handle. In Twitter’s case, it was too much to handle.

Gnip addresses this issue, standing between UGC sites and consuming sites:

gnip-flowThe UGC sites (aka producers) push updates for their various users to Gnip. These updates are either change notifications, or full content for each user. So Gnip becomes the reference layer for anything occurring for a UGC site’s users.

The consuming sites then look to Gnip as a “single throat to choke” in terms of updates. Gnip handles the updates, and gets them out to consuming sites in real-time. Gnip also removes the burden on consuming sites to write and maintain polling scripts for all the various UGC sites of users.

The idea is a good one, as it offloads a lot of the burden for both producers and consumers. Of course, it shines a light on Gnip’s scalability and uptime stats.

Initial consuming sites using Gnip include:

Gnip is off to a nice start. But what about FriendFeed?

FriendFeed’s Ex-Googlers Roll Their Own

FriendFeed is one of those consuming sites. But they’ve not signed on for Gnip so far. Not surprising, considering their Google background. Lots of good knowledge about scalability to be learned from Google.

Rather than sign on to Gnip’s service, Friendfeed has proposed the simple update protocol (SUP). What’s SUP?

SUP (Simple Update Protocol) is a simple and compact “ping feed” that web services can produce in order to alert the consumers of their feeds when a feed has been updated.

The idea is that the UGC sites provide a single point for posting notifications of new user activities. Rather than the consuming sites running the “got anything new?” query for every single user on their platform, they go to a single place to see what’s new. They have a list of the user IDs they want to check, which they run against the SUP location. Much more efficient.

Which does sound a little like Gnip, doesn’t it? Here’s a Q&A between Marshall Kirkpatrick of ReadWriteWeb and FriendFeed co-founder Paul Buchheit:

RWW: [Where is this relative to] Gnip? (See our coverage of Gnip, a startup that appears to be aiming to do what SUP will do and more.)

Buchheit: We’re talking with several companies about supporting SUP, but aren’t ready to announce anything.

On the TechCrunch post about Gnip 2.0, commenter Nikolay Kolev writes:

Even if I like Gnip’s concept a lot at this moment, I think it’s just a temporary solution of the real problem. It solves deficiencies of the vast majority of the data producers nowadays, yes, but if more implement XMPP PubSub, FriendFeed SUP and other similar technologies, there will be less incentives for data consumers to make their business rely on a single provider that supposedly aggregates and replicates all of the Web…

On FriendFeed, user Dani Radu writes this in response to Gnip:

pretty interesting, I mean making all this data handling drop dead simple is great – but this means they want to cache and route the direction of interest. Own the process and sell it. Again, perfectly sane – but I’d rather go for the SUP (simple update protocol) which in a way – if adopted widely – does the same but keeps the handling free (as free as the services are anyway) We shall see what future brings tho…

The Gnip vs. SUP question came up on Hacker News, which included this exchange with FriendFeed’s Paul Buchheit:

ryanwaggoner: Isn’t this what Gnip is doing, except that Gnip’s solution is readily available to anyone who wants it? In fact, I believe Gnip uses XMPP to push notifications to data consumers, which seems even more efficient. Am I missing something?

paul: No, Gnip is a complementary service and will likely consume SUP. SUP is intended to make it easier for feed publishers to expose information about which feeds have been updated. Without this information, Gnip can’t know when feeds have updated except by polling all of them. SUP allows them to poll a single URL instead.

ryanwaggoner: Got it. So this is designed to be the piece that allows publishers to easily integrate with intermediate services like Gnip, or with aggregation services like FF, SocialThing, etc.

paul: Exactly

Paul’s right, but the earlier comments are also right. Gnip may want to get updates for its UGC producing sites using SUP. But there’s truth to the idea that if producers offer SUP, some of the value proposition of Gnip is eroded.

Gnip: More Than Real-Time Updates

But Gnip appears to provide a range of services above and beyond simple update notifications. My guess is that those extra services Gnip provides above and beyond providing a single place to get notifications will be their secret sauce.

On the Gnip blog, product head Shane Pearson writes the four use cases on which Gnip is focused:

  1. Eliminate the need for developers to write dozens of pollers for UGC sites, all of which must be maintained and updated
  2. Target business specific applications that need this data. There may be interesting functional or vertical application that SUP won’t cover.
  3. Offload the overhead on UGC producers’ sites (which sounds like SUP). But beyond that, create an alternative channel for their content, provide analytics on the data consumed through Gnip ,and add filters an d target endpoints.
  4. Use Gnip as a source of market research and brand analysis for what consumers are saying about companies.

So what you see here is that the developer world sees SUP competition in the infrastructure part of Gnip. Gnip is looking beyond the developer world in terms of where it delivers value.

I wouldn’t be surprised if other companies enter the mix. “Glue” is an early, interesting space right now.

*****

See this post on FriendFeed: http://friendfeed.com/search?q=%22Social+Media+%E2%80%9CGlue%E2%80%9D+and+Gnip%E2%80%99s+Co-opetition+with+FriendFeed%22&who=everyone

Advertisements

Bit.ly Gets Better with New Data…Are You Using It Yet?

Lately, I’ve been using bit.ly for shortening the URLs I tweet, on the advice of Marshall Kirkpatrick at ReadWriteWeb. I started using it instead of is.gd, which had been my previous favorite.

Why? Because bit.ly offers an array of useful data. Who knew that a simple URL shortener could open up so much interesting data?  I can’t believe people still use tinyurl and other services that “only” shorten URLs. The tracking of metadata around a posted URL – for free – makes bit.ly really powerful.

Here’s what bit.ly was offering before the latest data features…

  • Last 15 URLs: Bit.ly knows your last 15 shortened URLs, courtesy of a cookie.
  • Post to Twitter: Post shortened URLs from bit.ly to your Twitter account
  • Archived web page: Yup, see that page anytime because there’s a cached version of it, even if the source link changes or disappears.
  • Traffic sources: See how much click action that bit.ly URL got once you put it out there. And from what apps.
  • Conversations: Tracks which users on Twitter and FriendFeed put the URL out there. This is really cool, as you can see others who liked the same thing you did.
  • Browser bookmarklet: Easy way to create a shortened URL, stay on the page you’re reading.
  • Semantic metadata: According to Marshall’s July post, bit.ly was going to add semantic analysis via Reuter’s OpenCalais API. Looks like it’s there. Cool to see per link, probably more interesting with a critical mass of URLs.

On October 30, bit.ly announced several nice additions to their service.

  • Full referring domains: Not just the top-level domain.
  • Graph of click activity by time: The dates and times that a URL got clicked.
  • Clicks by Country: The countries of people who click on your URL. This is really fascinating.

Seriously, if you’re not using bit.ly, why not?

*****

See this post on FriendFeed: http://friendfeed.com/search?q=%22Bit.ly+Gets+Better+with+New+Data%E2%80%A6Are+You+Using+It+Yet%3F%22&who=everyone

Weekly Recap 062708: The FriendFeed Immigration Continues

Twitter’s replies tab has been disabled most of this week, causing a fair amount of consternation…without Replies, it’s hard to maintain asynchronous conversations, or even synchronous ones if you’re conversing with 10,000+ people…

So a few more of the bigger technorati are discovering the merits of FriendFeed…

Michael Arrington: “Friendfeed for most users was just a place to bookmarks all their activities on other social networks. Now, more and more, it’s a place that people start conversations. The early adopters got that a while ago. Now, the not so early adopters are using it as a Twitter replacement, too.”

Dave Winer: “I’m steering people to FriendFeed, can’t help it. My discussions are happening there. And bonus: It pisses off Steve Gillmor. :-)”

Shel Israel: “Really tired of Replies being broken here. Spending more time in FF, but still subscribing only to close friends over there.”

Steve Gillmor: “friendfeed is getting very close to being usable”

Chris Saad: “So is the idea we use friendfeed instead of Twitter? Does that actually work?”

Not bad at all…but we’ll see how long it lasts…collectively, the last four (excluding Arrington) have 18,670 followers, which is hard to match any time soon on FriendFeed…as Corvida noted:

when you get out of one relationship that you’ve put so much time and effort into, do you really feel like going out there, just to find a replacement to try to rebuild what you had with someone else?

Once Twitter rights its ship (in several months), we’ll see how many of the Twitter refugees stick around on FriendFeed…

*****

Have you heard of the Persian Cam Room on FriendFeed? Join it! Amazing pictures can be found there.

.

Amazing pictures can be found there.

*****

I often see complaints on FriendFeed about too much FriendFeed talk…to which I say, that’s why they provide the Hide function…

*****

Facebook has added FriendFeed-like functionality, allowing comments on the activity streams of your friends…I’m trying it out a bit, here are a few initial impressions:

  • It’s hard to comment on someone adding the “Hug Me” application
  • Status updates are easier content on which to comments
  • You really get used to the speed with which commenting and accessing new content occurs. Facebook is so painfully slow in comparison
  • I was pleased with the comments I got back after I did my first round of comments. Will continue to play with it.

*****

Finally, I wanted to note the moves of several solo bloggers into the “big time”

Interestingly enough, Frederic had just the prior week written a post in which he noted:

It’s close to impossible for a solo blogger to make a living in the tech blogosphere.

Now he’s part of the big time…congrats to all!

*****

See this item on FriendFeed: http://friendfeed.com/search?q=%22Weekly+Recap+062708%3A+The+FriendFeed+Immigration+Continues%22&public=1