16 metrics for tracking Collaborative Innovation performance

In a recent PwC survey, 61% of CEOs said innovation was a key priority for their company (pdf). The only surprising result there is that it wasn’t 100%. Innovation efforts come in a variety of forms: innovation and design labs, jobs-to-be-done analysis, corporate venturing, distributed employee experiments, open innovation, TRIZ, etc.

In this post, I want to focus on another type of innovation initiative: Collaborative Innovation. A good way to think about Collaborative Innovation is that it integrates social and crowdsourcing principles:

Collaborative innovation - social and crowdsourcing

 

A definition I use for this approach:

Collaborative Innovation is defined as activities organizations use to improve their rates of innovation and problem solving by more effectively leveraging the diverse ideas and insights of employees, customers and partners.

While it seems straightforward, Collaborative Innovation is actually a fairly sophisticated activity. People with a cursory understanding say all you need to do is: (i) stand up an ideas portal; (ii) let people post ideas; (iii) collect votes on those ideas; and (iv) pick the winners.

Unfortunately, that’s just plain wrong. I’ve seen too many cases where organizations launch idea portals, only to see them die off six months later. The practice of Collaborative Innovation is a rich realm, with solid results for those who apply it thoughtfully.

This post is a look at several key metrics that corporate innovation teams should focus on as they lead Collaborative Innovation programs. The metrics are segmented by the different phases of innovation:

  1. Sourcing
  2. Decisioning
  3. Acting

The metrics below rest on two key assumptions: use of an innovation management software platform; use of campaigns to target innovation efforts.

Sourcing

Sourcing refers to the generation of ideas, as well as eliciting others’ insights about an idea.

Phase objectives

  • Distinct, differentiated ideas
  • Ideas matching needs of customers (incl. internal customers)
  • Ideas matching the innovation appetite of the organization
  • Capturing the cognitive diversity of participants
  • Growing the culture of innovation

Metrics

Metric Description Why
Trend in unique logins Measure the ratio of logins/invited over time for multiple campaigns. Want to see a rise over time until reaching a steady state (~60%).
  • No logins, no ideas
  • Confirm that the credibility of program increasing
  • Demonstrate better targeting of relevant innovation topics
Trend in multiple logins Determine the number of people who log in to each campaign 3 times or more. Divide these multi-login people by the total number of people logging in to each campaign. Look for increasing ratios over time.
  • Returning to a campaign repeatedly is a measure of engagement
  • More repeat logins increases opportunities for collaboration
Ratio of ideators to unique logins Divide the number of people who post at least one idea by the number of unique logins. Want to see a rise over time until reaching a steady state (10 – 15%).
  • Ensure those with valuable ideas are being invited
  • Track whether the campaign questions are accessible to those invited
  • Confirm credibility of the program is increasing
Average number comments per idea Divide the number of comments by the number of ideas, per campaign. Target an average of 2 comments per idea.
  • Ideas are the start, but need others’ insights to evolve and grow
  • Track the collaboration culture of the organization, and of specific org units
  • Ensure participants understand that more than ideas are desired
Average number of replies per comment Divide the number of comment replies by the number of comments. Target an average of 0.5 replies per comment.
  • Innovation dialogues are healthy for both ideas and the organization’s innovation culture
  • Sharing of insights among employees is a second level objective, and this helps track that
Average number of votes per idea Divide the number of votes by the number of ideas, per campaign. Target an average of 3 votes per idea.
  • Participants can help identify ideas with greater potential
  • Ensure the voice of the community is captured, to complement the views of experts and campaign sponsors
Unique org units | departments | locations contributing Count the number of different org units, departments and/or locations with at least one person posting an idea, posting a comment or voting. This count needs to be considered against the number of org units, departments or locations invited.
  • Cognitive diversity is a key driver of value
  • Seek inputs from people who normally aren’t working closely together, to ensure different perspectives are brought into the campaign

Decisioning

Decisioning refers to identifying which ideas move forward for next steps. This phase is the bridge between getting a lot of different ideas, and determining which ones will be acted on.

Phase objectives

  • Identify ideas presenting enough possibility to warrant further review
  • Acknowledge value of community’s perspective
  • Timely assessments of ideas

Metrics

Metric Description Why
Ratio of ideas selected for further review Some number of ideas submitted for each campaign will be selected for the next round of review. Calculate the ratio of selected ideas to total ideas submitted. Watch how this ratio changes over time.
  • Track whether campaigns are generating the level of possibilities expected
  • Look for cases of being overly pessimistic on ideas’ possibilities (too-low or declining ratio over time)
Ratio of top 5 voted/commented ideas selected for further review Of the ideas that were the top 5 for either votes or number of unique commenters, track how many were selected for further review.
  • When the community is invited to comment and vote, they have a natural expectation that their interactions will be part of the decision calculus
  • Failure to regularly consider what the community coalesces around will reduce enthusiasm to participate
Percentage of initially reviewed ideas sent back for iteration & information Of the ideas that were selected for further assessment, track the number where the idea submitter (and team) are asked to iterate the idea and/or provide more information.
  • Ideas rarely have enough “meat” on their initial sourcing, and benefit from further development
  • Watch out for too conservative a mindset by those making decisions on ideas; are they too quick to say ‘no’ without seeking more information?
Time to complete decisions Measure the time between selection of ideas for further review and selection of ideas to move forward into the Acting phase. The time will vary by the level of risk attendant to a campaign.
  • Participants will have a reasonable expectation that promising ideas move forward; delays signal a lack of commitment
  • From the world of finance, the time value of money argues for moving sooner rather than later on ideas with value
Ratio of reviewed ideas that advance to Acting phase Divide the number of ideas selected to move into the Acting phase by the number of ideas selected for review. Watch this ratio over time.
  • Moving ideas forward to action is core to developing an innovation culture; ensure this key step is occurring as expected
  • Too-low or declining ratios indicate a breakdown in the innovation process
  • Ideas that move forward are critical for ensuring the credibility of the innovation effort

Acting

Acting refers to the activities to prove out an idea, develop it and prepare it for full launch. Or to learn why an idea won’t be feasible, ultimately.

Phase objectives

  • Develop deeper understanding for whether the idea passes the three jobs-to-be-done tests that determine market adoption
  • Optimize features that best deliver on the outcomes that the idea’s targeted beneficiaries have
  • Maximize the probability of success by eliminating ideas that just aren’t working

Metrics

Metric Description Why
Average number of experiments per idea Tally the total number of experiments for a “class” of selected ideas for Acting phase, calculate the average per idea.
  • Because of the inherent risk of trying something new, many ideas need different looks
  • Learning mentality to understand an idea’s strengths and weaknesses
  • Valuable for establishing a strong innovation culture for the organization
Time to make final decision on selected ideas Track the amount of time between the decision to put an idea into the Acting phase, and the decision whether to pursue the idea at scale.
  • While this process shouldn’t be rushed, it should be treated with appropriate diligence
  • Participants will expect final decisions; failure to do so undermines the program credibility
Ratio of ideas selected for full launch Divide the number of ideas selected for full launch by the number of ideas selected for the Acting phase. Watch how this ratio tracks over time.
  • The determinant of success for this phase is the number of ideas that make to full launch
  • Ideas in this phase passed muster during the prior Deciding phase; the percentage that make it to full launch should be high
Projected and realized value of ideas that have been moved to full launch Aggregate projected and realized value of the ideas that will be or have been put into full launch.
  • The bottom line rationale for the innovation program
  • Critical for establishing credibility of the program with senior executives

The above list is solid foundation of metrics to track for your Collaborative Innovation program. It’s not exhaustive. And there are likely elements for each phase that will vary for each organization.

But these are good for watching how your program is tracking. Behind each metric, there are techniques to enhance outcomes. The key is knowing where to look.

I’m @bhc3 on Twitter.

Radio Show Interview: Collaborative Innovation at Scale

The area of collaborative innovation is a natural extension of the social business movement. It’s the extension of social into purposeful collaboration, a term Alan Lepofsky uses to describe the evolution of the social business market.

In the innovation-focused radio show, Women Who Innovate, host LeAnna Carey, innovation expert John Lewis and I talk about collaborative innovation at scale. In other words, what are the benefits of, issues with, and techniques for getting hundreds and thousands of people to share ideas and insights, toward a common goal. It’s a different task than getting small teams to collaborate. The recording of the show is below:

This event had a unique twist. It was run in conjunction with the weekly innovation conversation on Twitter, Innochat. In both the radio show on on Twitter, the following topics were covered:

  1. How important is it to get diverse people to contribute to innovation, vs. singular creatives to generate innovations?
    • Doesn’t Steve Jobs point to the primacy of singular genius?
    • What is the model for cognitive diversity to generate innovation outcomes?
  2. What differentiates sharing in large groups vs. small teams?
    • How much does familiarity mean trust?
    • How to handle different personalities that will intersect?
  3. In environments where employee skepticism reigns, how do you change attitudes to open up sharing?
    • What are the ways in which skepticism can creep in?
    • What is the #1 issue that must be addressed?
  4. What are motivations for employees to contribute to an innovation program?
    • How much does “what’s in it for me?” come into play?
    • What are the intrinsic and extrinsic motivations?
  5. What techniques help drive participation in crowdsourced innovation programs?
    • What influence do senior executives have?
    • What influence does peer participation have?
    • How can gamification drive greater participation?

It was a thorough, fast-paced discussion. If you’re considering crowdsourced innovation programs, it’s worth a listen.

I’m @bhc3 on Twitter and I’m a Senior Consultant with HYPE Innovation.

Talk-n-Tweet | Collaborative Innovation at Scale

Previously, I’ve described Why Crowdsourcing Works. Crowdsourcing is a case where you get many people who don’t one another collaborating toward a defined outcome.Talk-n-Tweet Collaborative Innovation at ScaleTo reiterate the principle points about the value of crowdsourcing:

  • Diverse inputs drive superior solutions
  • Cognitive diversity requires spanning gaps in social networks

Simple enough, yet actually a rich field for work and analysis. To that end, I invite to two events happening simultaneously on Thursday 25 September 2014 (12 noon Eastern):

  • LeAnna Carey’s radio show (link)
  • Twitter Innochat (link)

I’ll be on the radio show talking with Lea Carey, Renee Hopkins and John Lewis. At the same time, the weekly #innochat will follow along with the radio program. It’s a unique chance to blend live conversation with online discussion. The main questions to be tackled will be:

  1. How important is it to get diverse people to contribute to innovation, vs. singular creatives to generate innovations?
    • Doesn’t Steve Jobs point to the primacy of singular genius?
    • What is the model for cognitive diversity to generate innovation outcomes?
  2. What differentiates sharing in large groups vs. small teams?
    • How much does familiarity mean trust?
    • How to handle different personalities that will intersect?
  3. In environments where employee skepticism reigns, how do you change attitudes to open up sharing?
    • What are the ways in which skepticism can creep in?
    • What is the #1 issue that must be addressed?
  4. What are motivations for employees to contribute to an innovation program?
    • How much does “what’s in it for me?” come into play?
    • What are the intrinsic and extrinsic motivations?
  5. What techniques help drive participation in crowdsourced innovation programs?
    • What influence do senior executives have?
    • What influence does peer participation have?
    • How can gamification drive greater participation?

As a reminder, the event time across time zones:

Thursday 25 September 2014
9 am Pacific
12 noon Eastern
6 pm Central European Time

I look forward to hearing your take on this issue.

Is Google+ More Facebook or More Twitter? Yes

Quick, what existing social network is Google+ most likely to displace in terms of people’s time?

Another Try by Google to Take On Facebook

Claire Cain Miller, New York Times

This isn’t a Facebook-killer, it’s a Twitter-killer.

Yishan Wong, Google+ post

A hearty congrats to Google for creating an offering that manages to be compared to both Facebook and Twitter. The initial press focused on Google+ as a Facebook competitor. But as people have gotten to play with it, more and more they are realizing that it’s just as much a Twitter competitor.

I wanted to understand how that’s possible. How is it Google+ competes with both of those services? To do so, I plotted Google+’s features against comparable features in both Facebook and Twitter. The objective was to understand:

  • Why are people thinking of Google+ as competitor to both existing social networks?
  • How did the Google team make use of the best of both services?

The chart below is shows where Google+ is more like Facebook or Twitter. The red check marks () and gray shading highlight which service a Google+ feature is more like.

A few notes about the chart.

Circles for tracking: Twitter has a very comparable feature with its Lists. Facebook also lets you put connections into lists; I know because I’ve put connections into lists (e.g. Family, High School, etc.). But I had a hard time figuring out where those lists are. in the Facebook UI. Seriously, where are they for accessing? They may be available somewhere, but it’s not readily accessible. So I didn’t consider Facebook as offering this as a core experience.

+1 voting on posts: Both Google+ and Facebook allow up votes on people’s posts.Twitter has the ‘favorite’ feature. Which is sort of like up voting. But not really. It’s not visible to others, and it’s more a bookmarking feature.

Posts in web search results: Google+ posts, the public ones, show up in Google search results. Not surprising there. Tweets do as well. Facebook posts for the most part do not. I understand some posts on public pages can. But the vast majority of Wall posts never show up in web search results.

Google+ One-Way Following Defines Its Experience

When you look at the chart above, on a strict feature count, Google+ is more like Facebook. It’s got comment threading, video chat,  inline media, and limited sharing.

But for me, the core defining design of Google+ is the one-way following. I can follow anyone on Google+. They may not follow back (er…put me in a circle), but I can see their public posts. This one-way following is what makes the experience more like Twitter for me. Knowing your public posts are out there for anyone to find and read is both boon and caution. For instance, I’ll post pics of my kids on Facebook, because I know who can see those pics – the people I’ve connected with. I don’t tend to post their pics on Twitter. Call me an old fashioned protective parent.

That’s my initial impression. Now as Google+ circles gain ground in terms of usage, they will become the Facebook equivalent of two-way following. Things like sharing and +mentions are issues that are hazy to me right now. Can someone reshare my “circle-only” post to others outside my circle? Do I have to turn off reshare every time? Does +mentioning someone outside my circle make them aware of the post?

Google has created quite a powerful platform here. While most features are not new innovations per se, Google+ benefits from the experience of both Twitter and Facebook. They’re off to a good start.

I’m @bhc3 on Twitter.

When Should Management Push Enterprise 2.0 Adoption?

After the Boston edition of the Enterprise 2.0 Conference, IBM’s Rawn Shah wrote a great follow-up post outlining ten observations from the event. A couple points that I found myself agreeing with wholeheartedly were:

Adoption is about transforming human behaviors at work – More folks are starting to recognize that it is not trivial to bring communities and other social environments to life.

‘Let’s get beyond “adoption”’ – This was another sentiment I heard several times, but I attribute it to short-attention span. The general statement was ‘adoption’ was last-year’s thing, and we needed a new ‘thing’.

[tweetmeme source=”bhc3″]

The underlying philosophy of his post contrasts with that of Paula Thornton, who finds talk of driving adoption to be antithetical to the true nature of Enterprise 2.0. As she described in a post from several months ago:

If you have to “drive adoption” you’ve failed at 2.0 design and implementation. The fundamentals of 2.0 are based on design that is organic — meets the individual where they are and adapts based on feedback — it emerges. The ‘adoption’ comes from rigorous ‘adaptation’ — it continuously morphs based on involvement from the ‘masses’. If done right, you can’t keep them away…because you’ve brought the scratch for their itch.

While I empathize with her design-driven perspective, I personally find there to be more to people’s adoption patterns. Sometimes the superior design does not win. Existing network effects may prove a high barrier to adoption of something new. Embedded history makes the current approach valuable. And other reasons intrude.

In considering adoption, we have the push strategy (by management), and the pull strategy (viral, organically spreads). Both are viable approaches. The key factor is to determine when each needs to be employed.

A Decision Framework for Pushing Enterprise 2.0 Adoption

The graphic below outlines a basis for determining when Enterprise 2.0 adoption must be pushed, and when to let adoption be pulled:

The two key factors in the framework are user-centric and organization-centric.

The X-axis highlights a key reality. If a current approach/technology is working well enough for users, there is an inertia to making a switch of any kind. This principle is nicely captured in the “9x problem”, an explanation by Harvard professor John Gourville that was highlighted by Andrew McAfee. The 9x problem is this:

Users will overvalue existing products/solutions by 3 times, and undervalue the benefits of a new products/solutions by 3 times.

We’re for the most part risk-averse (e.g. technology adoption lifecycle is back-end loaded), and giving up existing ways presents a level of uncertainty. It’s the devil we know versus the devil we don’t. We place a value on the certainty of current methods, even if flawed.

The other part of the 9x equation is that users will place an uncertainty discount against new products/solutions enumerated benefits. Yes, it’s true. We don’t always buy everything we’re told.

The Y-axis speaks to the value of E2.0 to organizations. Certainly there will be use cases that can drive high value for the organization. And just as certainly, there will be those use cases that contribute little to organizational value.

Let’s run through the different approaches mapped on the graph, clockwise from top right.

Requires a Top-Down Push

Situation:

  • Existing ways are ‘good enough’ for employees
  • Executives see great potential for value from adoption

What might this be? Imagine management has seen too many examples of people missing key information and connecting the dots well with others are working on. An enlightened C-level type knows there is an opportunity to pick it up a level.

So some sort of social software – e.g. wiki, collaboration groups, etc. – is selected to make this a reality. But guess what? People keep emailing to one another and saving docs to the LAN.

Why? Because those are the tools they know, there is no learning curve and everyone operates on a shared set of processes and assumptions. Things work “as is”.

This is where management needs to wield its power, and come up with ways to influence employees to alter their entrenched behaviors that work “good enough”.

Mix a Push-Pull Strategy

Situation:

  • Existing ways are actually not “good enough”
  • There is high value in large-scale adoption

This is the home run of initiatives. Solves a “what’s in it for me” need of individuals, while also presenting a great chance to advance the value of the organization.

An innovation platform is a good example here. A place for individuals to express those ideas that fire them up or just plain solve annoyances. Which get lost in the email inbox.

But the opportunity for new ideas that deliver to the bottom line gets management’s attention.

Pull works here, as word spreads about the initiative. But management has an interest in making sure everyone is aware of the initiative, as soon as possible. Push tactics are good supplements.

Let It Grow Organically

Situation:

  • Existing ways are actually not “good enough”
  • There is low value in large-scale adoption

This is a tough one. Clearly the “Enterprise 2.0 way” can solve a problem for employees, but its adoption cannot be seen to lead to high impact on company value. An example here? Hmm…tough one. Enterprise bookmarking might be one area. Solves the, “how do I find things?” conundrum, for me personally and for others. But hard to see just how it will increase firm value. At least on a standalone basis.

Best to let these initiatives grow of their own accord. Let their value emerge, often with stories.

Don’t Waste Your Time

Situation:

  • Existing ways are ‘good enough’ for employees
  • There is low value in large-scale adoption

Suffice to say, this one should be killed before it ever starts.

[tweetmeme source=”bhc3″]

My Ten Favorite Tweets – Week Ending 043010

From the home office on the Gulf Coast, where I just have to note that you never hear about “sunshine spills” or “wind slicks”.

#1: RT @mattgaston If foursquare gets it right, they could go big. Very big! NYTimes: Linking Customer Loyalty With Social Networking http://nyti.ms/cfzclt

#2: RT @TechCrunch The Huffington Post Starts To Give Out Badges To Readers http://tcrn.ch/bQopLB > Just getting started…

#3: WSJ has its own Foursquare badges http://bit.ly/a6EjmX by @mathewi > WSJ also provides news items for locations

#4: RT @tacanderson Cool webcast today by HP: An economist’s view of crowdsourcing http://j.mp/czYrSY

#5: Getting the Most from Your Crowdsourcing Initiative (via Spigit blog) http://bit.ly/cugk0z #innovation

#6: RT @jacobm spigit announces its innovation summit, should be a great one http://bit.ly/csHfNE cc @bhc3

#7: 42: Why innovation is a hard sell http://bit.ly/b83pWs by @deb_lavoy > #Innovation is problem-solving, not ideation

#8: RT @Renee_Innosight Yes! RT @MARTYneumeier: The secret to collaboration is finding a rhythm that alternates between team creativity and individual creativity.

#9: NBC’s Parenthood cracks me up. Love it. Until it inevitably jumps the shark somewhere along the line with a “very special” Parenthood.

#10: About to start Stuart Hall Miller’s Mile with my son — at Warming Hut Park Store & Cafe http://gowal.la/c/E4ah

My Ten Favorite Tweets – Week Ending 031210

From the home office at SXSW in Austin, where I’m not…

#1: Is Collaboration Enough for Knowledge Management? http://bit.ly/bXdNhj by @deb_lavoy #e20 #km

#2: What Enterprise 2.0 vendors can learn from FourSquare http://tinyurl.com/y9bsxc6 by @markfidelman

#3: RT @Irregulars Wikipedia’s Decline and the 7 Types of Human Motivation http://bit.ly/atzPLC

#4: White House expands Gov 2.0 with landmark crowdsourcing directive (via Spigit blog) http://bit.ly/auo6FK #gov20 #innovation

#5: “Contests are increasingly being used as a tool to solve society’s most entrenched problems” http://bit.ly/9KFJmy #crowdsourcing

#6: RT @VentureBeat Spigit offers social media platform for company contests http://ow.ly/1q0m44 #crowdsourcing

#7: RT @elldir Woops! Too long ago I told @bhc3 that I would post how I think about different dimensions of innovation. http://bit.ly/dcNd7s

#8: Five inter-related innovation problems that an organizational structure should address – Scott Anthony HBR #innovation http://post.ly/SOB2

#9: Reading @bokeen‘s write-up of his chatroulette experience. Damn funny, and pretty much what I’d expect. http://bit.ly/9Wnd20

#10: RT @anildash I’m surprised none of you dorks camped outside of your own house last night, then ran back in to order an iPad

Follow

Get every new post delivered to your Inbox.

Join 759 other followers