Advertisements

16 metrics for tracking Collaborative Innovation performance

In a recent PwC survey, 61% of CEOs said innovation was a key priority for their company (pdf). The only surprising result there is that it wasn’t 100%. Innovation efforts come in a variety of forms: innovation and design labs, jobs-to-be-done analysis, corporate venturing, distributed employee experiments, open innovation, TRIZ, etc.

In this post, I want to focus on another type of innovation initiative: Collaborative Innovation. A good way to think about Collaborative Innovation is that it integrates social and crowdsourcing principles:

Collaborative innovation - social and crowdsourcing

 

A definition I use for this approach:

Collaborative Innovation is defined as activities organizations use to improve their rates of innovation and problem solving by more effectively leveraging the diverse ideas and insights of employees, customers and partners.

While it seems straightforward, Collaborative Innovation is actually a fairly sophisticated activity. People with a cursory understanding say all you need to do is: (i) stand up an ideas portal; (ii) let people post ideas; (iii) collect votes on those ideas; and (iv) pick the winners.

Unfortunately, that’s just plain wrong. I’ve seen too many cases where organizations launch idea portals, only to see them die off six months later. The practice of Collaborative Innovation is a rich realm, with solid results for those who apply it thoughtfully.

This post is a look at several key metrics that corporate innovation teams should focus on as they lead Collaborative Innovation programs. The metrics are segmented by the different phases of innovation:

  1. Sourcing
  2. Decisioning
  3. Acting

The metrics below rest on two key assumptions: use of an innovation management software platform; use of campaigns to target innovation efforts.

Sourcing

Sourcing refers to the generation of ideas, as well as eliciting others’ insights about an idea.

Phase objectives

  • Distinct, differentiated ideas
  • Ideas matching needs of customers (incl. internal customers)
  • Ideas matching the innovation appetite of the organization
  • Capturing the cognitive diversity of participants
  • Growing the culture of innovation

Metrics

Metric Description Why
Trend in unique logins Measure the ratio of logins/invited over time for multiple campaigns. Want to see a rise over time until reaching a steady state (~60%).
  • No logins, no ideas
  • Confirm that the credibility of program increasing
  • Demonstrate better targeting of relevant innovation topics
Trend in multiple logins Determine the number of people who log in to each campaign 3 times or more. Divide these multi-login people by the total number of people logging in to each campaign. Look for increasing ratios over time.
  • Returning to a campaign repeatedly is a measure of engagement
  • More repeat logins increases opportunities for collaboration
Ratio of ideators to unique logins Divide the number of people who post at least one idea by the number of unique logins. Want to see a rise over time until reaching a steady state (10 – 15%).
  • Ensure those with valuable ideas are being invited
  • Track whether the campaign questions are accessible to those invited
  • Confirm credibility of the program is increasing
Average number comments per idea Divide the number of comments by the number of ideas, per campaign. Target an average of 2 comments per idea.
  • Ideas are the start, but need others’ insights to evolve and grow
  • Track the collaboration culture of the organization, and of specific org units
  • Ensure participants understand that more than ideas are desired
Average number of replies per comment Divide the number of comment replies by the number of comments. Target an average of 0.5 replies per comment.
  • Innovation dialogues are healthy for both ideas and the organization’s innovation culture
  • Sharing of insights among employees is a second level objective, and this helps track that
Average number of votes per idea Divide the number of votes by the number of ideas, per campaign. Target an average of 3 votes per idea.
  • Participants can help identify ideas with greater potential
  • Ensure the voice of the community is captured, to complement the views of experts and campaign sponsors
Unique org units | departments | locations contributing Count the number of different org units, departments and/or locations with at least one person posting an idea, posting a comment or voting. This count needs to be considered against the number of org units, departments or locations invited.
  • Cognitive diversity is a key driver of value
  • Seek inputs from people who normally aren’t working closely together, to ensure different perspectives are brought into the campaign

Decisioning

Decisioning refers to identifying which ideas move forward for next steps. This phase is the bridge between getting a lot of different ideas, and determining which ones will be acted on.

Phase objectives

  • Identify ideas presenting enough possibility to warrant further review
  • Acknowledge value of community’s perspective
  • Timely assessments of ideas

Metrics

Metric Description Why
Ratio of ideas selected for further review Some number of ideas submitted for each campaign will be selected for the next round of review. Calculate the ratio of selected ideas to total ideas submitted. Watch how this ratio changes over time.
  • Track whether campaigns are generating the level of possibilities expected
  • Look for cases of being overly pessimistic on ideas’ possibilities (too-low or declining ratio over time)
Ratio of top 5 voted/commented ideas selected for further review Of the ideas that were the top 5 for either votes or number of unique commenters, track how many were selected for further review.
  • When the community is invited to comment and vote, they have a natural expectation that their interactions will be part of the decision calculus
  • Failure to regularly consider what the community coalesces around will reduce enthusiasm to participate
Percentage of initially reviewed ideas sent back for iteration & information Of the ideas that were selected for further assessment, track the number where the idea submitter (and team) are asked to iterate the idea and/or provide more information.
  • Ideas rarely have enough “meat” on their initial sourcing, and benefit from further development
  • Watch out for too conservative a mindset by those making decisions on ideas; are they too quick to say ‘no’ without seeking more information?
Time to complete decisions Measure the time between selection of ideas for further review and selection of ideas to move forward into the Acting phase. The time will vary by the level of risk attendant to a campaign.
  • Participants will have a reasonable expectation that promising ideas move forward; delays signal a lack of commitment
  • From the world of finance, the time value of money argues for moving sooner rather than later on ideas with value
Ratio of reviewed ideas that advance to Acting phase Divide the number of ideas selected to move into the Acting phase by the number of ideas selected for review. Watch this ratio over time.
  • Moving ideas forward to action is core to developing an innovation culture; ensure this key step is occurring as expected
  • Too-low or declining ratios indicate a breakdown in the innovation process
  • Ideas that move forward are critical for ensuring the credibility of the innovation effort

Acting

Acting refers to the activities to prove out an idea, develop it and prepare it for full launch. Or to learn why an idea won’t be feasible, ultimately.

Phase objectives

  • Develop deeper understanding for whether the idea passes the three jobs-to-be-done tests that determine market adoption
  • Optimize features that best deliver on the outcomes that the idea’s targeted beneficiaries have
  • Maximize the probability of success by eliminating ideas that just aren’t working

Metrics

Metric Description Why
Average number of experiments per idea Tally the total number of experiments for a “class” of selected ideas for Acting phase, calculate the average per idea.
  • Because of the inherent risk of trying something new, many ideas need different looks
  • Learning mentality to understand an idea’s strengths and weaknesses
  • Valuable for establishing a strong innovation culture for the organization
Time to make final decision on selected ideas Track the amount of time between the decision to put an idea into the Acting phase, and the decision whether to pursue the idea at scale.
  • While this process shouldn’t be rushed, it should be treated with appropriate diligence
  • Participants will expect final decisions; failure to do so undermines the program credibility
Ratio of ideas selected for full launch Divide the number of ideas selected for full launch by the number of ideas selected for the Acting phase. Watch how this ratio tracks over time.
  • The determinant of success for this phase is the number of ideas that make to full launch
  • Ideas in this phase passed muster during the prior Deciding phase; the percentage that make it to full launch should be high
Projected and realized value of ideas that have been moved to full launch Aggregate projected and realized value of the ideas that will be or have been put into full launch.
  • The bottom line rationale for the innovation program
  • Critical for establishing credibility of the program with senior executives

The above list is solid foundation of metrics to track for your Collaborative Innovation program. It’s not exhaustive. And there are likely elements for each phase that will vary for each organization.

But these are good for watching how your program is tracking. Behind each metric, there are techniques to enhance outcomes. The key is knowing where to look.

I’m @bhc3 on Twitter.

Advertisements