Why crowdsourcing works
March 4, 2014 12 Comments
Crowdsourcing is a method of solving problems through the distributed contributions of multiple people. It’s used to address tough problems that happen everyday. Ideas for new opportunities. Ways to solve problems. Uncovering an existing approach that addresses your need.
Time and again, crowdsourcing has been used successfully to solve challenges. But…why does it work? What’s the magic? What gives it an advantage over talking with your pals at work, or doing some brainstorming on your own? In a word: diversity. Cognitive diversity. Specifically these two principles:
- Diverse inputs drive superior solutions
- Cognitive diversity requires spanning gaps in social networks
These two principles work in tandem to deliver results.
When trying to solve a challenge, what is the probability that any one person will have the best solution for it? It’s a simple mathematical reality: the odds of any single person providing the top answer are low.
How do we get around this? Partly by more participants; increased shots on goal. But even more important is diversity of thinking. People contributing based on their diverse cognitive toolkits:
As described by University of Michigan Professor Scott Page in The Difference, our cognitive toolkits consist of: different knowledge, perspectives and heuristics (problem-solving methods). Tapping into people’s cognitive toolkits brings fresh perspectives and novel approaches to solving a challenge. Indeed, a research study found that the probability of solving tough scientific challenges is three times higher if a person’s field of expertise is seven degrees outside the domain of the problem.
In another study, researchers analyzed the results of an online protein-folding game, Foldit. Proteins fold themselves, but no one understands how they do so. This is particularly true of experts in the field of biochemistry. So the online game allows users to simulate it, with an eye towards better understanding the ways the proteins fold themselves. As reported by Andrew McAfee, the top players of Foldit were better than both computers and experts in the field at understanding the folding sequence. The surprising finding? None had taken chemistry beyond a high school course. It turns out spatial skills are more important to solve the problem than deep domain knowledge of proteins.
Those two examples provide real-world proof for the models and solution-seeking benefits of cognitive diversity described by Professor Page.
Problem solving can be thought of as building a solutions landscape, planted with different ideas. Each person achieves their local optimum, submitting the best idea they can for a given challenge based on their cognitive assets.
But here’s the rub: any one person’s idea is unlikely to be the best one that could be uncovered. This makes sense as both a probabilistic outcome, and based on our own experiences. However in aggregate, some ideas will stand out clearly from the rest. Cognitive diversity is the fertile ground where these best ideas will sprout.
In addition to being a source of novel ideas, cognitive diversity is incredibly valuable as feedback on others’ ideas. Ideas are improved as people contribute their distinct points of view. The initial idea is the seedling, and feedback provides the nutrients that allow it to grow.
Cognitive diversity clearly has a significant positive effect on problem-solving. Generally when something has proven value to outcomes, companies adopt it as a key operating principle. Yet getting this diversity has not proven to be as easy and common as one might expect.
Because it’s dependent on human behavior. Left to our own devices, we tend to turn to our close connections for advice and feedback. These strong ties are the core of our day-in, day-out interactions.
But this natural human tendency to turn to our strong ties is why companies are challenged to leverage their cognitive diversity. University of Chicago Professor Ron Burt describes the issue as one of structural holes between nodes in a corporate social network in his paper, Structural Holes and Good Ideas (pdf). A structural hole is a gap between different groups in the organization. Information does not flow across structural holes.
In and of themselves, structural holes are not the problem. Rather, the issue is that when people operate primarily within their own node, their information sources are redundant. Over time, the people in the node know the same facts, develop the same assumptions and optimize to work together in harmony. Sort of like a silo of social ties.
The impact of this is a severe curtailment of fresh thinking, which impacts the quality of ideas. Professor Burt found empirical evidence for this in a study of Raytheon’s Supply Chain Group. 673 employees were characterized by their social network connections, plotting them on a spectrum from insular to diverse. These employees then provided one idea to improve supply chain management at Raytheon. Their ideas were then assessed by two senior executives.
The results? Employees with more diverse social connections provided higher quality ideas. To the right is a graph of the rated ideas, with a curve based on the average idea ratings versus the submitter’s level of network diversity. The curve shows that with each increase in the diversity of a person’s connections, the higher the value of their idea.
Employees with access to diverse sources of information provided better ideas. Their access to nonredundant information allowed them to generate more novel, higher potential ideas. Inside organizations, there are employees who excel at making diverse connections across the organization. These people are the ones who will provide better ideas. They are brokers across the structural holes in social networks.
Professor Burt provides the key insight about these brokers:
People connected to groups beyond their own can expect to ﬁnd themselves delivering valuable ideas, seeming to be gifted with creativity. This is not creativity born of genius; it is creativity as an import-export business. An idea mundane in one group can be a valuable insight in another.
An “import-export business”. Consider that for a moment. It’s a metaphor that well describes the key value of the brokers. They are exchange mechanisms for cognitive diversity. They are incredibly valuable to moving things forward inside organizations. But are organizations overly dependent on these super-connectors? Yes. Companies are leaving millions on the table by not enabling a more scalable, comprehensive and efficient means for exchanges of cognitive diversity.
Would if we could systematize what the most connected employees do?
Crowdsourcing doesn’t eliminate the need for the super-connectors. They play a number of valuable roles inside organizations. But by crowdsourcing to solve problems, companies gain the following:
- Deeper reach into the cognitive assets of all employees
- Avoiding the strong ties trap of problem-solving
- Faster surfacing of the best insights
- Neutralize the biases that the super-connectors naturally have
As you consider ways to improve your decision-making and to foster greater cross-organizational collaboration, make crowdsourcing a key element of your strategic approach.