The Challenge Dividend: Netflix Prize - Getting Closer to a Winner

In October 2006, I wrote about Netflix's innovative use of a challenge to get hundreds or thousands of smart people to help it improve its service. The company launched "The Netflix Prize" - an awards of $1 million for the first person or team to improve the quality of member movie predictions by 10%.

Just a little over a year since the contest began, Wired magazine recently updated its readers on how the contest is going so far. A few highlights:

Over 25,000 people and teams have entered the contest to date.
The best score is an 8.82% improvement in recommendations.
Netflix has seen an improvement in customer retention alongside an improvement in its recommendation algorithm. The company is not sure the two are directly related, but believes better recommendations are at least part of the story.

Much of the Wired story focuses on the rapid rose of Gavin Potter into the top ranks of the Netflix challengers. Unlike most of the leading teams, which are math experts in universities, Potter is a retired operations consultant working on his own at home under the screen name "Just a Guy in a Garage." A recent newcomer to the top 10, Potter has used theories of human psychology to build his prediction algorithm. It is a strategy that the math majors have under-utilized to date, but now they, too are building off Potter's approach.

While it is still unlikely that Potter will win - his approach reminds us that many of the biggest advancements in science come from outsiders who challenge the status quo thinking. Einstein was a patent clerk who barely got published. Genetics was advanced by Gregor Mendel, a monk working alone in Austria. And the human genetic code, in turn, was cracked thanks to the revolutionary approach of outsider Craig Venter. Outsiders often win because they are not tied to the accepted norms of the current system, and they are attracted by the chance for quick fame and fortune.

Thanks to establishing challenges and opportunities, companies like Netflix and their customers benefit from thousands of hours of brainpower from the smartest men and women in the world.