Category Archives: trends

computer science, economics, people, politics, prediction markets, probability, research, science, technology, trends, yahoo

A toast to the number 303: A redemptive election night for science, and The Signal

3 Comments

The night of February 15, 2012, was an uncomfortable one for me. Not a natural talker, I was out of my element at a press dinner organized by Yahoo! with journalists from the New York Times, Fast Company, MIT Tech Review, Forbes, SF Chronicle, WIRED, Reuters, and several more [1]. Even worse, the reporters kept leading with, “wow, this must a big night for you, huh? You just called the election.”

We were there to promote The Signal, a partnership between Yahoo! Research and Yahoo! News to put a quantitative lens on the election and beyond. The Signal was our data-driven antidote to two media extremes: the pundits who commit to statements without evidence; and some journalists who, in the name of balance, commit to nothing. As MIT Tech Review billed it, The Signal would be the “mother of all political prediction engines”. We like to joke that that quote undersold us: our aim was to be the mother of all prediction engines, period. The Signal was a broad project with many moving parts, featuring predictions, social media analysis, infographics, interactives, polls, and games. Led by David “Force-of-Nature” Rothschild, myself, and Chris Wilson, the full cast included over 30 researchers, engineers, and news editors [2]. We confirmed quickly that there’s a clear thirst for numeracy in news reporting: The Signal grew in 4 months to 2 million unique users per month [3].

On that night, though, the journalists kept coming back to the Yahoo! PR hook that brought them in the door: our insanely early election “call”. At that time in February, Romney hadn’t even been nominated.

No, we didn’t call the election, we predicted the election. That may sound like the same thing but, in scientific terms, there is a world of difference. We estimated the most likely outcome – Obama would win 303 Electoral College votes, more than enough to return him to the White House — and assigned a probability to it. Of less than one. Implying a probability of more than zero of being wrong. But that nuance is hard to explain to journalists and the public, and not nearly as exciting.

Although most of our predictions were based on markets and polls, the “303” prediction was not: it was a statistical model trained on historical data of past elections, authored by economists Patrick Hummel and David Rothschild. It doesn’t even care about the identities of the candidates.

I have to give Yahoo! enormous credit. It took a lot of guts to put faith in some number-crunching eggheads in their Research division and go to press with their conclusions. On February 16, Yahoo! went further. They put the 303 prediction front and center, literally, as an “Exclusive” banner item on Yahoo.com, a place that 300 million people call home every month.

The Signal 303 prediction "Exclusive" top banner item on Yahoo.com 2012-02-16

The firestorm was immediate and monstrous. Nearly a million people read the article and almost 40,000 left comments. Writing for Yahoo! News, I had grown used to the barrage of comments and emails, some comic, irrelevant, or snarky; others hateful or alert-the-FBI scary. But nothing could prepare us for that day. Responses ranged from skeptical to utterly outraged, mostly from people who read the headline or reactions but not the article itself. How dare Yahoo! call the election this far out?! (We didn’t.) Yahoo! is a mouthpiece for Obama! (The model is transparent and published: take it for what it’s worth.) Even Yahoo! News editor Chris Suellentrop grew uncomfortable, especially with the spin from Homepage (“Has Obama won?”) and PR (see “call” versus “predict”), keeping a tighter rein on us from then on. Plenty of other outlets “got it” and reported on it for what it was – a prediction with a solid scientific basis, and a margin for error.

This morning, with Florida still undecided, Obama had secured exactly 303 Electoral College votes.

New York Times 2012 election results Big Board 2011-11-07

Just today Obama wrapped up Florida too, giving him 29 more EVs than we predicted. Still, Florida was the closest vote in the nation, and for all 50 other entities — 49 states plus Washington D.C. — we predicted the correct outcome back in February. The model was not 100% confident about every state of course, formally expecting to get 6.8 wrong, and rating Florida the most likely state to flip from red to blue. The Hummel-Rothschild model, based only on a handful of variables like approval rating and second-quarter economic trends, completely ignored everything else of note, including money, debates, bail outs, binders, third-quarter numbers, and more than 47% of all surreptitious recordings. Yet it came within 74,000 votes of sweeping the board. Think about that the next time you hear an “obvious” explanation for why Obama won (his data was biggi-er!) or why Romney failed (too much fundraising!).

Kudos to Nate Silver, Simon Jackman, Drew Linzer, and Sam Wang for predicting all 51 states correctly on election eve.

As Felix Salmon said, “The dominant narrative, the day after the presidential election, is the triumph of the quants.” Mashable’s Chris Taylor remarked, “here is the absolute, undoubted winner of this election: Nate Silver and his running mate, big data.” ReadWrite declared, “This is about the triumph of machines and software over gut instinct. The age of voodoo is over.” The new news quants “bring their own data” and represent a refreshing trend in media toward accountability at least, if not total objectivity, away from rhetoric and anecdote. We need more people like them. Whether you agree or not, their kind — our kind — will proliferate.

Congrats to David, Patrick, Chris, Yahoo! News, and the entire Signal team for going out on a limb, taking significant heat for it, and correctly predicting 50 out of 51 states and an Obama victory nearly nine months prior to the election.

Footnotes

[1] Here was the day-before guest list for the February 15 Yahoo! press dinner, though one or two didn’t make it:
-  New York Times, John Markoff
-  New York Times, David Corcoran
-  Fast Company, EB Boyd
-  Forbes, Tomio Geron
-  MIT Tech Review, Tom Simonite
-  New Scientist, Jim Giles
-  Scobleizer, Robert Scoble
-  WIRED, Cade Metz
-  Bloomberg/BusinessWeek, Doug MacMillan
-  Reuters, Alexei Oreskovic
-  San Francisco Chronicle, James Temple

[2] The extended Signal cast included Kim Farrell, Kim Capps-Tanaka, Sebastien Lahaie, Miro Dudik, Patrick Hummel, Alex Jaimes, Ingemar Weber, Ana-Maria Popescu, Peter Mika, Rob Barrett, Thomas Kelly, Chris Suellentrop, Hillary Frey, EJ Lao, Steve Enders, Grant Wong, Paula McMahon, Shirish Anand, Laura Davis, Mridul Muralidharan, Navneet Nair, Arun Kumar, Shrikant Naidu, and Sudar Muthu.

[3] Although I continue to be amazed at how greener the grass is at Microsoft compared to Yahoo!, my one significant regret is not being able to see The Signal project through to its natural conclusion. Although The Signal blog was by no means the sole product of the project, it was certainly the hub. In the end, I wrote 22 articles and David Rothschild at least three times that many.

artificial intelligence, computer science, economics, people, research, technology, trends

Congratulations Pete Wurman and Kiva Systems, a bellwether of the automated economy

4 Comments

Congratulations to my academic sibling, friend, and Detroit Red Wings fan Pete Wurman, whose company Kiva Systems just became Amazon’s second largest acquisition ever.

In short, Kiva Systems designs, builds, and operates intelligent autonomous robots to pick and stow products in giant distribution centers for companies like Toys R Us, Walgreens, and Zappos. (The latter is an Amazon subsidiary.) The best way to understand Kiva Systems is to watch their robots in action: an amazing sight to see. Here is a clip from IEEE Spectrum:

In 2003, I remember sitting in the back seat of a car with Pete, him excitedly demo-ing the concept to me via an animated simulation on his laptop, little dots representing robots weaving in and out of each on the screen. (Pete’s laptop was a mac. In grad school, Pete was every bit the Apple fan I was and more. He and I programmed HyperCard and Newton together. Pete advocated for simplicity in design before it was cool. When I briefly switched to Windows, he never wavered.)

By 2006, the robots were real. Pete took me and our shared academic parent, Mike Wellman (who I believe also played an early role in the company), on a tour. Dots on a laptop had become squat orange robots receiving orders, fetching products, avoiding each other, seeking power, and otherwise navigating around a complex environment with computational minds of their own. The designs were inspired: for example, to lift a box, the robot spun underneath it to extend a corkscrew so that the product wouldn’t get jarred. They even added noise in the robots’ paths, so their wheels wouldn’t wear grooves in the floor (call it a floorsaver algorithm).

By coincidence, a few weeks ago, I was speaking to someone from Amazon who works on optimizing the way people (ha!) retrieve, store, and pack items in their distribution centers and I mentioned Pete’s company. He said “until that happens” he would focus on optimizing their current systems. Little did we (or at least I) know how quickly “until” would come.

Kiva Systems isn’t just an incredibly cool company run by amazing people. It’s a harbinger of things to come as the world moves inexorably toward an Automated Economy.

By the way, if you’re worried that robots will take jobs away from people, don’t. The world is a better place with mechanical devices doing mechanical tasks, leaving people to do more interesting and creative things, for example turning crazy ideas into companies. Remember that the purpose of jobs is to produce valuable things and improve the world. Despite political rhetoric, jobs are not an end to themselves. Otherwise, we should all be happy digging ditches and filling them back up, or pumping gas for people who would rather do it themselves. Think about where society should go in fifty or a hundred years when automation can handle more and more tasks. It would be a real shame if at that time people were still “working for a living” in jobs they don’t enjoy simply for the sake of keeping them occupied.

commentary, economics, prediction markets, startups, trends

It’s Arab Spring, but is it Prediction Market Winter?

16 Comments

Is the growing prediction market industry graveyard an omen?

It’s hard to ignore the accumulating bodies, including, may they rest in peace, PPX, Hubdub, Protrade, Tradesports, Newsfutures, Hedgestreet, Yoonew, TheTicketReserve, FirstDibz, BettorFan, ipreo, Tech Buzz Game, The WSX, Storage Markets, FTPredict, real HSX, BizPredict [1], CasualObserver.net, Cenimar, Alexadex [2], Askmarkets, Truth Markets, BetBubble, Betocracy, CrowdIQ, Media Mammon, Owise, RIMDEX, Trendio [3], TwoCrowds, BBC celebdaq/sportdaq, Betfair Predicts, chrisfmasse.com [4], and more.

Is this churn rate normal for startups in general, even healthy? Is it a sign of PM’s place in the trough of the hype cycle? Is the current climate an opportunity for those left standing or someone new? Or does it simply suggest that prediction market proponents like me have lost?

A number of media-PM partnerships which on their face seem perfectly natural are history: USA Today+Newsfutures, Popular Science+HSX, Business 2.0+ConsensusPoint, Financial Times+Intrade, Techcrunch+Askmarkets [5], ABC7+Inkling [6], and CFO Magazine+Crowdcast.

At least two former PM companies found success only after switching gears: Protrade became Citizen Sports before being acquired by Yahoo! and Nigel euthanized Hubdub to focus on FanDuel. Cocision, launched just last fall, has already abandoned its PM roots in favor of breezy Q&A and voting.

Usable Marketeer Alex Kirtland nails exactly why all the “predict Wall Street” games may be fun but aren’t likely to be predictive. Research papers, including my own, report that the accuracy advantage of prediction markets, while real, may often be small compared to statistical models or polls.

Intrade, one of the most cited and well studied PMs, is trying hard with a radical remake that looks great and a new fee structure that’s likely to improve low-probability predictions. I don’t have any inside knowledge but the company and the exchange don’t seem especially strong; I even spotted some bugs in their exchange rules. The venerable Iowa Electronic Market and Foresight Exchange that, together with Robin, started it all, look, well, venerable. Betfair is still a powerhouse and soared in its IPO just last fall, but is perhaps showing signs of age as personnel turn over and the product remains decidedly 1.0.

A few startups like Crowdcast, MediaPredict, smarkets, betable, socialico/PremierX, and InklingMarkets are nimble and promising, but none have hit home runs yet. The SimExchange is well designed and chugging along. Bet2Give [7] and CentSports are both fascinating concepts and still alive, two of the most intriguing real-money markets. Others like ExtZy, RealityMarkets and PublicGyan are hanging on. New entrants like Prediculous, Predictalot, Predictopus, 4cast, beansight, I Called It, IBET, Prediction Book, HuffPo’s Predict the News, Slate’s Lean/Lock, Ultrinsic, Knew The News, Cantor Gaming’s Oracle [8], and the MNI Forecast Competition (Lumenogic) are still coming up, though at an admittedly slower pace than four years ago.

Update 2011/5/10: Crowdpark, a German company with an office in San Francisco, launched in English last December with a web game and an impressive, well-designed Facebook game that’s already attracted 500,000 monthly active users, the 11th fastest growing Facebook app in April. They have an interesting “patent pending” automated market maker that I can’t find any details about (yet).

One PM mailing list is of questionable transparency and another is often silent. The Prediction Market Industry Association is inactive.

The final post on Newfutures Blog in 2009 declares that “resistance is futile”. But is it the world’s resistance of PMs, or PMs resistance of irrelevance, that is futile?

Despite the negative tone of this post, I believe it’s the former. The prediction market spring will come. Here’s why. Prediction markets offer:

  1. Accountability
  2. Meritocracy
  3. A marketplace to reward information release
  4. Real-time updates
  5. Accuracy
  6. Increasing ease of use, as the technology matures and diffuses
  7. Self funding

No other prediction technology offers the same. There’s a great opportunity here for the companies that have squirreled away enough nuts to survive the winter.

P.S. Also read Paul Hewitt’s Prediction Market Prospects 2010.

Footnotes:

[1] In 2006, the teaser prediction for BizPredict was “Do you know when MySpace’s traffic will surpass Yahoo’s?”.

[2] Techcrunch declared Alexadex “the web 2.0 stock market”, back when Techcrunch encouraged Diggs

[3] I like Trendio’s post-mortem:

..Trendio rapidly became popular and attracted massive traffic from all over the world, as well as attention from major newspapers, TV-channels and blogs. To develop Trendio as a large-scale web property and an income-generating business would however have required to dedicate time and resources that I wasn’t able to provide.

I still believe there is a massive potential for prediction markets, both as games and for their predictive power…

[4] A truly sad loss, and not just because of the 2005 awards. Someone should archive the archive to be sure this gem, as information-rich as it was verbose and disorganized, survives. Hang in there Midas Oracle!

[5] Ironically, upon launch of Askmarkets in 2008 Techcrunch asked “who’s going to the deadpool?”

[6] Technically not dead, but seems neglected.

[7] We independently considered an idea similar to bet2give at Yahoo! in 2007 but never pursued it.

[8] Cantor Gaming’s odd-setting mechanism seems effectively like an automated market maker with intelligent prior.

commentary, computer science, gambling, people, prediction markets, trends

Where is the betting market for P=NP when you need it?

12 Comments

HP research scientist Vinay Deolalikar has constructed the most credible proof yet of the most important open question in computer science. If his proof is validated (and there are extremely confident skeptics as you’ll see) he proved that P≠NP, or loosely speaking that some of the most widespread computational problems — everything from finding a good layout of circuits on a chip to solving Sudoku puzzles to computing LMSR prices in a combinatorial market — cannot be solved efficiently. Most computer scientists believe that P≠NP, but after decades of some of the smartest people in the world trying, and despite the promise of worldwide accolades and a cool $1 million from the Clay Mathematics Institute, no one has been able to prove it, until possibly now.

Scott Aaronson is a skeptic, to say the least. He made an amazing public bet to demonstrate his confidence. He pledged that if Deolalikar wins the $1 million prize, Aaronson will top it off with $200,000 of own money. Even more amazing: Aaronson made the bet without even reading the proof. [Update: I should have said “without reading the proof in detail”: see comments] (Perhaps more amazing still: a PC World journalist characterized Aaronson’s stance as “noncommittal” without a drip of sarcasm.) [Hat tip to Dan Reeves.]

As Aaronson explains:

The point is this: I really, really doubt that Deolalikar’s proof will stand. And while I haven’t studied his long, interesting paper and pinpointed the irreparable flaw… I have a way of stating my prediction that no reasonable person could hold against me: I’ve literally bet my house on it.

Aaronson is effectively offering infinite odds [Update: actually more like 2000/1 odds: see comments] that the question “P=NP?” will not be resolved in the near future. Kevin McCurley and Ron Fagin made a different (conditional) bet: Fagin offered 5/1 odds (at much lower stakes) that if the question is resolved in 2010, the answer will be P≠NP. Bill Gasarch says that he, like Aaronson, would bet that the proof is wrong… if only he were a betting man. Richard Lipton recounts a discussion about the odds of P=NP with Ken Steiglitz.

But beyond a few one-off bets and declarations, where is the central market where I can bet on P=NP? I don’t even necessarily want in on the action, I just want the odds. (Really!)

My first thought was the Foresight Exchange. It does list one related contract — Good 3SAT Algorithm by 2020 — which should presumably go to zero if Deolalikar’s proof is correct. It hasn’t budged much, consistent with skepticism (or with apathy). My second thought was the PopSci Predictions Exchange (PPX), though sadly it has retired. InklingMarkets has a poll about whether P=NP will be resolved before the other Clay Institute prize questions, but not about how it will be resolved or the odds of it happening. (The poll is one of several markets sponsored by the Woodrow Wilson Center’s Science and Technology Innovation Program — hat tip to Vince Conitzer.) I don’t see anything at longbets, and anyway longbets doesn’t provide odds despite it’s name.

In 1990 Robin Hanson provocatively asked: Could gambling save science?. That question and his thoughtful answers inspired a number of people, including me, to study prediction markets. Indeed, the Foresight Exchange was built largely in his image. P=NP seems one of the most natural claims for any scitech prediction market.

All these years later, when I really need my fix, I can’t seem to get it!

2010/08/14 Update: Smarkets comes the closest: they have real-money betting on whether P=NP will be resolved before the other Clay Institute prize questions. They report a 53% chance as of 2010/08/14 (for the record, I would bet against that). What’s missing is when the award might happen and how the question might be resolved, P=NP or P≠NP. I also don’t see a graph to check whether Deolalikar’s proof had any effect.

If it wasn’t clear in my original post, I found Aaronson’s bet incredibly useful and I am thrilled he did it. I believe he should be commended: his bet was exactly what more scientists should do. Scientists should express their opinion, and betting is a clear, credible, and quantitative way to express it. It would be as shame if some of the negative reactions caused him or others not to make similar bets in the future.

I just wish there were a central place to make bets on scientific claims and follow the odds in the vision of Robin Hanson, rather than every scientist having to declare their bet on their own individual blogs.

blogging, commentary, new media, oddhead blog, search, trends

Famous for 15 tweets

TV era: $quote = “In the future, everyone will be world-famous for 15 minutes”;
Search era: $quote =~ s/minutes/links/;
Social era: $quote =~ s/links/tweets/;

This month I’ve had five times more traffic than in any other month since I began blogging in Oct 2006, even during woblomo.

Why? I paid Paul Graham a compliment that struck a minor viral nerve, spreading through twitter, facebook, and blogs and sending over six thousand people my way on July 16 alone according to quantcast. Of course most have since dispersed.

Oddhead Blog traffic according to Quantcast July 2010

Power on the web flows backward through referrals to the sites that people begin their day with, the sources of traffic. Referrals from social media, unpredictable and bursty though they may be, are inexorably on the rise. As they grow, power will shift away from search engines, today’s referral kings. Who knows, this may embolden publishers to take previously unthinkable steps like voluntary delisting, further eroding the value of search. This has all been said before, perhaps best by Mark Cuban starting in 2008. It would be a blow to openness and hurt users, but would spark a fascinating battle.

Another meta note: I installed a new WordPress theme: Suffusion. It’s fantastic: endlessly configurable, bug free, fast, and well designed. I happened upon it by accident when WP 3.0 broke my old theme and I couldn’t be happier. Apparently written by a teenager, I donated to his beer, er, coffee fund.

mad scientist geek with test tube & lab coat
commentary, finance, gambling, ideas, prediction markets, technology, trends

Casinos for geeks

2 Comments

Gambling has been mocked as “a tax on the mathematically challenged”. Gamblers are stereotyped as losers in life. Casinos reinforce this by literally kicking out people who display too much intelligence. They ban card counting and people who simply win too much. They don’t allow computers or Internet connections in the sports book to block out information. They emphasize familiarity over innovation, cementing their appeal to habitual gamblers over geeks.

floating dice 2 and 5In the eyes of a casino, a sharp is indistinguishable from a cheat. More than boring, this seems fundamentally unfair and unsustainable, inviting disguise. It also turns off a wealthy, influential, and game-loving segment of the population.

What I want: A casino by geeks for geeks that celebrates innovation, encourages cleverness, welcomes gadgets and wifi, and fosters hacking, outwit, and outplay.

At least one casino has seen the light. The M Casino in Las Vegas is parterning with Cantor Fitzgerald to support in-game sports betting with few rules and caps, inviting sharps to, more or less, “bring it on”.

…gamblers can bet on the game even during play, accepting ever-changing point-spreads and odds. They can invest money on a Knicks foul shot going through the hoop or a Dodger getting to first base — contending with ever-evolving odds.

More critically, bettors can create hedges while jumping in and out of positions. But instead of buying into the fast-breaking moves of Microsoft, they’re betting on the Mariners’ impending fortunes.

This form of wagering is new to Las Vegas but old-hat in other markets.

…unlike in most other casinos, laptop computers are welcome.

…”The M wants [sharp bettors] to be there,” believes [professional sports-bettor Steve] Fezzik. “They want your information, and that’s a progressive attitude.”

Kudos to M for taking a chance on a more interesting future and to Cantor for making it happen.

What Cantor is debuting may not be a whole lot more than betfair indoors, but it’s a long overdue start. Here’s to hoping we see even more innovation, including smarter and more expressive markets.

fun, people, probability, technology, trends

The most prescient footnote ever

25 Comments

Back in 2004, who could have imagined Apple’s astonishing rise to overtake Microsoft as the most valuable tech company in the world?

At least one person.

Paul Graham wins the award for the most prescient parenthetical statement inside a footnote ever.

In footnote 14 of Chapter 5 (p. 228) of Graham’s classic Hackers and Painters, published in 2004, Graham asks “If the the Mac was so great why did it lose?”. His explanation ends with this caveat, in parentheses:

And it hasn’t lost yet. If Apple were to grow the iPod into a cell phone with a web browser, Microsoft would be in big trouble.

Wow. Count me impressed.

To find this quote, search inside the book for “ipod cell”.

To get into a 2004 midset, look here and here.

prediction markets, probability, trends, woblomo

Revisiting predictions: Google and Avatar

2 Comments

In September 2007 I predicted that “Google [will buy] a TV ad for Google.com aimed at mass consumers”… before September 2008. I only missed it by a year and a half.

In December, before Avatar was released and while some still thought it more likely to sink than swim, Slate journalist Josh Levin asked us to predict its opening weekend box office earnings using our models. We projected between $65 and $84 million. The actual number? $77 million.

Confession: In this post I am guilty of exactly the sins I’ve complained about in the past: cherry picking positive outcomes in hindsight, and measuring probabilistic predictions categorically and in isolation. Oops.

finance, trends

Microfunding: the next big small thing?

9 Comments

First micro lending, then microblogging, now microfunding.* Announcements of three funds recently sixdegreed their way to my doorstep, each smaller and faster than the next, a trend iconified by the famously speedy and minuscule Y Combinator:

1) George “Greek geek” Tziralis’s openfund; 2) Kevin Dick’s Black Swan Fund [via Daniel Horowitz]; and 3) the Awesome Foundation [via Foo Camp list].

The beginning of a trend?

Update 2010/03/19: The Black Swan fund, now called RightSide Capital is open for business.

___________

*Yet still no micropayments! 🙁