The night of February 15, 2012, was an uncomfortable one for me. Not a natural talker, I was out of my element at a press dinner organized by Yahoo! with journalists from the New York Times, Fast Company, MIT Tech Review, Forbes, SF Chronicle, WIRED, Reuters, and several more [1]. Even worse, the reporters kept leading with, “wow, this must a big night for you, huh? You just called the election.”
We were there to promote The Signal, a partnership between Yahoo! Research and Yahoo! News to put a quantitative lens on the election and beyond. The Signal was our data-driven antidote to two media extremes: the pundits who commit to statements without evidence; and some journalists who, in the name of balance, commit to nothing. As MIT Tech Review billed it, The Signal would be the “mother of all political prediction engines”. We like to joke that that quote undersold us: our aim was to be the mother of all prediction engines, period. The Signal was a broad project with many moving parts, featuring predictions, social media analysis, infographics, interactives, polls, and games. Led by David “Force-of-Nature” Rothschild, myself, and Chris Wilson, the full cast included over 30 researchers, engineers, and news editors [2]. We confirmed quickly that there’s a clear thirst for numeracy in news reporting: The Signal grew in 4 months to 2 million unique users per month [3].
On that night, though, the journalists kept coming back to the Yahoo! PR hook that brought them in the door: our insanely early election “call”. At that time in February, Romney hadn’t even been nominated.
No, we didn’t call the election, we predicted the election. That may sound like the same thing but, in scientific terms, there is a world of difference. We estimated the most likely outcome – Obama would win 303 Electoral College votes, more than enough to return him to the White House — and assigned a probability to it. Of less than one. Implying a probability of more than zero of being wrong. But that nuance is hard to explain to journalists and the public, and not nearly as exciting.
Although most of our predictions were based on markets and polls, the “303″ prediction was not: it was a statistical model trained on historical data of past elections, authored by economists Patrick Hummel and David Rothschild. It doesn’t even care about the identities of the candidates.
I have to give Yahoo! enormous credit. It took a lot of guts to put faith in some number-crunching eggheads in their Research division and go to press with their conclusions. On February 16, Yahoo! went further. They put the 303 prediction front and center, literally, as an “Exclusive” banner item on Yahoo.com, a place that 300 million people call home every month.
The firestorm was immediate and monstrous. Nearly a million people read the article and almost 40,000 left comments. Writing for Yahoo! News, I had grown used to the barrage of comments and emails, some comic, irrelevant, or snarky; others hateful or alert-the-FBI scary. But nothing could prepare us for that day. Responses ranged from skeptical to utterly outraged, mostly from people who read the headline or reactions but not the article itself. How dare Yahoo! call the election this far out?! (We didn’t.) Yahoo! is a mouthpiece for Obama! (The model is transparent and published: take it for what it’s worth.) Even Yahoo! News editor Chris Suellentrop grew uncomfortable, especially with the spin from Homepage (“Has Obama won?”) and PR (see “call” versus “predict”), keeping a tighter rein on us from then on. Plenty of other outlets “got it” and reported on it for what it was – a prediction with a solid scientific basis, and a margin for error.
This morning, with Florida still undecided, Obama had secured exactly 303 Electoral College votes.
Just today Obama wrapped up Florida too, giving him 29 more EVs than we predicted. Still, Florida was the closest vote in the nation, and for all 50 other entities — 49 states plus Washington D.C. — we predicted the correct outcome back in February. The model was not 100% confident about every state of course, formally expecting to get 6.8 wrong, and rating Florida the most likely state to flip from red to blue. The Hummel-Rothschild model, based only on a handful of variables like approval rating and second-quarter economic trends, completely ignored everything else of note, including money, debates, bail outs, binders, third-quarter numbers, and more than 47% of all surreptitious recordings. Yet it came within 74,000 votes of sweeping the board. Think about that the next time you hear an “obvious” explanation for why Obama won (his data was biggi-er!) or why Romney failed (too much fundraising!).
Kudos to Nate Silver, Simon Jackman, Drew Linzer, and Sam Wang for predicting all 51 states correctly on election eve.
As Felix Salmon said, “The dominant narrative, the day after the presidential election, is the triumph of the quants.” Mashable’s Chris Taylor remarked, “here is the absolute, undoubted winner of this election: Nate Silver and his running mate, big data.” ReadWrite declared, “This is about the triumph of machines and software over gut instinct. The age of voodoo is over.” The new news quants “bring their own data” and represent a refreshing trend in media toward accountability at least, if not total objectivity, away from rhetoric and anecdote. We need more people like them. Whether you agree or not, their kind — our kind — will proliferate.
Congrats to David, Patrick, Chris, Yahoo! News, and the entire Signal team for going out on a limb, taking significant heat for it, and correctly predicting 50 out of 51 states and an Obama victory nearly nine months prior to the election.
Footnotes
[1] Here was the day-before guest list for the February 15 Yahoo! press dinner, though one or two didn’t make it:
- New York Times, John Markoff
- New York Times, David Corcoran
- Fast Company, EB Boyd
- Forbes, Tomio Geron
- MIT Tech Review, Tom Simonite
- New Scientist, Jim Giles
- Scobleizer, Robert Scoble
- WIRED, Cade Metz
- Bloomberg/BusinessWeek, Doug MacMillan
- Reuters, Alexei Oreskovic
- San Francisco Chronicle, James Temple
[2] The extended Signal cast included Kim Farrell, Kim Capps-Tanaka, Sebastien Lahaie, Miro Dudik, Patrick Hummel, Alex Jaimes, Ingemar Weber, Ana-Maria Popescu, Peter Mika, Rob Barrett, Thomas Kelly, Chris Suellentrop, Hillary Frey, EJ Lao, Steve Enders, Grant Wong, Paula McMahon, Shirish Anand, Laura Davis, Mridul Muralidharan, Navneet Nair, Arun Kumar, Shrikant Naidu, and Sudar Muthu.
[3] Although I continue to be amazed at how greener the grass is at Microsoft compared to Yahoo!, my one significant regret is not being able to see The Signal project through to its natural conclusion. Although The Signal blog was by no means the sole product of the project, it was certainly the hub. In the end, I wrote 22 articles and David Rothschild at least three times that many.
Now that I’ve 
On Thursday April 26, 2012, I resigned from Yahoo! after nearly 10 without actively changing jobs. Here is the full text of the goodbye letter(s) I sent. It’s the kind of long-winded last salvo that few people actually read, and now I’m foisting it upon you, dear reader, but I can’t help myself. Writing it brought back many wonderful memories and a tinge of sadness at the end of a truly amazing work environment for me, but I found the exercise rewarding. I really appreciate the many kind words and well wishes: some were poignant and immensely gratifying. The feeling is mutual. If nothing else, throughout my career I have had the great fortune of working with amazing people who are equal parts brilliant, effective, and nice, including my bosses, peers, reports, and students.
A professional goodbye and a personal thanks to Carol Bartz
My geek CEO was fired. If you’re wondering whether she deserved it, or Yahoo! is better off for it, or Roy Bostock is a doofus or dorfus, I don’t really know.* But I do have a personal story about Carol Bartz that’s indicative of the kind of CEO she was and the kind of person she is, perfect for Ada Lovelace day, a day to blog about women in science and technology who inspire you.
In May 2010, my wife Lauren was diagnosed with breast cancer. On Sunday, May 9, 2010—Mother’s Day no less—I received a phone call. “Hello?,” I said. “Hi, this is Carol Bartz,” she said. “Wow!,” I couldn’t help saying. I had never spoken to her before. She proceeded to say how sorry she was for me and Lauren, to reassure us, to ask me questions, and to answer mine.
More than a year, multiple surgeries, and six chemo sessions later, I’m happy to say that Lauren is past the worst part of the treatment and, to the best of anyone’s knowledge, cancer free. At the time, we were frightened, bewildered, and angry. To me, the most overwhelming feeling was disbelief. Was this really happening to us? It was surreal. Lauren’s strength and sheer will to keep our home life as normal as possible, and her ability to turn the ordeal into a positive is amazing and helped me cope. That my mom and Lauren’s mom went through the same thing also helped. The more we looked into it the more we realized breast cancer was everywhere—shockingly common even at Lauren’s age. (Especially in New Jersey, one of only five states in the top tier for both incidence of and mortality from breast cancer.) The calls to increase the age of first mammogram border on criminal. One silver lining for Lauren has been meeting the amazing support community of breast cancer sufferers, survivors, and their friends. They have inspired her to give back in many ways. My mom, a radiologist and ACR fellow, was herself inspired to specialize in mammography and pursue breast cancer research.
It turns out, Carol Bartz is a survivor herself and, in addition to being one of the fifty most powerful women in business, is just another member of the breast cancer support community who cares deeply. Carol had over twelve thousand employees. To take the time to call one of them on a holiday weekend to address personal problems and pain shows the kind of leader she is. (And shows the kind of bosses Preston and Prabhakar are, who thought enough to bring it to her attention.) It’s a “Yahoo! moment” and a Carol moment that I remember vividly and continues to stick out in my mind. I suspect most stereotypes of corporate and public leaders as conniving powermad ladder climbers are just that: stereotypes. But still, I’m convinced that not all—probably few—CEOs would do what Carol Bartz did. Goodbye, good luck, and, most of all: Thanks, Carol.
* I will say that I respect Carol’s willingness give her blunt assessment of the board, possibly risking $10 million to do so, and to come right out and say “I was fired” rather than hide behind “more time with family” cliches. I’m not surprised that the board gave their full confidence to her in public just two months before firing her—of course a board always has to say that they have confidence in their current CEO. I am surprised and dismayed that, at least judging by her reaction, it seems the board was also giving their confidence to her in private. That’s HR 101: No one who’s fired should be surprised.