Oddhead Blog

As Jed points out, “an idea is only the first step in innovation, and it’s by far the easiest step”.

Yahoo! Hack Day was created precisely to summon and celebrate the hard step of innovation: the build it step. The goal is simple: take an idea and make it real — in 24 hours. Spend all day and all night coding until a working, useable, if brittle prototype of your idea emerges. Then show it off.

Hack Day is a religion inside Yahoo!, but on September 12-13, 2008, Yahoo! will open up its Sunnyvale campus, inviting any developer who feels like it to join the geek-out frenzy. Sign up here.

Schedule

8am-6pm PT Friday: Over 20 workshops covering YUI and the newest API offerings from Yahoo!, including BOSS, SearchMonkey, Fire Eagle, and more, and previews of what’s next.

8pm Friday: A surprise musical guest takes the stage (it’s not 2006 guest Beck, but apparently the lyrics are “hacker-friendly” and “may not be appropriate for young children”). Hacking continues all night.

2pm Saturday: Judging, including a special hack-off for the winners of the University Hack Days.

Saturday evening: Awards.

History, thoughts, and notes

At the first Open Hack Day in 2006 in Sunnyvale (see photos), 400 developers fueled by 500 pizzas and a live Beck performance cranked out 54 hacks. At Open Hack Day London lightening struck twice and it rained indoors. Bangalore followed.

If you’re a student, Yahoo! Hack Week may be coming to a campus near you. We’ve held Hack Weeks at Georgia Tech, CMU, UIUC, UC Berkeley, and Stanford, and I believe Waterloo is next. Here’s a quote describing these Hack U events: “Computer science students fueled by fast food, ultra-caffeinated beverages, and alternative music, are free to let their imaginations run wild, tapping the Yahoo! library of APIs to create hacks that advance the Internet experience.”

Why Hack Day? Many an engineer join Jed in lamenting how the PowerPointy set co-opted the term innovation, rendering it almost meaningless. Hack Day was created in part to reclaim innovation for the makers.

Why does it work? Hack Day is to programmers what NaNoWriMo is to writers: a timed artistic challenge that on it’s face is a ludicrous and artificial pretense for accomplishing a goal. Yet somehow the exercise induces a psychological state perfect for making progress on the initial “80% phase” of creation. The punishing deadline forces all meta processing aside — no critic, no perfectionist, no planner, no lazy dreamer — and encourages the raw energy embodied by Nike’s Kirk-beats-Spock slogan “just do it”.

A number of Yahoo! products, services, and features were born on a Hack Day (here are two). Yoopick was too.

Why open?

Openness is one of only three overarching goals for Yahoo!. The other two goals — starting point for users and must buy for advertisers — are in some sense incontrovertible, yet the openness goal reflects a riskier “if we build it they will come” stand that’s grounded in Yahoo!’s respect for and debt of gratitude to Internet culture. Open Hack Day, Hadoop, Pig, cloud computing, academic relations, publications, APIs, BOSS, SearchMonkey, YUI, Pipes, OpenSocial, and OpenID are among the many examples showing that Yahoo!’s commitment to openness is real. (Jeremy, RIP 2008, said it better.)

Update 2008/09/11: Blueprint mobile SDK and more Y! Open announcements (music, homepage, mail, ads)

Update 2008/09/26: CNET says: Yahoo Open: Finally, a real answer to Google. Also, Google spouse Kara Swisher gets defensive and rewrites history.

Cross-posted on midasoracle.org

The Yahoo! News Political Dashboard has re-launched for the general election stretch run of the 2008 US Presidential election.

From the main map you can see the status of the election in every state according to either polls or Intrade prediction market odds. Hover your mouse over a state to see current numbers or click on a state to see historical trends. On the side, you can see search trends, blogs, news, and demographic breakdowns at national and state levels.

You can also “create your own scenario” by picking who will win in every state. You can save and share your prediction and compare against markets, polls, history, or celebrities. More on ycorpblog.

In the markets view, states are colored either bright red or bright blue, regardless of how close the race is in that state. To see a visualization that blends colors to reflect the tightness of the race, see electoralmarkets.com.

Yahoo! News also offers a candidate badge that you can display on your blog declaring your choice. The badge features national-level polls, prediction markets, search buzz, and money raised.

We invite participants to the first New York Computer Science and Economics Day (NYCE Day), October 3 2008, at the New York Academy of Sciences, 7 World Trade Center.

NYCE Day is a gathering for people in the NYC metropolitan area with interests in auction algorithms, economics, game theory, e-commerce, marketing, and business to discuss common research problems and topics in a relaxed environment. The aim is to foster collaboration and the exchange of ideas.

The program features invited speakers Asim Ansari (Columbia), Susan Athey (Harvard), Constantinos Daskalakis (MIT), and Tuomas Sandholm (CMU), and a rump session with short contributed presentations.

You can indicate your interest in the event on upcoming.yahoo but official registration should go through NYAS.

Your participation and suggestions are greatly welcome. Please distribute this announcement to people and groups who may be interested.

Thanks,
NYCE Day Organizers
 Anindya Ghose, NYU
 S. Muthu Muthukrishnan, Google
 David Pennock, Yahoo!
 Sergei Vassilvitskii, Yahoo!

P.S. This is one week prior and in the same location as the Symposium on Machine Learning.

P.P.S. For those familiar, NYCE Day is inspired as a Right Coast version of BAGT.

P.P.P.S. The New York Academy of Sciences in a spectacular venue. See for yourself.

If you haven’t played around with Yahoo! Pipes, I highly recommend it. It’s a usable and useful service that brings web mashups to the masses, making this favorite hacker pastime as easy as dragging objects around on the screen.

For example, it took me probably about ten minutes as a first-time user to create a map mashup showing Barack Obama’s upcoming campaign stops. I “piped” the output of Washington Post’s RSS feed to a location-extractor module that identifies and geo-codes place names and renders them on a map. Here’s a screenshot of the output:

The easiest way to get started is to find an existing Pipe, clone it, and modify it as your own. Using this feature, I cloned my Obama map and in about one minute had a McCain map too.

Pipes uses a visual programming interface. The idea of “programming by picture” (I recall playing with one in the 1980s) never took hold as a mainstream tool. However, as a metaphor for mashups, where to goal is to chain together a number of sources and services, the visual approach seems exactly right. The implementation in a browser is a feat of ajaxian magic that I still find remarkable, even as Yahoo! and others are commoditizing the art. I imagine that even non-programmers should have little trouble constructing their own Pipes. Here is a screenshot of the source “code” for my Obama map:

Pipes has dozens of useful modules, including user input, Yahoo! Search, Flickr, and regular expressions.

You can embed the Pipe on your own website with a single line of javascript. I did this with my Obama and McCain campaign travel maps here. Or you can grab the output as an XML feed to use however you wish.

Pipes allows you to create human-readable URLs (e.g., http://pipes.yahoo.com/oddhead/obamatravelmap), a nice touch.

The icing on the cake for me is how Pipes — unlike so many other web sites, including some on Yahoo! — treats me and my Opera browser like adults:

(BTW, Pipes seems to work fine on Opera).

Unfortunately, Daniel Raffel, one of the key founders of Yahoo! Pipes, left Yahoo!. However, the team seems to be strong and continues to innovate, so I’m hopeful this fantastic service will continue to improve and thrive.

We just added a new feature to Yoopick designed especially for Frenchmen Chris and Emile and citizens of nineteen other countries to place their swagor* on how many Olympic medals they think their country will win.

We’ve argued that the Yoopick interface is useful for predicting almost any kind of number, and since medal count is indeed a number, we thought we’d give it a try.

Besides, Lance told us it would be a good idea.

Sign up, play, enjoy, and don’t forget to tell us what you think!

Thanks,
Sharad Goel
David Pennock
Dan Reeves

* Scientific wild-ass guess, on record

I’m happy to announce the public beta launch of Yoopick, a sports prediction contest with a twist.

You pick any range you think the score difference or point spread of the game will fall into, for example you might pick Pittsburgh wins by between 2 and 11 points.

The more your prediction is viewed as unlikely by others, and the more you’re willing to stake on your prediction, the more you stand to gain. Of course it’s all for fun: you win and lose bragging rights only.

You can play with and against your friends on Facebook.

You can settle a pick even before the game is over, much like selling a stock in the stock market. Depending on what other players have done in the interim, you may be left with a gain or loss. You gain if you were one of the first to pick a popular outcome.

If you run out of credit, you can “work off your debt” by helping to digitize old books via the recaptcha project.

Those are the highlights if you want to go play the game. If you’re interested in more details, read on…

Motivation, Design, and Research Goals

There are a great many sources of sports predictions, including expert communities, statistical number crunchers, bookmakers, and betting exchanges. Many of these sources are highly accurate, however they typically focus on predicting the outright or spread-adjusted winner of the game. Our goal is to obtain more information about the final score, including the relative likelihood of each point spread. For example, if our system is working, on average there should be more weight put on point spreads of 3 and 7 in NFL games than on 2,4,6, or 8.

We chose sports as a test domain to tap into the avid fan base and the armies of arm chair (and Aeron chair) prognosticators out there. However, the same approach should translate well to any situation where you’d like to predict a number, for example, the vote share of a politician or the volume of sales of your company’s widget. In addition to giving you the expected value of the number, our approach gives you the confidence or variance of the prediction — in fact, it gives you the entire probability distribution, or the likelihood of every possible value of the number.

Underneath the hood, Yoopick is a type of combinatorial prediction market where the possible outcomes are the values of the point spread, and each pick is a purchase of a bundle of outcomes in a given interval. We use Hanson’s logarithmic market scoring rules market maker to price the picks — that is, to set the risk/reward ratio. This pricing mechanism also determines the gain or loss when picks are settled early.

Wins and losses on Yoopick are measured in milliyootles, a social currency useful for expressing thanks.

Our market maker can — and we expect will — lose yootles on average. Stated another way, we expect players as a whole to gain on average. At the same time, we actively work to improve our market maker to limit its losses to control inflation in the game.

Because the outcomes of a game are tied together in a unified market, picks in one region automatically affect the price of picks in other regions in a logically consistent way. Players have considerable flexibility in how and what information they can inject into the market. In particular, players can replicate the standard picks like outright winner and spread-adjusted winner if they want, or they can go beyond to pick any interval of the point spread. No matter the form of the pick, all the information flows into a single market that aggregates everything in a unified prediction. In contrast, at venues from Wall Street to Churchill Downs to High Street to Las Vegas Boulevard, markets with many outcomes are usually split into independent one-dimensional markets.

Our goal is to test whether our market design is indeed able to elicit more information than traditional methods. We hope you have fun playing in our Petri dish.

Sharad Goel
David Pennock
Daniel Reeves
Prasenjit Sarkar
Cong Yu

Two New York Post video reporters came to Yahoo!’s midtown NYC office last Friday to interview me for a piece they were producing on intrade’s political prediction markets. The video is now up on NYPOST.COM and in the embedded player below. The reporters were friendly and professional — thankfully they cut out most of my word-fumbling moments — and the end result is an entertaining, polished, informative video geared toward newbies. My own role came out at least not terrible.

If you look carefully, you’ll see subtle product placement of the Yahoo! Election Dashboard, which aggregates a ton of election numbers including intrade prices. You can also see short clips of the conference room, whiteboard scribbles, ylogo, and cubes at our Y! Research NYC office.

See also: Chris Masse’s comments

The absurdity of gambling laws in the US leads to such silliness as:

• In 2007, Google, Microsoft, and Yahoo! paid millions in penalties for placing gambling ads, something they haven’t done since they were told to stop in 2004.
• Yahoo! can quote prices from intrade, but can’t link to intrade.
• Google can’t advertise for intrade/tradesports, but can place AdSense ads on intrade.com and tradesports.com. In other words, Google can’t sell eyeballs to gambling sites, but can sell eyeballs on gambling sites.

I am happy to report that on my suggestion intrade has listed futures contracts for 2008 search engine market share.

Here is how they work:

A contract will expire according to the percentage share of internet searches conducted in the United States in 2008. For example, if 53.5% of searches conducted in the United States in 2008 are made using Google then the contract listed for Google will expire at 53.5…

…Expiry will be based on the United States search share rankings published by Nielson Online.

I think this could be a fascinating market because:

• Search engine market share is very important to these major companies, with dramatic effects on their share prices.
• Search engine market share is fluid, so far with Google growing inexorably. However, Microsoft has cash, determination, Internet Explorer, and the willingness to experiment. Ask.com has erasers, 3D, ad budgets, and The Algorithm. Yahoo!, second in market share, often tests equal or better than Google, and new features like Search Assist are impressive.
• The media loves to write about it.
• A major search company might use the market to hedge. Well, this seems far-fetched but you never know. Certainly, from an economic risk management standpoint it would seem to make a great deal of sense. (Here, as always on this blog, I speak on behalf of myself and not my company.)

Finally, I have to comment on how refreshingly easy the process was in working with intrade. They went from suggestion to implementation in a matter of days. It’s a shame that US-based companies are in contrast stuck in stultifying legal and regulatory mud.

Addendum 2008/01/26: Here are links to some market research reports:
Nielsen | ComScore | HitWise | Compete

(It seems that Nielsen Netratings homepage is down; getting 404 error at the moment)

Addendum 2008/03/07: If you prefer, you can now also bet on search share just for fun with virtual currency at play.intrade.com.

(Nielsen Netratings homepage is still down, now for over a month. It’s even more ridiculous given that their own Nielsen Online website points to this page.)

I’m happy to report the launch of the Yahoo! Election ‘08 Political Dashboard. Using the dashboard, you can navigate through a wealth of election-related data, including prediction market data from intrade.com, polling data from Real Clear Politics, search buzz data from Yahoo!, and financial contributions data, regional demographic data, and historical voting records from AP. You can view the election landscape from the national level or dive in deeper to investigate trends state by state.

Yiling, Tej, Lance, and I played supporting roles among a cast that includes fantastic teams at Yahoo! News, UI/Design, Ops, Q&A, and more.

We’ve come a long way since 2006.

See also coverage from TechCrunch and the Yahoo! corporate blog.