Category Archives: prediction markets

New York Post Video: Gambling on Politics

Two New York Post video reporters came to Yahoo!’s midtown NYC office last Friday to interview me for a piece they were producing on intrade‘s political prediction markets. The video is now up on NYPOST.COM and in the embedded player below. The reporters were friendly and professional — thankfully they cut out most of my word-fumbling moments — and the end result is an entertaining, polished, informative video geared toward newbies. My own role came out at least not terrible.

If you look carefully, you’ll see subtle product placement of the Yahoo! Election Dashboard, which aggregates a ton of election numbers including intrade prices. You can also see short clips of the conference room, whiteboard scribbles, ylogo, and cubes at our Y! Research NYC office.

See also: Chris Masse’s comments

Crowdsourcing meets crowd wisdom

I met Lukas Biewald at CI Foo [1 2 3 4 5 6]. Lukas is involved in a fascinating startup called Dolores Labs that helps crowdsource your problem to Amazon’s Mechanical Turk. Read his manifesto.

As an experiment, they hired Turkers to label a sample of news items about Barack Obama and Hillary Clinton as either positive or negative for each of the candidates. As it turns out, every news source was pro-Obama except ABC News, with Digg being the pro-est of the pro-Obama camp.

They then plotted changes in news sentiment alongside the price of Obama’s intrade contract:

News sentiment and intrade price for Obama vs Clinton Feb-March 2008

Visually, there appears to be a correlation and news sentiment may actually be the leading indicator between the two, however it would be great to see statistical confirmation, if it’s even possible with such a small sample.

I sent Lukas some poll data and search buzz data that we’ve been collecting for the Yahoo! Election Dashboard. I’ll post an update if anything interesting results from lining up all four signals.

The right way to implement a multi-outcome prediction market: Linear programming

There are many examples of multi-outcome prediction markets, for example election markets with more than two candidates, or sports championship markets with dozens of teams.

What is the best way to implement a multi-outcome prediction market?

The simplest way is to effectively ignore the fact that there are multiple outcomes, breaking up the market into a bunch of separate binary markets, one for each outcome. Each outcome-market is an independent instrument with its own order flow and processing.

This seems to be the most common approach, taken by for example intrade, IEM, racetracks, and most financial exchanges. IMHO, it’s the wrong way, for three reasons.

  1. Splitting up a market can hurt liquidity. In a split market, there are effectively two ways to do everything (e.g., buy outcome 1 equals sell outcomes 2 through N), so traders may not see the best price for what they want to do, and orders may not fill at the best price available. There may even be orders that together constitute an agreeable trade, yet are stuck waiting in separate queues.
  2. A split market may also slow information propagation. Price changes in one outcome do not directly affect prices of other outcomes; it’s left to arbitrageurs to propagate logical implications.
  3. Finally, a naïve implementation of a split market may limit traders’ leverage, forcing them set aside more money than necessary to complete a set of trades. For example, on IEM, short selling one share at $0.99 requires that you have $1 in your account, even though the most you could possibly lose in this transaction is $0.01. The reason is that to short sell on IEM you must first buy the bundle of all outcomes for $1, then sell off the outcome that you don’t want.

IEM has possibly the worst implementation, suffering from all three problems.

Intrade’s implementation is slightly better: they at least handle leverage correctly.

Newsfutures is smarter still.1 They generate phantom bids to reflect the redundant ways to place bets. For example, if there are bids for outcomes 2 through N that add up to $0.80, they place a phantom ask on outcome 1 for $0.20. A trader who accepts the ask, buying outcome 1 for $0.20, actually sells outcomes 2 through N behind the scenes, an entirely equivalent transaction. Chris Hibbert has a more elaborate methodology for eking out as much liquidity as possibly using phantom bids, an approach he has implemented plans to implement in his Zocalo platform.

Yet phantom bids are a band-aid that cannot entirely heal a fractured market. Still missing is the ability to trade bundles of outcomes in a single transaction.

For example, consider the US National Basketball Association championship market, with 30 teams. A split market (possibly with phantom bids) works great for betting on individual teams one at a time, but is terribly cumbersome for betting on groups of teams. For example, betting that a Western conference team will win requires 15 separate transactions. A common fix is to open yet another market in each popular bundle, however this limits choice and exacerbates all three problems above.

Bundling is especially useful with interval bets. For example, consider this bet on the peak price of gasoline through September 2008, broken up into intervals $3-$3.25, $3.25-$3.40, etc. In order to bet that gas prices will peak between, say, $3.40 and $4.30, you must buy all six outcomes spanning the interval, one at a time. (Moreover, you must sum the six outcome prices manually to compute a price quote.)

Fortunately, there is a trading engine that solves all three problems above and also allows bundle bets…

It’s linear programming!

Bossaerts et al. call it combined value trading. Baron & Lange, Lange & Economides and Peters et al. call it a parimutuel call market. Fortnow et al. and Chen et al. describe it in the context of combinatorial call markets.

Whatever you call it, the underlying principle is relatively straightforward, and it seems inherently the right way to implement a multi-outcome market. Yet I’ve rarely seen it done. The only example I know of is the now defunct economic derivatives markets run by Longitude, Goldman Sachs, and Deutsche Bank.

The set up of the linear program is as follows. Each order is associated with a decision variable x that ranges between 0 and 1, encoding the fraction of the order that the auctioneer can accept.2 There is one constraint per outcome that ensures that the auctioneer never loses money across all outcomes. The choice of objective function depends on the auctioneer’s goals, but something like maximizing the fill fraction makes sense.

Once the program is set up, the auctioneer solves for the x variables to determine which orders to accept in full (x=1), which to accept partially (0<x<1), and which to reject (x=0). The program can be solved either in batch mode, after waiting to collect a number of orders, or in continuous mode immediately as new orders arrive. Batch mode corresponds to a call market. Continuous mode corresponds to a continuous auction, a generalization of the continuous double auction mechanism of the stock market.

Each order consists of a price, a quantity, and an outcome bundle. Traders can just as easily bet on single outcomes, negations of outcomes, or sets of outcomes (e.g., all Western Conference NBA teams). Every order goes into the same pool of liquidity no matter how it is phrased.

Price quotes are queries to the linear program of the form “at what price p will this order be accepted in full?” (I believe that bounds on the dual variables of the LP can be interpreted as bid and ask price quotes.)

Lange & Economides and Peters et al. devise clever ways to make prices unique rather than bid-ask ranges, by injected a small subsidy to seed the market at the onset.

Note that Hanson’s market scoring rules market maker also elegantly solves all the same problems as the LP formulation, including handling bundle bets. However, the market maker requires a patron to subsidize the market, while the LP auctioneer formulation is budget balanced — that is, can never lose money.

Also note that I am not talking about a combinatorial-outcome market here. In this post, I am imagining that the number of outcomes is tractable — small enough so that we can explicitly list, store, and compute across all of the outcomes. A true combinatorial-outcome market, on the other hand, has an exponentially large number of outcomes making it impossible to even list them all explicitly, and forcing all calculations to operate on an implicit representation of outcomes, for example Boolean combinations of base events.

1Apparently worked out in conjunction with Brian Galebach, a mathematician and Newsfutures fan extraordinaire who runs the prediction contest probabilitysports.com.
2Alternatively, the variables can range between 0 and q, where q is the quantity of shares ordered.

The proverbial wisdom of crowds

I am fascinated by thingnaming.

In some ways there is no more straightforward way to certify your influence on the world than to count the number of times people use a word or phrase you invented.

On this count, James Surowiecki is a champion.1 His catch phrase the wisdom of crowds — a brilliant feat of thingnaming — has in four short years spread to over 2.1 million nooks and crannies around the web.2

In fact, BusinessWeek reporter Jennifer L. Schenker recently termed it the “proverbial wisdom of the crowd”. [Finding faces in the e-crowd, Businessweek, Dec 24, 2007, p.70]

At first I meant to poke fun at Schenker for attributing this adjective associated with adages of ancient origin to a four-year-old artifact.

However, digging further, I noticed that Schenker is right. Another use of the word proverbial is “having become an object of common mention or reference”, for example “your proverbial inability to get anywhere on time”.

Interestingly, a pun on Surowiecki’s phrase appears in the same issue of BusinessWeek. Stephen Baker’s long (yet remarkably content-free) piece on cloud computing is titled Google and the wisdom of clouds.

It’s amazing how crucial a good thingname can be to the success of a thing. Thanks James!

1Of course, beyond thingnaming, Surowiecki wrote a fantastic book that helped catalyze an industry, among his other plentiful contributions and accomplishments.
2For examples of unsuccessful thingnaming look here and here.

Search engine futures!

I am happy to report that on my suggestion intrade has listed futures contracts for 2008 search engine market share.

Here is how they work:

A contract will expire according to the percentage share of internet searches conducted in the United States in 2008. For example, if 53.5% of searches conducted in the United States in 2008 are made using Google then the contract listed for Google will expire at 53.5…

…Expiry will be based on the United States search share rankings published by Nielson Online.

I think this could be a fascinating market because:

  • Search engine market share is very important to these major companies, with dramatic effects on their share prices.
  • Search engine market share is fluid, so far with Google growing inexorably. However, Microsoft has cash, determination, Internet Explorer, and the willingness to experiment. Ask.com has erasers, 3D, ad budgets, and The Algorithm. Yahoo!, second in market share, often tests equal or better than Google, and new features like Search Assist are impressive.
  • The media loves to write about it.
  • A major search company might use the market to hedge. Well, this seems far-fetched but you never know. Certainly, from an economic risk management standpoint it would seem to make a great deal of sense. (Here, as always on this blog, I speak on behalf of myself and not my company.)

Finally, I have to comment on how refreshingly easy the process was in working with intrade. They went from suggestion to implementation in a matter of days. It’s a shame that US-based companies are in contrast stuck in stultifying legal and regulatory mud.

Addendum 2008/01/26: Here are links to some market research reports:
Nielsen | ComScore | HitWise | Compete

(It seems that Nielsen Netratings homepage is down; getting 404 error at the moment)

Addendum 2008/03/07: If you prefer, you can now also bet on search share just for fun with virtual currency at play.intrade.com.

(Nielsen Netratings homepage is still down, now for over a month. It’s even more ridiculous given that their own Nielsen Online website points to this page.)

FYI 2 CFPs: WWW2008-IM & ACM EC'08

Here are two Call For P*s for upcoming academic/research conferences:

  1. Call for Participation: For the first time, the World Wide Web Conference has a track on Internet Monetization, including topics in electronic commerce and online advertising. The conference will be held in Beijing April 21-25, 2008. If the Olympics in China are all about image, then the Internet in China is all about, well, Monetization. (A lot of it, growing fast.)
  2. Call for Papers: The 2008 ACM Conference on Electronic Commerce will be held in Chicago July 8-12, 2008 in proximity to AAAI-08 and GAMES 2008. Research papers on all aspects of electronic commerce — including personal favorites prediction markets and online advertising — are due February 7, 2008.

You can signal your interest on social events calendar upcoming.org: WWW2008 | EC’08

Hope to see some of you in either the Forbidden or Windy City, as the case may be.

Yahoo! Election 2008 Political Dashboard

I’m happy to report the launch of the Yahoo! Election ’08 Political Dashboard. Using the dashboard, you can navigate through a wealth of election-related data, including prediction market data from intrade.com, polling data from Real Clear Politics, search buzz data from Yahoo!, and financial contributions data, regional demographic data, and historical voting records from AP. You can view the election landscape from the national level or dive in deeper to investigate trends state by state.

Yiling, Tej, Lance, and I played supporting roles among a cast that includes fantastic teams at Yahoo! News, UI/Design, Ops, Q&A, and more.

We’ve come a long way since 2006.

See also coverage from TechCrunch and the Yahoo! corporate blog.

Computational aspects of prediction markets: Book chapter and extended bibliography

Rahul Sami and I wrote a chapter called “Computational aspects of prediction markets” in the book Algorithmic Game Theory, Cambridge University Press, forthcoming 2007.

You can download an almost-final version of our chapter here.

Update 2007/09/19: You can now also download the entire book Algorithmic Game Theory: username agt1user , password camb2agt . If you like it, you can buy it.

In the course of writing the chapter, we compiled an extended annotated bibliography that ended up being too long to publish in its entirety in the book. So we trimmed the bibliographic notes in the book to cover only the most directly relevant citations. You can download the full extended bibliography here.

Here is the abstract of our chapter:

Prediction markets (also known as information markets) are markets established to aggregate knowledge and opinions about the likelihood of future events. This chapter is intended to give an overview of the current research on computational aspects of these markets. We begin with a brief survey of prediction market research, and then give a more detailed description of models and results in three areas: the computational complexity of operating markets for combinatorial events; the design of automated market makers; and the analysis of the computational power and speed of a market as an aggregation tool. We conclude with a discussion of open problems and directions for future research.

If you’re interested in this topic, you might also take a look at our recent paper on Betting on permutations, published after the book chapter was completed.

Finally, for a higher-level treatment, here is a pre-print version of a short letter on “Combinatorial betting” that we submitted to SIGecom Exchanges.

Predictions: Apple bites, Google eats

Happy 5768 everyone!

Time for some predictions.

  1. Apple bites into PC pie. Apple Computer (remember them?) will attain at least 30% PC market share by 5772.

    Probability: 40% ; Willing to stake: $Y20

    On the front lines, silver Powerbooks are infiltrating in droves. At techie conventions and computer science conferences, penetration has gone from almost zero to something approaching 1/3 by anecdotal evidence. Wandering about these venues, it’s not terribly uncommon to see a table of three or four who apparently all agree to think different. At Yahoo!, more and more of Jobs’s ministers are simply preaching to the converted. In our Yahoo! Research New York office, for example, laps are topped at least two to one with half-eaten half-glowing apples. Even tech celeb Marc Andreessen has returned to the fold.

    But can the Apple bug jump from geeks to grandmas? (Well, my daughters’ grandma is already infected.) I’m guessing so. After all, these same alphadopters led the way to mp3s, Google, Wikipedia, Slashdot, blogs, Firefox, Digg, and Homestar Runner, unlocking remarkable truths along the way like “web search can be monetized”, “Really Simple trumps Really Smart”, and “give up now, Friendster has already won”. (Oops.)

    Why is there an Apple renaissance on the desktop? A big reason is that the OS’s natural monopoly is not so natural anymore. Today, the browser is the most important piece of software on your computer, and a viable cross-platform browser (Firefox) exists that almost every web site designs to. A second reason: it turns out that Intel chips are faster and better than PowerPC chips after all, despite decades of vehement Apple fanboy arguments to the contrary. Third, Apple’s built-in iLife software suite really is astonishingly useful and well designed and speaks to the new killer apps of the desktop: pictures, music, video, web, and email. A final reason is, well, Apple is cool, and technology is at least as much about fashion as function, or at least more than geeks would like to admit.

    Disagreers can accept my yootleoffer or put your play money where your mouth is on related bets at PPX and Inkling.

    (Side note: My take on Apple’s fumbled iPhone price cut: I believe that Apple reacted in fear of the looming gPhone. However, if history is a guide, that fear may be an exaggerated fear of the unknown.)

  2. Google eats its own dog food. Google buys an advertisement by the end of 5768.

    Probability: 60% ; Willing to stake: $Y20

    Google is the king of selling advertisements. So they must believe that advertising is effective, right? Then why doesn’t Google advertise for itself? (I’m not counting recruiting ads.) I’m guessing the reason is that they don’t have to. As a media darling, they get more than enough free press to catalyze their already monstrous word of mouth. I expect that as the glow wears off, as some of the not not evil jabs — deserved or not — start to stick, and as they settle into Big Company mode, you will start to see Google spots on TV and elsewhere.

2007/09/17 Update: Sean McNee noticed that Google is advertising Google Apps to enterprise customers on VentureBeat and the Seattle Times [example ad image]. As a result, let me update my prediction to “Google buys a TV ad for Google.com aimed at mass consumers”.

2007/09/19 Update: Maverick blogger, Maverick owner, Yahoo! benefactor, and uber alphadopter Mark Cuban is dancing with the Steves.

2010 Update: I was right, just 1.5 years too early. In other words, I was wrong.

Betcha's gambit

Betcha is bold. To say the least. The founder Nick Jenkins is either crazy, brilliant, or, like many founders, both. Betcha is a platform for peer to peer betting not unlike gottabet, betfair, or intrade. Except for two (intimately related) details: (1) all debts are on the honor system, and (2) it’s based in Seattle, WA, UIGEA. Betcha makes no bones about it ( no “wink wink” here): they expect users to bet on anything and everything including sports. But because coughing up is not strictly enforced, the site evades the letter of the gambling laws. To engender trust, Betcha verifies its users’ credit cards and tracks their reputation scores, but in the end all payments are voluntary. The site earns money via listing fees.

I can’t help but admire Jenkins and Co., and I hope their gambit succeeds: my heart is with them even if my head is a step behind. (For more legal discussion see Tom Bell and The Boston Globe.)

And as much as I like the concept, I do have to ding Betcha for one of the most convoluted, head-scratching explainers I’ve heard in a long time:

“As an open, honor-based betting platform, Betcha is like an auction site, Las Vegas, a marketplace of ideas, and The Golden Rule — all rolled into one. [1]

[1] “The Golden Rule” refers to the idea that you should do unto others as you’d have them do unto you. It is the fundamental principle behind most of the world’s major religions. And while we aren’t here to push religion on anyone, doing well by others is a principle we’d like to see more of.

Whaa? Four (weak) analogies plus a long-winded footnote? C’mon, Betcha, please KISS.