Oddhead Logo

Oddhead Blog

Musings of a computer scientist and yahoo1,2 about
prediction markets, gambling, and estimating the odds of everything

October 21st, 2009

Notes from Yahoo! Open Hack Day NYC

Here are my notes from Yahoo! Open Hack Day NYC. For other perspectives read New York Times open sourcerer Nick Thuesen or the Yahoo! devel blog. You can watch videos of some of the talks or browse pictures.

First off, I cheated. I went to sleep in a hotel room rather than hack all through the night. (Even in college I woke up at 4am rather than pull an all nighter.) Still, I made decent progress on some pet projects including combinatorial betting. Daniel, Sharad, and Winter from Yahoo! Research New York participated for real, working through the night. Returning in the morning showered and caffeinated to greet the sleepwalkers was a little surreal. A number of ex-Yahoos joined the festivities including David Yang, Mor Naaman, and Chad Dickerson. (Havi joked that Yahoo! is like finishing school for entrepreneurs. If you count Yahoo! capture and releases like Mark Cuban and Paul Graham, the spreading influence is enormous.)

Clay Shirky kicked off the event. He’s a fantastic speaker — watch his talk here. His punch line — that successful communities like facebook, twitter, flickr, and wikipedia start small and cohesive (as opposed to large and fragmented: see Yahoo! 360) — was aimed perfectly at the many founders and foundreamers in the audience. There were speakers from Mint and foursquare and tutorials on the Yahoo! Application Platform, Yahoo! Query Language (the most popular service), Yahoo! TV widgets, and more. There was a round of Ignite NYC, a barrage of twenty-slides-in-five-minutes talks, some educational (geek’s guide to patents), some charitable (aid to South America), some hilarious (spaceman from outerspace), some thought provoking (makerbot 3d printers), and many all of the above (meta mechanical turk; the Emoji translation of Moby Dick). Watch the Ignite talks here.

A bunch of small touches made the event memorable, including a steampunk-themed hacking hall complete with retroRed Victorian couches, portraits of hackers through history, funky tweet-streaming sculptures, chalk drawings of old patents, power cords dangling from hanging bird cages, and a guitarhero-foosball corner. The food was tasty and at times eccentric, like the hot dog stand and toppings bar under a rainbow umbrella, ice cream cart, and old-fashioned popcorn machine. There was plenty of beer, coffee, red bull, sliders, and cookies, and even (gasp) vegan fare, salmon, and salad.

I give the event an A for style (decor, food) and content (talks, hacks, organization). The one sour note was the wireless — certainly a key ingredient for a good hack day — which began flaky and ended slow but acceptable.

I attended the YAP tutorial and created a rudimentary application. I was pleasantly surprised how simple the process was — the documentation and sample code are great. You can get the hello world app (complete with social hooks) running and add some ajax magic within minutes.

By far one of the coolest sights was the MakerBot Industries 3D printer in action. It sucks in plastic wire, melts it, and deposits it in perfect formation to produce coins, busts, parts for itself, or almost anything in the thingiverse. For Hack Day, the device printed news headlines in peanut butter on toast. We met an nyc resistor who was working on a conveyer belt mechanism for his own MakerBot printer, and he invited us to craft night at their shared hackspace in Brooklyn (a place that would be heaven for my dad and brother; Sharad, Jake, Daniel, and Bethany went to check it out).

I missed the tutorial on Yahoo! TV widgets but I’d like to learn more. They are now in most major TV brands including Sony, Samsung, and LG — millions of sets around the world in the coming months. (The Sony won editor’s choice in the Sept 2009 issue of Wired magazine; the Samsung and LG rated close behind. The sole TV reviewed without Yahoo! Widgets, a Panasonic, was ridiculed for is clunky Viera Cast online interface.) If you’re an internet video startup, like my friend, you need a widget channel. Personally, I’d love to see a sports game tracker that highlights pivotal moments by monitoring in-game betting odds.

Footnote: Two Yahoos made a humorous video (that’s both self-promotional and -deprecating) on what people in Times Square think ‘hacker’ means:

See Paul Tarjan and Christian Heilmann for real definitions.

October 2nd, 2009

Yahoo! Open Hack Day NYC, Oct 9-10, 2009

Yahoo! Open Hack Day NYC 2009Join us on October 9, 2009 at the Millennium Broadway Hotel in New York City for Yahoo! Open Hack Day NYC. Come to listen, learn, and meet, but mainly come to make. Your goal: in 24 hours hackmash something together for bragging rights and prizes. Speakers include Clay Shirky (NYU), Carrie Cronkey (Mint.com), Dennis Crowley (foursquare), and Rasmus Lerdorf (inventor PHP). Register here. It’s free.

The 24-Hour Hackathon begins Friday afternoon. We encourage you to play around with Yahoo!’s Open Platforms and APIs like YAP, YQL, YUI, TVWidgets, our Social APIs, and more. And of course, feel free to use other APIs, developer tools and whatever software/hardware floats your boat…

At the end of the 24 hours, the hackers will have the chance to debut their hack and winners will be awarded with some enviable prizes…

And of course we will keep you well fed and hydrated throughout the two days. There will also be sleeping areas in case you want to take a nap.

Previous: the what and why of Open Hack.

October 2nd, 2009

Upcoming CS-econ events: New York Computer Science and Economics Day and ACM Conference on Electronic Commerce

1. New York Computer Science and Economics Day (NYCE Day)

Monday, November 9, 2009 | 9:00 AM – 5:00 PM
The New York Academy of Sciences, New York, NY, USA

NYCE 2009 is the Second Annual New York Computer Science and Economics Day. The goal of the meeting is to bring together researchers in the larger New York metropolitan area with interests in Computer Science, Economics, Marketing and Business and a common focus in understanding and developing the economics of internet activity. Examples of topics of interest include theoretical, modeling, algorithmic and empirical work on advertising and marketing based on search, user-generated content, or social networks, and other means of monetizing the internet.

The workshop is soliciting rump session speakers until October 12. Rump session speakers will have 5 minutes to describe a problem and result, an experiment/system and results, or an open problem or a big challenge.

Invited Speakers

  • Larry Blume, Cornell University
  • Shahar Dobzinski, Cornell University
  • Michael Kearns, University of Pennsylvania
  • Jennifer Rexford, Princeton University

CFP: New York Computer Science and Economics Day (NYCE Day), Nov 9 2009

2. 11th ACM Conference on Electronic Commerce (EC’10)

June 7-11, 2010
Harvard University, Cambridge, MA, USA

Since 1999 the ACM Special Interest Group on Electronic Commerce (SIGecom) has sponsored the leading scientific conference on advances in theory, systems, and applications for electronic commerce. The Eleventh ACM Conference on Electronic Commerce (EC’10) will feature invited speakers, paper presentations, workshops, and tutorials covering all areas of electronic commerce. The natural focus of the conference is on computer science issues, but the conference is interdisciplinary in nature. The conference is soliciting full papers and workshop and tutorial proposals on all aspects of electronic commerce.

June 17th, 2009

Thank you Bangalore

Sunday I returned from a trip to Bangalore, India, where I gave a talk on “The Automated Economy” about how computers can and should take over the mechanical aspects of economic activity, optimizing and learning from data in the way people cannot, with detailed case studies in online advertising and prediction markets. You can read the abstract, watch archive video of the talk, view my talk slides, browse the official pictures of the event, or see my personal pictures of the trip.

Some say everything’s bigger in Texas (most vociferously Texans). They haven’t been to India. My talk is part of Yahoo!’s Big Thinkers India series — four talks a year from (so far) Yahoo! Research speakers. If the Thinking isn’t Big, the crowds certainly are — the events can draw close to 1000 attendees from, apparently, all over India. Duncan Watts says its the largest crowd he’s spoken too; me too. This time they disallowed Yahoo! employees to attend the main event and the hotel ballroom still filled to capacity.

Here is a linked-up version of my journal entry for the trip, a kind of windy and winded thank you letter to Bangalore. If you’re not interested in personal details, you might skip to Thoughts on Bangalore.

Getting there

The Philadelphia airport international terminal is dead empty. I breeze through security — the only one in line. I’m inside security two hours early thinking that either the recession is still in full force or traveling internationally on a Monday night out of Philadelphia is the best ever. Maybe not. Get on plane. Wait two hours on tarmac. Apparently a two hour layover isn’t enough leeway on international flights. Miss my connecting flight in Frankfurt by a few minutes. Team up with a fellow passenger in the same boat. We are rebooked via Dubai. Fly directly over Bagdad. Dubai is an impressive airport. Endless terminals lined with upscale shopping. Packed with Asians, Europeans at midnight and beyond. From there, Emerites Air to Bangalore. Only 9 hours behind schedule. Sneezing fits begin after 28 hours of airplane air.

Day 0: Yahoo! internal practice talk

Driver right there outside baggage claim, nice guy. Takes me to hotel. Over an hour. Traffic. Time for shower, NeilMed nasal rinses (bottled water), Sudafed, but not sleep. Call home. Yahoo! Messenger with Voice doesn’t roll off the tongue like ‘Skype’, but it rocks. Super clear and dirt cheap. Lauren and the girls are so sweet. Miss them. To Yahoo! office. Meet Anita, Mani. Time for Yahoo! internal version of Big Thinkers talk. Nose is still running. Drips and wipes during my talk. Talk goes well but I run out of time for prediction market section and this seems what people are most interested in. I’m glad I had the practice run to work out the kinks and rebalanced the talk. Back to hotel. Call home again for a recharging dose of home. I missed Ashley’s graduation from pre-school: she did great: they sang six songs and she knew them all. She was dressed up in a yellow cap and gown. I’m upset I had to miss such an adorable milestone but am proud of my little girl (and dismayed she is rapidly becoming not so little!). More NeilMed. Room service. (Called “private dining” here — sounds illicit.) Sleep! For a few hours at least. Wake up in the middle of the night since it’s NY daytime. Finally get back to sleep again.

Day 1: Meetings

Hard to wake up at 9am = midnight. Shower. Feel 1000% better. Driver takes me to the Yahoo! office. It’s in a complex with Microsoft, Google, Target, Dell, and many other US brands. Once you’re inside it’s like every other Yahoo! office except the food — built essentially to corporate spec. Meet with Anita, Raghu, and Rajeev: go over PR angles and they brief me on the media interviews. These guys and gal are on top of things. Meet with Mani and her team: great group. Skip intern pizza talks because I can’t eat cheese, going for the cafeteria instead. Mistake. Order a veggie grill thinking that since it’s grilled, it’s cooked enough. I only take a few bites of this before thinking it’s too risky. I eat some bread and Indian mixtures. Not sure what the culprit is but something doesn’t sit well in my stomach. Give prediction markets portion of my talk to a few interested people in labs. Very sharp group. Meet with Dinesh and Sachin, their intern, and one other. Interesting work. Meet with Chid and Preeti on Webscope. Back to hotel. Call Lauren. Good to hear her voice. Ashley wants to say hi. She’s so adorable. She finds it hilarious that I am about to have dinner while she is eating breakfast. I can hear her laughing uncontrollably at the thought. Sarah says hi too and even ends our conversation without prompting with a “bye, love you”. I go down to the restaurant for dinner. Have a chicken Indian dish with paratha (is it lachha paratha?) bread. Spicy (sweat inducing) yet so delicious. The bread is fantastic — round white with flaky layers. Back to room. TV. CNN. CNBC. ESPN. Hard to sleep. There is an incredible thunderstorm with torrents of rain. I open my balcony door briefly to catch its power. I find out later that monsoon season is just beginning. I also find out that it rained so hard and so long that the roads flooded to the point of becoming impassible. In fact, Anita, the Bangalore PR lead, had a near-disastrous experience in the rapidly flooding streets on her way home and had to turn back and check into a hotel before going home briefly in the morning and then back to Yahoo! for our am meeting. Finally get to sleep.

Day 2: My talk!

Hard to wake up at 8:30am too. Talk’s today! Nerves begin. Media interviews are first! Even worse. Turns out they went fine. Two nice/sharp reporters, especially the second one who really knows her stuff and spoke to us (Rajeev and I) for 1.5 hours. She’s especially interested in the prediction market stuff since that is something new. She may write two articles (for Business World India). Lunch, then a bit of time to rest and freshen up. Stomach is not doing well. Pepto to the rescue. Back down to lobby. They take my picture in the courtyard. Then into the ballroom. Miked. Soundchecked. They accept a final last minute change to my slides: hooray! Room starts filling. 100 people. 200. 300. Now 500. It’s time to start! Rajeev gives a very nice intro. I walk up the stairs onto the stage. I’m miked, in lights, speaking in front of 500 people expecting a Big Thinker. Here I go! “Four score and seven years…” Ha ha. Actually: “Thanks Rajeev, and thanks everyone for your time and attention. I am happy and honored to be here. I’m going to talk about trends in automation in the economy…”

David Pennock speaking at Yahoo! Big Thinkers India June 2009Audience at Yahoo! Big Thinkers India June 2009

65 minutes later “Thank you very much.” Applause. I think it went well: one of my better talks. I covered everything, including the prediction market stuff. It turns out, like at Yahoo!, and like the journalists, the audience is more interested in prediction markets than advertising. Lots of questions. Some I follow, some I can’t parse the words, others I hear the words but just don’t understand. I do my best. Several people mention they follow my blog: gratifying. After the official Q&A session ends, there is a line up of folks with questions or comments and business cards. It’s the closest I’ll ever be to a rock star. A handful of people wait patiently around me while I try to get to everyone. Eventually the PR folks rescue me and take me to a “high tea” event with Yahoo! Bangalore execs and some recruiting targets. Relief and euphoria kick in. It’s over. I talk with a number of people. I make my exit. Private dining. Call home. Lauren has explained to Ashley that I am on the other side of the world, so when she has the sun, I have the moon. So I can hear Ashley asking in the background, “does Daddy have the moon?” I do. She can’t stop laughing. A repeat of game 6 of the Stanley Cup is on Ten Sports India. I watch it, getting psyched for Game 7. I check online for Ten Sports schedule. Game 7 will be on at 5:30am! I can’t miss that! Set my alarm. Try to sleep. Can’t sleep. Try to sleep. Can’t sleep. Try with TV on. Can’t sleep. Try with TV off. Can’t sleep. Finally fall asleep… Alarm!

Day 3a: Penguins win the Stanley Cup!

Really hard to wake up at 5:30am. Actually maybe not quite as hard since it’s 8pm in my head. Game on! Nerves are racked up. Can’t sit down: bad luck. Pacing. No score first period. Tons of commercials, all for Ten Sports programming: wrestling, cricket, tennis. Every commercial repeats three times. Is period two coming? Yes, it’s back on! Pens score first! Fist pumping and muted cheering. Can they really do this? No sitting rule in full effect. Pacing. Pens score again! Talbot second goal. Wow, is this real? Can it be? Don’t think about it yet. Don’t celebrate to soon. Plenty of time left. Period two end at 2-0. Unbelievable. All the same commercials come back, three times each. Period three begins. Stand up. Pace. Clock ticks. Pens are playing too defensive: not taking shots, just throwing the puck out of their zone. This isn’t good. Detroit is getting tons of chances. Fleury is awesome. Five minutes left. I let myself think about winning the cup. Mistake! Detroit scores! It’s 2-1! Nerves are ratcheted up beyond ratcheting. I think about it all slipping away. How awful that would feel. If Detroit ties it up, imagine the let down, the blown opportunity. Clock ticks. More chances. More saves. More defense. It’s working! Detroit pulls their goalie. Pressure. Final seconds. Faceoff in our zone. Detroit wins control. Shot. Rebound. Right to a Red Wing — Nick Lidstrom — in perfect position. He shoots. Fleury swings around. He saves it! It’s over! Pens win the Cup! Super fist pumping, jumping around, dancing, muted cheering. They did it! How amazing it feels after last year’s loss to the same team. After falling behind 2-0 and 3-2 in the series. They came back! A delicious payback with the same but opposite script as last year: a two goal lead cut in half in the waning minutes, a flurry of attempts at the end including a few-inch miss of the tying goal in the last seconds. These guys are young and have the potential to rule hockey for several years if they’re lucky. Mario Lemieux is on the ice. How sweet. Twice as player, now as owner, the one who saved hockey in Pittsburgh. What a year for Pittsburgh sports! Two nail biter games, two comebacks, two championships. City of Champions again. Too bad the Pirates have no shot to join them in a trifecta. Back to sleep.

Day 3b: Sightseeing

Phone rings at 11am — my driver is here. Off to do some whirlwind sightseeing. Everyone here who finds out I have a day off recommends I leave Bangalore — Bangalore is just not that nice, nothing really to see, they say. They all recommend Mysore, 3.5 hours away, but that is too far for my comfort level given that my flight is late tonight and it’s supposed to thunderstorm. We start with some souvenir shopping on “MG Road”. My driver takes me to a store and waits in the car outside. I walk in an instantly there are people greeting me and showing me things. One aggressive man takes over and remains my “tour guide” through the whole store. The fact that I reward his aggressiveness by following along and eventually buying stuff will only bolster him to do more of the same in the future. Annoying but clearly it works. I do negotiate him down, but I leave still feeling I didn’t bargain hard enough and with a bit of distaste in my mouth that I fueled and validated the pushy tactics. Next we drive past parliament and the courthouse. Impressive, large, old buildings. But I can just gaze and take photos from the car — can’t go inside. Next we drive past Cubbon Park — tree lined paths and flower gardens in center city. Next is ISKCON temple. But it’s closed. So one more round of shopping at a place called Cottage Industries. I’m wary given the last experience, but go anyway. This one is better. Again one person escorts me around but I feel less pressure. Plus I’m more prepared to say no and negotiate harder. I leave with what seems like a fair amount of value in goods. I recommend Cottage Industries to future visitors: more professional, more familiar (items have price tags), lower pressure, greater variety, and higher quality than at least the first shop I visited. Now we’ve killed enough time and the ISKCON temple is open. It’s a giant Hare Krishna temple. The parking lot is full. I tell the driver it’s ok — we don’t need to go. He says “you go, you go”. “Ok” I say. We drive around again to the same full parking lot. The attendant waves at us to leave, blowing a whistle. My driver is talking to him. They are talking quite heatedly. The attendant in his official looking uniform is waving us on vigorously. Although I can’t understand the words, he is clearly telling us the lot is full and we must leave immediately — we are holding up traffic. My driver is getting more insistent. They are yelling back and forth. I have no idea what he says but it works. The guard let’s us in. Meanwhile another car sees our success and tries to argue his way in too but to no avail. I ask my driver what he said: he simply replies “don’t talk”. Indeed once we’re in, there is an empty spot. We put all my bags in my suitcase in the trunk and cover my backpack. We take off our shoes and my driver leads me to the temple. He knows the back entrance and is guiding me to cut in front of lines everywhere. We walk past the main attraction: the altar with some people on the floor worshiping. Then the line weaves past a gift shop of course: I buy a crazy looking book (Easy Journey to Other Planets). We need to kill some time. We go to the gardens again to walk around. We walk into the public library. Most books are in English. Most seem old and worn. The attendant says the library is 110 years old. We start walking through the garden but I am paranoid about mosquitoes/malaria so we turn around early to return to the car. We go to UB City where I meet Rajeev. It’s a thoroughly modern office tower half owned by Kingfisher of Kingfisher Airlines. The building is full of high-end shopping like almost any upscale western mall with all the same brands. Here is the Apple Store. Here is Louis Vuitton. We have dinner at an Italian restaurant that could be anywhere in the western world, owned by an Italian expat. The only seating is outside and I remain worried about mosquitoes but don’t see any. The food is good and the conversation is good. This place is the closest I’ve seen of the future of Bangalore. In the center of town, a gorgeous building filled with gleaming shops and tantalizing restaurants and bars, with apartments and condos within walking distance, and a palm-tree-lined street leading to the central town circle and the park. As Rajeev says, though, whereas New York has hundreds of similar scenes, Bangalore has one. For now.


Thoughts on Bangalore

Bangalore is a city of jarring contradictions, a hard-to-fathom mix of modernity and poverty. Signs with professional logos and familiar brands are set askew on dilapidated shacks and garages lining the road. While many live on dollars and day and others beg, the majority are smartly dressed (men invariably in button-down shirts), have mobile phones, and are intelligent and friendly. There are gleaming office towers indistinguishable from their western counterparts, yet a strong rain can flood the roads to the point of become impassible for hours and day-long blackouts aren’t uncommon. Many billboards are in English, sporting familiar brands and messages. Others, like sexy stars promoting a Bollywood film, are entirely familiar, English or not. Others are impenetrable. Still another advertises a phone number to learn why Obama quoted the Koran.

BMWs and Toyotas join bikes, motorcycles, pedestrians, aging trucks and buses, and colorful open-air motorized rickshaws in a sea of disorganized line-ignoring sign-ignoring traffic. People drive here the way New Yorkers walk sidewalks: weaving past one another in a noisy self-organized tangle that somehow — mostly — works. You can eat outside in a restaurant bar next to upscale shops, a fountain, and smiling yuppies, yet worry that a malaria-infected mosquito lurks nearby or that a washed vegetable will turn a western-coddled stomach deathly ill. When two people ride a motorcycle, as is common, only the driver wears a helmet — the passenger clinging on behind does not: new and old rules on display atop a single vehicle. And the traffic. Oh, the traffic. Roads are clogged nearly every hour of every day. My Saturday of sightseeing was as bad or worse than weekday rush hour. The extent of congestion itself illustrates Bangalore’s two faces: so many people with youth (India is one of the youngest countries in the world), energy, purpose, and the means and intelligence to accomplish it overtaxing a primitive infrastructure. Buildings are going up according to western specs, but under old-time rules where corruption reins and bribery is an accepted fact of life by even the western-educated aspirational class (about 20% and growing, according to Rajeev).

Thoughts on Yahoo! Labs Bangalore

The folks I met are impressive. Rajeev has done a great job hiring talented, driven folks. Mani’s group of research engineers is fantastic. One is headed to Berkeley for grad school and asks great questions about CentMail. Another proposes an attack on Pictcha. Another (Rahul Agrawal) has read up deeply on prediction markets, including Hanson’s LMSR.

Thoughts on the Yahoo! Big Thinkers India program

The whole event was organized to precision. Anita, the PR lead, was incredible. I especially appreciated the extra “above and beyond” touches like having someone pick up Yahoo! India schwag for my family and send it to my hotel after I forgot: so nice. Raghu, who arranged the media interviews, is supremely organized and on top of his game. The fact that the event draws such a large crowd shows that there is great thirst for events like this in Bangalore. I’m not sure whose idea it is, but it’s a brilliant one: great marketing and great for recruiting.

Thank you Bangalore

In sum, thanks to the people of Bangalore for a fascinating and rewarding trip. Thanks to Rahul at the travel desk whose instant replies about the driver arrangements calmed my nerves on the stressful day of my departure. Thanks to the Yahoo! folks who arranged and organized my talk, and the Yahoo! Labs members for seeding an exceptional science organization. Thanks to my driver who got me everywhere — including into full parking lots, back entrances, and fronts of lines — with efficiency, safety, and a smile (when I tipped him, I tried to think wwsd: what would Sharad do). Thanks to those who attending my talk and whom I met afterward: it’s gratifying and invigorating to see your level of interest and enthusiasm (and your numbers). And thanks Bangalore chefs for keeping any stomach upset relatively mild and brief.

At the airport on the way out, the flight is overbooked and they are offering close to US$1000 plus hotel to leave tomorrow. Not a chance. It’s been fun and an adventure but my nerves are on high and I miss my family: it’s time to make the 20+ hour journey home.

May 20th, 2009

Applause please

I recently spent two days at an economics workshop. In some ways it felt like visiting a foreign country. For one, the audience doesn’t clap. Especially when the speaker ends with “thank you”, the silence is deafening. I hadn’t realized how instinctual the reaction to applaud had become. Of course, it’s arbitrary whether a community claps or not when one of its members concludes a speech. If a community always claps for every speaker, the signal is meaningless as a gauge of satisfaction, like restaurant patrons tipping 18% regardless of service. In fact, almost surely the speaker is just as grateful to have the attention as the audience is to receive the information. It’s not like a political rally where clapping indicates loyalty. Still, it seems like a nice gesture with near zero cost, so why not? Maybe it’s because computer scientists are generally poor speakers that we like to reassure one another. It reminds me of my first international flight. When we landed, all the passengers cheered — the tradition on international flights at the time and apparently at one time on all flights. It seems that now even international flights do not culminate in a round of applause for the pilot. I find it sad that apparently “don’t clap” is the stable equilibrium.

Second, each session was organized with two presentations followed by a lengthy review given by a “discussant”, usually a senior member of the community. I found the format useful: the discussant highlights the main points of the papers in a different voice, helping to reinforce the message, and provides some of their own opinions and insights. The main drawback is that covering two papers takes a full hour and a half, with almost no time for questions and discussion from the audience.

Luckily, even though some of the rituals were foreign, the language was familiar. It so happens that economists and computer scientists speak a remarkably similar dialect of math. Those of us working on market design are especially close: we inhabit similar circles at meetings, universities, and now industry labs (“mini universities” according to Susan Athey) like Google, Microsoft, and Yahoo!, and even co-author papers. Al Roth may have inadvertently suggested why. He encourages thinking of economics as engineering. Computer “science”, like the design branch of economics, seems less science than an amalgam of math, engineering, and art.

May 1st, 2009

Where to find the Yahoo!-Google letter to the CFTC about prediction markets

At the Prediction Markets Summit1 last Friday April 24 2009, I mentioned that Yahoo! and Google jointly wrote a letter to the U.S. Commodity Futures Trading Commission encouraging the legalization of small-stakes real-money prediction markets, and that Microsoft had recently written its own letter in support of the effort.

I told the audience that they could learn more by searching for “cftc yahoo google” in their favorite search engine, showing the Yahoo! Search results with MidasOracle’s coverage at the top.2

It turns out that was poor advice. 63.7% of the audience probably won’t find what they’re looking for using that search.3


Yahoo! versus Google search for "cftc yahoo google"

If some search engines don’t surface the MidasOracle post, I’m hoping they’ll find this.

And back to the effort to guide the CFTC: I hope other people and companies will join. The CFTC’s request for help itself displays a clear understanding of the science and practice of prediction markets and a real willingness to listen. The more organizations that speak out in support, the greater chance we have of convincing the CFTC to take action and open the door to innovation and experimentation.

1Which I hesitated to attend and host a reception for and now regret endorsing in any way.
2In September 2008, journalist Chris Masse uncovered the letter on the CFTC website before Google or Yahoo! had announced it. We should have known: Masse is extraordinarily skilled at finding anything relevant anywhere, and has been a tireless, invaluable (and unpaid) chronicler of all-things-prediction-markets for years now.
3Even Microsoft Live has the “right” result in position 3. Interestingly, Daniel Reeves got slightly different, presumably personalized, results in Google, even less excuse for not knowing what two MO junkies were looking for with that query.
October 10th, 2008

NYCE Day: Thanks and thoughts

NYCE Day 2008 went very well, with over 100 attendees, great talks, and valuable discussion. Many thanks to the four plenary speakers — Costis, Asim, Susan, and Tuomas — and ten rump session speakers who came in from various NYC suburbs like Boston, Pittsburgh, and Palo Alto.

At dinner the night before,1 the organizers agreed that we were nervous because we weren’t at all nervous. Karin and Renee from the New York Academy of Sciences had taken care of almost everything, leaving little for us to fret about. It turned out we were right to not worry and wrong to worry about not worrying: indeed Karin, Renee, and NYAS were absolutely fantastic, orchestrating every detail of the event flawlessly, from technology to catered breaks. The venue itself is gorgeous — a well laid-out space in a modern building in the World Trade Center complex with stunning views2 and a number of nice touches, from an alcove with a computer station to check email to a subtle gradient in the wallpaper that slowly pixilates as your gaze moves from the center toward the side of the room. I came away incredibly impressed with NYAS and delighted to become a member.

Muthu provides an excellent summary of the event, divided into before and after lunch. Read that first and then come back here for my additional thoughts/notes:

  1. Costis gave us mostly bad news. He summarized some of his award winning work with Christos Papadimitriou and Paul Goldberg proving that computing equilibrium behavior in almost any moderately complex game may be beyond the reach of our computers,3 let alone our brains. As a saying goes, “if your laptop can’t find it, then neither can the market” [attribution: Kamal Jain?]. Still, all may not be lost. These results, as is the nature of computational complexity results, say only that some games are extremely hard to solve, not all games or even most games. Since nature is not adversarial (Murphy’s Law aside), it may be the case that among games that arise in the real world that we care about, a number of them can be solved for equilibrium. The problem is defining what “realistic” means in this context: an almost impossibly fuzzy task. Costis did end with some positive results, showing that anonymous games can be solved efficiently. Anonymous games crop up in realistic situations, for example in analyzing traffic, where only the quantity of cars near you matters and not the identity of the drivers inside.

  2. Asim described a sophisticated Bayesian model well suited for social network data that handles non-existant links — meaning the lack of connection between two people, by far the most common situation — much better than previous approaches. The approach is good for digging deeply into a small data set but at least for now has difficulty with moderately large amounts of data. (To get results in a reasonable amount of time, Asim had to down sample his already fairly modest sized corpus.) The talk didn’t help me overcome my bias that Bayesian methods ala UAI often don’t work well at Internet scales without modification.

  3. Susan gave a fantastic and energetic talk. She advocates economic models of online advertising that include more sophisticated users, as opposed to typical models that assume users scan from the top of the page down in a precise sequence. She went further to claim that users may actually choose their search engine based on the quality of the ads. Personally, I’m a bit skeptical about that, though I do agree that there is an indirect effect: search engines with better paying ads can afford to buy more traffic and improve their algorithmic search more. Susan highlighted the enormous shift in mindset required between economic theory and practice when just computing the mean of a data stream can take weeks (though this is changing with tools like Hadoop that can bring such computations down to hours or minutes as Sebastien confirms).

  4. Someone asked Tuomas why his expressive commerce company CombineNet uses first-price auctions instead of VCG pricing. He listed four of what he said were dozens of reasons on top of Rothkopf’s thirteen and Ausubel and Milgrom’s list. In fact he went further to say that as far as he knew no real auction anywhere in the world has ever used true VCG pricing for anything more complicated than selling a single good at a time.

  5. For those not familiar, a rump session is open to anyone to speak briefly on any relevant topic. As it turns out, in part because brevity forces clarity, and in part because editorial filtering overweights mediocrity, the rump session is often the most interesting part of a conference. The “NYCE rump” session was no exception, with topics spanning ad auctions, reputation, Internet routing, and user generated content. Ivy Li proposed a clever scheme whereby eBay sellers are motivated to reward buyers for honest feedback. Sebastien presented work with Sihem and I on an expressive bidding language for online advertising with fast allocation and pricing algorithms, with the goal of moving the industry toward an open standard. Sampath Kannan on leave at NSF had encouraging news on the funding front, laying out his vision for CS theory funding with an explicit call for proposals at the boundary of CS and economics.

  6. I think we did a good job of attracting a diversity of speakers and participants, with talks ranging from computational complexity to Bayesian models of social networks, with academia and industry represented, and with CS, economics, and business backgrounds represented.
1We had dinner at Gobo, a fantastic restaurant Muthu recommended that truly opened my eyes in terms of the tastes and textures possible with a vegetarian menu. Delicious.
2Speaking of views, I had a stunning and fascinating one from my hotel the night before, looking straight down onto ground zero of the World Trade Center complex from a relatively high floor of the Millenium Hilton (apparently intentionally misspelled). I booked the room for $185 on Hotwire, and then found out why. Though the WTC site still looks nearly empty, builders appear to be making up for lost time with round the clock construction. Put it this way: the hotel kindly provided complementary earplugs. All in all though the room and view were well worth the cost in dollars and sounds.
3Specifically, computing Nash equilibrium is PPAD-complete for most games. In terms of complexity classes, PPAD is a superset of P and a subset of NP. Almost surely there is no polynomial time algorithm, though the problem is not quite as hard as the classic NP-complete problems like traveling salesman.
October 1st, 2008

Reminder: NYCE Day and nicer views Oct 3 2008

If you were wavering on whether to attend NYCE Day (New York Computer Science and Economics Day) this Friday October 3, take a look at the New York Academy of Sciences venue where the event will be held. It’s spectacular.

Hope to see many folks there.

New York Academy of Sciences view

September 9th, 2008

Yahoo! Open Hack Day Sunnyvale, Sept 12-13, 2008

Yahoo! Open Hack Day 2008 As Jed points out, “an idea is only the first step in innovation, and it’s by far the easiest step”.

Yahoo! Hack Day was created precisely to summon and celebrate the hard step of innovation: the build it step. The goal is simple: take an idea and make it real — in 24 hours. Spend all day and all night coding until a working, useable, if brittle prototype of your idea emerges. Then show it off.

Hack Day is a religion inside Yahoo!, but on September 12-13, 2008, Yahoo! will open up its Sunnyvale campus, inviting any developer who feels like it to join the geek-out frenzy. Sign up here.

Schedule

8am-6pm PT Friday: Over 20 workshops covering YUI and the newest API offerings from Yahoo!, including BOSS, SearchMonkey, Fire Eagle, and more, and previews of what’s next.

8pm Friday: A surprise musical guest takes the stage (it’s not 2006 guest Beck, but apparently the lyrics are “hacker-friendly” and “may not be appropriate for young children”). Hacking continues all night.

2pm Saturday: Judging, including a special hack-off for the winners of the University Hack Days.

Saturday evening: Awards.

History, thoughts, and notes

At the first Open Hack Day in 2006 in Sunnyvale (see photos), 400 developers fueled by 500 pizzas and a live Beck performance cranked out 54 hacks. At Open Hack Day London lightening struck twice and it rained indoors. Bangalore followed.

If you’re a student, Yahoo! Hack Week may be coming to a campus near you. We’ve held Hack Weeks at Georgia Tech, CMU, UIUC, UC Berkeley, and Stanford, and I believe Waterloo is next. Here’s a quote describing these Hack U events: “Computer science students fueled by fast food, ultra-caffeinated beverages, and alternative music, are free to let their imaginations run wild, tapping the Yahoo! library of APIs to create hacks that advance the Internet experience.”

Why Hack Day? Many an engineer join Jed in lamenting how the PowerPointy set co-opted the term innovation, rendering it almost meaningless. Hack Day was created in part to reclaim innovation for the makers.

Why does it work? Hack Day is to programmers what NaNoWriMo is to writers: a timed artistic challenge that on it’s face is a ludicrous and artificial pretense for accomplishing a goal. Yet somehow the exercise induces a psychological state perfect for making progress on the initial “80% phase” of creation. The punishing deadline forces all meta processing aside — no critic, no perfectionist, no planner, no lazy dreamer — and encourages the raw energy embodied by Nike’s Kirk-beats-Spock slogan “just do it”.

A number of Yahoo! products, services, and features were born on a Hack Day (here are two). Yoopick was too.

Why open?

Openness is one of only three overarching goals for Yahoo!. The other two goalsstarting point for users and must buy for advertisers — are in some sense incontrovertible, yet the openness goal reflects a riskier “if we build it they will come” stand that’s grounded in Yahoo!’s respect for and debt of gratitude to Internet culture. Open Hack Day, Hadoop, Pig, cloud computing, academic relations, publications, APIs, BOSS, SearchMonkey, YUI, Pipes, OpenSocial, and OpenID are among the many examples showing that Yahoo!’s commitment to openness is real. (Jeremy, RIP 2008, said it better.)

Update 2008/09/11: Blueprint mobile SDK and more Y! Open announcements (music, homepage, mail, ads)

Update 2008/09/26: CNET says: Yahoo Open: Finally, a real answer to Google. Also, Google spouse Kara Swisher gets defensive and rewrites history.

September 5th, 2008

New York Computer Science and Economics Day (NYCE Day) October 3, 2008

We invite participants to the first New York Computer Science and Economics Day (NYCE Day), October 3 2008, at the New York Academy of Sciences, 7 World Trade Center.

NYCE Day is a gathering for people in the NYC metropolitan area with interests in auction algorithms, economics, game theory, e-commerce, marketing, and business to discuss common research problems and topics in a relaxed environment. The aim is to foster collaboration and the exchange of ideas.

The program features invited speakers Asim Ansari (Columbia), Susan Athey (Harvard), Constantinos Daskalakis (MIT), and Tuomas Sandholm (CMU), and a rump session with short contributed presentations.

You can indicate your interest in the event on upcoming.yahoo but official registration should go through NYAS.

Your participation and suggestions are greatly welcome. Please distribute this announcement to people and groups who may be interested.

Thanks,
NYCE Day Organizers
 Anindya Ghose, NYU
 S. Muthu Muthukrishnan, Google
 David Pennock, Yahoo!
 Sergei Vassilvitskii, Yahoo!

P.S. This is one week prior and in the same location as the Symposium on Machine Learning.

P.P.S. For those familiar, NYCE Day is inspired as a Right Coast version of BAGT.

P.P.P.S. The New York Academy of Sciences in a spectacular venue. See for yourself.