Category Archives: writing

challenges, economics, fun, oddhead blog, woblomo

World Blogging Year

3 Comments

First: I did it! A perfect 16 out of 31. I completed the (ok, my) World Blogging Month challenge to blog every odd day in the month of March.

Last year WoBloMo leapt out of the gates with five participants but I fell five hours short of the goal. As far as I know only Anthony and I returned for year two. He succeeded too according to official Australian Rules.

Again, I found the exercise worthwhile, clearing a number of items out of my queue, albeit mostly the easy and inane ones (c.f. the barking), and boosting readership.

In fact, I enjoyed it so much that I’ve signed up for World Blogging Year (WoBloYe). I will blog every odd day of every month at least through the end of 2010, starting today.

In fact I have formally pledged to stickk to my goal. Moreover, I am putting my money where my mouth is, PM-style. For every odd day of the month that passes blog-post-free I will donate $100 to my anticharity, the re-election fund for Don McLeroy. If I miss two deadlines in a row, my antidonation will double. Three missed deadlines in a row and it will quadruple, etc.

I’ve enlisted kibotzer’s help and you can follow my progress there. Wish me luck!

Update 2010/04/02: April Fools!

P.S. In all seriousness, read that New York Times article about Don McLeroy. It’s one of the scariest articles I’ve read in a long time. It’s about how ultra conservatives on the Texas board of education are rewriting history and science according to biblical and republican dogma, and how standards in that enormous state can dictate what gets printed in textbooks nationwide. They’ve done things like add Newt Gingrich and delete Edward Kennedy as significant Americans. They’ve banned classic children’s books by Bill Martin Jr. because they confused him with a different Bill Martin, author of “Ethical Marxism”.

It is the most crazy-making thing to sit there and watch a dentist and an insurance salesman rewrite curriculum standards in science and history. Last year, Don McLeroy believed he was smarter than the National Academy of Sciences, and he now believes he’s smarter than professors of American history.

fun, hacking, personal, woblomo

Meet my maker, or Bernie Pennock and the blowtorch fountain

2 Comments

My dad is an original maker. When I didn’t want to pay $200 to replace a broken car key housing, he sent me this vice made out of quarters he fashioned and all the parts I needed to attach it to the key.

Using quarters as a vice to hold a key

Aa biomedical engineer, he led a study showing that a non-invasive mask can save people from respiratory failure as well as intubation. The technique is now common practice and, fittingly, the device helped saved his own life several years ago. He also invented a piezoelectric band to measure heart rate and breathing during sleep more comfortably than electrodes.

He did his Ph.D. dissertation on, in a sense, protien folding, in the days when cut and paste meant scissors and glue. I have an original copy of his dissertation and it’s a beautiful object to behold.

Bernie Pennock's Ph.D. dissertation 1Bernie Pennock's Ph.D. dissertation 2Bernie Pennock's Ph.D. dissertation 3

And what about that blowtorch fountain?

Bernie Pennock and the Blowtorch Fountain

Read about it in this profile of my dad by Maureen Simpson highlighting both his hacker and painter sides.

In his retirement, Bernie Pennock found a way to turn fire into water.
The former medical research scientist said it was just one of the many problems that needed solving in his home, where art has become the answer.
“It’s really the same idea as what I did as a career,” Pennock said of his hobby. “You see a problem and think of how to solve it. I think of what I want to do and how to do it, and then I do it and see if it works.”
Using old brass blowtorches he has collected over the years from antique shops and friends, Pennock constructed a fountain next to the pathway leading up to his front door…
Instead of spitting flames, Pennock’s structure spouts water. He mounted the old-fashioned tools to a sheet of copper and then rigged a water pump and pipes behind it… Pennock said his friends describe the work of art as “very Rube Goldberg.”…
Inside his home — on lampshades, along walls and attached to windows — guests can see numerous examples of the former scientist’s artistic experiments. His most recent obsession, apart from the fountain, has been working with stained glass.
“It all started with this window that looks out on the pool,” Pennock said. “I wanted something that let in light, but wouldn’t allow you to see into the bathroom. When I got an estimate to find out how much it would cost to have someone do a stained glass window, I decided to make my own.”
The multi colored scene is based on a photograph Pennock took of two people walking on the beach. Since then, he has made at least a dozen more windows that include a copy of a Monet painting, the Talmadge Bridge in Savannah and his interpretation of
12 stained glass windows designed by Marc Chagall at the synagogue of the Hadassah-Hebrew University Medical Center in Jerusalem, Israel.
Pennock said he usually buys the windows from a Habitat for Humanity store and gets his stained glass from a supplier in Charleston. The next project he plans to take on is a bamboo sculpture, because he’s running out of windows.
“I dabble in a lot of things,” Pennock said. “I like to invent. I just start from scratch, get ideas and see what happens.”
Among his rules for living, which Pennock painted on leftover floor tiles that hang next to the blowtorch fountain, is fittingly: “Pay attention.”

Oh, my brother and sister are makers too. And my mom a trailblazer. I’ll leave those for another day.

__________
This more personal post inspired because Robin says Tyler says it’s OK.

prediction markets, woblomo

Prediction market news: HSX, Nadex, Predictalot, mailing lists, & EC’10

2 Comments

Some prediction market related news:

  1. The Hollywood Stock Exchange is the latest example of life imitating art imitating life. The venerable HSX, the second oldest play-money prediction market (and oldest that anyone has actually heard of) is getting real. You’ll soon be able bet real money on box office returns and deposit your winnings in First Life. The move has been a long time coming — Cantor Fitzgerald purchased HSX in 2001 with the explicit goal of converting it into a real-money exchange — but Cantor was decimated in the September 11 WTC attacks, and the road to regulatory approval has been slow. The real HSX will of course say goodbye to the virtual specialist and the opening weekend adjust, two facets of the game that make it fun to play, but that create significant amounts of (virtual) wealth out of thin air. The Cantor Gaming group is engaged in other interesting initiatives. They are taking over a sportsbook in Las Vegas and turning it into more of a derivatives exchange with live in-game betting, a step toward my dream of a geek-friendly casino. Interestingly, another company called Veriana Networks is close to launching a competing Hollywood derivatives market called the Trend Exchange.
  2. I’ve recently seen ads in the elevator in my building for Nadex, the new incarnation of HedgeStreet that was acquired, redesigned, and rebranded as the North American Derivatives Exchange. I haven’t checked in on the market since the rebranding — in fact I can’t remember if my HedgeStreet account transfered over. I wonder how widespread their ad campaign is and how Nadex is doing?
  3. Predictalot update: 50,000 people have logged in and 11,000 have made at least one prediction.* They’ve made 91,000 total predictions of 5,500 different kinds. We’ve been written up in NYTimes, VentureBeat, L’Atelier (“C’est ce que fait Yahoo!, avec Predictalot.”), and Wired. People are saying everything from “wicked fun”, “great idea”, and “love the game” to “disaster”. Either way, people care. It’s been a ton of fun and its popularity has wildly surpassed my expectations. The final four is this weekend. Then not much time to get in as many improvements as we can before the World Cup.
  4. If you’re not aware, there is a new prediction market mailing list that strives to be open, transparent, objective, and independent under some simple ground rules. I encourage you to join it. I abandoned the old PM list owned by John Maloney for a number of reasons. The old list clearly operates in a payola style — sponsors of Maloney’s events receive prominent billing. That might be ok, except Maloney bills his list and his events as open, non-profit affairs, yet he charges quite a bit and is evasive about financial details (actually he says he doesn’t keep records). Over the years, Maloney has asked me/Yahoo! to sponsor his events several times, occasionally berating me when the answer is no. He’s made some questionable wikipedia edits to highlight his group. Still, I’ve gone to some of his events and find that very good people go there and give interesting talks, and Maloney is very reasonable and personable face to face. My final straw came when Maloney censored me on his list for criticizing him, admittedly in a snarky way.** So I joined with Emile, Bo, Forrest, Justin, Jed, Adam, and others to start afresh.
  5. The 2010 ACM Conference on Electronic Commerce to be held June 7-11 at Harvard features at least six papers on prediction markets:
    • L. Jian and R. Sami, Aggregation and Manipulation in Prediction Markets: Effects of Trading Mechanism and Information Distribution
    • K. Iyer, R. Johari and C. Moallemi, Information Aggregation in Smooth Markets
    • A. Othman, D. Pennock, D. Reeves and T. Sandholm, A Practical Liquidity-Sensitive Automated Market Maker
    • S. Goel, D. Reeves, D. Watts and D. Pennock, Prediction Without Markets
    • A. Othman and T. Sandholm, Automated Market-Making in the Large: The Gates Hillman Prediction Market
    • Y. Chen and J. Wortman Vaughan, A New Understanding of Prediction Markets Via No-Regret Learning

__________
* I believe one in five logged in users placing a prediction is actually a high conversion rate for a prediction market and a testament to our user interface design. Mike Speiser told me that Bix had a tough time converting their users — who loved to vote — into traders in their American Idol prediction market, one reason why they abandoned the experiment.
** The exchange went like this: Maloney’s assistant Jennifer Hewitt announced that “Crowdcast, the leading provider of prediction market solutions for collective forecasting” was joining Maloney’s latest event. Emile asked “leading… Based on what metric, exactly?”. Then I quipped “leading recent contributor to John Maloney”, which Maloney censored. In Maloney’s defense, the “leading provider” language actually came from Crowdcast’s own website (their meta description tag) and so it appears in the snippet when you search for “crowdcast” in Google.

computer science, fun, prediction markets, woblomo

Gates Hillman Prediction Market: The Movie

2 Comments

From September 2008 to August 2009, Carnegie Mellon graduate student Abe Othman ran a prediction market to forecast when CMU’s two new computer science buildings, Gates and Hillman, would open. Abe designed the market to predict not just the magic day, but the likelihood of every possible opening day (in other words, the full probability distribution), at the time making his the largest prediction market built in terms of the number of outcomes.

Now Abe created a fascinating video showing the evolution of prices over time in his market. You can see qualitatively that the thing actually worked, zeroing in closer and closer to the actual opening day as the market progressed.

Figure 3 on page 7 of Abe’s paper with Tuomas Sandholm in the 2010 ACM Conference on Electronic Commerce conveys similar information.

Evolution of prices in the Gates Hillman prediction market

Despite plenty of precedent, and despite increasing evidence that non-market methods do surprisingly well too,* I still find it astonishing to see a bunch of people play a subtle betting game for nothing but bragging rights or a small prize and end up with something reasonably intelligent.

By implementing a working market used by over a hundred CMU students, Abe learned a great deal about practical yet important details, from the difficulty of crisply defining ground truth (when exactly is a building officially “open”?) to the black art of choosing the liquidity parameter of Hanson’s market maker.** Abe independently created an intuitive interval betting interface similar, and in some ways superior, to our own Yoopick interface and Leslie Fine’s Crowdcast interface. Abe went so far as to interview his top traders in great detail to learn about their strategies, which ran the gamut from building automated statistical arbitrage agents to calling construction crew members to learn inside information. Abe observed that interval betting using Hanson’s market maker leads to very “spiky” prices. Starting from this informal observation, Abe was able to actually prove an impossibility result of sorts that any price function with otherwise reasonable properties must be spiky in a formal sense. See Abe and Tuomas’s paper for the details.

__________
* Our paper “Prediction without markets”, by Sharad Goel, Daniel Reeves, Duncan Watts, and me, will be published in the 2010 ACM Conference on Electronic Commerce.
** Abe has now developed a flexible market maker that automatically adjusts liquidity to match trader activity. The paper, by Abe, Tuomas, Daniel Reeves, and me, will also be published in the 2010 ACM Conference on Electronic Commerce.

finance, fun, ideas, woblomo

Housing arbitrage, or the $1.4 million muse

10 Comments

Has anyone heard of the following trick, which might be called housing arbitrage?

Buy one house at the beach and a second house near a ski resort. You live in the beach house in the winter and the ski resort in the summer. You rent out the beach house in the summer and the ski resort in the winter.* Can your earnings (rental revenue minus mortgage costs) be enough to live on?

Why it could work: the cost of each house will be roughly proportional to the average annual rental income in that location. If you didn’t live in the properties at all, you should roughly break even (income = mortgage payments). But you are living in each location during the time when rent is essentially free (not contributing to the average) so you have no housing costs. If you find good enough deals (or put money down, or have some small income like freelance writing, etc.) your income may exceed your mortgage enough to live on.

What’s the minimum you could get started with on this strategy? Probably a minimum income to live comfortably as a starting point would be $70K before taxes: see justification below. Assume you can make about 5% of a home’s value in rental income: this seems feasible. Then you need $1.4 million invested in real estate (say two $700K houses) with no mortgage (completely paid). Suppose you can also borrow at 5%. Then if you put 50% down on two $1.4 million properties ($2.8 million total), your effective mortgage rate is 2.5% and your “spread” is 2.5%, so you again earn $70K, but now you have two twice as nice houses (but more risk, need to qualify for loan, etc.). Now here is some magic. Suppose you find an incredible deal (say, in a down real estate market) and you can earn 10% in rental income. You can borrow at 5% and only want to put 20% down, still a respectable portion that the bank may be willing to go for. You buy two $600K homes ($1.2 million total) needing only $240K in cash. Now your rental revenue is $120K and your mortgage payments are $48K, so your net income is, viola, $72K!

Didn’t I forget about taxes and insurance? No, I’m just assuming these can be covered by your $70K income. I did forget about health insurance, though: that could threaten the strategy, at least in the United States. You can can hope that the new health care law helps, or keep an enjoyable day job, or purchase insurance out of the $70K.

You might say $70K pretax is not enough to live the lifestyle you want. But remember, you effectively have no housing costs, and this is just meant as a starting point. This is your “muse” as Tim Ferriss calls it: a steady reliable income that is your buffer. You still should pursue freelance ideas or business ideas that you are passionate about, and one of those just might hit it big. This just gives you freedom to pursue other ideas on your own. Hopefully even at $70K you can save some money to purchase additional properties and increase your income. Note that once your mortgage is paid off, your income will go up.

One nice thing about this strategy, and real estate investments in general, is that they are naturally inflation adjusted: rental rates should go up if inflation goes up.

This really only seems practical for people without kids in school. Although I suppose if your kids went to school in the beach location it might work. You’d only spend 2.5 months in the ski resort.

Certainly there are downsides: constantly moving, living in off-season tourist towns, living in properties that are rented half the year, dealing with renters, risk of loss or default, and managing the business headaches.

If housing arbitrage could really work, why aren’t more people doing it? Maybe it requires too much capital and maybe my math is wildly optimistic. Probably it’s no more than a fun mental exercise. I’m sure it’s been thought of. I can’t find it on a cursory web search but it seems hard to articulate to a search engine. If enough people started doing it, by definition house prices would go up to eliminate the arbitrage.

__________
* Maybe take a week or two in the summer at the beach and the winter at the ski house.

advertising, computer science, economics, events, prediction markets, research, woblomo

CS ∩ Econ news

8 Comments

Here are some news items about the field with no name (at least not yet, see below) that lies at the intersection of computer science and economics.

  1. The Sixth Workshop on Ad Auctions is soliciting papers. The workshop will be held June 8, 2010, in Cambridge, MA, in conjunction with the ACM Conference on Electronic Commerce (EC’10). There is a terrific organizing committee this year spanning industry and academia, CS and business schools.

  2. The EC’10 list of accepted papers is out and looks great.

  3. The first-ever Behavioral and Quantitative Game Theory Conference on Future Directions will be held May 14-16 in Newport Beach, CA. The program looks fantastic.

  4. Last fall, the University of Pennsylvania announced the first-ever undergraduate degree program in Market and Social Systems Engineering. Kudos to UPenn: the move shows impressive vision and leadership.

  5. The NSF is funding research in the CS-Econ area. They support efforts to “explore the emerging interface between computer science and economics, including algorithmic game theory, automated mechanism design, computational tractability of basic economic problems, and the role of information, trust, and reputation in markets” (page 7).

  6. The NBER Market Design working group is soliciting papers for a workshop October 8-9, 2010 in Cambridge, MA.

  7. We are now reviewing some amazing submissions to Yahoo!’s 2010 Key Scientific Challenges program. Read the challenges for the area we call Algorithmic Economics.

  8. Members of Yahoo! Labs can submit proposals to fund collaborative research with academic colleagues through the Yahoo! Faculty Research and Engagement program. If you’re interested, contact a Yahoo! Labs employee.

What should be the name? CS ∩ Econ is accurate but cryptic. At Yahoo!, we call it Algorithmic Economics. At Google, they call it Market Algorithms. The ACM Special Interest Group in this area calls it Electronic Commerce, causing complaints every year. I’ve heard people suggest Economics and Computation. The name Algorithmic Game Theory has emerged as something of a standard within the CS theory community. [Update: Noam suggests Algorithmic Game Theory and Economics and even renamed his blog accordingly.] The phrase Computational Economics makes sense but is already in use by a different field. A fun suggestion is Economatics (or Autonomics), meant to invoke a mashup of economics and automation.

Prediction markets had a similar naming/identity crisis. They’ve been called information markets, idea markets, securities markets, event markets, binary options, market in uncertainty, and more. But now almost everyone has settled on prediction markets. I’ve come to like the name and I think it’s helped establish the field in it’s own right. I hope we can settle on a good name for CS ∩ Econ in part so we can create the Journal of PerfectNameForCSEcon, an outlet sorely missing from the field.

Update 2011/10/11: The journal now exists! Called the ACM Transactions on Economics and Computation, it circumvented the naming issue.

finance, woblomo

A beautiful model (of the stock market)

22 Comments

Here is a histogram of the daily changes of the S&P 500 from 1950 to 2009. The x-axis is the daily log difference* and the y-axis is the number of times that difference occurred.

Histogram of log differences of S&P500 from 1950 to 2009

It turns out that a Laplace distribution is a pretty good model of the stock market. The Laplace distribution is parametrized by its median u and its average absolute difference from median b. I computed these two parameters for the S&P 500 data and plugged them into a Laplace distribution in Mathematica, then used that to generate 59 years worth of random simulated S&P 500 prices. Here is the resulting histogram.

Histogram of log differences of simulated S&P500

This is not a best fit: this is simply the same u and b computed on the data and then plugged into the distribution. Here are the two histograms on top of each other.

Histogram of log differences of S&P500 from 1950 to 2009 and simulated S&P 500

At the aggregate level the stock market is well behaved: it’s randomness is remarkably predictable. It’s amazing that this social construct — created by people for people, and itself often personified — behaves so much like a physical process, more so than any other man-made entity I can think of.

The graphs also hint at the futility of attaching reasons to price movements every single day. Today, “the prospects for continued low borrowing costs buoyed investors’ hopes for the U.S. economy”. Tomorrow, there may be “profit taking”.** Why do reporters feel obligated to explain why, or more to the point why do readers demand to be told? Why must we infer reasons, see momentum where there may be none, and assign labels like bull and bear? Why can’t we accept randomness?***

Update: Anthony Towns conducted a fascinating and puzzling follow-up analysis showing that the volatility of the stock market has been going up steadily over the years, even though the mean return has not.
__________
* The log difference is the log of the price on day d minus the log of the price on day d-1, or equivalently ln(d/(d-1)).
** I’d love to see a controlled experiment where financial reporters are given randomized reports about the Dow and watch them manufacture explanations, I imagine occasionally invoking the same cause for both ups and downs… (a) The prospect of higher interest rates spooked investors today; or (b) the Fed’s willingness to raise rates signals a recovering economy — investors rejoiced.
*** Let me clarify. The efficient market hypothesis implies both randomness and rationality: Randomness in the sense of statistical independence (no momentum), and rationality in the sense that price changes reflect new information. So I’m not saying that prices change for no reason, just that there is rarely an easily-stated single explanation for the overall market. (BTW, “profit taking” is never a valid explanation under EMH.)

computer science, economics, finance, fun, gambling, hacking, prediction markets, research, woblomo, yahoo

Let the madness begin

3 Comments

Sixty-five men’s college basketball teams have been selected. Tomorrow there will be sixty-four. Half of the remaining teams will be eliminated twice every weekend for the next three weekends until only one team remains.

On April 5th, we will know who is champion. In the meantime, it’s anybody’s guess: any of 9.2 quintillion things could in principle happen.

At Predictalot it’s your guess. Make almost any prediction you can think of, like Duke will win go further than both Kansas and Kentucky, or the Atlantic Coast will lose more games than the Big East. There’s even the alphabet challenge: you pick six letters that include among them the first letters of all four final-four teams.

Following Selection Sunday yesterday, the full range of prediction types are now enabled in Predictalot encompassing hundreds of millions of predictions about your favorite teams, conferences, and regions. Check it out. Place a prediction or just lurk to see whether the crowd thinks St. Mary’s is this year’s Cinderella.

Come join our mad science experiment where crowd wisdom meets basketball madness. We’ve had many ups and down already — for example sampling is way trickier than I naively assumed initially — and I’m sure there is more to come, but that’s part of what makes building things based on unsolved scientific questions fun. Read more about the technical details in my previous posts and on the Yahoo! Research website.

And for the best general-audience description of the game, see the Yahoo! corporate blog.

Update: Read about us on the New York Times and VentureBeat.

You can even get your fix on Safari on iPhone!

Dave playing Predictalot on iPhone

Below is a graph of our exponential user growth over the last couple days. Come join the stampede!

graph of YAP installs for Predictalot