Richard Hollerith here. In the initial versions of this page I partially endorsed an approach (suggested by GreedyAlgorithm) that I become more sceptical of, so I took it out.

Eliezer wrote:

Nick Bostrom and I once took a taxi and split the fare. When we counted the money we'd assembled to pay the driver, we found an extra twenty there.

"I'm pretty sure this twenty isn't mine," said Nick.

"I'd have been sure that it wasn't mine either," I said.

"You just take it," said Nick.

"No, you just take it," I said.

We looked at each other, and we knew what we had to do.

"To the best of your ability to say at this point, what would have been your initial probability that the bill was yours?" I said.

"Fifteen percent," said Nick.

"I would have said twenty percent," I said.

Eliezer went on to describe how he and Nick split the extra twenty, but later conceded that it was not the right (fair) answer.

Of course, to get a numerical answer you have to make some assumptions. I think the following suffice: Nick and Eliezer's priors are the same; neither's estimate is better than the other; neither estimate relies on evidence relied on by the other.

If you have an explanation, please let me see it. Richard Hollerith out.

===

Comment by Hal:

Hello, Richard - I don't actually see a question here. Are you asking what are the correct odds, given this information (and perhaps other plausible assumptions), that the $20 came from Eliezer vs Nick?

Here's one approach. Suppose that the prior, due to symmetry, is that the bill is equally likely to have come from either. However each examines his personal memories of the transaction, thinking about whether he could have introduced an extra bill, and decides that there is strong evidence against that. The formula I would use is odds afterwards equals odds before times likelihood ratio. We will multiply the individual likelihood ratios on the theory that they are independent, each person just estimating his own chance of having accidentally contributed an extra $20.

Using his private information, Eliezer sees 4 to 1 odds in favor of the bill being Nick's. Using his own private information, Nick sees about 6 to 1 odds against the bill being his, or equivalently 1 to 6 in favor. The initial odds are 1:1, so the likelihood ratios are also respectively 4 and 1/6. We multiply the likelihood ratios to combine them and get 4/6 or 2/3. Multiplying by 1:1 odds gives us final odds of 2:3 that the bill is Nick's, or in other words 40% that it is Nick's and 60% that it is Eliezers, to one significant figure.

===

Richard Hollerith here again. Thanks, Hal Finney. I consider your way of arriving at the answer basically sound.

Probably the thing that most persuades me of the soundness of Hal's solution is considering a variation on the above scenario where the probabilities come not from fuzzy subjective considerations but from 'hard, empirical' data:

"I'm pretty sure this twenty isn't mine," said Nick.

"I'd have been sure that it wasn't mine either," I said.

"You just take it," said Nick.

"No, you just take it," I said.

At this point, the cab driver turned around and said, "You guys are in luck! I am not an ordinary cab driver. I am in fact a member of a secret organization which has obtained empirical evidence of how often each of you inadvertently includes an extra $20 bill when you pay a cab fare. In particular, my organization employs cab drivers in several of the cities both of you frequent. Moreover, we have conducted a certain amount of surveillance on you two to increase the probability that when you took a cab, the driver was in our employ. Consequently, I am in a position to tell you, Nick, that not counting this cab ride, you have taken 20 rides in cabs whose drivers were in our employ, and you included an extra twenty in your fare 3 times. Eliezer, you, too, have ridden in one of our cabs exactly 20 times, and you included an extra twenty 4 times.

In this variation, I am highly confident that turning the 4/20 and 3/20 into likelihood ratios and then multiplying (like Hal did) yields the right answer, provided the cab driver was telling the truth and provided (which is very likely) that there is no additional relevant evidence that is not screened off by what the cab driver said.

But in the absence of 'hard, empirical' data, subjective probabilities can be worth using, especially if the person giving the probability is as rational and trustworthy as Eliezer is.

But multiplying likelihood ratios yield the right answer only if the the likelihoods are independent of each other. In my variation above, that is assured. In Eliezer's original scenario, that is much less probable. Independence would have been much more probable if the dialog had gone as follows:

"I'm pretty sure this twenty isn't mine," said Nick.

"I'd have been sure that it wasn't mine either," I said.

"You just take it," said Nick.

"No, you just take it," I said.

We looked at each other, and we knew what we had to do.

"Since you know yourself better than I know you, tell me, what is your probability that you would include an extra twenty when paying for a cab ride like this one? In your answer, do not include the fact that one of us included an extra twenty today and do not include any information you may have accumulated about my propensity to give the wrong amount of cash when paying for a cab ride. That is important because we need a way to avoid double-counting a piece of evidence. If your probability incorporates all evidence you have accumulated about your tendency to overpay and mine incorporates all evidence I have about my tendency, then we avoid double counting."

===

Anyone who wants to add to this page should contact me

changed December 12, 2009