Couchsurfing positive hours
Heard on the Note to Self podcast:
Back in 2007, they tried a system that rewarded the people who spent the least amount of time on their site. Harris explains: "If I was going to Paris and I was staying there for four days, they would estimate how many hours would happen in those four days between me and the person who hosts me in Paris. And then they would ask both people, 'How positive were those hours? Did you have a good time together?' So they're getting kind of a count of the number of positive hours. And then what they do is subtract all of the time that both people spent on Couchsurfing's website. They take that as a cost to people's lives. 'Cause having people search and send messages and look at profiles, they don't view that as a contribution that's positive to people's lives. And what you're left with is just these new net positive hours that would have never existed if Couchsurfing didn't exist."
There's something I find really compelling about this metric as it has some really nice qualities. It's readily understandable, easy to calculate, hard to game (accidentally or otherwise for whatever reason), and doesn't have any obvious failure modes where it would actually cause the wrong thing to happen.
On the drawbacks front, it's obviously not totally precise. While one can argue that it is overly dependent on the fidelity of reports from the individuals that had the experience, but at the same time, perhaps their recollection of the number of positive hours is actually more important than the exact clock time.