The Living Thing / Notebooks :

Reputation systems

Karma, credit scores, pagerank…

Usefulness: 🔧
Novelty: 💡
Uncertainty: 🤪 🤪 🤪
Incompleteness: 🚧 🚧 🚧

Reputation systems are systems to work out how reliable a bet someone is by the rankings of society. We seem to have some status-seeking which makes us prone to such systems, and the modern technocratic versions are a potential source of mechanisms for collective accomplishment, and or course, control.

I’m mostly thinking about peer ranking systems here, although of course centrally supplied ranking systems are also reputation systems and do indeed fit in here.

As seen in the wild

Credit scores, rankings, bulletin board karma, China’s vaunted social credit system(s). Risk estimation based on qualities that are not transferred financially (although they might be about financial behaviour). Robin Hanson has speculated about status apps becoming ubiquitous.

Bulletin-board style

Most prominently now, stackexchange has a gamified reputation system designed to get people addicted and incentivised to be productive community members based on formalised status. The system was hard to design and need constant revision, but it seems to do something.

China-style

China has gotten a lot of attention for this recently, I suppose because their systems are being designed from the ground up and do not look like the reputation systems that exist in the West and have faded into the background for Western observers.

Coverage of these new systems can be alarming:

China builds the mother of all online reputation systems:

China is proposing to assess its citizens’ behavior over a totality of commercial and social activities, creating an uber-scoring system. When completed, the model could encompass everything from a person’s chat-room comments to their performance at work, while the score could be used to determine eligibility for jobs, mortgages, and social services.

“They’ve been working on the credit system for the financial industry for a while now,” says Rogier Creemers, a China expert at Oxford University. “But, in recent years, the idea started growing that if you’re going to assess people’s financial status, you should equally be able to do that with other modes of trustworthiness.”

The document talks about the “construction of credibility”—the ability to give and take away credits—across more than 30 areas of life, from energy saving to advertising.

See also WaPo on this theme.

Dev Lewis’ fieldwork on such systems makes them sound far less dystopian.

I’m not expert enough to comment on this disjunct. The intersection with automated surveillance could lead in weird directions. Or not.

The coverage mostly seems to radiate from a civil-liberties authoritarian panic, which might be justified. But one wonders if there is a utopian possibility here. Various formal and informal reputation systems are obviously already everywhere; maybe by designing technocratic ones, we have a chance to design better ones with better biases in. Or maybe this is an attempt to make legible what should not be made legible. Maybe the surveillance will be nasty even without reputation systems getting involved, and the places like China that let the reputation score be front-and-centre will at least get more status value out of the process of monitoring everyone than the less prominent credit rating systems and informal networks of the west?

In terms of practical outcomes, I find it hard to imagine China, for example, iterating as rapidly and effectively on a reputation system as does Stackexchange, and indeed they have much more complicated incentives to design, so it is hard to imagine things being smooth or simple.

Iterative reputation

”My rating of your reputation is worth more because other people rate my reputation highly.”

Pagerank and kin, and their application to people rather than webpages. Possible relation to game theory, network topologies, contagion processes, swarm sensing, recommender systems, worryingly technocratic utopians, and learning from gossip. Important in the practice of science, and other situations of trust.

Refs

Ciampaglia, Giovanni Luca, Prashant Shiralkar, Luis M. Rocha, Johan Bollen, Filippo Menczer, and Alessandro Flammini. 2015. “Computational Fact Checking from Knowledge Networks,” January. http://arxiv.org/abs/1501.03471.

Estrada, Ernesto, and Juan A. Rodríguez-Velázquez. 2005. “Subgraph Centrality in Complex Networks.” Physical Review E 71 (5): 056103. https://doi.org/10.1103/PhysRevE.71.056103.

Kamenica, Emir. 2019. “Bayesian Persuasion and Information Design.” Annual Review of Economics 11 (1): 249–72. https://doi.org/10.1146/annurev-economics-080218-025739.

Kamvar, Sepandar D., Mario T. Schlosser, and Hector Garcia-Molina. 2003. “The Eigentrust Algorithm for Reputation Management in P2P Networks.” In Proceedings of the 12th International Conference on World Wide Web, 640–51. WWW ’03. New York, NY, USA: ACM. https://doi.org/10.1145/775152.775242.

Mahoney, Michael W. 2016. “Lecture Notes on Spectral Graph Methods.” arXiv Preprint arXiv:1608.04845. http://arxiv.org/abs/1608.04845.

Newman, Mark E J. 2003. “The Structure and Function of Complex Networks.” SIAM Review 45: 167–256.

Priem, Jason. 2013. “Scholarship: Beyond the Paper.” Nature 495 (7442): 437–40. https://doi.org/10.1038/495437a.

Tadelis, Steven. 2016. “Reputation and Feedback Systems in Online Platform Markets.” Annual Review of Economics 8 (1): 321–40. https://doi.org/10.1146/annurev-economics-080315-015325.