Making Rankings Credible: How Ranking Creators Navigate Trials Of Strength
Keywords:
Ranking, credibility practices, knowledge production, trials of strengthAbstract
In this article, we take a closer look at the production of social knowledge beyond conventional academic sites in two NGOs that produce rankings of nation states. Whereas extant literature on rankings has mainly focused on the critique of methodologies or the inherent power of numbers, we suggest viewing them as webs of practices aimed at generating credible knowledge about the world. Using Latour’s concept of trials of strength and applying it to two in-depth case studies, we investigate these practices empirically to unpack how these organisations attempt to create epistemic credibility.
References
Asdal, Kristin, and Hilde Reinertsen (2022) Doing document analysis: A practice-oriented method. Los Angeles, London, New Delhi, Singapore, Washington DC, Melbourne: SAGE.
Austin, John L. (1971) How to do things with words: The William James lectures delivered at Harvard University in 1955. Cambridge, Mass.: Harvard Univ. Press.
Barnett, Michael N., and Martha Finnemore (2004) The power of liberal international organizations, in M. N. Barnett and R. Duvall (Eds.) Cambridge studies in international relations, Power in global governance. Cambridge: Cambridge University Press, 161–184.
Beaumont, Paul, and Ann E. Towns (2021) The Rankings Game: A Relational Approach to Country Performance Indicators, International Studies Review 23(4): 1467–94.
Bhuta, Nehal, Debora V. Malito, and Gaby Umbach (2018) Introduction: Of Numbers and Narratives. Indicators in Global Governance and the Rise of a Reflexive Indicator Culture, in D. V. Malito, G. Umbach, and N. Bhuta (Eds.) Palgrave handbooks, The Palgrave handbook of indicators in global governance. Cham: Palgrave Macmillan, 1–30.
Brankovic, Jelena, Leopold Ringel, and Tobias Werron (2018) How Rankings Produce Competition: The Case of Global University Rankings, Zeitschrift für Soziologie 47(4): 270–88. https://doi.org/10.1515/zfsoz-2018-0118 .
Brankovic, Jelena, Leopold Ringel, and Tobias Werron (2022) Spreading the gospel: Legitimating university rankings as boundary work, Research Evaluation 31(4): 463–74. https://doi.org/10.1093/reseval/rvac035 .
Breidenstein, Georg, Stefan Hirschauer, Herbert Kalthoff, and Boris Nieswand (2013) Ethnografie: Die Praxis der Feldforschung. Konstanz, München: UVK-Verl.-Ges; UVK/Lucius.
Broome, André, Alexandra Homolar, and Matthias Kranke (2018) Bad science: International organizations and the indirect power of global benchmarking, European Journal of International Relations 24(3): 514–39. https://doi.org/10.1177/1354066117719320 .
Broome, André, and Joel Quirk (2015) Governing the world at a distance: the practice of global benchmarking, Review of International Studies 41(5): 819–41.
Callon, Michel (1984) Some Elements of a Sociology of Translation: Domestication of the Scallops and the Fishermen of St Brieuc Bay, The Sociological Review 32(1_suppl): 196–233. https://doi.org/10.1111/j.1467-954X.1984.tb00113.x .
Camic, Charles, Neil Gross, and Michèle Lamont (2011) The Study of Social Knowledge Making, in Social knowledge in the making in C. Camic, N. Gross, and M. Lamont (Eds.). Chicago, Ill.: University of Chicago Press, 1–40.
Christensen, John (2007) Mirror, Mirror on the Wall, who’s the most corrupt of all?, (https://www.researchgate.net/publication/237827314_MIRROR_MIRROR_ON_THE_WALL_WHO'S_THE_MOST_CORRUPT_OF_ALL) (accessed: 10 Oct. 2024).
CCPI Climate Change Performance Index 2024: Background and Methodology (https://ccpi.org/download/climate-change-performance-index-2024-background-and-methodology/) (accessed: 3 May 2024).
Cobham, Alex, Petr Janský, and Markus Meinzer (2015) The Financial Secrecy Index: Shedding New Light on the Geography of Secrecy, Economic Geography 91(3): 281–303. https://doi.org/10.1111/ecge.12094 .
Cooley, Alexander (2015) The emerging politics of international rankings and ratings. A framework for analysis, in Ranking the world. Grading states as a tool of global governance in A. Cooley and J. L. Snyder (Eds.). Cambridge: Cambridge University Press, 1–38.
Cooley, Alexander, and Jack L. Snyder, editors (2015) Ranking the world: Grading states as a tool of global governance. Cambridge: Cambridge University Press.
Daston, Lorraine, and Peter Galison (2007) Objectivity. New York, New York State, Cambridge, Massachusetts: Zone Books.
Davies, William (2015) Spirits of neoliberalism: ‘competitiveness’ and ‘wellbeing’ indicators as rival orders of worth, in R. Rottenburg, S. E. Merry, S. J. Park, and J. Mugler (Eds.) Cambridge studies in law and society, The world of indicators. The making of governmental knowledge through quantification. Cambridge: Cambridge University Press, 238–306.
Dean, Steven A., and Attiya Waris (2020) Ten Truths About Tax Havens. Inclusion and the” Liberia” Problem, Emory Law Journal 70: 1659.
Desrosières, Alain (2015) Retoraction: how indicators feed back onto quantified, in R. Rottenburg, S. E. Merry, S. J. Park, and J. Mugler (Eds.) Cambridge studies in law and society, The world of indicators. The making of governmental knowledge through quantification. Cambridge: Cambridge University Press, 329–353.
Espeland, Wendy N., and Michael Sauder (2007) Rankings and Reactivity: How Public Measures Recreate Social Worlds, American Journal of Sociology 113(1): 1–40.
Eyal, Gil, and Larissa Buchholz (2010) From the Sociology of Intellectuals to the Sociology of Interventions, Annual Review of Sociology 36(1): 117–37.
Falkner, Robert (2016) The Paris Agreement and the new logic of international climate politics, International Affairs 92(5): 1107–25. https://doi.org/10.1111/1468-2346.12708 .
Flick, Uwe (2011) Triangulation: Eine Einführung. Wiesbaden: VS Verl. für Sozialwiss.
FSI Financial Secrecy Index 2022: Methodology. (https://web.archive.org/web/20231119065642/https://fsi.taxjustice.net/fsi2022/methodology.pdf) (accessed: 11 Nov. 2025)
Hamann, Julian, and Leopold Ringel (2023) The discursive resilience of university rankings, Higher Education 86(4): 845–63. https://doi.org/10.1007/s10734-022-00990-x .
Hamann, Julian, and Christian Schmidt-Wellenburg (2020) The double function of rankings. Consecration and dispositif in transnational academic fields, in C. Schmidt-Wellenburg and S. Bernhard (Eds.) Routledge research in transnationalism, vol. 38, Charting transnational fields. Methodology for a political sociology of knowledge. Abingdon, New York: Routledge, 160–177.
Hansen, Hans K., and Anne Vestergaard (2018) On the Contest of Lists and Their Governing Capacities. How ‘Tax Havens’ Became ‘Secrecy Jurisdictions’, in Education by the numbers and the making of society. The expertise of international assessments in S. Lindblad, D. Pettersson, and T. S. Popkewitz (Eds.). New York, London: Routledge, 35–52.
Kaiser, Robert (2014) Qualitative Experteninterviews: Konzeptionelle Grundlagen und praktische Durchführung. Wiesbaden: Springer VS.
Kelle, Udo, and Susann Kluge (2010) Vom Einzelfall zum Typus: Fallvergleich und Fallkontrastierung in der qualitativen Sozialforschung. Wiesbaden: VS Verlag.
Kelley, Judith G., and Beth A. Simmons (2019) Introduction: The Power of Global Performance Indicators, International Organization 73(03): 491–510. https://doi.org/10.1017/S0020818319000146 .
Krause, Monika (2014) The good project: Humanitarian relief NGOs and the fragmentation of reason. Chicago, London: The University of Chicago Press.
Latour, Bruno (1987) Science in action: How to follow scientists and engineers through society. Cambridge, Massachusetts: Harvard University Press.
Latour, Bruno (1988) The pasteurization of France. Cambridge, London: Harvard Univ. Press.
Latour, Bruno (1990) The Force and Reason of Experiment, in H. E. LeGrand (Ed.) Australasian studies in history and philosophy of science, vol. 8, Experimental inquiries. Historical, philosophical and social studies of experimentation in science. Dordrecht: Kluwer, 49–80.
Latour, Bruno (2007) Eine neue Soziologie für eine neue Gesellschaft: Einführung in die Akteur-Netzwerk-Theorie. Frankfurt am Main: Suhrkamp.
Leemans, Rik, and Pier Vellinga (2017) The scientific motivation of the internationally agreed ‘well below 2 °C’ climate protection target: a historical perspective, Current Opinion in Environmental Sustainability 26-27: 134–42. https://doi.org/10.1016/j.cosust.2017.07.010 .
Mansour, Mark Bou (2020) Slander and failed advertising campaigns how tax haven officials responded to the financial secrecy index 2020 (https://taxjustice.net/2020/03/05/slander-and-failed-advertising-campaigns-how-tax-haven-officials-responded-to-the-financial-secrecy-index-2020) (accessed: 3. May 2024).
Mayne, Ruth and Jenny Kimmis (2000) Tax Havens: Releasing the hidden billions for poverty eradication, (https://policy-practice.oxfam.org/resources/tax-havens-releasing-the-hidden-billions-for-poverty-eradication-114611/) (accessed: 10 Oct. 2024).
Medvetz, Thomas (2012) Murky Power: “Think Tanks” as Boundary Organizations, in D. Courpasson, D. Golsorkhi, and J. J. Sallaz (Eds.) Research in the sociology of organizations, vol. 34.2012, Rethinking power in organizations, institutions, and markets. Bingley: Emerald, 113–133.
Mehrpouya, Afshin, and Rita Samiolo (2016) Performance measurement in global governance: Ranking and the politics of variability, Accounting, Organizations and Society 55: 12–31. https://doi.org/10.1016/j.aos.2016.09.001 .
Merry, Sally E. (2011) Measuring the World, Current Anthropology 52(S3): S83-S95. https://doi.org/10.1086/657241 .
Merry, Sally E. (2016) The seductions of quantification: Measuring human rights, gender violence, and sex trafficking. Chicago, London: The University of Chicago Press.
Ringel, Leopold (2021) Stepping into the Spotlight. How Rankings Become Public Performances, in L. Ringel, W. N. Espeland, M. Sauder, and T. Werron (Eds.) Research in the sociology of organizations, volume 74, Worlds of rankings. Bingley, UK: Emerald Publishing, 53–76.
Ringel, Leopold, Wendy Espeland, Michael Sauder, and Tobias Werron (2021a) Worlds of Rankings, in L. Ringel, W. N. Espeland, M. Sauder, and T. Werron (Eds.) Research in the sociology of organizations, volume 74, Worlds of rankings. Bingley, UK: Emerald Publishing, 1–23.
Ringel, Leopold, Julian Hamann, and Jelena Brankovic (2021b) Unfreiwillige Komplizenschaft: Wie wissenschaftliche Kritik zur Beharrungskraft von Hochschulrankings beiträgt, in S. M. Büttner and T. Laux (Eds.) Leviathan Sonderband, vol. 38, Umstrittene Expertise. Zur Wissensproblematik der Politik: 386–407.
Ringel, Leopold, and Tobias Werron (2021) Serielle Vergleiche: Zum Unterschied, den Wiederholung macht, KZfSS Kölner Zeitschrift für Soziologie und Sozialpsychologie 73(S1): 301–31 https://doi.org/10.1007/s11577-021-00750-2 .
Rottenburg, Richard, and Sally E. Merry (2015) A world of indicators: the making of governmental konlwedge through quantification, in R. Rottenburg, S. E. Merry, S. J. Park, and J. Mugler (Eds.) Cambridge studies in law and society, The world of indicators. The making of governmental knowledge through quantification. Cambridge: Cambridge University Press, 1–33.
Savin-Baden, Maggi, and Claire H. Major (2012) Qualitative research: The essential guide to theory and practice. Milton Park, Abingdon, Oxon, New York: Routledge.
Schatzki, Theodore R. (2012) A Primer on Practices, in J. Higgs, R. Barnett, S. Billett, M. Hutchings, and F. Trede (Eds.) Educational Research E-Books Online, Collection 2005-2017, ISBN, vol. 6, Practice-Based Education. A Perspectives and Strategies. Leiden, Boston: Brill | Sense, 13–26.
Seabrooke, Leonard, and Duncan Wigan (2015) How activists use benchmarks: Reformist and revolutionary benchmarks for global economic justice, Review of International Studies 41(5): 887–904. https://doi.org/10.1017/S0260210515000376 .
Shore, Cris, and Susan Wright (2015a) Audit Culture Revisited, Current Anthropology 56(3): 421–44. https://doi.org/10.1086/681534 .
Shore, Cris, and Susan Wright (2015b) Governing by numbers: audit culture, rankings and the new world order, Social Anthropology 23(1): 22–28. https://doi.org/10.1111/1469-8676.12098 .
Timmermans, Stefan, and Steven Epstein (2010) A World of Standards but not a Standard World: Toward a Sociology of Standards and Standardization, Annual Review of Sociology 36: 69–89. https://doi.org/10.1146/annurev.soc.012809.102629 .
Wilbers, Stefan, and Jelena Brankovic (2023) The emergence of university rankings: a historical‑sociological account, Higher Education 86: 733–50. https://doi.org/10.1007/s10734-021-00776-7 .
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2026 Elisabeth Strietzel, Can David Tobias
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
CC-BY-NC-ND
