Measuring the Agreement of Mathematical Peer Reviewers

Axiomathes:1-15 (forthcoming)
  Copy   BIBTEX

Abstract

We investigate the possibility of arguing for or against the philosophical position that mathematics is an _epistemic exception_ on the basis of agreement data from the mathematical peer review process and argue that Cohen’s \(\kappa \), the standard agreement measure used for inter-rater agreement, is unable to detect epistemic exceptionality from peer review data.

Categories

Keywords

Reprint years

DOI

10.1007/s10516-022-09647-x

Other Versions

No versions found

Links

PhilArchive



    Upload a copy of this work     Papers currently archived: 100,752

External links

Setup an account with your affiliations in order to access resources via your University's proxy server

Through your library

My notes

Similar books and articles

When Zero May Not be Zero: A Cautionary Note on the Use of Inter-Rater Reliability in Evaluating Grant Peer Review.Elena A. Erosheva, Patrícia Martinková & Carole J. Lee - 2021 - Journal of the Royal Statistical Society: Series A 184:904-19.
Individual versus general structured feedback to improve agreement in grant peer review: a randomized controlled trial.Ida Svege, Pål Ulleberg, Knut Inge Fostervold & Jan-Ole Hesselberg - 2021 - Research Integrity and Peer Review 6 (1).
Editorial peer reviewers’ recommendations at a general medical journal: are they reliable and do editors care?Richard L. Kravitz, Peter Franks, Mitchell D. Feldman, Martha Gerrity, Cindy Byrne & William M. Tierney - 2010 - PLoS ONE 5 (4):e10072.
Refinement: Measuring informativeness of ratings in the absence of a gold standard.Sheridan Grant, Marina Meilă, Elena Erosheva & Carole Lee - 2022 - British Journal of Mathematical and Statistical Psychology 75 (3):593-615.
Peer review versus editorial review and their role in innovative science.Nicole Zwiren, Glenn Zuraw, Ian Young, Michael A. Woodley, Jennifer Finocchio Wolfe, Nick Wilson, Peter Weinberger, Manuel Weinberger, Christoph Wagner, Georg von Wintzigerode, Matt Vogel, Alex Villasenor, Shiloh Vermaak, Carlos A. Vega, Leo Varela, Tine van der Maas, Jennie van der Byl, Paul Vahur, Nicole Turner, Michaela Trimmel, Siro I. Trevisanato, Jack Tozer, Alison Tomlinson, Laura Thompson, David Tavares, Amhayes Tadesse, Johann Summhammer, Mike Sullivan, Carl Stryg, Christina Streli, James Stratford, Gilles St-Pierre, Karri Stokely, Joe Stokely, Reinhard Stindl, Martin Steppan, Johannes H. Sterba, Konstantin Steinhoff, Wolfgang Steinhauser, Marjorie Elizabeth Steakley, Chrislie J. Starr-Casanova, Mels Sonko, Werner F. Sommer, Daphne Anne Sole, Jildou Slofstra, John R. Skoyles, Florian Six, Sibusio Sithole, Beldeu Singh, Jolanta Siller-Matula, Kyle Shields, David Seppi, Laura Seegers, David Scott, Thomas Schwarzgruber, Clemens Sauerzopf, Jairaj Sanand, Markus Salletmaier & Sackl - 2012 - Theoretical Medicine and Bioethics 33 (5):359-376.
Author's Response-Peer review: Agreement and disagreement.D. V. Cicchetti - 1996 - Behavioral and Brain Sciences 19 (3):536-536.
A Kuhnian Critique of Psychometric Research on Peer Review.Carole J. Lee - 2012 - Philosophy of Science 79 (5):859-870.
“We Share All Data with Each Other”: Data-Sharing in Peer-to-Peer Relationships.Eva Barlösius - 2023 - Minerva 61 (2):243-263.
The ethics of peer review in bioethics.David Wendler & Franklin Miller - 2014 - Journal of Medical Ethics 40 (10):697-701.
The problem of humiliation in peer review.Debra R. Comer & Michael Schwartz - 2014 - Ethics and Education 9 (2):141-156.

Analytics

Added to PP
2023-01-05

Downloads
47 (#467,133)

6 months
11 (#337,502)

Historical graph of downloads
How can I increase my downloads?

Author's Profile

Benedikt Löwe
University of Amsterdam

Citations of this work

Add more citations

References found in this work

The derivation-indicator view of mathematical practice.Jody Azzouni - 2004 - Philosophia Mathematica 12 (2):81-106.

View all 6 references / Add more references