Peer Review with Multiple Reviewers 1.0.0
Building on the “Peer Review Model”, this ABM looks at the effect of multiple reviewers and their behavior on the quality and efficiency of peer review. It models a community of scientists who alternatively act as “author” or “reviewer” at each turn. Authors’ task is to submit an article and have it published, while each reviewer evaluates one submitted article.
In the “stochastic” version reviewers over or underestimate the quality of a paper according to a fixed probability value. In the “strategic” scenario reviewers’ bias is conditioned on the outcome of their last submitted paper while acting as authors, independently from the identity of the rejecting reviewer.
Reviewers assigned to each paper can be from 1 to 3.
Resources are needed both to submit and review an article. Each scientist has a variable set of resources, which are initially homogeneously distributed. At each tick the agents are endowed with a fixed amount of resources, equal for all. Plus, they cumulate resources according to their publication score.
The quality of author submissions depends on scientist’s resources.
The chance of being published is determined by the average evaluation score assigned by reviewers. Only a fixed number of submissions are published at each turn. If not published, authors lose all the resources invested for submitting. Successful publication multiplies author resources for a mutiplier value.
When selected as reviewers, scientists invest a given amount of resources for reviewing and simultaneously lose the opportunity to publish. Reviewing expenses grow linearly with the quality of authors’ submissions.
Release Notes
Associated publication:
F. Bianchi & F. Squazzoni, “Is three better than one? Simulating the effect of reviewer selection and behavior on the quality and efficiency of peer review”, WSC ‘15 Proceedings of the 2015 Winter Simulation Conference, Huntington Beach, California — December 06 - 09, 2015
IEEE Press Piscataway, NJ, USA, 2015, pages 4081-4089 .