Types of peer review and what the evidence says about bias

Every call for papers carries this information in a single line: “submissions will be assessed by peer review, in the double blind format.” Behind it lies a decision the scientific committee makes before the call opens: who knows whose identity during the evaluation. And that decision is not cosmetic. Experiments with thousands of reviewers show that the chosen format changes the outcome of the evaluation in a measurable way.

This guide presents the four types of peer review, what the evidence says about bias in each one, and what that means in practice for anyone organizing a scientific conference.

The four types of peer review

The entire classification answers a single question: who sees whose identity?

Type	Author anonymous?	Reviewer anonymous?
Single blind	No	Yes
Double blind	Yes	Yes
Triple blind	Yes, even to the committee	Yes
Open review	No	No, and the report may be public

Single blind

The reviewer knows who the author is; the author does not know who reviewed the work. It is historically the most common format in journals, and the one most exposed to the weight of prestige, as the evidence below shows.

Double blind

Neither side is identified: the reviewer assesses the work without knowing whose it is, and the author receives the review without knowing who it came from. It is one of the most common choices at conferences, especially when the audience includes many early-career authors.

Triple blind

Beyond the reviewers, not even the committee (or the editor) sees the authorship during triage and the distribution of submissions. It reduces the channels of bias even further, but the operational cost is high and, in practice, it is rare.

Open review (open peer review)

An umbrella of practices: identities visible on both sides, reports published alongside the work, community participation, or combinations of these. The term is so elastic that a systematic review mapped 122 different definitions in use.⁶

It is not a niche curiosity: it is the frontier of the field, and it grows for a concrete reason.

Galoa is an ORCID Member When your event runs on Galoa, authors and reviewers connect their ORCID with official authentication. Every contribution goes straight to their ORCID record. How our membership works →

Signing the review and recording it in ORCID tends to make the reviewer more rigorous and turns the review into a traceable academic credit. Publishing the reviews and opening space for the author’s reply shifts the focus from “who signs” to what the work actually contributes to the community.

What the evidence says about bias

The choice between these formats does not have to be made by intuition. It has already been tested, including under conditions that conferences know well.

Prestige weighs when it is visible. In the most cited experiment in the field, run at a real computing conference with a 15.6% acceptance rate, each submission was evaluated at the same time by two single-blind reviewers and two double-blind reviewers. The reviewers who could see the authorship significantly favored work from famous authors and prestigious institutions.¹

The name on the cover changes the review. In a field experiment with more than 3,300 researchers invited to assess the same article, the displayed authorship changed everything: when the only visible author was a Nobel laureate, 23% recommended rejection. With the article anonymized, 48%. When the visible author was an early-career researcher, 65%.² The same text, three fates.

Bias is not only about prestige. A systematic review examined the effect of double blind on gender bias in scientific publishing,⁴ and an analysis of more than 36 million articles indexed in PubMed showed that articles by women researchers spend longer under review than those by men in comparable conditions.⁵

And the honest counterpoint: the observational evidence is mixed. A study of computer systems conferences showed that, depending on the methodological choices, the same data support contradictory conclusions about the size of prestige bias.³ What double blind does, demonstrably, is close the most direct channel of bias: the author’s identity visible at the moment of judgment. It does not fix all of science, and it only works while the anonymity is sustained in the operation.

What this means for your conference

There is no single format that stamps a conference as serious: the choice depends on the community, and each option has trade-offs the evidence helps bring into view.

• Double blind is a strong choice, not an obligation. It protects the early-career author (precisely the audience of many conferences) and reduces the weight of prestige in the evaluation. But established conferences also use single blind by choice, and that does not make them less rigorous: the decision belongs to the committee, not a quality seal.
• In a small community, anonymity is always partial. When few groups work on a topic, authorship can be inferred from the very object of the work, even with everything hidden. This does not invalidate double blind, but it shows that the format is about the intent to judge on merit, not an absolute guarantee of secrecy.
• If you choose double blind, the label in the call is not enough. The double blind that the experiments measured presupposes real anonymity. A name left in the file metadata, a shared spreadsheet tab with the wrong column, and the format ceases to exist. That is why peer-review management has to hide identities by construction, not by manual discipline.
• Triple blind demands real automation. The committee has to distribute work by area and manage conflicts of interest, and doing that without seeing the authorship is unfeasible in a manual process, which is why it is usually rare. When distribution and conflict control run by construction in the system, as they do at Galoa, it stops being unfeasible, and the question becomes whether the marginal gain over double blind is worth it, not whether it can be done.
• Open review is the direction open science has been taking. Signed and recorded reviews, author replies, transparency: in journals it is already a reality, and it is reaching conferences little by little. At events with short deadlines and concentrated volume it is still the exception, but it is an exception that grows, not a deviation.

The outcome is also part of the rigor: the work approved by this evaluation deserves to become proceedings with DOIs, a permanent and citable record. As Crossref Sponsoring Members since 2015 and ORCID Institutional Members, Galoa ensures that the result of your event’s peer review ends up aligned with the global indexing standard, with authorship correctly attributed.

The evaluation format is a statement of rigor

When the scientific committee chooses a format deliberately and sustains it end to end, it is telling the community how the work will be judged, and taking public ownership of that choice. The evidence shows that the format changes the outcome of the evaluation in a real and measurable way, whether by closing the channel of prestige or by opening the review to scrutiny. What remains is to ensure that the conference operation lives up to the promise in the call, whatever it may be.