Assessing the Quality of Peer Reviews in Public Health: Use of a Standardised Proforma among Online Course Participants

This study, “Assessing the Quality of Peer Reviews in Public Health: Use of a Standardised Proforma among Online Course Participants,” evaluates a newly developed proforma designed to standardize peer reviews of public health research papers. The proforma was implemented within an online peer review training course for African health researchers, where 210 participants completed structured reviews of a public health preprint.

The proforma combined multiple-choice and open-ended questions assessing comprehension, critical appraisal, and editorial judgment. Participant responses were systematically compared with a model answer prepared by course tutors. Results showed that only 13% of participant recommendations matched the tutors’ “approve with major reservations” decision, and only 11% achieved six or more points out of a possible eight on the critical appraisal section. Free-text responses from these high scorers were further evaluated using a five-criteria rubric (mean score = 21/25), showing high inter-rater reliability (ICC = 0.939, 95% CI [0.876–0.972], p < 0.0001).

The findings demonstrate the feasibility of using a standardized proforma as a scalable tool for training and evaluating peer review quality in public health, with potential application in future educational and capacity-building programs.