---
name: academic-paper-review
description: Structured critical review of a paper (method, claims, threats to validity) in the voice of a third reviewer.
title: Academic Paper Review
category: research-analysis
difficulty: intermediate
license: Apache-2.0
author: admin
source_url: "https://github.com/allenai/scireviewer"
icon: 🎓
input: pdf
output: markdown
phase: post
domain: research
tags: paper-review,academic-critique,peer-review-automation,claim-evidence-mapping,methodology-assessment,research-validation,structured-feedback,neurips-iclr-calibrated,reproducibility-checklist,reviewer-bias-mitigation
best_for:
  - Machine learning conference submissions (NeurIPS, ICLR, ICML)
  - Academic paper quality assurance and editorial triage
  - Research methodology validation and threat-to-validity assessment
  - Structured peer feedback generation for pre-submission review
---

## Description

A review skill that produces a Reviewer 3-style critique: summary, strengths, concerns bucketed by category (method / stats / novelty / reproducibility), and concrete revision asks. Calibrated against NeurIPS / ICLR reviewer guidelines.

## Why it works

Paper review failure modes are well-studied: vague praise, missing the actual contribution, or surfacing nits while ignoring a broken methodology. Constraining the LLM to a taxonomy that *requires* a claim-evidence mapping forces it to either cite a line number or stay silent — which matches what a competent reviewer does. The 'Reviewer 3' voice also counterbalances the model's default sycophancy bias.

## How it works

1) Extract abstract + method + results via the PDF Structure Extraction skill. 2) For each of the 6 review categories, ask Claude to produce claim-evidence pairs quoting line numbers from the extracted text. 3) Empty categories become 'No concerns' rather than filler. 4) Generate an aggregate score (Weak Accept / Weak Reject / Borderline) justified by the count and severity of concerns. 5) Output is reviewer-form-ready markdown plus a side-by-side change-request table.