Reading a Paper
Ask what it actually proved first, then look at the details
Help me read the paper excerpt below. Output: 1) what problem it addresses; 2) the core idea of the method, in plain language; 3) what the experiments proved and what they didn't; 4) limitations the authors admit; 5) if I had to reproduce it, the hardest step. Don't restate the abstract. [Paste paper excerpt]
How to use
Ask what it didn't prove before you read the method section — it changes what you look for.
Good for
- You have twenty papers and half an hour to decide which three deserve a close read
- The abstract reads as if it solves your problem, but you suspect the experimental setting differs from your own data
Not for
- You need to cite exact numbers or experimental conditions in a report, and the model may invent metrics and setups
- You pasted only the abstract yet want answers about what was not proven or the hardest step to reproduce: the material cannot support it
Example
Input
We present a residual learning framework to ease the training of networks that are substantially deeper than those used previously. Instead of hoping each few stacked layers directly fit a desired underlying mapping, we explicitly let these layers fit a residual mapping. We provide comprehensive empirical evidence showing these residual networks are easier to optimize and can gain accuracy from considerably increased depth. On ImageNet, we evaluate residual nets with a depth of up to 152 layers, 8x deeper than VGG nets but still having lower complexity.
What you get
It should say the problem is degradation of very deep networks, not vanishing gradients or overfitting, and that the core idea is an identity shortcut which turns the optimisation target into a residual. It should mark that the evidence covers classification and detection gains on ImageNet and COCO only, with no theory of why residuals help. The hardest step to reproduce is usually the training recipe (augmentation, learning-rate warmup, batch size), not the block itself.
Common mistakes
- Pasting only the abstract and accepting a verdict on what the paper proved, treating the abstract's wording as the experimental result
- Asking only what it says and never what it did not prove, which turns the reading into free publicity for the paper
Try tweaking it like this
- Add 'as a reviewer, give three main objections and point to the missing section for each': reading becomes critical evaluation
- Paste only the method section and ask what unwritten assumptions it depends on: useful when you plan to reproduce or port it