Writing reviews for systems conferences

Purpose for paper review

• a chance to get your own thoughts on the paper straight by writing them down

• communicating your thoughts about the paper to other PC members

Steps in reviewing

• read each one in turn quickly from start to finish to get a general sense of what it’s like.

• Starting as soon as possible, read each paper in turn carefully, scribbling notes on them in the margins with whatever comments you think of at the time.

Structure of a review

1. summarize the paper
Give a neutral description of what you think the paper is about, where the authors are coming from, why they view the problem as important, and what they’ve done. This is a great way to start writing a review, particularly when you’re not sure how to get started.

2. state what you think the contributions are. No useful contribution? Contribution missed?

3. specific comments
1. Novelty
2. How well written
3. apparent technical flaws
4. gaps or unaddressed issues
5. anything really cool

Tone

• transform every negative comment into a constructive suggestion
For example: “This system doesn’t deal with unexpected vegetables” can be turned into the more positive “The paper would be much stronger if it discussed how the system deals with unexpected vegetables.”

• a remote possibility that you’ve misunderstood the paper
It’s better to write “The description in the paper left me worried that algorithm breaks in the presence of Byzantine faults.”