How to Read a Paper

Inspired by David Silver’s wonderful introductory course on reinforcement learning (RL), I’ve recently started learning about deep reinforcement learning (deep RL). My goal is to gain an in-depth understanding on this field, the learning path suggested by OpenAI is to read the key papers and re-implement the algorithms.

The first question to ask is “how to read a paper efficiently?” With the abundance of papers available, knowing how to efficiently read research papers is an essential skill to acquire. I will use the “three-pass” approach where we read the paper in three passes, instead of hoping to fully understand it after reading it once.

Pass 1

Do: Read the title, abstract, introduction, section and sub-section headings, conclusion and a glance at the references.

Goal: Know which category the paper belongs to, which other papers is it related to, does it make reasonable assumptions, what is its main contributions, and is it well written? Use the answers to these questions to decide whether to read further.

Pass 2

Do: Actively read through the paper carefully, write down notes as you read, pay attention to the figures and diagrams, but skip the math details.

Goal: Summarize the main context with supporting evidence.

Note that you may not understand a paper even after this second pass.

Pass 3

Do: Pay great attention to every detail, understand the math, and re-implement the algorithms.

Goal: Fully understand the paper

I will read a paper a week from this selection of key papers in deep RL and post a write-up here.