Reading a paper
For your master thesis (and during advanced master courses) you often need to read several papers.
Intended for: MSc, PhD
Paper reading is time consuming, since a full read of a paper will easily take several hours. However, not every paper will be equally relevant to you, and you therefore need to learn to read papers in a gradually increasing level of detail. With each step, a (significant) portion of papers should drop (they don't make it to the next step). Consider these steps:
Title & Abstract (1 minute):
This can usually still be done on your favorite search engine, such as Google Scholar.
The abstract should summarize the main paper content, and is your biggest help to determine whether you should continue. Look for keywords on the problem type, methodology and results to determine whether this paper is worth more attention.
Decide: Is the paper interesting to you, then open it's pdf.
Quick scan (2 - 5 minutes):
Browse the introduction: An extended version of the abstract. Try to find out what the main idea is the authors propose (the abstract is usually too brief for this). Look for keywords again (usually the first few words of a paragraph) to see where you need to be.
Glance through the paper: Glance at section headers (what is the structure of the document), and at figures and tables (what type of methods & results do you recognize).
Look at schematic figure: Check whether the authors have a schematic figure, usually Figure 1, that visually summarizes the overall method.
Look at a few result graphs/tables: Do the results look promising? What settings are tested, do you recognize baselines, etc.
Read the conclusion paragraph: How do the authors summarize their work?
Decide: Does the paper seem relevant to you, because of problem, methodology and/or results? If yes, then continue.
Superficial read (5 - 20 minutes):
If you are still interested, then it is worth your time to truly find out what the paper proposes.
Read the introduction: This should summarize all motivation, the main idea, the obtained results, and the structure of the paper.
Browse the methodology section: Focus on the part where the authors propose something novel (look for keywords). How exactly is the idea implemented?
Browse related work: Quickly scan to what other fields the authors connect their work. Do you recognize papers? How do they describe the relationships? This may suddenly improve your understanding of the proposed method, by putting it into context.
Browse the discussion: Scan how the authors reflect on their work, and what future work they propose. This may help you better understand the method, and whether it suits your setting.
Decide: Have you understood at a high level what method the paper proposes (this should be clear now for a well-written paper)? Are the methodology and results really interesting & relevant to you (would you consider working on this topic, using similar ideas, etc.). If yes: consider to read the full paper.
Note: Be careful, the next step gets more expensive, so you have to start becoming selective now. Only continue for papers that really interest you.
Full read (20 minutes - 2 hours):
If you are still interested in the paper, then it is probably worth it to fully read the paper (front to back). Some additional advice:
Get creative: Focus on the proposed novel idea. Do you find it smart? Can you think of an extension of this idea, or another way to use the same principle? Get your mind to be creative. Check the future work sections as well, for what the authors consider useful possible extensions.
Experimental set-up: This is also the moment to carefully check whether the results are fair. Did the authors only tune hyperparameters for their own method, and not for the baseline? That could have strongly influenced the results. Check whether you want to accept the results.
Skip: A full read does not mean you have to read every sentence (you can of course). Feel free to skip certain parts when you directly see the point. There is for example no point in reading a full 'Preliminaries' section in every paper, where they introduce the notation you are already familiar with. (See theory versus empirical sections below as well).
Code: Many papers have public Github repositories, which might also be worth a quick browse. Search for the part of the code where the actually methodology is implemented (often a small part of the total code base). Does this help you understand the methodology better?
Theory versus empirical sections: Many machine learning papers come with i) a theoretical section in which the authors prove things for an idealized setting and then ii) an empirical section with a practical algorithm that violates many assumptions of the theoretical part (and the authors for example use a neural network to solve the problem in approximate form). A mistake I often see is that students come for one aspect (either theory or empirical part), but get intimidated by the other half. Depending on you interest, determine whether you really need to understand everything (of course it's good to try to understand, but if this would take too much time, also dare to skip it and focus on what is relevant for your project).
Decide: For most papers the process will stop here. You have understood the method, thought about it, and you may in the future use similar ideas in your own work. You only continue to the final step if you are really planning to work on the exact method proposed in this paper, and you want to figure out all the details.
Implementation read (incl. appendix & code) (2-... hours):
You will only take this step on a limited number of papers. Maybe you really like the idea and you want to use it in your own work, or maybe you have come up with an extension.
Study the math in detail: Although you already did a full read, you may still need some time to fully grasp all equations and details of the method.
Read the appendix: Now it is worth to dive into the appendix of the paper, an figure out all implementation details, additional results, etc.
Run the code: If the paper has code, it might be worth to dive into it as well, see how the practical implementation works, and maybe even rerun the experiment code on you own computer.
Implement/extend: Finally, when you are convinced, you may want to implement the method yourself, or extend it in some form.