When you do research, there are two main approaches:
- Start with a grand idea, intuition, or otherwise behaviour you want to enforce. Perform steps in order to achieve this vision.
- Start off of an existing idea. Take tiny steps that change the existing work, until it is your own. I would also place addressing “what’s wrong” with the paper under this category.
These two approaches are fundamentally different, and essentially represent working backwards from a solution vs starting from a problem, and finding a new contribution.
This is a direct analogue of a bidirectional BFS in leetcode!
The second approach is generally a little more “in-the-weeds” and detail-oriented, where you must justify each step carefully.
There are lots of details when it comes to neural nets…