1. What did you do this week?
This week's main job is to build a calculation graph. The core of the automatic differential system is vjp, which is composed of calculation graph construction and gradient calculation.
2. Difficulty
The computational graph construction is not as simple as I thought. The main problem encountered in the process is that some basic differential operators not only need to pass in the tensors, but also need to pass in some parameters of the function, such as np.sum. This requires that some parameters required by the differential operator be passed in advance when constructing the calculation graph. In addition, according to the different calculation paths, the parents of each node in the calculation graph should be marked appropriately, and then they can be calculated along the correct path during back propagation.
3. What is coming up next?
The work for the next week is to implement simple back propagation and complete a complete differential operation.