Seonglae ChoDeep Equilibrium Models, DEQ
In this equation, is the fixed point where the network's output is fed back into itself as input. In other words, it's a state where repeatedly applying f no longer changes the value. Once we reach this fixed point after iterating (forward step), we can compute the gradient using IFT (Implicit Function Theorem) as follows:
Using this approach, we can calculate the gradient using only the final fixed point, without backpropagating through all recursion steps. → This enables "1-step gradient approximation".
