Automatic Differentiation

AD

Evaluation Traces

• Forward Mode -
  Dual number(primal + tangent) with
  Jacobian Matrix - 임력차원이 적은 문제에 적합

• Reverse Mode -
  Back Propagation using
  Chain Rule - 출력 차원이 적은 문제에 적함

2.5. 자동 미분(automatic differentiation) — Dive into Deep Learning documentation

https://ko.d2l.ai/chapter_crashcourse/autograd.html