Gradient Scaling

Creator

Created

2023 Aug 20 15:18

Editor

Edited

2023 Nov 3 12:23

Refs

forward f16 → backward f16 could be flush as 0

by multiplying gradient scaling (scale factor) to loss, scaled loss computed for backward pass

///////