Seonglae ChoEarth mover's distance, Optimal Transport Problem
The minimum cost required to transform one probability distribution into another by moving probability mass, where cost is defined as the total distance that mass needs to be moved
It does not diverges even if Support does not duplicate.
Flow
Algorithm
Earth mover's distance
In computer science, the earth mover's distance (EMD)[1] is a measure of dissimilarity between two frequency distributions, densities, or measures, over a metric space D.
Informally, if the distributions are interpreted as two different ways of piling up earth (dirt) over D, the EMD captures the minimum cost of building the smaller pile using dirt taken from the larger, where cost is defined as the amount of dirt moved multiplied by the distance over which it is moved.
https://en.wikipedia.org/wiki/Earth_mover%27s_distance
NTIL (Numerical Token Integrity Loss)
- Token level: Preserves order/distance between numbers by training with EMD (+ digit position weighting)
- Sequence level: Measures value error (relative·scale) between predicted and target numerical values
CE limitation: Treats each numerical token as an independent class, ignoring proximity between numbers