Sparsemax Function

Creator
Creator
Seonglae Cho
Created
Created
2025 Mar 10 12:41
Editor
Edited
Edited
2025 Mar 10 13:0

Probability simplex
Projection

Sparsemax(z)=argminpΔKpz22,where ΔK={pRKi=1Kpi=1,pi0}. \text{Sparsemax}(\mathbf{z}) = \arg\min_{\mathbf{p} \in \Delta^K} \|\mathbf{p} - \mathbf{z}\|_2^2, \quad \text{where } \Delta^K = \{\mathbf{p} \in \mathbb{R}^K \mid \sum_{i=1}^K p_i = 1,\, p_i \ge 0\}. 
This projects the input vector zz onto the
Probability simplex
, yielding a sparse
Probability Distribution
.

Solution

Sparsemax(z)i=max(ziτ,0),where τ satisfies i=1Kmax(ziτ,0)=1.\text{Sparsemax}(z)_i = \max\left(z_i - \tau,\,0\right), \quad \text{where } \tau \text{ satisfies } \sum_{i=1}^K \max\left(z_i - \tau,\,0\right) = 1.
and that τ\tau is
τ=jSzj1SandS={iziτ>0}.\tau = \frac{\sum_{j \in \mathcal{S}} z_j - 1}{|\mathcal{S}|} \quad \text{and} \quad \mathcal{S} = \{ i \mid z_i - \tau > 0 \}.
 
 
 
 
 
 
 

Recommendations