Advantage function

Creator
Creator
Seonglae ChoSeonglae Cho
Created
Created
2024 Mar 20 2:8
Editor
Edited
Edited
2024 Nov 21 10:21
Refs
Refs
GAE

Most commonly used in practice

How good advantage is an action compared to the policy?
Q - V
positive or negative
notion image
notion image
  • usually average is near 0
Advantage estimations
 
Can show that advantage-weighted objective approximates KL-constrained objective.
notion image
 
 
 
 

Recommendations