Texonom
Texonom/
Engineering
Engineering/Data Engineering/Artificial Intelligence/Machine Learning/Machine Learning Theory/Reinforcement Learning/Reinforcement Learning Method/Model based RL/
CEM Iteration
Search

CEM Iteration

Creator
Creator
Seonglae ChoSeonglae Cho
Created
Created
2024 May 10 7:34
Editor
Editor
Seonglae ChoSeonglae Cho
Edited
Edited
2024 Jun 17 4:28
Refs
Refs
Model-Predictive Control

Cross-Entropy Method

Sampling-based optimization
Sample mean and variance of elite samples are minimizing CE between the current sampling distribution and target distribution
CEM Iteration (more planning → better performance) 5 is a good point.
Set action distribution and update distribution parameters re-fit distribution using top-K elite actions
notion image
CEM can generate the action only by planning through the model, without any explicit policy.
 
 

Random shooting

Guess & check without elite fitting. We improved this distribution by the selected elites.
 
 
 
 
Cross-entropy method
The cross-entropy (CE) method is a Monte Carlo method for importance sampling and optimization. It is applicable to both combinatorial and continuous problems, with either a static or noisy objective.
Cross-entropy method
https://en.wikipedia.org/wiki/Cross-entropy_method
 
 

Recommendations

Texonom
Texonom/
Engineering
Engineering/Data Engineering/Artificial Intelligence/Machine Learning/Machine Learning Theory/Reinforcement Learning/Reinforcement Learning Method/Model based RL/
CEM Iteration