Seonglae ChoIt indicates the model can access or interact with the Environment
Model is trained and updated continuously as new data arrives, rather than in batches
This approach allows the model to adapt quickly to changes and new patterns in the data without needing a complete retraining process
Online machine learning
In computer science, online machine learning is a method of machine learning in which data becomes available in a sequential order and is used to update the best predictor for future data at each step, as opposed to batch learning techniques which generate the best predictor by learning on the entire training data set at once. Online learning is a common technique used in areas of machine learning where it is computationally infeasible to train over the entire dataset, requiring the need of out-of-core algorithms. It is also used in situations where it is necessary for the algorithm to dynamically adapt to new patterns in the data, or when the data itself is generated as a function of time, e.g., stock price prediction. Online learning algorithms may be prone to catastrophic interference, a problem that can be addressed by incremental learning approaches.
https://en.wikipedia.org/wiki/Online_machine_learning
RLPD(Reinforcement Learning with Prior Data)
Efficient Online Reinforcement Learning with Offline Data
Using “symmetric sampling,” at each update they sample data from the online replay buffer and the offline dataset in a 50:50 ratio to construct the training batch, thereby continuously preserving guidance from the offline data. Second, they apply layer normalization to the critic (value function) network, which plays a crucial role in preventing excessive over-extrapolation and divergence of the value function on out-of-distribution (OOD) data. Third, they use a large REDQ-style ensemble (Ensemble,
Efficient Online Reinforcement Learning with Offline Data
Sample efficiency and exploration remain major challenges in online reinforcement learning (RL). A powerful approach that can be applied to address these issues is the inclusion of offline data,...
https://arxiv.org/abs/2302.02948
