Q function, value function, State-Action-Value function
In practice, Q value is harder to fit than value function.
- linear value function
+ least squares or regression then minimize error
We only need to fit V
Q-Table stores how good each action is in each state
Generalizing states in a Q-table is challenging due to the large number of possible states. The solution is to generalize from previously experienced situations to new, similar situations. This is accomplished using Feature-Based Representations, which is a fundamental concept we frequently encounter in machine learning.
Action-value estimations