RL critics trained on actions, not returns
A new RL critic learns by asking which actions fit a given reward, instead of predicting rewards directly.
A new RL critic learns by asking which actions fit a given reward, instead of predicting rewards directly.