RLHF reward models break at known thresholds
Researchers pinpoint exactly when reward models stop reflecting human preferences during RLHF training.
Researchers pinpoint exactly when reward models stop reflecting human preferences during RLHF training.