Why AI forgets your prompt over time
A new theorem proves that a transformer's memory of its prompt fades as a power law as more tokens are generated.
A new theorem proves that a transformer's memory of its prompt fades as a power law as more tokens are generated.