Get ready to lose to Transformers on Lichess

They can hit 2895 Elo ... without memorizing patterns!

“Figure 4: Two options to win the game in 3 or 5 moves, respectively (more options exist). Since they both map into the highest-value bin our bot ignores Nh6+, the fastest way to win (in 3), and instead plays Nd6+ (matein-5). Unfortunately, a state-based predictor without explicit search cannot guarantee that it will continue playing the Nd6+ strategy and thus might randomly alternate between different strategies. Overall this increases the risk of drawing the game or losing due to a subsequent (low-probability) mistake, such as a bad softmax sample. Board from a game between our 9M Transformer (white) and a human (blitz Elo of 2145).”

Hello everyone! AI has revolutionized strategy games, but a core question remains: can AI achieve top-level planning without memorizing strategies or using brute-force search?

As a 2k Elo player myself (brag?), I was pretty excited to see someone tried to answer this problem in a unique way for chess!

A top paper on AImodels.fyi today takes a novel approach by training large transformers to play chess without memorization or explicit search. Instead, it uses a benchmark dataset, ChessBench, built from 10 million human chess games.

By analyzing this dataset, the study trains transformers to make strategic moves “intuitively” on unseen chess boards—that is, without relying on memorized game patterns. The findings reveal how far transformers can go in mastering chess through generalization rather than memorization.

So how’s this all work?