Format your numbers like this to dramatically improve your LLM's math skills

Positional description matters for transformers arithmetic

Data format and some other special tricks makes LLMs much better at math.

Arithmetic is an important yet challenging domain for LLMs. While they have achieved tremendous success in natural language, their performance on arithmetic is often lacking. In this blog post, we'll summarize a recent technical paper that dives deeper into transformers' capabilities and limitations with arithmetic. Key findings include a new insight into positional encoding and techniques to enhance models' multiplication and addition skills. Let's begin!