Coding agents think ahead of time

Researchers discovered that language models used in coding agents internally represent program properties—such as whether code parses or passes tests—in their hidden states, with probes achieving up to 0.83 AUC accuracy in predicting correctness. More remarkably, these internal representations appear to "look ahead" to predict the outcomes of future edits up to 25 steps in advance, a phenomenon the authors call the agent's "latent programming horizon." This finding advances mechanistic interpretability research and suggests that coding agents have more sophisticated internal planning capabilities than their observable actions reveal.