OpenArch – PyTorch implementations of modern LLM architectures

OpenArch is a PyTorch repository containing hand-written implementations of modern large language model architectures, designed to prioritize clarity and learning over production efficiency. The repository catalogs various LLM designs from papers and technical reports, making explicit the architectural choices like attention mechanisms, normalization methods, and positional encodings so they can be easily compared side-by-side. Currently implemented models include GPT-2, Llama variants, Mistral, Gemma, and others, with each model documented in a single readable file.