From-scratch PyTorch implementation of Sutskever et al. (2014), sequence to sequence learning with stacked LSTMs, source reversal and beam search, with an analysis of why it plateaus at small scale.

From-scratch PyTorch implementation of Bahdanau et al. (2014), "Neural Machine Translation by Jointly Learning to Align and Translate": bidirectional GRU encoder, additive attention, maxout decoder and beam search, with alignment heatmaps on a synthetic translation task.

From-scratch PyTorch implementation of Vaswani et al. (2017), the Transformer with multi-head attention, sinusoidal positional encodings and beam search, verified by built-in checks and trained on synthetic reverse and sort tasks, with an analysis of what the attention learns.

From-scratch PyTorch implementation of Radford et al. (2018), GPT-1, with a decoder-only post-LN Transformer, weight tying, GPT-1's training recipe, and fine-tuning with the input transformation and auxiliary LM loss, trained on CPU on Tiny Shakespeare, with ablations showing the auxiliary loss prevents catastrophic forgetting.

From-scratch PyTorch implementation of Dehghani et al. (2019), the Universal Transformer with weight sharing across depth, coordinate embeddings and Adaptive Computation Time, trained on CPU on copy, reverse and sort, with an analysis of how much computation each task gets.

A minimal from-scratch clone of vLLM's LLMEngine orchestration layer, built to understand how an inference engine works: input and output processors around a shared request table, token-by-token generation driven manually through the KV cache, concurrent requests with streaming output, mid-generation abort, and fan-out of one prompt into N completions, running on a GPT-2 backend.