WIKI.LOG

BUILD NOTES.

Living documentation on what I'm building right now.

PAUSEDJune 2026

LLM Maker — Model Registry & Design Documentation

Building decoder-only LLMs from scratch: pretraining tiny models on FineWeb-Edu, instruction-tuning, and scaling up — every model, the architecture and why, design decisions, measured performance, and I/O examples.

LLMPyTorchPretrainingTransformers
> open_doc.sh