doriandarko/texts-to-transformer

Texts to Transformer is a project that lets a Mac user train a very small language model from scratch using their own iMessage history, with everything running locally on the machine and nothing uploaded anywhere. A transformer is the type of neural network architecture behind most modern chat AI, and this project builds one entirely from an untrained starting point rather than starting from an existing pretrained model. The goal is not a generally capable assistant that can answer questions or reason, but a small model that picks up the owner's own phrasing, rhythm, slang, and typical responses well enough to suggest replies in their personal texting style. The README places heavy emphasis on privacy and safety throughout the pipeline. The Messages database on the Mac is only ever opened in read only mode, and all processing happens on a private backup copy rather than the live database. Names and chat identifiers are replaced with scrambled pseudonyms before anything is written to a file, and things like URLs, email addresses, and phone numbers are automatically redacted. Nothing in the pipeline sends a message or uploads data anywhere, and the finished datasets and trained models are deliberately excluded from being committed to Git. The full pipeline covers roughly nine steps: safely copying the Messages database, extracting and cleaning the text, building a custom tokenizer that turns text into model readable pieces, training a small transformer model from random initialization using Apple's MLX machine learning framework, evaluating it on messages it never saw during training, and finally exporting a local model you can chat with in the terminal to get suggested replies. The default configuration builds a model with under one and a half million parameters, though a larger preset exists for people with very large message histories. This project only works on an Apple Silicon Mac running a recent version of macOS, requires the Full Disk Access permission to read the Messages database, and depends on Apple's MLX library rather than more common tools like PyTorch. The README is explicit that the resulting model can memorize private text and must be kept private rather than shared or uploaded.