s-sahoo/phd-thesis

This repository holds Subham Sekhar Sahoo's PhD thesis, titled Foundations of Diffusion Language Models, completed at Cornell Tech. The repo contains the thesis document as a PDF, plus a link to the slides from the thesis talk. The thesis investigates diffusion language models, an alternative to the autoregressive models that power most of today's chatbots and text generators. The author's abstract explains that diffusion models already work well for images and video, but had not yet closed the gap with autoregressive models for text. The work looks at three separate contributions. First, it examines the noise schedule used to train diffusion models and introduces new noise processes learned from data, which the author says improve both output quality and how likely the model's predictions are. Second, it proposes a unified framework covering two types of discrete diffusion and derives tighter mathematical bounds used to train the models, producing what the author calls state of the art results among diffusion approaches. Third, it connects one type of discrete diffusion to continuous diffusion methods used for images, which allowed the author to borrow techniques from the image side to speed up both training and generation, including a way to sample text in far fewer steps. There is no code here to install or run. The repository is largely just a container for the dissertation and its citation information, formatted as a BibTeX entry so other researchers can reference it properly. Anyone interested in diffusion based text generation, or the mathematics behind newer alternatives to standard chatbot architectures, would find this useful as a reference document rather than a tool.