kn-neeraj/inference-engg-learnings

Inference Engineering Learnings is a personal collection of hands on notes and runnable notebooks about running and studying large language model inference. The repository is organized around a single notebooks folder that holds Jupyter notebooks and exports from Google Colab, along with a table in the README that indexes each notebook by file name and a short description of what it covers. At the time of this writing, the notebook index lists one entry: a notebook that runs Google's Gemma 4 12B model using 4 bit quantization on a free Google Colab T4 GPU, in order to study how inference behaves in practice, including time to first token, tokens generated per second, and how the underlying inference pipeline works. The README also documents the intended workflow for growing the collection: add a new notebook to the notebooks folder, update the index table with its file name and a one line description, and commit the notebook together with the README change so the index always matches what is actually in the repository. This is a small, early stage project with very little content so far beyond the single Gemma notebook and the instructions for adding more. It reads as a working notebook for one person to record inference engineering experiments over time rather than a packaged tool or library meant for other people to install and run directly, though anyone is free to open the notebook in Colab and follow along with the same experiment. The project is written primarily in Jupyter Notebook format, which mixes explanatory text with runnable Python code cells, and no license file is mentioned in what is shown here.