wdndev/llm_interview_note

★ 14,172HTMLAudience · researcherComplexity · 3/5Setup · easy

Mindmap

mindmap
  root((llm_interview_note))
    Foundations
      Tokenization
      Word vectors
      BERT basics
    Architecture
      Transformer design
      Attention mechanisms
      LLaMA and ChatGLM
    Training
      Distributed strategies
      Fine-tuning techniques
      RLHF alignment
    Applied Topics
      RAG systems
      Inference frameworks
      Hallucination causes

mindmap root((llm_interview_note)) Foundations Tokenization Word vectors BERT basics Architecture Transformer design Attention mechanisms LLaMA and ChatGLM Training Distributed strategies Fine-tuning techniques RLHF alignment Applied Topics RAG systems Inference frameworks Hallucination causes

Click or tap to explore — scroll the page freely

Things people build with this

USE CASE 1

Study Transformer architecture, attention mechanisms, and models like LLaMA and ChatGLM to prepare for an AI engineering interview

USE CASE 2

Learn how RLHF fine-tuning and distributed training strategies work for large-scale language model development

USE CASE 3

Understand RAG systems and LLM hallucination causes from both a technical and interview-question perspective

Tech stack

HTMLPython

Getting it running

Difficulty · easy Time to first run · 5min

License terms not specified in the explanation.

In plain English

This repository is a study guide and interview question collection for engineers working on large language models (LLMs). The content is written in Chinese and is aimed at people preparing for technical job interviews at AI companies, or those who want to build a structured understanding of how modern language models are built and deployed. The notes cover the full lifecycle of a language model in roughly ten topic areas. The first covers foundational concepts: how text is broken into tokens, how words are represented as numeric vectors, and how classic model components like BERT work. The second goes deeper into model architecture, including the Transformer design that most modern LLMs are based on, various attention mechanisms, and specific models like LLaMA and ChatGLM. A section on training covers distributed training strategies, which are needed when training on large clusters of hardware. Later sections address fine-tuning (how to adapt a general model to a specific task), inference frameworks (tools used to run the model efficiently once training is done), and reinforcement learning from human feedback (RLHF), which is a technique used to align model outputs with human preferences. There is also a section on retrieval-augmented generation (RAG), where the model consults an external database during generation to improve accuracy. The guide includes a section on LLM hallucinations: cases where the model produces confident-sounding but incorrect output. This is covered from both a technical and practical interview-question angle. The author also links several companion repositories for hands-on practice, including a small Chinese-language LLM built from scratch, a simple RAG system, and an implementation of the LLaMA 3 architecture. These are separate projects for those who want to move from reading notes to running experiments.

Copy-paste prompts

Prompt 1

Explain the key architectural differences between GPT, BERT, and LLaMA as I would need to know for an LLM engineering interview at a Chinese AI company

Prompt 2

Walk me through how RLHF works step by step, what is the reward model, how is it trained, and why is it used to align language models

Prompt 3

What are the main distributed training strategies for large language models such as data parallelism, tensor parallelism, and pipeline parallelism, and when would I choose each

Prompt 4

How does retrieval-augmented generation reduce LLM hallucinations, what are its main limitations, and how would I explain it in a technical interview

Prompt 5

What common LLM fine-tuning techniques like LoRA and prefix tuning are covered in wdndev/llm_interview_note and how do they differ

Open on GitHub → Explain another repo

← wdndev on gitmyhub — every repo by this author, as a profile.

Verify against the repo before relying on details.