da-southampton/nlp_ability

★ 7,491PythonAudience · developerComplexity · 1/5Setup · easy

Mindmap

mindmap
  root((nlp_ability))
    Transformer
      Interview questions
      Encoder and decoder
      Positional encoding
      Normalization choices
    BERT variants
      RoBERTa
      ALBERT
      UniLM
    Word embeddings
      Word2Vec
      FastText
      GloVe
    Other topics
      Knowledge distillation
      Text classification
      Named entity recognition

mindmap root((nlp_ability)) Transformer Interview questions Encoder and decoder Positional encoding Normalization choices BERT variants RoBERTa ALBERT UniLM Word embeddings Word2Vec FastText GloVe Other topics Knowledge distillation Text classification Named entity recognition

Click or tap to explore — scroll the page freely

Things people build with this

USE CASE 1

Study for NLP engineering interviews using detailed Q&A on Transformer and BERT internals.

USE CASE 2

Reference explanations of Word2Vec, FastText, and GloVe word embedding techniques with training details.

USE CASE 3

Learn knowledge distillation methods like TinyBERT and BERT-of-Theseus for compressing large language models.

USE CASE 4

Build understanding of text classification and named entity recognition for production NLP systems.

Tech stack

Python

Getting it running

Difficulty · easy Time to first run · 5min

No license information found in this repository.

In plain English

This repository is a Chinese-language collection of articles and notes for people working in or studying Natural Language Processing, the field of software that deals with understanding and generating human language. The maintainer assembled it from their own work experience, daily research notes, and paper summaries, with the goal of building up a reference that covers both theory and practical engineering skill. The content is organized by topic. A large section covers the Transformer architecture, which underpins most modern language AI systems. That section includes a set of common interview questions with detailed answers, explanations of how the encoder and decoder components work, and discussions of design choices like different position encoding approaches and normalization strategies. A separate section focuses on BERT, a well-known language model, and its variants such as RoBERTa, XLNET, ALBERT, and UniLM. Word embeddings are covered in another section, with multiple articles on Word2Vec covering the two training approaches, the optimization methods, negative sampling, and parameter selection. FastText and GloVe also have dedicated articles. There is a section on knowledge distillation, which is the technique of training a smaller, faster model to mimic a larger one, including articles on TinyBERT, PKD-BERT, and BERT-of-Theseus. Other sections address text classification, text similarity and matching, named entity recognition, and multimodal models that combine language with other types of data. The articles are written in Chinese and are primarily aimed at practitioners preparing for NLP engineering interviews or looking to deepen their understanding of specific techniques. The repository does not appear to include runnable code for most topics, it functions as a structured reading and reference guide.

Copy-paste prompts

Prompt 1

I'm preparing for an NLP engineering interview. Quiz me on common Transformer architecture questions covering self-attention, positional encoding, and layer normalization choices.

Prompt 2

Explain the difference between RoBERTa, XLNET, ALBERT, and UniLM as BERT variants, what does each one improve and when would you choose one over another?

Prompt 3

Walk me through how knowledge distillation works in TinyBERT, how does it compress a large BERT model into a smaller one that still performs well on downstream tasks?

Prompt 4

I'm studying Word2Vec for an interview. Explain the Skip-gram vs CBOW training approaches, negative sampling, and how to tune the window size.

Open on GitHub → Explain another repo

← da-southampton on gitmyhub — every repo by this author, as a profile.

Verify against the repo before relying on details.