tensorflow/tensor2tensor

★ 17,252PythonAudience · researcherComplexity · 3/5Setup · moderate

Mindmap

mindmap
  root((tensor2tensor))
    What it does
      Deep learning library
      Pre-built models
      Research benchmarks
    Tech Stack
      Python
      TensorFlow
      t2t-trainer CLI
    Use Cases
      Translation research
      Image classification
      Language modeling
    Status
      Deprecated
      Migrate to Trax

mindmap root((tensor2tensor)) What it does Deep learning library Pre-built models Research benchmarks Tech Stack Python TensorFlow t2t-trainer CLI Use Cases Translation research Image classification Language modeling Status Deprecated Migrate to Trax

Click or tap to explore — scroll the page freely

Things people build with this

USE CASE 1

Run a Transformer model on a standard machine translation benchmark without writing model code from scratch

USE CASE 2

Train an image classifier on ImageNet or CIFAR-10 using a pre-built ResNet or Xception problem definition

USE CASE 3

Experiment with hyperparameter sets on language modeling benchmarks to compare model architectures

USE CASE 4

Reproduce published deep learning results using the included problem, model, and hyperparameter configurations

Tech stack

PythonTensorFlow

Getting it running

Difficulty · moderate Time to first run · 30min

Now deprecated, new projects should use the successor library Trax, T2T still accepts bug fixes.

In plain English

Tensor2Tensor, often shortened to T2T, is a library of pre-built deep learning models and datasets that aims to make deep learning more accessible and to speed up machine learning research. Deep learning is a branch of machine learning that uses neural networks with many layers, and T2T packages up the building blocks so researchers do not have to start from scratch for each new experiment. Inside the library, a "problem" describes a dataset and the task you want to do with it, and a "model" describes the neural network shape. T2T comes with ready-made problems for image classification using datasets like ImageNet, CIFAR-10, CIFAR-100 and MNIST, image generation using CelebA, LSUN Bedrooms and MS-COCO, language modeling using PTB and the billion-word LM1B corpus, speech recognition using Librispeech and Mozilla Common Voice, sentiment analysis using IMDB, question answering using bAbi, and machine translation. It includes several flavours of the Transformer model, plus alternatives like ResNet, Xception and Shake-Shake. You pick a problem, pick a model, pick a set of hyperparameters, and a single command-line trainer called t2t-trainer runs the experiment. Someone would actually use this if they were doing machine learning research and wanted a head start on standard benchmarks, or if they wanted to learn how transformer-style models behave on a particular dataset. The maintainers note that T2T was created by the Google Brain team and is now deprecated, bug fixes are still accepted, but users are encouraged to move to the successor library called Trax. The full README is longer than what was provided.

Copy-paste prompts

Prompt 1

Using Tensor2Tensor, set up the WMT English-to-German translation problem and train a base Transformer model with the t2t-trainer command.

Prompt 2

Help me configure a Tensor2Tensor experiment to train a ResNet-50 on CIFAR-10 and log metrics to TensorBoard.

Prompt 3

Using Tensor2Tensor's hyperparameter sets, run a learning rate sweep for the Transformer on the LM1B language modeling benchmark.

Prompt 4

Show me how to add a custom problem to Tensor2Tensor so I can train a Transformer on my own parallel sentence pairs.

Prompt 5

Using Tensor2Tensor, compare the Transformer and LSTM architectures on the IMDB sentiment benchmark and report validation accuracy.

Open on GitHub → Explain another repo

← tensorflow on gitmyhub — every repo by this author, as a profile.

Verify against the repo before relying on details.