locuslab/tcn

This repository contains the code from a research paper that compares two broad families of neural network architectures for working with sequences of data. One family is called recurrent networks, which have been the standard approach for tasks like language modeling and music generation. The other is a convolutional network design called a Temporal Convolutional Network (TCN), which the paper tests against the recurrent approaches on their own benchmark tasks. The goal of the research was to find out whether a convolutional approach, one not traditionally associated with sequence tasks, could match or beat recurrent networks across a wide range of standard problems. The conclusion was that the convolutional design performed well across the board, sometimes better than the recurrent alternatives. The repository includes runnable experiments for eleven different tasks. These cover things like digit classification on images presented one pixel at a time, word-level and character-level language modeling using standard text datasets, polyphonic music datasets, and a couple of synthetic tests designed to measure how well a model can retain information over long time spans. Each task lives in its own folder and follows the same structure: a data folder, a script to run the test, and files for the model and helper utilities. Running the test script for a given task is all that is needed to reproduce the results. Hyperparameters can be adjusted through command-line flags. Some of the larger datasets are downloaded automatically through a helper package rather than bundled in the repository. The code requires Python and a machine learning library called PyTorch. It is intended for researchers and practitioners who want to reproduce the benchmark results or use the TCN architecture as a starting point for their own sequence modeling work.