What’s new in PyTorch tutorials?
Introduction to Distributed Pipeline Parallelism
Introduction to Libuv TCPStore Backend
Asynchronous Saving with Distributed Checkpoint (DCP)
Python Custom Operators
Updated Getting Started with DeviceMesh
Learn the Basics
Familiarize yourself with PyTorch concepts and modules. Learn how to load data, build deep neural networks, train and save your models in this quickstart guide.
Get started with PyTorch
PyTorch Recipes
Bite-size, ready-to-deploy PyTorch code examples.
Explore Recipes
Learn the BasicsA step-by-step guide to building a complete ML workflow with PyTorch.Getting-Started
Introduction to PyTorch on YouTubeAn introduction to building a complete ML workflow with PyTorch. Follows the PyTorch Beginner Series on YouTube.Getting-Started
Learning PyTorch with ExamplesThis tutorial introduces the fundamental concepts of PyTorch through self-contained examples.Getting-Started
What is torch.nn really?Use torch.nn to create and train a neural network.Getting-Started
Visualizing Models, Data, and Training with TensorBoardLearn to use TensorBoard to visualize data and model training.Interpretability,Getting-Started,TensorBoard
TorchVision Object Detection Finetuning TutorialFinetune a pre-trained Mask R-CNN model.Image/Video
Transfer Learning for Computer Vision TutorialTrain a convolutional neural network for image classification using transfer learning.Image/Video
Optimizing Vision Transformer ModelApply cutting-edge, attention-based transformer models to computer vision tasks.Image/Video
Adversarial Example GenerationTrain a convolutional neural network for image classification using transfer learning.Image/Video
DCGAN TutorialTrain a generative adversarial network (GAN) to generate new celebrities.Image/Video
Spatial Transformer Networks TutorialLearn how to augment your network using a visual attention mechanism.Image/Video
Inference on Whole Slide Images with TIAToolboxLearn how to use the TIAToolbox to perform inference on whole slide images.Image/Video
Semi-Supervised Learning Tutorial Based on USBLearn how to train semi-supervised learning algorithms (on custom data) using USB and PyTorch.Image/Video
Audio IOLearn to load data with torchaudio.Audio
Audio ResamplingLearn to resample audio waveforms using torchaudio.Audio
Audio Data AugmentationLearn to apply data augmentations using torchaudio.Audio
Audio Feature ExtractionsLearn to extract features using torchaudio.Audio
Audio Feature AugmentationLearn to augment features using torchaudio.Audio
Audio DatasetsLearn to use torchaudio datasets.Audio
Automatic Speech Recognition with Wav2Vec2 in torchaudioLearn how to use torchaudio's pretrained models for building a speech recognition application.Audio
Speech Command ClassificationLearn how to correctly format an audio dataset and then train/test an audio classifier network on the dataset.Audio
Text-to-Speech with torchaudioLearn how to use torchaudio's pretrained models for building a text-to-speech application.Audio
Forced Alignment with Wav2Vec2 in torchaudioLearn how to use torchaudio's Wav2Vec2 pretrained models for aligning text to speechAudio
Fast Transformer Inference with Better TransformerDeploy a PyTorch Transformer model using Better Transformer with high performance for inferenceProduction,Text
NLP from Scratch: Classifying Names with a Character-level RNNBuild and train a basic character-level RNN to classify word from scratch without the use of torchtext. First in a series of three tutorials.Text
NLP from Scratch: Generating Names with a Character-level RNNAfter using character-level RNN to classify names, learn how to generate names from languages. Second in a series of three tutorials.Text
NLP from Scratch: Translation with a Sequence-to-sequence Network and AttentionThis is the third and final tutorial on doing “NLP From Scratch”, where we write our own classes and functions to preprocess the data to do our NLP modeling tasks.Text
Text Classification with TorchtextLearn how to build the dataset and classify text using torchtext library.Text
Language Translation with TransformerTrain a language translation model from scratch using Transformer.Text
Pre-process custom text dataset using TorchtextLearn how to use torchtext to prepare a custom datasetText
(optional) Exporting a PyTorch model to ONNX using TorchDynamo backend and Running it using ONNX RuntimeBuild a image classifier model in PyTorch and convert it to ONNX before deploying it with ONNX Runtime.Production,ONNX,Backends
Introduction to ONNX RegistryDemonstrate end-to-end how to address unsupported operators by using ONNX Registry.Production,ONNX,Backends
Reinforcement Learning (DQN)Learn how to use PyTorch to train a Deep Q Learning (DQN) agent on the CartPole-v0 task from the OpenAI Gym.Reinforcement-Learning
Reinforcement Learning (PPO) with TorchRLLearn how to use PyTorch and TorchRL to train a Proximal Policy Optimization agent on the Inverted Pendulum task from Gym.Reinforcement-Learning
Train a Mario-playing RL AgentUse PyTorch to train a Double Q-learning agent to play Mario.Reinforcement-Learning
Recurrent DQNUse TorchRL to train recurrent policiesReinforcement-Learning
Code a DDPG LossUse TorchRL to code a DDPG LossReinforcement-Learning
Writing your environment and transformsUse TorchRL to code a PendulumReinforcement-Learning
Deploying PyTorch in Python via a REST API with FlaskDeploy a PyTorch model using Flask and expose a REST API for model inference using the example of a pretrained DenseNet 121 model which detects the image.Production
Introduction to TorchScriptIntroduction to TorchScript, an intermediate representation of a PyTorch model (subclass of nn.Module) that can then be run in a high-performance environment such as C++.Production,TorchScript
Loading a TorchScript Model in C++Learn how PyTorch provides to go from an existing Python model to a serialized representation that can be loaded and executed purely from C++, with no dependency on Python.Production,TorchScript
(optional) Exporting a PyTorch Model to ONNX using TorchScript backend and Running it using ONNX RuntimeConvert a model defined in PyTorch into the ONNX format and then run it with ONNX Runtime.Production,ONNX
Profiling PyTorchLearn how to profile a PyTorch applicationProfiling_static/img/thumbnails/default.png
Profiling PyTorchIntroduction to Holistic Trace AnalysisProfiling_static/img/thumbnails/default.png
Profiling PyTorchTrace Diff using Holistic Trace AnalysisProfiling_static/img/thumbnails/default.png
Building a Convolution/Batch Norm fuser in FXBuild a simple FX pass that fuses batch norm into convolution to improve performance during inference.FX
Building a Simple Performance Profiler with FXBuild a simple FX interpreter to record the runtime of op, module, and function calls and report statisticsFX
(beta) Channels Last Memory Format in PyTorchGet an overview of Channels Last memory format and understand how it is used to order NCHW tensors in memory preserving dimensions.Memory-Format,Best-Practice,Frontend-APIs
Using the PyTorch C++ FrontendWalk through an end-to-end example of training a model with the C++ frontend by training a DCGAN – a kind of generative model – to generate images of MNIST digits.Frontend-APIs,C++
Python Custom Operators Landing PageThis is the landing page for all things related to custom operators in PyTorch.Extending-PyTorch,Frontend-APIs,C++,CUDA
Python Custom OperatorsCreate Custom Operators in Python. Useful for black-boxing a Python function for use with torch.compile.Extending-PyTorch,Frontend-APIs,C++,CUDA
Custom C++ and CUDA OperatorsHow to extend PyTorch with custom C++ and CUDA operators.Extending-PyTorch,Frontend-APIs,C++,CUDA
Custom C++ and CUDA ExtensionsCreate a neural network layer with no parameters using numpy. Then use scipy to create a neural network layer that has learnable weights.Extending-PyTorch,Frontend-APIs,C++,CUDA
Extending TorchScript with Custom C++ OperatorsImplement a custom TorchScript operator in C++, how to build it into a shared library, how to use it in Python to define TorchScript models and lastly how to load it into a C++ application for inference workloads.Extending-PyTorch,Frontend-APIs,TorchScript,C++
Extending TorchScript with Custom C++ ClassesThis is a continuation of the custom operator tutorial, and introduces the API we’ve built for binding C++ classes into TorchScript and Python simultaneously.Extending-PyTorch,Frontend-APIs,TorchScript,C++
Dynamic Parallelism in TorchScriptThis tutorial introduces the syntax for doing *dynamic inter-op parallelism* in TorchScript.Frontend-APIs,TorchScript,C++
Real Time Inference on Raspberry Pi 4This tutorial covers how to run quantized and fused models on a Raspberry Pi 4 at 30 fps.TorchScript,Model-Optimization,Image/Video,Quantization
Autograd in C++ FrontendThe autograd package helps build flexible and dynamic nerural netorks. In this tutorial, exploreseveral examples of doing autograd in PyTorch C++ frontendFrontend-APIs,C++
Registering a Dispatched Operator in C++The dispatcher is an internal component of PyTorch which is responsible for figuring out what code should actually get run when you call a function like torch::add.Extending-PyTorch,Frontend-APIs,C++
Extending Dispatcher For a New Backend in C++Learn how to extend the dispatcher to add a new device living outside of the pytorch/pytorch repo and maintain it to keep in sync with native PyTorch devices.Extending-PyTorch,Frontend-APIs,C++
Facilitating New Backend Integration by PrivateUse1Learn how to integrate a new backend living outside of the pytorch/pytorch repo and maintain it to keep in sync with the native PyTorch backend.Extending-PyTorch,Frontend-APIs,C++
Custom Function Tutorial: Double BackwardLearn how to write a custom autograd Function that supports double backward.Extending-PyTorch,Frontend-APIs
Custom Function Tutorial: Fusing Convolution and Batch NormLearn how to create a custom autograd Function that fuses batch norm into a convolution to improve memory usage.Extending-PyTorch,Frontend-APIs
Forward-mode Automatic DifferentiationLearn how to use forward-mode automatic differentiation.Frontend-APIs
Jacobians, Hessians, hvp, vhp, and moreLearn how to compute advanced autodiff quantities using torch.funcFrontend-APIs
Model EnsemblingLearn how to ensemble models using torch.vmapFrontend-APIs
Per-Sample-GradientsLearn how to compute per-sample-gradients using torch.funcFrontend-APIs
Neural Tangent KernelsLearn how to compute neural tangent kernels using torch.funcFrontend-APIs
Performance Profiling in PyTorchLearn how to use the PyTorch Profiler to benchmark your module's performance.Model-Optimization,Best-Practice,Profiling
Performance Profiling in TensorBoardLearn how to use the TensorBoard plugin to profile and analyze your model's performance.Model-Optimization,Best-Practice,Profiling,TensorBoard
Hyperparameter Tuning TutorialLearn how to use Ray Tune to find the best performing set of hyperparameters for your model.Model-Optimization,Best-Practice
Parametrizations TutorialLearn how to use torch.nn.utils.parametrize to put constraints on your parameters (e.g. make them orthogonal, symmetric positive definite, low-rank...)Model-Optimization,Best-Practice
Pruning TutorialLearn how to use torch.nn.utils.prune to sparsify your neural networks, and how to extend it to implement your own custom pruning technique.Model-Optimization,Best-Practice
How to save memory by fusing the optimizer step into the backward passLearn a memory-saving technique through fusing the optimizer step into the backward pass using memory snapshots.Model-Optimization,Best-Practice,CUDA,Frontend-APIs
(beta) Accelerating BERT with semi-structured sparsityTrain BERT, prune it to be 2:4 sparse, and then accelerate it to achieve 2x inference speedups with semi-structured sparsity and torch.compile.Text,Model-Optimization
(beta) Dynamic Quantization on an LSTM Word Language ModelApply dynamic quantization, the easiest form of quantization, to a LSTM-based next word prediction model.Text,Quantization,Model-Optimization
(beta) Dynamic Quantization on BERTApply the dynamic quantization on a BERT (Bidirectional Embedding Representations from Transformers) model.Text,Quantization,Model-Optimization
(beta) Quantized Transfer Learning for Computer Vision TutorialExtends the Transfer Learning for Computer Vision Tutorial using a quantized model.Image/Video,Quantization,Model-Optimization
(beta) Static Quantization with Eager Mode in PyTorchThis tutorial shows how to do post-training static quantization.Quantization
Grokking PyTorch Intel CPU Performance from First PrinciplesA case study on the TorchServe inference framework optimized with Intel® Extension for PyTorch.Model-Optimization,Production
Grokking PyTorch Intel CPU Performance from First Principles (Part 2)A case study on the TorchServe inference framework optimized with Intel® Extension for PyTorch (Part 2).Model-Optimization,Production
Multi-Objective Neural Architecture Search with AxLearn how to use Ax to search over architectures find optimal tradeoffs between accuracy and latency.Model-Optimization,Best-Practice,Ax,TorchX
torch.compile TutorialSpeed up your models with minimal code changes using torch.compile, the latest PyTorch compiler solution.Model-Optimization
Inductor CPU Backend Debugging and ProfilingLearn the usage, debugging and performance profiling for ``torch.compile`` with Inductor CPU backend.Model-Optimization
(beta) Implementing High-Performance Transformers with SCALED DOT PRODUCT ATTENTIONThis tutorial explores the new torch.nn.functional.scaled_dot_product_attention and how it can be used to construct Transformer components.Model-Optimization,Attention,Transformer
Knowledge Distillation in Convolutional Neural NetworksLearn how to improve the accuracy of lightweight models using more powerful models as teachers.Model-Optimization,Image/Video
PyTorch Distributed OverviewBriefly go over all concepts and features in the distributed package. Use this document to find the distributed training technology that can best serve your application.Parallel-and-Distributed-Training
Distributed Data Parallel in PyTorch - Video TutorialsThis series of video tutorials walks you through distributed training in PyTorch via DDP.Parallel-and-Distributed-Training
Single-Machine Model Parallel Best PracticesLearn how to implement model parallel, a distributed training technique which splits a single model onto different GPUs, rather than replicating the entire model on each GPUParallel-and-Distributed-Training
Getting Started with Distributed Data ParallelLearn the basics of when to use distributed data paralle versus data parallel and work through an example to set it up.Parallel-and-Distributed-Training
Writing Distributed Applications with PyTorchSet up the distributed package of PyTorch, use the different communication strategies, and go over some the internals of the package.Parallel-and-Distributed-Training
Large Scale Transformer model training with Tensor ParallelLearn how to train large models with Tensor Parallel package.Parallel-and-Distributed-Training
Customize Process Group Backends Using Cpp ExtensionsExtend ProcessGroup with custom collective communication implementations.Parallel-and-Distributed-Training
Getting Started with Distributed RPC FrameworkLearn how to build distributed training using the torch.distributed.rpc package.Parallel-and-Distributed-Training
Implementing a Parameter Server Using Distributed RPC FrameworkWalk through a through a simple example of implementing a parameter server using PyTorch’s Distributed RPC framework.Parallel-and-Distributed-Training
Introduction to Distributed Pipeline ParallelismDemonstrate how to implement pipeline parallelism using torch.distributed.pipeliningParallel-and-Distributed-Training
Implementing Batch RPC Processing Using Asynchronous ExecutionsLearn how to use rpc.functions.async_execution to implement batch RPCParallel-and-Distributed-Training
Combining Distributed DataParallel with Distributed RPC FrameworkWalk through a through a simple example of how to combine distributed data parallelism with distributed model parallelism.Parallel-and-Distributed-Training
Getting Started with Fully Sharded Data Parallel(FSDP)Learn how to train models with Fully Sharded Data Parallel package.Parallel-and-Distributed-Training
Advanced Model Training with Fully Sharded Data Parallel (FSDP)Explore advanced model training with Fully Sharded Data Parallel package.Parallel-and-Distributed-Training
Introduction to Libuv TCPStore BackendTCPStore now uses a new server backend for faster connection and better scalability.Parallel-and-Distributed-Training
Exporting to ExecuTorch TutorialLearn about how to use ExecuTorch, a unified ML stack for lowering PyTorch models to edge devices.Edge
Running an ExecuTorch Model in C++ TutorialLearn how to load and execute an ExecuTorch model in C++Edge
Using the ExecuTorch SDK to Profile a ModelExplore how to use the ExecuTorch SDK to profile, debug, and visualize ExecuTorch modelsEdge
Building an ExecuTorch iOS Demo AppExplore how to set up the ExecuTorch iOS Demo App, which uses the MobileNet v3 model to process live camera images leveraging three different backends: XNNPACK, Core ML, and Metal Performance Shaders (MPS).Edge
Building an ExecuTorch Android Demo AppLearn how to set up the ExecuTorch Android Demo App for image segmentation tasks using the DeepLab v3 model and XNNPACK FP32 backend.Edge
Lowering a Model as a DelegateLearn to accelerate your program using ExecuTorch by applying delegates through three methods: lowering the whole module, composing it with another module, and partitioning parts of a module.Edge
Exploring TorchRec shardingThis tutorial covers the sharding schemes of embedding tables by using EmbeddingPlanner and DistributedModelParallel API.TorchRec,Recommender
Introduction to TorchMultimodalTorchMultimodal is a library that provides models, primitives and examples for training multimodal tasksTorchMultimodal
Examples of PyTorch
A set of examples around PyTorch in Vision, Text, Reinforcement Learning that you can incorporate in your existing work.
Check Out Examples
PyTorch Cheat Sheet
Quick overview to essential PyTorch elements.
Open
Run Tutorials on Google Colab
Learn how to copy tutorial data into Google Drive so that you can run tutorials on Google Colab.
Open
