(yet) CUDA Graph and Torch Compile

< 목차 >

• CUDA Graphs
  • tmp
• torch.compile
• References

CUDA Graphs

tmp

torch.compile

References

• Blogs, Docs and Others
  • Accelerating PyTorch with CUDA Graphs
  • Getting Started with CUDA Graphs
  • Docs > torch.compiler > CUDAGraph Trees
  • Docs > CUDA semantics

  • CUDA 스트림(stream)을 활용한 동시성 향상

  • torch.compile, the missing manual
  • Docs > torch > torch.compile
  • Tutorials > Introduction to torch.compile
  • Docs > torch.compiler > Getting Started
  • Docs > torch.export > Dynamo Overview
  • SimpleFSDP: Simpler Fully Sharded Data Parallel with torch.compile
• Videos
  • torch.compile: The Missing Manual
  • Torch.Compile for Autograd, DDP and FSDP - Will Feng , Chien-Chin Huang & Simon Fan, Meta
  • ACCELERATING PYTORCH NETWORKS WITH NATIVE CUDA GRAPHS SUPPORT from MICHAEL CARILLI
  • PyTorch 2.0 Live Q&A Series: A Deep Dive on TorchDynamo
  • CUDA Graphs Tutorial