(yet) Understanding NVIDIA GPU Utilization

< 목차 >

• CUDA Mem Alloc Retries
• References

CUDA Mem Alloc Retries

Fig.

Fig.

References

• Understanding NVIDIA GPU Performance: Utilization vs. Saturation (2023)
• Why Utilizing The Maximum Amount Of Memory (Almost) Never Leads To A Better Training Throughput

• lessw2020/transformer_central

• DeepSpeed Memory Flush Warning Logic
  • zero3 memory flush
  • memory_stats
  • torch.cuda.memory_stats