(yet) NanoGPT Speedrun and MomentUm Orthogonalized by Newton-Schulz (Muon)

< 목차 >

• tmp
• References

tmp

Fig. Source from link

References

• tweet from Keller Jordan
  • KellerJordan/modded-nanogpt
• Muon: An optimizer for hidden layers in neural networks

• tweet from Jeremy Bernstein

• The Power Method from Robert van de Geijn