Some Legendary Emails from OGs

< 목차 >

congratulations, @goodfellow_ian, for the test-of-time award at @NeurIPSConf !

this award reminds me of how GAN started with this one email ian sent to the @Mila_Quebec lab mailing list in May 2014. super insightful and amazing execution! pic.twitter.com/AEvcfdBfHz

— Kyunghyun Cho (@kchonyc) November 27, 2024

Ian Goodfellow가 NIPS 2024에서 test-of-time award를 수상한 기념으로 조경현 교수님이 관련 내용을 posting했는데, Adam도 나오기 전인 2014년에 learning dynamics까지 고려해서 learning algorithm을 설계한 것이 너무 감명깊은 것 같다.

+) 이런 email을 공유하는게 유행인건지 Andrej Karpathy도 최초의 (?) attention을 사용한 RNN encoder-decoder model인 Neural Machine Translation (NMT) 개발하게된 Bahdanau와의 email을 공개했는데 이것도 매우 인상깊다.

The (true) story of development and inspiration behind the "attention" operator, the one in "Attention is All you Need" that introduced the Transformer. From personal email correspondence with the author @DBahdanau ~2 years ago, published here and now (with permission) following… pic.twitter.com/hKD7gDcexS

— Andrej Karpathy (@karpathy) December 3, 2024