Seonglae ChoGenerative Pre-trained Transformer
Each 0.5 in the version is roughly 10X pretraining compute. - Andrej Karpathy fl
GPT Implementations
History
GPT in simple explanation
GPT in 60 Lines of NumPy | Jay Mody
January 30, 2023 In this post, we'll implement a GPT from scratch in just 60 lines of numpy . We'll then load the trained GPT-2 model weights released by OpenAI into our implementation and generate some text. Note: This post assumes familiarity with Python, NumPy, and some basic experience training neural networks.
https://jaykmody.com/blog/gpt-from-scratch/
