Seonglae ChoThe potential of a third learning paradigm that learns and edits text instructions without changing model parameters (Pretraining → Fine Tuning → )
Andrej Karpathy on Twitter / X
We're missing (at least one) major paradigm for LLM learning. Not sure what to call it, possibly it has a name - system prompt learning?Pretraining is for knowledge.Finetuning (SL/RL) is for habitual behavior.Both of these involve a change in parameters but a lot of human…— Andrej Karpathy (@karpathy) May 11, 2025
https://x.com/karpathy/status/1921368644069765486
Claude’s System Prompt: Chatbots Are More Than Just Models
A couple days ago, Ásgeir Thor Johnson convinced Claude to give up its system prompt. The prompt is a good reminder that chatbots are more than just their model. They’re tools and instructions that accrue and are honed, through user feedback and design.
https://www.dbreunig.com/2025/05/07/claude-s-system-prompt-chatbots-are-more-than-just-models.html
Meta Learning
MetaSPO with Inner Loop, Outer Loop for Unseen Generalization and Test-Time Adaptation