System Prompt Learning

Creator

Creator

Seonglae Cho

Created

Created

2025 Apr 27 22:10

Editor

Editor

Seonglae Cho

Edited

Edited

2025 Jun 13 18:54

Refs

Refs

Prompt Learning

The potential of a third learning paradigm that learns and edits text instructions without changing model parameters (Pretraining → Fine Tuning → )

Andrej Karpathy on Twitter / X

We're missing (at least one) major paradigm for LLM learning. Not sure what to call it, possibly it has a name - system prompt learning?Pretraining is for knowledge.Finetuning (SL/RL) is for habitual behavior.Both of these involve a change in parameters but a lot of human…— Andrej Karpathy (@karpathy) May 11, 2025

https://x.com/karpathy/status/1921368644069765486

Claude’s System Prompt: Chatbots Are More Than Just Models

A couple days ago, Ásgeir Thor Johnson convinced Claude to give up its system prompt. The prompt is a good reminder that chatbots are more than just their model. They’re tools and instructions that accrue and are honed, through user feedback and design.

https://www.dbreunig.com/2025/05/07/claude-s-system-prompt-chatbots-are-more-than-just-models.html

Claude’s System Prompt: Chatbots Are More Than Just Models

Meta Learning

MetaSPO with Inner Loop, Outer Loop for Unseen Generalization and Test-Time Adaptation

https://www.arxiv.org/pdf/2505.09666

Recommendations

//////////