Garcon

Creator

Creator

Seonglae Cho

Created

Created

2024 Mar 16 4:37

Editor

Editor

Seonglae Cho

Edited

Edited

2024 Apr 15 12:48

Refs

Refs

A tool of infrastructure built to enable interpretability research at Anthropic

Running remote model via garçon and Garçon client connected to a model

Hooks

Pytorch Module Hook exists. However, when the model scales beyond a single node, there's no obvious way to translate that workflow.

The basic interface to probes is that you can provide a “probe function”. Probe functions accept two arguments: a “save context” which can be used to save activations or data for later, and the tensor represented at this particular point in the model. Probe functions can return an updated tensor, which will replace the probed tensor in the computation, or can return None to use the original value. (this convention is borrowed from PyTorch hooks).

you need to create a probe, run the forward pass, and then separately retrieve the activations

rmodel.recordings() returns a dictionary indexed by probe point name.

Reduction on the server

2021

You can also watch a video covering similar content to this piece.

https://transformer-circuits.pub/2021/garcon/index.html

Infrastructure - Garcon [rough early thoughts]

See also: https://transformer-circuits.pub/2021/garcon/index.html This video is part of a series: https://www.youtube.com/playlist?list=PLoyGOS2WIonajhAVqKUgEMNmeq3nEeM51 As an experiment, we recorded a couple videos discussing our early stage thinking on trying to reverse engineer neural networks. We made them to share our very informal thoughts with colleagues at other institutions. Please treat these videos like talks one might give on early results at a research group meeting. Our thinking is very rough and errors are very possible. Please take all of these videos with a big grain of salt. We expect they're primarily of interest to people actively thinking about how to reverse engineer neural networks. Our thoughts have evolved a lot since we started recording these videos.

Infrastructure - Garcon [rough early thoughts]

https://www.youtube.com/watch?v=LqvCPmbg5KI&list=PLoyGOS2WIonajhAVqKUgEMNmeq3nEeM51

Infrastructure - Garcon [rough early thoughts]

Recommendations

//////////