Circuit-Model Distance

CMD

Circuit Model Distance (CMD) measures how similar the output distribution of a circuit is to the full model by calculating the area between the CPR curve and f = 1 (where 0 is optimal).

where are the low-level nodes not in the candidate circuit. CMD measures the proportion of test-time outputs that change when each non-circuit node is individually resample-ablated.

MIB (Mechanistic Interpretability Benchmark)

All sets consist of (original, n counterfactuals) pairs, which clearly create situations where "outputs should be the same/should be different."

mib-bench (Mechanistic Interpretability Benchmark)

Principled evaluation of mechanistic interpretability methods.

https://huggingface.co/mib-bench

arxiv.org

https://arxiv.org/pdf/2504.13151

Circuit-Model Distance

CMD

MIB (Mechanistic Interpretability Benchmark)

Backlinks

Recommendations