Power Sampling

Training-free reasoning improvement, inference-time inference algorithm with sampling-based decoding method using MCMC-based distribution resampling (distributional inference)

Traditional decoding methods adjust token-level probabilities for "local" exploration, while Power Sampling transforms the probability of entire sequences to perform "global" exploration as a meta-level decoding approach.

Power Transformation

This appears similar to changing decoding temperature, but is actually a power transformation on joint sequence likelihood. From to:

Sampling Framework

Uses Metropolis–Hastings to sample from this sharper distribution.

Reasoning with Sampling: Your Base Model is Smarter Than You Think

Reasoning with Sampling: Your Base Model is Smarter Than You Think - achieving RL-level reasoning performance through inference-time MCMC sampling.

https://aakaran.github.io/reasoning_with_sampling/

Reasoning with Sampling: Your Base Model is Smarter Than You Think

Frontier reasoning models have exhibited incredible capabilities across a wide array of disciplines, driven by posttraining large language models (LLMs) with reinforcement learning (RL). However,...

https://www.arxiv.org/abs/2510.14901v1

Power Sampling

Power Transformation

Sampling Framework

Recommendations