inference_mode also disables to forward-mode AD
The primary difference between
torch.no_grad() and torch.inference_mode() in PyTorch is that torch.no_grad() temporarily disables gradient calculation, while torch.inference_mode() not only disables gradient computation but also optimizes specific operations for inference.inference_mode — PyTorch 2.1 documentation
Community
https://pytorch.org/docs/stable/generated/torch.inference_mode.html

Seonglae Cho