Seonglae ChoRemote Direct Memory Access
RDMA (Remote Direct Memory Access) is commonly used alongside NVMe-oF. It allows direct memory access from one computer to another without CPU involvement, dramatically reducing CPU utilization.
Key Benefits
- Provides significant advantages when computing and storage are disaggregated
- Enables compute nodes to retrieve information from remote storage directly into memory without CPU intervention
- Allows virtual machines on compute nodes to utilize more physical CPU resources
Implementation Considerations
- Requires RDMA-compatible NICs and drivers
- Hardware support must be verified before deployment
Storage Protocols
- Several RDMA-based storage protocols exist:
- NVMe-oF (NVMe over Fabrics)
- SRP (Storage RDMA Protocol)
- iSER (iSCSI Extension for RDMA)
- Mellanox has endorsed NVMe-oF as their development standard due to its superior performance
RDMA Protocol
pplx-gardenperplexityai • Updated 2026 Jun 25 9:30
pplx-garden
perplexityai • Updated 2026 Jun 25 9:30
Despite the ConnectX-7 hardware dependency, it clearly analyzed the bottlenecks inherent in collective communication and presented practical engineering solutions to avoid them.
Notably, this technique increased the weight update speed by more than 100× in RL-based large language model training.
Weight Transfer for RL Post-Training in under 2 seconds
Ultra-fast cross-GPU model sync
https://research.perplexity.ai/articles/weight-transfer-for-rl-post-training-in-under-2-seconds
[네이버클라우드 기술&경험] 가상화 개념 이해하기#2, SPDK와 NVMe-oF 알아보기
SPDK는 User space에서 사용하는 스토리지 I/O 가속 솔루션입니다. 개발용 라이브러리, 가상머신용 vhost 등 다양한 사용 모델을 제공하고 있으며 iSCSI, NVMe-oF를 위한 모듈도 지원하고 있습니다. 가상머신에서 발생하는 I/O의 경우 지난 시간에 알아본 KVM의 vhost (이하 k-vhost)가 범용적으로 사용되어 왔습니다. 네트워크는 k-vhost-net, 스토리지는 k-vhost-scsi, k-vhost-blk 등이 있습니다.
![[네이버클라우드 기술&경험] 가상화 개념 이해하기#2, SPDK와 NVMe-oF 알아보기](https://miro.medium.com/1*m-R_BkNf1Qjr1YbyOIJY2w.png){kind=small}
https://medium.com/naver-cloud-platform/%EB%84%A4%EC%9D%B4%EB%B2%84%ED%81%B4%EB%9D%BC%EC%9A%B0%EB%93%9C-%EA%B8%B0%EC%88%A0-%EA%B2%BD%ED%97%98-%EA%B0%80%EC%83%81%ED%99%94-%EA%B0%9C%EB%85%90-%EC%9D%B4%ED%95%B4%ED%95%98%EA%B8%B0-2-spdk%EC%99%80-nvme-of-%EC%95%8C%EC%95%84%EB%B3%B4%EA%B8%B0-6eae4ef1dd4c