Texonom
Texonom
/
Engineering
Engineering
/Data Engineering/Artificial Intelligence/AI Development/AI Inference Tool/
SGLang
Search

SGLang

Creator
Creator
Seonglae Cho
Created
Created
2025 Mar 8 0:58
Editor
Editor
Seonglae Cho
Edited
Edited
2025 Mar 25 18:26
Refs
Refs
 
 

supported models

Supported Models — SGLang
python3 -m sglang.launch_server --model-path lmms-lab/llava-onevision-qwen2-7b-ov --port=30000 --chat-template=chatml-llava
Supported Models — SGLang
https://docs.sglang.ai/references/supported_models.html
Speculative Decoding
Speculative Decoding — SGLang
SGLang now provides an EAGLE2-based speculative decoding option. Our implementation aims to maximize speed and efficiency and is considered to be among the fastest in open-source LLM engines.
Speculative Decoding — SGLang
https://docs.sglang.ai/backend/speculative_decoding.html
 
 

Recommendations

Texonom
Texonom
/
Engineering
Engineering
/Data Engineering/Artificial Intelligence/AI Development/AI Inference Tool/
SGLang
Copyright Seonglae Cho