What it does
SGLang is an "inference" framework: software that runs a trained AI model and answers requests with it. It's designed for serving large language models, vision-language models and image and video generation models, especially to many users at once.
Its makers say it's built for heavy workloads like AI agents, large-scale serving and the training method called reinforcement learning. It supports many types of chips, from NVIDIA and AMD graphics cards to Google TPUs and Apple Silicon Macs.
Related projects add audio models for speech and tools for speeding up responses.
Who it's for
Engineers and companies who run AI models on their own servers for many users.
Can I use it without coding?
Developers only. It's installed with Docker or Python and configured from the command line, usually on powerful servers. New to this? Start here.
Can I run it?
- Cost
- FreeNothing to pay or sign up for.
- Runs on
- Mac, Server
- Computer
- Works best on a powerful computer or one with a graphics card (GPU)
- Privacy
- Runs on your own device: your data stays with you
- API key
- Not needed
Try it
Using Claude Code, Cursor or another AI coding assistant? Paste the prompt and it will walk you through the setup.New to this?
Before you install
- Only download from the official GitHub page. Copies with similar names exist for many popular projects.
- License (Apache-2.0): Permissive: free to use, change and share, including for business.
- More on staying safe