Plain-AyI
← All projects
Running AI at scale

Mooncake

The serving system behind the Kimi chatbot, built to handle many AI requests by sharing cached work.

New this week

What it does

Mooncake is the platform Moonshot AI uses to run its Kimi chatbot service. It is built around the KV cache, a kind of short-term memory AI models create while reading text, and shares this cache across many machines.

Its makers say this design lets Kimi handle many more requests while staying fast. Parts of Mooncake, such as its fast data-transfer engine and shared cache store, have been added to popular serving tools like vLLM, SGLang and TensorRT-LLM.

Who it's for

Engineering teams running large language models at very large scale.

Can I use it without coding?

Developers only. It's data-centre software for teams running AI serving clusters. New to this? Start here.

Can I run it?

Cost
FreeNothing to pay or sign up for.
Runs on
Server
Computer
Made for servers or data centres
Privacy
Runs on your own device: your data stays with you
API key
Not needed

Try it

Using Claude Code, Cursor or another AI coding assistant? Paste the prompt and it will walk you through the setup.New to this?

Before you install

  • Only download from the official GitHub page. Copies with similar names exist for many popular projects.
  • License (Apache-2.0): Permissive: free to use, change and share, including for business.
  • More on staying safe

Where people are talking about it