Plain-AyI
← All projects
Run AI on your computer

Shimmy

A tiny single-file program that runs AI chat models on your computer and works with existing AI apps.

New this week

What it does

Shimmy is a small inference server, a program that runs an AI language model and answers requests. It loads models saved in the common GGUF format and lets other AI tools connect to it as if it were OpenAI's service.

Its makers say it is a single file with no extra dependencies, written entirely in Rust, and uses your graphics card through WebGPU on NVIDIA, AMD, Intel and Apple hardware.

It lists a set of tested models from families such as Llama, Qwen, Phi, Gemma and DeepSeek, and the makers say it will always be free.

Who it's for

Developers and tinkerers who want a lightweight way to run AI models locally.

Can I use it without coding?

With some setup. It installs with Rust's package tool and runs from a command window. New to this? Start here.

Can I run it?

Cost
FreeNothing to pay or sign up for.
Computer
An ordinary computer is fine
Privacy
Runs on your own device: your data stays with you
API key
Not needed

Try it

  1. Install the Rust programming tools, then install Shimmy with Rust's package tool.
  2. Download a model file in GGUF format.
  3. Start Shimmy with that model and point your AI app at it.

Using Claude Code, Cursor or another AI coding assistant? Paste the prompt and it will walk you through the setup.New to this?

Before you install

  • Only download from the official GitHub page. Copies with similar names exist for many popular projects.
  • License (Apache-2.0): Permissive: free to use, change and share, including for business.
  • More on staying safe

Where people are talking about it