What it does
This is a reading list about inference, the work of running an already-trained AI model to get answers. It collects research papers, each with links to code where available.
Topics include splitting work across many graphics cards, shrinking models (quantization), memory tricks, faster attention methods and running models on phones and other devices. There is also a long PDF guide for beginners.
Who it's for
Engineers and researchers who make AI models run efficiently.
Can I use it without coding?
Yes. It is a list of papers you can read in any browser. New to this? Start here.
Can I run it?
- Cost
- FreeNothing to pay or sign up for.
- Runs on
- Web browser
- Computer
- An ordinary computer is fine
- API key
- Not needed
Try it
- Pick a topic from the contents and open the papers that interest you.
Using Claude Code, Cursor or another AI coding assistant? Paste the prompt and it will walk you through the setup.New to this?
Before you install
- Only download from the official GitHub page. Copies with similar names exist for many popular projects.
- License (GPL-3.0): Free to use; if you share a changed version, you must share your source code too.
- More on staying safe