What it does
OpenCompass checks how well large language models perform. It supports models from OpenAI, Anthropic, Gemini, Qwen, DeepSeek and others, plus open models you run yourself.
It includes more than a hundred test datasets covering knowledge, reasoning, coding, science, languages, long documents and safety. It handles the whole process, from running the model to scoring and summarising results.
The team also runs public leaderboards comparing models.
Who it's for
Researchers and companies who need to measure and compare AI model quality.
Can I use it without coding?
Developers only. You configure and run tests from the command line with Python. New to this? Start here.
Can I run it?
- Cost
- FreeNothing to pay or sign up for.
- Privacy
- Your choice: run it on your device or use an online service
- API key
- Not needed
Try it
Using Claude Code, Cursor or another AI coding assistant? Paste the prompt and it will walk you through the setup.New to this?
Before you install
- Only download from the official GitHub page. Copies with similar names exist for many popular projects.
- License (Apache-2.0): Permissive: free to use, change and share, including for business.
- More on staying safe