What it does
agent-desktop is a command-line tool (a program you run by typing commands) that gives AI agents "computer use": the ability to read and work in ordinary desktop apps. Instead of guessing from screenshots, it reads each app's accessibility tree, the built-in map of buttons, menus and fields that operating systems provide for assistive tools like screen readers.
An agent takes a snapshot of an app and gets back a list of every element it can act on, each with a label called a ref. It then clicks, types, scrolls or selects using that ref, and takes another snapshot to check the result. It can also manage windows, read notifications, use the clipboard and take screenshots. By default it works without moving your mouse or taking over your screen.
It comes with built-in instructions that AI agents can read, and a C library so other programs can use it directly. The README warns that snapshots, screenshots and clipboard contents can include private information, so they should be kept out of public logs.
Who it's for
Developers building AI agents that need to operate desktop apps, and people who use AI coding agents and want them to control other programs on their computer.
Can I use it without coding?
Developers only. It is installed and used through the command line and is meant to be driven by an AI agent or a program. New to this? Start here.
Can I run it?
- Cost
- FreeNothing to pay or sign up for.
- Runs on
- Mac, Windows, Linux
- Computer
- An ordinary computer is fine
- Privacy
- Runs on your own device: your data stays with you
Try it
- On a Mac, give it Accessibility permission using its built-in permissions guide
- Tell your AI agent to read the tool's built-in desktop and platform instructions before using it
Before you download
Plain-AyI is an independent guide: we link to each project's official pages, but we don't host, review or endorse the software, and we can't accept responsibility for how it works on your device. Please download and install at your own discretion.
Using Claude Code, Cursor or another AI coding assistant? Paste the prompt and it will walk you through the setup.New to this?
Before you install
- Only download from the official GitHub page or the official download page above. Copies with similar names exist for many popular projects.
- Plain-AyI is an independent guide and doesn't host or review the software it lists. Please check each project's official page before installing.
- License (Apache-2.0): Permissive: free to use, change and share, including for business.
- More on staying safe