OmniParser
Computer use
OmniParser is a Microsoft research tool that parses user interface screenshots into structured, easy-to-read elements. The repository describes it as a screen parsing tool towards pure vision based GUI agents. The repository includes a Gradio demo, a Python notebook with examples, and OmniTool, which controls a Windows 11 virtual machine with a vision model of the developer's choice.
The license is shown inconsistently on the repository page, so the license field is left empty.

Pinout
- MCP server: not checked yet
- HTTP API: not checked yet
- command line: not checked yet
- Python SDK: supported, verified
- TypeScript SDK: not checked yet
- OpenAPI description: not checked yet
| Interface | Status | Source we read | Read on |
|---|---|---|---|
| MCP server | Not checked yet | — | — |
| HTTP API | Not checked yet | — | — |
| Command line | Not checked yet | — | — |
| Python SDK | Supported, verified | OmniParser repository README, demo.ipynb examples | |
| TypeScript SDK | Not checked yet | — | — |
| OpenAPI description | Not checked yet | — | — |
Facts
| Description | A simple screen parsing tool towards pure vision based GUI agent. OmniParser repository README, read . |
|---|---|
| License shown | The repository sidebar lists CC-BY-4.0 while the README badge links to MIT; the page does not settle it. OmniParser repository, sidebar and README, read . |
| Install | conda environment with Python 3.12, pip install -r requirements.txt, model weights fetched from Hugging Face. OmniParser repository README, install, read . |
| Demo | python gradio_demo.py starts a Gradio demo. OmniParser repository README, usage, read . |
| Python usage | Examples are provided in demo.ipynb. OmniParser repository README, usage, read . |
| OmniTool | Controls a Windows 11 virtual machine with OmniParser and a vision model of the user's choice. OmniParser repository README, OmniTool, read . |
| OmniTool model providers named | OpenAI, DeepSeek, Qwen and Anthropic Computer Use. OmniParser repository README, OmniTool, read . |
| Last commit | . GitHub API, latest commit on the default branch of microsoft/OmniParser, read . |
| Frameworks | OpenAI (OmniParser repository README, OmniTool, read 2026-11-16); DeepSeek (OmniParser repository README, OmniTool, read 2026-11-16); Qwen (OmniParser repository README, OmniTool, read 2026-11-16); Anthropic Computer Use (OmniParser repository README, OmniTool, read 2026-11-16). |
Limits
- One weight is fetched from an open Hugging Face pull request until it is merged.
- The detector licenses differ between releases, as the README notes.
Strengths
- Model weights are hosted on Hugging Face.
- A Gradio demo and a notebook of examples are included.
- OmniTool pairs the parser with several named model providers.
Alternatives to OmniParser
All computer use tools for agents
CLI that lets an agent operate desktop apps through accessibility trees.

MCP server that lets AI agents operate Windows applications.

Simular's open-source framework for computer-use agents, plus a hosted API.

Python SDK that drives a Playwright browser from natural-language steps.
Last check: . Method: public repository and docs read automatically, page and day written down. Next check due within 90 days. How we check
Is this your tool? Write to hello@agent-tools.directory from an address on its domain to correct a pin or a fact.