Computer Use

An AI capability that lets a model operate a computer like a person: looking at the screen, moving the pointer, clicking and typing.

Last updated

AI & Automation

In plain English

Lots of apps and websites have no easy way for software to connect to them. Computer use gets around that. The AI looks at the screen and uses the mouse and keyboard, so it can work in almost any app.

Example

An AI agent fills in expense reports by opening the company's old expense website, typing in each receipt and clicking submit, just like an employee would.

In short

Mouse and keyboard included. AI that clicks and types on a screen like you do.

Why it matters

Computer use lets AI agents work with older software and websites that have no API. It's still slower and less reliable than a direct connection, and it raises security questions, so it usually runs in a sandbox with limits.

Common confusion

Computer use vs. tool calling. Tool calling connects an AI straight to software through an API, which is fast and precise. Computer use goes through the screen like a person, which is slower but works almost anywhere.

Sources

  1. Anthropic: Computer use tool
  2. OpenAI: Computer-Using Agent