OSWorld
OSWorld is a public benchmark that tests computer-use agents on 369 tasks on real computers, across desktop apps, files and websites, with a script checking each result. AGI, Inc. reported 76.26% task success on OSWorld in October 2025, above the 72.36% that people achieve.
Last updated
Related terms
AndroidWorld
AndroidWorld is a public benchmark that tests AI agents on 116 tasks across 20 real Android apps.
Computer use
Computer use is an AI agent operating a computer's apps and websites through the screen, the way a person does.
Vision-language-action (VLA) model
A vision-language-action (VLA) model takes an image and an instruction and returns an action, for a robot or on a screen.
Built by | Xie et al. (NeurIPS 2024) |
|---|---|
Tasks | 369, on real computers |
Operating systems | Ubuntu, Windows and macOS supported |
People | 72.36% |
AGI, Inc. | 76.26% (October 2025) |
What does OSWorld test?
It tests whether an agent can complete real computer tasks across desktop apps, files and websites, including workflows that span several apps. Examples include turning on a privacy setting in a web browser and changing an image's color mode in an image editor.
How is OSWorld scored?
Each task has a detailed starting setup and its own evaluation script that checks the result on the computer. The score is the share of tasks completed. In the original paper, people completed 72.36% of the tasks.
What did AGI, Inc. score?
AGI, Inc. reported 76.26% task success in October 2025, above the human baseline. OSWorld measures computer use. For phones, see AndroidWorld.
FAQ
What is OSWorld?
A public benchmark for computer-use agents with 369 tasks on real computers, each checked by its own evaluation script. It was introduced by Xie et al. and presented at NeurIPS 2024.
How do people score on OSWorld?
In the original paper, people completed 72.36% of the tasks. AGI, Inc. reported 76.26% in October 2025. Scores on benchmarks like this change quickly, so check the date of any result you compare.
How is OSWorld different from AndroidWorld?
OSWorld tests agents on desktop computers, across desktop apps, files and websites. AndroidWorld tests agents on Android phone apps, with 116 tasks across 20 apps. AGI-0 scored 100% on AndroidWorld (116/116 tasks).
