The ability to look directly at your monitor and control software with a mouse and keyboard is the cornerstone of this release. Here is a calm, practical breakdown of its mechanism, real costs compared to rivals, practical adoption criteria, and essential caveats.
3-Line Summary
- GPT-6 Astra moves beyond text answers to directly observe your computer screen, calculating pixel coordinates to click and type through multi-step OS workflows.
- Its complex task success rate on the benchmark OSWorld 2.0 jumped to 72.6%, featuring significantly sharper coordinate accuracy and faster mouse response than GPT-5.6 Sol.
- Because each action captures a full screenshot consuming vision tokens, it is not meant for quick answers, but delivers huge leverage when automating repetitive 15+ minute desktop chores.
Asking AI to draft an email or summarize a lengthy document has become second nature. Yet if you examine our daily screen time, actual work rarely stops at text chat.
You open a browser, close annoying modal popups, enter search queries, download spreadsheets, paste columns into another window, and organize files into folders. Traditional chat models gave fluent answers on "how" to do this, but the physical labor of moving the mouse and pressing buttons still fell squarely on human shoulders.
Announced by OpenAI on September 3, 2026, GPT-6 Astra is designed to take over this exact computer labor. By visually recognizing screen UI elements and generating native OS mouse and keyboard inputs, it advances toward genuine end-to-end task completion.
Let's look at how it navigates your desktop, what it actually costs, and where it makes financial sense to deploy.
How It Works: The 3-Step Screen-and-Act Mechanism
Astra doesn't hook into hidden APIs; it interacts with your computer just like a person sitting in front of a monitor.
[Step 1: Screen Capture & Coordinate Mapping]
OS Screenshot ──> Calculate (X, Y) pixel coordinates of buttons and text fields
│
▼
[Step 2: Action Planning]
Match intent with UI ──> Plan action: "Click close button" then "Type query into search bar"
│
▼
[Step 3: Virtual Input & Visual Confirmation]
Simulate mouse click & keyboard entry ──> Retake screenshot to confirm UI changed as intended
1. Screen Capture and Coordinate Mapping
Astra does not stream video continuously. Instead, it takes a high-resolution screenshot at every decision point and feeds it into its visual encoder. It calculates the exact (X, Y) pixel center of buttons, menus, and input fields.
2. Virtual Mouse and Keyboard Execution
Once coordinates are locked, the agent issues OS-level commands to move the virtual cursor, click, double-click, drag, type text, or scroll the wheel down to reveal hidden content.
3. Feedback Loop Confirmation
Clicking a button isn't the end of a task. The model snaps another screenshot to verify that the file download dialog or modal appeared. If the page is still loading or an unexpected error dialog pops up, it detects the state and either waits or works around it.
Benchmark Comparison: GPT-6 Astra vs. GPT-5.6 Sol
On the standard OSWorld 2.0 benchmark, which evaluates real-world desktop and office automation across hundreds of tasks, the generational leap is clear:
| Metric | Previous Gen (GPT-5.6 Sol) | Flagship (GPT-6 Astra) | Tangible Difference in Daily Use |
|---|---|---|---|
| OSWorld 2.0 Success Rate | 38.2% | 72.6% | Success rate across multi-window desktop tasks nearly doubled |
| Average Task Duration | ~4 min 12 sec | ~2 min 15 sec | Visual processing and decision latency reduced by ~47% |
| Coordinate Precision | 82.4% | 96.8% | Far fewer missed clicks on tiny close buttons or crowded menus |
| Multi-Window Navigation | Lost path after 2~3 windows | Up to 12 consecutive window switches | Smoothly bridges browser, Excel, and chat clients via Alt+Tab |
| Safety Guardrail Audit | Medium | Critical Pass | Meets rigorous standards blocking rogue scripts or unauthorized commands |
Where previous models excelled inside browser text but got hopelessly lost when switching between Excel and local folders, Astra uses native window shortcuts to coordinate multiple applications seamlessly.
Real-World Cost & Pricing Comparison
When evaluating computer-use models, real monetary cost is the first question to ask.
Because every action consumes a full screenshot (roughly 1,200 to 1,600 vision tokens per image), computer-use agents burn tokens much faster than standard text chat.
Here is how token rates and an estimated 10-step desktop task compare against rivals:
| Model | Input Price (1M Tokens) | Output Price (1M Tokens) | Estimated Cost per 10-Step Task | Value Proposition |
|---|---|---|---|---|
| GPT-6 Astra (OpenAI) | $10.00 | $50.00 | ~$0.20 – $0.35 (approx. 280–480 KRW) | Highest price, but highest success rate (72.6%) across complex multi-window software |
| Claude 3.7 Sonnet (Anthropic) | $3.00 | $15.00 | ~$0.07 – $0.12 (approx. 90–160 KRW) | Excellent balance for code editors and browser-centric tasks |
| Gemini 2.0 Pro (Google) | $1.25 | $5.00 | ~$0.03 – $0.06 (approx. 40–80 KRW) | Lowest cost for quick web research and data extraction |
| GPT-5.6 Sol (Previous Gen) | $2.50 | $10.00 | ~$0.05 – $0.08 (approx. 70–110 KRW) | Cheaper token cost, but high failure rate outside a single window forces costly retries |
Economic Rule of Thumb
Using Astra for a simple 30-second web search is a waste of money.
However, if an employee spends 15 to 20 minutes every morning opening three vendor sites, checking inventory numbers, and logging them into a spreadsheet, spending $0.25 to $0.35 (around 300 to 450 KRW) to automate that chore frees valuable human focus and pays for itself immediately.
Where to Deploy Right Now (And What to Avoid)
Great Fits for Immediate Automation
- Daily routine dashboard logging: Navigating to private portals, snapping status screenshots, or exporting weekly CSVs.
- Form replication & data entry: Reading entries from an uploaded PDF and typing them into internal CRM or ERP fields one by one.
- Batch file renaming & sorting: Sorting through dozens of downloaded files and renaming them according to uniform date and naming rules.
Tasks Better Left to Humans
- Financial payments & bank transfers: Credit card checkouts, OTP verifications, and wire transfers (safety guardrails will automatically halt the agent).
- Legally binding approvals: Final contract signatures and sending official external proposals to clients.
- Websites with aggressive CAPTCHAs: "I am not a robot" puzzle modals cannot be solved autonomously by the agent and require human intervention.
Frequently Asked Questions
Can I move my mouse while Astra is working?
No. Because Astra directly controls your operating system's hardware cursor, moving your physical mouse disrupts the visual coordinate tracking. It is best to let the machine run uninterrupted or dedicate a virtual desktop to the task.
Does it work in standard ChatGPT web chat?
No. Standard web chat lacks system-level OS permissions. You must use the dedicated desktop app in Work mode or utilize developer API hooks to enable computer control.
Is there a risk of leaking private data through screenshots?
OpenAI implements strict safeguards preventing unauthorized shell commands or file tampering. However, since the agent captures full-screen images, sensitive chat windows or financial balances open on your desktop could be included in the visual stream. Always close private personal windows before delegating work.
What happens when a CAPTCHA appears?
Safety policies forbid autonomous CAPTCHA solving. The agent will pause and send you a notification saying "Please resolve the CAPTCHA." Once you complete the verification, Astra will resume its sequence.
If a task fails midway, am I still charged?
Yes. Token consumption occurs on every individual screenshot and reasoning step regardless of eventual task outcome. Writing clear, unambiguous starting instructions with target URLs and file names is the best way to safeguard against wasted spend.