AGI-LEARN: how our agent improves itself from its own production traffic

AGI-LEARN, AGI, Inc.'s self-learning engine: every failed task in production is re-tested on our own device fleet, trained out, and kept as a regression test.

Our agent improves itself from live traffic. Every failed task is re-tested on our own device fleet, trained out, and kept as a regression test. Every model passes the full suite before shipping.

AGI Research Team · October 2026

TL;DR

We're introducing AGI-LEARN, the engine that securely uses our agent's production traffic to teach its next model. Every live trace maps onto a userflow in Atlas, our map of what people do inside apps. Failures are sorted by type, rebuilt on our own device fleet with equivalent accounts and data in place of the user's, and corrected with reinforcement learning that scores the agent step by step.
Training aims at named userflows and named failure types. Weakness is measured and fixed per flow (checkout, book a ride, send a message), so a fix carries to every app and task that flow appears in.
Nothing is promoted without clearing the full regression suite. Every fixed failure becomes a permanent test, grouped by domain and issue type, and the promoted model's own traffic re-enters the loop.

The loop

An agent that uses apps for people fails in specific, repeatable ways. Usually it is one wrong step on one screen of one flow. AGI-LEARN finds those failures in production, reproduces them, trains them out and keeps them out. It runs on our research fleet today. Partner deployments are next.

Figure 1. The AGI-LEARN cycle. Live traces land on their userflows in Atlas, failures are sorted by type and rebuilt on the device fleet, Pinpoint corrects the failing step, and Gate holds every candidate against the full regression suite before it is promoted.

Atlas

Atlas gives every trace, failure, and regression test a shared vocabulary of userflows.

Exploration. We explore each app programmatically and run our own algorithms to identify every distinct userflow. The result is that app's graph.
Userflows. A userflow is a group of similar tasks, such as checkout, book a ride, or send a message.
Library. We build graphs across a large set of apps, which gives a fixed library of userflows that live traces map onto.
Composition. Userflows are basic blocks. Complex and multi-app tasks decompose into several, so every task is categorizable.

Because everything is indexed by userflow, we measure and fix weakness per flow rather than per app or per one-off task.

Figure 2. Atlas. Userflows discovered by exploring apps form a shared library. A multi-app task decomposes into userflows from that library.

The pipeline

Map. Each live trace is categorized into one or more userflows from the Atlas library. Compound and multi-app tasks get several.
Triage. Our validation system analyzes each failed trace and sorts it into a bucket of our failure taxonomy by failure type.
Replay. Traces never contain personal data. We map the task's inputs and preconditions to equivalents that exist on our internal device fleet, generate a masked version of the task, and confirm automatically that it reproduces there.
Pinpoint. Replayed tasks enter our in-context reinforcement learning loop with a step reward. We score per dataset, localize the failing steps, and train to correct those behaviors.
Gate. Fixed failures become regression sets per domain and issue type. A model is promoted only after clearing the full suite, and its post-deploy traces re-enter step 1.

Why it matters for AGI

An agent deployed on a partner's devices has to get better at the tasks their users run, and it must not get worse at anything it already does well. AGI-LEARN does both from production traffic, with no manual triage.

Training is targeted. It aims at named userflows and failure buckets instead of bulk data, so each training run spends its budget on the flows that failed. Every training task is reproducible: it is rebuilt on our own fleet, with equivalent data in place of the user's, before it is used. Regressions cannot slip through silently, because every fixed failure is a test the next candidate has to pass.

Each promoted model's traces re-enter step 1, so every release starts from a larger regression suite and a sharper map of where the agent fails.

We share per-userflow results, failure buckets and the regression suites with the partners we deploy with. If you are putting an agent on your devices, get in touch.