NPU (neural processing unit)
An NPU (neural processing unit) is the part of a phone or computer chip built to run the math inside AI models quickly and at low power. AGI, Inc. can run its AGI-ACT action models on the NPU through AGI-RUN, its inference engine for phones and edge hardware.
Last updated
Related terms
On-device inference
On-device inference means running a trained AI model on the device itself, such as a phone or PC, instead of on a server.
Inference engine
An inference engine is the software that runs a trained AI model on the chips a device has.
Prefill throughput
Prefill throughput is how fast an AI model reads its input before it starts writing, measured in tokens per second.
Also called | AI accelerator |
|---|---|
Found in | Phones, PCs and other edge devices |
Measured in | TOPS, trillions of operations per second |
AGI use | AGI-ACT on the Snapdragon NPU through AGI-RUN |
What does an NPU do that a CPU or GPU does not?
An NPU is built for the operations AI models repeat millions of times, so it runs them faster than a CPU and with less power than a GPU. On a phone, battery and heat set the limits.
Chip | Built for | Running AI models |
|---|---|---|
CPU | General tasks | Works for small models, but slowly |
GPU | Graphics and other parallel math | Fast, with higher power use |
NPU | The math inside neural networks | Fast, at low power |
Why does an AI agent run on the NPU?
An agent completes a task over many steps, so speed and power use add up quickly. Running on the NPU lets a phone handle those steps on the device instead of sending them to a server. At Snapdragon Summit 2026, AGI's action model completed a task on a phone with no connection, running on the Snapdragon NPU.
FAQ
Do all phones have an NPU?
No. Most recent flagship phone chips include one, and NPU speed varies a lot between chips. An on-device action model needs a fast enough NPU and enough memory to hold the model.
What is TOPS?
TOPS means trillions of operations per second. Chip makers use it to describe the peak AI performance of an NPU. It is a rough guide, since real speed also depends on memory and on the inference engine.
Does AGI's model run on the NPU?
Yes. AGI-ACT runs on the Snapdragon NPU through AGI-RUN, AGI's inference engine, as part of the collaboration with Qualcomm Technologies that AGI, Inc. announced in March 2026.
