Hybrid inference
Hybrid inference is a setup where an AI model runs on the device for the steps the hardware can handle and sends heavier steps to the cloud or to on-prem servers. AGI, Inc. offers its AGI-ACT models on the device through AGI-RUN or in a hybrid setup fitted to each partner's hardware and compliance needs.
Last updated
Related terms
On-device inference
On-device inference means running a trained AI model on the device itself, such as a phone or PC, instead of on a server.
Agentic OS
An agentic OS is an operating system with an AI agent that finishes tasks across the device's apps when the user asks.
Inference engine
An inference engine is the software that runs a trained AI model on the chips a device has.
Also called | Hybrid AI |
|---|---|
On the device | Steps the hardware can handle |
Off the device | Heavier steps, in the cloud or on-prem |
AGI setup | AGI-ACT on the device through AGI-RUN, or hybrid |
Why use a hybrid setup?
A hybrid setup lets a product launch on the phones people own today. The device handles what fits on its chip and bigger steps go to a server, so the product does not have to wait until every phone has enough memory. As chips improve, more of the work moves onto the device.
What stays on the device in a hybrid setup?
Whatever runs on the device stays there. Steps sent to the cloud or to on-prem servers leave the device, so a hybrid product should say which steps those are. Running the heavier steps on-prem keeps that data inside a company's own infrastructure.
FAQ
Is hybrid inference less private than on-device inference?
Only for the steps that leave the device. Steps that run on the device keep their input there. Steps sent to the cloud or to on-prem servers do not, which is why it matters which steps run where.
Does AGI, Inc. use hybrid inference?
Yes, as one option. AGI-ACT runs on the device through AGI-RUN, and it can also run in a hybrid setup with cloud or on-prem components, fitted to each partner's hardware and compliance needs.
What is the difference between hybrid inference and cloud inference?
Cloud inference runs every step on a server. Hybrid inference runs the steps the device can handle on the device and sends only the heavier steps to a server.
