Microsoft finds offloading robot AI inference boosts task success and battery life

Microsoft Research says robots that send their AI computations to nearby or cloud-based computers, rather than doing the work onboard, complete tasks more successfully and last longer on a single charge. The finding comes from what the company calls the first systematic study of robotics workloads, published on its research blog.

The team tested a common household task: checking a kitchen for rubbish and carrying it to the trash. Completing it requires a robot to plan a route, recognize objects in its surroundings, navigate to the trash, pick it up, and find a path back. Microsoft Research evaluated three parts of that pipeline, semantic mapping and planning, navigation, and manipulation, across a range of onboard, edge, and cloud GPUs, the chips that run AI models.

The results favored moving the work off the robot. On smaller onboard GPUs, mapping and planning slowed by as much as 383% compared with an A100, a high-end data-center GPU Microsoft used as its benchmark. Navigation systems missed obstacles 30% more often when timely detection depended on lighter onboard hardware. Vision-language-action models, or VLA models, which turn a robot’s camera feed and instructions into physical movement, kept roughly the same speed on weaker GPUs but lost up to half their accuracy.

Battery life told a similar story. Microsoft Research compared robots running a power-hungry onboard GPU against ones fitted with a lightweight Raspberry Pi-5 board that shipped its data to an offloaded GPU elsewhere. On the Stretch-3 robot, swapping in a larger onboard chip such as Nvidia’s Jetson Thor cut battery life by up to 160%, costing several hours of runtime.

A toolkit for splitting the work

To put the approach into practice, Microsoft is adding an inference-offloading capability to its Physical AI Toolchain, an open-source framework the company released to combine Azure cloud services with Nvidia’s robotics software stack. The new tools let developers containerize robot AI workloads, meaning package them into portable, self-contained units, and distribute them across a robot, nearby edge hardware, and the cloud using Kubernetes, software built for managing clusters of containers. Microsoft says the toolkit also connects to robotics simulators, the LeRobot framework, and ROS2, a widely used robotics software platform.

The release includes example projects for two robot arms, the SO-101 and the UR10e. In demonstration videos, Microsoft shows its Rho model, built for dual-arm robots, controlling a Mobile Aloha robot to press a button and turn a knob while its inference runs on a Jetson Thor GPU rather than onboard.

Microsoft Research acknowledges the approach involves tradeoffs: offloading exchanges onboard cost and battery drain for dependence on network latency, bandwidth, and the availability of edge or cloud GPU capacity. The company has published a technical report detailing its test hardware and says it is now collecting feedback from developers using the toolkit’s SO-101 and UR10e example projects in real-world deployments.