Google has released ML Drift, an open-source GPU compute engine for on-device AI inference that already runs on millions of devices inside Chrome, YouTube Shorts, Photos and Meet. Licensed under Apache 2.0, it abstracts differences between GPU hardware and low-level APIs, supporting OpenGL ES, OpenCL, Metal and WebGPU. Benchmarks on Gemma models show 12% lower memory overhead than frameworks including llama. cpp and MLX LM, which matters for companies seeking to run generative AI outside the data center.
What ML Drift replaces in LiteRT stack
ML Drift serves as the core GPU acceleration engine inside LiteRT, Google's runtime for on-device machine learning. The release allows developers to build custom graphics and inference runtimes for computer vision, audio, depth processing and generative AI. Google confirmed it will stop updating the legacy TensorFlow Lite GPU delegate, recommending migration to the LiteRT ML Drift GPU accelerator for backward compatibility with existing models. Application developers use that accelerator directly, while GPU and system engineers get a lower-level C++ API for custom GpuModel compute graphs and shaders.
The engine introduces unified shaders through tensor virtualization, extensible custom operations and five-dimensional tensor support. This design decouples logical tensors from physical memory allocations, a change that AI engineer Pavithra Agraharam described as cutting memory fragmentation during live inference. Native 5D support removes layout workarounds for spatial and transformer models on edge GPUs. Google also added hardware-specific optimizations for GPUs from Arm, Intel and Qualcomm Technologies to raise utilization across modern mobile and desktop chips.
The move addresses fragmentation across GPU architectures, driver versions and low-level APIs on edge devices, where developers rarely know target hardware in advance. Google positions ML Drift as a foundation for portability plus stability for traditional ML and peak speed for generative AI. Desktop coverage now includes Windows and Linux through Dawn, Chromium's WebGPU implementation, while macOS keeps a native Metal backend. By relying on WebGPU hardware abstraction, Dawn avoids direct work with DirectX and Vulkan, simplifying cross-platform deployment.
What faster edge inference means for business
For product teams, the practical effect is lower latency and memory cost for camera, photo and video features without sending data to servers. In YouTube Shorts, segmentation effects run with up to 40% lower average frame latency on Android and iOS. Google Photos accelerates computational photography and segmentation pipelines by up to two seconds. Snapchat reports 30% lower model latency for face and style-generation effects on Android, which allows larger diffusion models on phones, while Adobe Lightroom and Photoshop gain up to 30% on functions such as Select Subject, Select Sky and Adaptive Portrait.
The release does not answer every procurement question, and teams should test claims on their own device mix. Founder of Tensorust Matthew Charles Vladislav Busel asked how ML Drift compares with Core ML on recent iPhones for latency and memory, and Google has published no comparative figures. Results cited for Shorts, Photos, Adobe and Snapchat reflect selected pipelines, not a guarantee across all models. Buyers should verify driver support, behavior of custom operations and migration effort from the old TFLite delegate before planning rollouts.
The marker to watch is migration away from the frozen TensorFlow Lite GPU delegate toward the LiteRT ML Drift accelerator in shipping apps. If independent developers report stable performance across Arm, Intel and Qualcomm GPUs on Windows, Linux, macOS, Android and iOS, the engine can become a default path for on-device generative features. Absence of such reports would signal that portability still requires platform-specific tuning.
