This section is intended to describe the necessary steps to take a PyTorch model and run it using ExecuTorch. To use the framework, you will typically need to take the following steps:
- Install the ExecuTorch python package and runtime libraries.
- Export the PyTorch model for the target hardware configuration.
- Run the model using the ExecuTorch runtime APIs on your development platform.
- Deploy the model to the target platform using the ExecuTorch runtime.
The following are required to install the ExecuTorch host libraries, needed to export models and run from Python. Requirements for target end-user devices are backend dependent. See the appropriate backend documentation for more information.
- Python 3.10 - 3.14
- g++ version 7 or higher, clang++ version 5 or higher, or another C++17-compatible toolchain.
- Linux (x86_64 or ARM64), macOS (ARM64), or Windows (x86_64).
- Intel-based macOS systems require building PyTorch from source (see Building From Source for instructions).
- On Windows, Visual Studio 2022 or later.
To use ExecuTorch, you will need to install both the Python package and the appropriate platform-specific runtime libraries. Pip is the recommended way to install the ExecuTorch python package. Consider installing it within a virtual environment, such as one provided by conda or venv.
Install PyTorch in the same command. The ExecuTorch package does not declare it
as a dependency, because the build you need depends on your hardware, so pip
cannot choose one for you. Installing ExecuTorch on its own gives an environment
where exporting a model stops with No module named 'torch'.
Both packages come from the same package index. Find your machine in the table
below and put the name from it in place of <variant>:
pip install executorch torch \
--index-url https://download.pytorch.org/whl/<variant> \
--extra-index-url https://pypi.org/simple
| Machine you export on | Variant |
|---|---|
| CPU only | cpu |
| NVIDIA GPU, CUDA 13.0 | cu130 |
| NVIDIA GPU, CUDA 13.2 | cu132 |
| NVIDIA GPU, CUDA 13.4 | cu134 |
The CUDA packages are built for Linux, on x86_64 and ARM64. Use the cpu
variant on macOS and on Windows. If your CUDA version is not in the table,
choose the closest lower one with the same major version, because CUDA works
across minor versions but not across major ones. There is no package for CUDA
12, so on CUDA 12 use the cpu variant or build from source.
To get a change that has landed on the main branch but is not in a release
yet, use a nightly build. These are rebuilt every day. Put nightly/ in front
of the variant name, for example nightly/cu130, and add --pre to the
command, otherwise pip skips development versions. Nightly builds cover the same
variants. CUDA 13.4 is the newest, and until the next release it is in nightly
builds only.
The second index is needed because a bare --index-url replaces PyPI instead of
adding to it, and some dependencies are published only on PyPI.
To build the framework from source, see Building From Source. Backend delegates may require additional dependencies. See the appropriate backend documentation for more information.
NOTE: On Windows, ExecuTorch requires a Visual Studio Developer Powershell. Running from outside of a developer prompt will manifest as errors related to CL.exe.
Exporting is the process of taking a PyTorch model and converting it to the
.pte format used by the ExecuTorch runtime. This is done using Python APIs.
Selected pre-exported artifacts are published by the
ExecuTorch Community on Hugging Face.
Use one only when its model configuration, precision, and target backend match
the runtime your application links.
A complete example of exporting, lowering, and verifying MobileNet V2 is available as a Colab notebook.
- A PyTorch model.
- Example model inputs, typically as PyTorch tensors. You should be able to successfully run the PyTorch model with these inputs.
- One or more target hardware backends.
ExecuTorch provides hardware acceleration for a wide variety of hardware. The most commonly used backends are XNNPACK, for Arm and x86 CPU, Core ML (for iOS), Vulkan (for Android GPUs), and Qualcomm (for Qualcomm-powered Android phones).
For mobile use cases, consider using XNNPACK for Android and Core ML or XNNPACK for iOS as a first step. See Hardware Backends for more information.
Exporting is done using Python APIs. ExecuTorch provides a high degree of customization during the export process, but the typical flow is as follows. This example uses the MobileNet V2 image classification model implementation in torchvision. Other models can follow the same flow when they are export-compliant and their operators are available in the ExecuTorch runtime or selected backend. For Hugging Face PreTrainedModel architectures, choose between the experimental Transformers ExecuTorch exporter for broad programmatic XNNPACK or CUDA export and Optimum ExecuTorch for higher-level, tested task workflows.
import torch
import torchvision.models as models
from torchvision.models.mobilenetv2 import MobileNet_V2_Weights
from executorch.backends.xnnpack.partition.xnnpack_partitioner import XnnpackPartitioner
from executorch.exir import to_edge_transform_and_lower
model = models.mobilenetv2.mobilenet_v2(weights=MobileNet_V2_Weights.DEFAULT).eval()
sample_inputs = (torch.randn(1, 3, 224, 224), )
et_program = to_edge_transform_and_lower(
torch.export.export(model, sample_inputs),
partitioner=[XnnpackPartitioner()]
).to_executorch()
with open("model.pte", "wb") as f:
f.write(et_program.buffer)If the model requires varying input sizes, you will need to specify the varying dimensions and bounds as part of the export call. See Model Export and Lowering for more information.
The hardware backend to target is controlled by the partitioner parameter to to_edge_transform_and_lower. In this example, the XnnpackPartitioner is used to target mobile CPUs. See the backend-specific documentation for information on how to use each backend.
Quantization can also be done at this stage to reduce model size and runtime. Quantization is backend-specific. See the documentation for the target backend for a full description of supported quantization schemes.
After successfully generating a .pte file, it is common to use the Python runtime APIs to validate the model on the development platform. This can be used to evaluate model accuracy before running on-device.
For the MobileNet V2 model from torchvision used in this example, image inputs are expected as a normalized, float32 tensor with dimensions of (batch, channels, height, width). The output is a tensor containing class logits. See torchvision.models.mobilenet_v2 for more information on the input and output tensor format for this model.
For more guidance on image preprocessing, channels-first and channels-last layouts, and CV output decoding, see Working with Computer Vision Models.
import torch
from executorch.runtime import Runtime
from typing import List
runtime = Runtime.get()
input_tensor: torch.Tensor = torch.randn(1, 3, 224, 224)
program = runtime.load_program("model.pte")
method = program.load_method("forward")
output: List[torch.Tensor] = method.execute([input_tensor])
print("Run successfully via executorch")
from torchvision.models.mobilenetv2 import MobileNet_V2_Weights
import torchvision.models as models
eager_reference_model = models.mobilenetv2.mobilenet_v2(weights=MobileNet_V2_Weights.DEFAULT).eval()
eager_reference_output = eager_reference_model(input_tensor)
print("Comparing against original PyTorch module")
print(torch.allclose(output[0], eager_reference_output, rtol=1e-3, atol=1e-5))For complete examples of exporting and running the model, please refer to our examples GitHub repository.
For generative models, Transformers' export_for_generation produces independent
.pte graph components; applications still need the appropriate preprocessing,
tokenization, generation loop, and runtime integration. Optimum ExecuTorch
supplies more of that task-level integration for its tested model paths.
ExecuTorch provides C++ runtime APIs and Module bindings for Java/Kotlin on
Android and Objective-C/Swift on Apple platforms. Stability follows the
API lifecycle policy and the annotations on each API.
Quick Links:
ExecuTorch provides Java bindings for Android usage, which can be consumed from both Java and Kotlin. To add the library to your app, add the following dependency to gradle build rule.
// app/build.gradle.kts
val executorchVersion = "X.Y.Z" // Replace with a version from Maven Central.
dependencies {
implementation("org.pytorch:executorch-android:$executorchVersion")
}Choose an available version on Maven Central.
Models can be loaded and run from Java or Kotlin using the experimental
Module class. For a stable native API on Android, use the C++ runtime.
import org.pytorch.executorch.EValue;
import org.pytorch.executorch.Module;
import org.pytorch.executorch.Tensor;
// …
Module model = Module.load("/path/to/model.pte");
Tensor input_tensor = Tensor.fromBlob(float_data, new long[] { 1, 3, height, width });
EValue input_evalue = EValue.from(input_tensor);
EValue[] output = model.forward(input_evalue);
float[] scores = output[0].toTensor().getDataAsFloatArray();Note that the C++ APIs can be used when targeting Android native.
For a full example of running a model on Android, see the DeepLabV3AndroidDemo. For more information on Android development, including building from source, a full description of the Java APIs, and information on using ExecuTorch from Android native code, see Using ExecuTorch on Android.
ExecuTorch supports iOS and macOS through C++ and through
Objective-C APIs that bridge to Swift. Core ML and XNNPACK provide accelerated
execution paths on Apple platforms. The runtime libraries are distributed as
.xcframework targets through a Swift Package Manager package.
To get started with Xcode, go to File > Add Package Dependencies and paste the
ExecuTorch repository URL into the search bar. For Dependency Rule, select
Branch and enter the swiftpm-X.Y.Z branch that matches the ExecuTorch
release used to export the model, for example swiftpm-1.4.1. The dependency
can also be added to Package.swift; see
Using ExecuTorch on iOS for details.
Models can be loaded and run through the Objective-C/Swift Module, Tensor,
and Value APIs, or through the C++ runtime from Objective-C++. The managed
APIs wrap the C++ Module and tensor extensions. See the
API lifecycle policy for stability guarantees.
For more information on iOS integration, including an API reference, logging setup, and building from source, see Using ExecuTorch on iOS.
ExecuTorch provides C++ APIs, which can be used to target embedded or mobile devices. The C++ APIs provide a greater level of control compared to other language bindings, allowing for advanced memory management, data loading, and platform integration.
On Linux and macOS, current main/nightly wheels ship the runtime as prebuilt libraries with headers and a CMake package, so there is nothing to build:
pip install --upgrade --pre executorch --extra-index-url https://download.pytorch.org/whl/nightly/cpu
find_package(executorch REQUIRED COMPONENTS kernels_optimized)
target_link_libraries(my_target PRIVATE executorch::runtime executorch::kernels_optimized)See Using the prebuilt libraries from the pip package for a complete walkthrough, and Running on a GPU with the CUDA package for GPU support.
For a platform the package does not cover, or to change build options, build from source instead.
CMake is the preferred build system for the ExecuTorch C++ runtime. To use with CMake, clone the ExecuTorch repository as a subdirectory of your project, and use CMake's add_subdirectory("executorch") to include the dependency. The executorch target, as well as kernel and backend targets will be made available to link against. The runtime can also be built standalone to support diverse toolchains. See Using ExecuTorch with C++ and Building from Source for a detailed description of build integration, targets, and cross compilation.
git clone -b viable/strict https://github.com/pytorch/executorch.git
# Set CMAKE_CXX_STANDARD to 17 or above.
set(CMAKE_CXX_STANDARD 17)
# CMakeLists.txt
set(EXECUTORCH_BUILD_PRESET_FILE ${CMAKE_SOURCE_DIR}/executorch/tools/cmake/preset/llm.cmake)
# Set other ExecuTorch options here.
add_subdirectory("executorch")
...
target_link_libraries(
my_target
PRIVATE executorch
executorch::backends
executorch::extensions
executorch::kernels)Both high-level and low-level C++ APIs are provided. The low-level APIs are platform independent, do not dynamically allocate memory, and are most suitable for resource-constrained embedded systems. The high-level APIs are provided as a convenience wrapper around the lower-level APIs, and make use of dynamic memory allocation and standard library constructs to reduce verbosity.
ExecuTorch uses CMake for native builds. Integration is typically done by cloning the ExecuTorch repository and using CMake add_subdirectory to add the dependency.
Loading and running a model using the high-level API can be done as follows:
#include <executorch/extension/module/module.h>
#include <executorch/extension/tensor/tensor.h>
using namespace ::executorch::extension;
// Load the model.
Module module("/path/to/model.pte");
// Create an input tensor.
float input[1 * 3 * 224 * 224];
auto tensor = from_blob(input, {1, 3, 224, 224});
// Perform an inference.
const auto result = module.forward(tensor);
if (result.ok()) {
// Retrieve the output data.
const auto output = result->at(0).toTensor().const_data_ptr<float>();
}For more information on the C++ APIs, see Running an ExecuTorch Model Using the Module Extension in C++ and Managing Tensor Memory in C++.
For complete examples of building and running C++ applications, please refer to our examples GitHub repository.
ExecuTorch provides a high-degree of customizability to support diverse hardware targets. Depending on your use cases, consider exploring one or more of the following pages:
- Export and Lowering for advanced model conversion options.
- Backend Overview for available backends and configuration options.
- Using ExecuTorch on Android and Using ExecuTorch on iOS for mobile runtime integration.
- Using ExecuTorch with C++ for embedded and mobile native development.
- Profiling and Debugging for developer tooling and debugging.
- API Reference for a full description of available APIs.
- Examples for demo apps and example code.