onnx

Developed collaboratively by Microsoft, Meta, and partners, ONNX defines a common graph format and operator specification that frameworks including PyTorch, TensorFlow, scikit-learn, and XGBoost can export to, and that inference runtimes including ONNX Runtime, TensorRT, OpenVINO, and CoreML can execute.

The practical value of ONNX is decoupling the training environment from the deployment environment a model trained in PyTorch for research can be exported to ONNX and deployed via ONNX Runtime on CPU, GPU, or NPU hardware with hardware-specific optimizations applied automatically by the runtime.

This separation allows ML engineers to choose the best training framework for productivity and the best inference runtime for performance independently, rather than being locked into a single framework's deployment capabilities.

ONNX Runtime, the primary production inference engine for ONNX models, provides graph optimization passes (operator fusion, constant folding, layout optimization) and hardware execution providers (CUDA, DirectML, CoreML, ROCm, ARM NN) that typically produce 2-10x inference speedups over framework-native serving.

ONNX Runtime powers inference for Microsoft's Office AI features, Azure AI services, Windows ML, and thousands of independent applications. The standard is maintained by the Linux Foundation's AI & Data foundation and is supported across the major cloud providers and hardware vendors.