truss

It provides a standardized, container-based model packaging format that handles the environment configuration, dependency installation, preprocessing, inference logic, and postprocessing that production model serving requirestaking a trained model and the code that wraps it and producing a self-contained, deployable artifact without requiring deep knowledge of Docker or Kubernetes.

The framework's model definition format captures everything needed to serve a model reproducibly: the model weights, the Python environment with pinned dependencies, the inference function with its input/output schema, and preprocessing and postprocessing code.

Truss generates optimized Docker images from this definition, handles model warmup, and produces an HTTP API with automatic request validation. It supports GPU-accelerated serving, batching, streaming outputs for generative models, and model caching for large weights that would otherwise cause slow cold starts.

ML engineers productionizing models trained in notebooks or research scripts, platform teams building internal model serving infrastructure, and companies deploying models to Baseten's cloud (where Truss is the native packaging format) use the framework to standardize the gap between model training and production serving.

The self-contained packaging model makes model serving reproducible across environmentsa meaningful improvement over ad-hoc inference scripts that accumulate environment-specific assumptions. Its open-source licensing allows teams to use the packaging format independently of Baseten's cloud platform.