Export Hugging Face models to ONNX
Request model evaluation on COCO val 2017 dataset
View and submit machine learning model evaluations
View and submit LLM benchmark evaluations
Explore and manage STM32 ML models with the STM32AI Model Zoo dashboard
Search for model performance across languages and benchmarks
Convert Hugging Face models to OpenVINO format
Display and filter leaderboard models
Evaluate LLM over-refusal rates with OR-Bench
Text-To-Speech (TTS) Evaluation using objective metrics.
Visualize model performance on function calling tasks
Browse and filter ML model leaderboard data
Browse and evaluate ML tasks in MLIP Arena
Export to ONNX is a tool designed to convert Hugging Face models into the ONNX (Open Neural Network Exchange) format. This allows users to export their models for deployment across various platforms, frameworks, and devices. By enabling this conversion, Export to ONNX enhances model portability, cross-framework compatibility, and efficiency in production environments.
• Converts Hugging Face models to ONNX format for broader compatibility
• Supports multiple deep learning frameworks for inference
• Optimizes models for faster inference and reduced latency
• Enables deployment on edge devices and cloud platforms
• Simplifies model sharing and collaboration across teams
• Integrates seamlessly with the Hugging Face ecosystem
pip install onnx to ensure you have the ONNX library installed.transformers or torch library to load your model.
from transformers import AutoModelForSequenceClassification, AutoTokenizer
model_name = "bert-base-uncased"
model = AutoModelForSequenceClassification.from_pretrained(model_name)
tokenizer = AutoTokenizer.from_pretrained(model_name)
torch.onnx.export function to convert the model.
import torch
dummy_input = "This is a sample input"
inputs = tokenizer(dummy_input, return_tensors="pt")
with torch.no_grad():
torch.onnx.export(
model,
inputs["input_ids"],
"model.onnx",
opset_version=12,
input_names=["input_ids"],
output_names=["output"],
)
What models are supported for export to ONNX?
Export to ONNX supports a wide range of Hugging Face models, including BERT, RoBERTa, and other popular architectures. However, some niche or custom models may require additional configuration.
How do I optimize my ONNX model for inference?
ONNX models can be optimized using tools like ONNX Runtime or TensorRT. These tools can reduce latency and improve performance on target hardware.
What if the model fails to convert to ONNX?
If conversion fails, check for unsupported operations in your model. Some custom layers or operations may not be compatible with ONNX. You may need to modify your model architecture or update your version of ONNX.