Export Hugging Face models to ONNX
Browse and submit model evaluations in LLM benchmarks
Find and download models from Hugging Face
Explain GPU usage for model training
Retrain models for new data at edge devices
Demo of the new, massively multilingual leaderboard
Create demo spaces for models on Hugging Face
Explore and benchmark visual document retrieval models
Display leaderboard for earthquake intent classification models
Optimize and train foundation models using IBM's FMS
Compare LLM performance across benchmarks
View and submit machine learning model evaluations
View and submit LLM evaluations
Export to ONNX is a tool designed to convert Hugging Face models into the ONNX (Open Neural Network Exchange) format. This allows users to export their models for deployment across various platforms, frameworks, and devices. By enabling this conversion, Export to ONNX enhances model portability, cross-framework compatibility, and efficiency in production environments.
• Converts Hugging Face models to ONNX format for broader compatibility
• Supports multiple deep learning frameworks for inference
• Optimizes models for faster inference and reduced latency
• Enables deployment on edge devices and cloud platforms
• Simplifies model sharing and collaboration across teams
• Integrates seamlessly with the Hugging Face ecosystem
pip install onnx to ensure you have the ONNX library installed.transformers or torch library to load your model.
from transformers import AutoModelForSequenceClassification, AutoTokenizer
model_name = "bert-base-uncased"
model = AutoModelForSequenceClassification.from_pretrained(model_name)
tokenizer = AutoTokenizer.from_pretrained(model_name)
torch.onnx.export function to convert the model.
import torch
dummy_input = "This is a sample input"
inputs = tokenizer(dummy_input, return_tensors="pt")
with torch.no_grad():
torch.onnx.export(
model,
inputs["input_ids"],
"model.onnx",
opset_version=12,
input_names=["input_ids"],
output_names=["output"],
)
What models are supported for export to ONNX?
Export to ONNX supports a wide range of Hugging Face models, including BERT, RoBERTa, and other popular architectures. However, some niche or custom models may require additional configuration.
How do I optimize my ONNX model for inference?
ONNX models can be optimized using tools like ONNX Runtime or TensorRT. These tools can reduce latency and improve performance on target hardware.
What if the model fails to convert to ONNX?
If conversion fails, check for unsupported operations in your model. Some custom layers or operations may not be compatible with ONNX. You may need to modify your model architecture or update your version of ONNX.