Export Hugging Face models to ONNX
Rank machines based on LLaMA 7B v2 benchmark results
Search for model performance across languages and benchmarks
Merge Lora adapters with a base model
Track, rank and evaluate open LLMs and chatbots
Create and manage ML pipelines with ZenML Dashboard
GIFT-Eval: A Benchmark for General Time Series Forecasting
Determine GPU requirements for large language models
Track, rank and evaluate open LLMs and chatbots
Display LLM benchmark leaderboard and info
Measure execution times of BERT models using WebGPU and WASM
Teach, test, evaluate language models with MTEB Arena
Convert PyTorch models to waifu2x-ios format
Export to ONNX is a tool designed to convert Hugging Face models into the ONNX (Open Neural Network Exchange) format. This allows users to export their models for deployment across various platforms, frameworks, and devices. By enabling this conversion, Export to ONNX enhances model portability, cross-framework compatibility, and efficiency in production environments.
• Converts Hugging Face models to ONNX format for broader compatibility
• Supports multiple deep learning frameworks for inference
• Optimizes models for faster inference and reduced latency
• Enables deployment on edge devices and cloud platforms
• Simplifies model sharing and collaboration across teams
• Integrates seamlessly with the Hugging Face ecosystem
pip install onnx to ensure you have the ONNX library installed.transformers or torch library to load your model.
from transformers import AutoModelForSequenceClassification, AutoTokenizer
model_name = "bert-base-uncased"
model = AutoModelForSequenceClassification.from_pretrained(model_name)
tokenizer = AutoTokenizer.from_pretrained(model_name)
torch.onnx.export function to convert the model.
import torch
dummy_input = "This is a sample input"
inputs = tokenizer(dummy_input, return_tensors="pt")
with torch.no_grad():
torch.onnx.export(
model,
inputs["input_ids"],
"model.onnx",
opset_version=12,
input_names=["input_ids"],
output_names=["output"],
)
What models are supported for export to ONNX?
Export to ONNX supports a wide range of Hugging Face models, including BERT, RoBERTa, and other popular architectures. However, some niche or custom models may require additional configuration.
How do I optimize my ONNX model for inference?
ONNX models can be optimized using tools like ONNX Runtime or TensorRT. These tools can reduce latency and improve performance on target hardware.
What if the model fails to convert to ONNX?
If conversion fails, check for unsupported operations in your model. Some custom layers or operations may not be compatible with ONNX. You may need to modify your model architecture or update your version of ONNX.