Evaluate LLM over-refusal rates with OR-Bench
Retrain models for new data at edge devices
Display and submit language model evaluations
Display LLM benchmark leaderboard and info
Track, rank and evaluate open LLMs and chatbots
Evaluate open LLMs in the languages of LATAM and Spain.
Analyze model errors with interactive pages
Merge machine learning models using a YAML configuration file
Display model benchmark results
Launch web-based model application
Browse and evaluate language models
Find and download models from Hugging Face
Measure execution times of BERT models using WebGPU and WASM
OR-Bench Leaderboard is a benchmarking platform designed to evaluate Large Language Models (LLMs) based on their over-refusal rates. It provides a comprehensive framework to assess how often models refuse to respond to prompts, offering insights into their reliability and responsiveness. This tool is particularly useful for researchers and developers aiming to optimize LLM performance and transparency.
• Benchmarking of LLMs: Comprehensive evaluation of models based on their refusal rates. • Performance Metrics: Detailed metrics on refusal rates across diverse scenarios and prompts. • Model Comparisons: Side-by-side comparisons to identify top-performing models. • Scenarios Support: Testing models against a wide range of scenarios. • Transparency: Open and accessible results for community review. • Community-Driven: Continuously updated with new models and data.
What does the OR-Bench Leaderboard measure?
The leaderboard measures the over-refusal rates of LLMs, indicating how often models refuse to respond to prompts.
How are the models evaluated?
Models are evaluated using a standardized set of scenarios designed to test their responsiveness and reliability.
Can I contribute to the leaderboard?
Yes, contributions are welcome. Submit your model or scenario suggestions through the platform's community portal.