Measure over-refusal in LLMs using OR-Bench
Convert PaddleOCR models to ONNX format
Evaluate code generation with diverse feedback types
Evaluate AI-generated results for accuracy
Optimize and train foundation models using IBM's FMS
Compare audio representation models using benchmark results
Find recent high-liked Hugging Face models
Calculate VRAM requirements for LLM models
Evaluate reward models for math reasoning
Convert Stable Diffusion checkpoint to Diffusers and open a PR
Browse and submit language model benchmarks
Display leaderboard of language model evaluations
View and compare language model evaluations
OR-Bench Leaderboard is a tool designed to measure and compare over-refusal (OR) behavior in large language models (LLMs). It provides a standardized framework to evaluate how models respond to refusal scenarios, ensuring consistent and fair benchmarking across different models. The leaderboard helps researchers and developers understand the limitations and capabilities of LLMs in handling refusal tasks.
What is over-refusal in LLMs?
Over-refusal refers to when a model refuses to respond to a query, even when it could provide a meaningful answer.
Why is benchmarking over-refusal important?
Benchmarking helps identify models that may excessively refuse to answer, potentially limiting their utility in real-world applications.
How do I interpret the results from OR-Bench Leaderboard?
Results show how often and in what contexts models refuse to respond, enabling comparisons of refusal behavior across different models.