Evaluate code generation with diverse feedback types
View and submit LLM benchmark evaluations
Convert Hugging Face model repo to Safetensors
View LLM Performance Leaderboard
Determine GPU requirements for large language models
Find and download models from Hugging Face
Browse and submit LLM evaluations
Measure over-refusal in LLMs using OR-Bench
Explain GPU usage for model training
Retrain models for new data at edge devices
Display and filter leaderboard models
Analyze model errors with interactive pages
Convert PyTorch models to waifu2x-ios format
ConvCodeWorld is a model benchmarking tool designed to evaluate and compare code generation models. It focuses on assessing models through diverse feedback types, making it a comprehensive platform for understanding and improving code generation capabilities.
• Multiple Feedback Types: Supports various feedback mechanisms, including user ratings, pairwise comparisons, and error detection tasks.
• Customizable Benchmarks: Allows users to define custom benchmarks tailored to specific use cases or programming languages.
• Detailed Metrics: Provides in-depth performance metrics, including correctness, efficiency, and user satisfaction scores.
• Model Agnostic: Compatible with a wide range of code generation models, ensuring versatility in evaluation.
• Version Tracking: Enables longitudinal analysis of model improvements over time.
• Collaborative Interface: Offers a shared workspace for teams to review and discuss model performance.
What makes ConvCodeWorld unique?
ConvCodeWorld stands out due to its diverse feedback mechanisms, which provide a holistic view of model performance beyond traditional metrics.
Which programming languages does ConvCodeWorld support?
ConvCodeWorld supports a wide range of programming languages, including Python, Java, C++, and JavaScript, with more languages being added regularly.
How long does it take to run a benchmark?
The time required to run a benchmark depends on the size of the test set and the complexity of the tasks. Small benchmarks can complete in minutes, while larger ones may take several hours.