Evaluate adversarial robustness using generative models
Evaluate and submit AI model results for Frugal AI Challenge
Browse and submit LLM evaluations
Display genomic embedding leaderboard
Convert Hugging Face model repo to Safetensors
Compare audio representation models using benchmark results
Convert Stable Diffusion checkpoint to Diffusers and open a PR
Convert Hugging Face models to OpenVINO format
Retrain models for new data at edge devices
Teach, test, evaluate language models with MTEB Arena
Browse and submit model evaluations in LLM benchmarks
Measure execution times of BERT models using WebGPU and WASM
Generate leaderboard comparing DNA models
GREAT Score is a tool designed to evaluate adversarial robustness using generative models. It provides a framework to assess how well machine learning models, particularly those in the realm of generative AI, can withstand adversarial attacks. By leveraging generative models, GREAT Score offers insights into the vulnerabilities of AI systems, helping developers and researchers improve model resilience and security.
• Adversarial Attack Generation: Utilizes advanced techniques to craft sophisticated adversarial examples to test model robustness.
• Multiple Threat Models: Supports various threat models, including L2, L∞, and other common adversarial attack constraints.
• Model Agnostic: Works with diverse types of machine learning models, including neural networks and traditional models.
• Flexible Evaluation Metrics: Provides detailed metrics to quantify model performance under adversarial conditions.
• Integration with Popular Libraries: Compatible with leading machine learning frameworks such as TensorFlow and PyTorch.
• Customizable Parameters: Allows users to tweak attack strength, step sizes, and other parameters for tailored evaluations.
• Comprehensive Reporting: Delivers detailed reports on model vulnerabilities and suggested improvements.
pip install great-score to install the package.import great_score in your Python script or notebook.What does GREAT Score stand for?
GREAT Score is an acronym that reflects its purpose of evaluating adversarial robustness using generative models.
Can GREAT Score be used with any type of machine learning model?
Yes, GREAT Score is designed to be model-agnostic and can work with various types of machine learning models, including neural networks and traditional models.
How do I interpret the results from GREAT Score?
The results provide metrics on model performance under adversarial conditions, highlighting vulnerabilities and suggesting potential improvements to enhance robustness.