CEB introduces a comprehensive benchmark to evaluate various fairness dimensions in Large Language Models simultaneously and consistently.

• Provides a compositional evaluation framework that measures biases across multiple dimensions and social groups
• Enables standardized comparisons between different models using consistent metrics
• Reveals that current LLMs still exhibit concerning biases that warrant attention
• Offers a practical tool for developers to identify and mitigate potential fairness issues

This research is critical for security professionals who need to ensure AI systems behave fairly and safely across diverse user populations, reducing potential harm and discrimination risks in deployed applications.

CEB: Compositional Evaluation Benchmark for Fairness in Large Language Models