FLEX is a novel benchmark designed to test how robustly Large Language Models maintain fairness when faced with adversarial prompts that attempt to elicit biased responses.

• Evaluates a critical gap in current safety testing by focusing on how simple adversarial instructions can bypass fairness guardrails
• Highlights intrinsic weaknesses where LLMs may generate biased content despite safety mechanisms
• Provides a more comprehensive assessment of model robustness against bias-inducing tactics
• Supports development of more resistant LLMs that maintain fairness under various prompt conditions

This research is crucial for security professionals working to identify and mitigate potential harm from deployed AI systems by revealing subtle vulnerabilities that standard testing might miss.

FLEX: A Benchmark for Evaluating Robustness of Fairness in Large Language Models