PT-ALIGN introduces a novel dual safety self-alignment approach that improves LLM safety while maintaining helpfulness through automatic refinement of training samples.

• Automatically refines positive and toxic samples for more effective safety training
• Uses a dual-objective approach balancing harmlessness and helpfulness
• Achieves comparable results to methods requiring extensive human annotation
• Demonstrates effectiveness across multiple safety benchmarks with minimal human intervention

This research addresses critical security concerns by providing a scalable method to improve LLM safety alignment without the extensive human labeling typically required, potentially reducing the deployment of unsafe AI systems while maintaining performance.

Refining Positive and Toxic Samples for Dual Safety Self-Alignment of LLMs with Minimal Human Interventions