Theoretical foundations and provable safety in AI systems

Training and fine-tuning methods for alignment and safety

Developing reliable training and fine-tuning strategies for AI models to ensure that their outputs remain safe, interpretable, and aligned with intended goals. This involves understanding how fine-tuning affects model behavior, employing adversarial training for robust alignment, carefully adjusting pre-training processes, and improving data quality and auditing methods.

Training and fine-tuning methods for alignment and safety > Understanding how fine-tuning changes a pretrained model

Investigating how fine-tuning alters a model’s internal representations and behaviors to better predict, and ultimately control, downstream safety outcomes.

Training and fine-tuning methods for alignment and safety > Develop output-based adversarial training techniques for more robust alignment

Developing training procedures, such as adversarial training focused on internal model representations, or ‘process supervision’, that directly optimize against adversarial examples and undesirable outputs, making models more resistant to manipulations that could lead to unsafe behaviors.

Training and fine-tuning methods for alignment and safety > Scalable techniques for targeted modifications of LLM behavior (including unlearning)

Creating scalable methods for precisely adjusting model outputs, such as removing unwanted content or refining responses to adhere to alignment constraints without broadly degrading performance. This may also include removal of unknown or latent undesirable capabilities that emerge in large models.

Training and fine-tuning methods for alignment and safety > Retrieval-augmented pre-training

Incorporating retrieval mechanisms during pre-training to better ground models in verified information.

Training and fine-tuning methods for alignment and safety > Pretraining alterations to improve interpretability

Altering pre-training protocols to produce models with clearer internal representations and decision-making pathways, allowing for more effective downstream analysis and intervention.