Benchmarking

Benchmark inaccuracy

Benchmark limitations

Good benchmarking practices

Benchmarking

Benchmarking is an evaluation method where different models are compared against a standardized dataset or a predetermined task. It allows comparison both across different models and over time, providing a reference point for model assessment. Benchmarks are usually open, where their question-answer pairs are publicly available [235].

Test robustness of GPAI system on relevant benchmarks

Various benchmarks [52, 236] have been developed to assess the robustness of GPAI systems when deployed in environments or scenarios that differ from their training conditions. These benchmarks typically evaluate the model’s ability to handle variations in inputs, unexpected data distributions, or adversarial examples, aiming to ensure reliable performance outside the original training domain.

Frequent benchmarking to identify when red teaming is needed

Benchmarks, once created, are inexpensive to apply but may be less informative than red teaming. One reason is that sensitive data (e.g., relating to CBRNrelated capabilities) cannot be included in the public questions and answers of benchmarks. On the other hand, red teaming can be more accurate given participants with diverse attack strategies, but it requires more resources to execute than benchmarking. If there is a correlation between benchmarking and redteaming scores, then employing frequent benchmarking during the development of the model can identify arising vulnerabilities and inform the developers when more thorough red teaming is required [21]. Benchmarks can act as early warning signs of a larger issue, and red teaming can then be employed to investigate the severity and extent of such an issue.

Contamination detection

Contamination detection refers to techniques for assessing whether and to what extent a given model has benchmark data in its training dataset [161, 170]. This can involve a set of technical and regulatory interventions to prevent or identify a model trained on contaminated data.

Preventing or mitigating data contamination and leakage

Developers of GPAIS and creators of benchmarks can take actions to prevent AI models from being trained on contaminated or leaked data, or mitigate such data contamination and leakage.

Tracking benchmark leakage

Constant monitoring and documentation of benchmark leakage can help with the early detection of benchmark leaks [224, 95]

Reporting data decontamination efforts

Decontamination analysis may involve comparing the training dataset with the benchmark data, and publishing a report with statistics such as data overlap [235]. Especially for already trained models, including contamination statistics can allow experts to reweigh the success of the model on affected benchmarks.

Avoiding benchmark data with publicly available solutions and releasing contextual information for internet-derived data

Evaluators can improve the integrity of benchmarks by avoiding data with publicly available solutions. When using internet-derived data, supplementing it with contextual information (such as entity linking) may help to better document and assess the integrity of the collected data [95]. This could help in detecting instances where contextual information (which may have been included in the training data) might inadvertently reveal details about the solution, ensuring that the data is suitable for accurate benchmarking.