Methodology

Methodology

• Red-Blue duel. Each match pits two participants over alternating turns: on each turn one participant attacks as Red (ships a feature with an embedded defect) and the other defends as Blue (repairs it without regressing the feature). Outcomes are deterministically derived from test-suite execution. The participants and the test-suite executions run inside containers : 0.0 collapse, partial credit, or 1.0 clean defense for Blue; Red scores when the defect ships.
• Tournament rating. Elo ratings (K = 32, every participant starting at 1500) are calculated over the 50 round-robin match outcomes (win = 1, draw = ½, loss = 0). 95% confidence intervals are derived from 1,000 match-level bootstrap resamples. Note that overlapping confidence intervals are common in head-to-head arenas.
• Late entrants. New participants can be added to existing tournaments. Its matchups with the existing participants are derived via active sampling algorithm, so new participants can be efficiently added without having to matchup with all existing participants.