Submit new entrants

Follow steps 01–05 of the Setup guide: install SWE-Duel, set SWE_DUEL_OPENROUTER_API_KEY, clone the target repositories, build the images, run swe-duel doctor, and add model configurations (if you want to evaluate a new model available on OpenRouter).

The tournament's exported ranking is stored at <swe-duel-project-root>/rankings/rankings_<tournament ID>.json by default. It contains Elo scores, Bradley–Terry log-rank ratings, head-to-head records, the raw match list used for sampling, and other metrics. The leaderboard's tournament ID is 72510128-7105-49c5-8028-2bef7d404c7e, and thus is stored at <swe-duel-project-root>/rankings/rankings_72510128-7105-49c5-8028-2bef7d404c7e.json. git pull before you start, so you sample against the latest field (including other late entrants submitted by other contributors).

Add the entrant's model to config/models.yaml with its selectable efforts, providers, and rate limits (see Setup · Choose the participants). The incumbents need no entries: models in the export that are not in your models.yaml get an auto-constructed config pinned to their recorded effort/provider.

models:
  my-new-model:
    model_id: "vendor/my-new-model"
    temperature: 0.2
    reasoning_efforts: ["high", "medium", "low"]
    providers: ["vendor"]
    provider_rate_limits:
      "vendor_cloud": 40
      "third_party_cloud": 20
      "default": 10

Then verify that it is reachable and that its cost/token telemetry comes back:

swe-duel doctor --live
swe-duel-probe-rate-limits --models my-new-model      # probe for provider_rate_limits

Your entrant's identity is the full 4-tuple; the harness rules apply (codex only takes minimal/low/medium/high/xhigh efforts and cannot pin a provider; claude-code takes neither).

# interactive
# paged selection of entrant: model → harness → effort → provider wizard
swe-duel-tournament-as --rankings path/to/rankings_<tournament ID>.json

The command will print:

• active-sampled matches: each entrant against its SE-reduction-selected incumbents;
• defense-turn plan and the task-slot plan per (participant, repository): slots that already have any record in your <swe-duel-project-root>/data/ (admitted, or an exhausted failure chain) are reused, and only never-attempted slots will be generated.

Two more steps after active-sampling of matches for the new entrant:

1. Task generation runs never-attempted slots (Red feature → feature gates → bug → full gates including self-review), then prints a task bank recap. A slot that never gets an admitted task is not an error: the defender auto-wins that turn.
2. Running the active-sampled matches plays the matches as one pooled batch of defenses. Defenses are cached in <swe-duel-project-root>/data/defenses/, and match records land in <swe-duel-project-root>/data/matches/.

Resuming active-sampled task generation and match runs.

Throughout the steps 04 and 05, the active-sampled matchups are persisted into a state file in <swe-duel-project-root>/data/tournaments/active_sampling_state_<unique ID>.json. Thus, in case of terminal hangs, no re-active-sampling and re-selection of model is needed. You can resume the active-sampled task generation and match running session with:

swe-duel-tournament-as --resume-state <state_id>
# <state_id>: the uuid, the file stem, or the path of <swe-duel-project-root>/data/tournaments/active_sampling_state_<unique ID>.json

Finally, after task generation and match runs, you will be prompted to continue with packaging the tasks, defenses, and matches inside of <swe-duel-project-root>/data/ that are relevant to the active-sampled matches, plus your whole <swe-duel-project-root>/config/ directory (the deployment configuration that shaped the packaged records), into <swe-duel-project-root>/submissions/.

Excluded: workspaces/, logs/, and the rest of your bank that is not relevant to the active-sampled matches.

# reach the submission packaging prompt in the console UI again:
swe-duel-tournament-as --resume-state <state_id>
sha256sum submissions/*.zip       # matches the printed SHA-256

Optional: To refresh the leaderboard on your machine and visualize it locally, you may run:

swe-duel-tournament-update path/to/submission.zip --tournament 72510128-7105-49c5-8028-2bef7d404c7e

Send the zip to the arena organizers via:

• Discord: Invite link to SWE-Duel Discord community.
• Email: wei.minn.2023@phdcs.smu.edu.sg; or mg.weiminn@gmail.com.

The organizers will merge the submitted records into our tournament data, and refresh the leaderboard to include your new entrant in it.