Skip to content

Refactor check_submissions - adding API hooks for judging and minor multithreading improvements - #490

Merged
pehrsoderman merged 3 commits into
Kattis:masterfrom
gkreitz:judge_hook_and_improved_multithreading
Sep 4, 2026
Merged

Refactor check_submissions - adding API hooks for judging and minor multithreading improvements#490
pehrsoderman merged 3 commits into
Kattis:masterfrom
gkreitz:judge_hook_and_improved_multithreading

Conversation

@gkreitz

@gkreitz gkreitz commented Sep 4, 2026

Copy link
Copy Markdown
Contributor

This PR is a large refactor on the check_submissions method. Main technical changes:

  • Added a class SubmissionsJudge which gets a call with a large batch of submissions to check. This is mainly to allow an API user to hook judging within problemtools, but also lets our multithreading schedule precompute a bit better.
  • A complete rewrite of the logic in check_submissions. I feel like it reads a lot cleaner now (gather up all submissions which lower bound the time limit, run those, compute time limit, run the rest)
  • Getting rid of directory hard coding (slowly making progress towards supporting submissions.yaml)

Behavioral changes:

  • If a submission is larger than the code size limit, we now still compile and test it (more in line with other non-fatal errors)
  • Minor change in the order we test submissions (alphabetical order, but forcing TLE last)
  • "AC submission foo.java" is now "accepted/foo.java". This is arguable a bit worse, but when we eventually support submissions.yaml, submissions can come in arbitrary directories.

Refactors check_submission with several goals in mind:
 - Easier to follow the logic
 - (some) preparation for submissions.yaml - a bit less directory hard coding
 - Use the submissions_judge_factory hook (for API users wanting to modify judging)
 - Start test cases in larger batches
@pehrsoderman
pehrsoderman merged commit 4be848f into Kattis:master Sep 4, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants