[ci] feat: implement a bot to suggest running tests before merging#14099
[ci] feat: implement a bot to suggest running tests before merging#14099sayakpaul wants to merge 2 commits into
Conversation
|
@DN6 can you take a look at this PR? down the road, we can streamline the workflow so final review will recommend a couple of tests in the serge UI, and the maintainer can just check a checkbox to run the slow test etc (not in the scope of this PR) |
|
@sayakpaul Why not automatically run the recommended tests? Why require a manual trigger? |
|
Recommended tests could be larger than intended. It would also interfere with the test fetcher mechanism (not just for pipelines but more generally) that we have been meaning to implement for a long time now. Additionally, in some cases, we might not even want to run the recommended slow/GPU TESTS. So, I wanted to keep it the scope very specific here. |
|
Also, we have a deterministic way to fetch tests in this workflow It fetches relevant tests to run based on which modules have been changed. We never activated it on PRs because we had some weird circular dependencies with SD1.5 in the library that would trigger the wrong tests to run. I think those have been fixed now. I've been working updating the workflow here (it's almost ready to PR): |
|
Yeah that is what I was referring to. Do you want to extend the workflow that would suggest maintainers to run tests that would not run in CI otherwise (outside fast tests for example)? |
|
Discussed with @DN6 today and I am going to close this because we will have a robust test fetcher. |
Example comment:
Where the diff in a PR maps cleanly to core modules and their corresponding tests (
peft.py-> LoRA-related tests), we use a deterministic system for the suggestion. However, when that gets convoluted, we also use Qwen3.5-35B-A3 model through HF Inference Provider to rank and polish the suggestion.