A cli rascal test runner for use from the maven plugin.

[jurgenvinju]

• This avoids having to write a Java class file with the JUnitTestRunner annotation, even in project that otherwise have no Java code. It also works for projects that do have Java dependencies.
• This closes the abstraction for Rascal as a language (you don't have to understand anymore that it's an interpreted and/or compiled language or how the test runners have been organized (via JUnit or otherwise).
• This makes sure that the generic surefire-report XML format is respected for integration with github actions and VScode reporting tools.

To use this add this to your pom.xml and make sure the <srcs> configuration for the rascal-maven-plugin matches what you want to test. You can also add a new <srcs> tag here with a different <configuration> under <execution>:

<execution>
     <id>yourproject-test</id>
     <phase>test</phase>
     <goals>
           <goal>test</goal>
     </goals>
</execution>

• CLI command to call the test runner with the same parameters as calling the checker or the compiler or the exec runner
• Optionally log in JUnit testsuite report format for picking up by CLI and CI tools
• Add test runner statistics for each testsuite/module XML file
• Documents the Surefire/JUnit test report XML format as a Rascal abstract grammar.
• Parallel testing, as parellel to the parallel checker

This does not (yet) work for compiled Rascal code, but the exact same interface could be implemented for generated test code (whether or not we use JUnit for that or not).

[codecov]

Codecov Report

❌ Patch coverage is 0% with 223 lines in your changes missing coverage. Please review.
✅ Project coverage is 45%. Comparing base (d9fc365) to head (072f503).

Files with missing lines	Patch %	Lines
...pl/test/infrastructure/JUnitXMLReportListener.java	0%	84 Missing ⚠️
src/org/rascalmpl/shell/RascalTest.java	0%	71 Missing ⚠️
...c/org/rascalmpl/shell/AbstractCommandlineTool.java	0%	61 Missing ⚠️
...g/rascalmpl/interpreter/env/GlobalEnvironment.java	0%	3 Missing ⚠️
src/org/rascalmpl/shell/CommandlineParser.java	0%	2 Missing ⚠️
src/org/rascalmpl/shell/RascalCompile.java	0%	2 Missing ⚠️

Additional details and impacted files

@@           Coverage Diff            @@
##              main   #2755    +/-   ##
========================================
- Coverage       46%     45%    -1%     
+ Complexity    6727    6715    -12     
========================================
  Files          842     844     +2     
  Lines        66809   66995   +186     
  Branches      9987   10011    +24     
========================================
- Hits         30791   30774    -17     
- Misses       33631   33838   +207     
+ Partials      2387    2383     -4     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

[DavyLandman]

I like this feature 👍 I'll try to allocate time for a review next week.

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

[jurgenvinju]

RascalTest does not have a test itself, parallel to RascalShell and RascalCheck and RascalCompile. I think we should have a different PR to resolve this omision for all commandline tools in one go.

[jurgenvinju]

@DavyLandman testing this on Windows is redundant since it uses the same setup as RascalCompile for running the checker.

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud