JUnit support doesn't count amount of tests correctly #431

let me clarify based on the JUnit XML specification.

The tests and failures attributes on <testsuite> are defined at the <testcase> level, not at the individual lint-finding level:

• tests is the total number of tests (i.e., <testcase> elements) in the suite, and failures is the total number of failed tests in the suite.

• The failures attribute specifically means "the total number of tests in the suite that failed," where a failure is a test which the code has explicitly failed by using the mechanisms for that purpose.

In the current junit.def, cpplint's JUnit output collapses all lint findings for a given file into a single <testcase> with a single <failure> element whose text body contains all 123 findings concatenated. So the actual structure is:

• 2 <testcase> nodes (one per file: src/sillycode.cpp, src/sillycode.w)
• 2 <failure> nodes (one per testcase)

This means the correct suite-level counts should be tests="2" failures="2", not tests="123" failures="123". The value 123 represents the count of individual lint findings, which is not what these XML attributes are meant to track — JUnit consumers (e.g., Jenkins, GitHub Actions test reporters) will interpret tests="123" as meaning there are 123 separate test cases, leading to incorrect pass/fail metrics.

If you'd prefer to represent 123 individual findings, each one would need its own <testcase> node.

Originally posted by @coderabbitai[bot] in #429 (comment)

I'm currently hoping that just turning each error into a failure so the one test case has 123 failure elements under it would conform. Not sure if it does though.