Skip to content

Enable JUnit report output during testRunner processing.#1901

Merged
d3r3kk merged 10 commits into
microsoft:masterfrom
d3r3kk:vsts_test_report
Jun 8, 2018
Merged

Enable JUnit report output during testRunner processing.#1901
d3r3kk merged 10 commits into
microsoft:masterfrom
d3r3kk:vsts_test_report

Conversation

@d3r3kk
Copy link
Copy Markdown

@d3r3kk d3r3kk commented Jun 7, 2018

Fixes #1897

If the environment indicates the tests are being run on CI servers, the following environment variables are now inspected during test runs:

  • MOCHA_REPORTER_JUNIT: any
    • if set in the environment (to any value) this enables the JUnit report file to be used, unlocks the rest of the MOCHA_CI... variables.
  • MOCHA_CI_REPORTFILE: string
    • path to a JUnit style output file.
  • MOCHA_CI_PROPERTIES: string
    • properties (in the format {KEY:VALUE;[KEY2:VALUE2;]}) to add to the JUnit report file

This pull request:

  • Has a title summarizes what is changing
  • Includes a news entry file (remember to thank yourself!)
  • Has unit tests & code coverage is not adversely affected (within reason)
  • Works on all actively maintained versions of Python (e.g. Python 2.7 & the latest Python 3 release)
  • Works on Windows 10, macOS, and Linux (e.g. considered file system case-sensitivity)

d3r3kk added 6 commits June 7, 2018 13:06
- Add reporter & reporterOptions to testRunner
- Add support for xunit support
- Add support for xunit output file
- change  format of gulp.task to coincide with v4
- change order of defined tasks to work correctly
- add `mocha-junit-reporter` to dev deps
- add `properties` reporterOption
- add `mochaFile` reporterOption
- set junit reporter options via env vars
- move MOCHA_ settings into constants
- export the values as const
- follow pattern of other env vars
@d3r3kk d3r3kk self-assigned this Jun 7, 2018
@d3r3kk d3r3kk requested a review from DonJayamanne June 7, 2018 22:04
@d3r3kk
Copy link
Copy Markdown
Author

d3r3kk commented Jun 7, 2018

Bit off too much, reverting the gulp upgrade...

Comment thread src/test/index.ts

// VSTS CI doesn't display colours correctly (yet).
if (IS_VSTS) {
options.useColors = false;
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please check useColors to true in line 26

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All done - good for you?

Comment thread src/test/index.ts Outdated
const options: testRunner.SetupOptions & { retries: number } = {
ui: 'tdd',
useColors: !IS_VSTS,
useColors: false,
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The default value must be true if you're changing this to false further down.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

agreed! Will make that change.

@codecov
Copy link
Copy Markdown

codecov Bot commented Jun 7, 2018

Codecov Report

Merging #1901 into master will increase coverage by 0.02%.
The diff coverage is n/a.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #1901      +/-   ##
==========================================
+ Coverage   74.42%   74.44%   +0.02%     
==========================================
  Files         282      282              
  Lines       13260    13260              
  Branches     2385     2385              
==========================================
+ Hits         9869     9872       +3     
+ Misses       3260     3258       -2     
+ Partials      131      130       -1
Impacted Files Coverage Δ
src/client/debugger/Main.ts 52.34% <0%> (+0.49%) ⬆️
src/client/linters/lintingEngine.ts 91.15% <0%> (+0.88%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 4ddebe2...f97c99f. Read the comment docs.

@d3r3kk d3r3kk closed this Jun 7, 2018
@d3r3kk
Copy link
Copy Markdown
Author

d3r3kk commented Jun 7, 2018

CI was interrupted for some reason...?

@d3r3kk d3r3kk reopened this Jun 7, 2018
@d3r3kk
Copy link
Copy Markdown
Author

d3r3kk commented Jun 7, 2018

Next todo: gather the logfiles and upload them into VSTS to see how we are doing. I don't love the test result logging at all, mocha doesn't seem to have a built-in 'tee' operation to get both the spec output to the console + JUnit report to a file. I'll explore some further options to see if I can get it to work in a more transparent way.

@d3r3kk d3r3kk closed this Jun 8, 2018
@d3r3kk
Copy link
Copy Markdown
Author

d3r3kk commented Jun 8, 2018

Rebuild-go

@d3r3kk d3r3kk reopened this Jun 8, 2018
@d3r3kk d3r3kk merged commit fd46c26 into microsoft:master Jun 8, 2018
@d3r3kk d3r3kk deleted the vsts_test_report branch June 8, 2018 02:56
@lock lock Bot locked as resolved and limited conversation to collaborators Jul 31, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add JUnit or XUnit output reporter to test runner to upload and use in VSTS

2 participants