Skip to content

test: add windows and c++ coverage #35670

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
bcoe wants to merge 12 commits into from
Closed

test: add windows and c++ coverage #35670

Show file tree
Hide file tree
Changes from 8 commits
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
df981ea
test: add windows and C++ coverage
Oct 17, 2020
d91b81d
build: improve coverage configuration
Oct 18, 2020
656160f
chore: attempt to build with gcov
Oct 18, 2020
9eac50b
build: ensure gcovr tool exists
Oct 18, 2020
f869255
chore: tweak codecov.yml file
Oct 18, 2020
f5d6272
multi-line command had issues
Oct 18, 2020
40cb785
chore: delete more output files
Oct 18, 2020
fc7c05c
build: run test-cov rather than test-ci
Oct 18, 2020
5939514
chore: address review
Oct 20, 2020
71f998a
chore: fiddle with gcovr settings
Oct 20, 2020
15d170e
chore: simplify command
Oct 20, 2020
c646f6d
build: a couple more small tweaks to reports
Oct 20, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 13 additions & 6 deletions .github/workflows/coverage-linux.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@
# TODO(bcoe): add similar job for Windows coverage.
name: coverage-linux

on:
Expand All @@ -12,7 +11,6 @@ env:
FLAKY_TESTS: dontcare

jobs:
# TODO(bcoe): add support for C++ coverage.
coverage-linux:
runs-on: ubuntu-latest
steps:
Comment thread
bcoe marked this conversation as resolved.
Expand All @@ -23,18 +21,27 @@ jobs:
python-version: ${{ env.PYTHON_VERSION }}
- name: Environment Information
run: npx envinfo
- name: Clone gcovr reporter
run: git clone -b 3.4 --depth=1 --single-branch https://github.com/gcovr/gcovr.git
- name: Clone patch for gcovr
Copy link
Copy Markdown
Contributor Author

@bcoe bcoe Oct 18, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mhdawson do you remember why we're running gcovr from a branch and floating a patch? It would be nice to simplify this.

Copy link
Copy Markdown
Member

@mhdawson mhdawson Oct 19, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It been long enough that I don't remember the specifics. I assume it failed to run properly with out that patch. It's possible a later version of gcovr may have fixed it.

Copy link
Copy Markdown
Contributor Author

@bcoe bcoe Oct 20, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mhdawson I've managed to get gcovr@4.2 working, installed directly from pip.

The trick was to continue running from the out/ folder, but to specify a --root pointing to the root project directory.

run: git clone --depth=1 --single-branch https://github.com/nodejs/build.git
- name: Patch gcovr
run: cd gcovr && patch -N -p1 < ../build/jenkins/scripts/coverage/gcovr-patches-3.4.diff
- name: Build
run: make build-ci -j2 V=1 CONFIG_FLAGS="--error-on-warn"
run: make build-ci -j2 V=1 CONFIG_FLAGS="--error-on-warn --coverage"
# TODO(bcoe): fix the couple tests that fail with the inspector enabled.
# The cause is most likely coverage's use of the inspector.
- name: Test
run: NODE_V8_COVERAGE=coverage/tmp make run-ci -j2 V=1 TEST_CI_ARGS="-p dots" || exit 0
- name: Report
run: NODE_V8_COVERAGE=coverage/tmp make test-cov -j2 V=1 TEST_CI_ARGS="-p dots" || exit 0
- name: Report js
run: npx c8 report --check-coverage
- name: Report C++
run: cd out && ../gcovr/scripts/gcovr --gcov-exclude='.*\b(deps|usr|out|cctest|embedding)\b' -v -r Release/obj.target --xml -o ../coverage/coverage.xml --gcov-executable=gcov
- name: Output file count
run: ls -l coverage/tmp/ | wc -l
# Clean temporary output from gcov and c8, so that it's not uploaded:
- name: Clean tmp
run: rm -rf coverage/tmp
run: rm -rf coverage/tmp && rm -rf out
- name: Upload
uses: codecov/codecov-action@v1
with:
Expand Down
41 changes: 41 additions & 0 deletions .github/workflows/coverage-windows.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
name: coverage-windows

on:
pull_request:
push:
branches:
- master

env:
PYTHON_VERSION: 3.9
FLAKY_TESTS: dontcare

jobs:
coverage-windows:
runs-on: windows-latest
steps:
- uses: actions/checkout@v2
- name: Set up Python ${{ env.PYTHON_VERSION }}
uses: actions/setup-python@v2
with:
python-version: ${{ env.PYTHON_VERSION }}
- name: Install deps
run: choco install nasm
- name: Environment Information
run: npx envinfo
- name: Build
run: ./vcbuild.bat experimental-quic
Comment thread
bcoe marked this conversation as resolved.
Outdated
# TODO(bcoe): investigate tests that fail with coverage enabled
# on Windows.
- name: Test
run: ./vcbuild.bat test-ci-js; node -e 'process.exit(0)'
env:
NODE_V8_COVERAGE: ./coverage/tmp
- name: Report
run: npx c8 report
- name: Clean tmp
run: npx rimraf ./coverage/tmp
- name: Upload
uses: codecov/codecov-action@v1
with:
directory: ./coverage
10 changes: 10 additions & 0 deletions codecov.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
comment:
# Only show diff and files changed:
layout: "diff, files"
Comment thread
bcoe marked this conversation as resolved.
# Don't post if no changes in coverage:
require_changes: true

codecov:
notify:
# Wait for all coverage builds:
after_n_builds: 2