Skip to content
This repository was archived by the owner on May 13, 2019. It is now read-only.

Commit 56fffa6

Browse files
maclover7mhdawson
maclover7
authored and
mhdawson
committed
Revert "coverage: remove folder (#57)"
This reverts commit 5d85de3. PR-URL: #58 Reviewed-By: Michael Dawson <michael_dawson@ca.ibm.com>
1 parent 5d85de3 commit 56fffa6

4 files changed

Lines changed: 384 additions & 0 deletions

File tree

‎coverage/README.md‎

Lines changed: 114 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,114 @@
1+
# Code Coverage Generation
2+
3+
We have nightly code coverage generation so that we can track test coverage
4+
for Node.js, make the information public and then use that information
5+
to improve coverage over time.
6+
7+
At this time we only capture coverage results once a day on linux x86. We
8+
believe that coverage will not vary greatly across platforms and that the
9+
process will be too expensive to run on every commit. We will re-evaluate
10+
these assumptions based on data once the process has been in place for
11+
a while.
12+
13+
This doc captures the infrastructure in place to support the generation
14+
of the coverage information published to https://coverage.nodejs.org.
15+
16+
# Steps
17+
18+
Generation/publication of the code coverage results consists of the following:
19+
20+
* Nightly scheduled job - We have a job in jenkins which is scheduled to run at
21+
11 EST each night.
22+
[node-test-commit-linux-coverage](https://ci.nodejs.org/view/All/job/node-test-commit-linux-coverage/).
23+
* At the end of the scheduled job it rsync's the generated data to the
24+
benchmarking data machine. We do this so that once the job is complete
25+
the data is in a place where we know we can pull it from, and that pulling
26+
that data will not affect any other jobs (for example jobs measuring
27+
performance on the benchmark machine).
28+
* At hourly intervals the the data is rsync'd from the benchmarking
29+
data machine to the website. This is triggered from the nodejs.org website
30+
machine and data is pulled from the benchmarking data machine. This allows
31+
us to minimize who can modify the nodejs.org website as no additional
32+
access is required.
33+
34+
# Coverage Job
35+
36+
The coverage job follows the same pattern as our other build jobs in order
37+
to check out the version of node to be build/tested. It requires the following
38+
additions:
39+
40+
1. Build/test with the coverage targets. This is currently:
41+
42+
```
43+
./configure --coverage
44+
make coverage-clean
45+
NODE_TEST_DIR=${HOME}/node-tmp PYTHON=python COVTESTS=test-ci make coverage -j $(getconf _NPROCESSORS_ONLN)
46+
```
47+
48+
2. Generate html summary page and push results to the benchmarking data machine:
49+
50+
```
51+
#!/bin/bash
52+
53+
# copy the coverage results to the directory where we keep them
54+
# generate the summaries and transfer to the benchmarking data
55+
# machine from which the website will pull them
56+
57+
export PATH="$(pwd):$PATH"
58+
59+
# copy over results
60+
COMMIT_ID=$(git rev-parse --short=16 HEAD)
61+
mkdir -p "$HOME/coverage-out"
62+
OUTDIR="$HOME/coverage-out/out"
63+
mkdir -p "$OUTDIR"
64+
rm -rf "$OUTDIR/coverage-$COMMIT_ID" || true
65+
cp -r coverage "$OUTDIR/coverage-$COMMIT_ID"
66+
67+
# add entry into the index and generate the html version
68+
JSCOVERAGE=$(grep -B1 Lines coverage/index.html | \
69+
head -n1 | grep -o '[0-9\.]*')
70+
CXXCOVERAGE=$(grep -A3 Lines coverage/cxxcoverage.html | \
71+
grep style | grep -o '[0-9]\{1,3\}\.[0-9]\{1,2\}')
72+
NOW=$(date -u +"%Y-%m-%dT%H:%M:%SZ")
73+
74+
echo "$JSCOVERAGE,$CXXCOVERAGE,$NOW,$COMMIT_ID" >> "$OUTDIR/index.csv"
75+
76+
cd $OUTDIR/..
77+
$WORKSPACE/testing/coverage/generate-index-html.py
78+
79+
# transfer results to machine where coverage data is staged.
80+
rsync -r out coveragedata:coverage-out
81+
```
82+
83+
The current setup depends on past runs being in /home/iojs/coverage-out/out
84+
on the machine that it is run on so that the generated index
85+
includes the current and past data. For this and other reasons described
86+
in the other sections, the job is pegged to run on:
87+
[iojs-softlayer-benchmark](https://ci.nodejs.org/computer/iojs-softlayer-benchmark/)
88+
89+
90+
# Tranfer to benchmarking data machine
91+
The rsync from the machine on which the job runs to the benchmarking
92+
data machine requires an ssh key. Currently we have pegged the job to the
93+
benchmarking machine
94+
[iojs-softlayer-benchmark](https://ci.nodejs.org/computer/iojs-softlayer-benchmark/),
95+
have installed the key there, and have added an entry in
96+
the ```.ssh/config``` file for the iojs user so that connections to the
97+
'coveragedata' go to the benchmarking machine and use the correct key
98+
(uses the softlayer internal network as opposed to public ip)
99+
100+
```
101+
Host coveragedata
102+
HostName 10.52.6.151
103+
User benchmark
104+
IdentityFile ~/coverage-out/key/id_rsa
105+
```
106+
107+
The results are pushed to /home/benchmark/coverage-out/out.
108+
109+
# Transfer to the website
110+
As mentioned earlier, the website will pull updates hourly from
111+
/home/benchmark/coverage-out/out and put
112+
them in the right place to be served at coverage.nodejs.org. The key
113+
required to do this is already in place in order to support the similar process
114+
for benchmarking.nodejs.org

‎coverage/gcovr-patches.diff‎

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,23 @@
1+
diff --git a/scripts/gcovr b/scripts/gcovr
2+
index 034779c86d29..e68b239c424f 100755
3+
--- a/scripts/gcovr
4+
+++ b/scripts/gcovr
5+
@@ -496,7 +496,7 @@ def process_gcov_data(data_fname, covdata, options):
6+
if filtered_fname is None:
7+
if options.verbose:
8+
sys.stdout.write(" Filtering coverage data for file %s\n" % fname)
9+
- return
10+
+ #return
11+
#
12+
# Return if the filename matches the exclude pattern
13+
#
14+
@@ -2141,6 +2141,9 @@ if options.objdir:
15+
for i in range(0, len(options.exclude)):
16+
options.exclude[i] = re.compile(options.exclude[i])
17+
18+
+if options.output is not None:
19+
+ options.output = os.path.abspath(options.output)
20+
+
21+
if options.root is not None:
22+
if not options.root:
23+
sys.stderr.write(

‎coverage/generate-index-html.py‎

Lines changed: 116 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,116 @@
1+
#!/usr/bin/env python
2+
# -*- coding: utf-8 -*-
3+
4+
import datetime
5+
6+
with open('out/index.csv') as index:
7+
index_csv = filter(lambda line: line, index.read().split('\n'))
8+
9+
with open('out/index.html', 'w') as out:
10+
out.write(
11+
'''
12+
<!DOCTYPE html>
13+
<html>
14+
<head>
15+
<meta charset="utf-8">
16+
<title>Node.js Code Coverage</title>
17+
18+
<link rel="dns-prefetch" href="http://fonts.googleapis.com">
19+
<link rel="dns-prefetch" href="http://fonts.gstatic.com">
20+
21+
<meta name="author" content="Node.js Foundation">
22+
<meta name="robots" content="index, follow">
23+
<meta name="viewport" content="width=device-width, initial-scale=1.0">
24+
25+
<link rel="apple-touch-icon" href="https://nodejs.org/static/apple-touch-icon.png">
26+
<link rel="icon" sizes="32x32" type="image/png" href="https://nodejs.org/static/favicon.png">
27+
28+
<link rel="stylesheet" href="https://nodejs.org/layouts/css/styles.css" media="all">
29+
<link rel="stylesheet" href="https://fonts.googleapis.com/css?family=Source+Sans+Pro:400,600">
30+
<style>
31+
#logo { margin-bottom: 1rem; }
32+
main { margin-bottom: 2rem; }
33+
.table-header,
34+
.table-row {
35+
box-sizing: border-box;
36+
display: flex;
37+
width: 100%;
38+
padding: 2px 10px;
39+
}
40+
.table-header { font-weight: bold;}
41+
.table-header > div,
42+
.table-row > div {
43+
flex-grow: 1;
44+
width: 100px;
45+
}
46+
.table-row:nth-child(even) { background-color: #eee; }
47+
.sha .cell-value { font-family: Consolas, "Liberation Mono", Menlo, Courier, monospace; }
48+
.cell-header { display: none; }
49+
@media screen and (min-width: 690px) and (max-width: 850px) {
50+
.table-header > div:nth-child(n+3),
51+
.table-row > div:nth-child(n+3) {
52+
flex-grow: 0.2;
53+
}
54+
.table-header > div:first-child,
55+
.table-row > div:first-child {
56+
flex-grow: 0.4;
57+
}
58+
}
59+
@media screen and (max-width: 690px) {
60+
.cell-header { display: block; font-weight: bold; }
61+
.table-header { display: none; }
62+
.table-row { display: block; }
63+
.table-row > div { width: 100%; text-align: center; margin-bottom: 12px;}
64+
}
65+
</style>
66+
</head>
67+
<body>
68+
<header>
69+
<div class="container" id="logo">
70+
<img src="https://nodejs.org/static/images/logos/nodejs-new-white-pantone.png" alt="node.js">
71+
</div>
72+
</header>
73+
<div id="main">
74+
<div class="container">
75+
<h1>Node.js Nightly Code Coverage</h1>
76+
<h3>
77+
Node.js Core&nbsp;&nbsp;<a href="https://github.com/nodejs/node">&rarr;</a>
78+
</h3>
79+
<main>
80+
<div class="page-content">
81+
<div class="table">
82+
<div class="table-header">
83+
<div>Date (UTC)</div>
84+
<div>HEAD</div>
85+
<div>JS Coverage</div>
86+
<div>C++ Coverage</div>
87+
</div>
88+
''')
89+
for line in reversed(index_csv):
90+
jscov, cxxcov, date, sha = line.split(',')
91+
date = datetime.datetime.strptime(date, '%Y-%m-%dT%H:%M:%S%fZ').strftime("%d/%m/%Y %H:%M")
92+
out.write('''
93+
<div class="table-row">
94+
<div><div class="cell-header">Date (UTC)</div><div class="cell-value">{0}</div></div>
95+
<div class="sha"><div class="cell-header">HEAD</div><div class="cell-value"><a href="https://github.com/nodejs/node/commit/{1}">{1}</a></div></div>
96+
<div><div class="cell-header">JS Coverage</div><div class="cell-value"><a href="coverage-{1}/index.html">{2:05.2f}&nbsp;%</a></div></div>
97+
<div><div class="cell-header">C++ Coverage</div><div class="cell-value"><a href="coverage-{1}/cxxcoverage.html">{3:05.2f}&nbsp;%</a></div></div>
98+
</div>'''.format(date, sha, float(jscov), float(cxxcov)))
99+
out.write('''
100+
</div>
101+
</div>
102+
</div>
103+
</div>
104+
<footer class="no-margin-top">
105+
<div class="linuxfoundation-footer">
106+
<div class="container">
107+
<a class="linuxfoundation-logo" href="http://collabprojects.linuxfoundation.org">Linux Foundation Collaborative Projects</a>
108+
<p>&copy; 2016 Node.js Foundation. All Rights Reserved. Portions of this site originally &copy; 2016 Joyent. </p>
109+
<p>Node.js is a trademark of Joyent, Inc. and is used with its permission. Please review the <a href="https://nodejs.org/static/documents/trademark-policy.pdf">Trademark Guidelines of the Node.js Foundation</a>.</p>
110+
<p>Linux Foundation is a registered trademark of The Linux Foundation.</p>
111+
<p>Linux is a registered <a href="http://www.linuxfoundation.org/programs/legal/trademark" title="Linux Mark Institute">trademark</a> of Linus Torvalds.</p>
112+
</div>
113+
</div>
114+
</footer>
115+
</body>
116+
</html>''')

‎coverage/patches.diff‎

Lines changed: 131 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,131 @@
1+
diff --git a/.gitignore b/.gitignore
2+
index c7361af80c79..e56b7f913845 100644
3+
--- a/.gitignore
4+
+++ b/.gitignore
5+
@@ -21,6 +21,8 @@ node_g
6+
icu_config.gypi
7+
8+
/out
9+
+/coverage
10+
+/lib_
11+
12+
# various stuff that VC++ produces/uses
13+
Debug/
14+
diff --git a/lib/internal/bootstrap_node.js b/lib/internal/bootstrap_node.js
15+
index 27f05a4fcf14..ae0fed9e1c00 100644
16+
--- a/lib/internal/bootstrap_node.js
17+
+++ b/lib/internal/bootstrap_node.js
18+
@@ -42,6 +42,7 @@
19+
NativeModule.require('internal/process/stdio').setup();
20+
_process.setupKillAndExit();
21+
_process.setupSignalHandlers();
22+
+ NativeModule.require('internal/process/write-coverage').setup();
23+
24+
// Do not initialize channel in debugger agent, it deletes env variable
25+
// and the main thread won't see it.
26+
diff --git a/lib/internal/process/write-coverage.js b/lib/internal/process/write-coverage.js
27+
new file mode 100644
28+
index 000000000000..666939bc3389
29+
--- /dev/null
30+
+++ b/lib/internal/process/write-coverage.js
31+
@@ -0,0 +1,46 @@
32+
+'use strict';
33+
+const process = require('process');
34+
+const path = require('path');
35+
+const fs = require('fs');
36+
+const mkdirSync = fs.mkdirSync;
37+
+const writeFileSync = fs.writeFileSync;
38+
+
39+
+var isWritingCoverage = false;
40+
+function writeCoverage() {
41+
+ if (isWritingCoverage || !global.__coverage__) {
42+
+ return;
43+
+ }
44+
+ isWritingCoverage = true;
45+
+
46+
+ const dirname = path.join(path.dirname(process.execPath), '.coverage');
47+
+ const filename = `coverage-${process.pid}-${Date.now()}.json`;
48+
+ try {
49+
+ mkdirSync(dirname);
50+
+ } catch (err) {
51+
+ if (err.code !== 'EEXIST') {
52+
+ console.error(err);
53+
+ return;
54+
+ }
55+
+ }
56+
+
57+
+ const target = path.join(dirname, filename);
58+
+ const coverageInfo = JSON.stringify(global.__coverage__);
59+
+ try {
60+
+ writeFileSync(target, coverageInfo);
61+
+ } catch (err) {
62+
+ console.error(err);
63+
+ }
64+
+}
65+
+
66+
+function setup() {
67+
+ var reallyReallyExit = process.reallyExit;
68+
+
69+
+ process.reallyExit = function(code) {
70+
+ writeCoverage();
71+
+ reallyReallyExit(code);
72+
+ };
73+
+
74+
+ process.on('exit', writeCoverage);
75+
+}
76+
+
77+
+exports.setup = setup;
78+
diff --git a/node.gyp b/node.gyp
79+
index 2254a6e..2e91bd9 100644
80+
--- a/node.gyp
81+
+++ b/node.gyp
82+
@@ -86,6 +86,7 @@
83+
'lib/internal/process/promises.js',
84+
'lib/internal/process/stdio.js',
85+
'lib/internal/process/warning.js',
86+
+ 'lib/internal/process/write-coverage.js',
87+
'lib/internal/process.js',
88+
'lib/internal/readline.js',
89+
'lib/internal/repl.js',
90+
diff --git a/test/common.js b/test/common.js
91+
index 5aefdc3bcee5..750c134d33ab 100644
92+
--- a/test/common.js
93+
+++ b/test/common.js
94+
@@ -258,6 +258,9 @@ exports.platformTimeout = function(ms) {
95+
if (process.config.target_defaults.default_configuration === 'Debug')
96+
ms = 2 * ms;
97+
98+
+ if (global.__coverage__)
99+
+ ms = 4 * ms;
100+
+
101+
if (exports.isAix)
102+
return 2 * ms; // default localhost speed is slower on AIX
103+
104+
@@ -348,7 +351,7 @@ function leakedGlobals() {
105+
if (-1 === knownGlobals.indexOf(global[val]))
106+
leaked.push(val);
107+
108+
- return leaked;
109+
+ return leaked.filter((varname) => !/^__cov/.test(varname));
110+
}
111+
exports.leakedGlobals = leakedGlobals;
112+
113+
diff --git a/test/parallel/test-fs-sync-fd-leak.js b/test/parallel/test-fs-sync-fd-leak.js
114+
index f7cfd25f4b9b..80ad8cf6b705 100644
115+
--- a/test/parallel/test-fs-sync-fd-leak.js
116+
+++ b/test/parallel/test-fs-sync-fd-leak.js
117+
@@ -1,8 +1,13 @@
118+
'use strict';
119+
-require('../common');
120+
+const common = require('../common');
121+
var assert = require('assert');
122+
var fs = require('fs');
123+
124+
+if (global.__coverage__) {
125+
+ common.skip('Not working with coverage');
126+
+ return;
127+
+}
128+
+
129+
// ensure that (read|write|append)FileSync() closes the file descriptor
130+
fs.openSync = function() {
131+
return 42;

0 commit comments

Comments
 (0)