Skip to content

Fix child process stdio data loss with slow piped consumers #7185

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
petrosagg wants to merge 3 commits into from
Closed

Fix child process stdio data loss with slow piped consumers #7185

Changes from 1 commit
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
3cfdf58
child-process: add stdio flush testcase with pipes
petrosagg Jun 7, 2016
b7d3f32
child-process: fix data loss with slow piped consumers
petrosagg Jun 7, 2016
f179bb6
test: fix testcase to actually detect regression
petrosagg Jun 7, 2016
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Next Next commit
child-process: add stdio flush testcase with pipes 
While commit 12274a5 fixed data loss when a readable event handler is
attached, the problem still exists when the stdio stream has a piped
consumer that doesn't read fast enough.

Signed-off-by: Petros Angelatos <petrosagg@gmail.com>
  • Loading branch information
@petrosagg
petrosagg committed Jun 7, 2016
commit 3cfdf58e341481e645f0421c2dcb65def05cae4d
31 changes: 31 additions & 0 deletions test/parallel/test-child-process-flush-stdio.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@
const cp = require('child_process');
const common = require('../common');
const assert = require('assert');
const stream = require('stream');

const p = cp.spawn('echo');

Expand All @@ -20,10 +21,40 @@ function spawnWithReadable() {
assert.strictEqual(code, 0);
assert.strictEqual(signal, null);
assert.strictEqual(Buffer.concat(buffer).toString().trim(), '123');
spawnWithPipe();
}));
p.stdout.on('readable', function() {
let buf;
while (buf = this.read())
buffer.push(buf);
});
}

function spawnWithPipe() {
const buffer = [];
const through = new stream.PassThrough();
const p = cp.spawn('seq', [ '36000' ]);
Copy link
Copy Markdown
Member

@addaleax addaleax Jun 7, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think something more cross-platform than seq might be nice here.

Copy link
Copy Markdown
Author

@petrosagg petrosagg Jun 7, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed. I can PR just the testcase separately although I'm not sure if it adds any value since the issue turned out to be in the stream implementation.

Copy link
Copy Markdown
Member

@addaleax addaleax Jun 7, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@petrosagg Too many tests never hurt anyone, so again thumbs up for a separate PR. ;)


const ret = [];
for (let i = 1; i <= 36000; i++) {
ret.push(i);
}

p.on('close', common.mustCall(function(code, signal) {
assert.strictEqual(code, 0);
assert.strictEqual(signal, null);
assert.strictEqual(Buffer.concat(buffer).toString().trim(), ret.join('\n'));
}));

p.on('exit', common.mustCall(function() {
Copy link
Copy Markdown
Member

@addaleax addaleax Jun 7, 2016 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should read close, as there’s currently no guarantee that 'close' will be emitted before 'exit'.

setImmediate(function() {
through.on('readable', function() {
let buf;
while (buf = this.read())
buffer.push(buf);
});
});
}));

p.stdout.pipe(through);
}