Skip to content

fs: buffer dir entries in opendir() #29893

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
addaleax wants to merge 6 commits into from
Closed

fs: buffer dir entries in opendir() #29893

Changes from 1 commit
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
8bf527e
fs: buffer dir entries in opendir()
addaleax Oct 8, 2019
6d209fc
fixup! fs: buffer dir entries in opendir()
addaleax Oct 10, 2019
c66840d
doc: fs dir modifications may not be reflected by dir.read
addaleax Oct 10, 2019
e2f1071
fixup! fixup! fs: buffer dir entries in opendir()
addaleax Oct 10, 2019
59cf84d
fixup! fs: buffer dir entries in opendir()
addaleax Oct 10, 2019
751a8b6
fixup! fixup! fs: buffer dir entries in opendir()
addaleax Oct 11, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
fixup! fs: buffer dir entries in opendir()
  • Loading branch information
@addaleax
addaleax committed Oct 10, 2019
commit 59cf84d2322af4b7a0e5d8c7bbc7e3fb0b575f4b
20 changes: 18 additions & 2 deletions benchmark/fs/bench-opendir.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ const path = require('path');
const bench = common.createBenchmark(main, {
n: [100],
dir: [ 'lib', 'test/parallel'],
mode: [ 'async', 'sync' ]
mode: [ 'async', 'sync', 'callback' ]
});

async function main({ n, dir, mode }) {
Expand All @@ -21,7 +21,23 @@ async function main({ n, dir, mode }) {
// eslint-disable-next-line no-unused-vars
for await (const entry of await fs.promises.opendir(fullPath))
counter++;
} else {
} else if (mode === 'callback') {
const dir = await fs.promises.opendir(fullPath);
await new Promise((resolve, reject) => {
function read() {
dir.read((err, entry) => {
if (err)
reject(err);
if (entry === null)
resolve(dir.close());
else
read();
});
}

read();
});
} {
Copy link
Copy Markdown
Contributor

@Fishrock123 Fishrock123 Oct 11, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@addaleax This patch seems wrong? There should be an else here... I think the benchmark you ran included readSync() along with the callback one...

Copy link
Copy Markdown
Member Author

@addaleax addaleax Oct 11, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oops, yes … funny the linter wouldn’t complain about this? The correct benchmark results for changing to setImmediate are

                                                                confidence improvement accuracy (*)    (**)   (***)
 fs/bench-opendir.js mode='callback' dir='lib' n=100                           3.40 %      ±15.95% ±21.22% ±27.62%
 fs/bench-opendir.js mode='callback' dir='test/parallel' n=100        ***    -43.16 %      ±10.70% ±14.27% ±18.66%

Copy link
Copy Markdown
Contributor

@Fishrock123 Fishrock123 Oct 11, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This result seems strange to me, how is that worse than before?

Regardless though after further though I think that nextTick is the reasonable thing to do.

Copy link
Copy Markdown
Member Author

@addaleax addaleax Oct 11, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Fishrock123 I think it makes sense – as you noted, the readSync() part was run unconditionally before, so its (unchanged) performance was included in the -30 %. If we only benchmark what has been slowed down, seeing more negative impact seems about right to me.

const dir = fs.opendirSync(fullPath);
while (dir.readSync() !== null)
counter++;
Expand Down