refactor: use flush option with writeFile#5058
Conversation
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## master #5058 +/- ##
=======================================
Coverage 99.78% 99.78%
=======================================
Files 221 221
Lines 16345 16351 +6
Branches 3933 3936 +3
=======================================
+ Hits 16310 16316 +6
Misses 35 35 ☔ View full report in Codecov by Sentry. |
Are you sure about that? The library itself is using it by awaiting internally: https://github.com/jprichardson/node-fs-extra/blob/master/lib/ensure/file.js#L24 Also the types say it's the promisified version. If it's true, it sounds like something worth reporting and fixing upstream. FWIW, I don't think I saw a flaky test that would point to this problem, so could be surely windows specific. |
|
I don't think this is true, when I console.log the method without awaiting, I get a Promise instance back. import { writeFile } from 'fs-extra';
console.log(writeFile('foo', 'bar'));
Promise { <pending> } |
Added "flush: true" to ensure that all files written are immediately accessible. The code already collects the entire contents of a file to write it in one go, and promises are used to prevent blocking the event loop. This in turn fixes potentially flaky tests, though the cost is that the overall time until all file writing promises are resolved will be a bit more. Also reordered imports in the affected files alphabetically.
d06c674 to
0100ed9
Compare
|
Hm... after digging through more sources, it seems the "core" of this is in So... yes, there is a promise there, but it seems there may be some kind of an edge case, at least on Windows, where the native promise writeFile handles things properly, while the universalify writeFile does not. The
which leads me to believe that may be what's happening here, except I don't get those errors... It's just that Or, equally likely, the native promise version is scheduling the flushes differently, and so without an explicit flush option or filehandle.sync() call, the end result is that the flush is scheduled later with the non-natively promised version. A more reliable fix would be to take advantage of the flush option in Node 21 I guess (see nodejs/node#50009 ), which would be ignored for versions that don't support it. I've now changed the PR to that. I re-ran the tests several times, and so far at least, I haven't gotten a single flaky test. |
flush option with writeFile
In all places of the code, fs-extra's writeFile is used. This package however is actually a re-export of the callback version of node:fs' writeFile.
Meanwhile, all packages seem to assume this is the version from node:fs/promises.
This fix in turn fixes tests that can sometimes be flaky, due to assuming file operations are done, when that isn't necessarily true.
Also reordered imports in the affected files alphabetically.
I noticed this while running tests locally. Random tests sometimes fail, and all they had in common was the fact that they were checking for a file's existence. I thought it's maybe an fs-extra specific issue with Windows, but then discovered it's not re-exporting the promise, hence this PR.