Skip to content

Commit 4a3c5c3

Browse files
Fishrock123juanarbol
authored andcommitted
fs: introduce opendir() and fs.Dir
This adds long-requested methods for asynchronously interacting and iterating through directory entries by using `uv_fs_opendir`, `uv_fs_readdir`, and `uv_fs_closedir`. `fs.opendir()` and friends return an `fs.Dir`, which contains methods for doing reads and cleanup. `fs.Dir` also has the async iterator symbol exposed. The `read()` method and friends only return `fs.Dirent`s for this API. Having a entry type or doing a `stat` call is deemed to be necessary in the majority of cases, so just returning dirents seems like the logical choice for a new api. Reading when there are no more entries returns `null` instead of a dirent. However the async iterator hides that (and does automatic cleanup). The code lives in separate files from the rest of fs, this is done partially to prevent over-pollution of those (already very large) files, but also in the case of js allows loading into `fsPromises`. Due to async_hooks, this introduces a new handle type of `DIRHANDLE`. This PR does not attempt to make complete optimization of this feature. Notable future improvements include: - Moving promise work into C++ land like FileHandle. - Possibly adding `readv()` to do multi-entry directory reads. - Aliasing `fs.readdir` to `fs.scandir` and doing a deprecation. Refs: nodejs/node-v0.x-archive#388 Refs: nodejs/node#583 Refs: libuv/libuv#2057 PR-URL: nodejs/node#29349 Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: David Carlier <devnexen@gmail.com>
1 parent 579d1c5 commit 4a3c5c3

7 files changed

Lines changed: 247 additions & 291 deletions

File tree

‎doc/api/fs.md‎

Lines changed: 95 additions & 70 deletions
Original file line numberDiff line numberDiff line change
@@ -286,13 +286,14 @@ performance implications for some applications. See the
286286

287287
## Class fs.Dir
288288
<!-- YAML
289-
added: v12.12.0
289+
added: REPLACEME
290290
-->
291291

292292
A class representing a directory stream.
293293

294-
Created by [`fs.opendir()`][], [`fs.opendirSync()`][], or
295-
[`fsPromises.opendir()`][].
294+
Created by [`fs.opendir()`][], [`fs.opendirSync()`][], or [`fsPromises.opendir()`][].
295+
296+
Example using async interation:
296297

297298
```js
298299
const fs = require('fs');
@@ -306,9 +307,19 @@ async function print(path) {
306307
print('./').catch(console.error);
307308
```
308309

310+
### dir.path
311+
<!-- YAML
312+
added: REPLACEME
313+
-->
314+
315+
* {string}
316+
317+
The read-only path of this directory as was provided to [`fs.opendir()`][],
318+
[`fs.opendirSync()`][], or [`fsPromises.opendir()`][].
319+
309320
### dir.close()
310321
<!-- YAML
311-
added: v12.12.0
322+
added: REPLACEME
312323
-->
313324

314325
* Returns: {Promise}
@@ -321,7 +332,7 @@ closed.
321332

322333
### dir.close(callback)
323334
<!-- YAML
324-
added: v12.12.0
335+
added: REPLACEME
325336
-->
326337

327338
* `callback` {Function}
@@ -334,104 +345,84 @@ The `callback` will be called after the resource handle has been closed.
334345

335346
### dir.closeSync()
336347
<!-- YAML
337-
added: v12.12.0
348+
added: REPLACEME
338349
-->
339350

340351
Synchronously close the directory's underlying resource handle.
341352
Subsequent reads will result in errors.
342353

343-
### dir.path
344-
<!-- YAML
345-
added: v12.12.0
346-
-->
347-
348-
* {string}
349-
350-
The read-only path of this directory as was provided to [`fs.opendir()`][],
351-
[`fs.opendirSync()`][], or [`fsPromises.opendir()`][].
352-
353-
### dir.read()
354+
### dir.read([options])
354355
<!-- YAML
355-
added: v12.12.0
356+
added: REPLACEME
356357
-->
357358

358-
* Returns: {Promise} containing {fs.Dirent|null}
359+
* `options` {Object}
360+
* `encoding` {string|null} **Default:** `'utf8'`
361+
* Returns: {Promise} containing {fs.Dirent}
359362

360363
Asynchronously read the next directory entry via readdir(3) as an
361364
[`fs.Dirent`][].
362365

363-
After the read is completed, a `Promise` is returned that will be resolved with
364-
an [`fs.Dirent`][], or `null` if there are no more directory entries to read.
366+
A `Promise` is returned that will be resolved with a [Dirent][] after the read
367+
is completed.
365368

366-
Directory entries returned by this function are in no particular order as
367-
provided by the operating system's underlying directory mechanisms.
368-
Entries added or removed while iterating over the directory may or may not be
369-
included in the iteration results.
369+
_Directory entries returned by this function are in no particular order as
370+
provided by the operating system's underlying directory mechanisms._
370371

371-
### dir.read(callback)
372+
### dir.read([options, ]callback)
372373
<!-- YAML
373-
added: v12.12.0
374+
added: REPLACEME
374375
-->
375376

377+
* `options` {Object}
378+
* `encoding` {string|null} **Default:** `'utf8'`
376379
* `callback` {Function}
377380
* `err` {Error}
378-
* `dirent` {fs.Dirent|null}
381+
* `dirent` {fs.Dirent}
379382

380383
Asynchronously read the next directory entry via readdir(3) as an
381384
[`fs.Dirent`][].
382385

383-
After the read is completed, the `callback` will be called with an
384-
[`fs.Dirent`][], or `null` if there are no more directory entries to read.
386+
The `callback` will be called with a [Dirent][] after the read is completed.
385387

386-
Directory entries returned by this function are in no particular order as
387-
provided by the operating system's underlying directory mechanisms.
388-
Entries added or removed while iterating over the directory may or may not be
389-
included in the iteration results.
388+
The `encoding` option sets the encoding of the `name` in the `dirent`.
390389

391-
### dir.readSync()
390+
_Directory entries returned by this function are in no particular order as
391+
provided by the operating system's underlying directory mechanisms._
392+
393+
### dir.readSync([options])
392394
<!-- YAML
393-
added: v12.12.0
395+
added: REPLACEME
394396
-->
395397

396-
* Returns: {fs.Dirent|null}
398+
* `options` {Object}
399+
* `encoding` {string|null} **Default:** `'utf8'`
400+
* Returns: {fs.Dirent}
397401

398402
Synchronously read the next directory entry via readdir(3) as an
399403
[`fs.Dirent`][].
400404

401-
If there are no more directory entries to read, `null` will be returned.
405+
The `encoding` option sets the encoding of the `name` in the `dirent`.
402406

403-
Directory entries returned by this function are in no particular order as
404-
provided by the operating system's underlying directory mechanisms.
405-
Entries added or removed while iterating over the directory may or may not be
406-
included in the iteration results.
407+
_Directory entries returned by this function are in no particular order as
408+
provided by the operating system's underlying directory mechanisms._
407409

408410
### dir\[Symbol.asyncIterator\]()
409411
<!-- YAML
410-
added: v12.12.0
412+
added: REPLACEME
411413
-->
412414

413-
* Returns: {AsyncIterator} of {fs.Dirent}
415+
* Returns: {AsyncIterator} to fully iterate over all entries in the directory.
414416

415-
Asynchronously iterates over the directory via readdir(3) until all entries have
416-
been read.
417-
418-
Entries returned by the async iterator are always an [`fs.Dirent`][].
419-
The `null` case from `dir.read()` is handled internally.
420-
421-
See [`fs.Dir`][] for an example.
422-
423-
Directory entries returned by this iterator are in no particular order as
424-
provided by the operating system's underlying directory mechanisms.
425-
Entries added or removed while iterating over the directory may or may not be
426-
included in the iteration results.
417+
_Directory entries returned by this iterator are in no particular order as
418+
provided by the operating system's underlying directory mechanisms._
427419

428420
## Class: fs.Dirent
429421
<!-- YAML
430422
added: v10.10.0
431423
-->
432424

433-
A representation of a directory entry, as returned by reading from an
434-
[`fs.Dir`][].
425+
A representation of a directory entry, as returned by reading from an [`fs.Dir`][].
435426

436427
Additionally, when [`fs.readdir()`][] or [`fs.readdirSync()`][] is called with
437428
the `withFileTypes` option set to `true`, the resulting array is filled with
@@ -2701,6 +2692,46 @@ Returns an integer representing the file descriptor.
27012692
For detailed information, see the documentation of the asynchronous version of
27022693
this API: [`fs.open()`][].
27032694

2695+
## fs.opendir(path[, options], callback)
2696+
<!-- YAML
2697+
added: REPLACEME
2698+
-->
2699+
2700+
* `path` {string|Buffer|URL}
2701+
* `options` {Object}
2702+
* `encoding` {string|null} **Default:** `'utf8'`
2703+
* `callback` {Function}
2704+
* `err` {Error}
2705+
* `dir` {fs.Dir}
2706+
2707+
Asynchronously open a directory. See opendir(3).
2708+
2709+
Creates an [`fs.Dir`][], which contains all further functions for reading from
2710+
and cleaning up the directory.
2711+
2712+
The `encoding` option sets the encoding for the `path` while opening the
2713+
directory and subsequent read operations (unless otherwise overriden during
2714+
reads from the directory).
2715+
2716+
## fs.opendirSync(path[, options])
2717+
<!-- YAML
2718+
added: REPLACEME
2719+
-->
2720+
2721+
* `path` {string|Buffer|URL}
2722+
* `options` {Object}
2723+
* `encoding` {string|null} **Default:** `'utf8'`
2724+
* Returns: {fs.Dir}
2725+
2726+
Synchronously open a directory. See opendir(3).
2727+
2728+
Creates an [`fs.Dir`][], which contains all further functions for reading from
2729+
and cleaning up the directory.
2730+
2731+
The `encoding` option sets the encoding for the `path` while opening the
2732+
directory and subsequent read operations (unless otherwise overriden during
2733+
reads from the directory).
2734+
27042735
## fs.read(fd, buffer, offset, length, position, callback)
27052736
<!-- YAML
27062737
added: v0.0.2
@@ -4862,21 +4893,14 @@ by [Naming Files, Paths, and Namespaces][]. Under NTFS, if the filename contains
48624893
a colon, Node.js will open a file system stream, as described by
48634894
[this MSDN page][MSDN-Using-Streams].
48644895

4865-
### fsPromises.opendir(path\[, options\])
4896+
## fsPromises.opendir(path[, options])
48664897
<!-- YAML
4867-
added: v12.12.0
4868-
changes:
4869-
- version: v13.1.0
4870-
pr-url: https://github.com/nodejs/node/pull/30114
4871-
description: The `bufferSize` option was introduced.
4898+
added: REPLACEME
48724899
-->
48734900

48744901
* `path` {string|Buffer|URL}
48754902
* `options` {Object}
48764903
* `encoding` {string|null} **Default:** `'utf8'`
4877-
* `bufferSize` {number} Number of directory entries that are buffered
4878-
internally when reading from the directory. Higher values lead to better
4879-
performance but higher memory usage. **Default:** `32`
48804904
* Returns: {Promise} containing {fs.Dir}
48814905

48824906
Asynchronously open a directory. See opendir(3).
@@ -4885,9 +4909,10 @@ Creates an [`fs.Dir`][], which contains all further functions for reading from
48854909
and cleaning up the directory.
48864910

48874911
The `encoding` option sets the encoding for the `path` while opening the
4888-
directory and subsequent read operations.
4912+
directory and subsequent read operations (unless otherwise overriden during
4913+
reads from the directory).
48894914

4890-
Example using async iteration:
4915+
Example using async interation:
48914916

48924917
```js
48934918
const fs = require('fs');
@@ -4901,7 +4926,7 @@ async function print(path) {
49014926
print('./').catch(console.error);
49024927
```
49034928

4904-
### fsPromises.readdir(path\[, options\])
4929+
### fsPromises.readdir(path[, options])
49054930
<!-- YAML
49064931
added: v10.0.0
49074932
changes:

0 commit comments

Comments
 (0)