You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
*[Why are library packages in a node_modules directory?](#lib-node-modules)
18
19
*[How can I contribute?](#contributing)
19
20
20
21
@@ -155,9 +156,9 @@ To better support numeric computing, standards bodies can do the following:
155
156
*__Universally unique identifiers__. Universally unique identifiers ([UUIDs][uuid]) are stored as 128-bit values.
156
157
*__Arbitrary-precision arithmetic__. [Arbitrary-precision arithmetic][arbitrary-precision-arithmetic] is beneficial for high precision applications and in preventing overflow, computing fundamental mathematical constants, and evaluating precision errors in fixed-precision calculations.
157
158
158
-
1.__Large arrays__: add support for large arrays. Arrays are currently limited to [`2**32-1`][ecma262-array-length] (approximately `4` billion) elements. Many applications will never reach this limit; however, as datasets continue to increase in size, the need for larger arrays will become more apparent. For example, consider a `100000 x 100000` dense matrix, which is not uncommon when working with sensor data and trying to find correlations. This matrix will have `10` billion elements. Given current length limitations, one cannot store this data contiguously in a plain JavaScript array, thus resulting in increased cache misses and decreased performance.
159
+
1.__Large arrays__: add support for large arrays. Arrays are currently limited to [`2**32-1`][ecma-262-array-length] (approximately `4` billion) elements. Many applications will never reach this limit; however, as datasets continue to increase in size, the need for larger arrays will become more apparent. For example, consider a `100000 x 100000` dense matrix, which is not uncommon when working with sensor data and trying to find correlations. This matrix will have `10` billion elements. Given current length limitations, one cannot store this data contiguously in a plain JavaScript array, thus resulting in increased cache misses and decreased performance.
159
160
160
-
__Aside:__ typed arrays and, more generally, array-like objects may have as many as [`2**53-1`][ecma262-tolength] elements. In the case of typed arrays, however, one must allocate memory upon instantiation; thus, growing a typed array as needed, while possible, is neither straightforward nor efficient.
161
+
__Aside:__ typed arrays and, more generally, array-like objects may have as many as [`2**53-1`][ecma-262-tolength] elements. In the case of typed arrays, however, one must allocate memory upon instantiation; thus, growing a typed array as needed, while possible, is neither straightforward nor efficient.
161
162
162
163
1.__Typed objects__: add support for [typed objects][typed-objects-proposal]. Typed objects would facilitate efficient memory storage of data, which is critical for [performant][five-things-that-make-go-fast] numeric computations. In short,
163
164
@@ -173,11 +174,11 @@ To better support numeric computing, standards bodies can do the following:
173
174
174
175
1.__Big numbers__: add support for big [integers][julia-bigint], [rationals][golang-big], and [floats][julia-bigfloat]. In addition to cryptography and computing irrational numbers, arbitrary precision arithmetic is useful for algorithms involving double-precision floating-point numbers. Currently, lack of efficient, and relatively performant, big number support limits the scope and types of implemented algorithms, including for basic transcendental functions.
175
176
176
-
1.__SIMD__: add support for long SIMD. Currently, [proposals][ecmascript-simd] for [SIMD][simd-js] in JavaScript have focused on [short SIMD][mozilla-simd], which is well-suited for graphics applications. However, [short SIMD][mozilla-simd] is __not__ particularly well-suited for large vector operations, which are common in numeric computing (e.g., BLAS).
177
+
1.__SIMD__: add support for long SIMD. Currently, [proposals][ecmascript-simd] for [SIMD][mdn-simd-js] in JavaScript have focused on [short SIMD][mozilla-simd], which is well-suited for graphics applications. However, [short SIMD][mozilla-simd] is __not__ particularly well-suited for large vector operations, which are common in numeric computing (e.g., BLAS).
177
178
178
179
__Aside:__ JavaScript may never have native SIMD support. Instead, SIMD may be possible only via [WebAssembly][wasm]. Lack of native JavaScript SIMD support would be unfortunate, as plenty of applications exist (e.g., scripting for purposes of analysis and data manipulation), which would benefit from SIMD operations without requiring a context switch to a lower-level language and additional compilation steps.
179
180
180
-
1.__Parallelism__: add support for lightweight threading (parallelism). Currently, [data parallelism][data-parallelism], i.e., the same operations performed on different subsets of the same data, is only achievable by manual data orchestration and task execution via either [web workers][web-workers] (browser) or [child processes][child-process] (Node.js). While [web workers][web-workers] support [Transferable Objects][transferable-objects] thus allowing shared memory access, the same is not true for Node.js. Particularly in Node.js, task parallelism is heavyweight and cumbersome, especially for use cases like parallel computation involving matrix elements (e.g., compare to MATLAB's [`parfor`][matlab-parfor]). The ability to easily distribute data to a worker pool (processors) would provide a significant performance boost to many data analysis tasks.
181
+
1.__Parallelism__: add support for lightweight threading (parallelism). Currently, [data parallelism][data-parallelism], i.e., the same operations performed on different subsets of the same data, is only achievable by manual data orchestration and task execution via either [web workers][mdn-web-workers] (browser) or [child processes][node-child-process] (Node.js). While [web workers][mdn-web-workers] support [Transferable Objects][mdn-transferable-objects] thus allowing shared memory access, the same is not true for Node.js. Particularly in Node.js, task parallelism is heavyweight and cumbersome, especially for use cases like parallel computation involving matrix elements (e.g., compare to MATLAB's [`parfor`][matlab-parfor]). The ability to easily distribute data to a worker pool (processors) would provide a significant performance boost to many data analysis tasks.
181
182
182
183
1.__GPGPU__: provide better support for [GPGPU][gpgpu]. Currently, performing general purpose GPU (GPGPU) computing tasks within a browser is only possible via [WebGL][webgl] and awkward usage of shaders, which are designed for generating graphics, not generic number crunching. Additionally, synchronous data transfers between the main thread and the GPU are expensive, debugging support is limited, and reading floating-point textures is not possible without workarounds which encode floating-point numbers into integer outputs (RGBA). While [compute shaders][compute-shaders] and [Vulkan][vulkan] promise better GPGPU support, we are years away from realizing their proposed benefits via JavaScript. Once realized, however, embarrassingly parallel computation tasks and machine learning techniques such as neural networks become more viable and efficient.
183
184
@@ -279,6 +280,126 @@ The reasons are as follows:
279
280
<!-- </faq-question> -->
280
281
281
282
283
+
<!-- <faq-question> -->
284
+
285
+
---
286
+
287
+
<aname="lib-node-modules"></a>
288
+
289
+
### Why are library packages in a node_modules directory?
290
+
291
+
This project leverages the Node.js module resolution [algorithm][node-require] to resolve dependencies by name rather than by path. Hence, the project avoids relative require paths. For example,
292
+
293
+
```javascript
294
+
var foo =require( './../../../../../@stdlib/foo' );
295
+
```
296
+
297
+
becomes
298
+
299
+
```javascript
300
+
var foo =require( '@stdlib/foo' );
301
+
```
302
+
303
+
In general, far too many developers are oblivious to the module resolution [algorithm][node-require], often resorting to various unnecessary hacks, such as setting environment variables (e.g., `NODE_PATH`), using globals, creating symbolic links (symlink), using `require` wrappers, running startup scripts, or actually hacking `require`, itself (see [here][modifying-node-path-hack] and [here][list-of-require-hacks] as representative references). A superior approach is to leverage the module resolution [algorithm][node-require] to scope internal packages to their relevant context. For example, consider the following application directory structure
304
+
305
+
```text
306
+
/
307
+
app/
308
+
index.js
309
+
a.js
310
+
b.js
311
+
node_modules/ # => local app dependencies
312
+
debug/
313
+
index.js
314
+
logger/
315
+
index.js
316
+
routes/
317
+
index.js
318
+
login/
319
+
index.js
320
+
login.js
321
+
post.js
322
+
onerror.js
323
+
logout/
324
+
index.js
325
+
logout.js
326
+
post.js
327
+
onerror.js
328
+
node_modules/ # => local routes dependencies
329
+
start/
330
+
index.js
331
+
end/
332
+
index.js
333
+
response-time/
334
+
index.js
335
+
models/
336
+
index.js
337
+
c.js
338
+
d.js
339
+
user/
340
+
index.js
341
+
e.js
342
+
f.js
343
+
node_modules/ # => local user model dependencies
344
+
foo/
345
+
index.js
346
+
bar/
347
+
index.js
348
+
data/
349
+
index.js
350
+
g.js
351
+
h.js
352
+
i.js
353
+
node_modules/ # => local data model dependencies
354
+
transform/
355
+
index.js
356
+
analyze/
357
+
index.js
358
+
morph/
359
+
index.js
360
+
node_modules/ # => local models dependencies
361
+
db-connect/
362
+
index.js
363
+
db-get/
364
+
index.js
365
+
db-set/
366
+
index.js
367
+
db-update/
368
+
index.js
369
+
db-delete/
370
+
index.js
371
+
node_modules/ # => 3rd party dependencies
372
+
beep/
373
+
boop/
374
+
bop/
375
+
```
376
+
377
+
where `g.js`
378
+
379
+
```javascript
380
+
var beep =require( 'beep' );
381
+
var debug =require( 'debug' );
382
+
var get =require( 'db-get' );
383
+
var transform =require( 'transform' );
384
+
var h =require( './h.js' );
385
+
386
+
// ...
387
+
```
388
+
389
+
By leveraging node_modules, each local node_modules dependency
390
+
391
+
a. is scoped to its relevant context
392
+
b. does not pollute the top-level node_modules directory which contains 3rd party dependencies
393
+
c. allows modules within a scope to require the dependency by name rather than by relative path in a manner similar to node_module dependencies in parent scopes (including 3rd party dependencies)
394
+
d. may be elevated to a higher scope without needing to update require paths
395
+
396
+
In short, the module resolution [algorithm][node-require] provides a simple and robust cross-platform solution for managing both external and local module dependencies.
397
+
398
+
__Aside__: A common objection to the directory structure above is that tools often ignore anything within a node_modules folder (e.g., linters, unit test runners, etc). That this project is able to configure tools to recognize files within node_modules folders is evidence to the contrary. If a tool cannot be configured otherwise, that is a flaw in the tool, not in the approach.
399
+
400
+
<!-- </faq-question> -->
401
+
402
+
282
403
<!-- <faq-question> -->
283
404
284
405
---
@@ -299,7 +420,11 @@ See the [contributing guide][contributing-guide].
0 commit comments