Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions doc/api/cli.md
Original file line number Diff line number Diff line change
Expand Up @@ -3729,6 +3729,15 @@ Enable the [module compile cache][] for the Node.js instance. See the documentat
When set to 1, the [module compile cache][] can be reused across different directory
locations as long as the module layout relative to the cache directory remains the same.

### `NODE_COMPILE_CACHE_READONLY=1`

<!-- YAML
added: REPLACEME
-->

When set to 1, the [module compile cache][] only reads existing entries from
its directory: nothing is written to it and it is not created if missing.

### `NODE_DEBUG=module[,…]`

<!-- YAML
Expand Down
18 changes: 18 additions & 0 deletions doc/api/module.md
Original file line number Diff line number Diff line change
Expand Up @@ -421,6 +421,15 @@

2. Setting the environment variable: [`NODE_COMPILE_CACHE_PORTABLE=1`][]

### Read-only compile cache

A cache that was generated ahead of time, for example at build time to be
shipped inside an application package, can be enabled with `readOnly: true`
(or [`NODE_COMPILE_CACHE_READONLY=1`][]). Node.js then loads whatever entries
the directory holds and never writes to it: modules without a usable entry are
compiled as usual but not persisted, [`module.flushCompileCache()`][] is a
no-op, and the directory is not created if it is missing.

### Limitations of the compile cache

Currently when using the compile cache with [V8 JavaScript code coverage][], the
Expand Down Expand Up @@ -494,6 +503,9 @@
<!-- YAML
added: v22.8.0
changes:
- version: REPLACEME
pr-url: https://github.com/nodejs/node/pull/00000

Check warning on line 507 in doc/api/module.md

View workflow job for this annotation

GitHub Actions / lint-pr-url

pr-url doesn't match the URL of the current PR.
description: Add the `readOnly` option.
- version:
- v25.4.0
- v24.15.0
Expand All @@ -520,6 +532,11 @@
the cache can be reused even if the project directory is moved. This is a best-effort
feature. If not specified, it will depend on whether the environment variable
[`NODE_COMPILE_CACHE_PORTABLE=1`][] is set.
* `readOnly` {boolean} Optional. If `true`, existing cache entries in `directory` are
used but nothing is ever written to it, and the directory is not created when it does
not exist (enabling then fails). Meant for caches generated ahead of time and shipped
with an application. If not specified, it will depend on whether the environment
variable [`NODE_COMPILE_CACHE_READONLY=1`][] is set.
* Returns: {Object}
* `status` {integer} One of the [`module.constants.compileCacheStatus`][]
* `message` {string|undefined} If Node.js cannot enable the compile cache, this contains
Expand Down Expand Up @@ -2059,6 +2076,7 @@
[`--require`]: cli.md#-r---require-module
[`NODE_COMPILE_CACHE=dir`]: cli.md#node_compile_cachedir
[`NODE_COMPILE_CACHE_PORTABLE=1`]: cli.md#node_compile_cache_portable1
[`NODE_COMPILE_CACHE_READONLY=1`]: cli.md#node_compile_cache_readonly1
[`NODE_DISABLE_COMPILE_CACHE=1`]: cli.md#node_disable_compile_cache1
[`NODE_V8_COVERAGE=dir`]: cli.md#node_v8_coveragedir
[`Object.freeze()`]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Object/freeze
Expand Down
4 changes: 4 additions & 0 deletions doc/node.1
Original file line number Diff line number Diff line change
Expand Up @@ -1862,6 +1862,10 @@ module compile cache for details.
When set to 1, the module compile cache can be reused across different directory
locations as long as the module layout relative to the cache directory remains the same.
.
.It Ev NODE_COMPILE_CACHE_READONLY Ar 1
When set to 1, the module compile cache only reads existing entries from
its directory: nothing is written to it and it is not created if missing.
.
.It Ev NODE_DEBUG Ar module[,…]
\fB','\fR-separated list of core modules that should print debug information.
.
Expand Down
15 changes: 11 additions & 4 deletions lib/internal/modules/helpers.js
Original file line number Diff line number Diff line change
Expand Up @@ -466,21 +466,25 @@ function stringify(body) {
* after this method is called.
* This method accepts either:
* - A string: path to the cache directory.
* - An options object `{directory?: string, portable?: boolean}`:
* - An options object `{directory?: string, portable?: boolean, readOnly?: boolean}`:
* - `directory`: A string path to the cache directory.
* - `portable`: If `portable` is true, the cache directory will be considered relative.
* Defaults to `NODE_COMPILE_CACHE_PORTABLE === '1'`.
* - `readOnly`: If `readOnly` is true, existing cache entries are used but nothing is
* written, and the cache directory is not created. Defaults to
* `NODE_COMPILE_CACHE_READONLY === '1'`.
* If cache directory is undefined, it defaults to the `NODE_COMPILE_CACHE` environment variable.
* If `NODE_COMPILE_CACHE` isn't set, it defaults to `path.join(os.tmpdir(), 'node-compile-cache')`.
* @param {string | { directory?: string, portable?: boolean } | undefined} options
* @param {string | { directory?: string, portable?: boolean, readOnly?: boolean } | undefined} options
* @returns {{status: number, message?: string, directory?: string}}
*/
function enableCompileCache(options) {
let portable;
let readOnly;
let directory;

if (typeof options === 'object' && options !== null) {
({ directory, portable } = options);
({ directory, portable, readOnly } = options);
} else {
directory = options;
}
Expand All @@ -490,7 +494,10 @@ function enableCompileCache(options) {
if (portable === undefined) {
portable = process.env.NODE_COMPILE_CACHE_PORTABLE === '1';
}
const nativeResult = _enableCompileCache(directory, portable);
if (readOnly === undefined) {
readOnly = process.env.NODE_COMPILE_CACHE_READONLY === '1';
}
const nativeResult = _enableCompileCache(directory, portable, readOnly);
const result = { status: nativeResult[0] };
if (nativeResult[1]) {
result.message = nativeResult[1];
Expand Down
70 changes: 52 additions & 18 deletions src/compile_cache.cc
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
#include "compile_cache.h"
#include <sys/stat.h> // S_ISDIR
#include <string>
#include "debug_utils-inl.h"
#include "env-inl.h"
Expand Down Expand Up @@ -250,7 +251,8 @@ CompileCacheEntry* CompileCacheHandler::GetOrInsert(Local<String> code,
// If the portable cache is enabled and it seems possible to compute the
// relative position from an absolute path, we use the relative position
// in the cache key.
if (portable_ == EnableOption::PORTABLE && IsAbsoluteFilePath(file_path)) {
if (HasOption(portable_, EnableOption::PORTABLE) &&
IsAbsoluteFilePath(file_path)) {
// Normalize the path to ensure it is consistent.
std::string normalized_file_path = NormalizeFileURLOrPath(env, file_path);
if (normalized_file_path.empty()) {
Expand Down Expand Up @@ -325,6 +327,10 @@ void CompileCacheHandler::MaybeSaveImpl(CompileCacheEntry* entry,
Debug("keeping the in-memory entry\n");
return;
}
if (read_only_) {
Debug("read-only, not serializing\n");
return;
}
Debug("%s the in-memory entry\n",
entry->cache == nullptr ? "initializing" : "refreshing");

Expand All @@ -349,6 +355,9 @@ void CompileCacheHandler::MaybeSave(CompileCacheEntry* entry,

void CompileCacheHandler::MaybeSave(CompileCacheEntry* entry,
std::string_view transpiled) {
if (read_only_) {
return;
}
CHECK(entry->type == CachedCodeType::kStrippedTypeScript);
Debug("[compile cache] saving transpilation cache for %s %s\n",
entry->type_name(),
Expand Down Expand Up @@ -381,6 +390,10 @@ void CompileCacheHandler::MaybeSave(CompileCacheEntry* entry,
*/
void CompileCacheHandler::Persist() {
DCHECK(!compile_cache_dir_.empty());
if (read_only_) {
Debug("[compile cache] read-only, skipping persistence\n");
return;
}

// TODO(joyeecheung): do this using a separate event loop to utilize the
// libuv thread pool and do the file system operations concurrently.
Expand Down Expand Up @@ -546,10 +559,11 @@ CompileCacheEnableResult CompileCacheHandler::Enable(Environment* env,
cache_tag,
cache_dir_with_tag);

if (!env->permission()->is_granted(
env,
permission::PermissionScope::kFileSystemWrite,
cache_dir_with_tag)) [[unlikely]] {
const bool read_only = HasOption(option, EnableOption::READ_ONLY);
if (!read_only && !env->permission()->is_granted(
env,
permission::PermissionScope::kFileSystemWrite,
cache_dir_with_tag)) [[unlikely]] {
result.message = "Skipping compile cache because write permission for " +
cache_dir_with_tag + " is not granted";
result.status = CompileCacheEnableStatus::FAILED;
Expand All @@ -566,25 +580,45 @@ CompileCacheEnableResult CompileCacheHandler::Enable(Environment* env,
return result;
}

fs::FSReqWrapSync req_wrap;
int err = fs::MKDirpSync(
nullptr, &(req_wrap.req), cache_dir_with_tag, 0777, nullptr);
if (is_debug_) {
Debug("[compile cache] creating cache directory %s...%s\n",
if (read_only) {
// A read-only cache is used as found and never created: without the
// directory there is nothing to read.
uv_fs_t stat_req;
int err =
uv_fs_stat(nullptr, &stat_req, cache_dir_with_tag.c_str(), nullptr);
bool is_dir = err == 0 && S_ISDIR(stat_req.statbuf.st_mode);
uv_fs_req_cleanup(&stat_req);
Debug("[compile cache] read-only cache directory %s...%s\n",
cache_dir_with_tag,
err < 0 ? uv_strerror(err) : "success");
}
if (err != 0 && err != UV_EEXIST) {
result.message =
"Cannot create cache directory: " + std::string(uv_strerror(err));
result.status = CompileCacheEnableStatus::FAILED;
return result;
is_dir ? "found" : "not found");
if (!is_dir) {
result.message =
"Cache directory does not exist (read-only): " + cache_dir_with_tag;
result.status = CompileCacheEnableStatus::FAILED;
return result;
}
} else {
fs::FSReqWrapSync req_wrap;
int err = fs::MKDirpSync(
nullptr, &(req_wrap.req), cache_dir_with_tag, 0777, nullptr);
if (is_debug_) {
Debug("[compile cache] creating cache directory %s...%s\n",
cache_dir_with_tag,
err < 0 ? uv_strerror(err) : "success");
}
if (err != 0 && err != UV_EEXIST) {
result.message =
"Cannot create cache directory: " + std::string(uv_strerror(err));
result.status = CompileCacheEnableStatus::FAILED;
return result;
}
}

result.cache_directory = absolute_cache_dir_base;
compile_cache_dir_ = cache_dir_with_tag;
portable_ = option;
if (option == EnableOption::PORTABLE) {
read_only_ = read_only;
if (HasOption(option, EnableOption::PORTABLE)) {
normalized_compile_cache_dir_ =
NormalizeFileURLOrPath(env, compile_cache_dir_);
}
Expand Down
19 changes: 18 additions & 1 deletion src/compile_cache.h
Original file line number Diff line number Diff line change
Expand Up @@ -60,7 +60,22 @@ struct CompileCacheEnableResult {
std::string message; // Set in case of failure.
};

enum class EnableOption : uint8_t { DEFAULT, PORTABLE };
enum class EnableOption : uint8_t {
DEFAULT = 0,
PORTABLE = 1 << 0,
// Only read existing cache entries: nothing is compiled into the in-memory
// store for persisting, nothing is written to disk, and the cache directory
// is not created if it does not exist.
READ_ONLY = 1 << 1,
};

inline constexpr EnableOption operator|(EnableOption a, EnableOption b) {
return static_cast<EnableOption>(static_cast<uint8_t>(a) |
static_cast<uint8_t>(b));
}
inline constexpr bool HasOption(EnableOption value, EnableOption flag) {
return (static_cast<uint8_t>(value) & static_cast<uint8_t>(flag)) != 0;
}

class CompileCacheHandler {
public:
Expand All @@ -82,6 +97,7 @@ class CompileCacheHandler {
bool rejected);
void MaybeSave(CompileCacheEntry* entry, std::string_view transpiled);
std::string_view cache_dir() { return compile_cache_dir_; }
bool read_only() const { return read_only_; }

private:
void ReadCacheFile(CompileCacheEntry* entry);
Expand All @@ -107,6 +123,7 @@ class CompileCacheHandler {
std::string compile_cache_dir_;
std::string normalized_compile_cache_dir_;
EnableOption portable_ = EnableOption::DEFAULT;
bool read_only_ = false;
std::unordered_map<uint32_t, std::unique_ptr<CompileCacheEntry>>
compiler_cache_store_;
};
Expand Down
10 changes: 8 additions & 2 deletions src/env.cc
Original file line number Diff line number Diff line change
Expand Up @@ -1195,8 +1195,14 @@ void Environment::InitializeCompileCache() {
DebugCategory::COMPILE_CACHE,
"[compile cache] using relative path\n");
}
EnableCompileCache(dir_from_env,
portable ? EnableOption::PORTABLE : EnableOption::DEFAULT);
std::string read_only_env;
bool read_only = credentials::SafeGetenv(
"NODE_COMPILE_CACHE_READONLY", &read_only_env, this) &&
read_only_env == "1";
EnableOption option = EnableOption::DEFAULT;
if (portable) option = option | EnableOption::PORTABLE;
if (read_only) option = option | EnableOption::READ_ONLY;
EnableCompileCache(dir_from_env, option);
}

CompileCacheEnableResult Environment::EnableCompileCache(
Expand Down
5 changes: 4 additions & 1 deletion src/node_modules.cc
Original file line number Diff line number Diff line change
Expand Up @@ -508,7 +508,10 @@ void EnableCompileCache(const FunctionCallbackInfo<Value>& args) {

EnableOption option = EnableOption::DEFAULT;
if (args.Length() > 1 && args[1]->IsTrue()) {
option = EnableOption::PORTABLE;
option = option | EnableOption::PORTABLE;
}
if (args.Length() > 2 && args[2]->IsTrue()) {
option = option | EnableOption::READ_ONLY;
}

Utf8Value value(isolate, args[0]);
Expand Down
65 changes: 65 additions & 0 deletions test/parallel/test-compile-cache-api-readonly.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
'use strict';

// This tests module.enableCompileCache({ directory, readOnly: true }): existing
// entries are read, nothing is written, and a missing directory is not created.

const common = require('../common');
const { spawnSyncAndAssert } = require('../common/child_process');
const assert = require('assert');
const fs = require('fs');
const path = require('path');
const tmpdir = require('../common/tmpdir');
const fixtures = require('../common/fixtures');

tmpdir.refresh();
const wrapper = fixtures.path('compile-cache-wrapper-options.js');
const target = path.join(tmpdir.path, 'target.js');
fs.writeFileSync(target, 'module.exports = 1;');
const other = path.join(tmpdir.path, 'other.js');
fs.writeFileSync(other, 'module.exports = 2;');
const directory = path.join(tmpdir.path, 'cache');
const list = () => fs.readdirSync(directory, { recursive: true }).sort();
const run = (options, extraEnv, requires, check) => spawnSyncAndAssert(
process.execPath,
[...requires.flatMap((r) => ['-r', r]), target],
{
env: {
...process.env,
NODE_DEBUG_NATIVE: 'COMPILE_CACHE',
NODE_TEST_COMPILE_CACHE_OPTIONS: JSON.stringify(options),
...extraEnv,
},
},
check);

// Read-only against a directory that does not exist: enabling fails and
// nothing is created.
run({ directory, readOnly: true }, {}, [wrapper], {
stderr: /read-only cache directory .*\.\.\.not found/,
});
assert(!fs.existsSync(directory));

// A normal run generates the cache for target.js.
run({ directory }, {}, [wrapper], {
stderr: /writing cache for .*target\.js.*success/,
});
const generated = list();
assert.notStrictEqual(generated.length, 0);

// Read-only against it: target.js's entry is accepted, other.js is compiled
// but not written, and persistence is skipped.
run({ directory, readOnly: true }, {}, [wrapper, other], {
stderr: common.mustCall((output) => {
assert.match(output, /cache for .*target\.js was accepted/);
assert.match(output, /read-only, skipping persistence/);
assert.doesNotMatch(output, /writing cache for/);
return true;
}),
});
assert.deepStrictEqual(list(), generated);

// The environment variable form.
run({ directory }, { NODE_COMPILE_CACHE_READONLY: '1' }, [wrapper, other], {
stderr: /read-only, skipping persistence/,
});
assert.deepStrictEqual(list(), generated);
Loading