Skip to content

repl: tab auto complete big arrays #22408

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
BridgeAR wants to merge 3 commits into from
Closed

repl: tab auto complete big arrays #22408

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
3f26e1f
repl: tab auto complete big arrays
BridgeAR Aug 20, 2018
7be3845
fixup: address comments
BridgeAR Aug 20, 2018
65993be
fixup: address c++ linter
BridgeAR Aug 20, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Next Next commit
repl: tab auto complete big arrays 
Due to a new API it's possible to skip the indices. That allows to
use auto completion with big (typed) arrays.

Fixes: #21446
  • Loading branch information
@BridgeAR
BridgeAR committed Aug 20, 2018
commit 3f26e1f071de56bc652e0020563a029ad5ff4556
9 changes: 9 additions & 0 deletions lib/internal/util.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -366,6 +366,14 @@ function isInsideNodeModules() {
return false;
}

const kPropertyFilter = Object.freeze({
Copy link
Copy Markdown
Member

@benjamingr benjamingr Aug 20, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it make sense to copy the whole enum as it is in V8 ( https://github.com/v8/v8/blob/master/src/property-details.h#L36-L45 ) for consistency and keeping a reference to the V8 source here?

Copy link
Copy Markdown
Member

@addaleax addaleax Aug 20, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So far, we usually go all the way here and provide the real V8 constants as exports from the binding, e.g.

node/src/pipe_wrap.cc

Lines 105 to 114 in 9bcb744

// Define constants
Local<Object> constants = Object::New(env->isolate());
NODE_DEFINE_CONSTANT(constants, SOCKET);
NODE_DEFINE_CONSTANT(constants, SERVER);
NODE_DEFINE_CONSTANT(constants, IPC);
NODE_DEFINE_CONSTANT(constants, UV_READABLE);
NODE_DEFINE_CONSTANT(constants, UV_WRITABLE);
target->Set(context,
FIXED_ONE_BYTE_STRING(env->isolate(), "constants"),
constants).FromJust();

I’d encourage you to keep up with that pattern to prevent unexpected breakage from changes to the V8 definition, which is the source-of-truth for these values, even if it seems like a bit of extra work at this point (or leave to TODO comment for me, I’m happy to do the work myself if you prefer)

Copy link
Copy Markdown
Member Author

@BridgeAR BridgeAR Aug 20, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks a lot for the comments. I definitely appreciate it as I try to teach myself more and more about C++ and the V8 side. So please always mention these things.

ALL_PROPERTIES: 0,
ONLY_WRITABLE: 1,
ONLY_ENUMERABLE: 2,
ONLY_CONFIGURABLE: 4,
SKIP_STRINGS: 8,
SKIP_SYMBOLS: 16
});

module.exports = {
assertCrypto,
Expand All @@ -386,6 +394,7 @@ module.exports = {
promisify,
spliceOne,
removeColors,
kPropertyFilter,

// Symbol used to customize promisify conversion
customPromisifyArgs: kCustomPromisifyArgsSymbol,
Expand Down
11 changes: 7 additions & 4 deletions lib/internal/util/comparisons.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ const {
isSet
} = internalBinding('types');
const { getOwnNonIndexProperties } = process.binding('util');
const { kPropertyFilter: { ONLY_ENUMERABLE } } = require('internal/util');

const ReflectApply = Reflect.apply;

Expand Down Expand Up @@ -118,8 +119,9 @@ function strictDeepEqual(val1, val2, memos) {
if (val1.length !== val2.length) {
return false;
}
const keys1 = getOwnNonIndexProperties(val1);
if (keys1.length !== getOwnNonIndexProperties(val2).length) {
const keys1 = getOwnNonIndexProperties(val1, ONLY_ENUMERABLE);
const keys2 = getOwnNonIndexProperties(val2, ONLY_ENUMERABLE);
if (keys1.length !== keys2.length) {
Copy link
Copy Markdown
Member

@benjamingr benjamingr Aug 20, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unrelated optimization idea: would it make sense to return only the length from V8 rather than the properties themselves in the case of keys2?

Copy link
Copy Markdown
Member Author

@BridgeAR BridgeAR Aug 20, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I already spoke with @nodejs/v8 to implement a API that does exactly that. Until that exists, I would rather keep it as is.

Copy link
Copy Markdown
Member

@jdalton jdalton Aug 20, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The reason for needing getOwnNonIndexProperties should be brought up to the TC39 (if it hasn't already). It'd be good to let them know that Node is having to use engine level APIs to work around a JS lang limitation (no way to return a value with methods that return arrays of keys).

Copy link
Copy Markdown
Member Author

@BridgeAR BridgeAR Aug 20, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I already spoke to @littledan and I am writing the proposal at the moment. It's almost done.

return false;
}
return keyCheck(val1, val2, kStrict, memos, kIsArray, keys1);
Expand Down Expand Up @@ -150,8 +152,9 @@ function strictDeepEqual(val1, val2, memos) {
// Buffer.compare returns true, so val1.length === val2.length. If they both
// only contain numeric keys, we don't need to exam further than checking
// the symbols.
const keys1 = getOwnNonIndexProperties(val1);
if (keys1.length !== getOwnNonIndexProperties(val2).length) {
const keys1 = getOwnNonIndexProperties(val1, ONLY_ENUMERABLE);
const keys2 = getOwnNonIndexProperties(val2, ONLY_ENUMERABLE);
if (keys1.length !== keys2.length) {
return false;
}
return keyCheck(val1, val2, kStrict, memos, kNoIterator, keys1);
Expand Down
31 changes: 4 additions & 27 deletions lib/repl.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,6 @@ const {
isIdentifierChar
} = require('internal/deps/acorn/dist/acorn');
const internalUtil = require('internal/util');
const { isTypedArray } = require('internal/util/types');
const util = require('util');
const utilBinding = process.binding('util');
const { inherits } = util;
Expand All @@ -74,6 +73,7 @@ const {
const { sendInspectorCommand } = require('internal/util/inspector');
const { experimentalREPLAwait } = process.binding('config');
const { isRecoverableError } = require('internal/repl/recoverable');
const { getOwnNonIndexProperties } = process.binding('util');

// Lazy-loaded.
let processTopLevelAwait;
Expand Down Expand Up @@ -927,34 +927,11 @@ function isIdentifier(str) {
return true;
}

const ARRAY_LENGTH_THRESHOLD = 1e6;

function mayBeLargeObject(obj) {
if (Array.isArray(obj)) {
return obj.length > ARRAY_LENGTH_THRESHOLD ? ['length'] : null;
} else if (isTypedArray(obj)) {
return obj.length > ARRAY_LENGTH_THRESHOLD ? [] : null;
}

return null;
}

function filteredOwnPropertyNames(obj) {
if (!obj) return [];
const fakeProperties = mayBeLargeObject(obj);
if (fakeProperties !== null) {
this.outputStream.write('\r\n');
process.emitWarning(
'The current array, Buffer or TypedArray has too many entries. ' +
'Certain properties may be missing from completion output.',
'REPLWarning',
undefined,
undefined,
true);

return fakeProperties;
}
return Object.getOwnPropertyNames(obj).filter(isIdentifier);
const { kPropertyFilter } = internalUtil;
const filter = kPropertyFilter.ALL_PROPERTIES | kPropertyFilter.SKIP_SYMBOLS;
return getOwnNonIndexProperties(obj, filter).filter(isIdentifier);
}

function getGlobalLexicalScopeNames(contextId) {
Expand Down
13 changes: 8 additions & 5 deletions src/node_util.cc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,17 +21,20 @@ static void GetOwnNonIndexProperties(
Environment* env = Environment::GetCurrent(args);
Local<Context> context = env->context();

if (!args[0]->IsObject())
if (!args[0]->IsObject() || !args[1]->IsUint32())
Copy link
Copy Markdown
Member

@benjamingr benjamingr Aug 20, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wouldn't it make sense for this to be a CHECK since we should never end up here without said args?

(Not critical for this PR)

return;

v8::Local<v8::Object> object = args[0].As<v8::Object>();
Local<Object> object = args[0].As<Object>();

Local<Array> properties;

// Return only non-enumerable properties by default.
v8::Local<v8::Array> properties;
v8::PropertyFilter filter =
static_cast<v8::PropertyFilter>(
args[1]->Uint32Value(env->context()).FromJust());
Copy link
Copy Markdown
Member

@addaleax addaleax Aug 20, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since you already know at this point that args[1] is an Uint32 here, I’d suggest using args[1].As<Uint32>()->Value() (which is a static cast that fully compiles away, whereas Uint32Value() performs the equivalent of value|0 in JS, i.e. performs a real conversion that can potentially run userland code for non-integer input)


if (!object->GetPropertyNames(
context, v8::KeyCollectionMode::kOwnOnly,
v8::ONLY_ENUMERABLE,
filter,
v8::IndexFilter::kSkipIndices)
.ToLocal(&properties)) {
return;
Expand Down
19 changes: 4 additions & 15 deletions test/parallel/test-repl-tab-complete.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -393,12 +393,6 @@ testMe.complete('obj.', common.mustCall((error, data) => {
assert(data[0].includes('obj.key'));
}));

// tab completion for large buffer
const warningRegEx = new RegExp(
'\\(node:\\d+\\) REPLWarning: The current array, Buffer or TypedArray has ' +
'too many entries\\. Certain properties may be missing from completion ' +
'output\\.');

[
Array,
Buffer,
Expand Down Expand Up @@ -428,11 +422,7 @@ const warningRegEx = new RegExp(
putIn.run([`var ele = new ${type.name}(1e6 + 1); ele.biu = 1;`]);
}

common.hijackStderr(common.mustCall((err) => {
process.nextTick(() => {
assert.ok(warningRegEx.test(err));
});
}));
common.hijackStderr(common.mustNotCall());
testMe.complete('ele.', common.mustCall((err, data) => {
common.restoreStderr();
assert.ifError(err);
Expand All @@ -443,13 +433,12 @@ const warningRegEx = new RegExp(
Buffer.alloc(0) :
new type(0));

assert.strictEqual(data[0].includes('ele.biu'), true);
Copy link
Copy Markdown
Member

@benjamingr benjamingr Aug 20, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be great to have tests that specify the behaviour for long arrays better.

Copy link
Copy Markdown
Member Author

@BridgeAR BridgeAR Aug 20, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This test is specifically for long arrays. Since this value shows up, we can be certain that it works. I would not know what to improve. Do you have a suggestion?


data[0].forEach((key) => {
if (!key) return;
if (!key || key === 'ele.biu') return;
assert.notStrictEqual(ele[key.substr(4)], undefined);
});

// no `biu`
assert.strictEqual(data.includes('ele.biu'), false);
}));
});

Expand Down