Skip to content

Improve missing-attribute suggestions: Jaro-Winkler similarity, gated containment - #147

Merged
jhonabreul merged 4 commits into
QuantConnect:masterfrom
jhonabreul:bug-attribute-suggestion-quality
Aug 12, 2026
Merged

Improve missing-attribute suggestions: Jaro-Winkler similarity, gated containment#147
jhonabreul merged 4 commits into
QuantConnect:masterfrom
jhonabreul:bug-attribute-suggestion-quality

Conversation

@jhonabreul

@jhonabreul jhonabreul commented Aug 11, 2026

Copy link
Copy Markdown
Collaborator

What does this implement/fix? Explain your changes.

Improves the quality of the "Did you mean" hints on missing-attribute errors, fixing two defects in how similar member names are selected:

  1. 1–2 letter members flooded the hint for long missed names. The substring-containment test admitted any member that appears inside the missed name, and types like QCAlgorithm expose many 1–2 letter members (indicator shortcuts, greek letters), every one of which is a substring of a long name:

    'QCAlgorithm' object has no attribute 'set_account_type' Did you mean: 'cc', 'co', 'a', 'c', 't'?
    

    Containment now only counts for fragments of at least 3 characters, and a candidate contained in the missed name must additionally cover at least half of it.

  2. Suffix-extended members were never suggested. The Levenshtein threshold max(2, len/3) rejected members whose real name extends the guess with a suffix, so the member the user most likely meant could be missing from its own hint:

    type object 'BrokerageName' has no attribute 'InteractiveBrokers' Did you mean: 'INTERACTIVE_BROKERS_FIX'?
    

    (INTERACTIVE_BROKERS_BROKERAGE — 11 edits away — was not suggested.) The edit-distance threshold is replaced with Jaro-Winkler similarity (threshold 0.87), a prefix-favoring measure that keeps suffix extensions and, as a bonus, rejects the junk matches short names produced under the distance cutoff (e.g. PII no longer suggests 'min', 'pow', 'sin' alongside 'PI').

Both fleet-reported cases now produce the intended hints against real Lean types:

'QCAlgorithm' object has no attribute 'set_account_type' Did you mean: 'set_account_currency'?
type object 'BrokerageName' has no attribute 'InteractiveBrokers' Did you mean: 'INTERACTIVE_BROKERS_FIX', 'INTERACTIVE_BROKERS_BROKERAGE'?

Algorithm selection. Two candidate replacements were evaluated over real member pools (590 QCAlgorithm members, BrokerageName, System.String, System.Math, DayOfWeek, plus the existing test fixtures): a token-based clause (split on _/camel humps, relate on shared tokens, keep Levenshtein for typos) and Jaro-Winkler. Both pass all 13 graded cases; Jaro-Winkler was chosen because it replaces the edit-distance clause instead of augmenting it, so it also fixes the short-name junk the token variant inherits, with less code:

Missed name Before Token-based Jaro-Winkler (chosen)
QCAlgorithm.set_account_type cc, co, a, c, t set_account_currency set_account_currency
BrokerageName.InteractiveBrokers INTERACTIVE_BROKERS_FIX both IB members both IB members
BrokerageName.INTERACTIVE_BROKERS both IB members both both
String.lenght length length length
Math.PII PI, min, pow, sin PI, min, pow, sin PI
String.Empy EMPTY, copy EMPTY, copy EMPTY
QCAlgorithm.cash ASI, asi, CKS, cks, CRSI ASI, asi, CKS, cks, CRSI set_cash
QCAlgorithm.market_ordr market_order, AR, ar, A, a market_order market_order, market_on_open_order, …

Threshold margins at 0.87: all intended targets score ≥ 0.90; the trimmed noise sits ≤ 0.851. Substring containment is kept (gated) as a fallback for fragment lookups Jaro-Winkler's match window cannot see (cashset_cash), scored by coverage so such matches always rank below similarity matches. The per-type candidate cache, per-(type, name) hint memoization, kind filtering and the 5-suggestion cap are unchanged.

Does this close any currently open issues?

No open issue in this repo; addresses the enum/member did-you-mean defects from QuantConnect/Agents#305 (improvement 1).

Any other comments?

Tests: py -3.11 -m pytest --runtime netcore tests → 459 passed, 1 failed (test_module.py::test_explicit_assembly_load, pre-existing environmental failure, reproduced on unmodified master), 23 skipped. dotnet test src/embed_tests/Python.EmbeddingTest.csproj -c Release --filter "FullyQualifiedName!~SetPythonPath" → 971 passed, 0 failed.

Note: #144 moves LevenshteinDistance into the same Util.cs this PR adds the Jaro-Winkler helpers to (and this PR removes that method's ClassBase copy), so whichever lands second has a trivial both-append merge in Util.cs and a deletion conflict in ClassBase.cs.

Checklist

Check all those that are applicable and complete.

  • Make sure to include one or more tests for your change
  • If an enhancement PR, please create docs and at best an example
  • Ensure you have signed the .NET Foundation CLA
  • Add yourself to AUTHORS
  • Updated the CHANGELOG

@jhonabreul
jhonabreul marked this pull request as ready for review August 11, 2026 17:51
Keeps the Jaro-Winkler scoring from this branch in ClassBase while
retaining Util.LevenshteinDistance, which MethodBinder's unexpected-kwarg
suggestion (QuantConnect#144) still uses.

Also moves the DiagnoseClosestOverloadMismatch block inside the
bind-failure try so master compiles again: QuantConnect#144 wrapped the message
construction (including the candidates declaration) in try/catch after
QuantConnect#145 had added the mismatch diagnosis below it, leaving 'candidates' out
of scope at its use site.
@jhonabreul
jhonabreul merged commit e907acd into QuantConnect:master Aug 12, 2026
9 checks passed
@jhonabreul
jhonabreul deleted the bug-attribute-suggestion-quality branch August 12, 2026 20:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants