Add SafeDict/SafeList pillar wrapping with Pydantic secrets, output redaction, and no_log#68907
Open
Akm0d wants to merge 53 commits intosaltstack:3008.xfrom
Open
Add SafeDict/SafeList pillar wrapping with Pydantic secrets, output redaction, and no_log#68907Akm0d wants to merge 53 commits intosaltstack:3008.xfrom
Akm0d wants to merge 53 commits intosaltstack:3008.xfrom
Conversation
…oken circular references
…egration tests pass locally
…tion regressions by relying on loader packs
…es, fixed integration regressions, and improved matcher fallback logic
…ressions, and linting issues
…and non-ext_pillar_first paths
If reap_stray_processes cannot kill a process, it logs which process it gived up on. However, if the process exits after the alive check and before the log, this causes psutil.NoSuchProcess exception
…inja iter_pillar_secret_literals collected every wrapped string, so public paths and identifiers became substring redaction targets and broke YAML rendering, diffs, and beacon/cron output. Gate literals on sensitive pillar key names. Unwrap pillar for Jinja render context and in json/yaml serializers so pillar|json and state templates see real values. Made-with: Cursor
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
This PR adds context-aware masking so sensitive pillar data is much harder to leak into state return payloads, minion logs, and event/log forwarding, while keeping explicit pillar APIs useful for debugging.
Highlights
pydantic>=2.4in base requirements;SecretStr/SecretBytesfor string/byte leaves.salt.utils.safepillar:SafeDict/SafeList,wrap_pillar_tree/unwrap_pillar_tree, literal collection, substring redaction,no_logmasking.no_log: trueon a state chunk (runtime keyword) masks that chunk’scommentandchangesregardless of whether the secret text appears in pillar literals.salt-callfrom thin keeps working (fixesModuleNotFoundError: pydantic/ follow-on import errors in CI).pillar.itemsandpillar.getreturn plain structures (unwrap_pillar_tree) so operators still see real values when they intentionally call those functions; in-memory pillar and accidental stringification stay protected.pillar.rstupdates.Fixes: #67367 (mask pillar in state output / logs).
Previous Behavior
{'name': 'curl https://api.example.com', 'comment': 'Request failed: invalid key sk-live-abc123', 'changes': {'stdout': 'error: sk-live-abc123'}, 'result': False}New Behavior
{'name': 'curl https://api.example.com', 'comment': 'Request failed: invalid key **********', 'changes': {'stdout': 'error: **********'}, 'result': False}2)
no_log: trueon a stateBefore:
commentandchangesare whatever the state module returned (may include secrets not present as pillar literals).After: both are masked to the placeholder shape:
3) In-memory pillar vs
pillar.items/pillar.getstr(pillar['db']['password'])→**********(safe if something logs or stringifies the object).pillar.items-style unwrap →{'db': {'password': 'plain-in-pillar'}}(plain values for intentional inspection).Merge requirements
pillar.rst)test_thin_dir)Reviewer notes
isinstance(x, str)for every pillar leaf may need to acceptSecretStror call.get_secret_value()after unwrap at boundaries.