Skip to content

feat(pii): gate data retention PII redaction behind feature flag#5144

Merged
TheodoreSpeaks merged 2 commits into
stagingfrom
feat/ff-pii
Jun 20, 2026
Merged

feat(pii): gate data retention PII redaction behind feature flag#5144
TheodoreSpeaks merged 2 commits into
stagingfrom
feat/ff-pii

Conversation

@TheodoreSpeaks

Copy link
Copy Markdown
Collaborator

Summary

  • Add a runtime pii-redaction feature flag (AppConfig-gated by org/user/admin on prod, PII_REDACTION secret fallback off-prod) gating the Data Retention PII redaction feature end-to-end.
  • Backend (persist choke point): logger.applyPiiRedaction short-circuits to the unredacted payload when the flag is off for the workspace's org, before resolving rules — covers all persist paths.
  • Backend (config write): PUT /api/organizations/[id]/data-retention returns 403 if a caller sets piiRedaction while the flag is off; GET/PUT now return server-resolved piiRedactionEnabled.
  • UI: the PII Redaction section in Data Retention settings only renders when piiRedactionEnabled (resolved server-side, piped through the existing GET response). Gated just that section, not the whole tab, since the tab also owns log-retention / soft-delete / task-cleanup.

Type of Change

  • New feature (flag gating)

Testing

  • bun run lint — clean
  • bun run check:api-validation:strict — passed
  • bun run type-check — clean
  • feature-flags.test.ts — 13 passed (added pii-redaction registry assertion)

Checklist

  • Code follows project style guidelines
  • Self-reviewed my changes
  • Tests added/updated and passing
  • No new warnings introduced
  • I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

@vercel

vercel Bot commented Jun 20, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated (UTC)
docs Skipped Skipped Jun 20, 2026 2:12am

Request Review

@TheodoreSpeaks

Copy link
Copy Markdown
Collaborator Author

@greptile review

@cursor

cursor Bot commented Jun 20, 2026

Copy link
Copy Markdown

PR Summary

Medium Risk
Changes the execution log persist choke point and data-retention API around PII handling; when the flag is off, logs stay unredacted even if org rules exist—intentional but security-sensitive for rollout.

Overview
Introduces a runtime pii-redaction feature flag (AppConfig with PII_REDACTION env fallback) so PII redaction can be rolled out or turned off globally without diverging behavior between persist and settings.

Persist path: applyPiiRedaction in the execution logger returns the payload unchanged when the flag is off, before org rules are resolved—so no Presidio masking runs on any log write path.

Config API: GET and PUT on organization data retention include server-resolved piiRedactionEnabled. PUT rejects piiRedaction body updates with 403 when the flag is off.

UI: The PII Redaction block in Data Retention settings renders only when piiRedactionEnabled is true; log retention and other sections are unchanged.

Contract schema and feature-flag tests were updated for the new flag and response field.

Reviewed by Cursor Bugbot for commit ebdd773. Bugbot is set up for automated code reviews on this repo. Configure here.

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit f8ca77c. Configure here.

Comment thread apps/sim/app/api/organizations/[id]/data-retention/route.ts Outdated
@greptile-apps

greptile-apps Bot commented Jun 20, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR gates the Data Retention PII redaction feature behind a pii-redaction feature flag, evaluating it globally (no org/user context) so the logger persist path and the config API routes always agree on the flag's state.

  • Adds the pii-redaction registry entry (PII_REDACTION env fallback) and the PII_REDACTION env var declaration; the flag description explicitly documents the global-only evaluation constraint.
  • Both the GET and PUT /api/organizations/[id]/data-retention route handlers now resolve piiRedactionEnabled without org context and return it in the response; the PUT blocks piiRedaction writes with a 403 when the flag is off.
  • ExecutionLogger.applyPiiRedaction short-circuits before any DB I/O when the flag is off, and the PII Redaction section in the settings UI is hidden when data.piiRedactionEnabled is false.

Confidence Score: 5/5

Safe to merge — the flag is evaluated consistently without context across all three paths (logger, GET, PUT), and all changes are additive with a clean default-off fallback.

The logger, GET route, and PUT route all call isFeatureEnabled with no context, so the flag state is always in sync. The fail-safe direction (flag off → no redaction skipped, UI hidden, writes blocked) is correct for a PII feature. No schema migrations, no new DB writes outside the existing piiRedaction field, and the existing enterprise billing gate is unchanged.

No files require special attention. The route.ts 403 error message wording is a minor polish item.

Important Files Changed

Filename Overview
apps/sim/lib/core/config/feature-flags.ts Adds 'pii-redaction' registry entry with PII_REDACTION env fallback; description explicitly documents global-only evaluation to keep logger and routes consistent
apps/sim/lib/core/config/env.ts Adds PII_REDACTION boolean env var declaration with clear inline comment
apps/sim/app/api/organizations/[id]/data-retention/route.ts GET returns piiRedactionEnabled; PUT blocks piiRedaction writes with 403 when flag is off — but the 403 error message says 'for this organization' despite the flag being evaluated globally (no org context passed)
apps/sim/lib/logs/execution/logger.ts Adds global-only isFeatureEnabled guard as early-exit before DB lookups; consistently evaluated without context, matching route handlers
apps/sim/lib/api/contracts/organization.ts Adds required piiRedactionEnabled: z.boolean() field to the data-retention response schema for both GET and PUT contracts
apps/sim/ee/data-retention/components/data-retention-settings.tsx Wraps the PII Redaction settings section in a data?.piiRedactionEnabled guard; rest of the tab (log retention, soft-delete, task cleanup) is unaffected
apps/sim/lib/core/config/feature-flags.test.ts Adds pii-redaction registry assertions to two existing test scenarios (no AppConfig, malformed AppConfig); confirms default-off behavior

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[Request: GET /data-retention] --> B[isFeatureEnabled 'pii-redaction'\nno context — global only]
    B -->|false| C[Return response\npiiRedactionEnabled: false]
    B -->|true| D[Return response\npiiRedactionEnabled: true]
    C --> E[UI hides PII Redaction section]
    D --> F[UI renders PII Redaction section]

    G[Request: PUT /data-retention] --> H{body.piiRedaction\npresent?}
    H -->|no| I[Update other settings normally]
    H -->|yes| J[isFeatureEnabled 'pii-redaction'\nno context — global only]
    J -->|false| K[Return 403]
    J -->|true| L[Merge piiRedaction rules into DB]

    M[ExecutionLogger.applyPiiRedaction] --> N{workspaceId present?}
    N -->|no| O[Return payload unchanged]
    N -->|yes| P[isFeatureEnabled 'pii-redaction'\nno context — global only]
    P -->|false| Q[Return payload unchanged\nskip all DB lookups]
    P -->|true| R[Resolve effective PII rules\nfrom org/workspace settings]
    R --> S{Rules enabled?}
    S -->|no| T[Return payload unchanged]
    S -->|yes| U[redactPIIFromExecution]
Loading
%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
flowchart TD
    A[Request: GET /data-retention] --> B[isFeatureEnabled 'pii-redaction'\nno context — global only]
    B -->|false| C[Return response\npiiRedactionEnabled: false]
    B -->|true| D[Return response\npiiRedactionEnabled: true]
    C --> E[UI hides PII Redaction section]
    D --> F[UI renders PII Redaction section]

    G[Request: PUT /data-retention] --> H{body.piiRedaction\npresent?}
    H -->|no| I[Update other settings normally]
    H -->|yes| J[isFeatureEnabled 'pii-redaction'\nno context — global only]
    J -->|false| K[Return 403]
    J -->|true| L[Merge piiRedaction rules into DB]

    M[ExecutionLogger.applyPiiRedaction] --> N{workspaceId present?}
    N -->|no| O[Return payload unchanged]
    N -->|yes| P[isFeatureEnabled 'pii-redaction'\nno context — global only]
    P -->|false| Q[Return payload unchanged\nskip all DB lookups]
    P -->|true| R[Resolve effective PII rules\nfrom org/workspace settings]
    R --> S{Rules enabled?}
    S -->|no| T[Return payload unchanged]
    S -->|yes| U[redactPIIFromExecution]
Loading

Reviews (3): Last reviewed commit: "fix(pii): evaluate pii-redaction flag gl..." | Re-trigger Greptile

Comment thread apps/sim/lib/logs/execution/logger.ts Outdated
@greptile-apps

greptile-apps Bot commented Jun 20, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR gates the existing Data Retention PII redaction feature behind a pii-redaction feature flag, evaluated via AWS AppConfig on hosted deployments and a PII_REDACTION env var fallback elsewhere. The flag is checked at three enforcement points: the logger persist choke point, the PUT config write, and the settings UI.

  • logger.ts: Early-return in applyPiiRedaction when the flag is off for the workspace's org; also extends the DB select to include orgId for the flag context.
  • route.ts (GET + PUT): GET resolves and pipes piiRedactionEnabled through the response; PUT returns 403 if a caller sets piiRedaction while the flag is off.
  • data-retention-settings.tsx: PII Redaction section renders only when data.piiRedactionEnabled is true; the rest of the Data Retention tab (log retention, soft-delete, task cleanup) is unaffected.

Confidence Score: 4/5

Safe to merge for org-level flag targeting; a gap exists if the AppConfig document targets this flag by userId or adminEnabled rather than orgId.

The logger evaluates the flag with only orgId — no userId is available at execution persist time. If the AppConfig document is ever configured with userIds or adminEnabled for this flag, those users would see the PII section in the UI and successfully save rules via PUT, but the logger would silently skip redaction at persist time. This is an inherent design limitation given the lack of user context in background processing, but it is currently undocumented in the flag description, making misconfiguration easy. In practice the PR intends org-level rollout and the env-var fallback is org-agnostic, so day-one deployments are unaffected.

apps/sim/lib/logs/execution/logger.ts — the isFeatureEnabled call should document that only orgId context is available here, and the flag should not be targeted by userIds or adminEnabled in AppConfig.

Important Files Changed

Filename Overview
apps/sim/lib/logs/execution/logger.ts Adds feature flag short-circuit at the top of applyPiiRedaction; also now selects orgId from the DB row to pass to the flag evaluator. The flag context omits userId, creating a gap when the flag is configured with userId/admin targeting in AppConfig.
apps/sim/app/api/organizations/[id]/data-retention/route.ts GET now resolves and exposes piiRedactionEnabled in the response; PUT returns 403 when a caller attempts to set piiRedaction while the flag is off. Both use full userId+orgId context, which is correct at request time.
apps/sim/lib/core/config/feature-flags.ts Adds pii-redaction to the FEATURE_FLAGS registry with a PII_REDACTION env fallback. Straightforward registry addition; no issues.
apps/sim/lib/api/contracts/organization.ts Adds piiRedactionEnabled: z.boolean() to the shared data retention response schema used by both GET and PUT contracts. Clean and complete.
apps/sim/ee/data-retention/components/data-retention-settings.tsx Wraps the PII Redaction SettingsSection in {data?.piiRedactionEnabled && ...}. The optional-chain on data is redundant given the earlier !data guard at line 432, but harmless.
apps/sim/lib/core/config/feature-flags.test.ts Adds pii-redaction assertions to both existing non-AppConfig test cases. Tests are accurate and complete.

Sequence Diagram

%%{init: {'theme': 'neutral'}}%%
sequenceDiagram
    participant UI as DataRetentionSettings (UI)
    participant GET as GET /data-retention
    participant PUT as PUT /data-retention
    participant Logger as ExecutionLogger
    participant FF as isFeatureEnabled
    participant AppConfig as AppConfig / ENV

    UI->>GET: fetch settings (userId + orgId)
    GET->>FF: "isFeatureEnabled('pii-redaction', {userId, orgId})"
    FF->>AppConfig: getFeatureFlags() (~30s TTL)
    AppConfig-->>FF: FeatureFlagsConfig
    FF-->>GET: piiRedactionEnabled: boolean
    GET-->>UI: "{..., piiRedactionEnabled}"
    note over UI: Renders PII section only if piiRedactionEnabled

    UI->>PUT: save piiRedaction rules (userId + orgId)
    PUT->>FF: "isFeatureEnabled('pii-redaction', {userId, orgId})"
    FF-->>PUT: piiRedactionEnabled
    alt flag OFF
        PUT-->>UI: 403 PII redaction not enabled
    else flag ON
        PUT-->>UI: 200 updated settings + piiRedactionEnabled
    end

    note over Logger: Execution persist path (no user context)
    Logger->>FF: "isFeatureEnabled('pii-redaction', {orgId})"
    FF-->>Logger: flagEnabled
    alt flag OFF
        Logger-->>Logger: return unredacted payload
    else flag ON
        Logger->>Logger: resolveEffectivePiiRedaction
        Logger->>Logger: redactPIIFromExecution
    end
Loading
%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
sequenceDiagram
    participant UI as DataRetentionSettings (UI)
    participant GET as GET /data-retention
    participant PUT as PUT /data-retention
    participant Logger as ExecutionLogger
    participant FF as isFeatureEnabled
    participant AppConfig as AppConfig / ENV

    UI->>GET: fetch settings (userId + orgId)
    GET->>FF: "isFeatureEnabled('pii-redaction', {userId, orgId})"
    FF->>AppConfig: getFeatureFlags() (~30s TTL)
    AppConfig-->>FF: FeatureFlagsConfig
    FF-->>GET: piiRedactionEnabled: boolean
    GET-->>UI: "{..., piiRedactionEnabled}"
    note over UI: Renders PII section only if piiRedactionEnabled

    UI->>PUT: save piiRedaction rules (userId + orgId)
    PUT->>FF: "isFeatureEnabled('pii-redaction', {userId, orgId})"
    FF-->>PUT: piiRedactionEnabled
    alt flag OFF
        PUT-->>UI: 403 PII redaction not enabled
    else flag ON
        PUT-->>UI: 200 updated settings + piiRedactionEnabled
    end

    note over Logger: Execution persist path (no user context)
    Logger->>FF: "isFeatureEnabled('pii-redaction', {orgId})"
    FF-->>Logger: flagEnabled
    alt flag OFF
        Logger-->>Logger: return unredacted payload
    else flag ON
        Logger->>Logger: resolveEffectivePiiRedaction
        Logger->>Logger: redactPIIFromExecution
    end
Loading

Reviews (2): Last reviewed commit: "feat(pii): gate data retention PII redac..." | Re-trigger Greptile

Comment thread apps/sim/lib/logs/execution/logger.ts Outdated
@TheodoreSpeaks

Copy link
Copy Markdown
Collaborator Author

@greptile review

@TheodoreSpeaks TheodoreSpeaks merged commit 3b78436 into staging Jun 20, 2026
16 checks passed
@TheodoreSpeaks TheodoreSpeaks deleted the feat/ff-pii branch June 20, 2026 02:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant