Skip to content

fix(confluence): preserve panel/callout macro semantics through sync#5896

Merged
waleedlatif1 merged 7 commits into
stagingfrom
fix/confluence-panel-callout-fidelity
Jul 23, 2026
Merged

fix(confluence): preserve panel/callout macro semantics through sync#5896
waleedlatif1 merged 7 commits into
stagingfrom
fix/confluence-panel-callout-fidelity

Conversation

@waleedlatif1

Copy link
Copy Markdown
Collaborator

Summary

  • Confluence's rendered view HTML wraps Info/Note/Warning/Tip and custom Panel macros in divs whose class/color convey meaning — the shared htmlToPlainText tag-stripper discards that along with the tags, so a red "do not use" warning panel becomes indistinguishable from a plain paragraph once flattened
  • Reported production impact: Ask IT (all of RVTech) misrouted a GitLab access request because the KB's exclusion callout ("Do NOT use this form for: GitLab...") lost its warning semantics after sync, so RAG had no signal it was a negative rule
  • Adds preserveConfluenceCallouts, a Confluence-specific pre-pass that rewrites each detected panel into a single bracketed label (e.g. [WARNING] Do NOT use this form for: GitLab) before the generic plain-text conversion runs, so the callout marker survives both the tag strip and the trailing whitespace collapse
  • Handles both markup shapes Confluence's view format emits: built-in Info/Note/Warning/Tip macros (.confluence-information-macro-{type}) and the customizable Panel macro (.panel / .panelHeader / .panelContent)
  • Bumps the connector's content-representation marker so already-synced pages get one automatic re-hydration under the new extraction, instead of silently keeping stale flattened content until their next edit

Type of Change

  • Bug fix

Testing

  • Added preserveConfluenceCallouts unit tests covering: built-in info/note/warning/tip macros, custom Panel macro with/without a header title, empty input, no-macro passthrough, and an end-to-end case through htmlToPlainText proving the label stays attached to its content across whitespace collapse
  • Full connector/knowledge-base test suite passes (529 tests)
  • bun run check:api-validation passes

Checklist

  • Code follows project style guidelines
  • Self-reviewed my changes
  • Tests added/updated and passing
  • No new warnings introduced
  • I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

@vercel

vercel Bot commented Jul 23, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated (UTC)
docs Skipped Skipped Jul 23, 2026 9:01pm

Request Review

@cursor

cursor Bot commented Jul 23, 2026

Copy link
Copy Markdown

PR Summary

Medium Risk
Changes how all Confluence page bodies are extracted for RAG (content quality and chunk boundaries), but scope is limited to the connector with extensive tests and a one-time re-hydration via contentHash.

Overview
Confluence sync now runs rendered page HTML through preserveConfluenceCallouts before htmlToPlainText, so Info/Note/Warning/Tip and custom Panel macros become explicit bracketed labels (e.g. [WARNING] …, [CALLOUT: title] …) instead of losing their meaning when tags and classes are stripped.

The preprocessor uses Cheerio with a custom DOM walk that separates block boundaries from inline formatting (so lists, paragraphs, and nested panels keep readable spacing without breaking words like un<b>believe</b>able). Nested panels/macros are rewritten innermost-first so each level keeps its own label.

The connector contentHash representation marker is bumped from view to view-callouts so unchanged pages are re-fetched once and pick up the new indexed text. Broad unit tests cover macro types, nesting, and the full path through htmlToPlainText.

Reviewed by Cursor Bugbot for commit f44c85c. Configure here.

@greptile-apps

greptile-apps Bot commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

Preserves Confluence callout semantics during knowledge synchronization.

  • Rewrites built-in information macros and custom panels into labeled plain text before generic HTML stripping.
  • Preserves structural and inline text boundaries, including nested callouts.
  • Changes the content-representation marker to rehydrate previously synchronized pages.
  • Adds regression coverage for labels, nesting, spacing, and line breaks.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failures remain.

Important Files Changed

Filename Overview
apps/sim/connectors/confluence/confluence.ts Adds callout-aware HTML preprocessing, integrates it into Confluence document hydration, and advances the representation marker.
apps/sim/connectors/confluence/confluence.test.ts Adds comprehensive regression tests for built-in and custom callouts, nested markup, and text-boundary preservation.

Reviews (7): Last reviewed commit: "test(confluence): add explicit regressio..." | Re-trigger Greptile

Comment thread apps/sim/connectors/confluence/confluence.ts Outdated
Comment thread apps/sim/connectors/confluence/confluence.ts Outdated
@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@greptile

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@cursor review

Comment thread apps/sim/connectors/confluence/confluence.ts
Comment thread apps/sim/connectors/confluence/confluence.ts Outdated
@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@greptile

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@cursor review

Comment thread apps/sim/connectors/confluence/confluence.ts Outdated
Comment thread apps/sim/connectors/confluence/confluence.ts Outdated
@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@greptile

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@cursor review

Comment thread apps/sim/connectors/confluence/confluence.ts

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit 6ad1a38. Configure here.

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@greptile

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@cursor review

Comment thread apps/sim/connectors/confluence/confluence.ts Outdated
@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@greptile

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@cursor review

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit 7e2115c. Configure here.

Comment thread apps/sim/connectors/confluence/confluence.ts
Confluence's rendered view HTML wraps Info/Note/Warning/Tip and custom Panel
macros in divs whose class/color convey meaning that the shared
htmlToPlainText tag-stripper discards along with the tags — a red "do not
use" warning panel becomes indistinguishable from a plain paragraph once
flattened, so RAG has no signal that a bullet under it is an exclusion rule
rather than a normal one.

Adds preserveConfluenceCallouts, a Confluence-specific pre-pass that rewrites
each detected panel into a single bracketed label (e.g. "[WARNING] Do NOT use
this form for: GitLab") before the generic plain-text conversion runs, so the
callout semantic survives both the tag strip and htmlToPlainText's trailing
whitespace collapse. Bumps the connector's content-representation marker so
already-synced pages get one automatic re-hydration under the new extraction,
rather than silently keeping their stale flattened content until their next
edit.
…y text

Greptile P1: cheerio's .text() concatenates every descendant text node with
no separator, so pulling a macro body's text in one call fused adjacent
blocks together (e.g. a paragraph ending in "for:" immediately followed by a
list item "GitLab" became "for:GitLab"), corrupting the exact word boundaries
RAG chunking and keyword matching depend on.

extractBlockJoinedText now extracts each paragraph/list-item/heading/cell/quote
individually and joins them with a single space, keeping every block's text
intact and properly separated, matching how htmlToPlainText already treats
the rest of the page.
Greptile P1: filtering the found blocks to only top-level ones still wasn't
enough — a nested block (an outer <li> containing its own nested <ul><li>, a
<td> containing a <blockquote>) matched the selector once, but its .text()
call recurses into and flattens its own matched descendants with no
separator, reproducing the exact word-fusion bug one level deeper (and any
duplicate-selection would have double-counted the same text).

Replaces the block-selector approach with a recursive text-node walk:
extractBlockJoinedText now visits every text node individually and joins them
all with a single space, so word boundaries are preserved at every nesting
depth with no double-counting, matching the pattern html-parser.ts already
uses elsewhere in this codebase for the same class of problem.
…l headers

Greptile: panelHeader text extraction was left on the plain .text() call
while panel/macro body extraction was already fixed to use
extractBlockJoinedText, so a rich multi-node header (e.g. <b>Warning:</b>
followed by a sibling <span>) could still fuse into "Warning:Do not use"
with no space. Panel headers now go through the same recursive text-node
walk as bodies, for consistency across every text extraction in this file.
…in extraction

Greptile: the recursive text-node walk unconditionally inserted a space
between every text node, which fixed block-boundary fusion but broke
genuinely inline-formatted text — "un<b>believe</b>able" became
"un believe able" and "Hello<b>!</b>" became "Hello !", corrupting valid
callout content on its way into the index.

Adds an INLINE_FORMATTING_TAGS allowlist (b, strong, i, em, span, a, etc.):
text flowing through those tags accumulates with no artificial separator,
preserving exact source adjacency, while every other tag boundary (p, li,
td, headings, br, ...) still flushes to a new segment — a block always
implies a break even with no literal whitespace in the source, but an inline
tag never does. Fixed one test that had encoded the old, incorrect
expectation for two genuinely adjacent inline tags with no source whitespace
between them, and added regression tests for mid-word inline formatting,
punctuation attached to an inline tag, and a header with real source spacing.
Cursor: processing matches in document order (outermost first) read a
nested, not-yet-converted panel/macro as plain body text before it ever got
its own bracketed label, silently dropping the inner callout's type. Worse,
an untitled outer panel's `.find('.panelHeader')` could reach past its own
missing header into a nested panel's header and adopt it as its own title.

Replaces the two independent .each() passes with a loop that converts only
"leaf" macros (no remaining nested macro/panel inside them) and repeats
until none are left. This processes innermost-first, so a nested macro is
already its own bracketed <p> by the time its parent's body/header text is
read, and an untitled outer panel's .find() can no longer reach a header
that isn't its own, since a leaf by definition has no nested panel left to
reach into.
… repro

Formalizes an explicit test for the exact <br>-separated string Greptile's
review cited as broken (verified manually not to reproduce, but wasn't
directly asserted in the suite before this).
@waleedlatif1
waleedlatif1 force-pushed the fix/confluence-panel-callout-fidelity branch from 7e2115c to f44c85c Compare July 23, 2026 21:01
@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@greptile

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@cursor review

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit f44c85c. Configure here.

@waleedlatif1
waleedlatif1 merged commit 5c42779 into staging Jul 23, 2026
14 checks passed
@waleedlatif1
waleedlatif1 deleted the fix/confluence-panel-callout-fidelity branch July 23, 2026 21:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant