Repository navigation
Line normalization before diff #219
Description
Description
The DiffRowGenerator class offers the lineNormalizer property. By default, it is used to replace < and > by their escaped versions < and >.
The lineNormalizer is applied to the input texts before the diff is calculated. While I see this is as a useful feature, in case of the default settings it might be surprising that the resulting text might not have HTML escaping anymore:
final var generator = DiffRowGenerator.create() //
.mergeOriginalRevised(true) //
.showInlineDiffs(true) //
.inlineDiffByWord(true) //
.build();
final var rows = generator.generateDiffRows(List.of("hello <world>"), List.of("bye >world<"));
final var resultingText = rows.stream() //
.map(DiffRow::getOldLine) //
.collect(Collectors.joining(StringUtils.LF));The resulting text is
<span class="editOldInline">hello</span><span class="editNewInline">bye</span> &<span class="editOldInline">lt</span><span class="editNewInline">gt</span>;world&<span class="editOldInline">gt</span><span class="editNewInline">lt</span>;
Note the part & is considered as an equal text part because both replacements < and > start with an ampersand. This resulting text is therefore no valid HTML anymore.
In order for this behaviour to be a problem, the following conditions must all be true:
- The
inlineDiffByWordmust be used - The default
lineNormalizermust be used - The two provided texts must differ at a position which starts with a character that is replaced by the
lineNormalizer - A release >= 4.15 must be used.
Workaround
Override the lineNormalizer. E.g., by using the SPLIT_BY_WORD_PATTERN of release 4.12, in which the ampersand was not considered a character that splits words.
Solution approaches
IMHO, the SPLIT_BY_WORD_PATTERN of release 4.15+ is fine and I do not consider it to be the problem.
The library could offer one of the following features:
- a parameter which defines when the 'lineNormalizer' should be applied (before diff-ing or after)
- a second type of line-normalizer that is applied after diff-ing
- an option to have the library apply the
processDiffsfunction to non-diffs as well
Activity
epictecch commentedon Oct 14, 2025
@wumpz Do you have any feedback/opinion on this topic? When you tell me which solution is the best fit for this library, I can submit a pull request.
IMHO, my option 3 is the one that covers most use-cases and does not introduce new problems. Instead of adding an additional parameter to
processDiffs()(whether the passed tag is a diff or an equality), we could add a new (-> backward compatible) feature: we could let theprocessDiffs()method remain as it is and we introduce a newprocessEqualities()method that defaults to not changing the tag.wumpz commentedon Nov 5, 2025
Seems as if processDiffs should be called anyway for Equal parts.
Option 1 seems to be kind of there, since normalize is called before and after. Only the option is missing.
Option 2: Do anyone need different normalizers (before, after). I don't think so.
So frankly I am with you for option 3.
feat(diffrow): add processEqualities hook and ensure equalities are p…
tusharsoni52 commentedon Nov 29, 2025
Hi @epictecch ,
Thank you for detailing the problem and the solution options.
Since @wumpz confirmed option 3, I implemented a PR following that design (#224).
I’d really appreciate your review or feedback
Happy to make any improvements.
Thanks again for raising this issue and guiding the solution direction!
feat(diffrow): add processEqualities hook and ensure equalities are p…