Message.text_html / text_markdown_v2 duplicate characters when two entities partially overlap #5341

Summary

Message._parse_html (and the identical logic in _parse_markdown) handles nested entities specially, but partially overlapping siblings are neither nested nor disjoint: each gets emitted in full, and the inter-entity gap slice degenerates to an empty byte range with start > stop. The reconstructed HTML then renders duplicated text.

Static-analysis finding based on reading master; not executed here.

Location

• File: src/telegram/_message.py
• Function: Message._parse_html (same pattern in _parse_markdown)
• Relevant code:

nested_entities = {
    e: t
    for (e, t) in sorted_entities
    if e.offset >= entity.offset
    and e.offset + e.length <= entity.offset + entity.length
    and e != entity
}
...
html_text += (
    escape(utf_16_text[last_offset * 2 : (entity.offset - offset) * 2]
           .decode(TextEncoding.UTF_16_LE))
    + insert
)
last_offset = entity.offset - offset + entity.length

Problem

For text abcdefgh (UTF-16 byte length 16) with entities BOLD(offset=0, length=5) ("abcde") and ITALIC(offset=3, length=5) ("defgh"):

1. Sorted order: [BOLD, ITALIC].
2. BOLD: ITALIC fails the nesting predicate (3+5 <= 0+5 is false), so BOLD emits <b>abcde</b> and last_offset = 0 + 5.
3. ITALIC: BOLD fails nesting (0 >= 3 false). Gap slice becomes utf_16_text[10:6] - Python slicing with start > stop yields b"" silently instead of raising, so no gap text is emitted.
4. ITALIC emits <i>defgh</i> in full.

Concatenated output: <b>abcde</b><i>defgh</i>, whose visible text is abcdedefgh (10 chars) - de appears twice; the original 8-char text is not recoverable from the parse.

Such overlapping entities are constructible by any bot: Bot API accepts arbitrary entities lists on send/copy methods, and PTB Message objects built from such responses flow into text_html.

Trigger / Reproduction

Based on source reading:

msg = Message(..., text="abcdefgh", entities=[MessageEntity(type='bold', offset=0, length=5),
                                              MessageEntity(type='italic', offset=3, length=5)])
msg.text_html  # '<b>abcde</b><i>defgh</i>' -> 'abcdedefgh'

Expected: HTML whose decoded text equals abcdefgh (overlap split between tags, e.g. <b>abc<i>d</i></b><i>defgh</i>).

Impact

Silent corruption of formatted-text round-trips (text_html*, text_markdown_v2*, caption variants): characters are duplicated (or, with other overlap geometries, skipped when last_offset overshoots). Downstream consumers that re-parse or re-send the produced markup propagate wrong message content without any error.

Suggested Direction

Treat non-nested overlaps explicitly: clamp the next entity's start against last_offset (skip already-emitted prefix and close/reopen the previous tag), or reject/split overlapping-but-not-nested entity pairs before emission. The same fix applies to _parse_markdown, which duplicates this code path.

Evidence

• The nesting predicate requires full containment; partial overlaps fall through both special cases.
• The gap slice uses raw UTF-16 offsets; when last_offset > entity.offset - offset, Python returns an empty bytes object rather than failing, so the duplication goes unnoticed internally.