Repository navigation
Conversation
PgServerMock can now play a walsender: CopyBothResponse, CopyData, XLogData and keepalive messages, CopyDone, CopyOutResponse and NoticeResponse writers, NULL DataRow columns, a configurable server_version, the parsed startup parameters (e.g. "replication") and Expect helpers for CopyDone, Terminate and standby status updates. This also fixes length bugs that were hidden because the client parses these messages by content (CopyInResponse, ParameterDescription and ErrorResponse sent wrong lengths), and ExpectMessage skipped synchronously without ensuring the message was fully received. ReplicationMock opens replication connections against the mock with the feedback timers disabled, and ReplicationMockTests smoke-tests startup, IDENTIFY_SYSTEM, physical streaming, keepalive replies and cancellation, with timeouts on both the client and the mock side. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ections
- TimelineHistory threw an InvalidCastException on PostgreSQL 14 and
later, which declare the content column as text instead of bytea. The
column carries the raw bytes of the history file either way, so it is
now read as raw bytes. That also stops bytea unescaping from mangling
backslashes in the content on older servers, and leaves the bytea
parsing without a use, so it is removed.
- Show("all") returns many rows, which made the command helper expect
CommandComplete, get another DataRow and break the connection. Further
rows are now skipped, and Show throws NotSupportedException for results
that aren't a single value, leaving the connection usable.
- The ReplicationConnection.CommandTimeout property was ignored for
replication commands, since every command reset the timeouts to the
connection string's command timeout. It now applies to all commands,
and setting it before Open no longer throws. Timeouts that don't fit
into an int of milliseconds (e.g. TimeSpan.MaxValue) mean no timeout.
- StartReplication(PhysicalReplicationSlot, CancellationToken) throws
ArgumentNullException for a null slot instead of a
NullReferenceException.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
PgTestCluster creates PostgreSQL clusters in temporary directories with the binaries that NPGSQL_TEST_PG_BIN points to (initdb, pg_ctl, pg_basebackup): a primary that allows physical and logical replication, streaming standbys of it, promotion and shutdown in any mode. This covers tests that can't run against the shared test server, like timeline switches and server shutdowns. Tests using it are ignored when the variable isn't set, so CI is unaffected. The clusters listen on 127.0.0.1 only (no Unix sockets, whose default directory isn't writable for unprivileged users on Debian-based builds), pick a free port right before each start, retry when another process took it, and limit every step to well below CI's 30 seconds per test. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- At the end of a historic timeline (e.g. after the standby we stream
from got promoted), the server ends copy mode with CopyDone. Npgsql
expected only CopyData and broke the connection ("Received backend
message CopyDone while expecting CopyDataMessage"). It now answers
with CopyDone, reads the next timeline the server sends, both
CommandCompletes and ReadyForQuery, and ends the enumeration normally,
leaving the connection usable. Status updates stop before our
CopyDone, since the protocol doesn't allow CopyData after it. The next
timeline is logged, and the StartReplication docs explain how to
continue: IDENTIFY_SYSTEM returns the current timeline and
TIMELINE_HISTORY the position where it branched off, which can be
before the end of the WAL received on the old timeline.
- Asked to start at the end of a historic timeline, the server doesn't
enter copy mode but sends the next timeline right away, which Npgsql
treated as an unexpected message. The enumeration now ends right away.
- A walsender that exits after everything has been replicated (e.g. at
server shutdown, npgsql#5752) sends CommandComplete, which also broke the
connection with a misleading error. It now throws an NpgsqlException
saying that the server ended the replication stream.
- In logical replication, unknown CopyData message types are skipped
instead of breaking the connection, like the logical replication
worker, which ignores them on purpose. Physical replication keeps
rejecting them, like walreceiver, since it must not lose WAL.
Keepalive fields that newer servers may add are skipped in both.
- In physical replication, the received position reported to the server
was the server's own WAL end, not the end of the WAL received, so
pg_stat_replication.write_lsn and synchronous_commit = remote_write
treated WAL as written before it arrived. It is now the end of the
data the consumer is done with, like walreceiver reports what it
wrote.
- LastReceivedLsn starts over at the start position of each stream
instead of keeping the maximum of all previous streams, which was
wrong when continuing on a new timeline at a lower position.
- The XLogData stream is disposed after each message instead of later in
cleanup, where it could skip data based on a stale position: a server
error that ended the stream after a keepalive (e.g. after
pg_cancel_backend) turned into a timeout and a broken connection.
The new cluster tests stream physically across a standby's promotion
and through a fast shutdown, using the throwaway clusters from
NPGSQL_TEST_PG_BIN.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This was referenced Oct 25, 2026
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Depends on #6689, #6690 and #6709, which aren't merged yet; their commits show up here until they are.
StartReplicationdocs explain how to continue: IDENTIFY_SYSTEM returns the current timeline and TIMELINE_HISTORY the position where it branched off, which can be before the end of the WAL received on the old timeline.pg_stat_replication.write_lsnandsynchronous_commit = remote_writetreated WAL as written before it arrived. It is now the end of the data the consumer is done with, as walreceiver reports what it wrote.LastReceivedLsnalso starts over at the start position of each stream instead of keeping the maximum of all previous streams, which was wrong when continuing on a new timeline at a lower position.pg_cancel_backend) turned into a timeout and a broken connection.How a real server gets there
On PostgreSQL 18.6 unless noted, main against this PR:
START_REPLICATION … TIMELINE n-1, which is deterministic). With a promotion, main broke the connection at the CopyDone. With this PR the wire showed both CopyDones, the result set and ReadyForQuery, the enumeration ended normally, and IDENTIFY_SYSTEM on the same connection returned timeline 2. Asking for the older timeline gave the same results on 18.6 and 9.6.24.pg_ctl stop -m fastwhile a client streams. Main reported "Received backend message CommandComplete while expecting CopyDataMessage. Please file a bug."; this PR logs event 50007 and throws "The server ended the replication stream, e.g. because it is shutting down." Both break the connection, since the server is gone.sent_lsnwas 0/90D4000. With the client insynchronous_standby_namesand a session usingsynchronous_commit = remote_write, main's status update released a commit after 6 ms, about 1 MB before the client had the commit record; with this PR the commit waited until the consumer had passed it.LastReceivedLsnacross streams: pg_receivewal's pattern (continue on the next timeline on the same connection, from a lower position), or a second START_REPLICATION at a lower position, e.g. from the last flushed position after a cancellation. With test_decoding, main's status update reported the end of the first stream while the second one started lower; this PR reported the second stream's own start.pg_cancel_backendon the walsender (a job that cancels long-running statements). Main waited for WalReceiverTimeout (60 s by default) and then broke the connection with a TimeoutException instead of the server's error. With this PR the PostgresException (57014) came after 25 ms, and IDENTIFY_SYSTEM worked on the same connection. The mock test for this isStream_ended_by_a_server_error.The mock tests' standard stop, ErrorResponse 57014 and ReadyForQuery after CopyBothResponse, is what a real walsender sends for a cancellation (checked on the wire).
Tests
ReplicationStreamEndMockTests(11 cases, mock): on the code before this PR, 9 of them fail. The other two pass there too: a guard (physical replication still breaks on an unknown type) and a control (the logical received position).PhysicalReplicationClusterTests(live): streaming from a standby across its promotion and on from the new timeline, and streaming through a fast shutdown. They use the throwaway clusters of Add a throwaway primary/standby cluster for replication tests #6690 and needNPGSQL_TEST_PG_BIN, which CI doesn't set; they passed with the binaries of PostgreSQL 10 to 18.For the reviewer
NpgsqlEventId: 50006 (ReplicationTimelineEnded) and 50007 (ReplicationStreamEndedByServer). The class suppresses RS0016, soPublicAPI.Unshipped.txtdoesn't change; there is no other API change.Dependencies and merge order
Fixes #5752
Fixes #6697