Repository navigation
Conversation
This was referenced Oct 25, 2026
When an ErrorResponse ended a binary export, the connector read up to
the ReadyForQuery and threw, but the exporter stayed ready. Such an
error comes at a row, after CopyDone instead of CommandComplete (from an
AFTER trigger such as a foreign key check, or from a deferred constraint
at commit), or instead of ReadyForQuery when a second statement in the
command fails. A cancellation or a command timeout while StartRow waits
for a row ends the export with one too. The exporter's dispose then
skipped to the end of the last message it had read and waited for
CopyDone. BeginBinaryExport runs this dispose itself when the error
comes before the first row.
Without a buffer switch, that skip was negative: it failed a debug
assertion, or in release builds moved the read position back. If the
bytes it moved back over were still in the buffer, the dispose read the
error a second time. Otherwise, the usual case with a real server, which
flushes the error and the ReadyForQuery separately, the read position
ended up before the start of the buffer: the connection stayed open, but
every later command on it failed with IndexOutOfRangeException,
including the first one after it went back to the pool, which then broke
it. After a message longer than the read buffer (a notice before the
COPY response or right before the error, or a long error), the connector
had switched to a bigger buffer, the skip only moved the old one, and
the dispose waited for a CopyDone that never came: until the command
timeout broke the connection, or forever without one.
After such an error, StartRow threw InvalidOperationException ("Already
in the middle of a row"), and Read, IsNull and Skip read a column length
from the idle connection: they waited for the command timeout, which
broke the connection, or forever without one.
The exporter now treats such an error in a read of the header, of a
row, of CommandComplete or of ReadyForQuery as the end of the export, so
that its dispose neither skips nor waits, and the COPY TO activity
records the error. StartRow, Read, IsNull and Skip then throw
InvalidOperationException at once, with the error as the inner
exception.
When the connection broke while the dispose read to the end of an
unfinished export, the connector disposed the exporter a second time,
and the outer dispose threw NullReferenceException. The dispose now
clears the connector's current COPY operation first, as
NpgsqlRawCopyStream's does.
When BeginBinaryExport failed at the COPY response, e.g. for a missing
table, its dispose of the exporter logged "Binary COPY operation
completed"; the dispose now logs that only for an export that was read
to its end.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Brar
force-pushed
the
core/binary-export-error-dispose
branch
from
November 1, 2026 13:38
c074aae to
0e58ec7
Compare
There was a problem hiding this comment.
🟢 Approval recommended
No active review issues were identified in the binary COPY export cleanup changes.
No open issues 🎉
What changed in this PR
Fixes binary COPY export cleanup after server errors or connection breaks, preventing connection corruption, unnecessary waits, and recursive disposal failures without changing the public API.
Changes:
- Tracks failed exports, preserves the original error for later calls, and records it in tracing.
- Prevents disposal from draining failed exports or running recursively; logs completion only for fully consumed exports.
- Adds regression tests for error timing, oversized messages, subsequent calls, connection reuse, logging, and tracing.
| File | Description |
|---|---|
| test/Npgsql.Tests/CopyTests.cs | Adds regression coverage for binary export failures and cleanup. |
| src/Npgsql/ThrowHelper.cs | Adds an overload that preserves an inner exception. |
| src/Npgsql/NpgsqlBinaryExporter.cs | Adds failed-state handling and safeguards disposal and completion logging. |
Review effort: Balanced
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
When an ErrorResponse ended a binary export, the connector read up to the ReadyForQuery and threw, but the exporter stayed ready. Such an error comes at a row; after CopyDone instead of CommandComplete, from an AFTER trigger such as a foreign key check or from a deferred constraint at commit; or instead of ReadyForQuery, when a second statement in the command fails. A cancellation or a command timeout while
StartRowwaits for a row ends the export with one too. The exporter's dispose (whichBeginBinaryExportruns itself when the error comes before the first row, and closing the connection runs for an undisposed exporter) then skipped to the end of the last message it had read and waited for CopyDone:StartRowthrew "Already in the middle of a row", andRead,IsNullandSkipread a column length from the idle connection until the command timeout broke it.BeginBinaryExportfailed at the COPY response, e.g. for a missing table, the dispose logged "Binary COPY operation completed".Now:
StartRowandStartRowAsync, all fourReadoverloads,IsNull,SkipandSkipAsyncthrow InvalidOperationException "The COPY operation ended with an error." at once, with the original as InnerException. After the user's dispose they throw ObjectDisposedException, as before.CurrentCopyOperationfirst, asNpgsqlRawCopyStream's does, so a break while it reads logs the break, leaves the connection broken and throws nothing.How a real server gets there
Each case was run live on PostgreSQL 18.6 (most also on 14.24), in Debug and Release:
COPY (SELECT 1 / (i - 2) FROM generate_series(1, 2) i) TO STDOUT (FORMAT BINARY), at the first row (throughBeginBinaryExport's own dispose) or at a later one, also when the error comes in a separate flush after a notice that the client has already read: in Release, main poisoned 20 of 20 fresh connections in each of these cases. A token cancellation or a 1-s command timeout whileStartRowwaited did the same without this change, since the server's 57014 ends the export like any error. With this change, all of them stayed usable.EndCopyTorunsExecutorFinish, which fires the queued AFTER triggers (copyto.c:1118and:1015,copy.c:358-359,execMain.c:443), and it commits before it reports command completion (postgres.c:1284-1298and:1337; all REL_18_STABLE). So the immediate foreign key check ofCOPY (INSERT ... RETURNING ...) TO STDOUT, or a deferred constraint at commit, fails after CopyDone. main poisoned the connection; inside BEGIN even ROLLBACK threw.COPY ... TO STDOUT (FORMAT BINARY); SELECT 1 / 0: its ErrorResponse comes where the exporter expects ReadyForQuery, and without this change the dispose poisoned the connection in Release.pg_terminate_backend, a fast shutdown, a failover or the command timeout breaks the connection. main's DisposeAsync threw NullReferenceException; with this change it throws nothing.StartRowreads, a guard of existing behavior:pg_terminate_backendwhileStartRowwaited for a row. The call threw 57P01, the connection was broken, a laterStartRowthrew ObjectDisposedException, and two disposes were fine.COPY missing_table TO STDOUT (FORMAT BINARY), and likewise 42703, 42601, 42501, a 22012 that the planner folds, a COPY to a server file, a text-format command, and a command timeout or cancellation while Begin waits, no longer log "completed". The exception, the connection state and the activity are unchanged.Tests
39 mock cases in
CopyTests: errors before the first row, before a later row, after CopyDone and after CommandComplete, with short and long notices and long errors, also after the client had read everything before the error; every member called after an error (18 cases); the tracing; a break during the dispose and one duringStartRow; a Begin failure and its controlBinary_export_read_to_end.StartRowreads, and in Release also the two tracing cases. The two Begin-failure cases fail with the exporter before that part of the change, at the "completed" log, and their control passes.StartRowand in the column calls, the inner exception, the tracing, the helper at ReadyForQuery, the dispose's log and skip conditions) fails exactly its own cases.For the reviewer
NpgsqlBinaryExporter, plus an internalThrowHelperoverload, with no API change.NpgsqlCommandrecords any exception.NpgsqlDataReader, whoseReadreturns false after a failedRead.StartRow's CopyDone read deliberately doesn't go through the helper: PostgreSQL sends the binary trailer and then CopyDone with nothing in between that can fail (copyto.c:381-383and:1113-1118, REL_18_STABLE), so only a mock could put an error there.StartRowleaves the COPY TO activity unstopped.Dependencies and merge order
NpgsqlBinaryExporter(theReadHeaderandStartRowread lines) and inCopyTests.cs(new tests at the same place). Merge that PR first, then rebase this one, keeping both changes: this PR's read line followed by that PR's two lines, both test blocks, and this PR'susing System.Linq;. The reverse order works with the same resolution.Fixes #6703