Skip to content

Fix NpgsqlBinaryExporter's dispose after an error or a connection break - #6713

Open
Brar wants to merge 1 commit into
npgsql:mainfrom
Brar:core/binary-export-error-dispose
Open

Brar wants to merge 1 commit into
npgsql:mainfrom
Brar:core/binary-export-error-dispose

Conversation

@Brar

@Brar Brar commented Oct 25, 2026 •

Copy link
Copy Markdown
Member

When an ErrorResponse ended a binary export, the connector read up to the ReadyForQuery and threw, but the exporter stayed ready. Such an error comes at a row; after CopyDone instead of CommandComplete, from an AFTER trigger such as a foreign key check or from a deferred constraint at commit; or instead of ReadyForQuery, when a second statement in the command fails. A cancellation or a command timeout while StartRow waits for a row ends the export with one too. The exporter's dispose (which BeginBinaryExport runs itself when the error comes before the first row, and closing the connection runs for an undisposed exporter) then skipped to the end of the last message it had read and waited for CopyDone:

  • Without a buffer switch, that skip was negative. In Release builds, with a real server, which flushes the error and the ReadyForQuery separately, the read position usually ended up before the start of the buffer: the connection stayed open, but every later command on it failed with IndexOutOfRangeException, including the first one after it went back to the pool, which then broke it. If the bytes were still in the buffer, the dispose read the error a second time instead; Debug builds failed an assertion.
  • After a message longer than the read buffer (a notice before the COPY response or right before the error, or a long error), the skip only moved the old buffer, and the dispose waited for a CopyDone that never came: until the command timeout broke the connection, or forever without one.
  • Calls after the error didn't report it: StartRow threw "Already in the middle of a row", and Read, IsNull and Skip read a column length from the idle connection until the command timeout broke it.
  • When the connection broke while the dispose read to the end of an unfinished export, the connector disposed the exporter a second time, and the outer dispose threw NullReferenceException.
  • When BeginBinaryExport failed at the COPY response, e.g. for a missing table, the dispose logged "Binary COPY operation completed".

Now:

  • The exporter reads the header, each row, CommandComplete and ReadyForQuery through a small helper. An error there ends the export in a failed state: the dispose neither skips nor waits, and the COPY TO activity records the error. The failing call throws the original exception, as before; later StartRow and StartRowAsync, all four Read overloads, IsNull, Skip and SkipAsync throw InvalidOperationException "The COPY operation ended with an error." at once, with the original as InnerException. After the user's dispose they throw ObjectDisposedException, as before.
  • The dispose clears the connector's CurrentCopyOperation first, as NpgsqlRawCopyStream's does, so a break while it reads logs the break, leaves the connection broken and throws nothing.
  • The dispose logs "Binary COPY operation completed" only for an export that was read to its end.

How a real server gets there

Each case was run live on PostgreSQL 18.6 (most also on 14.24), in Debug and Release:

  • An error at a row, e.g. COPY (SELECT 1 / (i - 2) FROM generate_series(1, 2) i) TO STDOUT (FORMAT BINARY), at the first row (through BeginBinaryExport's own dispose) or at a later one, also when the error comes in a separate flush after a notice that the client has already read: in Release, main poisoned 20 of 20 fresh connections in each of these cases. A token cancellation or a 1-s command timeout while StartRow waited did the same without this change, since the server's 57014 ends the export like any error. With this change, all of them stayed usable.
  • An error after CopyDone: PostgreSQL sends CopyDone before EndCopyTo runs ExecutorFinish, which fires the queued AFTER triggers (copyto.c:1118 and :1015, copy.c:358-359, execMain.c:443), and it commits before it reports command completion (postgres.c:1284-1298 and :1337; all REL_18_STABLE). So the immediate foreign key check of COPY (INSERT ... RETURNING ...) TO STDOUT, or a deferred constraint at commit, fails after CopyDone. main poisoned the connection; inside BEGIN even ROLLBACK threw.
  • A failing second statement, COPY ... TO STDOUT (FORMAT BINARY); SELECT 1 / 0: its ErrorResponse comes where the exporter expects ReadyForQuery, and without this change the dispose poisoned the connection in Release.
  • A long message. Before the COPY response, e.g. debug_print_plan's plan of an ordinary join or a notice that the planner raises (an IMMUTABLE function with a constant argument), with an error at the first row: main threw the error only after the command timeout, with the connection broken. At the failing read, e.g. a function that logs more than 8 KB and then fails, "stack depth limit exceeded" with a long CONTEXT, or an immediate foreign key violation on a long text key, whose DETAIL carries the key: main's dispose waited for the command timeout and then threw NullReferenceException, and with Command Timeout 0 it never ended. With this change, both returned at once, and the connection stayed usable.
  • Calls after the error, with a 23503 after CopyDone: without this change, the connection was broken after the 5-s command timeout. With it, every later call threw the InvalidOperationException at once, the connection served the next query, also after going back to the pool, and the COPY TO activity ended with status Error and the 23503.
  • A break during the dispose: the caller stops reading a large export early, and pg_terminate_backend, a fast shutdown, a failover or the command timeout breaks the connection. main's DisposeAsync threw NullReferenceException; with this change it throws nothing.
  • A break while StartRow reads, a guard of existing behavior: pg_terminate_backend while StartRow waited for a row. The call threw 57P01, the connection was broken, a later StartRow threw ObjectDisposedException, and two disposes were fine.
  • Closing the connection without disposing the exporter: on main in Release, the reopened pooled connection's first SELECT failed with IndexOutOfRangeException; with this change it works.
  • A Begin failure: COPY missing_table TO STDOUT (FORMAT BINARY), and likewise 42703, 42601, 42501, a 22012 that the planner folds, a COPY to a server file, a text-format command, and a command timeout or cancellation while Begin waits, no longer log "completed". The exception, the connection state and the activity are unchanged.

Tests

39 mock cases in CopyTests: errors before the first row, before a later row, after CopyDone and after CommandComplete, with short and long notices and long errors, also after the client had read everything before the error; every member called after an error (18 cases); the tracing; a break during the dispose and one during StartRow; a Begin failure and its control Binary_export_read_to_end.

  • Red first: of the 36 cases other than the Begin failure and its control, 33 fail with main's exporter in Debug and 31 in Release; the ones that pass are the break control and the two guards of a break while StartRow reads, and in Release also the two tracing cases. The two Begin-failure cases fail with the exporter before that part of the change, at the "completed" log, and their control passes.
  • Removing one part of the change at a time (the state filter, the checks in StartRow and in the column calls, the inner exception, the tracing, the helper at ReadyForQuery, the dispose's log and skip conditions) fails exactly its own cases.
  • The mock writes each response in one flush, so its Release red reason is often the milder "error read twice", where a real server poisons the connection.

For the reviewer

  • A core change in NpgsqlBinaryExporter, plus an internal ThrowHelper overload, with no API change.
  • Logs: the dispose of a failed export logs nothing; before, it logged its swallowed exception at Debug level, or "COPY operation was cancelled". Tracing: the COPY TO activity ends with status Error and the original exception, including an OperationCanceledException, as NpgsqlCommand records any exception.
  • A deliberate difference from NpgsqlDataReader, whose Read returns false after a failed Read.
  • StartRow's CopyDone read deliberately doesn't go through the helper: PostgreSQL sends the binary trailer and then CopyDone with nothing in between that can fail (copyto.c:381-383 and :1113-1118, REL_18_STABLE), so only a mock could put an error there.
  • After an error, the exporter keeps the connection in the Copy state until its dispose (another command meanwhile throws NpgsqlOperationInProgressException), while the raw and text exports release it at the failing read (Fix NpgsqlRawCopyStream's dispose and later reads after an export error #6711). Both patterns predate these PRs; neither waits. The exporter still has to be disposed before the connection runs another command.
  • Not changed (pre-existing): a break during StartRow leaves the COPY TO activity unstopped.
  • Backport: planned for the 10.0 hotfix branch, after Read COPY data from the connector's current read buffer #6712's. The source merges cleanly there.

Dependencies and merge order

  • It depends on no other PR.
  • Conflicts textually with Read COPY data from the connector's current read buffer #6712, in NpgsqlBinaryExporter (the ReadHeader and StartRow read lines) and in CopyTests.cs (new tests at the same place). Merge that PR first, then rebase this one, keeping both changes: this PR's read line followed by that PR's two lines, both test blocks, and this PR's using System.Linq;. The reverse order works with the same resolution.

Fixes #6703

When an ErrorResponse ended a binary export, the connector read up to
the ReadyForQuery and threw, but the exporter stayed ready. Such an
error comes at a row, after CopyDone instead of CommandComplete (from an
AFTER trigger such as a foreign key check, or from a deferred constraint
at commit), or instead of ReadyForQuery when a second statement in the
command fails. A cancellation or a command timeout while StartRow waits
for a row ends the export with one too. The exporter's dispose then
skipped to the end of the last message it had read and waited for
CopyDone. BeginBinaryExport runs this dispose itself when the error
comes before the first row.

Without a buffer switch, that skip was negative: it failed a debug
assertion, or in release builds moved the read position back. If the
bytes it moved back over were still in the buffer, the dispose read the
error a second time. Otherwise, the usual case with a real server, which
flushes the error and the ReadyForQuery separately, the read position
ended up before the start of the buffer: the connection stayed open, but
every later command on it failed with IndexOutOfRangeException,
including the first one after it went back to the pool, which then broke
it. After a message longer than the read buffer (a notice before the
COPY response or right before the error, or a long error), the connector
had switched to a bigger buffer, the skip only moved the old one, and
the dispose waited for a CopyDone that never came: until the command
timeout broke the connection, or forever without one.

After such an error, StartRow threw InvalidOperationException ("Already
in the middle of a row"), and Read, IsNull and Skip read a column length
from the idle connection: they waited for the command timeout, which
broke the connection, or forever without one.

The exporter now treats such an error in a read of the header, of a
row, of CommandComplete or of ReadyForQuery as the end of the export, so
that its dispose neither skips nor waits, and the COPY TO activity
records the error. StartRow, Read, IsNull and Skip then throw
InvalidOperationException at once, with the error as the inner
exception.

When the connection broke while the dispose read to the end of an
unfinished export, the connector disposed the exporter a second time,
and the outer dispose threw NullReferenceException. The dispose now
clears the connector's current COPY operation first, as
NpgsqlRawCopyStream's does.

When BeginBinaryExport failed at the COPY response, e.g. for a missing
table, its dispose of the exporter logged "Binary COPY operation
completed"; the dispose now logs that only for an export that was read
to its end.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Brar
Brar force-pushed the core/binary-export-error-dispose branch from c074aae to 0e58ec7 Compare November 1, 2026 13:38
Copilot AI balanced review requested due to automatic review settings November 1, 2026 13:38

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues were identified in the binary COPY export cleanup changes.

No open issues 🎉

What changed in this PR

Fixes binary COPY export cleanup after server errors or connection breaks, preventing connection corruption, unnecessary waits, and recursive disposal failures without changing the public API.

Changes:

  • Tracks failed exports, preserves the original error for later calls, and records it in tracing.
  • Prevents disposal from draining failed exports or running recursively; logs completion only for fully consumed exports.
  • Adds regression tests for error timing, oversized messages, subsequent calls, connection reuse, logging, and tracing.
File Description
test/​Npgsql.Tests/​CopyTests.cs Adds regression coverage for binary export failures and cleanup.
src/​Npgsql/​ThrowHelper.cs Adds an overload that preserves an inner exception.
src/​Npgsql/​NpgsqlBinaryExporter.cs Adds failed-state handling and safeguards disposal and completion logging.

Review effort: Balanced

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Binary COPY export: after a server error, the exporter can poison the connection or wait until Command Timeout

2 participants