Read COPY data from the connector's current read buffer - #6712

A backend message longer than the read buffer (e.g. a notice over 8 KB) makes the connector switch to a bigger buffer. NpgsqlRawCopyStream, which BeginTextExport also reads through, and NpgsqlBinaryExporter kept the buffer they found when they were created. After such a message they read the following data from the old buffer, which the switch had emptied, i.e. directly from the socket, bypassing what the connector had already received into the new buffer. From there they misread the stream, or waited for data that had already arrived. If an earlier message had already made the connector switch, the next switch returned that bigger buffer's array to the ArrayPool, and the reader went on reading socket data into it, even after the pool had handed it out again (shown with a mock probe).

The readers now take the connector's current buffer whenever they read a CopyData message, like the data reader does for each row. The raw stream also takes it when it reads the response to the COPY command: after an oversize message before that response, its ReadTimeout (which the text reader's and writer's Timeout set) went to the old buffer, so asynchronous reads ignored it. #6721 fixes the same pattern in the pgoutput decoder.

How a real server gets there

Notices may come between CopyData messages (protocol.sgml:1341-1345), and PostgreSQL flushes each one at once (elog.c:3695; references to REL_18_STABLE). Sources of a notice longer than the read buffer during COPY ... TO STDOUT, each run live:

• a PL/pgSQL RAISE NOTICE of a large value in a function the COPY query calls;
• client_min_messages = log with debug_print_plan = on, both user-settable, which print an ordinary join's plan of 11 to 13 KB (postgres.c:954): at a row when a function plans an SPI query there, or before CopyOutResponse for the COPY query itself;
• an IMMUTABLE function with constant arguments, which the planner runs before CopyOutResponse.

The binary header goes out inside the first row's CopyData, so a notice raised while the first row is computed lands between CopyOutResponse and the header. These triggers are realistic but uncommon; with the default Command Timeout the failures below take 30 s, and with 0 they never end.

Live on 18.6 with Command Timeout 5 s, COPY (SELECT f(i) FROM generate_series(1, 3) i) TO STDOUT with f raising a 20000-character notice for each row: without this change, BeginRawBinaryCopy returned 29 bytes of a notice's text as COPY data and then failed with "Unknown message code: 80", BeginBinaryExport failed with "Invalid COPY binary signature at beginning!", and BeginTextExport timed out and left the connection broken. With it, all three read the same data as with a 10-character notice.

The tests' scenarios, live on 18.6 and 14.24 with Command Timeout 5 s:

• A raw binary export with a 20 KB notice at row 2, or the debug_print_plan output: main returned the header and row 1, then timed out and broke the connection; with this change, all 51 bytes.
• A binary export with a notice before the header or before a later row: main read no row, or only row 1, and timed out (with Command Timeout 0 it was still waiting after 10 s); with this change, every row.
• The raw export's ReadTimeout after a notice before CopyOutResponse (debug_print_plan, ReadTimeout = 500 ms, a 3-s stall at the first row): main timed out after 3.5 s, this change after 0.5 s. Only async reads are affected; sync reads use the socket's timeout. The import's variant, with a long notice before CopyInResponse (e.g. raised by an IMMUTABLE function in the default of a column the COPY doesn't list), behaves the same, but that trigger is contrived.

Tests

Three mock tests in CopyTests with 10 cases, each oversize case with a 10-character-notice control; they use the smallest read buffer (4096 bytes), so that the mock can write an oversize notice in one piece:

• Raw_binary_export_with_notice_between_CopyData_messages;
• Raw_binary_copy_timeout_with_notice_before_copy_response (export and import);
• Binary_export_with_notice_before_row (before the header, or before a later row).

Red first: without the source change, the 5 oversize cases fail in Debug and Release (NpgsqlException "Exception while reading from stream" at the reads on the old buffer, or the timeout assertion), and the 5 controls pass. The read cases fail the same way with main's COPY code. Removing any one of the four lookups fails exactly its own cases.

For the reviewer

• A core change in NpgsqlRawCopyStream and NpgsqlBinaryExporter, with no API change. The buffer is looked up per message, not per use.
• The exporter's Init doesn't take the buffer: its only effect would be on the error path that Fix NpgsqlBinaryExporter's dispose after an error or a connection break #6713 fixes.
• Imports aren't affected otherwise: the import stream and NpgsqlBinaryImporter read only messages, through the connector.
• The bug is on main and, by reading, on the 10.0 hotfix branch. Backport: planned for it.

Dependencies and merge order

• It depends on no other PR.
• Conflicts textually with Fix NpgsqlBinaryExporter's dispose after an error or a connection break #6713, in NpgsqlBinaryExporter (the ReadHeader and StartRow read lines) and in CopyTests.cs (new tests at the same place). Merge this PR first; Fix NpgsqlBinaryExporter's dispose after an error or a connection break #6713 is then rebased keeping both changes: its read line followed by this PR's two lines, and both test blocks.

Fixes #6702