Add MAXCOUNT and MAXSIZE options to XREAD and XREADGROUP#15282
Conversation
…tive reply across streams
| return; | ||
| } | ||
|
|
||
| /* MAXCOUNT is a cumulative cap across all streams, so it must not be |
There was a problem hiding this comment.
Should we also error if (maxcount && maxsize && maxcount > maxsize)?
This is most likely a user error, though it will cause no harm.
There was a problem hiding this comment.
MAXCOUNT is in entries and MAXSIZE is in bytes — different units, so maxcount > maxsize is fine. They're independent limits, and whichever one is hit first wins. MAXCOUNT 1000 MAXSIZE 64 is perfectly valid.
There was a problem hiding this comment.
It is valid, but doesn't make much sense, since each entry requires at least one byte.
There was a problem hiding this comment.
@LiorKogan I can think of another example. If the
user uses the following configuration.
set-max-listpack-entries is no longer effective.
Since the user wants to do this, I don't think it's appropriate to cause any inconvenience to the user.
This way, the user won't have to wonder why each entry requires at least one byte. Therefore, I cannot use this parameter in this way.
set-max-listpack-entries 128
set-max-listpack-value 1
|
This change touches performance-sensitive code paths. Adding the |
Co-authored-by: debing.sun <debing.sun@redis.com>
|
net_output_bytes_curr_cmd is reset only in commandProcessed(), so et_output_bytes_curr_cmd was accumulated across the whole EXEC, that is to say, even with a very small stream entries, we can still reach the maxsize limit. |
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.
Reviewed by Cursor Bugbot for commit 4ee7db8. Configure here.
Co-authored-by: debing.sun <debing.sun@redis.com>

Overview
This PR adds two new options,
MAXCOUNTandMAXSIZE, to theXREADandXREADGROUPstream commands. They cap the cumulative reply across all streams named in a single command:MAXCOUNTbounds the total number of entries returned, andMAXSIZEbounds the total reply size in bytes. Unlike the existingCOUNToption — which limits entries on a per-stream basis — these new options apply a global budget for the whole command, giving clients a reliable way to bound a multi-stream read.Problem Statement
XREAD/XREADGROUPaccept multiple streams in one call, andCOUNTonly limits entries per stream. A read over N streams withCOUNT Ccan therefore return up toN * Centries, and the total reply size is effectively unbounded — a single command can return a very large payload depending on how many streams match and how large each entry is.For clients that need to bound the work and memory of a single round trip (e.g. fan-in consumers reading many streams, or latency-sensitive paths), there is no way today to say "give me at most K entries total" or "at most B bytes total" across all streams in one command. Working around this requires issuing per-stream reads and aggregating client-side, costing extra round trips.
Solution
Introduces two cumulative caps, applied across all streams in the command:
XREAD [COUNT count] [MAXCOUNT maxcount] [MAXSIZE maxsize] [BLOCK milliseconds] STREAMS key [key ...] id [id ...]XREADGROUP GROUP group consumer [COUNT count] [MAXCOUNT maxcount] [MAXSIZE maxsize] [BLOCK milliseconds] [CLAIM min-idle-time] [NOACK] STREAMS key [key ...] id [id ...]Semantics:
MAXCOUNTcaps the total number of entries returned across all streams. It must be a positive integer, and must be>= COUNTwhen both are given (since it is a cumulative cap over a per-stream limit). WhenCOUNTis omitted,MAXCOUNTalone bounds the total.MAXSIZEcaps the total reply size in bytes (tracked viac->net_output_bytes_curr_cmd). It must be a positive integer.MAXSIZEis still returned rather than yielding an empty reply.>) reads and for history/PEL reads inXREADGROUP. ForMAXSIZE, the limit is checked before delivering the next PEL/new entry, so a skipped entry is neither sent nor added to the consumer's PEL.Implementation details:
MAXCOUNTis enforced inxreadCommand(): a runningtotal_entriescounter shrinks each stream's effectivecountto the remaining budget and stops scanning further streams once the cap is reached.MAXSIZEis enforced inside the emit loops ofstreamReplyWithRange()andstreamReplyWithRangeFromConsumerPEL(), guarded by(emitted_before + arraylen) > 0to preserve the "always serve one entry" exception. All existing callers (XRANGE,XCLAIM,XAUTOCLAIM,XINFO, and the consumer-PEL path) are migrated to the new struct form.Examples
Given three streams, each with 100 entries:
MAXCOUNTcaps the cumulative entry count (vs. per-streamCOUNT):MAXCOUNTwithoutCOUNT— bounds the total directly, filling from the first stream onward:MAXSIZEbounds the reply size in bytes:MAXSIZEalways returns a single oversized message:Both together — whichever bound triggers first wins:
XREADGROUPbehaves identically for both new (>) reads and history/PEL reads, and the caps are honored after a blocking client is unblocked.Backward Compatibility
This is a fully additive change.
MAXCOUNTandMAXSIZEare new optional tokens recognized only withinXREAD/XREADGROUP; when omitted, behavior is exactly as before. No existing command, argument, default, or reply format is modified, and no new command is introduced.Note
Medium Risk
Touches core stream read/PEL delivery paths and consumer-group side effects when MAXSIZE stops before the next entry; behavior is additive but multi-stream reply shaping is easy to get wrong for clients.
Overview
XREAD and XREADGROUP gain optional
MAXCOUNTandMAXSIZE(8.10.0) to cap the cumulative reply across all streams in one command—unlike per-streamCOUNT.MAXCOUNTlimits total entries;xreadCommandtrackstotal_entries, shrinks each stream’s effective count, and stops scanning streams once the cap is hit.MAXSIZElimits reply bytes viac->net_output_bytes_curr_cmd; enforcement is instreamReplyWithRange()/ PEL paths, with a baseline soMULTI/EXECbudgets apply perXREADonly. At least one entry is always returned (a single oversized message can exceedMAXSIZE). Validation rejects non-positive values andMAXCOUNT < COUNT.streamReplyWithRange()now takes astreamReplyRangeArgsstruct (includingmaxsize/emitted_before); existing callers are updated. Command metadata and Tcl tests cover validation, multi-stream caps, blocking unblock, andMULTI/EXECbehavior.Reviewed by Cursor Bugbot for commit 30f52e1. Bugbot is set up for automated code reviews on this repo. Configure here.