N+1 problems with FilterExpired #296

Please forgive me if I'm incorrect.

I use SqlStreamStore.MsSql, and when I call ReadAllForwards to retrieve 1000 events, I experience retrieve times of over 30 seconds. For 2000 events, it's almost a minute.

Running the query to select those rows from the Messages table, it takes < 1 second.

Investigating this, I find that for each page retrieved, there is a call to FilterExpired, which:

• calls GetMaxAge, GetStreamMetadata, then GetStreamMetadataInternal to read the last message of the stream, get the message's metadata, to finally read the MaxAge.
• if MaxAge doesn't exist or is in the future, the message is kept.
• if MaxAge is in the past, the message is deleted from the stream.

Not only do we get one additional database call per message where the overhead increases with the message count, but there is a decision during a read operation to delete messages!

I will not question the need for a MaxAge, but now that it's there:

• No read operations should try to delete anything. But if that's the most pragmatic solution, it should be optional and opt-in.
• The read operation should retrieve enough information at the first read to decide if the message has expired and then remove it from the result.
• All deletes, when opted, should be batched into one command.

As it is now, we pay a huge performance penalty for the rare occasion a message has expired.