Skip to content

Allow using cache disk for query result cache. #52141

Closed as duplicate
Closed as duplicate

Description

@alexey-milovidov

Describe the solution you'd like

A query cache can be globally configured to use the cache disk with a specified name.

If enabled, every entry in the query cache is written (write-through mode) in compressed Native format into the query cache, similar to the temporary data.

When a requested entry is not found in the query cache in memory, it is looked up in the cache disk and put into the query cache in memory if found.

The eviction logic for the query cache entries in the cache disk is the same as for the cache disk.

There is no special logic around the maximum size of entries, and there are no changes in the eviction algorithm. It is simply another cache layer, inclusive, write-through.

Two settings can be introduced to turn on/off: - writing to disk; - looking up on disk.

The cache on disk is preserved on restarts and even on upgrades. It can be cleaned up only with the whole filesystem cache (for simplicity). Potential incompatibilities in query results have to be handled manually.

ProfileEvents should be introduced to count lookups on disk and to distinguish the hit rate for memory and disk.

Activity

  1. self-assigned this
    on Jul 17, 2023
  2. dyee665 commented on Jan 20, 2025

    @dyee665
  3. self-assigned this
    on Aug 5, 2026
  4. added
    comp-query-executionRuntime execution pipeline, processors, vectorized operators, resource management during execution.
    on Sep 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

comp-query-executionRuntime execution pipeline, processors, vectorized operators, resource management during execution.feature

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions