Skip to content

rpc : do not leave or serve truncated cache files - #30648

Open
TonyWu2333 wants to merge 1 commit into
ggml-org:masterfrom
TonyWu2333:rpc-cache-truncated-files
Open

TonyWu2333 wants to merge 1 commit into
ggml-org:masterfrom
TonyWu2333:rpc-cache-truncated-files

Conversation

@TonyWu2333

Copy link
Copy Markdown

Overview

When rpc-server -c fails to write a cache file (e.g. disk full), it leaves a truncated or empty file under the cache dir. On later requests get_cached_file treats that file as a valid hit and serves the truncated data, which silently corrupts weights.

This PR:

  • writes cache files to <hash>.tmp first and renames to <hash> only after a successful write; on failure the temp file is removed and a warning is logged
  • treats empty or unreadable cache files as a cache miss in get_cached_file, so the client falls back to sending the full tensor

Fixes #30633

Additional information

test-rpc-multi-server now starts server A with -c and a temp LLAMA_CACHE, pre-creates an empty cache file for a weights tensor, and checks that set_tensor still round-trips the correct data and that no empty or .tmp files are left in the cache dir afterwards.

Requirements

Write cache files to a temp path and rename them only after a successful write, and treat empty or unreadable cache files as a cache miss.

Fixes ggml-org#30633
@TonyWu2333
TonyWu2333 requested review from a team and ggerganov as code owners October 16, 2026 06:25
@github-actions github-actions Bot added testing Everything test related ggml changes relating to the ggml tensor library for machine learning labels Oct 16, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ggml changes relating to the ggml tensor library for machine learning testing Everything test related

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Eval bug: rpc-server -c: cache files truncated on disk-full are later served as valid cache hits → silent weight corruption / garbage output

1 participant