ROX-36398: support forcing a genuine Scanner V4 re-index - #23817

…ndexing

Gives the indexer's CreateIndexReport RPC a real "force" semantic. On an
already-indexed manifest, claircore's own checkManifest state machine
short-circuits on its internal ManifestScanned check, which is independent
of StackRox's own index_report cache -- so GetOrCreateIndexReport vs
CreateIndexReport made no actual behavioral difference once a manifest had
already been indexed.

Fixed by giving IndexContainerImage a Force option that, before indexing,
deletes the manifest's claircore-native state via a new, single-manifest
targeted IndexerMetadataStore.DeleteManifest. Deliberately not reusing the
existing threshold-based GCManifests for this: throttling it to one row
could non-deterministically delete a different, unrelated manifest instead
(no ORDER BY, no per-ID filter). CreateIndexReport now passes force=true;
GetOrCreateIndexReport's internal fallback (used only for genuinely new
manifests) stays force=false.

Context: ROX-36398 - Scanner V4 has no way to re-scan an already-indexed
image after enabling ROX_SCANNER_V4_MAVEN_SEARCH, short of waiting on a
slow (7-30 day) background GC. Verified live on a test cluster, including
confirming the native deletion correctly leaves layers shared with other
manifests untouched.

Partially generated with AI assistance.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

…gnal

ScanImage's force:true, the SBOM-generation handler's scanner-mismatch
retry, and the detection service's force handling all already set
EnrichmentContext.FetchOpt to UseImageNamesRefetchCachedValues. That
option already causes fetchFromDatabase to treat the existing image as
not found for scan-reuse purposes, so Central already bypasses its own
cached scan and re-asks the configured scanner for all three of these
callers -- this was confirmed by tracing the fetchFromDatabase/
updateImageFromDatabase call chain and the history behind this option
(#6814, which fixed a names-overwriting bug for roxctl-ingested images
with multiple names/tags sharing a digest; it was never about scan
freshness specifically, that behavior is a side effect of the "existing
image not found" short-circuit).

What a scanner's own internal cache can't know is that Central already
decided to bypass its own cache -- Scanner V4's claircore-backed indexer
has no way to tell "being asked again" apart from "being asked for the
first time", so it can still silently return a stale, previously-indexed
result regardless of what Central does on its own side (the actual
ROX-36398 symptom, now fixed at the indexer by the previous commits in
this PR).

Add FetchOption.forceReindex(), recognizing ForceRefetch,
ForceRefetchScansOnly (the semantically dormant option this PR
originally targeted), and UseImageNamesRefetchCachedValues (what all
three real callers actually use) as reindex triggers. This required no
changes to any of the three existing call sites, and carries no risk of
reintroducing the names bug #6814 fixed, since none of their FetchOpt
choices change.

ForceRefetchCachedValuesOnly hits the same fetchFromDatabase
short-circuit as ForceRefetch but is deliberately excluded: it drives
the continuous periodic reprocessing loop (central/reprocessor), so
forcing a real reindex on every cycle would be a severe performance
regression. ForceRefetch is included because its only callers are
one-off/startup paths (ScanImageInternal, Central's one-time startup
reprocessing pass) -- caught by CodeRabbit review, which flagged that
these paths would otherwise still receive a stale Scanner V4 result
after this PR.

Context: ROX-36398. Verified live end-to-end against a real cluster
through Central's actual, unmodified /v1/images/scan API (force: true)
-- confirmed via indexer logs ("manifest to be scanned", not "manifest
already scanned") that this is a genuine fresh re-index through the real
production code path, not a cache hit.

Partially generated with AI assistance.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

…ps found by CodeRabbit

1. IndexContainerImage silently ignored a forced reindex request when
   ROX_SCANNER_V4_REINDEX was disabled (or no metadata store was wired
   in): the force branch was itself gated on
   features.ScannerV4ReIndex.Enabled(), so if that flag were off, the
   caller got back whatever ClairCore already had cached with no
   indication force was never attempted. Added an explicit, fail-fast
   error instead, checked before any registry I/O (so it's also cheaply
   unit-testable without a fake registry).

2. DeleteManifest could silently commit without ClairCore's own state
   actually being cleared. claircore.IndexerStore.DeleteManifests takes
   a per-manifest Postgres advisory lock; on lock contention it returns
   (nil, nil) for that digest -- the exact same observable result as
   the manifest never having existed in ClairCore's store at all
   (confirmed by reading claircore's actual implementation,
   datastore/postgres/deletemanifests.go). Our wrapper ignored the
   returned slice entirely, so a transient lock holder (e.g. another
   goroutine finishing its own index of the same manifest) looked
   identical to "nothing to delete," and the metadata transaction would
   commit anyway -- the subsequent libIndex.Index call would then find
   ClairCore's manifest state untouched and silently return the stale
   result, defeating the forced reindex with no visible error.

   Fixed by using whether our own manifest_metadata row existed a
   moment ago (in the same transaction) as ground truth: if it did, an
   empty DeleteManifests result can only mean lock contention, so retry
   briefly and fail loudly if it never clears; if it didn't, an empty
   result is the expected, common case (e.g. every first-ever
   CreateIndexReport call goes through this path with nothing to
   discard) and is not treated as an error.

Verified live against a real Postgres: added
Test_ManifestMetadata_DeleteManifest_ClairCoreStore, which wires in a
real claircore datastore/postgres.IndexerStore (the existing
DeleteManifest test only covers the manifest_metadata bookkeeping,
with IndexerStore left nil, so it never exercised this code path).
True lock-contention retry behavior couldn't be exercised in an
automated test, since the competing advisory lock's key is derived by
claircore's unexported internal/dblock package, not reachable from
this module.

Found by CodeRabbit review on #23817. A third
CodeRabbit comment on the same PR, claiming IndexAndScanImage/
CreateAndScanImage should pass ir.GetContents() to the matcher instead
of nil, was investigated and is a false positive: the matcher's
GetVulnerabilities independently looks up previously-indexed contents
via HashId when Contents is nil (scanner/services/matcher.go), which
is exactly what IndexAndScanImage's existing, pre-existing production
behavior already relies on.

Partially generated with AI assistance.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>