Skip to content

security_policy

Directory actions

More options

Directory actions

More options

Latest commit

 

History

History
 
 

security_policy

Folders and files

NameName
Last commit message
Last commit date

parent directory

..
README.md
README.md
 
 
run_security_policy_bench_with_bin.py
run_security_policy_bench_with_bin.py
 
 
security_policy_bench.py
security_policy_bench.py
 
 

README.md

Security Policy Benchmark

Local benchmark for measuring Row Access Policy (RAP) and Data Mask Policy performance overhead in Databend.

What It Measures

  1. RAP vs WHERE: Overhead of row access policy (auto prewhere pushdown) compared to an equivalent hand-written WHERE clause.
  2. Mask vs Function: Overhead of masking policy (auto function replacement) compared to an equivalent hand-written function call in SELECT.
  3. Planner Cache: Cold vs warm plan cache latency with and without security policies.

Usage

Against a running Databend instance

# Default: 5M rows, all complexity levels, 5 bench rounds
python3 scripts/security_policy/security_policy_bench.py \
    --dsn "databend://root@127.0.0.1:8000/default?sslmode=disable" \
    --output tmp/security_policy_bench_result.json

# Quick run: single complexity, fewer rounds
python3 scripts/security_policy/security_policy_bench.py \
    --complexity simple --bench-rounds 3 --output tmp/quick.json

# Dry run: print generated SQL without executing
python3 scripts/security_policy/security_policy_bench.py --dry-run

With a specific binary (auto start/stop services)

python3 scripts/security_policy/run_security_policy_bench_with_bin.py \
    --query-bin target/release/databend-query \
    --output tmp/result.json

Cross-version A/B comparison

python3 scripts/security_policy/run_security_policy_bench_with_bin.py \
    --query-bin target/release/databend-query \
    --compare-bin /path/to/other/databend-query \
    --output tmp/result.json

This runs the benchmark on both binaries and outputs a diff report. Regressions (>5% slower) are flagged.

CLI Arguments

security_policy_bench.py

Argument Default Description
--dsn env BENDSQL_DSN bendsql connection string
--rows 5,000,000 Total rows to insert
--rows-per-batch 1,000,000 Rows per INSERT batch
--rows-per-block 10,000 ROW_PER_BLOCK table option
--blocks-per-segment 10 BLOCK_PER_SEGMENT table option
--complexity all RAP/mask complexity: simple, range, complex, all
--pattern all Query pattern filter
--warmup-rounds 2 Warmup iterations before measurement
--bench-rounds 5 Measurement iterations
--output none JSON output path
--keep false Don't drop database before run
--drop-after false Drop database after run
--dry-run false Print SQL without executing

run_security_policy_bench_with_bin.py

Argument Default Description
--query-bin required Path to databend-query binary
--meta-bin auto-detect Path to databend-meta binary
--compare-bin none Second binary for A/B comparison
--bench-arg none Extra args passed to bench script (repeatable)
--no-stop false Leave services running after bench

Metrics

Primary metrics (per query)

  • duration_ms: Best-of-N wall time (ms)
  • duration_avg_ms: Average wall time across rounds
  • all_ms: All individual round timings

Pruning stats (RAP queries)

  • segments_before/after: Segment range pruning
  • blocks_before/after: Block range pruning
  • *_pruned_pct: Percentage pruned

Summary output

  • rap_vs_where.overhead_pct: Policy overhead relative to manual WHERE
  • mask_vs_func.overhead_pct: Policy overhead relative to manual function

Test Table

CREATE TABLE bench_security (
    id         INT NOT NULL,
    tenant_id  INT NOT NULL,
    email      VARCHAR NOT NULL,
    amount     DECIMAL(18,2) NOT NULL,
    status     VARCHAR NOT NULL,
    created_at TIMESTAMP NOT NULL
) CLUSTER BY (created_at);
  • Clustered by time (realistic production pattern)
  • 100 distinct tenant_id values scattered across all blocks
  • RAP predicates on tenant_id force prewhere filtering (no cluster pruning shortcut)

Policy Complexity Levels

Level RAP predicate Mask expression
simple tenant_id = 1 '***'
range tenant_id BETWEEN 10 AND 30 CONCAT(LEFT(val,3), '***@***.com')
complex tenant_id % 10 = 0 AND status = 'active' CASE WHEN ... END