Skip to content

Add an optional stack paint helper. - #27

Draft
kristjanvalur wants to merge 3 commits into
stackless-dev:masterfrom
kristjanvalur:stack-guard
Draft

kristjanvalur wants to merge 3 commits into
stackless-dev:masterfrom
kristjanvalur:stack-guard

Conversation

@kristjanvalur

@kristjanvalur kristjanvalur commented Oct 16, 2026 •

Copy link
Copy Markdown
Collaborator

Optional paint of a caller-owned block. Plain C, compiled by the program that wants it, not part of libstackman, no library calls. The primary use is a pre-allocated block passed to stackman_call. A stackman_switch snapshot is a different range and is not what this paints.

Paint and check

You allocated a block, you are about to run a callback on it, and you want to know if that callback grew into the end of the block. The near end is the overflow side. Paint a band there, make the call, then read the band back. The check does not stop the write. It tells you the block was too small.

stackman_block_call turns the malloc pair into far and extent. far is the pointer you pass to stackman_call. extent is the byte count from there toward near. Alignment can make that a few bytes shorter than the malloc size. The sliver outside far is not part of the stack.

void *far;
size_t extent;
char *base = malloc(size);             /* low address, as malloc returns it */
if (stackman_block_call(base, size, &far, &extent) == 0)
    ; /* alignment left no usable byte */

stackman_guard_paint(far, extent, 64, 0xa5);
stackman_call(worker, ctx, far);
if (stackman_guard_check(far, extent, 64, 0xa5) != 1)
    ; /* the near band was overwritten */

/* How deep did this run go? Paint the whole block first. */
stackman_guard_paint(far, extent, extent, 0xa5);
stackman_call(worker, ctx, far);
size_t unused = stackman_stack_unused(far, extent, 0xa5);
size_t used = extent - unused;

stackman_stack_unused counts a solid run of the pattern from the near end back toward far. It is only meaningful after the whole extent was painted and the stack then grew from far. A pattern byte the program itself stores counts as untouched.

Why this is not the switch snapshot

stackman_switch, as libtealet uses it, saves a slice of the real C stack. The saved range is the bytes already in use: from the near pointer (the stack pointer at save) out to that tealet's stack_far. A copy of that range has no untouched tail. Painting it and scanning for a pattern does not measure overflow, and it does not measure how large a stack the coroutine needs. Tealet already stores that size on the chunk.

The main stack is the case where the extent is genuinely unknown. Its far marker is often STACKMAN_SP_FURTHEST, a sentinel, not an address you can paint from. The near limit is the operating system's guard, which this library does not ask about. Extent 0 is that input. The helpers then leave memory alone.

So the two ranges are:

  • Switch snapshot: near and far of a stack that already exists. The useful number is STACKMAN_SP_DIFF(far, near), and tealet copies those bytes. No paint.
  • stackman_call block: a malloc region whose far end you choose and whose near end is the end of the allocation. Both ends are known. This is the paint.

stackman_stack_unused stays (far, extent)

stackman_block_call is the conversion from a low address and a size. The paint, the check, and the unused count take the pair it returns, the same pointer stackman_call takes.

On a descending stack the low address is the near end. On an ascending stack the low address is the far end. A paint function that took the malloc pair directly would change which end that pointer names when STACKMAN_STACK_DIR changes. The helper does that conversion once and drops the alignment sliver.

bin/test_guard runs the descending case, including a stackman_call on a block obtained from stackman_block_call. bin/test_guard_up rebuilds the unit with STACKMAN_STACK_DIR=1, checks the other branch of the address math, and paints a block from the same helper. No supported ABI grows that way. Native make test passed.

Closes #14.

stackman_guard.c is plain C and is not part of the switch. It paints
a band at the near end of a caller-owned block. The near bound is the
overflow side. Extent 0 means that bound is unknown, and the helpers
leave memory alone. A stackman_call block knows both ends.
A switch snapshot is the used range between near and far and has no
untouched tail. stackman_stack_unused keeps those same coordinates.
A malloc (low, size) pair swaps which end it names when the stack
direction changes.
The malloc pair becomes far and extent in one place. The paint, the
check, and the unused count keep those coordinates.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add stack overflow detection mechanism

1 participant