Skip to content

fix(provisioner/terraform): prevent deadlock when terraform emits log lines > 64 KiB#26238

Open
goingforstudying-ctrl wants to merge 1 commit into
coder:mainfrom
goingforstudying-ctrl:fix/terraform-scanner-large-log-lines
Open

fix(provisioner/terraform): prevent deadlock when terraform emits log lines > 64 KiB#26238
goingforstudying-ctrl wants to merge 1 commit into
coder:mainfrom
goingforstudying-ctrl:fix/terraform-scanner-large-log-lines

Conversation

@goingforstudying-ctrl

Copy link
Copy Markdown

Ran into this while debugging an Azure deployment with TF_LOG=DEBUG set. The build would just hang forever — no error, no timeout, no progress. Took a while to figure out why.

The azurerm provider logs full HTTP response bodies (e.g. the subscription providers list, ~3 MB) as a single hclog [DEBUG] entry. That blows past bufio.Scanner's default 64 KiB limit. When scanner.Scan() fails with "token too long", the goroutine reading from the pipe exits, the pipe writer blocks, terraform's stdout/stderr block, and cmd.Wait() never returns. The provisioner is deadlocked.

I bumped the scanner buffer to 4 MiB and added scanner.Err() handling so any remaining failures get logged instead of causing silent hangs. Also added a fallback helper for truly enormous lines.

Fixes #24766

Not 100% sure about the 4 MiB constant — happy to adjust if there's a project preference.

@github-actions github-actions Bot added the community Pull Requests and issues created by the community. label Jun 11, 2026
@github-actions

github-actions Bot commented Jun 11, 2026

Copy link
Copy Markdown

All contributors have signed the CLA ✍️ ✅
Posted by the CLA Assistant Lite bot.

@goingforstudying-ctrl

Copy link
Copy Markdown
Author

I have read the CLA Document and I hereby sign the CLA

cdrci2 added a commit to coder/cla that referenced this pull request Jun 11, 2026
@goingforstudying-ctrl goingforstudying-ctrl force-pushed the fix/terraform-scanner-large-log-lines branch 8 times, most recently from ba54328 to d6c4f23 Compare June 12, 2026 05:01
…ines

Terraform providers (e.g. azurerm with TF_LOG=DEBUG) can emit single log
lines that exceed the default 64 KiB bufio.MaxScanTokenSize.  When this
happens bufio.Scanner fails with 'token too long', the scanner loop exits,
and the goroutine stops consuming from the io.Pipe.  The pipe writer blocks,
terraform's stdout/stderr block, and cmd.Wait() never returns, causing the
provisioner to hang indefinitely with no error or timeout.

Fix by:
- Increasing the scanner buffer to 4 MiB via a newLogScanner helper.
- Adding scanner.Err() handling in readAndLog and provisionReadAndLog so
  scan failures are surfaced as ERROR logs instead of silent deadlocks.
- Adding a readLineWithFallback helper for future use with lines that may
  still exceed the enlarged buffer.

Fixes coder/coder coder#24766
@goingforstudying-ctrl goingforstudying-ctrl force-pushed the fix/terraform-scanner-large-log-lines branch from d6c4f23 to f4f459a Compare June 12, 2026 07:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

community Pull Requests and issues created by the community.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Provisioner hangs indefinitely when terraform emits a single log line larger than 64 KiB (e.g. azurerm + TF_LOG=DEBUG)

1 participant