Skip to content

docs: normalize code-fence languages for Shiki compatibility#27161

Draft
nickvigilante wants to merge 6 commits into
mainfrom
vigilante/docs-476-normalize-docs-code-fence-languages-de-risk-shikifumadocs
Draft

docs: normalize code-fence languages for Shiki compatibility#27161
nickvigilante wants to merge 6 commits into
mainfrom
vigilante/docs-476-normalize-docs-code-fence-languages-de-risk-shikifumadocs

Conversation

@nickvigilante

@nickvigilante nickvigilante commented Jul 10, 2026

Copy link
Copy Markdown
Contributor

Normalizes non-standard code-fence language tags across docs/** so a strict highlighter (Shiki, used by Fumadocs) won't fail the build on an unrecognized language, and unifies redundant synonym tags onto one canonical form per language. The current renderer (Speed-Highlight) detects the language from the code content, not the fence label, so this drift wasn't visible until now.

Changes

  • hcl -> tf (199 fences, including indented ones nested in numbered/bulleted lists). Shiki ships hcl and terraform as two distinct grammars (not aliases); every hcl-tagged fence in docs/** is actually Terraform resource/data/provider syntax, so the more specific terraform grammar is correct for all of them. tf is Shiki's own alias for that grammar, and it's also what GitHub's own markdown renderer resolves to the same HCL/Terraform highlighting.
  • pwsh/powershell -> ps1. Both ps and ps1 are registered PowerShell aliases in Shiki, but on GitHub's renderer only .ps1 is a registered file extension (.ps isn't), so ps1 renders identically to powershell there today while bare ps would silently lose highlighting.
  • env -> dotenv (a dedicated Shiki grammar for KEY=VALUE files)
  • text/output/none/url -> txt. Same built-in plain-text fallback either way, just shorter.
  • Dockerfile -> dockerfile (lowercase)
  • bash/shell -> sh (732 fences). Shiki and GitHub both alias all three to a single shell grammar; this was already the style guide's stated preference, just not enforced across the existing corpus until now.
  • markdown -> md (4 fences). Alias of the same grammar in both Shiki and GitHub.
  • jsonc -> json (1 fence). The block has no comments or trailing commas, so it doesn't need the comments-capable grammar.
  • ts -> tsx (2 fences, docs/about/contributing/frontend.md). Verified the actual content tokenizes identically under both grammars, and a sibling block in the same file already needs tsx for real JSX, so unifying to one tag is safe for this file. Documented a caveat: tsx mis-tokenizes the legacy angle-bracket type-assertion syntax (<Type>value), which is invalid in real .tsx files anyway, so use value as Type instead.
  • yml -> yaml (1 fence)
  • Updated docs/.style/style-guide/formatting.md to document all canonical tags

promql (2 fences) and caddyfile (2 fences) are left as-is. Shiki doesn't bundle a grammar for either, so they need a custom grammar registration when the site adopts Shiki, rather than degrading to txt. Tracked as follow-up work under DOCS-118 and DOCS-544 (promql).

Does not touch offlinedocs/.

Linear: DOCS-476

How the fence tags were verified

Each tag was tested against a real shiki@latest highlighter instance (codeToHtml/codeToTokens) and cross-checked against GitHub's @wooorm/starry-night grammar sources (the renderer that actually displays these .md files today, in repo browsing and PR diffs), since that's what determines whether brevity is safe before Shiki adoption:

FAIL  env        -- Language `env` is not included in this bundle.
FAIL  Dockerfile -- Language `Dockerfile` is not included in this bundle.
FAIL  promql     -- Language `promql` is not included in this bundle.
FAIL  caddyfile  -- Language `caddyfile` is not included in this bundle.
FAIL  pwsh       -- Language `pwsh` is not included in this bundle.
FAIL  output     -- Language `output` is not included in this bundle.

hcl doesn't error in Shiki, since it's a real grammar, but that's exactly the trap: it was silently rendering every fence with the generic HCL grammar instead of the Terraform-specific one. Every hcl-tagged fence in docs/** was manually checked against origin/main and is genuinely Terraform content.

For ts/tsx, tokenizing the actual doc content confirmed identical output under both grammars; a synthetic test with the legacy angle-bracket cast syntax confirmed tsx degrades on that specific construct, which the style guide now calls out.

The first normalization pass only matched fence tags at column 0 (^```tag$), missing tags indented inside numbered/bulleted lists. A follow-up pass caught the remaining occurrences at any indentation level.


This PR description and the underlying changes were prepared with Coder Agents assistance.

Normalize non-standard code-fence language tags across docs/** so a
strict highlighter (Shiki, used by Fumadocs) won't fail the build on
an unrecognized language:

- tf, terraform -> hcl
- pwsh -> powershell
- env, output -> text
- Dockerfile -> dockerfile

promql fences are left as-is; Shiki doesn't bundle a promql grammar,
so it needs a custom grammar registration when the site adopts Shiki
(DOCS-118), rather than degrading to text.

Updates the formatting style guide to document the canonical tags.

DOCS-476
@linear-code

linear-code Bot commented Jul 10, 2026

Copy link
Copy Markdown

DOCS-476

@github-actions

github-actions Bot commented Jul 10, 2026

Copy link
Copy Markdown

Docs preview

📖 View docs preview for docs/about/contributing/CONTRIBUTING.md

Correcting the normalization from the previous commit:

- tf and hcl both mapped to hcl, but Shiki ships terraform and hcl as
  two distinct grammars (Terraform's own resource/provider syntax vs
  generic HCL used by other HashiCorp tools like Vault). Nearly every
  fenced block in docs/** is literal Terraform code, so terraform is
  the correct canonical tag, not hcl.
- env mapped to text, discarding real syntax highlighting, but Shiki
  bundles a dedicated dotenv grammar for exactly this KEY=VALUE
  content, so env blocks now map to dotenv instead.

Verified every fence tag now used under docs/** against the full list
of Shiki's 235 bundled languages and aliases (only promql remains
unsupported, as already documented).

DOCS-476
@nickvigilante

Copy link
Copy Markdown
Contributor Author

Pushed a fixup commit correcting two mistakes from the original normalization pass:

  • env now maps to dotenv instead of text. Shiki ships a dedicated dotenv grammar for KEY=VALUE content, so these blocks keep real syntax highlighting instead of degrading to plain text.
  • tf/hcl now map to terraform instead of hcl. terraform and hcl are two distinct Shiki grammars (Terraform's own resource/provider syntax vs. the generic HCL grammar used by other HashiCorp tools like Vault). Every fenced block in docs/** turned out to be literal Terraform code, so terraform is the correct, more specific canonical tag.

Re-verified every fence tag now used under docs/** against Shiki's full list of 235 bundled languages and aliases (https://shiki.style/languages) rather than just the subset flagged in the original pass. promql remains the only unsupported one, unchanged from before.


Generated with Coder Agents assistance.

- Revert the canonical Terraform/HCL fence tag from Usage: terraform [global options] <subcommand> [args]

The available commands for execution are listed below.
The primary workflow commands are given first, followed by
less common or more advanced commands.

Main commands:
  init          Prepare your working directory for other commands
  validate      Check whether the configuration is valid
  plan          Show changes required by the current configuration
  apply         Create or update infrastructure
  destroy       Destroy previously-created infrastructure

All other commands:
  console       Try Terraform expressions at an interactive command prompt
  fmt           Reformat your configuration in the standard style
  force-unlock  Release a stuck lock on the current workspace
  get           Install or upgrade remote Terraform modules
  graph         Generate a Graphviz graph of the steps in an operation
  import        Associate existing infrastructure with a Terraform resource
  login         Obtain and save credentials for a remote host
  logout        Remove locally-stored credentials for a remote host
  metadata      Metadata related commands
  modules       Show all declared modules in a working directory
  output        Show output values from your root module
  providers     Show the providers required for this configuration
  query         Search and list remote infrastructure with Terraform
  refresh       Update the state to match remote systems
  show          Show the current state or a saved plan
  stacks        Manage HCP Terraform stack operations
  state         Advanced state management
  taint         Mark a resource instance as not fully functional
  test          Execute integration tests for Terraform modules
  untaint       Remove the 'tainted' state from a resource instance
  version       Show the current Terraform version
  workspace     Workspace management

Global options (use these before the subcommand, if any):
  -chdir=DIR    Switch to a different working directory before executing the
                given subcommand.
  -help         Show this help output or the help for a specified subcommand.
  -version      An alias for the "version" subcommand. to :
  both resolve to the same, correct grammar (Shiki's terraform grammar
  bundles tf/tfvars as aliases; GitHub's starry-night renderer matches
  tf via its HCL grammar's file extensions), so there's no functional
  difference, and tf is shorter to type.
- Normalize indented fence tags (nested inside numbered/bulleted lists)
  that the original column-0-only sed pass never touched: hcl/terraform
  -> tf, env -> dotenv, output/none/url -> text, Dockerfile -> dockerfile,
  across ~20 additional files.
- Document that caddyfile (2 fences in reverse-proxy-caddy.md) is, like
  promql, a grammar Shiki doesn't bundle and stays as-is pending a
  custom grammar registration.
@nickvigilante

Copy link
Copy Markdown
Contributor Author

Follow-up correction (commit 54d50ff46a):

  1. Switched the canonical Terraform/HCL tag from terraform back to tf. Both resolve to the identical, correct grammar: Shiki bundles tf/tfvars as aliases of the terraform grammar (not hcl), and GitHub's own markdown renderer (starry-night) also resolves tf to the same HCL/Terraform highlighting via its extension-matching fallback (it checks the fence tag against registered language names first, then falls back to treating it as a file extension, and .tf is registered there). No functional difference between the two tags in either renderer, so shorter wins.
  2. Fixed a real gap from the original pass: the first normalization only matched fence tags at column 0 (^```tag$), so it silently skipped every fence indented inside a numbered or bulleted list. Re-scanning at any indentation level turned up ~20 more files still tagged hcl, env, output, none, url, or Dockerfile. All normalized now to the same canonical set.
  3. Found one more genuinely-unsupported grammar: caddyfile (2 fences in docs/tutorials/reverse-proxy-caddy.md) fails against Shiki the same way promql does, no bundled grammar or alias exists. Left as-is and documented alongside promql in the style guide, rather than silently degrading to text.

pnpm run check-docs (503 files, 0 errors), vale on all changed files (0 errors/warnings), and make pre-commit-light all pass on the updated commit.


Prepared with Coder Agents assistance.

- text -> txt for command output / no-highlight blocks. Both are
  Shiki's built-in plain-text fallback (txt/plain are explicit aliases
  of text, neither uses a real grammar), and GitHub's starry-night
  renderer doesn't recognize either as a language, so both already
  render unhighlighted there today. No functional difference; txt is
  shorter.
- powershell -> ps1. Confirmed via shiki.bundledLanguagesInfo that ps1
  is a registered PowerShell alias in Shiki. On GitHub's starry-night,
  the PowerShell grammar's extensions list includes .ps1 (not bare
  .ps), and flagToScope() falls back to extension matching when a flag
  isn't a recognized language name, so ps1 resolves to the same
  highlighting as powershell there too. ps does not: it's neither a
  registered name nor a registered extension, so it would silently
  render unhighlighted on GitHub today.
- Updated docs/.style/style-guide/formatting.md accordingly.
@nickvigilante

Copy link
Copy Markdown
Contributor Author

Follow-up (commit eb204ebd78): switched two more tags to shorter equivalents, same reasoning as the terraform -> tf swap:

  • text -> txt (62 fences): both are Shiki's plain-text fallback with no real grammar behind either, and GitHub's starry-night renderer doesn't recognize either as a language, so nothing changes there. Also covers the previously-invalid output/none/url tags, which already mapped to text.
  • powershell -> ps1 (16 fences): ps1 is a registered PowerShell alias in Shiki, and .ps1 is a registered extension in starry-night's PowerShell grammar (bare .ps is not), so ps1 renders identically to powershell on GitHub today. Confirmed ps would silently lose highlighting there, so didn't use it.

pnpm run check-docs (503 files, 0 errors), vale on all changed files (0 errors/warnings), and make pre-commit-light all pass.


Prepared with Coder Agents assistance.

…to tsx, yml to yaml

- bash, shell -> sh (732 fences). Shiki bundles bash/sh/shell/zsh as
  aliases of a single shellscript grammar; GitHub's starry-night shell
  grammar likewise treats all three as names for source.shell. Applies
  the style guide's existing (but previously unenforced) sh preference
  across the whole corpus.
- markdown -> md (4 fences). Shiki's markdown grammar has md as an
  explicit alias; GitHub's markdown grammar likewise lists md among its
  names.
- jsonc -> json (1 fence, docs/ai-coder/ai-gateway/clients/mux.md). The
  block has no comments or trailing commas, so it doesn't need the
  JSON-with-comments grammar; jsonc stays reserved for blocks that
  actually use it.
- ts -> tsx (2 fences, docs/about/contributing/frontend.md). Verified
  the actual content tokenizes identically under both grammars (no
  legacy angle-bracket type-assertion syntax present), and one sibling
  block in the same file already needs tsx for real JSX, so tsx is safe
  as the single tag for that file's TypeScript blocks.
- yml -> yaml (1 fence). Shiki and GitHub both alias yml to yaml.
- Updated docs/.style/style-guide/formatting.md with the new canonical
  tags, including a note on tsx's legacy-cast-syntax caveat and on
  jsonc's narrower scope.
@nickvigilante

Copy link
Copy Markdown
Contributor Author

Follow-up (commit e78b5e830c): unified redundant synonym tags onto single canonical forms, verified against both Shiki and GitHub's starry-night renderer as before:

  • bash/shell -> sh (732 fences). Enforces the style guide's existing but previously-unenforced preference across the whole corpus.
  • markdown -> md (4 fences)
  • jsonc -> json (1 fence): checked the content, no comments/trailing commas present, so it doesn't need the comments-capable grammar.
  • ts -> tsx (2 fences in docs/about/contributing/frontend.md): confirmed via codeToTokens that the actual content tokenizes identically under both grammars, and a sibling block in the same file needs tsx for real JSX anyway. Added a style-guide caveat: tsx isn't a lossless superset in general (it mis-tokenizes the legacy <Type>value cast syntax, which isn't valid in real .tsx files), so use value as Type instead, which is already the idiomatic style.
  • yml -> yaml (1 fence)

pnpm run check-docs (503 files, 0 errors) and make pre-commit-light both pass. vale is clean on every changed file except one pre-existing false positive in docs/.style/style-guide/word-choice.md (a "Don't" example row intentionally showing wrong brand-name casing; confirmed present on main too, unrelated to this PR).


Prepared with Coder Agents assistance.

The earlier normalization sweeps only touched checked-in docs/**
output, not the generator sources. A future 'make gen' would have
reverted docs/reference/api/*.md fences back to 'shell' since the
widdershins --language_tabs flag and postprocess/main.go template
still emitted the old tag.

widdershins loads a user-template file named code_${language_tab}.dot
to render each language tab's code sample, so renaming the shell tab
to 'sh' required renaming code_shell.dot to code_sh.dot as well;
otherwise widdershins silently drops the '### Code samples' curl
block for every endpoint (verified by regenerating and diffing).

- generate.sh: --language_tabs "shell:curl" -> "sh:curl"
- markdown-template/code_shell.dot -> code_sh.dot (renamed to match)
- security.def: shell -> sh for the 'coder tokens create' example
- postprocess/main.go: shell -> txt (bare URL, no shell syntax) and
  shell -> sh (real curl command) in the apiIndexContent template
- docs/reference/api/index.md: synced checked-in output to match

Verified by running 'rm coderd/apidoc/.gen && make coderd/apidoc/.gen'
end to end: regenerated output is now byte-identical to the checked-in
docs/reference/api/*.md files.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant