Skip to content

Add configurable OTLP resource attributes to workflow observability#39636

Merged
pelikhan merged 4 commits into
mainfrom
copilot/feature-custom-otel-resource-attributes
Jun 16, 2026
Merged

Add configurable OTLP resource attributes to workflow observability#39636
pelikhan merged 4 commits into
mainfrom
copilot/feature-custom-otel-resource-attributes

Conversation

Copilot AI commented Jun 16, 2026

Copy link
Copy Markdown
Contributor

Workflows could configure OTLP endpoints and headers, but not custom OTel resource attributes. This adds a first-class observability.otlp.resource-attributes field that is appended to the generated OTEL_RESOURCE_ATTRIBUTES, so workflow-defined metadata is carried across gh-aw and engine-emitted spans.

  • Schema and frontmatter support

    • Adds observability.otlp.resource-attributes to the workflow schema and typed frontmatter parsing.
    • Keeps the field shape as a string map so values can use GitHub Actions expressions.
  • OTEL resource attribute generation

    • Merges user-defined resource attributes into OTEL_RESOURCE_ATTRIBUTES rather than replacing the built-in gh-aw attributes.
    • Reuses spec-correct percent-encoding for static keys and values.
    • Preserves GitHub Actions expressions in values so runtime context can still be injected.
  • Safety guardrails

    • Rejects secrets.* and vars.* references in resource attribute values.
    • This prevents exporting sensitive configuration into tracing backends via resource metadata.
  • Imported workflow behavior

    • Merges imported resource-attributes with main-workflow precedence, matching the existing observability merge model.

Example:

observability:
  otlp:
    endpoint: ${{ secrets.OTLP_ENDPOINT }}
    resource-attributes:
      my.target-repo: ${{ github.event.client_payload.repository.full_name || github.repository }}
      my.actor: ${{ github.actor }}
      my.event: repository_dispatch

This compiles into OTEL_RESOURCE_ATTRIBUTES alongside the existing gh-aw attributes, with static values percent-encoded and expression values preserved for runtime evaluation.

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copilot AI changed the title [WIP] Add support for configurable custom OTel resource attributes Add configurable OTLP resource attributes to workflow observability Jun 16, 2026
Copilot AI requested a review from pelikhan June 16, 2026 18:37
@pelikhan pelikhan marked this pull request as ready for review June 16, 2026 18:50
Copilot AI review requested due to automatic review settings June 16, 2026 18:50

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Note

Copilot was unable to run its full agentic suite in this review.

Adds support for workflow-defined OTLP resource attributes so custom metadata is appended to OTEL_RESOURCE_ATTRIBUTES (without replacing built-ins), including schema/frontmatter support, imported-workflow merge behavior, and safety validation to prevent leaking secrets.* / vars.*.

Changes:

  • Adds observability.otlp.resource-attributes to schema + typed frontmatter parsing.
  • Appends sorted user resource attributes to generated OTEL_RESOURCE_ATTRIBUTES with percent-encoding (static values) while preserving expressions.
  • Merges imported/main workflow resource attributes with main-workflow precedence and validates that values don’t reference secrets.* / vars.*.
Show a summary per file
File Description
pkg/workflow/observability_otlp.go Implements resource-attribute collection, merge helpers, validation, and OTEL_RESOURCE_ATTRIBUTES formatting/appending.
pkg/workflow/observability_otlp_test.go Adds unit tests for injection, parsing, collection, and secrets/vars validation.
pkg/workflow/frontmatter_types.go Extends OTLPConfig with a ResourceAttributes map and documentation.
pkg/workflow/compiler_validators.go Hooks resource-attribute validation into the compiler validation pipeline.
pkg/workflow/compiler_orchestrator_workflow.go Merges imported/main resource-attributes into raw frontmatter.
pkg/workflow/compiler_orchestrator_workflow_test.go Tests imported observability merge behavior for resource attributes.
pkg/parser/schemas/main_workflow_schema.json Adds observability.otlp.resource-attributes to the JSON schema.
pkg/parser/schema_test.go Adds schema validation test for resource-attributes.

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

  • Files reviewed: 8/8 changed files
  • Comments generated: 5

var otlpLog = logger.New("workflow:observability_otlp")

var sentryEndpointExpressionPattern = regexp.MustCompile(`(?i)^\$\{\{\s*secrets\.` + regexp.QuoteMeta(constants.OTELSentryEndpointSecretName) + `\s*\}\}$`)
var otlpResourceAttributeSecretRefPattern = regexp.MustCompile(`\$\{\{\s*(secrets|vars)\.`)
Comment on lines +342 to +352
func validateOTLPResourceAttributes(workflowData *WorkflowData) error {
for key, value := range getOTLPResourceAttributes(workflowData) {
if otlpResourceAttributeSecretRefPattern.MatchString(value) {
return fmt.Errorf(
"observability.otlp.resource-attributes.%s must not reference secrets.* or vars.*; OTEL resource attributes are exported to tracing backends and are not treated as secret values",
key,
)
}
}
return nil
}
Comment on lines +837 to +844
func formatOTELResourceAttribute(key, value string) string {
trimmedKey := strings.TrimSpace(key)
trimmedValue := strings.TrimSpace(value)
if strings.Contains(trimmedValue, "${{") {
return encodeOTELResourceAttributeValue(trimmedKey) + "=" + trimmedValue
}
return encodeOTELResourceAttributeValue(trimmedKey) + "=" + encodeOTELResourceAttributeValue(trimmedValue)
}
Comment on lines +863 to 873
resourceAttrs := getOTLPResourceAttributes(workflowData)
if len(resourceAttrs) > 0 {
keys := make([]string, 0, len(resourceAttrs))
for key := range resourceAttrs {
keys = append(keys, key)
}
sort.Strings(keys)
for _, key := range keys {
attrs = append(attrs, formatOTELResourceAttribute(key, resourceAttrs[key]))
}
}
Comment on lines +543 to +547
assert.Contains(
t,
wd.Env,
"OTEL_RESOURCE_ATTRIBUTES: 'gh-aw.workflow.name=triage%20weekly,gh-aw.repository=${{ github.repository }},gh-aw.run.id=${{ github.run_id }},github.run_id=${{ github.run_id }},gh-aw.engine.id=copilot,my.actor=${{ github.actor }},my.target%20repo=owner%2Frepo%2Cweekly'",
)
@github-actions

Copy link
Copy Markdown
Contributor

@copilot review all comments and address unresolved review feedback for this PR.

Generated by 👨‍🍳 PR Sous Chef ·

@github-actions github-actions Bot added the otel label Jun 16, 2026
@github-actions

Copy link
Copy Markdown
Contributor

🔭 OTel Triage — This PR introduces configurable OTLP resource attributes (observability). Per customer triage rules, it has been labeled otel and is routed to @mnkeifer for review.

Generated by 🔧 PR Triage Agent ·

@@ -290,11 +302,12 @@ func applyMergedRawObservability(
rawFrontmatter map[string]any,
mergedEndpoints []any,
mergedAttrs map[string]string,

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot check that the next feature is not just a duplicate of mergeAttrs

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Confirmed — mergeOTLPCustomAttributes was indeed just a one-line wrapper around mergeOTLPStringMaps and added no value. Removed it and updated both the mergeImportedObservability call site and the test to use mergeOTLPStringMaps directly, so both span-attribute and resource-attribute merges now use the same function consistently (commit Remove redundant mergeOTLPCustomAttributes wrapper).

…ngMaps directly

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copilot AI requested a review from pelikhan June 16, 2026 20:23
@pelikhan

Copy link
Copy Markdown
Collaborator

@copilot review existing otel attribute to see if it wasn't supported yet

Copilot AI commented Jun 16, 2026

Copy link
Copy Markdown
Contributor Author

@copilot review existing otel attribute to see if it wasn't supported yet

The existing observability.otlp.attributes is a different, pre-existing feature — it carries custom per-span attributes via GH_AW_OTLP_ATTRIBUTES (a JSON env var consumed by the gh-aw runtime to attach attributes to individual spans). This PR adds observability.otlp.resource-attributes, which is a distinct OTel concept: resource attributes describe the emitting resource and are injected into OTEL_RESOURCE_ATTRIBUTES, which is inherited by all spans including those from child OTel SDKs (Copilot CLI, MCP gateway, etc.). No duplication — both fields are needed and serve different OTel layers.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants