Add configurable OTLP resource attributes to workflow observability#39636
Conversation
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
Note
Copilot was unable to run its full agentic suite in this review.
Adds support for workflow-defined OTLP resource attributes so custom metadata is appended to OTEL_RESOURCE_ATTRIBUTES (without replacing built-ins), including schema/frontmatter support, imported-workflow merge behavior, and safety validation to prevent leaking secrets.* / vars.*.
Changes:
- Adds
observability.otlp.resource-attributesto schema + typed frontmatter parsing. - Appends sorted user resource attributes to generated
OTEL_RESOURCE_ATTRIBUTESwith percent-encoding (static values) while preserving expressions. - Merges imported/main workflow resource attributes with main-workflow precedence and validates that values don’t reference
secrets.*/vars.*.
Show a summary per file
| File | Description |
|---|---|
| pkg/workflow/observability_otlp.go | Implements resource-attribute collection, merge helpers, validation, and OTEL_RESOURCE_ATTRIBUTES formatting/appending. |
| pkg/workflow/observability_otlp_test.go | Adds unit tests for injection, parsing, collection, and secrets/vars validation. |
| pkg/workflow/frontmatter_types.go | Extends OTLPConfig with a ResourceAttributes map and documentation. |
| pkg/workflow/compiler_validators.go | Hooks resource-attribute validation into the compiler validation pipeline. |
| pkg/workflow/compiler_orchestrator_workflow.go | Merges imported/main resource-attributes into raw frontmatter. |
| pkg/workflow/compiler_orchestrator_workflow_test.go | Tests imported observability merge behavior for resource attributes. |
| pkg/parser/schemas/main_workflow_schema.json | Adds observability.otlp.resource-attributes to the JSON schema. |
| pkg/parser/schema_test.go | Adds schema validation test for resource-attributes. |
Copilot's findings
Tip
Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- Files reviewed: 8/8 changed files
- Comments generated: 5
| var otlpLog = logger.New("workflow:observability_otlp") | ||
|
|
||
| var sentryEndpointExpressionPattern = regexp.MustCompile(`(?i)^\$\{\{\s*secrets\.` + regexp.QuoteMeta(constants.OTELSentryEndpointSecretName) + `\s*\}\}$`) | ||
| var otlpResourceAttributeSecretRefPattern = regexp.MustCompile(`\$\{\{\s*(secrets|vars)\.`) |
| func validateOTLPResourceAttributes(workflowData *WorkflowData) error { | ||
| for key, value := range getOTLPResourceAttributes(workflowData) { | ||
| if otlpResourceAttributeSecretRefPattern.MatchString(value) { | ||
| return fmt.Errorf( | ||
| "observability.otlp.resource-attributes.%s must not reference secrets.* or vars.*; OTEL resource attributes are exported to tracing backends and are not treated as secret values", | ||
| key, | ||
| ) | ||
| } | ||
| } | ||
| return nil | ||
| } |
| func formatOTELResourceAttribute(key, value string) string { | ||
| trimmedKey := strings.TrimSpace(key) | ||
| trimmedValue := strings.TrimSpace(value) | ||
| if strings.Contains(trimmedValue, "${{") { | ||
| return encodeOTELResourceAttributeValue(trimmedKey) + "=" + trimmedValue | ||
| } | ||
| return encodeOTELResourceAttributeValue(trimmedKey) + "=" + encodeOTELResourceAttributeValue(trimmedValue) | ||
| } |
| resourceAttrs := getOTLPResourceAttributes(workflowData) | ||
| if len(resourceAttrs) > 0 { | ||
| keys := make([]string, 0, len(resourceAttrs)) | ||
| for key := range resourceAttrs { | ||
| keys = append(keys, key) | ||
| } | ||
| sort.Strings(keys) | ||
| for _, key := range keys { | ||
| attrs = append(attrs, formatOTELResourceAttribute(key, resourceAttrs[key])) | ||
| } | ||
| } |
| assert.Contains( | ||
| t, | ||
| wd.Env, | ||
| "OTEL_RESOURCE_ATTRIBUTES: 'gh-aw.workflow.name=triage%20weekly,gh-aw.repository=${{ github.repository }},gh-aw.run.id=${{ github.run_id }},github.run_id=${{ github.run_id }},gh-aw.engine.id=copilot,my.actor=${{ github.actor }},my.target%20repo=owner%2Frepo%2Cweekly'", | ||
| ) |
|
@copilot review all comments and address unresolved review feedback for this PR.
|
|
🔭 OTel Triage — This PR introduces configurable OTLP resource attributes (observability). Per customer triage rules, it has been labeled
|
| @@ -290,11 +302,12 @@ func applyMergedRawObservability( | |||
| rawFrontmatter map[string]any, | |||
| mergedEndpoints []any, | |||
| mergedAttrs map[string]string, | |||
There was a problem hiding this comment.
@copilot check that the next feature is not just a duplicate of mergeAttrs
There was a problem hiding this comment.
Confirmed — mergeOTLPCustomAttributes was indeed just a one-line wrapper around mergeOTLPStringMaps and added no value. Removed it and updated both the mergeImportedObservability call site and the test to use mergeOTLPStringMaps directly, so both span-attribute and resource-attribute merges now use the same function consistently (commit Remove redundant mergeOTLPCustomAttributes wrapper).
…ngMaps directly Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
|
@copilot review existing otel attribute to see if it wasn't supported yet |
The existing |
Workflows could configure OTLP endpoints and headers, but not custom OTel resource attributes. This adds a first-class
observability.otlp.resource-attributesfield that is appended to the generatedOTEL_RESOURCE_ATTRIBUTES, so workflow-defined metadata is carried across gh-aw and engine-emitted spans.Schema and frontmatter support
observability.otlp.resource-attributesto the workflow schema and typed frontmatter parsing.OTEL resource attribute generation
OTEL_RESOURCE_ATTRIBUTESrather than replacing the built-in gh-aw attributes.Safety guardrails
secrets.*andvars.*references in resource attribute values.Imported workflow behavior
resource-attributeswith main-workflow precedence, matching the existing observability merge model.Example:
This compiles into
OTEL_RESOURCE_ATTRIBUTESalongside the existing gh-aw attributes, with static values percent-encoded and expression values preserved for runtime evaluation.