feat(config): add resource detection parsing#6435
feat(config): add resource detection parsing#6435trentm merged 16 commits intoopen-telemetry:mainfrom
Conversation
- Fix ExperimentalResourceDetection.detectors type from object to array (per spec schema) - Add parseDetectionDevelopment() to FileConfigFactory to parse detection/development block from YAML, including attribute include/exclude filters - Map OTEL_NODE_RESOURCE_DETECTORS env var to detection/development.detectors (container, host, process, serviceinstance→service; skips JS-only env/os) - Update configFromKitchenSinkFile test expectation with detection/development - Add env var tests for all, none, and JS-only detector cases Claude Sonnet 4.6 assisted with this change.
Claude Sonnet 4.6 assisted with this change.
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #6435 +/- ##
=======================================
Coverage 95.73% 95.74%
=======================================
Files 364 364
Lines 11826 11871 +45
Branches 2765 2789 +24
=======================================
+ Hits 11322 11366 +44
- Misses 504 505 +1
🚀 New features to boost your workflow:
|
…o mike/resource-detection-parsing
config.resource is always initialised before the detection/development block is parsed, so the null check was dead code causing a codecov miss. Use non-null assertion instead. Claude Sonnet 4.6 assisted with this change.
Consistent with the rest of FileConfigFactory which uses for loops rather than filter/map chains. Claude Sonnet 4.6 assisted with this change.
…o mike/resource-detection-parsing
|
Thank you for working on this! When I initially created these parsers, there was no place to add the node resource attributes, so I created the parameter
We don't want to lose any existing functionality, so I would not mark the issue itself as fixed until all items are added. You can have a PR on the config repo to update the schema to include the missing ones, and then add them into here on a following PR. Or you can hold on to this PR until the schema is updated and have it all here. It's up to you, either way works |
…detection/development model - Remove node_resource_detectors field from ConfigurationModel; detection/development.detectors now owns this data - Remove redundant constructor code in EnvironmentConfigFactory that set node_resource_detectors - Update getResourceDetectorsFromConfiguration() in sdk-node to map ExperimentalResourceDetector objects to detector instances - Update setupResource() to check detection/development.detectors instead of node_resource_detectors - Update start.test.ts: os detector is JS-only with no spec equivalent, not included via config model path - Add unit tests for getResourceDetectorsFromConfiguration() covering all detector types and edge cases Made-with: Cursor
|
Not sure the test failure is related to my changes - could someone re-run it for me please? |
Made-with: Cursor
|
Never mind - merging main into this branch fixed it 😄 |
- Add `os` and `env` to `ExperimentalResourceDetector` as Node.js-specific extensions (no spec equivalent) - Map `os`/`env` in `EnvironmentConfigFactory` OTEL_NODE_RESOURCE_DETECTORS parsing, including `all` - Handle `os`/`env` in `FileConfigFactory` `parseDetectionDevelopment()` - Map `detector.os`/`detector.env` to `osDetector`/`envDetector` in `getResourceDetectorsFromConfiguration()` - Update tests and kitchen-sink fixture - Note Node.js-only status in sdk-node README Assisted-by: Claude Sonnet 4.6
os is now included in OTEL_NODE_RESOURCE_DETECTORS=all via config model Assisted-by: Claude Sonnet 4.6
maryliag
left a comment
There was a problem hiding this comment.
Thanks for adding the missing ones, it makes things easier 😄
maryliag
left a comment
There was a problem hiding this comment.
removing the approval, just so it doesn't get merged accidentally until the changes are made
env detector must run last so its attributes take highest priority over other detectors Assisted-by: Claude Sonnet 4.6
Assisted-by: Claude Sonnet 4.6
Assisted-by: Claude Sonnet 4.6
| if (detector.host != null) result.push(hostDetector); | ||
| if (detector.os != null) result.push(osDetector); | ||
| if (detector.process != null) result.push(processDetector); | ||
| if (detector.service != null) result.push(serviceInstanceIdDetector); |
There was a problem hiding this comment.
We are allowed to have a breaking change in v3. I wonder if we want to change our (currently experimental, I think?) serviceInstanceIdDetector and deprecate it in favour of a serviceDetector that does the same thing.
(I'm not sure if the intent is that OTEL_SERVICE_NAME is handled by service or by env.)
There was a problem hiding this comment.
Agree this is worth discussing, but feels like a separate concern from this PR. Happy to open a follow-up issue to track renaming serviceInstanceIdDetector → serviceDetector for v3 if that's the direction.
There was a problem hiding this comment.
Absolutely a separate PR kind of thing.
I'm curious for @maryliag's opinion. I.e. whether changes in OTel JS here are worth it to have a service resource detector be closer in naming and scope to other languages (or at least closer to what I assume OTel Java does).
There was a problem hiding this comment.
I like the idea of a serviceDetector that handles everything service related, meaning instance ID and name, and making it clear the priority of where the value comes from, so it's worth opening a new issue/PR to handle that.
To answer your other question:
OTEL_SERVICE_NAME is handled by service or by env
I think being handled by the service would make more obvious to users, because is in the name, and we can make the priority clear of where the value comes from.
Using a serviceDetector would also help with the decision between the random uuid and defined service instance ID, where today depending on the order the users adds (serviceInstanceId, env VS env, serviceInstanceId) it returns different results.
That would be a breaking change, so as long as we make that clear to users, then should be fine
There was a problem hiding this comment.
Thanks both - I like the idea of replacing serviceInstanceDetecto -> serviceDetector and simplifying to cover name and ID.
I opened the follow-up issue to track doing that
There was a problem hiding this comment.
FWIW, I spent some time looking at OTel Java and Python implementations, and ... gave up after a little bit. Python doesn't have anything for service.instance.id I don't think, yet.
OTel Java has a ServiceInstanceIdResourceProvider (https://github.com/open-telemetry/opentelemetry-java/blob/main/sdk-extensions/incubator/src/main/java/io/opentelemetry/sdk/extension/incubator/resources/ServiceInstanceIdResourceProvider.java#L20-L47). It and the EnvironmentResourceProvider are using .order() SPI things (with values -1 and MAX_INT) to handle priority.
It also has a ServiceResourceDetector (with name "service") in the declarative config code (https://github.com/open-telemetry/opentelemetry-java/blob/main/sdk-extensions/incubator/src/main/java/io/opentelemetry/sdk/extension/incubator/fileconfig/ServiceResourceDetector.java#L18) that is slightly different in that it handles service.name and service.instance.id.
Anyway, clarity to seek on another day.
There was a problem hiding this comment.
Thanks for the notes @trentm - I've been helping out on the Java implementation and am leading the Python implementation too (which is still very early).
I'd like to consolidate on a consistent pattern too and will keep an eye on it the linked issue 👍🏻
- Fix service detector JSDoc: only populates service.instance.id, not service.name - Move CHANGELOG entry to Unreleased section - Sort detectors alphabetically in kitchen-sink.yaml (order is user-controlled in YAML configs) Assisted-by: Claude Sonnet 4.6
- resolve conflicts from PR open-telemetry#6435 (resource detection parsing) - keep Zod-based FileConfigFactory, drop upstream's manual parser - port OTEL_NODE_RESOURCE_DETECTORS env var parsing into EnvironmentConfigFactory - fix null-check pattern: use !== undefined so YAML-produced null values work - rename OTEL_EXPERIMENTAL_CONFIG_FILE → OTEL_CONFIG_FILE - fix TS7056: annotate OpenTelemetryConfigurationSchema as ZodTypeAny - replace z.infer<> in types.ts with explicit Configuration interface - fix sdk-node type refs: ConfigurationModel → Configuration - fix sdk-node test fixture logger.yaml TLS field names - all 53 configuration tests and 150 sdk-node tests passing Assisted-by: Claude Sonnet 4.6
Which problem is this PR solving?
Fixes #5980
RC3 spec requires parsing of
detection/developmentblock in declarative config. Currently:ExperimentalResourceDetection.detectorsis incorrectly typed as a single object; the spec schema defines it as an arrayFileConfigFactorynorEnvironmentConfigFactoryparse thedetection/developmentblockShort description of the changes
ExperimentalResourceDetection.detectorstype fromExperimentalResourceDetectortoExperimentalResourceDetector[]to match spec schemaparseDetectionDevelopment()toFileConfigFactoryto parse thedetection/developmentblock from YAML config files, including detector list and attribute include/exclude filtersOTEL_NODE_RESOURCE_DETECTORSenv var todetection/development.detectorsinEnvironmentConfigFactory(container, host, process, serviceinstance→service, env, os)envandosas Node.js-specific extensions toExperimentalResourceDetector(no spec equivalent; supported under the experimentaldetection/developmentblock)node_resource_detectorsfromConfigurationModel;detection/development.detectorsnow owns this datagetResourceDetectorsFromConfiguration()in sdk-node to mapExperimentalResourceDetectorobjects to detector instancesType of change
How Has This Been Tested?
configFromKitchenSinkFileexpected config to includedetection/developmentwith all detectors includingenvandosall,none, specific detectors includingenvandosgetResourceDetectorsFromConfiguration()covering all detector types and edge casesChecklist:
Assisted-by: Claude Sonnet 4.6