improve: related resource reference change#3425
Conversation
There was a problem hiding this comment.
Pull request overview
This PR improves handling of secondary→primary reference changes by extending the primary-to-secondary index update semantics, adding targeted tests (unit + integration), and introducing a new sample scenario demonstrating reconciliations when a secondary’s referenced primary set changes.
Changes:
- Updated the informer eventing/indexing pipeline so updates can reconcile both newly referenced and no-longer-referenced primaries.
- Added new core tests covering narrowing/moving references across updates.
- Added a new operator-framework integration test + sample resources demonstrating subset reference changes.
Reviewed changes
Copilot reviewed 15 out of 15 changed files in this pull request and generated 7 comments.
Show a summary per file
| File | Description |
|---|---|
| operator-framework/src/test/java/io/javaoperatorsdk/operator/baseapi/secondarytoprimaryreferencechange/TargetStatus.java | Adds a simple status model used by the sample primary CR. |
| operator-framework/src/test/java/io/javaoperatorsdk/operator/baseapi/secondarytoprimaryreferencechange/TargetReconciler.java | Sample reconciler driven by a referencing secondary CR via an informer event source. |
| operator-framework/src/test/java/io/javaoperatorsdk/operator/baseapi/secondarytoprimaryreferencechange/TargetCustomResource.java | Sample primary custom resource type for the new scenario. |
| operator-framework/src/test/java/io/javaoperatorsdk/operator/baseapi/secondarytoprimaryreferencechange/SecondaryToPrimaryReferenceChangeIT.java | New IT demonstrating subset reference changes and expected reconciliation behavior. |
| operator-framework/src/test/java/io/javaoperatorsdk/operator/baseapi/secondarytoprimaryreferencechange/ConfigToTargetMapper.java | Secondary→primary mapper for the sample that maps config.targetNames to targets. |
| operator-framework/src/test/java/io/javaoperatorsdk/operator/baseapi/secondarytoprimaryreferencechange/ConfigSpec.java | Spec model for the sample secondary CR, including targetNames and value. |
| operator-framework/src/test/java/io/javaoperatorsdk/operator/baseapi/secondarytoprimaryreferencechange/ConfigCustomResource.java | Sample secondary custom resource type. |
| operator-framework-core/src/test/java/io/javaoperatorsdk/operator/processing/event/source/informer/PrimaryToSecondaryIndexTest.java | Adds tests verifying index updates remove obsolete primaries on update. |
| operator-framework-core/src/test/java/io/javaoperatorsdk/operator/processing/event/source/informer/InformerEventSourceTest.java | Updates mocks/verifications for new index method signatures and delete behavior. |
| operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/source/SecondaryToPrimaryMapper.java | Updates mapper Javadoc wording. |
| operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/source/informer/PrimaryToSecondaryIndex.java | Changes index callbacks to return affected primary IDs. |
| operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/source/informer/NOOPPrimaryToSecondaryIndex.java | Adapts NOOP implementation to new return types. |
| operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/source/informer/InformerEventSource.java | Uses index-returned primaries to drive reconciliation propagation and unions old/new mappings. |
| operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/source/informer/DefaultPrimaryToSecondaryIndex.java | Tracks and removes obsolete primaries on update; returns affected primaries for reconciliation. |
| docs/content/en/docs/documentation/eventing.md | Documents secondary→primary mapping behavior (currently inconsistent with the API). |
| * Maps a secondary resource to the set of primary resources that should be reconciled in | ||
| * response. | ||
| * | ||
| * @param resource the secondary resource for which an event was received | ||
| * @return set of primary resource IDs to enqueue for reconciliation; an empty set means the event | ||
| * is irrelevant and no reconciliation is triggered. This is called also the old and the new | ||
| * resources, in case of an update. |
| if (primaryResourceIdSet == null) { | ||
| primaryResourceIdSet = new HashSet<>(); | ||
| primaryResourceIdSet.addAll( | ||
| configuration().getSecondaryToPrimaryMapper().toPrimaryResourceIDs(resource)); | ||
| if (oldResource != null) { |
There was a problem hiding this comment.
This is relevant will take a look
8170175 to
74c83aa
Compare
| @Override | ||
| public void onAddOrUpdate(R resource) { | ||
| // empty method because of noop implementation | ||
| public Set<ResourceID> onAddOrUpdate(R resource, R oldResource) { | ||
| return null; | ||
| } | ||
|
|
||
| @Override | ||
| public void onDelete(R resource) { | ||
| public Set<ResourceID> onDelete(R resource) { | ||
| // empty method because of noop implementation | ||
| return null; | ||
| } |
There was a problem hiding this comment.
THis is relevant, but will need a refactoring to remove, will get rid of it in a subsequent PR
|
I updated my reproducer and it works as expected (dvob/java-operator-sdk-repo-example@1ce1404). Also I like that with this approach we no longer need a special wiring. @csviri thanks a lot for your effort! |
| public void onAddOrUpdate(R resource) { | ||
| // empty method because of noop implementation | ||
| public Set<ResourceID> onAddOrUpdate(R resource, R oldResource) { | ||
| return null; |
There was a problem hiding this comment.
This is very ugly, might be better to remove the NOOP index
…urce This might needed in some corner cases where might help with optimizations to which resource to trigger. Signed-off-by: Attila Mészáros <a_meszaros@apple.com>
Signed-off-by: Attila Mészáros <a_meszaros@apple.com>
Signed-off-by: Attila Mészáros <a_meszaros@apple.com>
Signed-off-by: Attila Mészáros <a_meszaros@apple.com>
Signed-off-by: Attila Mészáros <a_meszaros@apple.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> Signed-off-by: Attila Mészáros <a_meszaros@apple.com>
Signed-off-by: Attila Mészáros <a_meszaros@apple.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> Signed-off-by: Attila Mészáros <a_meszaros@apple.com>
dcaa6c0 to
22e4a58
Compare
Thank you @dvob for feedback and highlighting this use case, that helps a lot! |
|
Can we focus this PR on what it needs to accomplish instead of also renaming methods and changing their visibility (if possible)? |
Signed-off-by: Attila Mészáros <a_meszaros@apple.com>
Signed-off-by: Attila Mészáros <a_meszaros@apple.com>
yep, np, I did |
When there is a resource that references (not owner reference) one or more custom resources (see sample), and the reference changes, we want to trigger reconciliation for both old and new resources referenced.
This PR fixes it by calling also the SeondaryToPrimary mapper twice (also for old resource) on update operation.
It adjusts also
PrimaryToSecondaryIndexso when a reference changes it removed from the secondary from the old primary. That is a more strict, but correct behavior.We did not have this issue for now, since usually the reference for (typically) "config resources" that serve as an input for the reconciliation (among the spec) is usually is other way around.
Note that for this use case
PrimaryToSecondaryMapperis NOT needed, since basically this is very similar to classic owner reference scenario.TODO: