IDEX l1b: filter upstream dependencies correctly before processing#3094
Merged
lacoak21 merged 2 commits intoApr 28, 2026
Merged
Conversation
tech3371
approved these changes
Apr 28, 2026
1449521
into
IMAP-Science-Operations-Center:dev
14 checks passed
maxinelasp
pushed a commit
to maxinelasp/imap_processing
that referenced
this pull request
May 1, 2026
…MAP-Science-Operations-Center#3094) * idex updated l1b logic * fix idex test
lacoak21
added a commit
to lacoak21/imap_processing
that referenced
this pull request
May 4, 2026
…MAP-Science-Operations-Center#3094) * idex updated l1b logic * fix idex test
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Change Summary
closes #3092
Overview
This issue is caused by this problem #2843.
Currently IDEX packets
This is a pretty major problem because our infrastructure is not built to deal with this.
The issue:
Since IDEX l1b processing needs spice kernels, I added a buffer time in the upstream dependency querying logic for l1b jobs only. This ensures that we have spice coverage but also provides the job with extraneous l1a files. I thought I was handling this correctly by choosing the l1a file with the greatest epoch value. This was returning the incorrect l1a file for processing because we should really be using the file with the start date of the job. I thought these two would always be the same but Jamey noticed a case where a file with an older start date has a more recent epoch value! This is due to the jumbling of IDEX events (I think) because of the issue described in #2843. Once this issue is resolved, this will no longer be a problem but as a small fix, I can pass the correct file to l1b processing by finding the one with the same start date.
EXAMPLE:
The max epoch value for the following files will illuminate the issue
File with start date 20251217 has a greater max epoch than teh file with 20251218
File changes
imap_processing/cli.py
Fix file sorting logic
Testing
This was tested with the fie Jamey mentioned in this email.