Handling for duplicate packet fragments in idex_l1a.py#3222
Conversation
There was a problem hiding this comment.
Pull request overview
This PR updates IDEX L1A science packet parsing to better handle retransmitted/duplicate science packet fragments, preventing duplicate fragments from corrupting waveform assembly and ensuring conflicting duplicates can be detected and skipped.
Changes:
- Introduces a stable
EventKeyand updates science event tracking to avoid collisions when grouping packets into events. - Reworks
RawDustEventfragment handling to de-duplicate fragment slots (drop exact duplicates, prefer longer retransmits, and skip events on conflicting duplicates). - Adds targeted tests covering exact-duplicate, shorter/longer duplicate, and conflicting duplicate fragment cases.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
imap_processing/idex/idex_l1a.py |
Implements fragment-slot de-duplication and event-keyed science event assembly logic. |
imap_processing/tests/idex/test_idex_l1a.py |
Adds regression tests for duplicate fragment handling and conflict behavior. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| scitype = Scitype(int(packet["IDX__SCI0TYPE"])) | ||
| fragoff = int(packet["IDX__SCI0FRAGOFF"]) | ||
| raw_fragment = bytes(packet["IDX__SCI0RAW"]) | ||
| fragment_slot = (int(scitype), fragoff) | ||
| stored_fragments = self.fragments_by_scitype[scitype] |
| Dataset containing processed dust events. | ||
| """ | ||
| dust_events = {} | ||
| dust_events: dict[EventKey, RawDustEvent] = {} |
|
|
||
| self.event_number = header_packet["IDX__SCI0EVTNUM"] | ||
| self.event_number = int(header_packet["IDX__SCI0EVTNUM"]) | ||
| self.event_key = self.get_event_key(header_packet) |
There was a problem hiding this comment.
Instead of calling this twice maybe you should just add event_key as a constructor parameter to RawDustEvent
| self.Target_Low_bits = "" | ||
| self.Target_High_bits = "" | ||
| self.Ion_Grid_bits = "" | ||
| # Keep one fragment per (science type, fragment offset) slot and assemble the |
There was a problem hiding this comment.
I dont know enough about this. What is a fragment offset? Can you add more info about this.
Change Summary
Overview
Patch l1a to handle duplicate pkts segments.
File changes
idex_l1a.py, test_idex_l1a.py
Testing
processed locally on 20260513 v001 and v002 pkts files.