Skip to content

2769 hi goodtimes fix esa step field in output txt file#2863

Merged
tmplummer merged 5 commits into
IMAP-Science-Operations-Center:devfrom
tmplummer:2769-hi-goodtimes---fix-esa_step-field-in-output-txt-file
Mar 24, 2026
Merged

2769 hi goodtimes fix esa step field in output txt file#2863
tmplummer merged 5 commits into
IMAP-Science-Operations-Center:devfrom
tmplummer:2769-hi-goodtimes---fix-esa_step-field-in-output-txt-file

Conversation

@tmplummer
Copy link
Copy Markdown
Contributor

@tmplummer tmplummer commented Mar 24, 2026

Summary

This PR adds unique bit-flag cull codes for each goodtimes filter, enabling tracking of which specific checks flagged each time bin. It also includes fixes to the goodtimes txt file format and spin_bin coordinate metadata.

Changes

Unique Cull Codes (Bit Flags)

  • Replaced single CullCode.LOOSE with unique bit flags for each filter:
    • INCOMPLETE_SPIN = 1 (incomplete 8-spin periods)
    • DRF = 2 (drift restabilization)
    • BAD_TDC_CAL = 4 (failed TDC calibration)
    • OVERFLOW = 8 (DE packet overflow)
    • STAT_FILTER_0 = 16 (penetrating background changes)
    • STAT_FILTER_1 = 32 (isotropic count rate increases)
    • STAT_FILTER_2 = 64 (short-lived event pulses)
  • Updated mark_bad_times() to use bitwise OR (|=) so multiple flags can be combined
  • Each filter function now defaults to its specific cull code

Other Fixes

  • Fixed INTERVAL_DTYPE to include cull_value and rename n_good_bins to n_bins
  • Fixed get_good_intervals() to track cull values in intervals
  • Updated dataset attributes from sensor/pointing to Sensor/Repointing for consistency

Closes: #2769

@tmplummer tmplummer requested a review from Copilot March 24, 2026 17:02
@tmplummer tmplummer self-assigned this Mar 24, 2026
@tmplummer tmplummer added this to IMAP Mar 24, 2026
Copilot AI reviewed Mar 24, 2026
View reviewed changes
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates IMAP-Hi goodtimes to (1) use per-filter bit-flag cull codes that can be combined, and (2) fix the goodtimes TXT interval output to represent ESA steps as a 10-element boolean mask (per issue #2769), while also adjusting interval grouping/metadata.

Changes:

  • Replace the single CullCode.LOOSE with per-filter bit flags and combine flags via |= in mark_bad_times().
  • Update interval extraction/output: merge consecutive identical cull patterns across METs; emit esa_step_mask and cull_value (and update the TXT format/tests accordingly).
  • Fix spin-bin coordinate metadata in the HI variable-attrs YAML (fill value, dtype, label dependency).

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 4 comments.

File Description
imap_processing/hi/hi_goodtimes.py Introduces bit-flag cull codes, combines flags with OR, changes interval grouping, and updates TXT output format (including ESA step mask).
imap_processing/tests/hi/test_hi_goodtimes.py Updates and expands tests for bit flags, combined flags behavior, merged intervals, and new TXT format/values.
imap_processing/cdf/config/imap_hi_variable_attrs.yaml Adjusts HI goodtimes spin_bin/spin_bin_label variable metadata (fill value, dtype, DEPEND key).

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread imap_processing/hi/hi_goodtimes.py Outdated
Comment thread imap_processing/hi/hi_goodtimes.py
Comment thread imap_processing/hi/hi_goodtimes.py
Comment thread imap_processing/tests/hi/test_hi_goodtimes.py Outdated
tmplummer and others added 2 commits March 24, 2026 12:53
@tmplummer
Apply suggestions from code review
1b222b4 
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
@tmplummer
Copilot feedback changes
17bccf2
lacoak21
lacoak21 approved these changes Mar 24, 2026
View reviewed changes
Copy link
Copy Markdown
Contributor

@lacoak21 lacoak21 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These changes look solid to me.

Comment thread imap_processing/hi/hi_goodtimes.py
Comment thread imap_processing/hi/hi_goodtimes.py
@lacoak21
Copy link
Copy Markdown
Contributor

lacoak21 commented Mar 24, 2026

I did have one overall question. So there is a goodtimes txt and CDF? Is the Txt basically just for a quicklook?

@tmplummer tmplummer merged commit a0a0092 into IMAP-Science-Operations-Center:dev Mar 24, 2026
14 checks passed
@github-project-automation github-project-automation Bot moved this to Done in IMAP Mar 24, 2026
@tmplummer
Copy link
Copy Markdown
Contributor Author

tmplummer commented Mar 24, 2026

@lacoak21

I did have one overall question. So there is a goodtimes txt and CDF? Is the Txt basically just for a quicklook?

I am including the txt function so that Paul can convert the CDF to a txt file using a command line utility. He has code that parses the txt file and displays data overlayed with goodtimes.

@tmplummer tmplummer deleted the 2769-hi-goodtimes---fix-esa_step-field-in-output-txt-file branch March 24, 2026 21:20
@lacoak21
Copy link
Copy Markdown
Contributor

lacoak21 commented Mar 24, 2026

@lacoak21

I did have one overall question. So there is a goodtimes txt and CDF? Is the Txt basically just for a quicklook?

I am including the txt function so that Paul can convert the CDF to a txt file using a command line utility. He has code that parses the txt file and displays data overlayed with goodtimes.

Ok cool - thats nice to have.

sapols pushed a commit to sapols/imap_processing that referenced this pull request Apr 15, 2026
@tmplummer @sapols
2769 hi goodtimes fix esa step field in output txt file (IMAP-Science…
6bbc8c8 
…-Operations-Center#2863)

* Fix hi goodtimes to txt file format

* Fix goodtimes spin_bin coordinate metadata

* Add cull code for each of the goodtimes checks

* Apply suggestions from code review

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Copilot feedback changes

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

Copilot code review Copilot Copilot left review comments

@lacoak21 lacoak21 lacoak21 approved these changes

@laspsandoval laspsandoval Awaiting requested review from laspsandoval

Assignees

@tmplummer tmplummer

Labels

None yet

Projects

IMAP
Status: Done

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Hi Goodtimes - Fix esa_step field in output txt file

3 participants

@tmplummer @lacoak21