Skip to content

3088 hi l1c esa dependent backgrounds#3140

Merged
tmplummer merged 4 commits into
IMAP-Science-Operations-Center:devfrom
tmplummer:3088-hi-l1c---esa-dependent-backgrounds
May 5, 2026
Merged

3088 hi l1c esa dependent backgrounds#3140
tmplummer merged 4 commits into
IMAP-Science-Operations-Center:devfrom
tmplummer:3088-hi-l1c---esa-dependent-backgrounds

Conversation

@tmplummer
Copy link
Copy Markdown
Contributor

Change Summary

Summary

Adds ESA-dependent background scaling to IMAP-Hi L1C processing. Background counts are still computed using the same TOF windows across all ESA energy steps, but each ESA step now has its own scaling_factor and uncertainty values.

Key Changes

imap_processing/hi/utils.py

  • Updated BackgroundConfig index to include esa_energy_step as a third level: (calibration_prod, background_index, esa_energy_step)
  • Added _validate() override to ensure TOF windows and coincidence types are consistent across ESA energy steps within each (calibration_prod, background_index) group
  • Added get_tof_config() method to return a 2-level index DataFrame for counting (since TOF windows are shared across ESA steps)
  • Added forward-fill support in from_csv() for sparse CSV format where TOF columns only need to be specified on the first row of each group

imap_processing/hi/hi_l1c.py

  • Updated _compute_background_counts() to use get_tof_config() for event counting
  • Updated pset_backgrounds() to apply ESA-specific scaling factors and uncertainties
  • Added validation to ensure ESA energy steps in config match PSET coordinates
  • Used dynamic dimension ordering from output variables for transpose operations

imap_processing/tests/hi/data/l1/imap_hi_90sensor-backgrounds_20240101_v001.csv

  • Updated test calibration file format to include esa_energy_step with ESA-varying scaling factors
  • Reordered columns for readability (scaling_factor, uncertainty before TOF columns)
  • Uses sparse format with forward-fill for repeated TOF values

Tests

  • Added tests for TOF consistency validation (passes, fails on different TOF windows, fails on different coincidence types)
  • Added test for get_tof_config() method
  • Added test for forward-fill functionality
  • Added test verifying different backgrounds are applied per ESA step
  • Added test for ESA energy step mismatch error

Closes: #3088

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds ESA-energy-step–dependent background scaling to IMAP-Hi L1C processing while keeping background event counting based on TOF/coincidence windows shared across ESA steps. This updates the background config schema and adjusts L1C background application logic accordingly.

Changes:

  • Extend Hi background config indexing to include esa_energy_step, add cross-ESA TOF/coincidence consistency validation, add get_tof_config(), and support sparse CSV via forward-fill.
  • Update Hi L1C background counting to use the shared TOF config and apply scaling/uncertainty per ESA step (with additional coordinate validation and safer dimension handling).
  • Update test background CSV format and expand unit tests for new validation/helpers and ESA-dependent behavior.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
imap_processing/hi/utils.py Extend BackgroundConfig to be ESA-indexed; add TOF consistency validation, get_tof_config(), and sparse CSV forward-fill support.
imap_processing/hi/hi_l1c.py Use TOF-only config for background counting; apply ESA-dependent scaling/uncertainty and validate coords; transpose/broadcast using output dims.
imap_processing/tests/hi/test_utils.py Add/adjust tests for new BackgroundConfig behavior (3-level index, validation, get_tof_config, forward-fill).
imap_processing/tests/hi/test_hi_l1c.py Extend tests to verify ESA-dependent background outputs and new mismatch validation.
imap_processing/tests/hi/data/l1/imap_hi_90sensor-backgrounds_20240101_v001.csv Update test ancillary file format to include esa_energy_step and sparse rows for TOF/coincidence fields.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread imap_processing/hi/utils.py
Comment thread imap_processing/hi/utils.py
@tmplummer tmplummer force-pushed the 3088-hi-l1c---esa-dependent-backgrounds branch from 8d5ae3a to ae6254b Compare May 5, 2026 15:20
Copy link
Copy Markdown
Contributor

@laspsandoval laspsandoval left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

Comment thread imap_processing/hi/utils.py
@tmplummer tmplummer merged commit b10319a into IMAP-Science-Operations-Center:dev May 5, 2026
14 checks passed
@github-project-automation github-project-automation Bot moved this to Done in IMAP May 5, 2026
@tmplummer tmplummer deleted the 3088-hi-l1c---esa-dependent-backgrounds branch May 5, 2026 22:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

Hi L1C - esa dependent backgrounds

3 participants