Skip to content

Filter empty HistogramL1A data points in GLOWS L1A processing#2756

Merged
tech3371 merged 3 commits into
devfrom
copilot/filter-empty-l1a-histogram-data
Feb 24, 2026
Merged

Filter empty HistogramL1A data points in GLOWS L1A processing#2756
tech3371 merged 3 commits into
devfrom
copilot/filter-empty-l1a-histogram-data

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented Feb 23, 2026

Some histogram packets contain timestamps with no actual data (NBINS == 0), typically from validation data. These empty entries were being passed through the pipeline and producing spurious output records.

Changes

  • glows_l1a.py: Filter out HistogramL1A objects with number_of_bins_per_histogram == 0 at the top of generate_histogram_dataset before any processing:

    hist_l1a_list = [hist for hist in hist_l1a_list if hist.number_of_bins_per_histogram > 0]
  • test_glows_l1a_cdf.py: Add test_generate_histogram_dataset_filters_empty — mixes mock empty histograms (number_of_bins_per_histogram == 0) with real ones and asserts only the non-empty entries appear in the output dataset.


🔒 GitHub Advanced Security automatically protects Copilot coding agent pull requests. You can protect all pull requests by enabling Advanced Security for your repositories. Learn more about Advanced Security.

…dd test

Co-authored-by: tech3371 <36522642+tech3371@users.noreply.github.com>
Copilot AI changed the title [WIP] Filter GLOWS L1A histogram data points that are empty Filter empty HistogramL1A data points in GLOWS L1A processing Feb 23, 2026
Copilot AI requested a review from tech3371 February 23, 2026 23:43
@tech3371 tech3371 marked this pull request as ready for review February 24, 2026 20:54
@tech3371
Copy link
Copy Markdown
Contributor

pre-commit.ci autofix

@tech3371 tech3371 requested a review from maxinelasp February 24, 2026 21:00
Copy link
Copy Markdown
Contributor

@tech3371 tech3371 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I checked the work locally and added validation notes in the attached ticket of this PR. Looks good

@tech3371 tech3371 merged commit 2683e11 into dev Feb 24, 2026
14 checks passed
@tech3371 tech3371 deleted the copilot/filter-empty-l1a-histogram-data branch February 24, 2026 23:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ENH - Filter GLOWS L1A Histogram data points which are empty.

3 participants