Skip to content

Eliminate xarray->numpy->xarray conversion in lo_l2 - #3371

Open
jaredclaypoole wants to merge 2 commits into
IMAP-Science-Operations-Center:devfrom
jaredclaypoole:issue3357
Open

Eliminate xarray->numpy->xarray conversion in lo_l2#3371
jaredclaypoole wants to merge 2 commits into
IMAP-Science-Operations-Center:devfrom
jaredclaypoole:issue3357

Conversation

@jaredclaypoole

Copy link
Copy Markdown

Closes #3357

  • Also had to add mypy error suppression in an unrelated class definition in lo_l2.py, because mypy is set to ignore typing from out-of-module imports

Change Summary

File changes

In imap_processing/lo/l2/lo_l2.py:

  • Function _calculate_rates_and_intensities
    • Change return type from dict[str, np.ndarray] to dict[str, xr.DataArray]
    • No need to introduce a newaxis for energy, geometric_factor, gf_low, and gf_high arrays
      • This is because they are named dimensions in xarray, so the broadcasting is handled automatically
    • Change _divide helper to take, operate on, and return DataArrays instead of numpy arrays
  • Function _build_map_dataset
    • change variables parameter type to match the output of _calculate_rates_and_intensities
  • Class LoSpinAnglePointingSet
    • Added #type: ignore[misc] to prevent a mypy error:
      • Mypy error: imap_processing/lo/l2/lo_l2.py:441: error: Class cannot subclass "PointingSet" (has type "Any") [misc]
      • Apparently this happens because the pyproject.toml's mypy configuration has settings strict = true and follow_imports = skip
        • This means types aren't imported from other modules, so mypy thinks PointingSet (which was defined in another module) has type Any, meaning strict = True prevents it from being subclassed

* Also had to add mypy error suppression in an unrelated class definition
  in lo_l2.py, because mypy is set to ignore typing from out-of-module
  imports
@jaredclaypoole

Copy link
Copy Markdown
Author

Okay, after bypassing some pre-commit weirdness I no longer have to use a type: ignore[misc] comment. That's good, because it was completely irrelevant to the changes I intended to make.

@tmplummer tmplummer left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall, this looks like a good improvement. I'd like @vineetbansal to look at this too.

out=np.zeros_like(exposure),
where=exposed,
)
return (numerator / denominator.where(exposed)).where(exposed, 0)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this can be simplified by calling xr.where directly:

Suggested change
return (numerator / denominator.where(exposed)).where(exposed, 0)
return xr.where(exposed, numerator / denominator, 0)

f"error bound; their systematic errors are left at zero."
)
intensity_upper = _divide(count_rate, np.where(valid, gf_low, 1.0) * energy)
intensity_upper = _divide(count_rate, gf_low.where(valid, 1.0) * energy)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With the simplification of _divide it seems like this code might be easier to read by directly implementing the division using the where function here. What do you think. IMO, it would make it clearer that it is using the exposed mask. For exampe:

intensity_upper = xr.where(exposed, count_rate / (gf_low.where(valid, 1.0) * energy), 0)

Just a thought.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Lo - Round-trip between numpy/xarray not needed

2 participants