Skip to content

ENH: optimize MaskedArray reduction methods by using the where parameter#31181

Draft
Prtm2110 wants to merge 1 commit intonumpy:mainfrom
Prtm2110:use_where_in_masked
Draft

ENH: optimize MaskedArray reduction methods by using the where parameter#31181
Prtm2110 wants to merge 1 commit intonumpy:mainfrom
Prtm2110:use_where_in_masked

Conversation

@Prtm2110
Copy link
Copy Markdown

@Prtm2110 Prtm2110 commented Apr 7, 2026

PR summary

As mentioned in the title. Directly changed methods are ["sum", "prod", "min", "max", "all", "any"] and out of these significant performance boost is seen in sum and prod for most of the scenarios. For which functions should I remove where arg?

▶ Small  1D float64   N=1k   mask=20%
  Op                New min    New med    Old min    Old med   Speedup(med)  Result
  ---------------------------------------------------------------------------------
  sum               0.0079ms    0.0081ms    0.0066ms    0.0067ms          0.82x   
  prod              0.0079ms    0.0084ms    0.0072ms    0.0072ms          0.86x   
  max               0.0125ms    0.0129ms    0.0066ms    0.0066ms          0.51x   
  min               0.0124ms    0.0126ms    0.0065ms    0.0066ms          0.52x   
  all               0.0111ms    0.0112ms    0.0067ms    0.0068ms          0.61x   
  any               0.0105ms    0.0107ms    0.0067ms    0.0068ms          0.63x   

▶ Medium 1D float64   N=100k mask=20%
  Op                New min    New med    Old min    Old med   Speedup(med)  Result
  ---------------------------------------------------------------------------------
  sum               0.3475ms    0.3574ms    0.3077ms    0.3164ms          0.89x   
  prod              0.3384ms    0.3435ms    0.3601ms    0.3658ms          1.06x   
  max               0.4409ms    0.4475ms    0.2991ms    0.3018ms          0.67x   
  min               0.4378ms    0.4519ms    0.2991ms    0.3055ms          0.68x   
  all               0.4211ms    0.4301ms    0.3076ms    0.3120ms          0.73x   
  any               0.3547ms    0.3638ms    0.3057ms    0.3114ms          0.86x   

▶ Large  1D float64   N=1M   mask=20%
  Op                New min    New med    Old min    Old med   Speedup(med)  Result
  ---------------------------------------------------------------------------------
  sum               3.6068ms    3.7183ms    5.2006ms    5.2888ms          1.42x   
  prod              3.4766ms    3.5215ms    5.3408ms    5.3939ms          1.53x   
  max               4.4872ms    4.5484ms    5.2012ms    5.2639ms          1.16x   
  min               4.4627ms    4.5187ms    3.9981ms    4.0326ms          0.89x   
  all               4.5346ms    4.6265ms    4.0663ms    4.1352ms          0.89x   
  any               3.8997ms    3.9392ms    4.1023ms    4.1200ms          1.05x   

▶ Large  1D float64   N=1M   mask=1%
  Op                New min    New med    Old min    Old med   Speedup(med)  Result
  ---------------------------------------------------------------------------------
  sum               0.9028ms    0.9309ms    1.4672ms    1.4890ms          1.60x   
  prod              1.2549ms    1.2702ms    1.8141ms    1.8315ms          1.44x   
  max               0.9919ms    1.0070ms    1.4527ms    1.4700ms          1.46x   
  min               0.9895ms    1.0089ms    1.4498ms    1.4741ms          1.46x   
  all               1.2289ms    1.2522ms    1.5332ms    1.5531ms          1.24x   
  any               1.0493ms    1.0754ms    1.5125ms    1.5383ms          1.43x   

▶ Large  1D float64   N=1M   mask=80%
  Op                New min    New med    Old min    Old med   Speedup(med)  Result
  ---------------------------------------------------------------------------------
  sum               3.8412ms    3.8604ms    4.0272ms    4.0492ms          1.05x   
  prod              3.5981ms    3.6377ms    4.3019ms    4.3361ms          1.19x   
  max               4.5078ms    4.5771ms    3.9845ms    4.0037ms          0.87x   
  min               4.5201ms    4.5760ms    3.9728ms    4.0031ms          0.87x   
  all               4.4302ms    4.4554ms    4.0797ms    4.1210ms          0.92x   
  any               4.2633ms    4.3046ms    4.0566ms    4.0896ms          0.95x   

▶ Large  1D int32     N=1M   mask=20%
  Op                New min    New med    Old min    Old med   Speedup(med)  Result
  ---------------------------------------------------------------------------------
  sum               3.6090ms    3.6616ms    3.5847ms    3.6151ms          0.99x   
  prod              3.6640ms    3.6853ms    4.2849ms    4.3169ms          1.17x   
  max               3.5813ms    3.5978ms    3.4428ms    3.4725ms          0.97x   
  min               3.6016ms    3.6247ms    3.4490ms    3.4713ms          0.96x   
  all               3.5913ms    3.6224ms    3.5223ms    3.5553ms          0.98x   
  any               3.6266ms    3.6790ms    3.5058ms    3.5506ms          0.97x   

▶ 2D     float64  1k×1k      mask=20%
  Op                New min    New med    Old min    Old med   Speedup(med)  Result
  ---------------------------------------------------------------------------------
  sum               3.4474ms    3.5000ms    4.0155ms    4.0386ms          1.15x   
  prod              3.3033ms    3.3368ms    4.2999ms    4.3191ms          1.29x   
  max               4.3165ms    4.3331ms    3.9876ms    4.0043ms          0.92x   
  min               4.3145ms    4.3820ms    4.0072ms    4.0422ms          0.92x   
  all               4.4943ms    4.5561ms    4.0866ms    4.1338ms          0.91x   
  any               3.8819ms    3.9308ms    4.0865ms    4.1126ms          1.05x   
  sum(axis=0)       3.5954ms    3.6128ms    4.0210ms    4.0521ms          1.12x   
  max(axis=0)       4.6133ms    4.6380ms    4.0950ms    4.1164ms          0.89x   

▶ No mask 1D float64  N=1M   mask=0%
  Op                New min    New med    Old min    Old med   Speedup(med)  Result
  ---------------------------------------------------------------------------------
  sum               0.3822ms    0.4544ms    0.3759ms    0.3864ms          0.85x   
  prod              0.7764ms    0.7967ms    0.7667ms    0.7828ms          0.98x   
  max               0.3915ms    0.4601ms    0.3608ms    0.3733ms          0.81x   
  min               0.3929ms    0.4595ms    0.3613ms    0.3725ms          0.81x   
  all               0.4498ms    0.5237ms    0.4252ms    0.4416ms          0.84x   
  any               0.4387ms    0.5127ms    0.4174ms    0.4321ms          0.84x   

==============================================================================

Fixes: #27206

AI Disclosure

Local llm to genreate benchmark script.

@Prtm2110 Prtm2110 force-pushed the use_where_in_masked branch from 5c02516 to b8a1e42 Compare April 7, 2026 16:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

01 - Enhancement component: numpy.ma masked arrays

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

ENH: Use "where" argument in masked array logic, rather than filling

2 participants

@Prtm2110 @jorenham