Can we have a little more context why we want this? Speed, correctness, something else?

The benefit is performance. I see a 5% speedup in ns->D conversion, 18% in ns->M.

close/reopen

Can we get some concrete benchmarks demonstrating the speedup? Can you also verify that our test coverage for these conversions is good? I'm a little nervous about completely changing the implementation of a poorly tested codepath.

If you know of an expert we can ping to review this, that might also help assuage concerns about correctness.

Can we get some concrete benchmarks demonstrating the speedup?

In [3]: arr = np.arange("2000-01-01", "2010-01-01", dtype="datetime64[s]").astype("datetime64[ns]")

In [4]: %timeit arr.astype("datetime64[M]")
3.55 s ± 25.2 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)  # <- main
2.95 s ± 58.6 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)  # <- PR

In [5]: %timeit arr.astype("datetime64[D]")
416 ms ± 99.6 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)  # <- main
350 ms ± 32.1 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)  # <- PR

Can you also verify that our test coverage for these conversions is good?

claude says "The existing 470 tests already cover pre-1970 dates, leap year boundaries, year 0/negatives, Y2038, and int64 limits"

I'm a little nervous about completely changing the implementation of a poorly tested codepath.

That's similar to the comment in pandas asking me to open this PR. The hope is that by keeping the implementations aligned we can benefit from each others' coverage.

FWIW my understanding is that the same algorithm is used in c++20's chrono implementation. We just can't use it directly because they don't support int64 years.

I added a triage review label to make sure we talk about this at next week's triage meeting. Feel free to come, see the calendar.

FWIW my understanding is that the same algorithm is used in c++20's chrono implementation.

Oh nice, it might also help if you add a link to an implementation of this as a comment. Someone could also try comparing the two implementations.

Yeah, I think it probably makes sense to just follow pandas, it seems like the sources are trustworthy, although reviewing that it is identical code isn't all that easy (bots help a bit with confidence, I guess).
It seems the GCC version: https://gcc.gnu.org/git/?p=gcc.git;a=blob;f=libstdc%2B%2B-v3/include/std/chrono;h=10e868e5a036924177cf952a81f4bce10c497106;hb=HEAD#l1678
uses the same/very similar approach, but very slightly different constants (no full understanding here, but it seems related to the valid range of years).