Thank you for your contribution to Astropy! 🌌 This checklist is meant to remind the package maintainers who will review this pull request of some common things to look for.

• Do the proposed changes actually accomplish desired goals?
• Do the proposed changes follow the Astropy coding guidelines?
• Are tests added/updated as required? If so, do they follow the Astropy testing guidelines?
• Are docs added/updated as required? If so, do they follow the Astropy documentation guidelines?
• Is rebase and/or squash necessary? If so, please provide the author with appropriate instructions. Also see instructions for rebase and squash.
• Did the CI pass? If no, are the failures related? If you need to run daily and weekly cron jobs as part of the PR, please apply the "Extra CI" label. Codestyle issues can be fixed by the bot.
• Is a change log needed? If yes, did the change log check pass? If no, add the "no-changelog-entry-needed" label. If this is a manual backport, use the "skip-changelog-checks" label unless special changelog handling is necessary.
• Is this a big PR that makes a "What's new?" entry worthwhile and if so, is (1) a "what's new" entry included in this PR and (2) the "whatsnew-needed" label applied?
• At the time of adding the milestone, if the milestone set requires a backport to release branch(es), apply the appropriate "backport-X.Y.x" label(s) before merge.

👋 Thank you for your draft pull request! Do you know that you can use [ci skip] or [skip ci] in your commit messages to skip running continuous integration tests until you are ready?

Interesting. Looks like the --run-slow flag (and corresponding @pytest.mark.slow marker) are defined in some plugin that's pulled from pytest-astropy, but I haven't been able to identified which one yet.

of course, it's from pytest-astropy itself.

Haha yes... It also has --run-hugemem, see https://github.com/astropy/pytest-astropy/blob/main/pytest_astropy/plugin.py

So yeah I am not sure if this is feasible without refactoring pytest-astropy itself... Maybe open discussion on astropy-dev if you still want to pursue?

I do want to continue experimenting with this, but I don't think the problems I'm after can be addressed from pytest-astropy itself without breaking its users.

(and with the impeding feature freeze, I'll have to postpone this work until at least next week)

Turns out vendoring the contents of that plugin within our root conftest.py is actually trivial. I just suspected it could not be, which is why I didn't prioritize this earlier.
Any way this should be much more stable now. Note that it's still based of #18780 though, but I think I could avoid that coupling. I'll try this next once CI is clear.

I'm seeing a couple unrelated failures caused by #18880

other failures I'm seeing are already handled in existing PRs

The challenge isn't code, but people. We wanted the plugin stuff in pytest-astropy so other packages could also use it without requiring astropy, but they were so trivial that we did not want to create a whole new package just for them.

Now, they been upstream long enough that we probably have a number of packages using them, and have to keep maintaining them. So copying them over is just duplicating code.

This definitely need more discussion.

we could easily make these markers into their own plugin if duplication must be avoided, but for what it's worth:

• duplicating ~50 lines of code seems like a (much) lesser evil than irreproducible builds
• I don't think we have to make this change throughout the entire ecosystem; it's diminishing returns from here, but astropy being a core package means (to me) it needs to adhere to strict(er) quality requirements

Moving them anywhere now would create churn. We have to find and inform all the affected downstream packages. And then go through deprecation period and face some resistance. And then still keep them going while deprecated. And then finally we have to remove them.

We have to find and inform all the affected downstream packages.

why ? couldn't we just keep pytest-astropy as the "batteries included" option, even if we stop dog-fooding it in astropy itself ? I glanced over it and it didn't seem too burdensome to maintain (no release for 2 years and it's still working for everyone, as far as I can tell). Am I missing anything ?

So you are saying we create a new package for the plugin and then include that in pytest-astropy, instead of moving it here?

Not really. I would prefer that the code we use to test the core library live in the same repo, which is what the current state of the branch does, and leave pytest-astropy live its life as the provider of these functionality for everyone else that choose to adopt it.

But if we must avoid code duplication at all cost, there are other avenues. Though, it seems we agree that they are highly disruptive and undesirable.

I'll remove the "extra CI" label for now. I agree it should run when this is stable enough, but it doesn't seem worth the extra CPU time at the current proof-of-concept stage.

Can the markers be defined in astropy/conftest.py and also in docs/conftest.py? Such duplication isn't good but we are duplicating conftest.py contents already. And perhaps the markers are not needed in docs, in which case we could define them only in astropy/conftest.py.

Can the markers be defined in astropy/conftest.py and also in docs/conftest.py? Such duplication isn't good but we are duplicating conftest.py contents already

neither will be discovered by pytest when running from an arbitrary location with --pyargs so I think the marker have to be provided by a plugin.

Hi humans 👋 - this pull request hasn't had any new commits for approximately 4 months. I plan to close this in 30 days if the pull request doesn't have any new commits by then.

In lieu of a stalled pull request, please consider closing this and open an issue instead if a reminder is needed to revisit in the future. Maintainers may also choose to add keep-open label to keep this PR open but it is discouraged unless absolutely necessary.

If this PR still needs to be reviewed, as an author, you can rebase it to reset the clock.

If you believe I commented on this pull request incorrectly, please report this here.

I think pyinstaller fails because pytest-skip-slow has an unusual structure and is distributed as a single file source rather than a tree. I take this back upstream.

Nevermind, it's fixed. The current error on pyinstaller is unrelated and addressed in #19818