@webknjaz It looks like there's still a CI failure most likely related to the changes in this PR: https://dev.azure.com/ansible/ansible/_build/results?buildId=136730&view=logs&jobId=3fee36c7-7a44-5eaa-c749-45e02c4a1c76&j=3fee36c7-7a44-5eaa-c749-45e02c4a1c76&t=6167ccff-5f17-5a7d-c531-0c64601cc36b

The other failure appears to be the usual macOS pwd issue. We're working on a separate fix for that, as DT changes have significantly increased the frequency at which it occurs.

@mattclay pkg_resources is being imported by https://github.com/ansible/ansible/blob/f3ccd78/test/integration/targets/egg-info/lookup_plugins/import_pkg_resources.py#L3. It's a part of setuptools. Older versions of CPython used to bundle setuptools as a part of ensurepip in stdlib. Python 3.12 stopped doing that: python/cpython#95299 / python/cpython#101039.

Since, I've bumped the Python version within VM from 3.11 to 3.13 (I think I probably skipped the bump for the 14.3 image, so we didn't hit 3.12), this manifested itself due to the missing dependency declaration — it just happened to work in the part due to the side effect of setuptools being bundled within Python distributions used.

I'm curious why this hasn't happened earlier — I'd expect this integration test to fail in other jobs with Python 3.12+. Does this mean that none of the currently running integration testing environments has a modern enough Python version?

I'd like to understand the nature of this import — is it necessary? Does it have to be pkg_resources specifically? The fix might be either declaring a dependency on setuptools in that test target or using some other import.

I'm curious why this hasn't happened earlier — I'd expect this integration test to fail in other jobs with Python 3.12+. Does this mean that none of the currently running integration testing environments has a modern enough Python version?

All of our other test environments wither newer Python versions have setuptools installed.

I'd like to understand the nature of this import — is it necessary? Does it have to be pkg_resources specifically? The fix might be either declaring a dependency on setuptools in that test target or using some other import.

It does need to be pkg_resources, since the point of that particular test is to ensure that it can be imported by plugins. In the past there were bugs where plugins couldn't use it due to things we had done within ansible-core. I don't recall the specifics. It's possible we no longer need to be concerned about pkg_resources working, in which case we could just eliminate that test. Either that, or the test should be updated to create a venv and install setuptools first.

I'm leaning towards just removing the test. It's likely to be a non-issue. If we ever do introduce a change which prevents it from working, we'll eventually get a bug report about it. That will at least give us a reproducer to justify having a test for it.

Looks like the test user directory race condition manifests itself flakily, too:

26:13 TASK [get real path of home directory of the unprivileged user] ****************
26:13 task path: /private/var/root/ansible/test/results/.tmp/integration/module_utils-343_xhtm-ÅÑŚÌβŁÈ/test/integration/targets/module_utils/module_utils_basic_setcwd.yml:44
26:13 <testhost> ESTABLISH LOCAL CONNECTION FOR USER: root
26:13 <testhost> EXEC sudo -H -S -p "[sudo via ansible, key=nzjokuyrmpzwabkafxgftugnelnsohxm] password:" -u ansibletest0 /bin/sh -c 'echo BECOME-SUCCESS-nzjokuyrmpzwabkafxgftugnelnsohxm ; /usr/local/bin/python3.13 -c '"'"'import os.path; print(os.path.realpath(os.path.expanduser("~")))'"'"''
26:13 fatal: [testhost]: FAILED! => {
26:13     "changed": true,
26:13     "msg": "non-zero return code",
26:13     "rc": 1,
26:13     "stderr": "job-working-directory: error retrieving current directory: getcwd: cannot access parent directories: Permission denied\nTraceback (most recent call last):\n  File \"<frozen importlib._bootstrap>\", line 1360, in _find_and_load\n  File \"<frozen importlib._bootstrap>\", line 1322, in _find_and_load_unlocked\n  File \"<frozen importlib._bootstrap>\", line 1262, in _find_spec\n  File \"<frozen importlib._bootstrap_external>\", line 1559, in find_spec\n  File \"<frozen importlib._bootstrap_external>\", line 1531, in _get_spec\n  File \"<frozen importlib._bootstrap_external>\", line 1510, in _path_importer_cache\nPermissionError: [Errno 13] Permission denied\n",
26:13     "stderr_lines": [
26:13         "job-working-directory: error retrieving current directory: getcwd: cannot access parent directories: Permission denied",
26:13         "Traceback (most recent call last):",
26:13         "  File \"<frozen importlib._bootstrap>\", line 1360, in _find_and_load",
26:13         "  File \"<frozen importlib._bootstrap>\", line 1322, in _find_and_load_unlocked",
26:13         "  File \"<frozen importlib._bootstrap>\", line 1262, in _find_spec",
26:13         "  File \"<frozen importlib._bootstrap_external>\", line 1559, in find_spec",
26:13         "  File \"<frozen importlib._bootstrap_external>\", line 1531, in _get_spec",
26:13         "  File \"<frozen importlib._bootstrap_external>\", line 1510, in _path_importer_cache",
26:13         "PermissionError: [Errno 13] Permission denied"
26:13     ],
26:13     "stdout": "",
26:13     "stdout_lines": []
26:13 }
26:13 NOTIFIED HANDLER setup_test_user : delete test user for testhost

(https://dev.azure.com/ansible/ansible/_build/results?buildId=136785&view=logs&j=df31b47f-64d3-5652-fdb8-5dc578f08384&t=01a156f1-333a-5ffb-fd86-8f5dafa9d292&l=14575)

Oh, interesting… It's recurring.

@nitzmahone does your thing fix this problem?

@mattclay I submitted that last commit separately: #84681

Yes, the sudo fix we're polishing up should prevent this problem from occurring (for both the Python and shell cases).

@mattclay I submitted that last commit separately: #84681

It turned into a removal: #84686.