Skip to content

[3.5] bpo-30675: Fix multiprocessing code in regrtest#2220

Merged
vstinner merged 1 commit into
python:3.5from
vstinner:regrtest_slaveargs
Jun 15, 2017
Merged

[3.5] bpo-30675: Fix multiprocessing code in regrtest#2220
vstinner merged 1 commit into
python:3.5from
vstinner:regrtest_slaveargs

Conversation

@vstinner

Copy link
Copy Markdown
Member
  • Rewrite code to pass slaveargs from the master process to worker
    processes: reuse the same code of the Python master branch
  • Move code to initialize tests in a new setup_tests() function,
    similar change was done in the master branch
  • In a worker process, call setup_tests() with the namespace built
    from slaveargs to initialize correctly tests

Before this change, warm_caches() was not called in worker processes
because the setup was done before rebuilding the namespace from
slaveargs. As a consequence, the huntrleaks feature was unstable. For
example, test_zipfile reported randomly false positive on reference
leaks.

Comment thread Lib/test/regrtest.py Outdated

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does it make sense to do this on subprocesses?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think so. It's not done in worker processes in master neither, so I moved this code out of setup_tests() again.

Comment thread Lib/test/regrtest.py Outdated

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

use_resources is not passed.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, nice catch, fixed.

It's a bug in my change. I copied the runtest() call from above, but above, there is a first "support.use_resources = ns.use_resources" call.

* Rewrite code to pass slaveargs from the master process to worker
  processes: reuse the same code of the Python master branch
* Move code to initialize tests in a new setup_tests() function,
  similar change was done in the master branch
* In a worker process, call setup_tests() with the namespace built
  from slaveargs to initialize correctly tests

Before this change, warm_caches() was not called in worker processes
because the setup was done before rebuilding the namespace from
slaveargs. As a consequence, the huntrleaks feature was unstable. For
example, test_zipfile reported randomly false positive on reference
leaks.
@vstinner vstinner merged commit 33cf0c4 into python:3.5 Jun 15, 2017
@vstinner vstinner deleted the regrtest_slaveargs branch June 15, 2017 22:08
@vstinner

Copy link
Copy Markdown
Member Author

I fixed the two bugs that you spotted @serhiy-storchaka, thanks for your review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants