No coverage reports created when running mypy with input file

Hello,
I am not that familiar with coverage.py, as I am only using it for part of an evaluation, but I ran into strange behavior and couldn't find anything related here or on stackoverflow. Hopefully I am not doing something completely wrong (or stupid).

- What happened:
I am trying to produce coverage reports for the mypy module, using generated inputs. The problem is whenever I run mypy with an input file no .coverage file is created, but otherwise (meaning no file given) a report is created.

- What you did:
I cloned the mypy repository (https://github.com/python/mypy) and installed it locally in a venv using the pip -e . command. I then run the command coverage run -m mypy foo.py, with foo.py beeing an empty file (or containing real code, does not seem to matter) in some folder that is not a python project(for simplicity). Though mypy returns with exactly the same output as it does, when I run
mypy foo.py, no .coverage file is created.

The weird thing is, when i run the command coverage run -m mypy (without the required file parameter) it as expected, returns the error: mypy: error: Missing target module, package, files, or command., however the .coverage file is created. This is also the case when I run coverage run -m mypy --help but I guess this is just for the same reason, as also no input file is given.

- What kind of help you need:
Can you explain to me, why no coverage file is created under this circumstance or how I can get coverage reports for the mypy module when inserting python files?

Could you provide more detail about what you did? Specifically:

which commit of mypy was current after you cloned? You can find out by cd-ing into the mypy directory, and then typing git log -1. When I do that, for example, I see

commit 0412590ba13395a04f122a5dd83764db9d001fe6 (HEAD -> master, origin/master, origin/HEAD)
Author: wyattscarpenter <wyattscarpenter@gmail.com>
Date:   2024-09-02 06:26:53 -0700

    [nit] conf.py: annotate the type, instead of ignoring the error (#17727)

could you describe precisely how you installed "mypy"? You said using the pip -e . command, but pip -e . is not a complete command. I assume you typed pip install -e ., but am not certain; it's important to know what your current directory was when you typed that.
similarly, please describe how you installed coverage
tell us exactly what you typed in order to run coverage. For example, I typed "/Users/not-workme/git-repositories/3rd-party/coveragepy-issues-1845/.venv/bin/coverage run -m mypy foo.py but I saw something very different than what you described (namely

mypy: "mypy/mypy/typeshed/stdlib/types.pyi" shadows library module "types"
note: A user-defined top-level module with name "types" is not supported

), so precision is important here.

I'm sorry, of course the command i used to install mypy was pip install -e ..

Here is what I do precisely:
First I create a virtual environment with: python3.10 -m venv venv and activate it using:
source venv/bin/activate. I have a folder repos/ containing multiple cloned python packages, also containing mypy (which I cloned with the command: git clone https://github.com/python/mypy.git). After cloning mypy into the repos/ folder, I entered it (via cd mypy/) and run pip install -e .. As far as I can tell the installation succeeded, since pip show mypy shows under editable project location: some_path/repos/mypy.

The commit shown using git log -1 is: fe15ee69b9225f808f8ed735671b73c31ae1bed8, which is the same as shown when running pip show mypy under version.

I installed coverage, using pip in the same virtual environment venv, (command: pip install coverage). The installed version is 7.4.3, however the same happens when I create a completely new environment and install version 7.6.1 (using also pip install coverage)

I though I had seen the error you described and maybe had done something earlier on, to fix it. However using a different computer everything worked the same, as I described.

Thanks, I certainly see what you're talking about. Here's how I reproduced it on recent MacOS:

$ mkdir coverage-1845
$ cd coverage-1845
$ pyenv install 3.10
python-build: use openssl from homebrew
python-build: use readline from homebrew
Downloading Python-3.10.15.tar.xz...
-> https://www.python.org/ftp/python/3.10.15/Python-3.10.15.tar.xz
Installing Python-3.10.15...
python-build: use tcl-tk from homebrew
python-build: use readline from homebrew
python-build: use ncurses from homebrew
python-build: use zlib from xcode sdk
Installed Python-3.10.15 to /Users/not-workme/.pyenv/versions/3.10.15
$ ~/.pyenv/versions/3.10.15/bin/python3.10 -m venv venv
$ source venv/bin/activate
$ mkdir repos
$ cd repos
$ git clone https://github.com/python/mypy.git
Cloning into 'mypy'...
remote: Enumerating objects: 97399, done.
remote: Counting objects: 100% (437/437), done.
remote: Compressing objects: 100% (337/337), done.
remote: Total 97399 (delta 193), reused 304 (delta 100), pack-reused 96962 (from 1)
Receiving objects: 100% (97399/97399), 65.95 MiB | 28.27 MiB/s, done.
Resolving deltas: 100% (75387/75387), done.
$ cd mypy/
$ git reset --hard fe15ee69b9225f808f8ed735671b73c31ae1bed8
HEAD is now at fe15ee69b [mypyc] Avoid uses of _PyObject_CallMethodOneArg on 3.13 (#17526)
$ pip install -e .
Obtaining file:///Users/not-workme/git-repositories/me/coverage-1845/repos/mypy
  Installing build dependencies ... done
  Checking if build backend supports build_editable ... done
  Getting requirements to build editable ... done
  Preparing editable metadata (pyproject.toml) ... done
Collecting mypy-extensions>=1.0.0
  Using cached mypy_extensions-1.0.0-py3-none-any.whl (4.7 kB)
Collecting tomli>=1.1.0
  Using cached tomli-2.0.1-py3-none-any.whl (12 kB)
Collecting typing-extensions>=4.6.0
  Using cached typing_extensions-4.12.2-py3-none-any.whl (37 kB)
Building wheels for collected packages: mypy
  Building editable for mypy (pyproject.toml) ... done
  Created wheel for mypy: filename=mypy-1.12.0+dev.fe15ee69b9225f808f8ed735671b73c31ae1bed8-0.editable-py3-none-any.whl size=8239 sha256=6bcc0b76a4eea3e6fcac8fb0a32f5176d642f0740945bd807230cf50c4c5c490
  Stored in directory: /private/var/folders/b0/_p0c_57s5n90cy4njdgd8vhw0000gq/T/pip-ephem-wheel-cache-y9zfb65z/wheels/5d/66/8f/ed18142883942a03238d09c4eeb9c71708f5bba980d4b6db9f
Successfully built mypy
Installing collected packages: typing-extensions, tomli, mypy-extensions, mypy
Successfully installed mypy-1.12.0+dev.fe15ee69b9225f808f8ed735671b73c31ae1bed8 mypy-extensions-1.0.0 tomli-2.0.1 typing-extensions-4.12.2

[notice] A new release of pip is available: 23.0.1 -> 24.2
[notice] To update, run: pip install --upgrade pip
$ pip install coverage
Collecting coverage
  Downloading coverage-7.6.1-cp310-cp310-macosx_11_0_arm64.whl (207 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 207.1/207.1 kB 6.3 MB/s eta 0:00:00
Installing collected packages: coverage
Successfully installed coverage-7.6.1

[notice] A new release of pip is available: 23.0.1 -> 24.2
[notice] To update, run: pip install --upgrade pip
$ cd ..
$ touch foo.py
$ coverage run -m mypy foo.py
mypy: "mypy/mypy/typeshed/stdlib/typing_extensions.pyi" shadows library module "typing_extensions"
note: A user-defined top-level module with name "typing_extensions" is not supported
$ ls -al
total 0
drwxr-xr-x  4 not-workme staff  128 Sep  9 08:34 .
drwxr-xr-x  5 not-workme staff  160 Sep  9 08:33 ..
-rw-r--r--  1 not-workme staff    0 Sep  9 08:34 foo.py
drwxr-xr-x 34 not-workme staff 1088 Sep  9 08:33 mypy
$ coverage run mypy foo.py
Can't find '__main__' module in 'mypy'
$  mypy foo.py
Success: no issues found in 1 source file
$ coverage run foo.py
$ ls -al
total 52
drwxr-xr-x  6 not-workme staff   192 Sep  9 08:38 .
drwxr-xr-x  5 not-workme staff   160 Sep  9 08:33 ..
-rw-r--r--  1 not-workme staff 53248 Sep  9 08:38 .coverage
drwxr-xr-x  5 not-workme staff   160 Sep  9 08:35 .mypy_cache
-rw-r--r--  1 not-workme staff     0 Sep  9 08:34 foo.py
drwxr-xr-x 34 not-workme staff  1088 Sep  9 08:33 mypy
$

Ok, using your procedure is was able to create a very similar bug:

$ mkdir tmp
$ cd tmp/
/tmp$ python3.10 -m venv venv
/tmp$ source venv/bin/activate
/tmp$ mkdir repos
/tmp$ cd repos/
/tmp/repos$ git clone https://github.com/python/mypy.git
Cloning into 'mypy'...
remote: Enumerating objects: 97399, done.
remote: Counting objects: 100% (437/437), done.
remote: Compressing objects: 100% (337/337), done.
remote: Total 97399 (delta 194), reused 302 (delta 100), pack-reused 96962 (from 1)
Receiving objects: 100% (97399/97399), 65.95 MiB | 1.70 MiB/s, done.
Resolving deltas: 100% (75396/75396), done.
/tmp/repos$ cd mypy/
/tmp/repos/mypy$ git reset --hard fe15ee69b9225f808f8ed735671b73c31ae1bed8
HEAD is now at fe15ee69b [mypyc] Avoid uses of _PyObject_CallMethodOneArg on 3.13 (#17526)
/tmp/repos/mypy$ pip install -e .
Obtaining file:.../tmp/repos/mypy
  Installing build dependencies ... done
  Checking if build backend supports build_editable ... done
  Getting requirements to build editable ... done
  Preparing editable metadata (pyproject.toml) ... done
Collecting typing-extensions>=4.6.0
  Using cached typing_extensions-4.12.2-py3-none-any.whl (37 kB)
Collecting tomli>=1.1.0
  Using cached tomli-2.0.1-py3-none-any.whl (12 kB)
Collecting mypy-extensions>=1.0.0
  Using cached mypy_extensions-1.0.0-py3-none-any.whl (4.7 kB)
Building wheels for collected packages: mypy
  Building editable for mypy (pyproject.toml) ... done
  Created wheel for mypy: filename=mypy-1.12.0+dev.fe15ee69b9225f808f8ed735671b73c31ae1bed8-0.editable-py3-none-any.whl size=8223 sha256=03e1a63e251dc464296b104428325f40afe5629cfab7fe09e8834422ab61b819
  Stored in directory: /tmp/pip-ephem-wheel-cache-vvnelq28/wheels/b8/a6/6d/4374bfa2c38d232f2607a6f70432a43da17ecfc4208baf971f
Successfully built mypy
Installing collected packages: typing-extensions, tomli, mypy-extensions, mypy
Successfully installed mypy-1.12.0+dev.fe15ee69b9225f808f8ed735671b73c31ae1bed8 mypy-extensions-1.0.0 tomli-2.0.1 typing-extensions-4.12.2
/tmp/repos/mypy$ pip install coverage
Collecting coverage
  Using cached coverage-7.6.1-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (234 kB)
Installing collected packages: coverage
Successfully installed coverage-7.6.1
/tmp/repos/mypy$ cd ..
/tmp/repos$ touch foo.py
/tmp/repos$ coverage run -m mypy foo.py 
mypy: "mypy/mypy/typeshed/stdlib/sys/__init__.pyi" shadows library module "sys"
note: A user-defined top-level module with name "sys" is not supported

It is not exactly the same error, but maybe my fix still applies. If I just get one more directory up, create foo.py and run coverage there, mypy reports success (though no .coverage file is created, as I mentioned in my initial post):

tmp/repos$ cd ..
/tmp$ touch foo.py
/tmp$ coverage run -m mypy foo.py 
Success: no issues found in 1 source file
/tmp$ ls -a
.  ..  foo.py  .mypy_cache  repos  venv

This is also the case when I specify an Outfile:

/tmp$ coverage run --data-file=".cov" -m mypy foo.py 
Success: no issues found in 1 source file
/tmp$ ls -a
.  ..  foo.py  .mypy_cache  repos  venv

Normaly coverage warns that no data was collected, but that is also not the case here.

Hopefully this is useful in recreating the bug.

(editet because I missed that the command ls does not show hidden files, but the coverage file is also not there using
ls -a)

Furthermore when I run:

/tmp$ coverage run --data-file=".cov" -m mypy
usage: mypy [-h] [-v] [-V] [more options; see below]
            [-m MODULE] [-p PACKAGE] [-c PROGRAM_TEXT] [files ...]
mypy: error: Missing target module, package, files, or command.
/tmp$ ls -a
.  ..  .cov  foo.py  .mypy_cache  repos  venv

now there is a coverage file.

Any new information?
I am on a deadline and would shift to other programs for what i am testing, if it is unclear why coverage is sometimes not producing reports for mypy.

This is because mypy does something unusual. In some error paths, they use their util.hard_exit function:

def hard_exit(status: int = 0) -> None:
    """Kill the current process without fully cleaning up.

    This can be quite a bit faster than a normal exit() since objects are not freed.
    """
    sys.stdout.flush()
    sys.stderr.flush()
    os._exit(status)

os._exit is a blunt tool that prevent Python code from cleaning up (as the docstring here describes). That prevents coverage from writing its data file. This is a duplicate of #310.

In commit 44a309f, I added a new configuration setting, [run] patch that you can use to fix os._exit().

Add this to you .coveragerc (or other configuration file), and the coverage data will be saved when os._exit is used:

[run]
patch = _exit

This is now released as part of coverage 7.10.0.

Navigation Menu

Repository navigation

No coverage reports created when running mypy with input file #1845

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

No coverage reports created when running mypy with input file #1845

Description

Activity

offby1 commented on Sep 5, 2024

KonstiB commented on Sep 9, 2024

offby1 commented on Sep 9, 2024

KonstiB commented on Sep 10, 2024

KonstiB commented on Sep 10, 2024

KonstiB commented on Sep 28, 2024

nedbat commented on Jul 5, 2025

nedbat commented on Jul 9, 2025

nedbat commented on Jul 24, 2025

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions