Commit a modified version of typeshed-stats into pyperformance

python · AlexWaygood · Mar 1, 2023 · Mar 1, 2023 · Mar 1, 2023 · Apr 27, 2023
commit 9e811d1d5be04285cab9181b94bb2151dcd25d8b
diff --git a/pyperformance/data-files/benchmarks/bm_typeshed_stats/typeshed-stats/LICENSE b/pyperformance/data-files/benchmarks/bm_typeshed_stats/typeshed-stats/LICENSE
@@ -0,0 +1,19 @@
+Copyright (c) 2018 The Python Packaging Authority
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.
diff --git a/pyperformance/data-files/benchmarks/bm_typeshed_stats/typeshed-stats/README.md b/pyperformance/data-files/benchmarks/bm_typeshed_stats/typeshed-stats/README.md
@@ -0,0 +1,62 @@
+<div align=center>
+
+# typeshed-stats
+
+<img src="https://user-images.githubusercontent.com/66076021/202873196-3af493e6-bca0-4c1a-8853-73635b5c1ca8.png" width="500">
+
+<br>
+
+---
+
+## A CLI tool and library to gather stats on [typeshed](https://github.com/python/typeshed)
+
+<br>
+
+[![website](https://img.shields.io/website?down_color=red&down_message=Offline&style=for-the-badge&up_color=green&up_message=Running&url=https%3A%2F%2Falexwaygood.github.io%2Ftypeshed-stats%2F)](https://alexwaygood.github.io/typeshed-stats/)[![build status](https://img.shields.io/github/actions/workflow/status/AlexWaygood/typeshed-stats/test.yml?branch=main&label=Tests&style=for-the-badge)](https://github.com/AlexWaygood/typeshed-stats/actions/workflows/test.yml)[![Coveralls](https://img.shields.io/coverallsCoverage/github/AlexWaygood/typeshed-stats?style=for-the-badge)](https://coveralls.io/github/AlexWaygood/typeshed-stats)
+<br>
+[![Checked with mypy](https://img.shields.io/badge/mypy-checked-blue?style=for-the-badge)](http://mypy-lang.org/)[![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg?style=for-the-badge)](https://github.com/psf/black)[![Imports: isort](https://img.shields.io/badge/%20imports-isort-%231674b1?style=for-the-badge&labelColor=ef8336)](https://pycqa.github.io/isort/)[![pre-commit](https://img.shields.io/badge/pre--commit-enabled-brightgreen?logo=pre-commit&logoColor=white&style=for-the-badge)](https://pre-commit.ci)
+<br>
+[![PyPI](https://img.shields.io/pypi/v/typeshed-stats?style=for-the-badge)](https://pypi.org/project/typeshed-stats/)![PyPI - Python Version](https://img.shields.io/pypi/pyversions/typeshed-stats?style=for-the-badge)![PyPI - Wheel](https://img.shields.io/pypi/wheel/typeshed-stats?style=for-the-badge)[![license](https://img.shields.io/github/license/AlexWaygood/typeshed-stats?style=for-the-badge)](https://opensource.org/licenses/MIT)
+
+---
+
+<br>
+</div>
+
+## What's this project for?
+
+This project is for easy gathering of statistics relating to [typeshed](https://github.com/python/typeshed)'s stubs. As well as being a CLI tool and library, it also powers [a website](https://alexwaygood.github.io/typeshed-stats/) where stats about typeshed's stubs are uploaded twice a day.
+
+This project was created by Alex Waygood. It is not necessarily endorsed by any of the other typeshed maintainers.
+
+Some examples of things you can do from the command line:
+
+- Create a `.csv` file with stats on all typeshed stubs: `typeshed-stats --typeshed-dir <PATH_TO_TYPESHED_CLONE> --to-file stats.csv` (the `.csv` file extension will be automatically detected by the script to identify the format required).
+- Pretty-print stats on typeshed stubs for emoji and redis to the terminal, in JSON format: `typeshed-stats --typeshed-dir <PATH_TO_TYPESHED_CLONE> --to-json emoji redis`
+- Generate a MarkDown file detailing stats on typeshed's stubs for protobuf and the stdlib: `typeshed-stats --typeshed-dir <PATH_TO_TYPESHED_CLONE> --to-file stats.md stdlib protobuf`
+
+Example usage of the Python-level API:
+
+```python
+from typeshed_stats.gather import tmpdir_typeshed, gather_stats
+
+with tmpdir_typeshed() as typeshed:
+    stats_on_all_packages = gather_stats_on_multiple_packages(typeshed_dir=typeshed)
+```
+
+## How can I use this?
+
+1. Run `pip install typeshed-stats[rich]` to install the package
+1. Run `typeshed-stats --help` for information about various options
+
+## Are there any examples of things this script can produce, other than [the website](https://alexwaygood.github.io/typeshed-stats/)?
+
+I'm glad you asked! They're in the `examples/` folder in this repo.
+(These examples are generated using the `regenerate.py` script in the `scripts/` directory.)
+
+## How do I run tests/linters?
+
+1. Clone the repo and `cd` into it
+1. Create and activate a virtual environment
+1. Run `pip install -e .[dev]`
+1. Either run the linters/tests individually (see the `.github/workflows` directory for details about what's run in CI) or use the `scripts/runtests.py` convenience script to run them all in succession.
diff --git a/pyperformance/data-files/benchmarks/bm_typeshed_stats/typeshed-stats/pyproject.toml b/pyperformance/data-files/benchmarks/bm_typeshed_stats/typeshed-stats/pyproject.toml
@@ -0,0 +1,40 @@
+[build-system]
+requires = ["hatchling"]
+build-backend = "hatchling.build"
+
+[project]
+name = "typeshed_stats"
+dynamic = ["version"]
+authors = [
+    { name="Alex Waygood", email="alex.waygood@gmail.com" },
+]
+description = "Library and command-line tool to gather stats on typeshed packages"
+license = { text = "MIT" }
+readme = "README.md"
+requires-python = ">=3.11"
+keywords = [
+    "typeshed",
+    "typing",
+    "stubs",
+]
+classifiers = [
+    "Development Status :: 3 - Alpha",
+    "Intended Audience :: Developers",
+    "Programming Language :: Python",
+    "Programming Language :: Python :: 3 :: Only",
+    "Programming Language :: Python :: 3.11",
+    "License :: OSI Approved :: MIT License",
+    "Operating System :: OS Independent",
+    "Topic :: Software Development",
+    "Typing :: Typed",
+]
+
+[project.urls]
+"Homepage" = "https://github.com/AlexWaygood/typeshed-stats"
+"Bug Tracker" = "https://github.com/AlexWaygood/typeshed-stats/issues"
+
+[project.scripts]
+typeshed-stats = "typeshed_stats._cli:main"
+
+[tool.hatch.version]
+path = "src/typeshed_stats/__init__.py"
diff --git a/...nce/data-files/benchmarks/bm_typeshed_stats/typeshed-stats/src/typeshed_stats/__init__.py b/...nce/data-files/benchmarks/bm_typeshed_stats/typeshed-stats/src/typeshed_stats/__init__.py
@@ -0,0 +1,18 @@
+"""Library and command-line tool for getting stats on various typeshed packages."""
+
+__all__ = [
+    "__title__",
+    "__summary__",
+    "__about__",
+    "__author__",
+    "__email__",
+    "__license__",
+    "__version__",
+]
+
+__title__ = "typeshed-stats"
+__summary__ = __about__ = __doc__
+__author__ = "Alex Waygood"
+__email__ = "alex.waygood@gmail.com"
+__license__ = "MIT"
+__version__ = "23.2.1"
diff --git a/...nce/data-files/benchmarks/bm_typeshed_stats/typeshed-stats/src/typeshed_stats/__main__.py b/...nce/data-files/benchmarks/bm_typeshed_stats/typeshed-stats/src/typeshed_stats/__main__.py
@@ -0,0 +1,13 @@
+"""Library and command-line tool for getting stats on various typeshed packages."""
+
+import sys
+
+if sys.version_info < (3, 11):
+    raise ImportError("Python 3.11+ is required!")  # pragma: no cover
+
+__all__: list[str] = []
+
+if __name__ == "__main__":
+    from ._cli import main
+
+    main()
diff --git a/...ormance/data-files/benchmarks/bm_typeshed_stats/typeshed-stats/src/typeshed_stats/_cli.py b/...ormance/data-files/benchmarks/bm_typeshed_stats/typeshed-stats/src/typeshed_stats/_cli.py
@@ -0,0 +1,155 @@
+"""Command-line interface."""
+
+from __future__ import annotations
+
+import argparse
+import logging
+import pprint
+import sys
+from collections.abc import Sequence
+from contextlib import ExitStack
+from dataclasses import dataclass
+from pathlib import Path
+from typing import Literal, TypeAlias, get_args
+
+from typeshed_stats.gather import gather_stats_on_multiple_packages, tmpdir_typeshed
+
+__all__ = ["main"]
+
+
+_LoggingLevels: TypeAlias = Literal[
+    "NOTSET", "DEBUG", "INFO", "WARNING", "ERROR", "CRITICAL"
+]
+
+
+def _get_argument_parser() -> argparse.ArgumentParser:
+    """Parse arguments and do basic argument validation.
+
+    *Don't* do any querying of whether paths actually exist, etc.
+    Leave that to _validate_options().
+    """
+    parser = argparse.ArgumentParser(
+        prog="typeshed-stats", description="Tool to gather stats on typeshed"
+    )
+    parser.add_argument(
+        "packages",
+        type=str,
+        nargs="*",
+        action="extend",
+        help=(
+            "Packages to gather stats on"
+            " (defaults to all third-party packages, plus the stdlib)"
+        ),
+    )
+    parser.add_argument(
+        "--log",
+        choices=get_args(_LoggingLevels),
+        default="INFO",
+        help="Specify the level of logging (defaults to logging.INFO)",
+        dest="logging_level",
+    )
+    parser.add_argument(
+        "--pretty",
+        action="store_true",
+        help="Whether to pprint results or not (defaults to False)",
+    )
+
+    typeshed_options = parser.add_argument_group(title="Typeshed options")
+    typeshed_options_group = typeshed_options.add_mutually_exclusive_group(
+        required=True
+    )
+    typeshed_options_group.add_argument(
+        "-t",
+        "--typeshed-dir",
+        type=Path,
+        help="Path to a local clone of typeshed, to be used as the basis for analysis",
+    )
+    typeshed_options_group.add_argument(
+        "-d",
+        "--download-typeshed",
+        action="store_true",
+        help=(
+            "Download a fresh copy of typeshed into a temporary directory,"
+            " and use that as the basis for analysis"
+        ),
+    )
+
+    return parser
+
+
+@dataclass(init=False)
+class _CmdArgs:
+    logging_level: _LoggingLevels
+    packages: list[str]
+    typeshed_dir: Path | None
+    download_typeshed: bool
+    overwrite: bool
+    writefile: Path | None
+    pretty: bool
+
+
+def _validate_packages(
+    package_names: list[str], typeshed_dir: Path, *, parser: argparse.ArgumentParser
+) -> None:
+    stubs_dir = typeshed_dir / "stubs"
+    for package_name in package_names:
+        if package_name != "stdlib":
+            package_dir = stubs_dir / package_name
+            if not (package_dir.exists() and package_dir.is_dir()):
+                parser.error(f"{package_name!r} does not have stubs in typeshed!")
+
+
+def _validate_typeshed_dir(
+    typeshed_dir: Path, *, parser: argparse.ArgumentParser
+) -> None:
+    for folder in typeshed_dir, (typeshed_dir / "stdlib"), (typeshed_dir / "stubs"):
+        if not (folder.exists() and folder.is_dir()):
+            parser.error(f'"{typeshed_dir}" is not a valid typeshed directory')
+
+
+def _setup_logger(str_level: _LoggingLevels) -> logging.Logger:
+    assert str_level in get_args(_LoggingLevels)
+    logger = logging.getLogger("typeshed_stats")
+    level = getattr(logging, str_level)
+    assert isinstance(level, int)
+    logger.setLevel(level)
+    handler = logging.StreamHandler()
+    handler.setLevel(level)
+    logger.addHandler(handler)
+    return logger
+
+
+def _run(argv: Sequence[str] | None = None) -> None:
+    parser = _get_argument_parser()
+    args: _CmdArgs = parser.parse_args(argv, namespace=_CmdArgs())
+    logger = _setup_logger(args.logging_level)
+
+    with ExitStack() as stack:
+        if args.download_typeshed:
+            logger.info("Cloning typeshed into a temporary directory...")
+            typeshed_dir = stack.enter_context(tmpdir_typeshed())
+        else:
+            assert args.typeshed_dir is not None
+            typeshed_dir = args.typeshed_dir
+            _validate_typeshed_dir(typeshed_dir, parser=parser)
+
+        packages: list[str] | None = args.packages or None
+        if packages:
+            _validate_packages(packages, typeshed_dir, parser=parser)
+
+        logger.info("Gathering stats...")
+        stats = gather_stats_on_multiple_packages(packages, typeshed_dir=typeshed_dir)
+
+    pprint.pprint({info_bundle.package_name: info_bundle for info_bundle in stats})
+
+
+def main(argv: Sequence[str] | None = None) -> None:
+    """CLI entry point."""
+    try:
+        _run(argv)
+    except KeyboardInterrupt:
+        sys.stderr.write("Interrupted!")
+        code = 2
+    else:
+        code = 0
+    raise SystemExit(code)