chore: add types (#225)

miketheman · web-flow · commit 5d721e12cf62 · 2022-03-11T15:04:32.000-05:00
* chore: add mypy testing Scaffolding in the testing, and will not complete succssfully yet. Will be followed by code changes to pass. Signed-off-by: Mike Fiedler <miketheman@gmail.com> * lint: ignore empty dict Surfaced via `mypy`, in that an empty dict could not infer what types could be there. Instead of importing `typing` and annotating the empty Dict, I opted to ignore the line, as we do not expect to populate the dict at all, and are using it to **prevent** additions to the value. Signed-off-by: Mike Fiedler <miketheman@gmail.com> * chore: remove unused styles parameter Surfaced via `mypy`, recommended adding a type to the empty list. The list was originally empty back in 0.1.0. Instead of adding a type, remove the constant, and the code that uses it from `clean()` - as it was partially reverted in #121. The default `ALLOWED_STYLES` in the underlying library is `[]`. Related: #114 (comment) Signed-off-by: Mike Fiedler <miketheman@gmail.com> * fix: correct import for unescape Surfaced via `mypy`, in that the `html.parser` module does not have a direct implementation of `unescape`. Refs: https://docs.python.org/3.6/library/html.html#html.unescape In #192 support for Python 2.7 was removed, the import path changed. This works due to imports placing the imported code in the local scope. If the `html.parser` module ever stopped importing `unescape`, this import would break as a result. Signed-off-by: Mike Fiedler <miketheman@gmail.com> * chore: update pytest markers cli flag Currently emits a warning: PytestRemovedIn8Warning: The --strict option is deprecated, use --strict-markers instead. Signed-off-by: Mike Fiedler <miketheman@gmail.com> * chore(types): add types to clean module Surfaced by running mypy in strict mode, and added types where relevant. Signed-off-by: Mike Fiedler <miketheman@gmail.com> * chore(types): add types to txt module Signed-off-by: Mike Fiedler <miketheman@gmail.com> * chore(types): add types to markdown module Signed-off-by: Mike Fiedler <miketheman@gmail.com> * chore: add types to rst module The types-docutils hints are still incomplete, good progress is being made. See: python/typeshed#7256 I've had to use an ignore on the class inheritance, and a couple of `typing.Any` annotations until that package implements more type hints. Signed-off-by: Mike Fiedler <miketheman@gmail.com> * chore: ignore distutils module from types `mypy` strict mode is having a hard time with the `distutils` imports, since they are wrapped in `setuptools` right now as a private package. This pacakge's distutils integration will need to be reworked anyhow. Left a comment with details at the top of the file. Signed-off-by: Mike Fiedler <miketheman@gmail.com> * test: use strict mode for mypy Prevent new things from creeping in during development. Signed-off-by: Mike Fiedler <miketheman@gmail.com> * chore: tell the world we've got types Include a blank `py.typed` file in the package to inform `mypy` that there's types to be found in this package. Signed-off-by: Mike Fiedler <miketheman@gmail.com> * chore: move strict flag to config Allows other tools to ebenfit from a consistent configuration. Signed-off-by: Mike Fiedler <miketheman@gmail.com> * refactor: change imports to be consistent with others Signed-off-by: Mike Fiedler <miketheman@gmail.com> * docs: add inline details for specific ignores Signed-off-by: Mike Fiedler <miketheman@gmail.com> * lint: apply more specific type Signed-off-by: Mike Fiedler <miketheman@gmail.com> * docs: add comment to why source is Any Signed-off-by: Mike Fiedler <miketheman@gmail.com> * lint: replace typing imports with relative ones Signed-off-by: Mike Fiedler <miketheman@gmail.com>
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
@@ -32,6 +32,8 @@ jobs:
         run: python -m pip install tox
       - name: Run linting
         run: python -m tox -e pep8
+      - name: Run mypy
+        run: python -m tox -e mypy
   packaging:
     name: Packaging
     runs-on: ubuntu-latest
diff --git a/MANIFEST.in b/MANIFEST.in
@@ -1,5 +1,6 @@
 include LICENSE README.rst CHANGES.rst
 include tox.ini .coveragerc pytest.ini
+include readme_renderer/py.typed
 
 recursive-include tests *.html
 recursive-include tests *.py
diff --git a/pyproject.toml b/pyproject.toml
@@ -1,3 +1,13 @@
 [build-system]
 requires = ["setuptools>=40.8.0", "wheel", "bleach>=2.1.0", "docutils>=0.13.1", "Pygments>=2.5.1"]
 build-backend = "setuptools.build_meta:__legacy__"
+
+[tool.mypy]
+strict = true
+warn_unused_configs = true
+[[tool.mypy.overrides]]
+# These modules do not yet have types available.
+module = [
+    "cmarkgfm.*"
+]
+ignore_missing_imports = true
diff --git a/readme_renderer/clean.py b/readme_renderer/clean.py
@@ -14,6 +14,7 @@
 from __future__ import absolute_import, division, print_function
 
 import functools
+from typing import Any, Dict, Iterator, List, Optional
 
 import bleach
 import bleach.callbacks
@@ -59,15 +60,14 @@
     "input": ["type", "checked", "disabled"],
 }
 
-ALLOWED_STYLES = [
-]
-
 
 class DisabledCheckboxInputsFilter:
-    def __init__(self, source):
+    # The typeshed for bleach (html5lib) filters is incomplete, use `typing.Any`
+    # See https://github.com/python/typeshed/blob/505ea726415016e53638c8b584b8fdc9c722cac1/stubs/bleach/bleach/html5lib_shim.pyi#L7-L8 # noqa E501
+    def __init__(self, source: Any) -> None:
         self.source = source
 
-    def __iter__(self):
+    def __iter__(self) -> Iterator[Dict[str, Optional[str]]]:
         for token in self.source:
             if token.get("name") == "input":
                 # only allow disabled checkbox inputs
@@ -85,23 +85,24 @@ def __iter__(self):
             else:
                 yield token
 
-    def __getattr__(self, name):
+    def __getattr__(self, name: str) -> Any:
         return getattr(self.source, name)
 
 
-def clean(html, tags=None, attributes=None, styles=None):
+def clean(
+    html: str,
+    tags: Optional[List[str]] = None,
+    attributes: Optional[Dict[str, List[str]]] = None
+) -> Optional[str]:
     if tags is None:
         tags = ALLOWED_TAGS
     if attributes is None:
         attributes = ALLOWED_ATTRIBUTES
-    if styles is None:
-        styles = ALLOWED_STYLES
 
     # Clean the output using Bleach
     cleaner = bleach.sanitizer.Cleaner(
         tags=tags,
         attributes=attributes,
-        styles=styles,
         filters=[
             # Bleach Linkify makes it easy to modify links, however, we will
             # not be using it to create additional links.
diff --git a/readme_renderer/integration/distutils.py b/readme_renderer/integration/distutils.py
@@ -11,6 +11,13 @@
 # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 # See the License for the specific language governing permissions and
 # limitations under the License.
+
+# The `distutils` integration is going to need to get updated to `setuptools``
+# soon enough, as `distutils` is deprecated and will be removed in Python 3.12.
+# There's currently some pass-through imports that allow this to work, but is
+# challenging for `mypy` in `strict` mode, so let's skip this file for now.
+# See https://peps.python.org/pep-0632/
+# mypy: ignore-errors
 from __future__ import absolute_import, division, print_function
 
 import cgi
diff --git a/readme_renderer/markdown.py b/readme_renderer/markdown.py
@@ -15,8 +15,9 @@
 
 import re
 import warnings
+from typing import Any, Match, Optional
 
-from html.parser import unescape
+from html import unescape
 
 import pygments
 import pygments.lexers
@@ -51,7 +52,11 @@
 }
 
 
-def render(raw, variant="GFM", **kwargs):
+def render(
+    raw: str,
+    variant: str = "GFM",
+    **kwargs: Any
+) -> Optional[str]:
     if not variants:
         warnings.warn(_EXTRA_WARNING)
         return None
@@ -61,7 +66,8 @@ def render(raw, variant="GFM", **kwargs):
     if not renderer:
         return None
 
-    rendered = renderer(raw)
+    # The renderer is a lambda function, and mypy fails lambdas right now.
+    rendered = renderer(raw)  # type: ignore
 
     if not rendered:
         return None
@@ -71,7 +77,7 @@ def render(raw, variant="GFM", **kwargs):
     return cleaned
 
 
-def _highlight(html):
+def _highlight(html: str) -> str:
     """Syntax-highlights HTML-rendered Markdown.
 
     Plucks sections to highlight that conform the the GitHub fenced code info
@@ -94,7 +100,7 @@ def _highlight(html):
         '(?(in_code)|<code>)(?P<code>.+?)'
         r'</code></pre>', re.DOTALL)
 
-    def replacer(match):
+    def replacer(match: Match[Any]) -> str:
         try:
             lang = match.group('lang')
             lang = _LANG_ALIASES.get(lang, lang)
diff --git a/readme_renderer/py.typed b/readme_renderer/py.typed
diff --git a/readme_renderer/rst.py b/readme_renderer/rst.py
@@ -14,20 +14,28 @@
 from __future__ import absolute_import, division, print_function
 
 import io
+from typing import Any, Dict, IO, Optional, Union
 
 from docutils.core import publish_parts
+from docutils.nodes import colspec, image
 from docutils.writers.html4css1 import HTMLTranslator, Writer
 from docutils.utils import SystemMessage
 
 from .clean import clean
 
 
-class ReadMeHTMLTranslator(HTMLTranslator):
+class ReadMeHTMLTranslator(HTMLTranslator):  # type: ignore[misc] # docutils is incomplete, returns `Any` python/typeshed#7256 # noqa E501
 
     # Overrides base class not to output `<object>` tag for SVG images.
-    object_image_types = {}
-
-    def emptytag(self, node, tagname, suffix="\n", **attributes):
+    object_image_types: Dict[str, str] = {}
+
+    def emptytag(
+        self,
+        node: Union[colspec, image],
+        tagname: str,
+        suffix: str = "\n",
+        **attributes: Any
+    ) -> Any:
         """Override this to add back the width/height attributes."""
         if tagname == "img":
             if "width" in node:
@@ -95,7 +103,11 @@ def emptytag(self, node, tagname, suffix="\n", **attributes):
 }
 
 
-def render(raw, stream=None, **kwargs):
+def render(
+    raw: str,
+    stream: Optional[IO[str]] = None,
+    **kwargs: Any
+) -> Optional[str]:
     if stream is None:
         # Use a io.StringIO as the warning stream to prevent warnings from
         # being printed to sys.stderr.
diff --git a/readme_renderer/txt.py b/readme_renderer/txt.py
@@ -14,6 +14,7 @@
 from __future__ import absolute_import, division, print_function
 
 import sys
+from typing import Any, Optional
 
 from .clean import clean
 
@@ -26,6 +27,6 @@ def html_escape(s):
         return escape(s, quote=True).replace("'", '&#x27;')
 
 
-def render(raw, **kwargs):
+def render(raw: str, **kwargs: Any) -> Optional[str]:
     rendered = html_escape(raw).replace("\n", "<br>")
     return clean(rendered, tags=["br"])
diff --git a/tox.ini b/tox.ini
@@ -1,13 +1,22 @@
 [tox]
-envlist = py36,py37,py38,py39,py310,pep8,packaging,noextra
+envlist = py36,py37,py38,py39,py310,pep8,packaging,noextra,mypy
 
 [testenv]
 deps =
     pytest
 commands =
-    pytest --strict {posargs}
+    pytest --strict-markers {posargs}
 extras = md
 
+[testenv:mypy]
+basepython = python3
+deps =
+    mypy
+    types-bleach
+    types-docutils
+    types-Pygments
+commands = mypy readme_renderer
+
 [testenv:pep8]
 basepython = python3
 deps =
