allow object type in dict.get and dict.pop#15471
Conversation
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
|
I count 28 errors going away in the ecosystem and 9 errors being added |
|
Well those 9 errors are as expected all Btw. is it really approapriate that def is_config(obj: object):
if not isinstance(obj, dict):
return False
if not isinstance(obj.get("_target_"), str): # ty: expected Never, found "_target_"
return False
return True |
Yeah I agree! Sorry, I only got partway through a review last night |
We've discussed this in the past (#7642) — we could certainly discuss it again now that there are even more type checkers on the scene. Mypy is still the most popular type checker around, so it will continue to have a lot of weight in our decisions here for a while, but as the author of another type checker I do obviously agree that we should consider other type checkers as well when making decisions in this repo 😄 |
This comment has been minimized.
This comment has been minimized.
|
Diff from mypy_primer, showing the effect of this PR on open source code: beartype (https://github.com/beartype/beartype)
+ beartype/_check/convert/_reduce/redmain.py:644: error: Unused "type: ignore" comment [unused-ignore]
+ beartype/_check/convert/_reduce/redmain.py:722: error: Unused "type: ignore" comment [unused-ignore]
+ beartype/_check/code/_pep/pep484585/codepep484585container.py:99: error: Unused "type: ignore" comment [unused-ignore]
+ beartype/door/_cls/pep/pep484585/doorpep484585subscripted.py:50: error: Unused "type: ignore" comment [unused-ignore]
+ beartype/door/_cls/util/doorclsmap.py:91: error: Unused "type: ignore" comment [unused-ignore]
starlette (https://github.com/encode/starlette)
+ starlette/middleware/sessions.py:114: error: Signature of "pop" incompatible with supertype "builtins.dict" [override]
+ starlette/middleware/sessions.py:114: note: Superclass:
+ starlette/middleware/sessions.py:114: note: @overload
+ starlette/middleware/sessions.py:114: note: def pop(self, object, /) -> Any
+ starlette/middleware/sessions.py:114: note: @overload
+ starlette/middleware/sessions.py:114: note: def pop(self, object, Any, /) -> Any
+ starlette/middleware/sessions.py:114: note: @overload
+ starlette/middleware/sessions.py:114: note: def [_T] pop(self, object, _T, /) -> Any | _T
+ starlette/middleware/sessions.py:114: note: Subclass:
+ starlette/middleware/sessions.py:114: note: def pop(self, key: str, *args: Any) -> Any
pylint (https://github.com/pycqa/pylint)
+ pylint/config/arguments_provider.py:54: error: Incompatible types in "yield" (actual type "tuple[None, list[tuple[str, dict[str, str | bool | int | Pattern[str] | Iterable[str | int | Pattern[str]] | type[_CallbackAction] | Callable[[Any], Any] | Callable[[Any, Any, Any, Any], Any] | None], Any]]]", expected type "tuple[str, list[tuple[str, dict[str, str | bool | int | Pattern[str] | Iterable[str | int | Pattern[str]] | type[_CallbackAction] | Callable[[Any], Any] | Callable[[Any, Any, Any, Any], Any] | None], Any]]] | tuple[None, dict[str, list[tuple[str, dict[str, str | bool | int | Pattern[str] | Iterable[str | int | Pattern[str]] | type[_CallbackAction] | Callable[[Any], Any] | Callable[[Any, Any, Any, Any], Any] | None], Any]]]]") [misc]
+ pylint/config/arguments_provider.py:54: note: Error code "misc" not covered by "type: ignore" comment
+ pylint/config/arguments_provider.py:54: error: Unused "type: ignore" comment [unused-ignore]
scrapy (https://github.com/scrapy/scrapy)
+ scrapy/utils/datatypes.py:82: error: Signature of "get" incompatible with supertype "builtins.dict" [override]
+ scrapy/utils/datatypes.py:82: note: Superclass:
+ scrapy/utils/datatypes.py:82: note: @overload
+ scrapy/utils/datatypes.py:82: note: def get(self, object, None = ..., /) -> Any | None
+ scrapy/utils/datatypes.py:82: note: @overload
+ scrapy/utils/datatypes.py:82: note: def get(self, object, Any, /) -> Any
+ scrapy/utils/datatypes.py:82: note: @overload
+ scrapy/utils/datatypes.py:82: note: def [_T] get(self, object, _T, /) -> Any | _T
+ scrapy/utils/datatypes.py:82: note: Subclass:
+ scrapy/utils/datatypes.py:82: note: def [AnyStr: (str, bytes)] get(self, key: AnyStr, def_val: Any = ...) -> Any
+ scrapy/utils/datatypes.py:98: error: Signature of "pop" incompatible with supertype "builtins.dict" [override]
+ scrapy/utils/datatypes.py:98: note: Superclass:
+ scrapy/utils/datatypes.py:98: note: @overload
+ scrapy/utils/datatypes.py:98: note: def pop(self, object, /) -> Any
+ scrapy/utils/datatypes.py:98: note: @overload
+ scrapy/utils/datatypes.py:98: note: def pop(self, object, Any, /) -> Any
+ scrapy/utils/datatypes.py:98: note: @overload
+ scrapy/utils/datatypes.py:98: note: def [_T] pop(self, object, _T, /) -> Any | _T
+ scrapy/utils/datatypes.py:98: note: Subclass:
+ scrapy/utils/datatypes.py:98: note: def [AnyStr: (str, bytes)] pop(self, key: AnyStr, *args: Any) -> Any
+ scrapy/http/headers.py:76: error: Signature of "get" incompatible with supertype "builtins.dict" [override]
+ scrapy/http/headers.py:76: note: Superclass:
+ scrapy/http/headers.py:76: note: @overload
+ scrapy/http/headers.py:76: note: def get(self, object, None = ..., /) -> Any | None
+ scrapy/http/headers.py:76: note: @overload
+ scrapy/http/headers.py:76: note: def get(self, object, Any, /) -> Any
+ scrapy/http/headers.py:76: note: @overload
+ scrapy/http/headers.py:76: note: def [_T] get(self, object, _T, /) -> Any | _T
+ scrapy/http/headers.py:76: note: Subclass:
+ scrapy/http/headers.py:76: note: def [AnyStr: (str, bytes)] get(self, key: AnyStr, def_val: Any = ...) -> bytes | None
altair (https://github.com/vega/altair)
+ altair/utils/html.py:394: error: Unused "type: ignore" comment [unused-ignore]
+ altair/utils/html.py:399: error: Item "str" of "Literal['standard', 'universal', 'inline', 'olli'] | Template" has no attribute "render" [union-attr]
discord.py (https://github.com/Rapptz/discord.py)
+ discord/state.py:372: error: Unused "type: ignore" comment [unused-ignore]
+ discord/state.py:437: error: Unused "type: ignore" comment [unused-ignore]
+ discord/state.py:474: error: Unused "type: ignore" comment [unused-ignore]
+ discord/state.py:478: error: Unused "type: ignore" comment [unused-ignore]
+ discord/state.py:497: error: Unused "type: ignore" comment [unused-ignore]
- discord/ui/view.py:961: error: No overload variant of "pop" of "dict" matches argument types "tuple[int, str]", "None" [call-overload]
- discord/ui/view.py:961: note: Possible overload variants:
- discord/ui/view.py:961: note: def pop(self, Pattern[str], /) -> type[DynamicItem[Item[Any]]]
- discord/ui/view.py:961: note: def pop(self, Pattern[str], type[DynamicItem[Item[Any]]], /) -> type[DynamicItem[Item[Any]]]
- discord/ui/view.py:961: note: def [_T] pop(self, Pattern[str], _T, /) -> type[DynamicItem[Item[Any]]] | _T
+ discord/ui/view.py:986: error: Unused "type: ignore" comment [unused-ignore]
- discord/ui/view.py:981: error: No overload variant of "pop" of "dict" matches argument types "tuple[int, str]", "None" [call-overload]
- discord/ui/view.py:981: note: Possible overload variants:
- discord/ui/view.py:981: note: def pop(self, Pattern[str], /) -> type[DynamicItem[Item[Any]]]
- discord/ui/view.py:981: note: def pop(self, Pattern[str], type[DynamicItem[Item[Any]]], /) -> type[DynamicItem[Item[Any]]]
- discord/ui/view.py:981: note: def [_T] pop(self, Pattern[str], _T, /) -> type[DynamicItem[Item[Any]]] | _T
steam.py (https://github.com/Gobot1234/steam.py)
- steam/ext/commands/utils.py:43: note: def get(self, str, None = ..., /) -> _VT | None
+ steam/ext/commands/utils.py:43: note: def get(self, object, None = ..., /) -> _VT | None
- steam/ext/commands/utils.py:43: note: def get(self, str, _VT, /) -> _VT
+ steam/ext/commands/utils.py:43: note: def get(self, object, _VT, /) -> _VT
- steam/ext/commands/utils.py:43: note: def [_T] get(self, str, _T, /) -> _VT | _T
+ steam/ext/commands/utils.py:43: note: def [_T] get(self, object, _T, /) -> _VT | _T
- steam/ext/commands/utils.py:52: note: def pop(self, str, /) -> _VT
+ steam/ext/commands/utils.py:52: note: def pop(self, object, /) -> _VT
- steam/ext/commands/utils.py:52: note: def pop(self, str, _VT, /) -> _VT
+ steam/ext/commands/utils.py:52: note: def pop(self, object, _VT, /) -> _VT
- steam/ext/commands/utils.py:52: note: def [_T] pop(self, str, _T, /) -> _VT | _T
+ steam/ext/commands/utils.py:52: note: def [_T] pop(self, object, _T, /) -> _VT | _T
+ steam/ext/commands/bot.py:537: error: Unused "type: ignore" comment [unused-ignore]
werkzeug (https://github.com/pallets/werkzeug)
+ src/werkzeug/datastructures/mixins.py:265: error: Signature of "pop" incompatible with supertype "builtins.dict" [override]
+ src/werkzeug/datastructures/mixins.py:265: note: Superclass:
+ src/werkzeug/datastructures/mixins.py:265: note: @overload
+ src/werkzeug/datastructures/mixins.py:265: note: def pop(self, object, /) -> V
+ src/werkzeug/datastructures/mixins.py:265: note: @overload
+ src/werkzeug/datastructures/mixins.py:265: note: def pop(self, object, V, /) -> V
+ src/werkzeug/datastructures/mixins.py:265: note: @overload
+ src/werkzeug/datastructures/mixins.py:265: note: def [_T] pop(self, object, _T, /) -> V | _T
+ src/werkzeug/datastructures/mixins.py:265: note: Subclass:
+ src/werkzeug/datastructures/mixins.py:265: note: @overload
+ src/werkzeug/datastructures/mixins.py:265: note: def pop(self, key: K) -> V
+ src/werkzeug/datastructures/mixins.py:265: note: @overload
+ src/werkzeug/datastructures/mixins.py:265: note: def pop(self, key: K, default: V) -> V
+ src/werkzeug/datastructures/mixins.py:265: note: @overload
+ src/werkzeug/datastructures/mixins.py:265: note: def [T] pop(self, key: K, default: T) -> T
+ src/werkzeug/datastructures/structures.py:471: error: Signature of "pop" incompatible with supertype "builtins.dict" [override]
+ src/werkzeug/datastructures/structures.py:471: note: Superclass:
+ src/werkzeug/datastructures/structures.py:471: note: @overload
+ src/werkzeug/datastructures/structures.py:471: note: def pop(self, object, /) -> V
+ src/werkzeug/datastructures/structures.py:471: note: @overload
+ src/werkzeug/datastructures/structures.py:471: note: def pop(self, object, V, /) -> V
+ src/werkzeug/datastructures/structures.py:471: note: @overload
+ src/werkzeug/datastructures/structures.py:471: note: def [_T] pop(self, object, _T, /) -> V | _T
+ src/werkzeug/datastructures/structures.py:471: note: Subclass:
+ src/werkzeug/datastructures/structures.py:471: note: @overload
+ src/werkzeug/datastructures/structures.py:471: note: def pop(self, key: K) -> V
+ src/werkzeug/datastructures/structures.py:471: note: @overload
+ src/werkzeug/datastructures/structures.py:471: note: def pop(self, key: K, default: V) -> V
+ src/werkzeug/datastructures/structures.py:471: note: @overload
+ src/werkzeug/datastructures/structures.py:471: note: def [T] pop(self, key: K, default: T) -> V | T
+ src/werkzeug/debug/__init__.py:551: error: Unused "type: ignore" comment [unused-ignore]
jinja (https://github.com/pallets/jinja)
+ src/jinja2/compiler.py:1422: error: Unused "type: ignore" comment [unused-ignore]
+ src/jinja2/environment.py:514: error: Unused "type: ignore" comment [unused-ignore]
cki-lib (https://gitlab.com/cki-project/cki-lib)
- tests/utils.py:22: error: Argument 1 to "get" of "dict" has incompatible type "int | str"; expected "str" [arg-type]
- tests/utils.py:22: error: Argument 2 to "get" of "dict" has incompatible type "int | str"; expected "int" [arg-type]
operator (https://github.com/canonical/operator)
- ops/_private/harness.py:2649: error: No overload variant of "pop" of "dict" matches argument types "str", "None" [call-overload]
- ops/_private/harness.py:2649: note: Possible overload variants:
- ops/_private/harness.py:2649: note: def pop(self, int, /) -> dict[str, Any]
- ops/_private/harness.py:2649: note: def pop(self, int, dict[str, Any], /) -> dict[str, Any]
- ops/_private/harness.py:2649: note: def [_T] pop(self, int, _T, /) -> dict[str, Any] | _T
kornia (https://github.com/kornia/kornia)
- kornia/io/io.py:90: error: Argument 1 to "get" of "dict" has incompatible type "int | None"; expected "int" [arg-type]
jax (https://github.com/google/jax)
- jax/_src/interpreters/partial_eval.py:1270: error: Argument 1 to "get" of "dict" has incompatible type "Var | Literal | None"; expected "Var" [arg-type]
- jax/_src/interpreters/mlir.py:317: error: Argument 1 to "get" of "dict" has incompatible type "tuple[int, AbstractValue | None]"; expected "tuple[int, AbstractValue]" [arg-type]
mongo-python-driver (https://github.com/mongodb/mongo-python-driver)
+ bson/son.py:142: error: Signature of "pop" incompatible with supertype "builtins.dict" [override]
+ bson/son.py:142: note: Superclass:
+ bson/son.py:142: note: @overload
+ bson/son.py:142: note: def pop(self, object, /) -> _Value
+ bson/son.py:142: note: @overload
+ bson/son.py:142: note: def pop(self, object, _Value, /) -> _Value
+ bson/son.py:142: note: @overload
+ bson/son.py:142: note: def [_T] pop(self, object, _T, /) -> _Value | _T
+ bson/son.py:142: note: Subclass:
+ bson/son.py:142: note: def [_T] pop(self, key: _Key, *args: _Value | _T) -> _Value | _T
setuptools (https://github.com/pypa/setuptools)
+ setuptools/_static.py:170: error: Unused "type: ignore" comment [unused-ignore]
pylox (https://github.com/sco1/pylox)
+ pylox/containers/array.py:83: note: def pop(self, object, /) -> Any
- pylox/containers/array.py:83: note: def pop(self, Any, /) -> Any
+ pylox/containers/array.py:83: note: def pop(self, object, Any, /) -> Any
- pylox/containers/array.py:83: note: def pop(self, Any, Any, /) -> Any
- pylox/containers/array.py:83: note: def [_T] pop(self, Any, _T, /) -> Any | _T
+ pylox/containers/array.py:83: note: def [_T] pop(self, object, _T, /) -> Any | _T
rclip (https://github.com/yurijmikhalevich/rclip)
- rclip/utils/tokenizer.py:90: error: No overload variant of "get" of "dict" matches argument types "tuple[str, str]", "float" [call-overload]
- rclip/utils/tokenizer.py:90: note: Error code "call-overload" not covered by "type: ignore" comment
- rclip/utils/tokenizer.py:90: note: Possible overload variants:
- rclip/utils/tokenizer.py:90: note: def get(self, str, None = ..., /) -> int | None
- rclip/utils/tokenizer.py:90: note: def get(self, str, int, /) -> int
- rclip/utils/tokenizer.py:90: note: def [_T] get(self, str, _T, /) -> int | _T
rotki (https://github.com/rotki/rotki)
+ rotkehlchen/tasks/events.py:827: error: Unused "type: ignore" comment [unused-ignore]
+ rotkehlchen/chain/aggregator.py:706: error: Unused "type: ignore" comment [unused-ignore]
+ rotkehlchen/chain/aggregator.py:896: error: Unused "type: ignore" comment [unused-ignore]
pandas (https://github.com/pandas-dev/pandas)
+ pandas/core/arrays/arrow/array.py:3649: error: Unused "type: ignore" comment [unused-ignore]
django-stubs (https://github.com/typeddjango/django-stubs)
+ django-stubs/utils/datastructures.pyi:77: error: Signature of "get" incompatible with supertype "builtins.dict" [override]
+ django-stubs/utils/datastructures.pyi:77: note: Superclass:
+ django-stubs/utils/datastructures.pyi:77: note: @overload
+ django-stubs/utils/datastructures.pyi:77: note: def get(self, object, None = ..., /) -> _V | None
+ django-stubs/utils/datastructures.pyi:77: note: @overload
+ django-stubs/utils/datastructures.pyi:77: note: def get(self, object, _V, /) -> _V
+ django-stubs/utils/datastructures.pyi:77: note: @overload
+ django-stubs/utils/datastructures.pyi:77: note: def [_T] get(self, object, _T, /) -> _V | _T
+ django-stubs/utils/datastructures.pyi:77: note: Subclass:
+ django-stubs/utils/datastructures.pyi:77: note: @overload
+ django-stubs/utils/datastructures.pyi:77: note: def get(self, key: _K, default: None = ...) -> _V | None
+ django-stubs/utils/datastructures.pyi:77: note: @overload
+ django-stubs/utils/datastructures.pyi:77: note: def get(self, key: _K, default: _V) -> _V
+ django-stubs/utils/datastructures.pyi:77: note: @overload
+ django-stubs/utils/datastructures.pyi:77: note: def [_Z] get(self, key: _K, default: _Z) -> _V | _Z
+ django-stubs/contrib/sessions/backends/base.pyi:33: error: Signature of "pop" incompatible with supertype "builtins.dict" [override]
+ django-stubs/contrib/sessions/backends/base.pyi:33: note: Superclass:
+ django-stubs/contrib/sessions/backends/base.pyi:33: note: @overload
+ django-stubs/contrib/sessions/backends/base.pyi:33: note: def pop(self, object, /) -> Any
+ django-stubs/contrib/sessions/backends/base.pyi:33: note: @overload
+ django-stubs/contrib/sessions/backends/base.pyi:33: note: def pop(self, object, Any, /) -> Any
+ django-stubs/contrib/sessions/backends/base.pyi:33: note: @overload
+ django-stubs/contrib/sessions/backends/base.pyi:33: note: def [_T] pop(self, object, _T, /) -> Any | _T
+ django-stubs/contrib/sessions/backends/base.pyi:33: note: Subclass:
+ django-stubs/contrib/sessions/backends/base.pyi:33: note: @overload
+ django-stubs/contrib/sessions/backends/base.pyi:33: note: def pop(self, key: str) -> Any
+ django-stubs/contrib/sessions/backends/base.pyi:33: note: @overload
+ django-stubs/contrib/sessions/backends/base.pyi:33: note: def pop(self, key: str, default: Any) -> Any
+ django-stubs/http/request.pyi:226: error: Signature of "pop" incompatible with supertype "builtins.dict" [override]
+ django-stubs/http/request.pyi:226: note: Superclass:
+ django-stubs/http/request.pyi:226: note: @overload
+ django-stubs/http/request.pyi:226: note: def pop(self, object, /) -> str
+ django-stubs/http/request.pyi:226: note: @overload
+ django-stubs/http/request.pyi:226: note: def pop(self, object, str, /) -> str
+ django-stubs/http/request.pyi:226: note: @overload
+ django-stubs/http/request.pyi:226: note: def [_T] pop(self, object, _T, /) -> str | _T
+ django-stubs/http/request.pyi:226: note: Subclass:
+ django-stubs/http/request.pyi:226: note: @overload
+ django-stubs/http/request.pyi:226: note: def pop(self, str | bytes, /) -> Never
+ django-stubs/http/request.pyi:226: note: @overload
+ django-stubs/http/request.pyi:226: note: def [_Z] pop(self, str | bytes, str | _Z = ..., /) -> Never
|
|
I don't think this is a good idea. If you use the wrong thing as the key, you will not get an error. After all, the type checker is supposed to check the types. For this PR, mypy_primer is not a very good way to consider the real-world impact of this, because most projects don't contain the mistakes that the key parameter's type is supposed to catch: someone has already noticed and fixed those bugs likely before committing to git, perhaps because their editor complained that the key type was wrong. IMO we should start by allowing |
|
@Akuli This has already been discussed in some of the linked issues, the current behavior leads to false positives and doesn't align with runtime semantics. What you want is a linter rule, but such a rule should live as a special case inside a type-checker rather than typeshed itself. There are some very clear cases were the currently behavior hurts, such as literal vs dynamic string, union types, etc. If we get something like @srittau's suggested |
Fixes #15271
Supersedes: #15297
This PR only changes
dict, but notMutableMapping, and changes the signature ofdict.getanddict.popto allowobjectas key arguments. (allows these methods to work with partially overlapping types)