Abstract

This PEP proposes making function objects subscriptable for typing purposes. Doing so gives developers explicit control over the types produced by the type checker where bi-directional inference (which allows for the types of parameters of anonymous functions to be inferred) and other methods than specialisation are insufficient. It also makes functions consistent with regular classes in their ability to be subscripted.

Motivation

Currently, classes allow passing type annotations for generic containers. This is especially useful in common constructors such as list, tuple and dict etc.

my_integer_list = list[int]()
reveal_type(my_integer_list)  # type is list[int]

At runtime list[int] returns a GenericAlias that can be later called, returning an empty list.

Another example of this is creating a specialised dict type for a section of our code where we want to ensure that keys are str and values are int:

NameNumberDict = dict[str, int]

NameNumberDict(
   one=1,
   two=2,
   three="3"  # Invalid: Literal["3"] is not of type int
)

In spite of the utility of this syntax, when trying to use it with a function, an error is raised, as functions are not subscriptable.

def my_list[T](arr: Iterable[T]) -> list[T]:
   # do something...
   return list(arr)

my_integer_list = my_list[int]()  # TypeError: 'function' object is not subscriptable

There are a few workarounds:

1. Making a callable class:

class my_list[T]:
     def __call__(self, arr: Iterable[T]) -> list[T]:
         # do something...
         return list(arr)

my_string_list = my_list[str]([])

2. Using PEP 747's TypeForm, with an extra unused argument:

from typing import TypeForm

def my_list(*arr: Iterable[T], typ: TypeForm[T]) -> list[T]:
   # do something...
   return list(arr)

my_string_list = my_list([], str)

As we can see this solution increases the complexity with an extra argument. Additionally it requires the user to understand a new concept TypeForm.

3. Annotating the assignment:

my_integer_list: list[int] = my_list()

This solution isn’t optimal as the return type is repeated, is more verbose and would require the type updating in multiple places if the return type changes. Additionally, it adds unnecesary and distracting verbossity when the intention is to pass the specialized value into another call.

In conclusion, the current workarounds are too complex or verbose, especially compared to syntax that is consistent with the rest of the language.

NameNumberDict = dict[str, int]
NameNumberDict(one=1, two=2, three="3")  # Invalid: Literal["3"] is not of type int``

def constrained_addition[T](a: T, b: T) -> T: ...

# where we work exclusively with ints
int_addition = constrained_addition[int]
int_addition(2, 4+8j)  # Invalid: complex is not of type int

def factory[T](func: Callable[[T], Any]) -> Foo[T]: ...

reveal_type(factory(lambda x: "Hello World" * x)) # type is Foo[Unknown]

reveal_type(factory[int](lambda x: "Hello World" * x))  # type is Foo[int]

def foo[T](x: list[T]) -> T: ...
reveal_type(foo([]))  # type checker cannot infer T (yet again)

reveal_type(foo[int]([]))  # type is int

words = ["hello", "world"]
foo[int](words)  # Invalid: list[str] is incompatible with list[int]

make_int_list = make_list[int]
reveal_type(make_int_list())  # type is list[int]

def foo[T]():
   return T.__value__

assert foo[int]() is int

Rationale

This proposal improves the consistency of the type system, by allowing syntax that already looks and feels like a natural of the existing syntax for classes.

If accepted, this syntax will reduce the necessity to learn about PEP 747s TypeForm, reduce verbosity and cognitive load of safely typed python.

Specification

In this PEP “Function objects” is used to refer to FunctionType, MethodType, BuiltinFunctionType, BuiltinMethodType and MethodWrapperType.

For MethodType you should be able to write:

class Foo:
    def make_list[T](self, *args: T) -> list[T]: ...

Foo().make_list[int]()

and have it work similarly to a FunctionType.

For BuiltinFunctionType, so builtin generic functions (e.g. max and min) work like ones defined in Python. Built-in functions should behave as much like functions implemented in Python as possible.

BuiltinMethodType is the same type as BuiltinFunctionType.

MethodWrapperType (e.g. the type of object().__str__) is useful for generic magic methods.

Function objects should implement __getitem__ to allow for subscription at runtime and return an instance of types.GenericAlias with __origin__ set as the callable and __args__ as the types passed.

Type checkers should support subscripting functions and understand that the parameters passed to the function subscription should follow the same rules as a generic callable class.

class Foo[T]: ...

assert Foo[int]().__orig_class__ == Foo[int]

def bar[U]():
    return Foo[int]()

assert bar[str]().__orig_class__  == Foo[int]

@overload
def serializer_for[T: str]() -> StringSerializer: ...
@overload
def serializer_for[T: list]() -> ListSerializer: ...

def serializer_for():
    ...

@overload
def make[*Ts]() -> float: ...
@overload
def make[T]() -> int: ...

make[int]  # matches first and second overload
make[int, str]  # matches only first

def foo[*T, *U](bar: Bar[*T], baz: Baz[*U]): ...
def spam[*T](bar: Bar[*T]): ...

foo[int, str, bool, complex](Bar(), Baz())  # Invalid: cannot determine which parameters are passed to *T and *U. Explicitly parameterise the instances individually
spam[int, str, bool, complex](Bar())        # OK

class C[T]:
    def method[U](self, x: T, y: U): ...
    @classmethod
    def cls[U](cls, x: T, y: U): ...

C[int].method[str](0, "")  # OK
C[int].cls[str](0, "")     # OK
C.cls[int, str](0, "")     # Invalid: too many type parameters
C.cls[str](0, "")          # OK, U will be matched to str

Backwards Compatibility

Currently these classes are not subclassable and so there are no backwards compatibility concerns with regards to classes already implementing __getitem__.

Reference Implementation

The runtime changes proposed can be found here https://github.com/Gobot1234/cpython/tree/function-subscript

Acknowledgements

Thank you to Alex Waygood and Jelle Zijlstra for their feedback on this PEP and Guido for some motivating examples.

Copyright

This document is placed in the public domain or under the CC0-1.0-Universal license, whichever is more permissive.