diff --git a/peps/pep-0718.rst b/peps/pep-0718.rst index 13b4ccc1f9c..11259cff19b 100644 --- a/peps/pep-0718.rst +++ b/peps/pep-0718.rst @@ -1,6 +1,6 @@ PEP: 718 Title: Subscriptable functions -Author: James Hilton-Balfe +Author: James Hilton-Balfe , Pablo Ruiz Cuevas Sponsor: Guido van Rossum Discussions-To: https://discuss.python.org/t/28457/ Status: Draft @@ -17,41 +17,113 @@ This PEP proposes making function objects subscriptable for typing purposes. Doi gives developers explicit control over the types produced by the type checker where bi-directional inference (which allows for the types of parameters of anonymous functions to be inferred) and other methods than specialisation are insufficient. It -also brings functions in line with regular classes in their ability to be -subscriptable. +also makes functions consistent with regular classes in their ability to be +subscripted. Motivation ---------- -Unknown Types -^^^^^^^^^^^^^ +Currently, classes allow passing type annotations for generic containers, this +is especially useful in common constructors such as ``list``\, ``tuple`` and ``dict`` +etc. -Currently, it is not possible to infer the type parameters to generic functions in -certain situations: +.. code-block:: python + + my_integer_list = list[int]() + reveal_type(my_integer_list) # type is list[int] + +At runtime ``list[int]`` returns a ``GenericAlias`` that can be later called, returning +an empty list. + +Another example of this is creating a specialised ``dict`` type for a section of our +code where we want to ensure that keys are ``str`` and values are ``int``: + +.. code-block:: python + + NameNumberDict = dict[str, int] + + NameNumberDict( + one=1, + two=2, + three="3" # Invalid: Literal["3"] is not of type int + ) + +In spite of the utility of this syntax, when trying to use it with a function, an error +is raised, as functions are not subscriptable. + +.. code-block:: python + + def my_list[T](arr) -> list[T]: + # do something... + return list(arr) + + my_integer_list = my_list[int]() # TypeError: 'function' object is not subscriptable + +There are a few workarounds: + +1. Making a callable class: .. code-block:: python - def make_list[T](*args: T) -> list[T]: ... - reveal_type(make_list()) # type checker cannot infer a meaningful type for T + class my_list[T]: + def __call__(self, *args: T) -> list[T]: + # do something... + return list(args) -Making instances of ``FunctionType`` subscriptable would allow for this constructor to -be typed: +2. Using :pep:`747`\'s TypeForm, with an extra unused argument: .. code-block:: python - reveal_type(make_list[int]()) # type is list[int] + from typing import TypeForm + + def my_list(*args: T, typ: TypeForm[T]) -> list[T]: + # do something... + return list(args) + +As we can see this solution increases the complexity with an extra argument. +Additionally it requires the user to understand a new concept ``TypeForm``. -Currently you have to use an assignment to provide a precise type: +3. Annotating the assignment: .. code-block:: python - x: list[int] = make_list() - reveal_type(x) # type is list[int] + my_integer_list: list[int] = my_list() + +This solution isn't optimal as the return type is repeated and is more verbose and +would require the type updating in multiple places if the return type changes. + +In conclusion, the current workarounds are too complex or verbose, especially compared +to syntax that is consistent with the rest of the language. + +Generic Specialisation +^^^^^^^^^^^^^^^^^^^^^^ + +As in the previous example currently we can create generic aliases for different +specialised usages: + +.. code-block:: python + + NameNumberDict = dict[str, int] + NameNumberDict(one=1, two=2, three="3") # Invalid: Literal["3"] is not of type int`` + +This not currently possible for functions but if allowed we could easily +specialise operations in certain sections of the codebase: + +.. code-block:: python + + def constrained_addition[T](a: T, b: T) -> T: ... + + # where we work exclusively with ints + int_addition = constrained_addition[int] + int_addition(2, 4+8j) # Invalid: complex is not of type int + +Unknown Types +^^^^^^^^^^^^^ -but this code is unnecessarily verbose taking up multiple lines for a simple function -call. +Currently, it is not possible to infer the type parameters to generic functions in +certain situations. -Similarly, ``T`` in this example cannot currently be meaningfully inferred, so ``x`` is +In this example ``T`` cannot currently be meaningfully inferred, so ``x`` is untyped without an extra assignment: .. code-block:: python @@ -66,11 +138,11 @@ If function objects were subscriptable, however, a more specific type could be g reveal_type(factory[int](lambda x: "Hello World" * x)) # type is Foo[int] -Undecidable Inference -^^^^^^^^^^^^^^^^^^^^^ +Undecidable Inference and Type Narrowing +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -There are even cases where subclass relations make type inference impossible. However, -if you can specialise the function type checkers can infer a meaningful type. +There are cases where subclass relations make type inference impossible. However, if +you can specialise the function type checkers can infer a meaningful type. .. code-block:: python @@ -138,7 +210,16 @@ The syntax for such a feature may look something like: Rationale --------- -Function objects in this PEP is used to refer to ``FunctionType``\ , ``MethodType``\ , +This proposal improves the consistency of the type system, by allowing syntax that +already looks and feels like a natural of the existing syntax for classes. + +If accepted, this syntax will reduce the necessity to learn about :pep:`747`\s +``TypeForm``, reduce verbosity and cognitive load of safely typed python. + +Specification +------------- + +In this PEP "Function objects" is used to refer to ``FunctionType``\ , ``MethodType``\ , ``BuiltinFunctionType``\ , ``BuiltinMethodType`` and ``MethodWrapperType``\ . For ``MethodType`` you should be able to write: @@ -161,9 +242,6 @@ functions implemented in Python as possible. ``MethodWrapperType`` (e.g. the type of ``object().__str__``) is useful for generic magic methods. -Specification -------------- - Function objects should implement ``__getitem__`` to allow for subscription at runtime and return an instance of ``types.GenericAlias`` with ``__origin__`` set as the callable and ``__args__`` as the types passed. @@ -201,10 +279,68 @@ The following code snippet would fail at runtime without this change as Interactions with ``@typing.overload`` ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -Overloaded functions should work much the same as already, since they have no effect on -the runtime type. The only change is that more situations will be decidable and the -behaviour/overload can be specified by the developer rather than leaving it to ordering -of overloads/unions. +This PEP opens the door to overloading based on type variables: + +.. code-block:: python + + @overload + def serializer_for[T: str]() -> StringSerializer: ... + @overload + def serializer_for[T: list]() -> ListSerializer: ... + + def serializer_for(): + ... + +For overload resolution a new step will be required previous to any other, where the resolver +will match only the overloads where the subscription may succeed. + +.. code-block:: python + + @overload + def make[*Ts]() -> float: ... + @overload + def make[T]() -> int: ... + + make[int] # matches first and second overload + make[int, str] # matches only first + + +Functions Parameterized by ``TypeVarTuple``\ s +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +Currently, type checkers disallow the use of multiple ``TypeVarTuple``\s in their +generic parameters; however, it is currently valid to have a function as such: + +.. code-block:: python + + def foo[*T, *U](bar: Bar[*T], baz: Baz[*U]): ... + def spam[*T](bar: Bar[*T]): ... + +This PEP does not allow functions like ``foo`` to be subscripted, for the same reason +as defined in :pep:`PEP 646<646#multiple-type-variable-tuples-not-allowed>`. + +.. code-block:: python + + foo[int, str, bool, complex](Bar(), Baz()) # Invalid: cannot determine which parameters are passed to *T and *U. Explicitly parameterise the instances individually + spam[int, str, bool, complex](Bar()) # OK + +Binding Rules +^^^^^^^^^^^^^ +Method subscription (including ``classmethods``, ``staticmethods``, etc.) should only +have access to their function's type parameter and not the enclosing class's. +Subscription should follow the rules specified in :pep:`PEP 696<696#binding-rules>`; +methods should bind type parameters on attribute access. + +.. code-block:: python + + class C[T]: + def method[U](self, x: T, y: U): ... + @classmethod + def cls[U](cls, x: T, y: U): ... + + C[int].method[str](0, "") # OK + C[int].cls[str](0, "") # OK + C.cls[int, str](0, "") # Invalid: too many type parameters + C.cls[str](0, "") # OK, T is ideally bound to int here though this is open for type checkers to decide Backwards Compatibility -----------------------