overrides: False positive when checking parameter with Callable

I also just tried with the list example. I can’t get it to work with the built-in list (mypy says “Variance of TypeVar “T” incompatible with variance in parent type”), but I don’t think the issue is specific to Callable either. I managed to define a custom class that behaves the same way:

from typing import TypeVar, Generic
from overrides import override

class Food: pass
class DogFood(Food): pass

T = TypeVar("T", contravariant=True)
class ContraList(Generic[T]):
	pass

class AnimalHerd:
	def feed_animals(self, bucket: ContraList[Food]): pass
class DogHerd(AnimalHerd):
	@override
	def feed_animals(self, bucket: ContraList[DogFood]): pass

Not that a ContraList would be practically useful… But mypy is happy with that, and the @override is not.

You’re correct that function parameter types are contravariant. However, it’s important to understand what this means. When we say function parameter types are contravariant, it means that if B is a subtype of A, then a function that accepts an A can be replaced with a function that accepts a B (not the other way around). In other words, the function that accepts the more general type is considered the “super type” when comparing functions.

So in the case of feed_any_animal: Callable[[Animal], None] and feed_only_dogs: Callable[[Dog], None], feed_any_animal is the super type of feed_only_dogs. That’s because feed_any_animal can replace feed_only_dogs in any code that uses feed_only_dogs without causing a type error.

To understand why, consider the following scenario:

Let’s say we have a function that calls feed_only_dogs:

def take_dog_to_dinner(feed_func: Callable[[Dog], None], dog: Dog):
    feed_func(dog)

You can pass feed_any_animal to this function without any issues:

take_dog_to_dinner(feed_any_animal, Dog())

However, if we have a similar function that calls feed_any_animal:

def take_animal_to_dinner(feed_func: Callable[[Animal], None], animal: Animal):
    feed_func(animal)

You can’t pass feed_only_dogs to this function if you’re planning to feed an animal that’s not a Dog:

take_animal_to_dinner(feed_only_dogs, Cat())  # This would be a problem!

In this sense, feed_any_animal can be used anywhere feed_only_dogs can be used, but the reverse is not true. This is why feed_any_animal is a super type of feed_only_dogs.

---

To your analogy about Fruit and Apple, that example is different from the function example because it deals with return types of methods rather than input parameters. When it comes to return types, it’s true that they are covariant, meaning that a function returning a B could be replaced with a function returning an A if B is a subtype of A. That’s the opposite of what happens with function parameter types, which are contravariant. So it’s not accurate to compare a function’s input parameters to a function’s return type when discussing covariance and contravariance.

You keep giving me references I already agree with…

Yes, if my example was talking about lists in the abstract, and their parameter types were invariant, then indeed list[Food] and list[DogFood] are incompatible. They would have nothing to do with each other.

But that’s not the case. In feed_animals(self, bucket: list[Food]), I’m using the bucket strictly to put stuff into it, same way the parameters of a Callable object can only go in.

Those Callable object are the feed functions that were used as arguments to feed_animals a few examples ago. If we write out the types for those feed functions in Java, it would look like

class Food {}
class DogFood extends Food {}

// Java's Callable[..., None]
Consumer<? super Food> feed_food = (food) -> {};
Consumer<? super DogFood> feed_dog_food = (food) -> {};

// feed_food = feed_dog_food;  // does not compile: incompatible types
feed_dog_food = feed_food;

We have the wild cards because Java uses use-site variance, and we have to explicitly write out the contravariant property of function parameter types. Python uses declare-site variance, and that property is built in to Callable.

Notice that we can assign feed_food of type Consumer<? super Food> to a variable declared to be Consumer<? super DogFood>, because the latter is a supertype of the former. If you try to assign the other way, the compiler will error.

In general, when we override a method in a subclass, the principle of substitutability must hold. This principle, also known as Liskov Substitution Principle (LSP), is fundamental in object-oriented programming and type theory.

Specifically, in the context of method overriding:

1. The input parameters (arguments) of the overriding method in the subclass can be the same as or wider (more general) than those in the overridden method of the superclass. This is known as contravariance of method arguments.

2. The return type of the overriding method in the subclass can be the same as or narrower (more specific) than that in the overridden method of the superclass. This is known as covariance of return types.

In your case, the types in the parameter are inverted. You’re passing Callable[[X], None] in class A and Callable[[Y], None] in class B, where Y is a subclass of X. This contradicts the principle of contravariance for method arguments.

Therefore, your example doesn’t comply with Liskov’s Substitution Principle, which is likely to cause a TypeError when trying to override the function f in class B.

To correct this, you would want to ensure the principle of contravariance for method arguments. So, class A should take Callable[[Y], None] and class B should take Callable[[X], None], as X is a wider type compared to Y. This allows any function that can handle X to be passed in, ensuring substitutability when going from A to B.

Here is the corrected code:

from typing import Callable
from overrides import override

class X: pass
class Y(X): pass

class A:
	def f(self, o: Callable[[Y], None]) -> None:
		pass

class B(A):
	@override
	def f(self, o: Callable[[X], None]) -> None:
		pass

Please note that the actual functionality of the classes and methods isn’t defined in the code you provided, so my advice is based on type theory and the Liskov Substitution Principle, and not on the specifics of the functionality.