Conversation
| if TYPE_CHECKING: | ||
| from typing_extensions import Buffer | ||
| if sys.version_info >= (3, 12): | ||
| from collections.abc import Buffer | ||
| else: | ||
| if sys.version_info >= (3, 12): | ||
| from collections.abc import Buffer | ||
| else: | ||
| Buffer = Any |
There was a problem hiding this comment.
This will ensure that Buffer annotations aren't Any on python<3.12. The reason this doesn't influence mypy's reports in CI is because the mypy job uses Python 3.12
There was a problem hiding this comment.
But if you want we could just revert this. It's somewhat out of scope, after all.
There was a problem hiding this comment.
If I test this on Python 3.11, it still passes - https://github.com/radarhere/Pillow/actions/runs/21477253617/job/61864264358
There was a problem hiding this comment.
Ah yea of course, ignore my previous comment about Ci versions; Any is still valid. It's just (a lot) less type-safe than Buffer.
| _pulls_fd = True | ||
|
|
||
| def decode(self, buffer: Buffer | Image.SupportsArrayInterface) -> tuple[int, int]: | ||
| def decode(self, buffer: bytes | Image.SupportsArrayInterface) -> tuple[int, int]: |
There was a problem hiding this comment.
Hmm strange, I thought this was an LSP violation 🤔. Maybe it's a mypy false negative? Either way, I'm fine with reverting this :)
|
I've looked through the rest of the changes, and pushed another commit that passes - https://github.com/radarhere/Pillow/actions/runs/21507567669/job/61966864558 I primarily want to clarify that not all of the changes in the PR flow out of the change in mypy settings. |
There was a problem hiding this comment.
The bytes / Buffer changes are necessary to avoid LSP violations, and mypy will warn about that:
I'm not sure why mypy isn't reporting this in CI, because if I locally run tox -e mypy after reverting the Buffer/bytes change in DdsRgbDecoder.decode I see the following:
mypy: commands[0]> mypy conftest.py selftest.py setup.py checks docs src winbuild Tests
/home/joren/Workspace/Pillow/src/PIL/DdsImagePlugin.py:492: error: Argument 1 of "decode" is incompatible with supertype "PIL.ImageFile.PyDecoder"; supertype defines the argument type as "Buffer | SupportsArrayInterface" [override]
def decode(self, buffer: bytes | Image.SupportsArrayInterface) -> tuple[int, int]:
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/home/joren/Workspace/Pillow/src/PIL/DdsImagePlugin.py:492: note: This violates the Liskov substitution principle
/home/joren/Workspace/Pillow/src/PIL/DdsImagePlugin.py:492: note: See https://mypy.readthedocs.io/en/stable/common_issues.html#incompatible-overrides
Found 1 error in 1 file (checked 303 source files)
mypy: exit 1 (0.22 seconds) /home/joren/Workspace/Pillow> mypy conftest.py selftest.py setup.py checks docs src winbuild Tests pid=17711
mypy: FAIL code 1 (0.23=setup[0.01]+cmd[0.22] seconds)
evaluation failed :( (0.24 seconds)
I also see the error if I instead run uvx --with tox-uv tox -e mypy, like in CI.
So the only conclusion I can draw is that this is a CI problem, and that at least the bytes/Buffer changes are actually necessary.
| if sys.version_info >= (3, 12): | ||
| from collections.abc import Buffer | ||
| else: | ||
| if sys.version_info >= (3, 12): | ||
| from collections.abc import Buffer | ||
| else: | ||
| Buffer = Any | ||
| Buffer = Any |
There was a problem hiding this comment.
Ah I figured it out!
In pyproject.toml, mypy is configured to always run with python_version = "3.10", regardless of the enviroment.
So by undoing this change, mypy will interpret Buffer as Any. This will cause all the LSP violations (i.e. override errors) in the decode methods that use bytes instead of Buffer to not be picked up.
To illustrate: if you locally change the python_version in pyproject.toml to 3.12 or higher (or remove it if you're in a python>=3.12 env) and run tox -e mypy, you'll see the following:
mypy: commands[0]> mypy conftest.py selftest.py setup.py checks docs src winbuild Tests
src/PIL/PdfParser.py:329: error: Incompatible return value type (got "Buffer", expected "bytes") [return-value]
return self.buf
^~~~~~~~
src/PIL/PpmImagePlugin.py:287: error: Argument 1 of "decode" is incompatible with supertype "PIL.ImageFile.PyDecoder"; supertype defines the argument type as "Buffer | SupportsArrayInterface" [override]
def decode(self, buffer: bytes | Image.SupportsArrayInterface) -> tuple[int, int]:
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
src/PIL/PpmImagePlugin.py:287: note: This violates the Liskov substitution principle
src/PIL/PpmImagePlugin.py:287: note: See https://mypy.readthedocs.io/en/stable/common_issues.html#incompatible-overrides
src/PIL/PpmImagePlugin.py:303: error: Argument 1 of "decode" is incompatible with supertype "PIL.ImageFile.PyDecoder"; supertype defines the argument type as "Buffer | SupportsArrayInterface" [override]
def decode(self, buffer: bytes | Image.SupportsArrayInterface) -> tuple[int, int]:
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
src/PIL/PpmImagePlugin.py:303: note: This violates the Liskov substitution principle
src/PIL/PpmImagePlugin.py:303: note: See https://mypy.readthedocs.io/en/stable/common_issues.html#incompatible-overrides
src/PIL/BmpImagePlugin.py:330: error: Argument 1 of "decode" is incompatible with supertype "PIL.ImageFile.PyDecoder"; supertype defines the argument type as "Buffer | SupportsArrayInterface" [override]
def decode(self, buffer: bytes | Image.SupportsArrayInterface) -> tuple[int, int]:
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
src/PIL/BmpImagePlugin.py:330: note: This violates the Liskov substitution principle
src/PIL/BmpImagePlugin.py:330: note: See https://mypy.readthedocs.io/en/stable/common_issues.html#incompatible-overrides
src/PIL/GifImagePlugin.py:1193: error: Argument 1 to "len" has incompatible type "Buffer"; expected "Sized" [arg-type]
return len(data)
^~~~
src/PIL/GifImagePlugin.py:1201: error: Incompatible return value type (got "list[Buffer]", expected "list[bytes]") [return-value]
return fp.data
^~~~~~~
src/PIL/XpmImagePlugin.py:121: error: Argument 1 of "decode" is incompatible with supertype "PIL.ImageFile.PyDecoder"; supertype defines the argument type as "Buffer | SupportsArrayInterface" [override]
def decode(self, buffer: bytes | Image.SupportsArrayInterface) -> tuple[int, int]:
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
src/PIL/XpmImagePlugin.py:121: note: This violates the Liskov substitution principle
src/PIL/XpmImagePlugin.py:121: note: See https://mypy.readthedocs.io/en/stable/common_issues.html#incompatible-overrides
src/PIL/SgiImagePlugin.py:201: error: Argument 1 of "decode" is incompatible with supertype "PIL.ImageFile.PyDecoder"; supertype defines the argument type as "Buffer | SupportsArrayInterface" [override]
def decode(self, buffer: bytes | Image.SupportsArrayInterface) -> tuple[int, int]:
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
src/PIL/SgiImagePlugin.py:201: note: This violates the Liskov substitution principle
src/PIL/SgiImagePlugin.py:201: note: See https://mypy.readthedocs.io/en/stable/common_issues.html#incompatible-overrides
src/PIL/QoiImagePlugin.py:54: error: Argument 1 of "decode" is incompatible with supertype "PIL.ImageFile.PyDecoder"; supertype defines the argument type as "Buffer | SupportsArrayInterface" [override]
def decode(self, buffer: bytes | Image.SupportsArrayInterface) -> tuple[int, int]:
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
src/PIL/QoiImagePlugin.py:54: note: This violates the Liskov substitution principle
src/PIL/QoiImagePlugin.py:54: note: See https://mypy.readthedocs.io/en/stable/common_issues.html#incompatible-overrides
src/PIL/MspImagePlugin.py:115: error: Argument 1 of "decode" is incompatible with supertype "PIL.ImageFile.PyDecoder"; supertype defines the argument type as "Buffer | SupportsArrayInterface" [override]
def decode(self, buffer: bytes | Image.SupportsArrayInterface) -> tuple[int, int]:
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
src/PIL/MspImagePlugin.py:115: note: This violates the Liskov substitution principle
src/PIL/MspImagePlugin.py:115: note: See https://mypy.readthedocs.io/en/stable/common_issues.html#incompatible-overrides
src/PIL/FitsImagePlugin.py:129: error: Argument 1 of "decode" is incompatible with supertype "PIL.ImageFile.PyDecoder"; supertype defines the argument type as "Buffer | SupportsArrayInterface" [override]
def decode(self, buffer: bytes | Image.SupportsArrayInterface) -> tuple[int, int]:
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
src/PIL/FitsImagePlugin.py:129: note: This violates the Liskov substitution principle
src/PIL/FitsImagePlugin.py:129: note: See https://mypy.readthedocs.io/en/stable/common_issues.html#incompatible-overrides
src/PIL/DdsImagePlugin.py:491: error: Argument 1 of "decode" is incompatible with supertype "PIL.ImageFile.PyDecoder"; supertype defines the argument type as "Buffer | SupportsArrayInterface" [override]
def decode(self, buffer: bytes | Image.SupportsArrayInterface) -> tuple[int, int]:
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
src/PIL/DdsImagePlugin.py:491: note: This violates the Liskov substitution principle
src/PIL/DdsImagePlugin.py:491: note: See https://mypy.readthedocs.io/en/stable/common_issues.html#incompatible-overrides
src/PIL/BlpImagePlugin.py:298: error: Argument 1 of "decode" is incompatible with supertype "PIL.ImageFile.PyDecoder"; supertype defines the argument type as "Buffer | SupportsArrayInterface" [override]
def decode(self, buffer: bytes | Image.SupportsArrayInterface) -> tuple[int, int]:
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
src/PIL/BlpImagePlugin.py:298: note: This violates the Liskov substitution principle
src/PIL/BlpImagePlugin.py:298: note: See https://mypy.readthedocs.io/en/stable/common_issues.html#incompatible-overrides
docs/example/DdsImagePlugin.py:261: error: Argument 1 of "decode" is incompatible with supertype "PIL.ImageFile.PyDecoder"; supertype defines the argument type as "Buffer | SupportsArrayInterface" [override]
def decode(self, buffer: bytes | Image.SupportsArrayInterface) -> tuple[int, int]:
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
docs/example/DdsImagePlugin.py:261: note: This violates the Liskov substitution principle
docs/example/DdsImagePlugin.py:261: note: See https://mypy.readthedocs.io/en/stable/common_issues.html#incompatible-overrides
docs/example/DdsImagePlugin.py:274: error: Argument 1 of "decode" is incompatible with supertype "PIL.ImageFile.PyDecoder"; supertype defines the argument type as "Buffer | SupportsArrayInterface" [override]
def decode(self, buffer: bytes | Image.SupportsArrayInterface) -> tuple[int, int]:
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
docs/example/DdsImagePlugin.py:274: note: This violates the Liskov substitution principle
docs/example/DdsImagePlugin.py:274: note: See https://mypy.readthedocs.io/en/stable/common_issues.html#incompatible-overrides
Tests/test_imagefile.py:241: error: Argument 1 of "decode" is incompatible with supertype "PIL.ImageFile.PyDecoder"; supertype defines the argument type as "Buffer | SupportsArrayInterface" [override]
def decode(self, buffer: bytes | Image.SupportsArrayInterface) -> tuple[int, int]:
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Tests/test_imagefile.py:241: note: This violates the Liskov substitution principle
Tests/test_imagefile.py:241: note: See https://mypy.readthedocs.io/en/stable/common_issues.html#incompatible-overrides
Tests/test_file_jpeg.py:1067: error: Argument 1 of "decode" is incompatible with supertype "PIL.ImageFile.PyDecoder"; supertype defines the argument type as "Buffer | SupportsArrayInterface" [override]
self, buffer: bytes | Image.SupportsArrayInterface
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Tests/test_file_jpeg.py:1067: note: This violates the Liskov substitution principle
Tests/test_file_jpeg.py:1067: note: See https://mypy.readthedocs.io/en/stable/common_issues.html#incompatible-overrides
Tests/test_image.py:205: error: Argument 1 to "len" has incompatible type "Buffer"; expected "Sized" [arg-type]
return len(data)
^~~~
Found 18 errors in 15 files (checked 303 source files)
mypy: exit 1 (3.07 seconds) /home/joren/Workspace/Pillow> mypy conftest.py selftest.py setup.py checks docs src winbuild Tests pid=20145
mypy: FAIL code 1 (3.07=setup[0.01]+cmd[3.07] seconds)
evaluation failed :( (3.09 seconds)
So most of these changes are necessary. Without them, the annotations will be invalid on python>=3.12.
There was a problem hiding this comment.
Ok, to try and test different versions properly then, I've created python-pillow#9414
This is not an exhaustive exploration of python-pillow#9410, but... I find that I can revert a number of your changes without causing the Lint job to fail? https://github.com/radarhere/Pillow/actions/runs/21475441596