镜像站点 · 本页由第三方 GitHub 只读镜像提供,非 GitHub 官方站点,不接受任何登录或凭据输入。前往 github.com
Skip to content

Speed up uncompressed DDS reader - #9943

Open
akx wants to merge 2 commits into
python-pillow:mainfrom
akx:dds-speedup
Open

akx wants to merge 2 commits into
python-pillow:mainfrom
akx:dds-speedup

Conversation

@akx

@akx akx commented Sep 2, 2026 •

Copy link
Copy Markdown
Contributor

Follows up on #7589. I noticed this looking at pytest --durations; a fuzzer test that just loads a DDS was surprisingly slow, and turns out doing 1-byte reads is not very efficient...

Locally:

$ pytest-benchmark compare --between=min .benchmarks/Darwin-CPython-3.14-64bit/*.json
Name (time in us)                   0001_orig Min  0002_new Min      ΔMin
-------------------------------------------------------------------------
test_load[uncompressed_rgb.dds]      405,475.7920    1,589.0000    -99.6%
-------------------------------------------------------------------------

or 255.17 times lower minimum, or in average OPS terms, +22351.7% faster. 😂

Comment thread src/PIL/DdsImagePlugin.py Resolved
Comment thread src/PIL/DdsImagePlugin.py Resolved
Comment thread Tests/benchmarks.py Resolved
@radarhere

radarhere commented Sep 11, 2026 •

Copy link
Copy Markdown
Member

As far as uncompressed_rgb.dds is concerned, I think I can do you one better. #7589 added DdsRgbDecoder in order to generalise this decoding path, but before that, this file was read with the raw decoder.

So if apply the following change to main to use the raw decoder for this specific scenario, you will find it faster than the current state of this PR.

diff --git a/src/PIL/DdsImagePlugin.py b/src/PIL/DdsImagePlugin.py
index 40012bc27..a6cec0557 100644
--- a/src/PIL/DdsImagePlugin.py
+++ b/src/PIL/DdsImagePlugin.py
@@ -371,8 +371,11 @@ class DdsImageFile(ImageFile.ImageFile):
                 mask_count = 3
 
             masks = struct.unpack(f"<{mask_count}I", header[84 : 84 + mask_count * 4])
-            self.tile = [ImageFile._Tile("dds_rgb", extents, 0, (bitcount, masks))]
-            return
+            if masks == (0xFF0000, 0x00FF00, 0x0000FF):
+                rawmode = "BGR"
+            else:
+                self.tile = [ImageFile._Tile("dds_rgb", extents, 0, (bitcount, masks))]
+                return
         elif pfflags & DDPF.LUMINANCE:
             if bitcount == 8:
                 self._mode = "L"

@akx
akx force-pushed the dds-speedup branch 2 times, most recently from e7021fe to dc4eeba Compare September 15, 2026 05:42
@akx

akx commented Sep 15, 2026

Copy link
Copy Markdown
Contributor Author

@radarhere Good stuff! 🎉 Applied that on here too. I checked pytest-cov, and with that applied the speedups from dadbf4f are still tested and used, so this is a double win :)

@akx

akx commented Sep 22, 2026

Copy link
Copy Markdown
Contributor Author

This would probably be nice to merge for testing/DX's sake as well.

Looks like PyPy is hit hard by this and it spends ~22% of the entire test run time on uncompressed_rgb.dds tests:

23.18s call     Tests/test_file_dds.py::test_uncompressed[RGBA-size4-Tests/images/uncompressed_rgb.dds]
22.96s call     Tests/oss-fuzz/test_fuzzers.py::test_fuzz_images[Tests/images/uncompressed_rgb.dds]

@codspeed

codspeed Bot commented Oct 23, 2026 •

Copy link
Copy Markdown

Merging this PR will not alter performance

1 warning

⚠️ 2 benchmarks are dominated by system calls

System calls cannot be simulated consistently and are left out of the measure. The reported value understates the real cost. Switch to the Walltime instrument to measure them.

Learn more about system calls

See affected benchmarks

✅ 668 untouched benchmarks
⏩ 352 skipped benchmarks1


Comparing akx:dds-speedup (cda38df) with main (842928a)

Open in CodSpeed

Footnotes

  1. 352 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

Comment thread src/PIL/DdsImagePlugin.py Resolved
Comment thread src/PIL/DdsImagePlugin.py
self.set_as_raw(data)
return -1, 0
msg = f"Unsupported bitcount {bitcount}"
raise OSError(msg)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What was your thinking here? It would have previously been 'ValueError: not enough image data'. So you wanted to clarify?

No good faith image should ever trigger this, so I'm not too concerned.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep. The test needed changing for the new "Truncated File Read" error message, and once I did that, the test failed for that one file where we reached this path... so it was more obvious to just raise with a specific error here and catch that in the test.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants