Repository navigation
Please allow tweaking the SquashFS compression #35
Description
Hi devs,
first of all thanks a lot for this fine piece of software, which seems to be the only viable successor of linuxdeploy-qt :-)
I searched for ways to generate the smallest possible AppImage for a Qt application. I use a static build, which greatly reduces the size, but leaving out features doesn't help much anymore.
So I examined if the compression method used to create the AppImage itself could be tweaked. Zstd is used to compress the data. SquashFS seems to not only support zstd, but also other compression algorithms – above all LZMA.
However, linuxdeploy uses an embedded version of mksquashfs, which only supports zstd. I found out that one could pass other compression algorithms to be used via the LDAI_COMP env variable – but as the packaged mksquashfs only supports zstd, that is not really meaningful.
I did a simple test to check out if LZMA was worth it. I mounted a test AppImage and tared the contents. The uncompressed tarball is 17.0M. Compressing it with zstd results in 7.3M, almost the size of the AppImage itself (7.5M). However, compressing it using xz, the tarball only takes 5.7M – that is almost 1/4 less!
While xz takes way longer to compress the data compared to zstd (5.6s vs. 0.1s), decompression is way faster. Admittedly, it's still slower for xz compared with zstd (0.2s vs. less than 0.1s), but considering the significant smaller AppImage, it seems to be worth it – as least as an option.
So – it would be really nice if one could choose to either use LZMA or zstd when creating an AppImage, preferably with the possibility to tweak the compression settings to be able to create smaller AppImages.
Cheers, Tobias
Activity
CJYate commentedon May 23, 2025
TheAssassin commentedon May 23, 2025
TheAssassin commentedon May 23, 2025
@l3u Zstd is known for its good decompression performance, so I'm wondering how xz could be significantly faster. That said, I think 0.2s isn't a bad startup time. Zstd is also used by default nowadays by the AppImage plugin (to which I will transfer this issue soon).
In the end, the AppImage plugin is just a wrapper around appimagetool. You can configure the compression with the
LDAI_COMPenvironment variable, provided that appimagetool supports it. See https://github.com/linuxdeploy/linuxdeploy-plugin-appimage.TheAssassin commentedon May 23, 2025
I apologize, I didn't read your (very lengthy) statement correctly. It states correctly that the flag exists. But you are requesting xz (LZMA) support. I think that was removed by appimagetool a while ago because Zstd provides the best trade-off of all compression algorithms. I'll find you a link.
TheAssassin commentedon May 23, 2025
See AppImage/appimagetool#69. There it's also mentioned that mksquashfs is built without xz support and bundled with appimagetool. Please open an issue at https://github.com/AppImage/appimagetool/.
Edit: by the way, thanks for the kind words!
l3u commentedon May 24, 2025
Thanks for the insight! Well, if LZMA support was there and has been removed intentionally – I fear requesting to add it again won't be very promising, will it?!
TheAssassin commentedon May 24, 2025
I am sure it would be considered (given the fact that I'll be one of the reviewers of such an issue). It just needs to be well founded.
Currently, we are investigating DwarFS as an alternative to squashfs.
Edit: xz/lzma used to be the default (instead of the also-supported zlib/gzip).
l3u commentedon May 25, 2025
Okay, then I'll file an issue about that.
I didn't knew DwarFS until now, thanks for the info! This is interesting! I just tested the latest pre-built mkdwarfs:
My final, zstd-compressed AppImage is 8.9M. When I unpack it, the squashfs-root is 22.0M. When I create a DwarFS from that, it shrinks to 6.5M. Plus some overhead for the AppImage stuff of course, but it definitely would be way smaller than the zstd version.
Would be nice if we could achieve better compression. The best thing would be if we simply had a choice here. Of course, zstd is really fast for what it compresses. But for my use-case, size would be more important than decompression speed, esp. when we're talking about split seconds.
l3u commentedon May 25, 2025
Here we are :-)
l3u commentedon May 25, 2025
… aaand it's already closed :-(
CJYate commentedon May 26, 2025
Massively helpful response, thank you!