diff --git a/.github/workflows/scorecard.yml b/.github/workflows/scorecard.yml
new file mode 100644
index 00000000..2a5b7f71
--- /dev/null
+++ b/.github/workflows/scorecard.yml
@@ -0,0 +1,46 @@
+# OpenSSF Scorecard supply-chain security analysis, published to https://scorecard.dev for the README badge
+name: Scorecard
+
+on:
+ push:
+ branches: [main]
+ schedule:
+ - cron: '17 4 * * 1'
+ # For the Branch-Protection check to pick up changes to branch protection rules
+ branch_protection_rule:
+ workflow_dispatch:
+
+permissions: {}
+
+jobs:
+ analysis:
+ name: Scorecard analysis
+ runs-on: ubuntu-latest
+ permissions:
+ # Upload results to the code scanning dashboard
+ security-events: write
+ # Publish results to the OpenSSF REST API
+ id-token: write
+ contents: read
+ actions: read
+ steps:
+ - name: Checkout
+ uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
+ with:
+ persist-credentials: false
+ - name: Run analysis
+ uses: ossf/scorecard-action@2d1146689b8cda280b9bc96326124645441f03bc # v2.4.4
+ with:
+ results_file: results.sarif
+ results_format: sarif
+ publish_results: true
+ - name: Upload artifact
+ uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7
+ with:
+ name: scorecard-results
+ path: results.sarif
+ retention-days: 5
+ - name: Upload to code scanning
+ uses: github/codeql-action/upload-sarif@2892aa5e19bbd11bc0cff5427e3b750a04d9e3c2 # v4.38.2
+ with:
+ sarif_file: results.sarif
diff --git a/.github/workflows/test.yml b/.github/workflows/test.yml
index 0a7f30bc..ec021c17 100644
--- a/.github/workflows/test.yml
+++ b/.github/workflows/test.yml
@@ -6,15 +6,24 @@ on:
pull_request:
workflow_dispatch:
+permissions:
+ contents: read
+
jobs:
linux:
name: Linux test
runs-on: ${{ matrix.runs-on }}
+ permissions:
+ contents: read
+ # Codecov upload via OIDC
+ id-token: write
strategy:
fail-fast: false
matrix:
include:
- runs-on: ubuntu-24.04
+ coverage: true
+ maven-flags: -Pcoverage
- runs-on: ubuntu-24.04-arm
# Azul Zulu 7 not available on Aarch64
maven-flags: >-
@@ -57,3 +66,13 @@ jobs:
-Dsurefire.reportFormat=plain \
${{ matrix.maven-flags }} \
verify
+ - name: Upload coverage to Codecov
+ if: matrix.coverage
+ uses: codecov/codecov-action@303a32d7a59b442fa8d48b6a1cc6825c09c847a5 # v7.1.1
+ with:
+ files: target/site/jacoco/jacoco.xml
+ disable_search: true
+ # OIDC tokens are not available to pull requests from other repositories, which use tokenless upload instead.
+ # Don't use head.repo.fork here: this repository is itself a fork, so that is always true.
+ use_oidc: ${{ github.event_name != 'pull_request' || github.event.pull_request.head.repo.full_name == github.repository }}
+ fail_ci_if_error: false
diff --git a/CHANGES.md b/CHANGES.md
index c2ddb3bf..74f4936c 100644
--- a/CHANGES.md
+++ b/CHANGES.md
@@ -1,5 +1,8 @@
# Change log
+*This changelog covers lz4-java 1.8.0 and earlier. For newer versions, see
+[GitHub Releases](https://github.com/yawkat/lz4-java/releases).*
+
## 1.8.0
- Upgraded LZ4 to 1.9.3. Updated the JNI bindings. Minimum glibc version in GNU/Linux platforms: 2.17 on aarch64, 2.2.5 on amd64, 2.17 on ppc64le, 2.2 on s390x.
diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
new file mode 100644
index 00000000..99bfea0a
--- /dev/null
+++ b/CONTRIBUTING.md
@@ -0,0 +1,39 @@
+# Contributing to lz4-java
+
+Thanks for your interest in contributing!
+
+## Reporting bugs and requesting features
+
+- **Bugs and feature requests:** open a [GitHub issue](https://github.com/yawkat/lz4-java/issues). For bugs, include
+ the lz4-java version, Java version, OS and architecture, which `LZ4Factory`/`XXHashFactory` instance you use, and
+ ideally a minimal reproducer.
+- **Questions:** use [GitHub Discussions](https://github.com/yawkat/lz4-java/discussions).
+- **Security vulnerabilities:** do **not** open a public issue. Report them privately through
+ [GitHub's vulnerability reporting](https://github.com/yawkat/lz4-java/security/advisories/new).
+
+## Pull requests
+
+Pull requests are welcome. For larger changes, please open an issue first to discuss the approach.
+
+To be accepted, a contribution must meet these requirements:
+
+- **The build passes.** `./mvnw verify` must succeed, and CI must be green on all platforms. See the
+ [README](README.md#building) for build prerequisites.
+- **Java 7 compatibility.** The library is compiled with Java 7 and must not use newer language features or APIs.
+ Tests may use the newer Java version configured in `pom.xml`.
+- **No breaking API changes.** Public API must stay source and binary compatible. Deprecate instead of removing, and
+ keep new non-user-facing members as private as possible.
+- **Tests.** Bug fixes should come with a regression test, and new features with tests. New tests use JUnit 5 or
+ newer (existing JUnit 4 tests run through the vintage engine). Code that parses input (decompressors, streams)
+ should also be covered by a [Jazzer](https://github.com/CodeIntelligenceTesting/jazzer) fuzz test, run with the
+ `fuzz` Maven profile. Each fuzz test needs its own execution in that profile.
+- **Code style.** There is no automated formatter. Follow the style of the surrounding code and
+ [`.editorconfig`](.editorconfig) (2-space indentation), and keep diffs minimal: no unrelated reformatting.
+- **Generated code.** Most of the Java and Unsafe implementations are generated from the MVEL templates in
+ `src/build/source_templates`. Edit the templates, not the generated files under `target/`.
+- **Implementations stay consistent.** A change to one implementation (JNI, Java safe, Java unsafe) usually needs a
+ matching change in the others.
+- **License.** Contributions are accepted under the [Apache License 2.0](LICENSE.txt), the project's license. New
+ source files need the Apache license header used by existing files.
+
+Keep pull requests focused on a single change and describe what changed and why.
diff --git a/README.md b/README.md
index b2497ecd..d234a929 100644
--- a/README.md
+++ b/README.md
@@ -1,174 +1,228 @@
# LZ4 Java
-*Community-maintained fork of https://github.com/lz4/lz4-java to patch [CVE‐2025‐12183](https://sites.google.com/sonatype.com/vulnerabilities/cve-2025-12183). Maven coordinates:*
+[](https://central.sonatype.com/artifact/at.yawk.lz4/lz4-java)
+[](https://codecov.io/gh/yawkat/lz4-java)
+[](https://www.bestpractices.dev/projects/14930)
+[](https://scorecard.dev/viewer/?uri=github.com/yawkat/lz4-java)
+
+Fast [LZ4](https://github.com/lz4/lz4) compression and [xxHash](https://github.com/Cyan4973/xxHash) hashing for Java,
+with JNI bindings to the reference C implementation and a pure Java fallback.
+
+This is the community-maintained continuation of [lz4/lz4-java](https://github.com/lz4/lz4-java), which is no longer
+maintained. The fork was originally created to fix
+[CVE‐2025‐12183](https://sites.google.com/sonatype.com/vulnerabilities/cve-2025-12183) and has since received further
+security fixes, an updated lz4 library, new native platforms and a modernized build. It is a drop-in replacement: the
+Java package names (`net.jpountz.*`) and API are unchanged, only the Maven coordinates are different.
+
+## Installation
+
+Replace `VERSION` with the latest version shown on
+[Maven Central](https://central.sonatype.com/artifact/at.yawk.lz4/lz4-java) or the
+[releases page](https://github.com/yawkat/lz4-java/releases).
+
+Maven:
```xml
at.yawk.lz4
lz4-java
+ VERSION
```
-LZ4 compression for Java, based on Yann Collet's work available at
-https://github.com/lz4/lz4.
-
-This library provides access to two compression methods that both generate a
-valid LZ4 stream:
- - fast scan (LZ4):
- - low memory footprint (~ 16 KB),
- - very fast (fast scan with skipping heuristics in case the input looks
- incompressible),
- - reasonable compression ratio (depending on the redundancy of the input).
- - high compression (LZ4 HC):
- - medium memory footprint (~ 256 KB),
- - rather slow (~ 10 times slower than LZ4),
- - good compression ratio (depending on the size and the redundancy of the
- input).
- - interoperable (LZ4 Frame):
- - overlay on top of lz4, that makes it system and library independent
- - this is the version to employ in order to share lz4 compressed data
- with other systems using different lz4 frameworks.
-
-The streams produced by LZ4 and LZ4 HC use the same compression format,
-are very fast to decompress and can be decompressed by the same lz4-java decompressor instance.
-
-The streams produced by LZ4 Frame can be read by other independent implementations of LZ4 Frame in any other language, including the `lz4` CLI tool.
-
-## Implementations
-
-For LZ4 compressors, LZ4 HC compressors and decompressors, 3 implementations are
-available:
- - JNI bindings to the original C implementation by Yann Collet,
- - a pure Java port of the compression and decompression algorithms,
- - a Java port that uses the `sun.misc.Unsafe` API in order to achieve compression
- and decompression speeds close to the C implementation.
-
-Have a look at [LZ4Factory](https://lz4-java.yawk.at/current/javadoc/net/jpountz/lz4/LZ4Factory) for more information.
+Gradle:
-## Compatibility notes
+```kotlin
+implementation("at.yawk.lz4:lz4-java:VERSION")
+```
+
+If another library pulls in `org.lz4:lz4-java`, exclude it (or use Gradle's dependency substitution) and add this
+artifact instead. Both contain the same classes, so make sure only one of them ends up on the classpath.
+
+Requirements:
+
+- Java 7 or newer. The JAR has the `Automatic-Module-Name` `org.lz4.java` and OSGi metadata.
+- The JAR bundles native libraries for Linux (amd64, i386, aarch64, ppc64le, s390x, riscv64), macOS (x86_64,
+ aarch64) and Windows (amd64). On other platforms, the pure Java implementation is used automatically.
+- On Java 24 and newer, the JVM prints a warning when the native library is loaded. Pass
+ `--enable-native-access=ALL-UNNAMED` (or the name of your module) to silence it.
+
+> **Note:** Before 1.8.1, there was also an `lz4-pure-java` artifact without the native libraries. It has been
+> discontinued and its latest version is still vulnerable to CVE‐2025‐12183. Please open an issue if you have a use
+> case that requires it.
+
+## Choosing an API
+
+| You want to… | Use |
+|-----------------------------------------------------------------|------------------------------------------------------------------------|
+| Exchange data with other tools or languages (e.g. the `lz4` CLI) | `LZ4FrameOutputStream` / `LZ4FrameInputStream` (LZ4 frame format) |
+| Compress a single `byte[]` or `ByteBuffer` and get it back | `LZ4CompressorWithLength` / `LZ4DecompressorWithLength` |
+| Full control over raw LZ4 blocks | `LZ4Compressor` / `LZ4SafeDecompressor` |
+| Hash data | `XXHash32`, `XXHash64` and their streaming variants |
+
+`LZ4BlockOutputStream` / `LZ4BlockInputStream` use a format specific to lz4-java. Prefer the frame streams for new
+code, unless you need compatibility with data that was already written in the block stream format.
+
+### Compressors
+
+- **Fast compressor** (`fastCompressor()`): low memory footprint (~16 KB), very fast, reasonable compression ratio.
+- **High compressor** (`highCompressor()`): LZ4 HC, higher memory footprint (~256 KB) and roughly 10× slower than the
+ fast compressor, but a better compression ratio. The compression level can be tuned; higher levels cost
+ significantly more CPU.
+
+Both produce the same block format, and their output can be read by the same decompressor.
+
+### Decompressors
+
+**Use `safeDecompressor()`.** It takes the compressed length as input, is safe for untrusted data, and is the fastest
+decompressor with the native implementation.
+
+`fastDecompressor()` takes the *decompressed* length instead. It is kept for compatibility, but for security reasons it
+now always uses the (slower) safe Java implementation, regardless of which factory it came from. The
+`nativeInsecureInstance()` and `unsafeInsecureInstance()` factories return the old, faster fast decompressors, but
+**must never be used with untrusted input**.
+
+### Implementations
+
+`LZ4Factory` and `XXHashFactory` provide the following implementations, which all produce compatible output:
- - Compressors and decompressors are interchangeable: it is perfectly correct
- to compress with the JNI bindings and to decompress with a Java port, or the
- other way around.
+- `nativeInstance()`: JNI bindings to the reference C implementation.
+- `safeInstance()`: A pure Java port.
+- `unsafeInstance()`: A Java port using `sun.misc.Unsafe`. For LZ4, this is currently an alias for the safe instance
+ out of caution.
- - Compressors might not generate the same compressed streams on all platforms,
- especially if CPU endianness differs, but the compressed streams can be
- safely decompressed by any decompressor implementation on any platform.
+In most cases, just use `fastestInstance()`, which picks the native implementation if it can be loaded and falls back
+to Java otherwise. Obtaining a factory is expensive, so store it in a static field and reuse it. See the
+[LZ4Factory javadoc](https://lz4-java.yawk.at/current/javadoc/net/jpountz/lz4/LZ4Factory) for caveats of the native
+implementation (temporary files, class loaders).
## Examples
-```java
-LZ4Factory factory = LZ4Factory.fastestInstance();
+### LZ4 frame streams
-byte[] data = "12345345234572".getBytes("UTF-8");
-final int decompressedLength = data.length;
+Files written this way can be read by the `lz4` command line tool and any other LZ4 frame implementation, and vice
+versa.
-// compress data
-LZ4Compressor compressor = factory.fastCompressor();
-int maxCompressedLength = compressor.maxCompressedLength(decompressedLength);
-byte[] compressed = new byte[maxCompressedLength];
-int compressedLength = compressor.compress(data, 0, decompressedLength, compressed, 0, maxCompressedLength);
-
-// decompress data
-// - method 1: when the decompressed length is known
-LZ4FastDecompressor decompressor = factory.fastDecompressor();
-byte[] restored = new byte[decompressedLength];
-int compressedLength2 = decompressor.decompress(compressed, 0, restored, 0, decompressedLength);
-// compressedLength == compressedLength2
-
-// - method 2: when the compressed length is known (a little slower)
-// the destination buffer needs to be over-sized
-LZ4SafeDecompressor decompressor2 = factory.safeDecompressor();
-int decompressedLength2 = decompressor2.decompress(compressed, 0, compressedLength, restored, 0);
-// decompressedLength == decompressedLength2
+```java
+try (OutputStream out = new LZ4FrameOutputStream(new FileOutputStream("data.lz4"))) {
+ out.write(data);
+}
+
+try (InputStream in = new LZ4FrameInputStream(new FileInputStream("data.lz4"))) {
+ byte[] restored = in.readAllBytes();
+}
```
+### Compressing a byte array
+
+`LZ4CompressorWithLength` prepends the original length to the compressed data, so you don't have to track it yourself.
+
```java
-byte[] data = "12345345234572".getBytes("UTF-8");
-final int decompressedLength = data.length;
+LZ4Factory factory = LZ4Factory.fastestInstance();
-LZ4FrameOutputStream outStream = new LZ4FrameOutputStream(new FileOutputStream(new File("test.lz4")));
-outStream.write(data);
-outStream.close();
+LZ4CompressorWithLength compressor = new LZ4CompressorWithLength(factory.fastCompressor());
+byte[] compressed = compressor.compress(data);
-byte[] restored = new byte[decompressedLength];
-LZ4FrameInputStream inStream = new LZ4FrameInputStream(new FileInputStream(new File("test.lz4")));
-inStream.read(restored);
-inStream.close();
+LZ4DecompressorWithLength decompressor = new LZ4DecompressorWithLength(factory.safeDecompressor());
+byte[] restored = decompressor.decompress(compressed);
```
-# xxhash Java
-
-xxhash hashing for Java, based on Yann Collet's work available at https://github.com/Cyan4973/xxHash (old version
-http://code.google.com/p/xxhash/). xxhash is a non-cryptographic, extremly fast
-and high-quality ([SMHasher](http://code.google.com/p/smhasher/wiki/SMHasher)
-score of 10) hash function.
+To protect against decompression bombs, `LZ4DecompressorWithLength` refuses to allocate output buffers larger than
+64 MiB by default. Pass a different maximum to the constructor, or set the
+`net.jpountz.lz4.LZ4DecompressorWithLength.maxDecompressedLength` system property. For large data, use the frame
+streams instead.
-## Implementations
+### Raw LZ4 blocks
-Similarly to LZ4, 3 implementations are available: JNI bindings, pure Java port
-and pure Java port that uses `sun.misc.Unsafe`.
+With the low-level API, you are responsible for storing the original length alongside the compressed data.
-Have a look at [XXHashFactory](https://lz4-java.yawk.at/current/javadoc/net/jpountz/xxhash/XXHashFactory) for more information.
+```java
+LZ4Factory factory = LZ4Factory.fastestInstance();
-## Compatibility notes
+// compress
+LZ4Compressor compressor = factory.fastCompressor();
+byte[] compressed = new byte[compressor.maxCompressedLength(data.length)];
+int compressedLength = compressor.compress(data, 0, data.length, compressed, 0, compressed.length);
- - All implementation return the same hash for the same input bytes:
- - on any JVM,
- - on any platform (even if the endianness or integer size differs).
+// decompress
+LZ4SafeDecompressor decompressor = factory.safeDecompressor();
+byte[] restored = new byte[data.length];
+int decompressedLength = decompressor.decompress(compressed, 0, compressedLength, restored, 0);
+```
-## Example
+### xxHash
```java
XXHashFactory factory = XXHashFactory.fastestInstance();
-
-byte[] data = "12345345234572".getBytes("UTF-8");
-ByteArrayInputStream in = new ByteArrayInputStream(data);
-
-int seed = 0x9747b28c; // used to initialize the hash value, use whatever
- // value you want, but always the same
-StreamingXXHash32 hash32 = factory.newStreamingHash32(seed);
-byte[] buf = new byte[8]; // for real-world usage, use a larger buffer, like 8192 bytes
-for (;;) {
- int read = in.read(buf);
- if (read == -1) {
- break;
- }
- hash32.update(buf, 0, read);
+long seed = 0; // any value, but use the same one for all hashes you want to compare
+
+// one-shot
+long hash = factory.hash64().hash(data, 0, data.length, seed);
+
+// streaming
+try (StreamingXXHash64 streamingHash = factory.newStreamingHash64(seed)) {
+ byte[] buf = new byte[8192];
+ int read;
+ while ((read = in.read(buf)) != -1) {
+ streamingHash.update(buf, 0, read);
+ }
+ long streamedHash = streamingHash.getValue();
}
-int hash = hash32.getValue();
```
-# Download
+xxHash is a very fast, non-cryptographic hash function. All implementations return the same hash for the same input,
+on every JVM and platform.
-You can download released artifacts from [Maven Central](https://central.sonatype.com/artifact/at.yawk.lz4/lz4-java).
+## Compatibility notes
+
+- Implementations are interchangeable: data compressed with the native implementation can be decompressed with the
+ Java one, and vice versa.
+- Compressed output is not guaranteed to be byte-for-byte identical across implementations, versions or platforms, but
+ it can always be decompressed by any of them.
+
+## Security
-Prior to 1.8.1, there was also a lz4-pure-java artifact that excluded the JNI binary. This artifact has been discontinued because it is difficult to build and doesn't make a lot of sense from a design standpoint. The latest version of this artifact is still vulnerable to CVE‐2025‐12183. Please open an issue if you have a use case that requires this version.
+Security fixes are published as [GitHub security advisories](https://github.com/yawkat/lz4-java/security/advisories)
+and noted in the [release notes](https://github.com/yawkat/lz4-java/releases). Please report vulnerabilities privately
+through GitHub's [vulnerability reporting](https://github.com/yawkat/lz4-java/security/advisories/new).
-# Documentation
+## Feedback and contributing
- - [lz4](https://lz4-java.yawk.at/current/javadoc/net/jpountz/lz4/package-summary.html)
- - [xxhash](https://lz4-java.yawk.at/current/javadoc/net/jpountz/xxhash/package-summary.html)
- - [changelog (versions <= 1.8.0)](https://github.com/yawkat/lz4-java/blob/main/CHANGES.md)
- - [changelog / GitHub releases (versions >= 1.8.1)](https://github.com/yawkat/lz4-java/releases)
+- Report bugs and request features through [GitHub issues](https://github.com/yawkat/lz4-java/issues).
+- Ask questions in [GitHub Discussions](https://github.com/yawkat/lz4-java/discussions).
+- Pull requests are welcome. See
+ [CONTRIBUTING.md](https://github.com/yawkat/lz4-java/blob/main/CONTRIBUTING.md) for the requirements for
+ contributions.
-# Performance
+## Documentation
-Both lz4 and xxhash focus on speed. Although compression, decompression and
-hashing performance can depend a lot on the input (there are lies, damn lies
-and benchmarks), here are some benchmarks that try to give a sense of the
-speed at which they compress/decompress/hash bytes.
+- Javadoc: [lz4](https://lz4-java.yawk.at/current/javadoc/net/jpountz/lz4/package-summary.html),
+ [xxhash](https://lz4-java.yawk.at/current/javadoc/net/jpountz/xxhash/package-summary.html)
+- [Release notes](https://github.com/yawkat/lz4-java/releases) (1.8.1 and later)
+- [Changelog](https://github.com/yawkat/lz4-java/blob/main/CHANGES.md) (1.8.0 and earlier)
+- Benchmarks of the original project (1.8.0):
+ [compression](https://lz4.github.io/lz4-java/1.8.0/lz4-compression-benchmark/),
+ [decompression](https://lz4.github.io/lz4-java/1.8.0/lz4-decompression-benchmark/),
+ [xxhash](https://lz4.github.io/lz4-java/1.3.0/xxhash-benchmark/)
- - [lz4 compression](https://lz4.github.io/lz4-java/1.8.0/lz4-compression-benchmark/)
- - [lz4 decompression](https://lz4.github.io/lz4-java/1.8.0/lz4-decompression-benchmark/)
- - [xxhash hashing](https://lz4.github.io/lz4-java/1.3.0/xxhash-benchmark/)
+## Building
-# Build
+You need:
-Make sure you have a Java 7 installation in your [Maven toolchains.xml](https://maven.apache.org/guides/mini/guide-using-toolchains.html). You may also need a C compiler.
+- A JDK 25 to run Maven.
+- A JDK 7 registered in your [Maven toolchains.xml](https://maven.apache.org/guides/mini/guide-using-toolchains.html),
+ used to compile the library. Alternatively, pass `-Dmaven.compiler.source=8 -Dmaven.compiler.target=8` to build
+ with the default JDK.
+- A C compiler, to build the native library for your platform.
+
+```bash
+git submodule update --init
+./mvnw verify
+```
-First run `git submodule init` and then `git submodule update` to initialize the lz4 submodule in `src/lz4`.
+The released JAR contains native libraries for all supported platforms. These are built in GitHub Actions, see
+[`build-all-and-publish.yml`](https://github.com/yawkat/lz4-java/blob/main/.github/workflows/build-all-and-publish.yml).
-Then, run `./mvnw verify`.
+## License
-Building a full artifact with native libraries for all supported platforms is more complex. We do this through GitHub Actions.
+[Apache License 2.0](https://github.com/yawkat/lz4-java/blob/main/LICENSE.txt)
diff --git a/pom.xml b/pom.xml
index de9783c2..bac1faaf 100644
--- a/pom.xml
+++ b/pom.xml
@@ -341,6 +341,34 @@
+
+
+ coverage
+
+
+
+ org.jacoco
+ jacoco-maven-plugin
+ 0.8.15
+
+
+ prepare-agent
+
+ prepare-agent
+
+
+
+ report
+ verify
+
+ report
+
+
+
+
+
+
+
deploy