You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "MartijnVisser (via GitHub)" <gi...@apache.org> on 2026/08/25 14:45:33 UTC
[PR] [FLINK-40459][Build/Shaded] Bump Maven from 3.8.6 to 3.9.16 [flink-shaded]
MartijnVisser opened a new pull request, #170:
URL: https://github.com/apache/flink-shaded/pull/170
## What is the purpose of the change
Bumps the build and enforced Maven version for `flink-shaded` from 3.8.6 to 3.9.16, matching apache/flink#29009 for FLINK-40459.
## Brief change log
- `.github/workflows/ci.yml`: the `stCarolas/setup-maven` pin goes from 3.8.6 to 3.9.16
- `pom.xml`: the `release` profile's `requireMavenVersion` pin goes from `[3.8.6]` to `[3.9.16]` (exact pin retained)
- `.mvn/wrapper/maven-wrapper.properties`, `mvnw`, `mvnw.cmd`: add a Maven wrapper pinning Apache Maven 3.9.16 (maven-wrapper 3.3.4), matching Flink's
- `.gitignore`: ignore the downloaded `maven-wrapper.jar`
- `README.md`: document the required Maven version and the wrapper
- `.github/workflows/ci.yml`: normalize Maven 3.9 mojo banners before the license check (see below)
The wrapper is added **script-only** — `maven-wrapper.jar` is deliberately not checked in. `mvnw` downloads it on first use and verifies it against `wrapperSha256Sum`, so no binary artifact enters the ASF source release. Verified: a simulated `create_source_release.sh` tarball contains only the three text files and zero binaries, and `apache-rat` reports 0 unapproved / 0 binaries. A previous attempt to add the wrapper here (0a4baad) was reverted 18 minutes later (8f9e83c); that one checked in the jar and shipped a maven-wrapper 3.1.1 script against 3.3.2 properties, so its checksums were never actually verified.
### The license check needs a fix, or it silently stops checking
Maven 3.9 renders mojo banners using the plugin's **goal prefix** (`deploy:2.8.2:deploy`) instead of its **artifactId** (`maven-deploy-plugin:2.8.2:deploy`). This was confirmed by decompiling `ExecutionEventLogger.append(MessageBuilder, MojoExecution)` from both distributions: 3.8.6 calls `MojoExecution.getArtifactId()`, 3.9.16 calls `MojoDescriptor.getPluginDescriptor().getGoalPrefix()`.
`flink-ci-tools`' `DeployParser` and `DependencyParser` match on the artifactId spelling, so under 3.9.16 they match nothing. `NoticeFileChecker` then sees zero deployed modules, empties `modulesWithBundledDependencies`, and every declared NOTICE line degrades to "not bundled, but listed" at `TOLERATED` severity — so `severeIssueCount` stays 0 and **the build still exits 0**. The gate does not fail; it stops checking. (`ShadeParser` survives, because its regex starts with `.*`. `JarFileChecker` scans the deploy directory rather than the log and is unaffected.)
Measured on this reactor:
| Build | exit code | `Extracted N modules that were deployed` |
|---|---|---|
| 3.8.6 | 0 | **17** |
| 3.9.16, unremediated | 0 | **0** |
| 3.9.16 + this PR's normalization | 0 | **17** |
No released `flink-ci-tools` fixes this — I checked the 1.18.0, 1.20.4 and 2.2.1 jars from Maven Central and all three carry byte-identical broken patterns; the fix exists only on the unmerged apache/flink#29009 branch. So CI rewrites the two affected banners back to the 3.8 spelling before invoking the checker. Only `deploy:` and `dependency:` are rewritten, anchored to `^[INFO] --- `; no other plugin in this build owns those goal prefixes, and the step is a byte-identical no-op on 3.8.6-format logs. It should be removed once `flink-ci-tools` is bumped to a release containing the FLINK-40459 parser fix.
## Verifying this change
This change is a trivial rework / code cleanup without any test coverage.
It was validated with a local red/green differential of the full reactor under 3.8.6 and 3.9.16, on JDK 11 (CI parity) and JDK 17, across the default, `-Pinclude-netty-tcnative-static`, `-Plicense-check`, `-Dshade-sources` and `-Prelease` profile combinations. Every combination compared **equivalent**: identical deployed file sets, identical jar entry lists, identical SHA-256 per uncompressed entry, identical aggregated `META-INF/NOTICE` and `META-INF/DEPENDENCIES`, byte-identical `dependency-reduced-pom.xml` and flattened poms, and an identical mojo execution sequence.
The only artifact difference anywhere is the manifest `Created-By: Apache Maven 3.8.6` → `3.9.16` line in the nine non-Ant modules, plus the matching `pom.properties` comment in `flink-shaded-force-shading`. The three Ant-repackaged jars (`flink-shaded-netty`, `flink-shaded-netty-tcnative-dynamic`, `flink-shaded-netty-tcnative-static`) are unchanged, as expected — Ant's `<jar>` writes their manifest, so they carry no Maven version at all.
Dependency resolution is unchanged. Two independent cold local repositories, populated online by each Maven, are identical in all 2189 files, and `dependency:tree` / `dependency:list` match exactly. This matters most for the ZooKeeper modules, whose `<artifactSet><include>*</include></artifactSet>` would put any resolution change straight into the shipped jar.
Also verified:
- `./mvnw clean verify` passes, and `./mvnw -v` downloads Apache Maven 3.9.16 with the wrapper's own SHA-256 verification
- negative check: with a system Maven 3.8.6, `mvn -N validate -Prelease` now fails with `Detected Maven Version: 3.8.6 is not in the allowed range [3.9.16,3.9.16]`, and with 3.9.16 it passes
- before this PR's pin change, `-Prelease` on 3.9.16 failed with `Detected Maven Version: 3.9.16 is not in the allowed range [3.8.6,3.8.6]` and no other error
- the old plugin set still loads and executes under 3.9.16 with no linkage errors (enforcer 1.4.1, apache-rat 0.11, antrun 1.7, dependency 3.1.1, gpg 1.4), including negative controls confirming enforcer still detects an injected dependency-convergence conflict and RAT still detects an unlicensed file. MNG-6965 (plexus-utils no longer auto-injected) is a non-event here: every plugin in the effective set declares plexus-utils itself
- a runtime probe over the produced artifacts on JDK 11 and 17 behaves identically for both Maven versions: relocated classes resolve, no un-relocated leaks, and the Jackson multi-release classes are still selected from `META-INF/versions/`
- downstream, `flink-rpc/flink-rpc-akka` built against each artifact set with the *same* Maven produces uber-jars identical in all 12323 entries
One benign difference shows up in `help:effective-pom` and is not caused by this change: Maven 3.9's super-POM drops `<releases><updatePolicy>never</updatePolicy>` from the built-in `central` **pluginRepository**. Nothing in this repository sets `updatePolicy`.
## Does this pull request potentially affect one of the following parts:
- Dependencies (does it add or upgrade a dependency): no
- The public API, i.e., is any changed class annotated with `@Public(Evolving)`: no
- The serializers: no
- The runtime per-record code paths (performance sensitive): no
- Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no
- The S3 file system connector: no
## Documentation
- Does this pull request introduce a new feature? no
- If yes, how is the feature documented? The README is updated with the required Maven version and the wrapper
---
##### Was generative AI tooling used to co-author this PR?
- [X] Yes (please specify the tool below)
Generated-by: Claude Code (Claude Opus 5)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-40459][Build/Shaded] Bump Maven from 3.8.6 to 3.9.16 [flink-shaded]
Posted by "MartijnVisser (via GitHub)" <gi...@apache.org>.
MartijnVisser merged PR #170:
URL: https://github.com/apache/flink-shaded/pull/170
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org