[PR] [lake] Fail fast when a lake plugin is shadowed by a jar in lib [fluss]

["gezapeti (via GitHub)" <gi...@apache.org>]

gezapeti opened a new pull request, #4203:
URL: https://github.com/apache/fluss/pull/4203

   ### Purpose
   
   Linked issue: close #1314
   
   A `fluss-lake-<format>` jar placed in `<FLUSS_HOME>/lib` as well as `plugins/<format>/` makes the
   CoordinatorServer fail with `NoClassDefFoundError: org/apache/hadoop/hdfs/HdfsConfiguration` from
   `CatalogContext.<init>`.
   
   `org.apache.fluss.` is parent-first, so the `lib/` copy shadows the plugin and is loaded without the
   dependencies bundled next to it. This rejects that setup up front instead of failing later.
   
   Items (3) and (4) of the issue were already addressed upstream; this covers (1), (2), (5) and (6).
   
   ### Brief change log
   
   - `LakeStoragePluginSetUp` resolves the plugins directory and the classpath separately. Finding the
     same format in both now fails with a message naming both locations and the jar to remove.
   - Resolution is unchanged when a plugin comes from one location only, so the Tiering Service, which
     passes a null `PluginManager`, is unaffected.
   - Added `PluginDirectoryLayoutTest` covering the deployed `plugins/` layout.
   - Added a troubleshooting page and a `Filesystem JARs` section on the downloads page.
   
   Detecting duplicates means the classpath SPI is now always scanned, where before it was skipped once
   a plugin was found. A broken service entry there can therefore surface a `ServiceConfigurationError`.
   
   ### Tests
   
   - `LakeStorageTest` — the duplicate is rejected and classpath-only resolution still works. Verified
     the new case fails without the production change.
   - `PluginDirectoryLayoutTest` — multiple plugins loaded at once, isolation between them, main
     classpath shadowing, and resolution from the plugin jar when not shadowed.
   - `./mvnw verify -pl fluss-common` passes: 1798 unit and 186 integration tests. Spotless, Checkstyle
     and RAT pass.
   
   ### API and Format
   
   No public API or storage format change.
   
   ### Documentation
   
   - New `maintenance/troubleshooting-plugins.md`.
   - New `Filesystem JARs` section on the downloads page. Two pages already linked to
     `/downloads#filesystem-jars`, which did not exist. It also separates the bundled `fluss-fs-hdfs`
     from the thin `fluss-fs-hadoop`, which expects Hadoop via `HADOOP_CLASSPATH`.
   - The filesystem overview credited `fluss-fs-hadoop` for the bundled HDFS plugin, which has been
     `fluss-fs-hdfs` since #1522.
   
   ### Generative AI disclosure
   - [x] Yes (please specify the tool below)
   
   Generated-by: Claude Opus 5 (Claude Code) following the guidelines (https://github.com/apache/fluss/blob/main/AGENTS.md)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@fluss.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org