[PR] [FLINK-40473] Fix yarn-application mode failing to locate flink-cdc-dist jar with Flink version suffix [flink-cdc]

["llphxd (via GitHub)" <gi...@apache.org>]

llphxd opened a new pull request, #4514:
URL: https://github.com/apache/flink-cdc/pull/4514

   ## What is the purpose of this pull request?
   
   Fixes [FLINK-40473](https://issues.apache.org/jira/browse/FLINK-40473).
   
   Submitting a YAML pipeline with `-t yarn-application` using the official
   `flink-cdc-3.6.0-1.20` binary distribution always fails with:
   
   java.io.FileNotFoundException: Failed to fetch Flink CDC dist jar from path: /path/to/flink-cdc-3.6.0-1.20/lib
   
   The `lib` directory does contain the dist jar. `YarnApplicationDeploymentExecutor`
   locates it by matching file names against
   `^flink-cdc-dist-(\d+(\.\d+)*)(-SNAPSHOT)?\.jar$`, which only accepts `-SNAPSHOT`
   or `.jar` after the version number.
   
   Since 3.6.0 the binary distribution is released once per Flink minor version, and
   the release script sets `revision` to `<release-version>-<flink-version>`.  The `-1.20` segment never matches the
   pattern.
   ## Brief change log
   
   - Match the dist jar by its `flink-cdc-dist-` prefix and `.jar` suffix instead of a
     version pattern, so the lookup no longer depends on how the version is formatted.
   - Extract the lib directory scan into `getFlinkCDCDistJar(Path)` so it can be unit
     tested without mutating the `FLINK_CDC_HOME` environment variable.
   - Add `YarnApplicationDeploymentExecutorTest`.
   
   The set of names accepted by the old pattern is a strict subset of what the new
   predicate accepts, so no previously working setup is affected.
   
   ---
   
   ## Verifying this change
   
   This change added tests and can be verified as follows:
   
   - Added unit tests in
     `flink-cdc-composer/src/test/java/org/apache/flink/cdc/composer/flink/deployment/YarnApplicationDeploymentExecutorTest.java`:
     - `testFindDistJarWhateverVersionItCarries` covers `flink-cdc-dist-3.5.0.jar`,
       `flink-cdc-dist-3.7-SNAPSHOT.jar`, `flink-cdc-dist-3.6.0-1.20.jar` and
       `flink-cdc-dist-3.7.0-2.0.jar`. The last two fail without this fix. A connector
       jar and a directory named like a dist jar are placed in the same lib directory
       to verify neither is picked up.
     - `testDistJarNotFound` verifies that a lib directory holding only connector jars
       still raises `FileNotFoundException`, i.e. the prefix match does not over-match.
   
   The failure happens in the first statement of `deploy()`, before any interaction
   with YARN, so the unit test covers the full code path that was broken.
   
   ## Documentation
   
   - Does this pull request introduce a new feature? no
   - If yes, how is the feature documented? not applicable
   
   
   ---
   
   ##### Was generative AI tooling used to co-author this PR?
   
   <!--
   If generative AI tooling has been used in the process of authoring this PR, please
   change the checkbox below to `[X]` followed by the name of the tool, and uncomment the
   "Generated-by" line. See the ASF Generative Tooling Guidance for details:
   https://www.apache.org/legal/generative-tooling.html
   
   You are responsible for the quality and correctness of every change in this PR
   regardless of the tooling used. Low-effort AI-generated PRs will be closed.
   -->
   
   - [ ] Yes (please specify the tool below)
   
   <!--
   Generated-by: [Tool Name and Version]
   -->
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] [FLINK-40473] Fix yarn-application mode failing to locate flink-cdc-dist jar with Flink version suffix [flink-cdc]

["lvyanquan (via GitHub)" <gi...@apache.org>]

lvyanquan merged PR #4514:
URL: https://github.com/apache/flink-cdc/pull/4514


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org