You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "qiuyanjun888 (via GitHub)" <gi...@apache.org> on 2026/07/05 05:01:11 UTC

[PR] [FLINK-39429][cdc-base] Avoid shaded Guava in incremental source fetchers [flink-cdc]

qiuyanjun888 opened a new pull request, #4462:
URL: https://github.com/apache/flink-cdc/pull/4462

   ## Summary
   
   - Remove production references to Flink's shaded Guava 31 classes from `flink-cdc-base` incremental source fetcher paths.
   - Replace `ThreadFactoryBuilder` usage with a small JDK-only daemon thread factory.
   - Replace `Lists.partition` usage with a local JDK-only partition helper.
   - Add a regression test that blocks shaded Guava 31 classes and scans compiled production classes for shaded-Guava references.
   
   ## Root Cause
   
   FLINK-39429 reports `NoClassDefFoundError` for `org/apache/flink/shaded/guava31/...` when running CDC with Flink 2.2. The `flink-cdc-base` production classes referenced Flink's shaded Guava 31 package directly, but that package is not available in the reported runtime environment.
   
   ## Fix
   
   - Use JDK-only `ThreadFactory` implementations in:
     - `IncrementalSourceStreamFetcher`
     - `IncrementalSourceScanFetcher`
     - `SnapshotSplitAssigner`
   - Use a JDK-only `partition(...)` helper in `IncrementalSourceEnumerator`.
   - Add `IncrementalSourceFetcherDependencyTest` to prevent shaded Guava 31 references from returning in `flink-cdc-base` production classes.
   
   ## Validation
   
   - `mvn -pl flink-cdc-connect/flink-cdc-source-connectors/flink-cdc-base -am -Pflink2 -Dtest=IncrementalSourceFetcherDependencyTest -DfailIfNoTests=false -Dsurefire.failIfNoSpecifiedTests=false -DskipITs -Dcheckstyle.skip -Drat.skip=true -Dspotless.check.skip=true -Dmdep.skip=true test`
   - `mvn -pl flink-cdc-connect/flink-cdc-source-connectors/flink-cdc-base -am -Pflink2 -Dtest='!SplitKeyUtilsTest,!FinishedSnapshotSplitInfoTest' -DfailIfNoTests=false -Dsurefire.failIfNoSpecifiedTests=false -DskipITs -Dcheckstyle.skip -Drat.skip=true -Dspotless.check.skip=true -Dmdep.skip=true test`
   - `mvn -pl flink-cdc-connect/flink-cdc-source-connectors/flink-cdc-base -am -Pflink2 -DskipTests -Drat.skip=true -Dmdep.skip=true verify`
   
   Note: the broader module test command excludes two known Java 17 Mockito blocker tests (`SplitKeyUtilsTest`, `FinishedSnapshotSplitInfoTest`) that are unrelated to this change.
   
   ## Jira
   
   https://issues.apache.org/jira/browse/FLINK-39429
   
   ## Reviewers
   
   cc @leonardBang @haruki-830 — you recently reviewed/authored changes touching this area.
   
   ## AI Assistance Disclosure
   
   This PR was prepared with AI-assisted tooling and manually validated with the commands above.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] [FLINK-39429][cdc-base] Avoid shaded Guava in incremental source fetchers [flink-cdc]

Posted by "qiuyanjun888 (via GitHub)" <gi...@apache.org>.
qiuyanjun888 commented on PR #4462:
URL: https://github.com/apache/flink-cdc/pull/4462#issuecomment-4953291450

   @yuxiqian can you please help review this PR? Thanks.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] [FLINK-39429][cdc-base] Avoid shaded Guava in incremental source fetchers [flink-cdc]

Posted by "yuxiqian (via GitHub)" <gi...@apache.org>.
yuxiqian commented on PR #4462:
URL: https://github.com/apache/flink-cdc/pull/4462#issuecomment-4953617316

   Hello Yanjun, could you please elaborate on the use case when this exception occurs? Are you using SQL or the DataStream connector?
   
   I remember that @lvyanquan has prepared a [package manual](https://nightlies.apache.org/flink/flink-cdc-docs-release-3.6/docs/connectors/flink-sources/datastream-api-package-guidance/#example-for-pomxml-with-flink-22x) for DataStream connector with Flink 2.x (by some shading + relocating hack). Is that similar to your problem?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] [FLINK-39429][cdc-base] Avoid shaded Guava in incremental source fetchers [flink-cdc]

Posted by "qiuyanjun888 (via GitHub)" <gi...@apache.org>.
qiuyanjun888 closed pull request #4462: [FLINK-39429][cdc-base] Avoid shaded Guava in incremental source fetchers
URL: https://github.com/apache/flink-cdc/pull/4462


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org