[jira] [Created] (FLINK-39429) NoClassDefFoundError for flink-shaded-guava31 in IncrementalSourceStreamFetcher (Flink 2.2 / CDC 3.6.0)

["João Boto (Jira)" <ji...@apache.org>]

João Boto created FLINK-39429:
---------------------------------

             Summary: NoClassDefFoundError for flink-shaded-guava31 in IncrementalSourceStreamFetcher (Flink 2.2 / CDC 3.6.0)
                 Key: FLINK-39429
                 URL: https://issues.apache.org/jira/browse/FLINK-39429
             Project: Flink
          Issue Type: Bug
          Components: Flink CDC
            Reporter: João Boto


When running a Flink CDC pipeline on Flink 2.2 using Flink CDC 3.6.0, the job crashes with a java.lang.NoClassDefFoundError: org/apache/flink/shaded/guava31/com/google/common/util/concurrent/ThreadFactoryBuilder.

This occurs specifically when scan.incremental.snapshot.enabled is set to true, as the IncrementalSourceStreamFetcher and IncrementalSourceScanFetcher are strictly looking for the guava31 shade, which seems to be missing or incompatible with the default Flink 2.2 classpath environment.

*Environment:*
Flink version: 2.2
Flink CDC version: 3.6.0
Database / Connector: PostgreSQL
Java version: 21
Deployment mode: Kubernetes

*Steps to reproduce*:
Configure a CDC source using Flink CDC 3.6.0.

Ensure incremental snapshots are enabled: this.debeziumProps.setProperty("scan.incremental.snapshot.enabled", "true");

Deploy the job to a Flink 2.2 cluster.

The job fails immediately during the fetcher initialization.

*Expected behavior:*
The Flink CDC connector should successfully initialize the IncrementalSourceStreamFetcher and perform lock-free incremental snapshots without missing shaded Guava dependencies out of the box. If a specific shaded Guava jar is required for Flink 2.2 compatibility, it should be bundled or explicitly documented.

*Actual behavior / Stack trace:*
{code:java}
java.lang.RuntimeException: One or more fetchers have encountered exception
    at org.apache.flink.connector.base.source.reader.fetcher.SplitFetcherManager.checkErrors(SplitFetcherManager.java:307)
    at org.apache.flink.connector.base.source.reader.SourceReaderBase.getNextFetch(SourceReaderBase.java:269)
    at org.apache.flink.connector.base.source.reader.SourceReaderBase.pollNext(SourceReaderBase.java:181)
    ...
    at java.base/java.lang.Thread.run(Unknown Source)
Caused by: java.lang.NoClassDefFoundError: org/apache/flink/shaded/guava31/com/google/common/util/concurrent/ThreadFactoryBuilder
    at org.apache.flink.cdc.connectors.base.source.reader.external.IncrementalSourceStreamFetcher.<init>(IncrementalSourceStreamFetcher.java:77)
    at org.apache.flink.cdc.connectors.base.source.reader.IncrementalSourceSplitReader.getStreamFetcher(IncrementalSourceSplitReader.java:272)
    at org.apache.flink.cdc.connectors.base.source.reader.IncrementalSourceSplitReader.submitStreamSplit(IncrementalSourceSplitReader.java:254)
    at org.apache.flink.cdc.connectors.base.source.reader.IncrementalSourceSplitReader.pollSplitRecords(IncrementalSourceSplitReader.java:153)
    at org.apache.flink.cdc.connectors.base.source.reader.IncrementalSourceSplitReader.fetch(IncrementalSourceSplitReader.java:98)
    at org.apache.flink.connector.base.source.reader.fetcher.FetchTask.run(FetchTask.java:58)
    at org.apache.flink.connector.base.source.reader.fetcher.SplitFetcher.runOnce(SplitFetcher.java:193)
    ...
Caused by: java.lang.ClassNotFoundException: org.apache.flink.shaded.guava31.com.google.common.util.concurrent.ThreadFactoryBuilder
    ... 13 more
{code}





--
This message was sent by Atlassian Jira
(v8.20.10#820010)