You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "Márton Balassi (Jira)" <ji...@apache.org> on 2026/06/04 07:52:00 UTC
[jira] [Closed] (FLINK-39608) KafkaDatasetFacet should extend DatasetConfigFacet to avoid classloader issues with OpenLineage
[ https://issues.apache.org/jira/browse/FLINK-39608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Márton Balassi closed FLINK-39608.
----------------------------------
Resolution: Fixed
Fixed via https://github.com/apache/flink-connector-kafka/commit/fa1021ee09c2594a3087be7983f864574539015c
> KafkaDatasetFacet should extend DatasetConfigFacet to avoid classloader issues with OpenLineage
> ------------------------------------------------------------------------------------------------
>
> Key: FLINK-39608
> URL: https://issues.apache.org/jira/browse/FLINK-39608
> Project: Flink
> Issue Type: Bug
> Components: Connectors / Kafka
> Reporter: Swapna Marru
> Assignee: Swapna Marru
> Priority: Minor
> Labels: pull-request-available
>
> The OpenLineage event emitter relies on the availability of the custom KafkaDatasetFacet class in the classpath (https://github.com/OpenLineage/OpenLineage/blob/main/integration/flink/flink2/src/main/java/io/openlineage/flink/util/KafkaDatasetFacetUtil.java#L25). This
> fails when the Kafka connector is loaded as part of a user jar while the OpenLineage library is loaded as part of the Flink distribution.
>
> In this scenario:
> - OpenLineage emitters are loaded in ApplicationClassLoader
> - Kafka connector libs are loaded in FlinkUserClassLoader This classloader boundary cannot load KafkaDatasetFacet and not emit kafka dataset information , even though the Kafka source is available in lineage datasets information.
>
> Proposed Solution:
> Make KafkaDatasetFacet extend org.apache.flink.streaming.api.lineage.DatasetConfigFacet (which is part of Flink core and available to both classloaders). The OpenLineage integration can then:
> 1. Identify the facet by name
> 2. Cast to DatasetConfigFacet (a Flink core interface) to extract configuration
> This eliminates the dependency on Kafka connector classes being in the same classloader as the lineage emitter libs.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)