You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "Mao Jiayi (Jira)" <ji...@apache.org> on 2026/07/09 08:47:00 UTC

[jira] [Updated] (FLINK-39760) Fix missing JTS core dependency of PostgreSQL YAML connector

     [ https://issues.apache.org/jira/browse/FLINK-39760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Mao Jiayi updated FLINK-39760:
------------------------------
    Description: 
When building the PostgreSQL YAML connector as a standalone module, the build may fail because the connector uses JTS classes without declaring the required JTS core dependency.

The issue happens in {*}{{PostgresEventDeserializer}}{*}. The current implementation imports and uses classes such as *{{org.locationtech.jts.geom.Coordinate}} and {{org.locationtech.jts.io.WKBReader}}* when handling geometry values. However, *{{flink-cdc-pipeline-connector-postgres}}* does not explicitly declare *{{org.locationtech.jts:jts-core}}* in its module dependencies.

This issue may not be exposed when the connector is built together with modules that happen to provide JTS transitively, but it can fail in isolated module builds or packaging workflows where the PostgreSQL YAML connector must resolve its own compile dependencies.

  was:
When using CDC pipelines with schema merging operations that involve *complex types (ARRAY, MAP, ROW)* being coerced to {*}STRING type{*}, the job may fail due to missing type conversion logic in {*}SchemaMergingUtils{*}.

The failure happens in {*}SchemaMergingUtils.coerceObject(){*}. When attempting to coerce complex types to STRING during schema merging, the utility only handles primitive types and binary data conversions. If the original field is a MapData, ArrayData, or RecordData, the coercion logic falls through to the default toString() conversion, which either produces incorrect string representations or fails entirely because these complex types don't have proper toString() implementations for string coercion.

This issue is usually not exposed in simple type scenarios because primitive types (INTEGER, VARCHAR, BOOLEAN, TIMESTAMP, etc.) have direct coercion paths to STRING through existing conversion rules. The schema merging logic works correctly for these basic type conversions, as they can be directly transformed without special handling.

However, when CDC pipelines involve schema evolution or field type changes where complex types need to be merged into STRING columns, the missing conversion logic becomes critical. For example, when merging a table with *ARRAY<INT>* column into a target table with *STRING* column at the same position, *SchemaMergingUtils* cannot properly serialize the array data structure into its string representation, causing the merge operation to fail or produce incorrect results.

This issue is not specific to any particular source connector. It can affect any CDC pipeline with schema merging scenarios involving complex-to-STRING type coercion, regardless of whether the source is MySQL, PostgreSQL, Kafka, or any other distributed pipeline source with schema evolution capabilities.

        Summary: Fix missing JTS core dependency of PostgreSQL YAML connector  (was: Fix unable to coerce complex types (ARRAY/MAP/ROW) to STRING)

> Fix missing JTS core dependency of PostgreSQL YAML connector
> ------------------------------------------------------------
>
>                 Key: FLINK-39760
>                 URL: https://issues.apache.org/jira/browse/FLINK-39760
>             Project: Flink
>          Issue Type: Bug
>          Components: Flink CDC
>            Reporter: Mao Jiayi
>            Priority: Major
>              Labels: pull-request-available
>
> When building the PostgreSQL YAML connector as a standalone module, the build may fail because the connector uses JTS classes without declaring the required JTS core dependency.
> The issue happens in {*}{{PostgresEventDeserializer}}{*}. The current implementation imports and uses classes such as *{{org.locationtech.jts.geom.Coordinate}} and {{org.locationtech.jts.io.WKBReader}}* when handling geometry values. However, *{{flink-cdc-pipeline-connector-postgres}}* does not explicitly declare *{{org.locationtech.jts:jts-core}}* in its module dependencies.
> This issue may not be exposed when the connector is built together with modules that happen to provide JTS transitively, but it can fail in isolated module builds or packaging workflows where the PostgreSQL YAML connector must resolve its own compile dependencies.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)