You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@seatunnel.apache.org by "hswlxc (via GitHub)" <gi...@apache.org> on 2024/08/12 02:28:19 UTC

[I] [Bug] [Postgres-CDC] Postgres-CDC Unsupported type: ARRAY [seatunnel]

hswlxc opened a new issue, #7370:
URL: https://github.com/apache/seatunnel/issues/7370

   ### Search before asking
   
   - [X] I had searched in the [issues](https://github.com/apache/seatunnel/issues?q=is%3Aissue+label%3A%22bug%22) and found no similar issues.
   
   
   ### What happened
   
   ![image](https://github.com/user-attachments/assets/d5548382-f20e-40d8-aa36-43cb72dbc66d)
   When using postgress-cdc, it is indicated that the array type is not supported, but the usage documentation indicates that it is supported
   <img width="834" alt="image" src="https://github.com/user-attachments/assets/64e6954b-f2de-4faf-a1a7-236780d5a70f">
   
   
   ### SeaTunnel Version
   
   2.3.5
   
   ### SeaTunnel Config
   
   ```conf
   env {
     execution.parallelism = 1
     job.mode = "STREAMING"
     checkpoint.interval = 5000
     read_limit.bytes_per_second=7000000
     read_limit.rows_per_second=400
   }
   
   source {
    Postgres-CDC {
       result_table_name=Table14286033870336
       username="brt"
       password="jiupinbrt6789jr"
       hostname = "172.16.15.117"
       port = 6543
       database-names = ["db_brt_test"]
       schema-names = ["brt_case"]
       table-names = ["db_brt_test.brt_case.tb_base"]
       base-url="jdbc:postgresql://172.16.15.117:6543/db_brt_test"
       exactly_once = true
       table-names-config = [
         {
           table = "db_brt_test.brt_case.tb_base"
           primaryKeys = ["bse_id"]
         }
        ]
    }
   }
   transform {
   }
   sink {
    Jdbc {
       source_table_name = "Table14286033870336"
       database=db_brt_test
       generate_sink_sql = true
       table="db_brt_test.brt_case.tb_base"
       chema = "brt_case"
       #tablePrefix = "sink_"
       primary_keys = ["bse_id"]
       password="6789@jiupin"
       url="jdbc:postgresql://172.16.16.230:5432/db_brt_test"
       user=postgres
       #username=postgres
       driver="org.postgresql.Driver"
    }
   }
   ```
   
   
   ### Running Command
   
   ```shell
   bin/seatunnel.sh -c config/v2.streaming.conf -e local
   ```
   
   
   ### Error Exception
   
   ```log
   Exception in thread "main" org.apache.seatunnel.core.starter.exception.CommandExecuteException: SeaTunnel job executed failed
           at org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand.execute(ClientExecuteCommand.java:202)
           at org.apache.seatunnel.core.starter.SeaTunnel.run(SeaTunnel.java:40)
           at org.apache.seatunnel.core.starter.seatunnel.SeaTunnelClient.main(SeaTunnelClient.java:34)
   Caused by: org.apache.seatunnel.api.table.factory.FactoryException: ErrorCode:[API-06], ErrorDescription:[Factory initialize failed] - Unable to create a source for identifier 'Postgres-CDC'.
           at org.apache.seatunnel.api.table.factory.FactoryUtil.createAndPrepareSource(FactoryUtil.java:100)
           at org.apache.seatunnel.engine.core.parse.MultipleTableJobConfigParser.parseSource(MultipleTableJobConfigParser.java:332)
           at org.apache.seatunnel.engine.core.parse.MultipleTableJobConfigParser.parse(MultipleTableJobConfigParser.java:188)
           at org.apache.seatunnel.engine.client.job.ClientJobExecutionEnvironment.getLogicalDag(ClientJobExecutionEnvironment.java:88)
           at org.apache.seatunnel.engine.client.job.ClientJobExecutionEnvironment.execute(ClientJobExecutionEnvironment.java:156)
           at org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand.execute(ClientExecuteCommand.java:149)
           ... 2 more
   Caused by: java.lang.UnsupportedOperationException: Unsupported type: ARRAY<STRING>
           at org.apache.seatunnel.connectors.cdc.debezium.row.SeaTunnelRowDebeziumDeserializationConverters.createNotNullConverter(SeaTunnelRowDebeziumDeserializationConverters.java:178)
           at org.apache.seatunnel.connectors.cdc.debezium.row.SeaTunnelRowDebeziumDeserializationConverters.createConverter(SeaTunnelRowDebeziumDeserializationConverters.java:113)
           at org.apache.seatunnel.connectors.cdc.debezium.row.SeaTunnelRowDebeziumDeserializationConverters.lambda$new$0(SeaTunnelRowDebeziumDeserializationConverters.java:71)
           at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:195)
           at java.base/java.util.Spliterators$ArraySpliterator.forEachRemaining(Spliterators.java:948)
           at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484)
           at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474)
           at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:550)
           at java.base/java.util.stream.AbstractPipeline.evaluateToArrayNode(AbstractPipeline.java:260)
           at java.base/java.util.stream.ReferencePipeline.toArray(ReferencePipeline.java:517)
           at org.apache.seatunnel.connectors.cdc.debezium.row.SeaTunnelRowDebeziumDeserializationConverters.<init>(SeaTunnelRowDebeziumDeserializationConverters.java:73)
           at org.apache.seatunnel.connectors.cdc.debezium.row.SeaTunnelRowDebeziumDeserializeSchema.createTableRowConverters(SeaTunnelRowDebeziumDeserializeSchema.java:295)
           at org.apache.seatunnel.connectors.cdc.debezium.row.SeaTunnelRowDebeziumDeserializeSchema.<init>(SeaTunnelRowDebeziumDeserializeSchema.java:87)
           at org.apache.seatunnel.connectors.cdc.debezium.row.SeaTunnelRowDebeziumDeserializeSchema$Builder.build(SeaTunnelRowDebeziumDeserializeSchema.java:331)
           at org.apache.seatunnel.connectors.seatunnel.cdc.postgres.source.PostgresIncrementalSource.createDebeziumDeserializationSchema(PostgresIncrementalSource.java:107)
           at org.apache.seatunnel.connectors.cdc.base.source.IncrementalSource.<init>(IncrementalSource.java:113)
           at org.apache.seatunnel.connectors.seatunnel.cdc.postgres.source.PostgresIncrementalSource.<init>(PostgresIncrementalSource.java:64)
           at org.apache.seatunnel.connectors.seatunnel.cdc.postgres.source.PostgresIncrementalSourceFactory.lambda$createSource$1(PostgresIncrementalSourceFactory.java:103)
           at org.apache.seatunnel.api.table.factory.FactoryUtil.createAndPrepareSource(FactoryUtil.java:112)
           at org.apache.seatunnel.api.table.factory.FactoryUtil.createAndPrepareSource(FactoryUtil.java:73)
           ... 7 more
   ```
   
   
   ### Zeta or Flink or Spark Version
   
   _No response_
   
   ### Java or Scala Version
   
   java :11.0.23
   
   ### Screenshots
   
   _No response_
   
   ### Are you willing to submit PR?
   
   - [ ] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [Bug] [Postgres-CDC] Postgres-CDC Unsupported type: ARRAY [seatunnel]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #7370:
URL: https://github.com/apache/seatunnel/issues/7370#issuecomment-3064432032

   This issue has been closed because it has not received response for too long time. You could reopen it if you encountered similar problems in the future.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [Bug] [Postgres-CDC] Postgres-CDC Unsupported type: ARRAY [seatunnel]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #7370:
URL: https://github.com/apache/seatunnel/issues/7370#issuecomment-2345022285

   This issue has been automatically marked as stale because it has not had recent activity for 30 days. It will be closed in next 7 days if no further activity occurs.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [Bug] [Postgres-CDC] Postgres-CDC Unsupported type: ARRAY [seatunnel]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed issue #7370: [Bug] [Postgres-CDC] Postgres-CDC  Unsupported type: ARRAY<STRING>
URL: https://github.com/apache/seatunnel/issues/7370


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org