You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@paimon.apache.org by "DavidZ1 (via GitHub)" <gi...@apache.org> on 2025/09/16 02:25:57 UTC
[I] [Bug] MySQL add column sync Paimon failed [paimon]
DavidZ1 opened a new issue, #6259:
URL: https://github.com/apache/paimon/issues/6259
### Search before asking
- [x] I searched in the [issues](https://github.com/apache/paimon/issues) and found nothing similar.
### Paimon version
1.0.1
### Compute Engine
Flink 1.18.1
### Minimal reproduce step
## 1. started sync job
```
/data/module/flink-1.18.1/bin/flink run-application -d -t yarn-application \
-p 1 \
-Dyarn.application.queue=realtime \
-Dyarn.application.name=paimon-schema-test \
-Djobmanager.memory.process.size=1024mb \
-Dtaskmanager.memory.process.size=2048mb \
-Dtaskmanager.memory.network.max=64mb \
-Dtaskmanager.numberOfTaskSlots=1 \
-Dexecution.checkpointing.interval=60000 \
-c org.apache.paimon.flink.action.FlinkActions \
/paimon/paimon-flink-action-1.0.1.jar \
mysql_sync_database \
--mode combined \
--warehouse hdfs:///hive/paimon/ \
--database tmp \
--table_prefix "ods_" \
--mysql_conf hostname=xxx \
--mysql_conf username=xxx \
--mysql_conf password=xxx \
--mysql_conf database-name=test \
--catalog_conf metastore=hive \
--catalog_conf uri=thrift://xxx:7004,thrift://xxx:7004 \
--table_conf bucket=1 \
--table_conf changelog-producer=input \
--type_mapping tinyint1-not-bool,char-to-string \
--including_tables 'add_colum_table_test'
```
## 2. Dbeaver execute ddl
`alter table test.add_colum_table_test add column `city_name` varchar(32) NOT null default '0' COMMENT '城市名称';`
##3. paimon table sync job fail
```
2025-09-16 10:12:55
java.lang.IllegalArgumentException: Column city_name cannot specify NOT NULL in the test.add_colum_table_test table.
at org.apache.paimon.utils.Preconditions.checkArgument(Preconditions.java:149)
at org.apache.paimon.schema.SchemaManager.commitChanges(SchemaManager.java:333)
at org.apache.paimon.hive.HiveCatalog.alterTableImpl(HiveCatalog.java:905)
at org.apache.paimon.catalog.AbstractCatalog.alterTable(AbstractCatalog.java:375)
at org.apache.paimon.catalog.Catalog.alterTable(Catalog.java:278)
at org.apache.paimon.flink.sink.cdc.UpdatedDataFieldsProcessFunctionBase.applySchemaChange(UpdatedDataFieldsProcessFunctionBase.java:98)
at org.apache.paimon.flink.sink.cdc.MultiTableUpdatedDataFieldsProcessFunction.processElement(MultiTableUpdatedDataFieldsProcessFunction.java:84)
at org.apache.paimon.flink.sink.cdc.MultiTableUpdatedDataFieldsProcessFunction.processElement(MultiTableUpdatedDataFieldsProcessFunction.java:47)
at org.apache.flink.streaming.api.operators.LegacyKeyedProcessOperator.processElement(LegacyKeyedProcessOperator.java:88)
at org.apache.flink.streaming.runtime.io.RecordProcessorUtils.lambda$getRecordProcessor$0(RecordProcessorUtils.java:60)
at org.apache.flink.streaming.runtime.tasks.OneInputStreamTask$StreamTaskNetworkOutput.emitRecord(OneInputStreamTask.java:237)
at org.apache.flink.streaming.runtime.io.AbstractStreamTaskNetworkInput.processElement(AbstractStreamTaskNetworkInput.java:146)
at org.apache.flink.streaming.runtime.io.AbstractStreamTaskNetworkInput.emitNext(AbstractStreamTaskNetworkInput.java:110)
at org.apache.flink.streaming.runtime.io.StreamOneInputProcessor.processInput(StreamOneInputProcessor.java:65)
at org.apache.flink.streaming.runtime.tasks.StreamTask.processInput(StreamTask.java:562)
at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMailboxLoop(MailboxProcessor.java:231)
at org.apache.flink.streaming.runtime.tasks.StreamTask.runMailboxLoop(StreamTask.java:858)
at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:807)
at org.apache.flink.runtime.taskmanager.Task.runWithSystemExitMonitoring(Task.java:953)
at org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke(Task.java:932)
at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:746)
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:562)
at java.lang.Thread.run(Thread.java:748)
```
Why ` SchemaManager.java ` has check add column check not null ?
### What doesn't meet your expectations?
##3. paimon table sync job fail
```
2025-09-16 10:12:55
java.lang.IllegalArgumentException: Column city_name cannot specify NOT NULL in the test.add_colum_table_test table.
at org.apache.paimon.utils.Preconditions.checkArgument(Preconditions.java:149)
at org.apache.paimon.schema.SchemaManager.commitChanges(SchemaManager.java:333)
at org.apache.paimon.hive.HiveCatalog.alterTableImpl(HiveCatalog.java:905)
at org.apache.paimon.catalog.AbstractCatalog.alterTable(AbstractCatalog.java:375)
at org.apache.paimon.catalog.Catalog.alterTable(Catalog.java:278)
at org.apache.paimon.flink.sink.cdc.UpdatedDataFieldsProcessFunctionBase.applySchemaChange(UpdatedDataFieldsProcessFunctionBase.java:98)
at org.apache.paimon.flink.sink.cdc.MultiTableUpdatedDataFieldsProcessFunction.processElement(MultiTableUpdatedDataFieldsProcessFunction.java:84)
at org.apache.paimon.flink.sink.cdc.MultiTableUpdatedDataFieldsProcessFunction.processElement(MultiTableUpdatedDataFieldsProcessFunction.java:47)
at org.apache.flink.streaming.api.operators.LegacyKeyedProcessOperator.processElement(LegacyKeyedProcessOperator.java:88)
at org.apache.flink.streaming.runtime.io.RecordProcessorUtils.lambda$getRecordProcessor$0(RecordProcessorUtils.java:60)
at org.apache.flink.streaming.runtime.tasks.OneInputStreamTask$StreamTaskNetworkOutput.emitRecord(OneInputStreamTask.java:237)
at org.apache.flink.streaming.runtime.io.AbstractStreamTaskNetworkInput.processElement(AbstractStreamTaskNetworkInput.java:146)
at org.apache.flink.streaming.runtime.io.AbstractStreamTaskNetworkInput.emitNext(AbstractStreamTaskNetworkInput.java:110)
at org.apache.flink.streaming.runtime.io.StreamOneInputProcessor.processInput(StreamOneInputProcessor.java:65)
at org.apache.flink.streaming.runtime.tasks.StreamTask.processInput(StreamTask.java:562)
at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMailboxLoop(MailboxProcessor.java:231)
at org.apache.flink.streaming.runtime.tasks.StreamTask.runMailboxLoop(StreamTask.java:858)
at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:807)
at org.apache.flink.runtime.taskmanager.Task.runWithSystemExitMonitoring(Task.java:953)
at org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke(Task.java:932)
at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:746)
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:562)
at java.lang.Thread.run(Thread.java:748)
```
### Anything else?
_No response_
### Are you willing to submit a PR?
- [ ] I'm willing to submit a PR!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug] MySQL add column sync Paimon failed [paimon]
Posted by "DavidZ1 (via GitHub)" <gi...@apache.org>.
DavidZ1 commented on issue #6259:
URL: https://github.com/apache/paimon/issues/6259#issuecomment-4342700281
The root cause of the above issue is `https://issues.apache.org/jira/browse/FLINK-30545`. Paimon provides a solution: use the `--type_mapping to-nullable` parameter to resolve it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug] MySQL add column sync Paimon failed [paimon]
Posted by "DavidZ1 (via GitHub)" <gi...@apache.org>.
DavidZ1 closed issue #6259: [Bug] MySQL add column sync Paimon failed
URL: https://github.com/apache/paimon/issues/6259
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug] MySQL add column sync Paimon failed [paimon]
Posted by "sd4324530 (via GitHub)" <gi...@apache.org>.
sd4324530 commented on issue #6259:
URL: https://github.com/apache/paimon/issues/6259#issuecomment-3392938593
@DavidZ1 It should be that the columns of the paimon table do not support default values.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org