You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by GitBox <gi...@apache.org> on 2021/06/05 13:56:37 UTC
[GitHub] [iceberg] moon-fall opened a new issue #2679: error when run flink-sql with iceberg table that has identifierFieldIds
moon-fall opened a new issue #2679:
URL: https://github.com/apache/iceberg/issues/2679
error like this
Exception in thread "main" org.apache.flink.table.api.ValidationException: Could not create a PRIMARY KEY '17bb7db3-e030-4639-ba8a-ff74f78649f6'. Column 'id' is nullable.
at org.apache.flink.table.api.TableSchema.validatePrimaryKey(TableSchema.java:468)
at org.apache.flink.table.api.TableSchema.access$200(TableSchema.java:59)
at org.apache.flink.table.api.TableSchema$Builder.build(TableSchema.java:698)
at org.apache.iceberg.flink.FlinkSchemaUtil.toSchema(FlinkSchemaUtil.java:173)
at org.apache.iceberg.flink.FlinkCatalog.toCatalogTable(FlinkCatalog.java:535)
at org.apache.iceberg.flink.FlinkCatalog.getTable(FlinkCatalog.java:311)
at org.apache.iceberg.flink.FlinkCatalog.getTable(FlinkCatalog.java:90)
at org.apache.flink.table.catalog.CatalogManager.getPermanentTable(CatalogManager.java:412)
at org.apache.flink.table.catalog.CatalogManager.getTable(CatalogManager.java:378)
at org.apache.flink.table.planner.delegation.PlannerBase.getTableSink(PlannerBase.scala:335)
at org.apache.flink.table.planner.delegation.PlannerBase.translateToRel(PlannerBase.scala:220)
at org.apache.flink.table.planner.delegation.PlannerBase$$anonfun$1.apply(PlannerBase.scala:164)
at org.apache.flink.table.planner.delegation.PlannerBase$$anonfun$1.apply(PlannerBase.scala:164)
at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
at scala.collection.Iterator$class.foreach(Iterator.scala:891)
at scala.collection.AbstractIterator.foreach(Iterator.scala:1334)
at scala.collection.IterableLike$class.foreach(IterableLike.scala:72)
at scala.collection.AbstractIterable.foreach(Iterable.scala:54)
at scala.collection.TraversableLike$class.map(TraversableLike.scala:234)
at scala.collection.AbstractTraversable.map(Traversable.scala:104)
at org.apache.flink.table.planner.delegation.PlannerBase.translate(PlannerBase.scala:164)
at org.apache.flink.table.api.internal.TableEnvironmentImpl.translate(TableEnvironmentImpl.java:1329)
at org.apache.flink.table.api.internal.TableEnvironmentImpl.executeInternal(TableEnvironmentImpl.java:676)
at org.apache.flink.table.api.internal.TableEnvironmentImpl.executeOperation(TableEnvironmentImpl.java:767)
at org.apache.flink.table.api.internal.TableEnvironmentImpl.executeSql(TableEnvironmentImpl.java:666)
at CdcToIceberg.main(CdcToIceberg.java:89)
i think it should set the identifierFieldIds not null when convert iceberg schema to flink table schema
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] jackye1995 commented on issue #2679: error when run flink-sql with iceberg table that has identifierFieldIds
Posted by GitBox <gi...@apache.org>.
jackye1995 commented on issue #2679:
URL: https://github.com/apache/iceberg/issues/2679#issuecomment-856132676
I think this is because the Iceberg identifier field can be null, but Flink does not allow null primary key column. Is it the case for you? What is the type of `id` column in your Iceberg schema?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] jackye1995 commented on issue #2679: error when run flink-sql with iceberg table that has identifierFieldIds
Posted by GitBox <gi...@apache.org>.
jackye1995 commented on issue #2679:
URL: https://github.com/apache/iceberg/issues/2679#issuecomment-856930899
Sorry when I say type I was trying to ask if you have set your `id` column to be optional or not in Iceberg schema. If your `id` column is optional, I think this is expected behavior.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] moon-fall closed issue #2679: error when run flink-sql with iceberg table that has identifierFieldIds
Posted by GitBox <gi...@apache.org>.
moon-fall closed issue #2679:
URL: https://github.com/apache/iceberg/issues/2679
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] moon-fall commented on issue #2679: error when run flink-sql with iceberg table that has identifierFieldIds
Posted by GitBox <gi...@apache.org>.
moon-fall commented on issue #2679:
URL: https://github.com/apache/iceberg/issues/2679#issuecomment-856691813
> I think this is because the Iceberg identifier field column can be nullable, but Flink does not allow nullable primary key column. Is it the case for you? What is the type of `id` column in your Iceberg schema?
yes, the type of `id` column is string
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] jackye1995 edited a comment on issue #2679: error when run flink-sql with iceberg table that has identifierFieldIds
Posted by GitBox <gi...@apache.org>.
jackye1995 edited a comment on issue #2679:
URL: https://github.com/apache/iceberg/issues/2679#issuecomment-856132676
I think this is because the Iceberg identifier field column can be nullable, but Flink does not allow nullable primary key column. Is it the case for you? What is the type of `id` column in your Iceberg schema?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] moon-fall commented on issue #2679: error when run flink-sql with iceberg table that has identifierFieldIds
Posted by GitBox <gi...@apache.org>.
moon-fall commented on issue #2679:
URL: https://github.com/apache/iceberg/issues/2679#issuecomment-857663033
> Sorry when I say type I was trying to ask if you have set your `id` column to be optional or not in Iceberg schema. If your `id` column is optional, I think this is expected behavior.
I do set the `id` column to be optional，and I set it to required then it runs no error any more，thanks for your response.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org