You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by GitBox <gi...@apache.org> on 2022/04/18 04:19:07 UTC
[GitHub] [iceberg] yittg commented on a diff in pull request #4561: Flink: support modify table properties with primary key.
yittg commented on code in PR #4561:
URL: https://github.com/apache/iceberg/pull/4561#discussion_r851867733
##########
flink/v1.14/flink/src/main/java/org/apache/iceberg/flink/FlinkCatalog.java:
##########
@@ -400,6 +400,30 @@ void createIcebergTable(ObjectPath tablePath, CatalogBaseTable table, boolean ig
}
}
+ private static void validateTableSchemaAndPartition(CatalogTable ct1, CatalogTable ct2) {
+ TableSchema ts1 = ct1.getSchema();
+ TableSchema ts2 = ct2.getSchema();
+ boolean equalsPrimary = false;
+ if (ts1.getPrimaryKey().isPresent() && ts2.getPrimaryKey().isPresent()) {
+ equalsPrimary = ts1.getPrimaryKey().get().getType().equals(ts2.getPrimaryKey().get().getType()) &&
+ ts1.getPrimaryKey().get().getColumns().equals(ts2.getPrimaryKey().get().getColumns());
+ } else if (!ts1.getPrimaryKey().isPresent() && !ts2.getPrimaryKey().isPresent()) {
+ equalsPrimary = true;
+ }
Review Comment:
how about wrapping a function `primaryKeyEqualsIgnoreName`
##########
flink/v1.14/flink/src/main/java/org/apache/iceberg/flink/FlinkCatalog.java:
##########
@@ -400,6 +400,30 @@ void createIcebergTable(ObjectPath tablePath, CatalogBaseTable table, boolean ig
}
}
+ private static void validateTableSchemaAndPartition(CatalogTable ct1, CatalogTable ct2) {
+ TableSchema ts1 = ct1.getSchema();
+ TableSchema ts2 = ct2.getSchema();
+ boolean equalsPrimary = false;
+ if (ts1.getPrimaryKey().isPresent() && ts2.getPrimaryKey().isPresent()) {
+ equalsPrimary = ts1.getPrimaryKey().get().getType().equals(ts2.getPrimaryKey().get().getType()) &&
+ ts1.getPrimaryKey().get().getColumns().equals(ts2.getPrimaryKey().get().getColumns());
+ } else if (!ts1.getPrimaryKey().isPresent() && !ts2.getPrimaryKey().isPresent()) {
+ equalsPrimary = true;
+ }
+
+ // For current Flink Catalog API, support for adding/removing/renaming columns cannot be done by comparing
+ // CatalogTable instances, unless the Flink schema contains Iceberg column IDs.
+ if (!equalsPrimary &&
+ ts1.getTableColumns().equals(ts2.getTableColumns()) &&
+ ts1.getWatermarkSpecs().equals(ts2.getWatermarkSpecs())) {
+ throw new UnsupportedOperationException("Altering schema is not supported yet.");
Review Comment:
And would you mind validating it by `Objects.equals` as flink does originally?
##########
flink/v1.14/flink/src/main/java/org/apache/iceberg/flink/FlinkCatalog.java:
##########
@@ -400,6 +400,30 @@ void createIcebergTable(ObjectPath tablePath, CatalogBaseTable table, boolean ig
}
}
+ private static void validateTableSchemaAndPartition(CatalogTable ct1, CatalogTable ct2) {
+ TableSchema ts1 = ct1.getSchema();
+ TableSchema ts2 = ct2.getSchema();
+ boolean equalsPrimary = false;
+ if (ts1.getPrimaryKey().isPresent() && ts2.getPrimaryKey().isPresent()) {
+ equalsPrimary = ts1.getPrimaryKey().get().getType().equals(ts2.getPrimaryKey().get().getType()) &&
+ ts1.getPrimaryKey().get().getColumns().equals(ts2.getPrimaryKey().get().getColumns());
+ } else if (!ts1.getPrimaryKey().isPresent() && !ts2.getPrimaryKey().isPresent()) {
+ equalsPrimary = true;
+ }
+
+ // For current Flink Catalog API, support for adding/removing/renaming columns cannot be done by comparing
+ // CatalogTable instances, unless the Flink schema contains Iceberg column IDs.
+ if (!equalsPrimary &&
+ ts1.getTableColumns().equals(ts2.getTableColumns()) &&
+ ts1.getWatermarkSpecs().equals(ts2.getWatermarkSpecs())) {
+ throw new UnsupportedOperationException("Altering schema is not supported yet.");
Review Comment:
it should be `if (!(equalsPrimary && columns && watermarks)) {}`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org