You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by GitBox <gi...@apache.org> on 2021/06/29 00:34:56 UTC
[GitHub] [iceberg] hankfanchiu opened a new issue #2751: "ICEBERG" vs. "iceberg" for "table_type" parameter
hankfanchiu opened a new issue #2751:
URL: https://github.com/apache/iceberg/issues/2751
Is the uppercase `"ICEBERG"` the official value for the `table_type` parameter? If so, would it be useful to add a "Catalog" subsection under [Specification](https://iceberg.apache.org/spec/#specification) in the documentation?
Within this project, I believe that all existing setters of `table_type` use the uppercase `"ICEBERG"`:
Hive:
https://github.com/apache/iceberg/blob/90225d6c9413016d611e2ce5eff37db1bc1b4fc5/python/iceberg/hive/hive_table_operations.py#L132-L133
https://github.com/apache/iceberg/blob/d5443e3a34a4288441a015ab616d965557d78202/hive-metastore/src/main/java/org/apache/iceberg/hive/HiveTableOperations.java#L359
https://github.com/apache/iceberg/blob/15616941461aaad3f03568ce4880171b86a06547/mr/src/main/java/org/apache/iceberg/mr/hive/HiveIcebergMetaHook.java#L80-L81
DynamoDB:
https://github.com/apache/iceberg/blob/f81d8adc9e4413265b63ca092ca744a22a0c9c65/aws/src/main/java/org/apache/iceberg/aws/dynamodb/DynamoDbTableOperations.java#L160
Glue:
https://github.com/apache/iceberg/blob/68c48ec9bd8c3c3ae14a365f937d1538a4fbf826/aws/src/main/java/org/apache/iceberg/aws/glue/GlueTableOperations.java#L172
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] pvary commented on issue #2751: "ICEBERG" vs. "iceberg" for "table_type" parameter
Posted by GitBox <gi...@apache.org>.
pvary commented on issue #2751:
URL: https://github.com/apache/iceberg/issues/2751#issuecomment-881322021
@hankfanchiu: You are right that we either has to say that everyone should handle the `ICEBERG_TABLE_TYPE_VALUE` case independent or we should declare the exact value for it. I would say that by the current state of the things we handle it as a case independent, and everyone should make sure to prepare for this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] hankfanchiu commented on issue #2751: "ICEBERG" vs. "iceberg" for "table_type" parameter
Posted by GitBox <gi...@apache.org>.
hankfanchiu commented on issue #2751:
URL: https://github.com/apache/iceberg/issues/2751#issuecomment-881179252
> The Iceberg code currently uses this method to check if the table is an iceberg table or not:
> https://github.com/apache/iceberg/blob/d5443e3a34a4288441a015ab616d965557d78202/hive-metastore/src/main/java/org/apache/iceberg/hive/HiveTableOperations.java#L506-L510
>
> This should not be case sensitive.
Wondering whether my suggested change in #2722 is viable, comparing with the uppercase `"ICEBERG"` string value:
https://github.com/apache/iceberg/blob/a7e219453e5e56da7f3505b7163c44e92e35b9e6/hive-metastore/src/main/java/org/apache/iceberg/hive/HiveCatalog.java#L120-L123
Supporting lowercase certainly wouldn't be difficult, something like:
```java
String filterKey = hive_metastoreConstants.HIVE_FILTER_FIELD_PARAMS + BaseMetastoreTableOperations.TABLE_TYPE_PROP;
String filter = String.format("%s = \"%s\" OR %s = \"%s\"",
filterKey,
BaseMetastoreTableOperations.ICEBERG_TABLE_TYPE_VALUE.toUpperCase(Locale.ENGLISH),
filterKey,
BaseMetastoreTableOperations.ICEBERG_TABLE_TYPE_VALUE);
```
But what if some new system sets a Hive table type value of `"Iceberg"`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] hankfanchiu commented on issue #2751: "ICEBERG" vs. "iceberg" for "table_type" parameter
Posted by GitBox <gi...@apache.org>.
hankfanchiu commented on issue #2751:
URL: https://github.com/apache/iceberg/issues/2751#issuecomment-870141138
Trino's Iceberg connector currently sets the Hive table type parameter as the lowercase `"iceberg"`:
https://github.com/trinodb/trino/blob/e07325b501e7658e42579742182406ecd0a5fa62/plugin/trino-iceberg/src/main/java/io/trino/plugin/iceberg/HiveTableOperations.java#L181
https://github.com/trinodb/trino/issues/1592#issuecomment-536049023 suggested documenting the official `table_type` parameter value.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] pvary commented on issue #2751: "ICEBERG" vs. "iceberg" for "table_type" parameter
Posted by GitBox <gi...@apache.org>.
pvary commented on issue #2751:
URL: https://github.com/apache/iceberg/issues/2751#issuecomment-901651149
> @pvary, given that the Hive MetaStore does not support case-insensitive filtering on a table's parameter, should I close my #2722, then?
We could still query with case insensitivity form HMS and then filter the results with case sensitivity in Java code. This way we still can have a good performance improvement if there are too many tables.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] pvary commented on issue #2751: "ICEBERG" vs. "iceberg" for "table_type" parameter
Posted by GitBox <gi...@apache.org>.
pvary commented on issue #2751:
URL: https://github.com/apache/iceberg/issues/2751#issuecomment-878223980
@hankfanchiu: Have you find any problem with setting it differently?
The Iceberg code currently uses this method to check if the table is an iceberg table or not:
https://github.com/apache/iceberg/blob/d5443e3a34a4288441a015ab616d965557d78202/hive-metastore/src/main/java/org/apache/iceberg/hive/HiveTableOperations.java#L506-L510
This should not be case sensitive.
Thanks,
Peter
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] hankfanchiu commented on issue #2751: "ICEBERG" vs. "iceberg" for "table_type" parameter
Posted by GitBox <gi...@apache.org>.
hankfanchiu commented on issue #2751:
URL: https://github.com/apache/iceberg/issues/2751#issuecomment-870141138
Trino's Iceberg connector currently sets the Hive table type parameter as the lowercase `"iceberg"`:
https://github.com/trinodb/trino/blob/e07325b501e7658e42579742182406ecd0a5fa62/plugin/trino-iceberg/src/main/java/io/trino/plugin/iceberg/HiveTableOperations.java#L181
https://github.com/trinodb/trino/issues/1592#issuecomment-536049023 suggested documenting the official `table_type` parameter value.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] hankfanchiu commented on issue #2751: "ICEBERG" vs. "iceberg" for "table_type" parameter
Posted by GitBox <gi...@apache.org>.
hankfanchiu commented on issue #2751:
URL: https://github.com/apache/iceberg/issues/2751#issuecomment-896548451
> I would say that by the current state of the things we handle it as a case independent
@pvary, given that the Hive MetaStore does not support case-insensitive filtering on a table's parameter, should I close my #2722, then?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org