You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by GitBox <gi...@apache.org> on 2021/01/13 14:42:02 UTC
[GitHub] [iceberg] lcspinter opened a new pull request #2086: Hive: Schema conversion should convert field comments
lcspinter opened a new pull request #2086:
URL: https://github.com/apache/iceberg/pull/2086
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] lcspinter commented on pull request #2086: Hive: Schema conversion should convert field comments
Posted by GitBox <gi...@apache.org>.
lcspinter commented on pull request #2086:
URL: https://github.com/apache/iceberg/pull/2086#issuecomment-759492779
@marton-bod @pvary Could you please have a look at this change? Thank you
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rdblue merged pull request #2086: Hive: Schema conversion should convert field comments
Posted by GitBox <gi...@apache.org>.
rdblue merged pull request #2086:
URL: https://github.com/apache/iceberg/pull/2086
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] pvary commented on a change in pull request #2086: Hive: Schema conversion should convert field comments
Posted by GitBox <gi...@apache.org>.
pvary commented on a change in pull request #2086:
URL: https://github.com/apache/iceberg/pull/2086#discussion_r556648080
##########
File path: mr/src/test/java/org/apache/iceberg/mr/hive/TestHiveIcebergStorageHandlerWithEngine.java
##########
@@ -150,6 +151,19 @@ public void testScanTable() throws IOException {
Assert.assertArrayEquals(new Object[] {"Alice", 0L}, descRows.get(2));
}
+ @Test
+ public void testDescribeTable() throws IOException {
Review comment:
It would be good to have a test for:
```
CREATE TABLE alma (korte INT COMMENT 'barack')
```
I think the change need to be done here:
https://github.com/apache/iceberg/blob/969a1ffc8d7392cc4c54eba156647c448757b45e/mr/src/main/java/org/apache/iceberg/mr/hive/HiveIcebergSerDe.java#L135-L153
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] lcspinter commented on a change in pull request #2086: Hive: Schema conversion should convert field comments
Posted by GitBox <gi...@apache.org>.
lcspinter commented on a change in pull request #2086:
URL: https://github.com/apache/iceberg/pull/2086#discussion_r556711309
##########
File path: mr/src/test/java/org/apache/iceberg/mr/hive/TestHiveIcebergStorageHandlerWithEngine.java
##########
@@ -150,6 +151,19 @@ public void testScanTable() throws IOException {
Assert.assertArrayEquals(new Object[] {"Alice", 0L}, descRows.get(2));
}
+ @Test
+ public void testDescribeTable() throws IOException {
Review comment:
Yes, I added some dummy comments
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rdblue commented on a change in pull request #2086: Hive: Schema conversion should convert field comments
Posted by GitBox <gi...@apache.org>.
rdblue commented on a change in pull request #2086:
URL: https://github.com/apache/iceberg/pull/2086#discussion_r562306710
##########
File path: hive-metastore/src/test/java/org/apache/iceberg/hive/TestHiveSchemaUtil.java
##########
@@ -181,16 +182,16 @@ public void testComplexTypeAndTypeInfoConvert() {
protected Schema getSchemaWithSupportedTypes() {
return new Schema(
- optional(0, "c_float", Types.FloatType.get()),
- optional(1, "c_double", Types.DoubleType.get()),
- optional(2, "c_boolean", Types.BooleanType.get()),
- optional(3, "c_int", Types.IntegerType.get()),
- optional(4, "c_long", Types.LongType.get()),
- optional(5, "c_binary", Types.BinaryType.get()),
- optional(6, "c_string", Types.StringType.get()),
- optional(7, "c_timestamp", Types.TimestampType.withoutZone()),
- optional(8, "c_date", Types.DateType.get()),
- optional(9, "c_decimal", Types.DecimalType.of(38, 10)));
+ optional(0, "c_float", Types.FloatType.get(), ""),
+ optional(1, "c_double", Types.DoubleType.get(), ""),
+ optional(2, "c_boolean", Types.BooleanType.get(), ""),
+ optional(3, "c_int", Types.IntegerType.get(), ""),
+ optional(4, "c_long", Types.LongType.get(), ""),
+ optional(5, "c_binary", Types.BinaryType.get(), ""),
+ optional(6, "c_string", Types.StringType.get(), ""),
+ optional(7, "c_timestamp", Types.TimestampType.withoutZone(), ""),
+ optional(8, "c_date", Types.DateType.get(), ""),
+ optional(9, "c_decimal", Types.DecimalType.of(38, 10), ""));
Review comment:
Just to make sure: this should not use empty string if there is no comment. If there is no comment, then it should use null. It's a little misleading to see empty string like this in tests. Could you remove half of them and add non-empty doc strings to the other half?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] lcspinter commented on a change in pull request #2086: Hive: Schema conversion should convert field comments
Posted by GitBox <gi...@apache.org>.
lcspinter commented on a change in pull request #2086:
URL: https://github.com/apache/iceberg/pull/2086#discussion_r557163340
##########
File path: mr/src/test/java/org/apache/iceberg/mr/hive/TestHiveIcebergStorageHandlerWithEngine.java
##########
@@ -150,6 +151,19 @@ public void testScanTable() throws IOException {
Assert.assertArrayEquals(new Object[] {"Alice", 0L}, descRows.get(2));
}
+ @Test
+ public void testDescribeTable() throws IOException {
Review comment:
Yes, the comments are now available for each column.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rdblue commented on pull request #2086: Hive: Schema conversion should convert field comments
Posted by GitBox <gi...@apache.org>.
rdblue commented on pull request #2086:
URL: https://github.com/apache/iceberg/pull/2086#issuecomment-765600935
Thanks, @lcspinter!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] pvary commented on a change in pull request #2086: Hive: Schema conversion should convert field comments
Posted by GitBox <gi...@apache.org>.
pvary commented on a change in pull request #2086:
URL: https://github.com/apache/iceberg/pull/2086#discussion_r556679384
##########
File path: mr/src/test/java/org/apache/iceberg/mr/hive/TestHiveIcebergStorageHandlerWithEngine.java
##########
@@ -150,6 +151,19 @@ public void testScanTable() throws IOException {
Assert.assertArrayEquals(new Object[] {"Alice", 0L}, descRows.get(2));
}
+ @Test
+ public void testDescribeTable() throws IOException {
Review comment:
Do we create these tables with comments for the colums?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] pvary commented on a change in pull request #2086: Hive: Schema conversion should convert field comments
Posted by GitBox <gi...@apache.org>.
pvary commented on a change in pull request #2086:
URL: https://github.com/apache/iceberg/pull/2086#discussion_r556745909
##########
File path: mr/src/test/java/org/apache/iceberg/mr/hive/TestHiveIcebergStorageHandlerWithEngine.java
##########
@@ -150,6 +151,19 @@ public void testScanTable() throws IOException {
Assert.assertArrayEquals(new Object[] {"Alice", 0L}, descRows.get(2));
}
+ @Test
+ public void testDescribeTable() throws IOException {
Review comment:
Have we checked that the Hive table contains the comments?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rdsr commented on a change in pull request #2086: Hive: Schema conversion should convert field comments
Posted by GitBox <gi...@apache.org>.
rdsr commented on a change in pull request #2086:
URL: https://github.com/apache/iceberg/pull/2086#discussion_r556727997
##########
File path: hive-metastore/src/main/java/org/apache/iceberg/hive/HiveSchemaConverter.java
##########
@@ -110,7 +111,7 @@ Type convertType(TypeInfo typeInfo) {
case STRUCT:
StructTypeInfo structTypeInfo = (StructTypeInfo) typeInfo;
List<Types.NestedField> fields =
- convertInternal(structTypeInfo.getAllStructFieldNames(), structTypeInfo.getAllStructFieldTypeInfos());
+ convertInternal(structTypeInfo.getAllStructFieldNames(), structTypeInfo.getAllStructFieldTypeInfos(), null);
Review comment:
Should the comment parameter be an empty list instead of null? This may remove various null checks in the code e.g in `convertInternal` above
Also, for an empty struct I believe `fieldnames` and `typeinfos` will be empty instead of null so it maybe more consistent for it to be empty.
##########
File path: mr/src/main/java/org/apache/iceberg/mr/hive/HiveIcebergSerDe.java
##########
@@ -137,15 +137,21 @@ private static Schema hiveSchemaOrThrow(Properties serDeProperties, Exception pr
// Read the configuration parameters
String columnNames = serDeProperties.getProperty(serdeConstants.LIST_COLUMNS);
String columnTypes = serDeProperties.getProperty(serdeConstants.LIST_COLUMN_TYPES);
+ // No constant for column comments and column comments delimiter.
Review comment:
not sure I follow this comment..
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] lcspinter commented on pull request #2086: Hive: Schema conversion should convert field comments
Posted by GitBox <gi...@apache.org>.
lcspinter commented on pull request #2086:
URL: https://github.com/apache/iceberg/pull/2086#issuecomment-765310572
Thanks for the review @pvary @rdsr @rdblue !
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] pvary commented on a change in pull request #2086: Hive: Schema conversion should convert field comments
Posted by GitBox <gi...@apache.org>.
pvary commented on a change in pull request #2086:
URL: https://github.com/apache/iceberg/pull/2086#discussion_r558268127
##########
File path: mr/src/test/java/org/apache/iceberg/mr/hive/HiveIcebergStorageHandlerTestUtils.java
##########
@@ -36,9 +36,9 @@
new FileFormat[] {FileFormat.AVRO, FileFormat.ORC, FileFormat.PARQUET};
static final Schema CUSTOMER_SCHEMA = new Schema(
- optional(1, "customer_id", Types.LongType.get()),
- optional(2, "first_name", Types.StringType.get()),
- optional(3, "last_name", Types.StringType.get())
+ optional(1, "customer_id", Types.LongType.get(), "This is an ID"),
+ optional(2, "first_name", Types.StringType.get(), "This is first name"),
+ optional(3, "last_name", Types.StringType.get(), "This is last name")
Review comment:
do we have to have a comment for all of columns?
shall we test at least one without a comment?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rdblue commented on a change in pull request #2086: Hive: Schema conversion should convert field comments
Posted by GitBox <gi...@apache.org>.
rdblue commented on a change in pull request #2086:
URL: https://github.com/apache/iceberg/pull/2086#discussion_r562306320
##########
File path: mr/src/test/java/org/apache/iceberg/mr/hive/TestHiveIcebergStorageHandlerNoScan.java
##########
@@ -504,21 +506,60 @@ public void testCreateTableWithNotSupportedTypesWithAutoConversion() {
TableIdentifier identifier = TableIdentifier.of("default", "not_supported_types");
// Can not create INTERVAL types from normal create table, so leave them out from this test
Map<String, Type> notSupportedTypes = ImmutableMap.of(
- "TINYINT", Types.IntegerType.get(),
- "SMALLINT", Types.IntegerType.get(),
- "VARCHAR(1)", Types.StringType.get(),
- "CHAR(1)", Types.StringType.get());
+ "TINYINT", Types.IntegerType.get(),
+ "SMALLINT", Types.IntegerType.get(),
+ "VARCHAR(1)", Types.StringType.get(),
+ "CHAR(1)", Types.StringType.get());
Review comment:
Can you revert these whitespace-only changes?
##########
File path: hive-metastore/src/test/java/org/apache/iceberg/hive/TestHiveSchemaUtil.java
##########
@@ -181,16 +182,16 @@ public void testComplexTypeAndTypeInfoConvert() {
protected Schema getSchemaWithSupportedTypes() {
return new Schema(
- optional(0, "c_float", Types.FloatType.get()),
- optional(1, "c_double", Types.DoubleType.get()),
- optional(2, "c_boolean", Types.BooleanType.get()),
- optional(3, "c_int", Types.IntegerType.get()),
- optional(4, "c_long", Types.LongType.get()),
- optional(5, "c_binary", Types.BinaryType.get()),
- optional(6, "c_string", Types.StringType.get()),
- optional(7, "c_timestamp", Types.TimestampType.withoutZone()),
- optional(8, "c_date", Types.DateType.get()),
- optional(9, "c_decimal", Types.DecimalType.of(38, 10)));
+ optional(0, "c_float", Types.FloatType.get(), ""),
+ optional(1, "c_double", Types.DoubleType.get(), ""),
+ optional(2, "c_boolean", Types.BooleanType.get(), ""),
+ optional(3, "c_int", Types.IntegerType.get(), ""),
+ optional(4, "c_long", Types.LongType.get(), ""),
+ optional(5, "c_binary", Types.BinaryType.get(), ""),
+ optional(6, "c_string", Types.StringType.get(), ""),
+ optional(7, "c_timestamp", Types.TimestampType.withoutZone(), ""),
+ optional(8, "c_date", Types.DateType.get(), ""),
+ optional(9, "c_decimal", Types.DecimalType.of(38, 10), ""));
Review comment:
Just to make sure: this should not use empty string if there is no comment. If there is no comment, then it should use null. It's a little misleading to see empty string like this in tests. Could you remove half of them and add non-empty doc strings to the other half?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] lcspinter commented on a change in pull request #2086: Hive: Schema conversion should convert field comments
Posted by GitBox <gi...@apache.org>.
lcspinter commented on a change in pull request #2086:
URL: https://github.com/apache/iceberg/pull/2086#discussion_r557218244
##########
File path: mr/src/main/java/org/apache/iceberg/mr/hive/HiveIcebergSerDe.java
##########
@@ -137,15 +137,21 @@ private static Schema hiveSchemaOrThrow(Properties serDeProperties, Exception pr
// Read the configuration parameters
String columnNames = serDeProperties.getProperty(serdeConstants.LIST_COLUMNS);
String columnTypes = serDeProperties.getProperty(serdeConstants.LIST_COLUMN_TYPES);
+ // No constant for column comments and column comments delimiter.
Review comment:
What I meant here, that there is no constant definition among the serdeConstants, therefore I had to hardcode the column comment ("columns.comments") and column delimiter (Character.MIN_VALUE) property names.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] lcspinter commented on a change in pull request #2086: Hive: Schema conversion should convert field comments
Posted by GitBox <gi...@apache.org>.
lcspinter commented on a change in pull request #2086:
URL: https://github.com/apache/iceberg/pull/2086#discussion_r562537758
##########
File path: mr/src/test/java/org/apache/iceberg/mr/hive/TestHiveIcebergStorageHandlerNoScan.java
##########
@@ -504,21 +506,60 @@ public void testCreateTableWithNotSupportedTypesWithAutoConversion() {
TableIdentifier identifier = TableIdentifier.of("default", "not_supported_types");
// Can not create INTERVAL types from normal create table, so leave them out from this test
Map<String, Type> notSupportedTypes = ImmutableMap.of(
- "TINYINT", Types.IntegerType.get(),
- "SMALLINT", Types.IntegerType.get(),
- "VARCHAR(1)", Types.StringType.get(),
- "CHAR(1)", Types.StringType.get());
+ "TINYINT", Types.IntegerType.get(),
+ "SMALLINT", Types.IntegerType.get(),
+ "VARCHAR(1)", Types.StringType.get(),
+ "CHAR(1)", Types.StringType.get());
Review comment:
Done.
##########
File path: hive-metastore/src/test/java/org/apache/iceberg/hive/TestHiveSchemaUtil.java
##########
@@ -181,16 +182,16 @@ public void testComplexTypeAndTypeInfoConvert() {
protected Schema getSchemaWithSupportedTypes() {
return new Schema(
- optional(0, "c_float", Types.FloatType.get()),
- optional(1, "c_double", Types.DoubleType.get()),
- optional(2, "c_boolean", Types.BooleanType.get()),
- optional(3, "c_int", Types.IntegerType.get()),
- optional(4, "c_long", Types.LongType.get()),
- optional(5, "c_binary", Types.BinaryType.get()),
- optional(6, "c_string", Types.StringType.get()),
- optional(7, "c_timestamp", Types.TimestampType.withoutZone()),
- optional(8, "c_date", Types.DateType.get()),
- optional(9, "c_decimal", Types.DecimalType.of(38, 10)));
+ optional(0, "c_float", Types.FloatType.get(), ""),
+ optional(1, "c_double", Types.DoubleType.get(), ""),
+ optional(2, "c_boolean", Types.BooleanType.get(), ""),
+ optional(3, "c_int", Types.IntegerType.get(), ""),
+ optional(4, "c_long", Types.LongType.get(), ""),
+ optional(5, "c_binary", Types.BinaryType.get(), ""),
+ optional(6, "c_string", Types.StringType.get(), ""),
+ optional(7, "c_timestamp", Types.TimestampType.withoutZone(), ""),
+ optional(8, "c_date", Types.DateType.get(), ""),
+ optional(9, "c_decimal", Types.DecimalType.of(38, 10), ""));
Review comment:
Fixed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] pvary commented on a change in pull request #2086: Hive: Schema conversion should convert field comments
Posted by GitBox <gi...@apache.org>.
pvary commented on a change in pull request #2086:
URL: https://github.com/apache/iceberg/pull/2086#discussion_r556745239
##########
File path: mr/src/test/java/org/apache/iceberg/mr/hive/TestHiveIcebergStorageHandlerWithEngine.java
##########
@@ -150,6 +151,19 @@ public void testScanTable() throws IOException {
Assert.assertArrayEquals(new Object[] {"Alice", 0L}, descRows.get(2));
}
+ @Test
+ public void testDescribeTable() throws IOException {
Review comment:
Opps... missed that :( Sorry
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rdblue commented on a change in pull request #2086: Hive: Schema conversion should convert field comments
Posted by GitBox <gi...@apache.org>.
rdblue commented on a change in pull request #2086:
URL: https://github.com/apache/iceberg/pull/2086#discussion_r562306320
##########
File path: mr/src/test/java/org/apache/iceberg/mr/hive/TestHiveIcebergStorageHandlerNoScan.java
##########
@@ -504,21 +506,60 @@ public void testCreateTableWithNotSupportedTypesWithAutoConversion() {
TableIdentifier identifier = TableIdentifier.of("default", "not_supported_types");
// Can not create INTERVAL types from normal create table, so leave them out from this test
Map<String, Type> notSupportedTypes = ImmutableMap.of(
- "TINYINT", Types.IntegerType.get(),
- "SMALLINT", Types.IntegerType.get(),
- "VARCHAR(1)", Types.StringType.get(),
- "CHAR(1)", Types.StringType.get());
+ "TINYINT", Types.IntegerType.get(),
+ "SMALLINT", Types.IntegerType.get(),
+ "VARCHAR(1)", Types.StringType.get(),
+ "CHAR(1)", Types.StringType.get());
Review comment:
Can you revert these whitespace-only changes?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rdblue commented on pull request #2086: Hive: Schema conversion should convert field comments
Posted by GitBox <gi...@apache.org>.
rdblue commented on pull request #2086:
URL: https://github.com/apache/iceberg/pull/2086#issuecomment-763851131
@pvary, could you rebase this? Looks like it has conflicts.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] lcspinter commented on a change in pull request #2086: Hive: Schema conversion should convert field comments
Posted by GitBox <gi...@apache.org>.
lcspinter commented on a change in pull request #2086:
URL: https://github.com/apache/iceberg/pull/2086#discussion_r557211811
##########
File path: hive-metastore/src/main/java/org/apache/iceberg/hive/HiveSchemaConverter.java
##########
@@ -110,7 +111,7 @@ Type convertType(TypeInfo typeInfo) {
case STRUCT:
StructTypeInfo structTypeInfo = (StructTypeInfo) typeInfo;
List<Types.NestedField> fields =
- convertInternal(structTypeInfo.getAllStructFieldNames(), structTypeInfo.getAllStructFieldTypeInfos());
+ convertInternal(structTypeInfo.getAllStructFieldNames(), structTypeInfo.getAllStructFieldTypeInfos(), null);
Review comment:
Thanks for the review. Good point, I changed the parameter to an empty list.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] lcspinter commented on a change in pull request #2086: Hive: Schema conversion should convert field comments
Posted by GitBox <gi...@apache.org>.
lcspinter commented on a change in pull request #2086:
URL: https://github.com/apache/iceberg/pull/2086#discussion_r556652230
##########
File path: mr/src/test/java/org/apache/iceberg/mr/hive/TestHiveIcebergStorageHandlerWithEngine.java
##########
@@ -150,6 +151,19 @@ public void testScanTable() throws IOException {
Assert.assertArrayEquals(new Object[] {"Alice", 0L}, descRows.get(2));
}
+ @Test
+ public void testDescribeTable() throws IOException {
Review comment:
I have a few tests in TestHiveIcebergStorageHandlerLocalScan.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] lcspinter commented on a change in pull request #2086: Hive: Schema conversion should convert field comments
Posted by GitBox <gi...@apache.org>.
lcspinter commented on a change in pull request #2086:
URL: https://github.com/apache/iceberg/pull/2086#discussion_r558315619
##########
File path: mr/src/test/java/org/apache/iceberg/mr/hive/HiveIcebergStorageHandlerTestUtils.java
##########
@@ -36,9 +36,9 @@
new FileFormat[] {FileFormat.AVRO, FileFormat.ORC, FileFormat.PARQUET};
static final Schema CUSTOMER_SCHEMA = new Schema(
- optional(1, "customer_id", Types.LongType.get()),
- optional(2, "first_name", Types.StringType.get()),
- optional(3, "last_name", Types.StringType.get())
+ optional(1, "customer_id", Types.LongType.get(), "This is an ID"),
+ optional(2, "first_name", Types.StringType.get(), "This is first name"),
+ optional(3, "last_name", Types.StringType.get(), "This is last name")
Review comment:
Yes, it makes sense to remove one comment. I created a new test to check uncommented columns as well.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org