You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by GitBox <gi...@apache.org> on 2021/08/25 18:06:41 UTC
[GitHub] [iceberg] dmgcodevil opened a new issue #3025: Schema field ids overridden when using nested structs
dmgcodevil opened a new issue #3025:
URL: https://github.com/apache/iceberg/issues/3025
Why Iceberg overrides schema fields ids for nested structs. For example in my schema struct field 'LAST_TRADE_PRICE' has id 53, however in metadata json it's "id" : 49
Shema:
```java
Types.NestedField.optional(53, LAST_TRADE_PRICE, Types.StructType.of(
Types.NestedField.required(54, PRICE, Types.DoubleType.get()),
Types.NestedField.required(55, YIELD, Types.DoubleType.get()),
Types.NestedField.required(56, DISCOUNT, Types.DoubleType.get()),
Types.NestedField.required(57, SCORE, Types.IntegerType.get())
)),
``
metadata json:
```json
"id" : 49,
"name" : "last_trade_price",
"required" : false,
"type" : {
"type" : "struct",
"fields" : [ {
"id" : 56,
"name" : "price",
"required" : true,
"type" : "double"
}, {
"id" : 57,
"name" : "yield",
"required" : true,
"type" : "double"
}, {
"id" : 58,
"name" : "discount",
"required" : true,
"type" : "double"
}, {
"id" : 59,
"name" : "score",
"required" : true,
"type" : "int"
} ]
}
}
```
Is it possible to preserve original ids ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] openinx commented on issue #3025: Schema field ids overridden when using nested structs
Posted by GitBox <gi...@apache.org>.
openinx commented on issue #3025:
URL: https://github.com/apache/iceberg/issues/3025#issuecomment-919642056
@dmgcodevil Yes, that's the correct behavior in apache iceberg table metadata, what's the reason that you want to keep the use-defined field id in the persistent iceberg table metadata ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] dmgcodevil commented on issue #3025: Schema field ids overridden when using nested structs
Posted by GitBox <gi...@apache.org>.
dmgcodevil commented on issue #3025:
URL: https://github.com/apache/iceberg/issues/3025#issuecomment-925484647
we have a tool that automatically detects a schema change and executed corresponding schema update operations. Without consistent ids operations such as `rename` can't be automated
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] dmgcodevil edited a comment on issue #3025: Schema field ids overridden when using nested structs
Posted by GitBox <gi...@apache.org>.
dmgcodevil edited a comment on issue #3025:
URL: https://github.com/apache/iceberg/issues/3025#issuecomment-925484647
we have a tool that automatically detects a schema change and executes corresponding schema update operations. Without consistent ids operations such as `rename` can't be automated
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] dmgcodevil commented on issue #3025: Schema field ids overridden when using nested structs
Posted by GitBox <gi...@apache.org>.
dmgcodevil commented on issue #3025:
URL: https://github.com/apache/iceberg/issues/3025#issuecomment-905879411
Look like ids reassigned here https://github.com/apache/iceberg/blob/0b2f0657037c7a5be685ba5f082ba1cc1a145fa1/core/src/main/java/org/apache/iceberg/TableMetadata.java#L98-L99
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org