You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by "rdblue (via GitHub)" <gi...@apache.org> on 2025/04/17 17:26:07 UTC
[PR] API: Use normalized JSON path to identify Variant fields [iceberg]
rdblue opened a new pull request, #12835:
URL: https://github.com/apache/iceberg/pull/12835
This updates the Parquet metrics conversion to use normalized JSON paths instead of joining field names using `.`. A normalized path is a more reliable representation that preserves the distinction between `[a.b][c]`, `[a][b][c]` and `[a.b.c]`.
This implements the changes from #12658. Currently, the spec does not cover how bounds should be stored for variants, but I think we are in agreement about using normalized paths, so I would like to get this in to avoid writing incorrect bounds in the 1.9.0 release.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] API: Use normalized JSON path to identify Variant fields [iceberg]
Posted by "ajantha-bhat (via GitHub)" <gi...@apache.org>.
ajantha-bhat commented on PR #12835:
URL: https://github.com/apache/iceberg/pull/12835#issuecomment-2816245897
Will cherry-pick this to the next RC once merged. cc: @nastra
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] API: Use normalized JSON path to identify Variant fields [iceberg]
Posted by "rdblue (via GitHub)" <gi...@apache.org>.
rdblue commented on code in PR #12835:
URL: https://github.com/apache/iceberg/pull/12835#discussion_r2049461135
##########
api/src/main/java/org/apache/iceberg/expressions/PathUtil.java:
##########
@@ -61,4 +74,59 @@ static List<String> parse(String path) {
return names;
}
+
+ public static String toNormalizedPath(Iterable<String> fields) {
Review Comment:
This has to be public so that the Parquet conversion code can use it to translate a list of field names into a normalized path.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] API: Use normalized JSON path to identify Variant fields [iceberg]
Posted by "amogh-jahagirdar (via GitHub)" <gi...@apache.org>.
amogh-jahagirdar merged PR #12835:
URL: https://github.com/apache/iceberg/pull/12835
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] API: Use normalized JSON path to identify Variant fields [iceberg]
Posted by "amogh-jahagirdar (via GitHub)" <gi...@apache.org>.
amogh-jahagirdar commented on PR #12835:
URL: https://github.com/apache/iceberg/pull/12835#issuecomment-2819521337
thanks @rdblue and thank you @aihuaxu for the spec change for this and for reviewing! I'll go ahead and merge
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org