You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by GitBox <gi...@apache.org> on 2020/12/16 19:37:21 UTC
[GitHub] [iceberg] yyanyy opened a new pull request #1943: Spark: update list array copy behavior in StructInternalRow
yyanyy opened a new pull request #1943:
URL: https://github.com/apache/iceberg/pull/1943
I noticed this when I was working on avro metrics support #1935, that [this test case](https://github.com/apache/iceberg/pull/1935/files#diff-81688a1556370355eceb1e39ea9babd927c8ae4a2e6b1ae56743548e0ac833f5R259) will fail with cast exception (unable to cast int to collection). Before this change, I think when `elementType` is of type list, this line result in a second call to `collectionToArrayData` with `elementType` as list when it iterates through each element in the list (within `fillArray` L300), and the cast exception occurs when it tries to cast the element's type to collection for applying `setter.accept`.
I think this would occur when there's a list in a nested structure (e.g. list within map, etc), and I think this doesn't cause problem now since this class is only used for populating metadata tables in spark, and metadata table schema doesn't have such structure.
Currently there's no direct test coverage on this class so I didn't add test to it; this is indirectly tested by various `TestMergingMetrics` and `TestAvroMetricsBounds` test classes (latter introduced in PR mentioned above) though.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rdblue merged pull request #1943: Spark: update list array copy behavior in StructInternalRow
Posted by GitBox <gi...@apache.org>.
rdblue merged pull request #1943:
URL: https://github.com/apache/iceberg/pull/1943
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org