You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by "puchengy (via GitHub)" <gi...@apache.org> on 2024/01/04 20:46:48 UTC
[I] [BUG] parquet col array with null value is read in as empty list but should but none [iceberg-python]
puchengy opened a new issue, #251:
URL: https://github.com/apache/iceberg-python/issues/251
### Apache Iceberg version
main (development)
### Please describe the bug 🐞
An Iceberg table with column type array<int> that has null value is read in as empty list, however, it should be none instead.
reproducible scripts: https://github.com/puchengy/iceberg-python/commits/null-in-array/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] [BUG] parquet col array with null value is read in as empty list but should be none [iceberg-python]
Posted by "HonahX (via GitHub)" <gi...@apache.org>.
HonahX closed issue #251: [BUG] parquet col array<int> with null value is read in as empty list but should be none
URL: https://github.com/apache/iceberg-python/issues/251
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] Parquet column array> with null value is read in as empty list [iceberg-python]
Posted by "kevinjqliu (via GitHub)" <gi...@apache.org>.
kevinjqliu commented on issue #251:
URL: https://github.com/apache/iceberg-python/issues/251#issuecomment-2755074686
i believe this issue is already address and we have a test for this https://github.com/apache/iceberg-python/blob/278f7643cd62f9e14496177632cb48d9b52e553d/dev/provision.py#L328
i also tested the example above https://github.com/puchengy/iceberg-python/commit/3fd6d3d3e4b237bda98e40c36bb07e7e4035c2f2
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] Parquet column array> with null value is read in as empty list [iceberg-python]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #251:
URL: https://github.com/apache/iceberg-python/issues/251#issuecomment-2249099707
This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs. To permanently prevent this issue from being considered stale, add the label 'not-stale', but commenting on the issue is preferred when possible.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] [BUG] parquet col array with null value is read in as empty list but should be none [iceberg-python]
Posted by "Fokko (via GitHub)" <gi...@apache.org>.
Fokko commented on issue #251:
URL: https://github.com/apache/iceberg-python/issues/251#issuecomment-1879323375
I've found the issue. We don't respect the null count when fetching the array through the accessor:
<img width="1624" alt="image" src="https://github.com/apache/iceberg-python/assets/1134248/ba5225cc-1102-408f-8926-caf5563930f0">
We just return the array and then create a new array with offset 1, and then it just injects a `[]`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] Parquet column array> with null value is read in as empty list [iceberg-python]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #251:
URL: https://github.com/apache/iceberg-python/issues/251#issuecomment-2606006540
This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs. To permanently prevent this issue from being considered stale, add the label 'not-stale', but commenting on the issue is preferred when possible.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] Parquet column array> with null value is read in as empty list [iceberg-python]
Posted by "HonahX (via GitHub)" <gi...@apache.org>.
HonahX commented on issue #251:
URL: https://github.com/apache/iceberg-python/issues/251#issuecomment-2249538702
Reply to re-activate the issue : )
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] [BUG] parquet col array with null value is read in as empty list but should be none [iceberg-python]
Posted by "HonahX (via GitHub)" <gi...@apache.org>.
HonahX commented on issue #251:
URL: https://github.com/apache/iceberg-python/issues/251#issuecomment-1911626958
This issue isn't completely fixed. We need to wait for an upstream fix: https://github.com/apache/arrow/issues/38809
ref: https://github.com/apache/iceberg-python/pull/252#discussion_r1467065763
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] [BUG] parquet col array with null value is read in as empty list but should be none [iceberg-python]
Posted by "Fokko (via GitHub)" <gi...@apache.org>.
Fokko commented on issue #251:
URL: https://github.com/apache/iceberg-python/issues/251#issuecomment-1878563528
Great catch @puchengy, let me see what's needed to fix this
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] Parquet column array> with null value is read in as empty list [iceberg-python]
Posted by "kevinjqliu (via GitHub)" <gi...@apache.org>.
kevinjqliu closed issue #251: Parquet column array<struct<>> with null value is read in as empty list
URL: https://github.com/apache/iceberg-python/issues/251
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org