You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by "Fokko (via GitHub)" <gi...@apache.org> on 2024/08/07 15:00:09 UTC
[PR] Allow setting `write.parquet.page-row-limit` [iceberg-python]
Fokko opened a new pull request, #1017:
URL: https://github.com/apache/iceberg-python/pull/1017
Noticed this when working on https://github.com/apache/iceberg-python/pull/1016
It is being passed down to PyArrow here:
https://github.com/apache/iceberg-python/blob/8aeab4951080fa196c0d29c72cba1cbba824ffc4/pyiceberg/io/pyarrow.py#L2374-L2377
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Allow setting `write.parquet.page-row-limit` [iceberg-python]
Posted by "Fokko (via GitHub)" <gi...@apache.org>.
Fokko commented on PR #1017:
URL: https://github.com/apache/iceberg-python/pull/1017#issuecomment-2275263994
I tried coming up with a test in two ways:
- Inspecting the Parquet file through PyArrow to see if we can inspect the files, but the low-level page information is not exposed through the Arrow APIs.
- Passing in something bad (like a negative number, but this is all fine with Arrow).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Allow setting `write.parquet.page-row-limit` [iceberg-python]
Posted by "ndrluis (via GitHub)" <gi...@apache.org>.
ndrluis commented on PR #1017:
URL: https://github.com/apache/iceberg-python/pull/1017#issuecomment-2274196689
WDYT about add a test verifying that the configuration from `write.parquet.page-row-limit` is passed down through `write_batch_size`as expected?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Allow setting `write.parquet.page-row-limit` [iceberg-python]
Posted by "Fokko (via GitHub)" <gi...@apache.org>.
Fokko merged PR #1017:
URL: https://github.com/apache/iceberg-python/pull/1017
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org