You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by "rustyconover (via GitHub)" <gi...@apache.org> on 2023/07/15 17:33:33 UTC
[GitHub] [iceberg] rustyconover opened a new pull request, #8074: Python: Remove MemoryInputStream in favor of io.BytesIO
rustyconover opened a new pull request, #8074:
URL: https://github.com/apache/iceberg/pull/8074
Remove the implementation of MemoryInputStream in favor of the Python built in io.BytesIO.
This will improve efficiency of parsing Avro files when the blocks are entirely in memory.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] Fokko commented on pull request #8074: Python: Remove MemoryInputStream in favor of io.BytesIO
Posted by "Fokko (via GitHub)" <gi...@apache.org>.
Fokko commented on PR #8074:
URL: https://github.com/apache/iceberg/pull/8074#issuecomment-1636873434
> This will improve efficiency of parsing Avro files when the blocks are entirely in memory.
This is great! Avro files we always consume all the way. So we might also want to tune the buffer sizes, or even better, check if there is a file API to read the whole file.
This is great @rustyconover Thanks for working on this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] Fokko merged pull request #8074: Python: Remove MemoryInputStream in favor of io.BytesIO
Posted by "Fokko (via GitHub)" <gi...@apache.org>.
Fokko merged PR #8074:
URL: https://github.com/apache/iceberg/pull/8074
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rustyconover commented on pull request #8074: Python: Remove MemoryInputStream in favor of io.BytesIO
Posted by "rustyconover (via GitHub)" <gi...@apache.org>.
rustyconover commented on PR #8074:
URL: https://github.com/apache/iceberg/pull/8074#issuecomment-1636868418
Done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] Fokko commented on pull request #8074: Python: Remove MemoryInputStream in favor of io.BytesIO
Posted by "Fokko (via GitHub)" <gi...@apache.org>.
Fokko commented on PR #8074:
URL: https://github.com/apache/iceberg/pull/8074#issuecomment-1636857216
@rustyconover This is great! Can you run `pip3 install pre-commit && pre-commit run --all-files`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org