You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by GitBox <gi...@apache.org> on 2021/07/26 08:28:23 UTC
[GitHub] [iceberg] hameizi opened a new issue #2869: Auto compact small files
hameizi opened a new issue #2869:
URL: https://github.com/apache/iceberg/issues/2869
When we insert data into iceberg will generate much small files, so i try to auto compact small files when we use flink insert into iceberg in https://github.com/apache/iceberg/pull/2867. In this PR, flink opeator compact small files when FlinkFileCommitter commit transaction, and add partition filter when compact files.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] elferherrera commented on issue #2869: Auto compact small files
Posted by GitBox <gi...@apache.org>.
elferherrera commented on issue #2869:
URL: https://github.com/apache/iceberg/issues/2869#issuecomment-937958413
Sorry to hijack your thread, but I can find any information on files after inserting. Is this the expected iceberg behaviour? Every time I write a batch of files it creates a parquet per entry and not one for the whole batch
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] elferherrera commented on issue #2869: Auto compact small files
Posted by GitBox <gi...@apache.org>.
elferherrera commented on issue #2869:
URL: https://github.com/apache/iceberg/issues/2869#issuecomment-937958413
Sorry to hijack your thread, but I can find any information on files after inserting. Is this the expected iceberg behaviour? Every time I write a batch of files it creates a parquet per entry and not one for the whole batch
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org