You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by GitBox <gi...@apache.org> on 2020/10/26 07:35:59 UTC
[GitHub] [iceberg] zhangjun0x01 opened a new issue #1660: Add max parallelism for RewriteDataFilesAction
zhangjun0x01 opened a new issue #1660:
URL: https://github.com/apache/iceberg/issues/1660
The current default parallelism of RewriteDataFilesAction is the number of CombinedScanTask tasks, but if there is a lot of data in the partition, the task size will be very large, which will occupy a lot of system resources and affect the running tasks. I think we should set The max parallelism of RewriteDataFilesAction to avoid taking up too much system resources
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] zhangjun0x01 closed issue #1660: Add max parallelism for RewriteDataFilesAction
Posted by GitBox <gi...@apache.org>.
zhangjun0x01 closed issue #1660:
URL: https://github.com/apache/iceberg/issues/1660
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] zhangjun0x01 commented on issue #1660: Add max parallelism for RewriteDataFilesAction
Posted by GitBox <gi...@apache.org>.
zhangjun0x01 commented on issue #1660:
URL: https://github.com/apache/iceberg/issues/1660#issuecomment-722065664
we had add the max parallelism on #1623 ,close this issue
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org