You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by GitBox <gi...@apache.org> on 2022/11/18 12:52:33 UTC
[GitHub] [iceberg] rbalamohan opened a new issue, #6218: Set COMMIT_MIN_RETRY_WAIT_MS_DEFAULT to 1000 ms instead of 100 ms to avoid too frequent commit exceptions
rbalamohan opened a new issue, #6218:
URL: https://github.com/apache/iceberg/issues/6218
### Apache Iceberg version
_No response_
### Query engine
_No response_
### Please describe the bug 🐞
COMMIT_MIN_RETRY_WAIT_MS_DEFAULT has been set to 100ms. Though it has 4 retries, with exponential backoffs, it mostly would be complete within 1-2 seconds.
https://github.com/apache/iceberg/blob/master/core/src/main/java/org/apache/iceberg/TableProperties.java#L89
For large tables, it can take couple of seconds for commit operation and current default leads to frequent commit failures. Since this is table level property, it can't be set in session level as well causing inconvenience. It will be helpful to set this value to 1000ms instead of 100ms as the default. This should provide roughly 8+ seconds for exponential backoffs, giving better chance for commits.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] github-actions[bot] closed issue #6218: Set COMMIT_MIN_RETRY_WAIT_MS_DEFAULT to 1000 ms instead of 100 ms to avoid too frequent commit exceptions
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed issue #6218: Set COMMIT_MIN_RETRY_WAIT_MS_DEFAULT to 1000 ms instead of 100 ms to avoid too frequent commit exceptions
URL: https://github.com/apache/iceberg/issues/6218
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] github-actions[bot] commented on issue #6218: Set COMMIT_MIN_RETRY_WAIT_MS_DEFAULT to 1000 ms instead of 100 ms to avoid too frequent commit exceptions
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #6218:
URL: https://github.com/apache/iceberg/issues/6218#issuecomment-1556339404
This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs. To permanently prevent this issue from being considered stale, add the label 'not-stale', but commenting on the issue is preferred when possible.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] github-actions[bot] commented on issue #6218: Set COMMIT_MIN_RETRY_WAIT_MS_DEFAULT to 1000 ms instead of 100 ms to avoid too frequent commit exceptions
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #6218:
URL: https://github.com/apache/iceberg/issues/6218#issuecomment-1575841032
This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale'
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] nastra commented on issue #6218: Set COMMIT_MIN_RETRY_WAIT_MS_DEFAULT to 1000 ms instead of 100 ms to avoid too frequent commit exceptions
Posted by GitBox <gi...@apache.org>.
nastra commented on issue #6218:
URL: https://github.com/apache/iceberg/issues/6218#issuecomment-1323848621
I'm not sure we would want to increase the default as that would cause things to generally take longer in the case of retries. For large tables I think it would better to selectively increase `commit.retry.num-retries` / `commit.retry.min-wait-ms` for those particular tables.
I'm curious why commits in your case take several seconds to complete. Do you have lots of small data files that are being tracked in metadata files?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org