You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by GitBox <gi...@apache.org> on 2023/01/17 01:11:04 UTC
[GitHub] [iceberg] vapography opened a new pull request, #6605: Fix thread leak in hive-metastore caused by thread pool per HiveTableOperations
vapography opened a new pull request, #6605:
URL: https://github.com/apache/iceberg/pull/6605
Fix thread leak in hive-metastore caused by new thread pool instance created for each HiveTableOperations instance
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] vapography commented on pull request #6605: Fix thread leak in hive-metastore caused by thread pool per HiveTableOperations
Posted by GitBox <gi...@apache.org>.
vapography commented on PR #6605:
URL: https://github.com/apache/iceberg/pull/6605#issuecomment-1385829209
Yes, I have completely missed scenario with multiple hive configurations.
Great that issue is already resolved by #6570. Closing PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] pvary commented on pull request #6605: Fix thread leak in hive-metastore caused by thread pool per HiveTableOperations
Posted by GitBox <gi...@apache.org>.
pvary commented on PR #6605:
URL: https://github.com/apache/iceberg/pull/6605#issuecomment-1386019455
For the record #6570 has not been merged yet.
I was not clear enough, sorry for that
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] vapography closed pull request #6605: Fix thread leak in hive-metastore caused by thread pool per HiveTableOperations
Posted by GitBox <gi...@apache.org>.
vapography closed pull request #6605: Fix thread leak in hive-metastore caused by thread pool per HiveTableOperations
URL: https://github.com/apache/iceberg/pull/6605
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] pvary commented on pull request #6605: Fix thread leak in hive-metastore caused by thread pool per HiveTableOperations
Posted by GitBox <gi...@apache.org>.
pvary commented on PR #6605:
URL: https://github.com/apache/iceberg/pull/6605#issuecomment-1385069134
Funny thing that I have found the same issue when running the spark tests against my PR: #6570.
#6570 would fix this too by recreating the `hiveLockHeartbeatExecutorService` for every Lock (the same way that we were doing for the HiveTableOperations before #6570).
I have some concerns about the proposed solution:
- The size of the threadpool is depends on the configuration of the first executor to call the `initHiveLockHeartbeatScheduledExecutorService` - this would create hard to reproduce situations and it would be hard to understand why the new `iceberg.hive.lock-heartbeat-threadpool-size` is "not used" in some cases
- This creates a JVM wide threadpool, and if the pool is exhausted then the heartbeating of the Lock will not happen, and it is done silently, which can cause issues later
On the plus side, we do not recreate the pool every time when there is a new HiveTableOperations/Lock
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org