You are viewing a plain text version of this content. The canonical link for it is here.
Posted to gitbox@hive.apache.org by GitBox <gi...@apache.org> on 2021/01/18 14:42:42 UTC
[GitHub] [hive] klcopp opened a new pull request #1881: HIVE-24652: If compactor worker times out, compaction is not cleared from queue
klcopp opened a new pull request #1881:
URL: https://github.com/apache/hive/pull/1881
### What changes were proposed in this pull request?
### Why are the changes needed?
### Does this PR introduce _any_ user-facing change?
See HIVE-24652
### How was this patch tested?
Unit test
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] klcopp commented on pull request #1881: HIVE-24652: If compactor worker times out, compaction is not cleared from queue
Posted by GitBox <gi...@apache.org>.
klcopp commented on pull request #1881:
URL: https://github.com/apache/hive/pull/1881#issuecomment-762295707
Note: The compaction's txn will stay open if it is open when the Worker times out. But AcidHouseKeeperService should close it when the txn times out.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] pvary commented on a change in pull request #1881: HIVE-24652: If compactor worker times out, compaction is not cleared from queue
Posted by GitBox <gi...@apache.org>.
pvary commented on a change in pull request #1881:
URL: https://github.com/apache/hive/pull/1881#discussion_r560072484
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/Worker.java
##########
@@ -102,11 +102,16 @@ public void run() {
ExecutorService executor = getTimeoutHandlingExecutor();
try {
do {
- Future<Boolean> singleRun = executor.submit(() -> findNextCompactionAndExecute(computeStats));
+ Ref<CompactionInfo> ci = new Ref<>(null);
+ Future<Boolean> singleRun = executor.submit(() -> findNextCompactionAndExecute(computeStats, ci));
Review comment:
Wouldn't it be more natural to have a complex return type, like below? Definitely not a final proposal just some food for thought.
```
private static class ExecutionResult {
private Boolean singleRun;
private CompactionInfo failedCompaction;
}
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] pvargacl commented on pull request #1881: HIVE-24652: If compactor worker times out, compaction is not cleared from queue
Posted by GitBox <gi...@apache.org>.
pvargacl commented on pull request #1881:
URL: https://github.com/apache/hive/pull/1881#issuecomment-762760593
> Note: The compaction's txn will stay open if it is open when the Worker times out. But AcidHouseKeeperService should close it when the txn times out.
The txnId is saved in the compactionQueue when updateCompactorState is called, so you can abort it manually, if you want.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] klcopp commented on pull request #1881: HIVE-24652: If compactor worker times out, compaction is not cleared from queue
Posted by GitBox <gi...@apache.org>.
klcopp commented on pull request #1881:
URL: https://github.com/apache/hive/pull/1881#issuecomment-762879462
Initiator#recoverFailedCompactions already takes care of this.
Thanks for taking a look and pointing this out @pvary !
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] klcopp closed pull request #1881: HIVE-24652: If compactor worker times out, compaction is not cleared from queue
Posted by GitBox <gi...@apache.org>.
klcopp closed pull request #1881:
URL: https://github.com/apache/hive/pull/1881
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org