You are viewing a plain text version of this content. The canonical link for it is here.
Posted to gitbox@hive.apache.org by GitBox <gi...@apache.org> on 2020/07/13 08:55:22 UTC
[GitHub] [hive] deniskuzZ opened a new pull request #1243: HIVE-23832: Compaction cleaner fails to clean up deltas when using bl…
deniskuzZ opened a new pull request #1243:
URL: https://github.com/apache/hive/pull/1243
…ocking compaction
## NOTICE
Please create an issue in ASF JIRA before opening a pull request,
and you need to set the title of the pull request which starts with
the corresponding JIRA issue number. (e.g. HIVE-XXXXX: Fix a typo in YYY)
For more details, please see https://cwiki.apache.org/confluence/display/Hive/HowToContribute
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] deniskuzZ merged pull request #1243: HIVE-23832: Compaction cleaner fails to clean up deltas when using bl…
Posted by GitBox <gi...@apache.org>.
deniskuzZ merged pull request #1243:
URL: https://github.com/apache/hive/pull/1243
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] laszlopinter86 commented on a change in pull request #1243: HIVE-23832: Compaction cleaner fails to clean up deltas when using bl…
Posted by GitBox <gi...@apache.org>.
laszlopinter86 commented on a change in pull request #1243:
URL: https://github.com/apache/hive/pull/1243#discussion_r453682925
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/ddl/table/storage/compact/AlterTableCompactOperation.java
##########
@@ -99,7 +99,7 @@ private void waitForCompactionToFinish(CompactionResponse resp) throws HiveExcep
wait: while (true) {
//double wait time until 5min
waitTimeMs = waitTimeMs*2;
- waitTimeMs = Math.max(waitTimeMs, waitTimeOut);
+ waitTimeMs = Math.min(waitTimeMs, waitTimeOut);
Review comment:
Why do we need this change? Per the documentation, the minimum wait timeout is 2000 ms
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] laszlopinter86 commented on a change in pull request #1243: HIVE-23832: Compaction cleaner fails to clean up deltas when using bl…
Posted by GitBox <gi...@apache.org>.
laszlopinter86 commented on a change in pull request #1243:
URL: https://github.com/apache/hive/pull/1243#discussion_r453690376
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/ddl/table/storage/compact/AlterTableCompactOperation.java
##########
@@ -99,7 +99,7 @@ private void waitForCompactionToFinish(CompactionResponse resp) throws HiveExcep
wait: while (true) {
//double wait time until 5min
waitTimeMs = waitTimeMs*2;
- waitTimeMs = Math.max(waitTimeMs, waitTimeOut);
+ waitTimeMs = Math.min(waitTimeMs, waitTimeOut);
Review comment:
I've got your point.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] deniskuzZ commented on a change in pull request #1243: HIVE-23832: Compaction cleaner fails to clean up deltas when using bl…
Posted by GitBox <gi...@apache.org>.
deniskuzZ commented on a change in pull request #1243:
URL: https://github.com/apache/hive/pull/1243#discussion_r453686774
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/ddl/table/storage/compact/AlterTableCompactOperation.java
##########
@@ -99,7 +99,7 @@ private void waitForCompactionToFinish(CompactionResponse resp) throws HiveExcep
wait: while (true) {
//double wait time until 5min
waitTimeMs = waitTimeMs*2;
- waitTimeMs = Math.max(waitTimeMs, waitTimeOut);
+ waitTimeMs = Math.min(waitTimeMs, waitTimeOut);
Review comment:
default is 300000ms - 5 min
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] deniskuzZ commented on a change in pull request #1243: HIVE-23832: Compaction cleaner fails to clean up deltas when using bl…
Posted by GitBox <gi...@apache.org>.
deniskuzZ commented on a change in pull request #1243:
URL: https://github.com/apache/hive/pull/1243#discussion_r453686774
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/ddl/table/storage/compact/AlterTableCompactOperation.java
##########
@@ -99,7 +99,7 @@ private void waitForCompactionToFinish(CompactionResponse resp) throws HiveExcep
wait: while (true) {
//double wait time until 5min
waitTimeMs = waitTimeMs*2;
- waitTimeMs = Math.max(waitTimeMs, waitTimeOut);
+ waitTimeMs = Math.min(waitTimeMs, waitTimeOut);
Review comment:
hive.compactor.wait.timeout default is 300000ms (5 min) - that's max,
start wait time = (waitTimeMs = 1000) X 2 = 2000ms;
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] deniskuzZ commented on a change in pull request #1243: HIVE-23832: Compaction cleaner fails to clean up deltas when using bl…
Posted by GitBox <gi...@apache.org>.
deniskuzZ commented on a change in pull request #1243:
URL: https://github.com/apache/hive/pull/1243#discussion_r453686774
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/ddl/table/storage/compact/AlterTableCompactOperation.java
##########
@@ -99,7 +99,7 @@ private void waitForCompactionToFinish(CompactionResponse resp) throws HiveExcep
wait: while (true) {
//double wait time until 5min
waitTimeMs = waitTimeMs*2;
- waitTimeMs = Math.max(waitTimeMs, waitTimeOut);
+ waitTimeMs = Math.min(waitTimeMs, waitTimeOut);
Review comment:
default is 300000ms (5 min) that's max, min = waitTimeMs = 1000 X 2 = 2000ms;
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org