You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@impala.apache.org by "Joe McDonnell (Code Review)" <ge...@cloudera.org> on 2019/11/22 19:22:46 UTC
[Impala-ASF-CR] IMPALA-9165: Add back hard kill to kill-hbase.sh
Joe McDonnell has uploaded this change for review. ( http://gerrit.cloudera.org:8080/14789
Change subject: IMPALA-9165: Add back hard kill to kill-hbase.sh
......................................................................
IMPALA-9165: Add back hard kill to kill-hbase.sh
The fix for IMPALA-9150 changed kill-hbase.sh to use HBase's
stop-hbase.sh script. Around this time, the GVO timeout issues
started. GVO can reuse machines, so we don't know what state
they may be in. If something failed to kill HBase processes,
the next job would need to be able to kill them even without
access to the last run's files / logs.
This restores the original kill logic to kill-hbase.sh, after
trying a graceful shutdown using HBase's stop-hbase.sh script.
The original kill logic doesn't rely on anything from the
filesystem to know about the existence of processes, so it
would handle machine reuse.
Testing:
- Started with a running minicluster, ran bin/clean.sh,
then ran testdata/bin/kill-all.sh and verified that the
java processes were gone
Change-Id: Ie2f0b342bcd1d8abea8ef923adbb54a14518a7a6
---
M testdata/bin/kill-hbase.sh
1 file changed, 6 insertions(+), 2 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/89/14789/1
--
To view, visit http://gerrit.cloudera.org:8080/14789
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: Ie2f0b342bcd1d8abea8ef923adbb54a14518a7a6
Gerrit-Change-Number: 14789
Gerrit-PatchSet: 1
Gerrit-Owner: Joe McDonnell <jo...@cloudera.com>
[Impala-ASF-CR] IMPALA-9165: Add back hard kill to kill-hbase.sh
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14789 )
Change subject: IMPALA-9165: Add back hard kill to kill-hbase.sh
......................................................................
Patch Set 3: Verified+1
--
To view, visit http://gerrit.cloudera.org:8080/14789
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie2f0b342bcd1d8abea8ef923adbb54a14518a7a6
Gerrit-Change-Number: 14789
Gerrit-PatchSet: 3
Gerrit-Owner: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Lars Volker <lv...@cloudera.com>
Gerrit-Comment-Date: Sat, 23 Nov 2019 00:15:25 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9165: Add back hard kill to kill-hbase.sh
Posted by "Lars Volker (Code Review)" <ge...@cloudera.org>.
Lars Volker has posted comments on this change. ( http://gerrit.cloudera.org:8080/14789 )
Change subject: IMPALA-9165: Add back hard kill to kill-hbase.sh
......................................................................
Patch Set 4: Code-Review+2
--
To view, visit http://gerrit.cloudera.org:8080/14789
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie2f0b342bcd1d8abea8ef923adbb54a14518a7a6
Gerrit-Change-Number: 14789
Gerrit-PatchSet: 4
Gerrit-Owner: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Lars Volker <lv...@cloudera.com>
Gerrit-Comment-Date: Fri, 22 Nov 2019 21:45:14 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9165: Add back hard kill to kill-hbase.sh
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14789 )
Change subject: IMPALA-9165: Add back hard kill to kill-hbase.sh
......................................................................
Patch Set 1:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/5121/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/14789
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie2f0b342bcd1d8abea8ef923adbb54a14518a7a6
Gerrit-Change-Number: 14789
Gerrit-PatchSet: 1
Gerrit-Owner: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Fri, 22 Nov 2019 20:06:05 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9165: Add back hard kill to kill-hbase.sh
Posted by "Joe McDonnell (Code Review)" <ge...@cloudera.org>.
Hello Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/14789
to look at the new patch set (#3).
Change subject: IMPALA-9165: Add back hard kill to kill-hbase.sh
......................................................................
IMPALA-9165: Add back hard kill to kill-hbase.sh
The fix for IMPALA-9150 changed kill-hbase.sh to use HBase's
stop-hbase.sh script. Around this time, the GVO timeout issues
started. GVO can reuse machines, so we don't know what state
they may be in. If something failed to kill HBase processes,
the next job would need to be able to kill them even without
access to the last run's files / logs.
This restores the original kill logic to kill-hbase.sh, after
trying a graceful shutdown using HBase's stop-hbase.sh script.
The original kill logic doesn't rely on anything from the
filesystem to know about the existence of processes, so it
would handle machine reuse.
This also changes our Jenkins test scripts to shut down the
minicluster at the end.
Testing:
- Started with a running minicluster, ran bin/clean.sh,
then ran testdata/bin/kill-all.sh and verified that the
java processes were gone
Change-Id: Ie2f0b342bcd1d8abea8ef923adbb54a14518a7a6
---
M bin/jenkins/all-tests.sh
M bin/jenkins/dockerized-impala-run-tests.sh
M testdata/bin/kill-hbase.sh
3 files changed, 12 insertions(+), 2 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/89/14789/3
--
To view, visit http://gerrit.cloudera.org:8080/14789
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ie2f0b342bcd1d8abea8ef923adbb54a14518a7a6
Gerrit-Change-Number: 14789
Gerrit-PatchSet: 3
Gerrit-Owner: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
[Impala-ASF-CR] IMPALA-9165: Add back hard kill to kill-hbase.sh
Posted by "Joe McDonnell (Code Review)" <ge...@cloudera.org>.
Hello Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/14789
to look at the new patch set (#4).
Change subject: IMPALA-9165: Add back hard kill to kill-hbase.sh
......................................................................
IMPALA-9165: Add back hard kill to kill-hbase.sh
The fix for IMPALA-9150 changed kill-hbase.sh to use HBase's
stop-hbase.sh script. Around this time, the GVO timeout issues
started. GVO can reuse machines, so we don't know what state
they may be in. If something failed to kill HBase processes,
the next job would need to be able to kill them even without
access to the last run's files / logs.
This restores the original kill logic to kill-hbase.sh, after
trying a graceful shutdown using HBase's stop-hbase.sh script.
The original kill logic doesn't rely on anything from the
filesystem to know about the existence of processes, so it
would handle machine reuse.
This also changes our Jenkins test scripts to shut down the
minicluster at the end.
Testing:
- Started with a running minicluster, ran bin/clean.sh,
then ran testdata/bin/kill-all.sh and verified that the
java processes were gone
Change-Id: Ie2f0b342bcd1d8abea8ef923adbb54a14518a7a6
---
M bin/jenkins/all-tests.sh
M bin/jenkins/dockerized-impala-run-tests.sh
M testdata/bin/kill-hbase.sh
3 files changed, 18 insertions(+), 3 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/89/14789/4
--
To view, visit http://gerrit.cloudera.org:8080/14789
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ie2f0b342bcd1d8abea8ef923adbb54a14518a7a6
Gerrit-Change-Number: 14789
Gerrit-PatchSet: 4
Gerrit-Owner: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
[Impala-ASF-CR] IMPALA-9165: Add back hard kill to kill-hbase.sh
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14789 )
Change subject: IMPALA-9165: Add back hard kill to kill-hbase.sh
......................................................................
Patch Set 5: Verified+1
--
To view, visit http://gerrit.cloudera.org:8080/14789
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie2f0b342bcd1d8abea8ef923adbb54a14518a7a6
Gerrit-Change-Number: 14789
Gerrit-PatchSet: 5
Gerrit-Owner: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Lars Volker <lv...@cloudera.com>
Gerrit-Comment-Date: Sat, 23 Nov 2019 02:42:23 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9165: Add back hard kill to kill-hbase.sh
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14789 )
Change subject: IMPALA-9165: Add back hard kill to kill-hbase.sh
......................................................................
Patch Set 5: Code-Review+2
--
To view, visit http://gerrit.cloudera.org:8080/14789
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie2f0b342bcd1d8abea8ef923adbb54a14518a7a6
Gerrit-Change-Number: 14789
Gerrit-PatchSet: 5
Gerrit-Owner: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Lars Volker <lv...@cloudera.com>
Gerrit-Comment-Date: Fri, 22 Nov 2019 22:00:54 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9165: Add back hard kill to kill-hbase.sh
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14789 )
Change subject: IMPALA-9165: Add back hard kill to kill-hbase.sh
......................................................................
Patch Set 4:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/5123/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/14789
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie2f0b342bcd1d8abea8ef923adbb54a14518a7a6
Gerrit-Change-Number: 14789
Gerrit-PatchSet: 4
Gerrit-Owner: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Fri, 22 Nov 2019 20:24:36 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9165: Add back hard kill to kill-hbase.sh
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14789 )
Change subject: IMPALA-9165: Add back hard kill to kill-hbase.sh
......................................................................
Patch Set 3:
Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/5282/ DRY_RUN=true
--
To view, visit http://gerrit.cloudera.org:8080/14789
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie2f0b342bcd1d8abea8ef923adbb54a14518a7a6
Gerrit-Change-Number: 14789
Gerrit-PatchSet: 3
Gerrit-Owner: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Fri, 22 Nov 2019 19:32:53 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9165: Add back hard kill to kill-hbase.sh
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/14789 )
Change subject: IMPALA-9165: Add back hard kill to kill-hbase.sh
......................................................................
IMPALA-9165: Add back hard kill to kill-hbase.sh
The fix for IMPALA-9150 changed kill-hbase.sh to use HBase's
stop-hbase.sh script. Around this time, the GVO timeout issues
started. GVO can reuse machines, so we don't know what state
they may be in. If something failed to kill HBase processes,
the next job would need to be able to kill them even without
access to the last run's files / logs.
This restores the original kill logic to kill-hbase.sh, after
trying a graceful shutdown using HBase's stop-hbase.sh script.
The original kill logic doesn't rely on anything from the
filesystem to know about the existence of processes, so it
would handle machine reuse.
This also changes our Jenkins test scripts to shut down the
minicluster at the end.
Testing:
- Started with a running minicluster, ran bin/clean.sh,
then ran testdata/bin/kill-all.sh and verified that the
java processes were gone
Change-Id: Ie2f0b342bcd1d8abea8ef923adbb54a14518a7a6
Reviewed-on: http://gerrit.cloudera.org:8080/14789
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
---
M bin/jenkins/all-tests.sh
M bin/jenkins/dockerized-impala-run-tests.sh
M testdata/bin/kill-hbase.sh
3 files changed, 18 insertions(+), 3 deletions(-)
Approvals:
Impala Public Jenkins: Looks good to me, approved; Verified
--
To view, visit http://gerrit.cloudera.org:8080/14789
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: Ie2f0b342bcd1d8abea8ef923adbb54a14518a7a6
Gerrit-Change-Number: 14789
Gerrit-PatchSet: 6
Gerrit-Owner: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Lars Volker <lv...@cloudera.com>
[Impala-ASF-CR] IMPALA-9165: Add back hard kill to kill-hbase.sh
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14789 )
Change subject: IMPALA-9165: Add back hard kill to kill-hbase.sh
......................................................................
Patch Set 5:
Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/5285/ DRY_RUN=false
--
To view, visit http://gerrit.cloudera.org:8080/14789
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie2f0b342bcd1d8abea8ef923adbb54a14518a7a6
Gerrit-Change-Number: 14789
Gerrit-PatchSet: 5
Gerrit-Owner: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Lars Volker <lv...@cloudera.com>
Gerrit-Comment-Date: Fri, 22 Nov 2019 22:00:55 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9165: Add back hard kill to kill-hbase.sh
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14789 )
Change subject: IMPALA-9165: Add back hard kill to kill-hbase.sh
......................................................................
Patch Set 3:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/5122/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/14789
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie2f0b342bcd1d8abea8ef923adbb54a14518a7a6
Gerrit-Change-Number: 14789
Gerrit-PatchSet: 3
Gerrit-Owner: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Fri, 22 Nov 2019 20:16:03 +0000
Gerrit-HasComments: No