You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@impala.apache.org by "Thomas Tauber-Marshall (Code Review)" <ge...@cloudera.org> on 2019/09/19 17:51:28 UTC

[Impala-ASF-CR] IMPALA-4057: Fix --webserver interface 

Thomas Tauber-Marshall has uploaded this change for review. ( http://gerrit.cloudera.org:8080/14266


Change subject: IMPALA-4057: Fix --webserver_interface
......................................................................

IMPALA-4057: Fix --webserver_interface

This patch fixes two issues with --webserver_interface:
- When --webserver_interface was used start-impala-cluster.py with a
  value that's different from --hostname, minicluster startup would
  appear to fail as liveness is determined by checking for the webui's
  availability at the address specified for --hostname.
- The value of --webserver_interface was applied correctly for the
  catalogd and statestored but not for impalads, due to the way
  ExecEnv constructed the Webserver.

This patch also upgrades our version of psutil to the latest for the
function 'net_if_addrs'. This requires a few change to our use of
psutil, mostly adding '()' to call functions that previously were
variables.

Testing:
- Added a custom cluster test that finds all available interfaces,
  binds the webserver to one of them, and checks that its only
  available over that interface.

Change-Id: Ic7e75908426756d73f13a0fa3cfc21fc31da164c
---
M be/src/runtime/exec-env.cc
M be/src/util/webserver-test.cc
M be/src/util/webserver.cc
M be/src/util/webserver.h
M infra/python/deps/compiled-requirements.txt
M tests/common/impala_cluster.py
M tests/common/impala_service.py
M tests/common/impala_test_suite.py
M tests/custom_cluster/test_web_pages.py
9 files changed, 106 insertions(+), 62 deletions(-)



  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/66/14266/1
-- 
To view, visit http://gerrit.cloudera.org:8080/14266
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: Ic7e75908426756d73f13a0fa3cfc21fc31da164c
Gerrit-Change-Number: 14266
Gerrit-PatchSet: 1
Gerrit-Owner: Thomas Tauber-Marshall <tm...@cloudera.com>

[Impala-ASF-CR] IMPALA-4057, IMPALA-4050: Fix --webserver interface

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14266 )

Change subject: IMPALA-4057, IMPALA-4050: Fix --webserver_interface
......................................................................


Patch Set 2:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/4605/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/14266
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic7e75908426756d73f13a0fa3cfc21fc31da164c
Gerrit-Change-Number: 14266
Gerrit-PatchSet: 2
Gerrit-Owner: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Fri, 20 Sep 2019 17:10:58 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-4057: Fix --webserver interface

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14266 )

Change subject: IMPALA-4057: Fix --webserver_interface
......................................................................


Patch Set 1:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/4967/ DRY_RUN=true


-- 
To view, visit http://gerrit.cloudera.org:8080/14266
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic7e75908426756d73f13a0fa3cfc21fc31da164c
Gerrit-Change-Number: 14266
Gerrit-PatchSet: 1
Gerrit-Owner: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Thu, 19 Sep 2019 18:46:28 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-4057, IMPALA-4050: Fix --webserver interface

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/14266 )

Change subject: IMPALA-4057, IMPALA-4050: Fix --webserver_interface
......................................................................

IMPALA-4057, IMPALA-4050: Fix --webserver_interface

This patch fixes two issues with --webserver_interface:
- When --webserver_interface was used start-impala-cluster.py with a
  value that's different from --hostname, minicluster startup would
  appear to fail as liveness is determined by checking for the webui's
  availability at the address specified for --hostname.
- The value of --webserver_interface was applied correctly for the
  catalogd and statestored but not for impalads, due to the way
  ExecEnv constructed the Webserver.
- It is now possible to specify a hostname for webserver_interface
  instead of an IP. The webserver will resolve the hostname.

This patch also upgrades our version of psutil to the latest for the
function 'net_if_addrs'. This requires a few change to our use of
psutil, mostly adding '()' to call functions that previously were
variables.

Testing:
- Added a custom cluster test that finds all available interfaces,
  binds the webserver to one of them, and checks that its only
  available over that interface.

Change-Id: Ic7e75908426756d73f13a0fa3cfc21fc31da164c
Reviewed-on: http://gerrit.cloudera.org:8080/14266
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
---
M be/src/runtime/exec-env.cc
M be/src/util/webserver-test.cc
M be/src/util/webserver.cc
M be/src/util/webserver.h
M infra/python/deps/compiled-requirements.txt
M tests/common/impala_cluster.py
M tests/common/impala_service.py
M tests/common/impala_test_suite.py
M tests/custom_cluster/test_client_ssl.py
M tests/custom_cluster/test_web_pages.py
10 files changed, 112 insertions(+), 64 deletions(-)

Approvals:
  Impala Public Jenkins: Looks good to me, approved; Verified

-- 
To view, visit http://gerrit.cloudera.org:8080/14266
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: Ic7e75908426756d73f13a0fa3cfc21fc31da164c
Gerrit-Change-Number: 14266
Gerrit-PatchSet: 4
Gerrit-Owner: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>

[Impala-ASF-CR] IMPALA-4057, IMPALA-4050: Fix --webserver interface

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14266 )

Change subject: IMPALA-4057, IMPALA-4050: Fix --webserver_interface
......................................................................


Patch Set 3: Verified+1


-- 
To view, visit http://gerrit.cloudera.org:8080/14266
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic7e75908426756d73f13a0fa3cfc21fc31da164c
Gerrit-Change-Number: 14266
Gerrit-PatchSet: 3
Gerrit-Owner: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Fri, 20 Sep 2019 20:43:25 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-4057: Fix --webserver interface

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14266 )

Change subject: IMPALA-4057: Fix --webserver_interface
......................................................................


Patch Set 1: Verified-1

Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/4967/


-- 
To view, visit http://gerrit.cloudera.org:8080/14266
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic7e75908426756d73f13a0fa3cfc21fc31da164c
Gerrit-Change-Number: 14266
Gerrit-PatchSet: 1
Gerrit-Owner: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Thu, 19 Sep 2019 22:55:11 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-4057: Fix --webserver interface

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14266 )

Change subject: IMPALA-4057: Fix --webserver_interface
......................................................................


Patch Set 1:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/4598/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/14266
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic7e75908426756d73f13a0fa3cfc21fc31da164c
Gerrit-Change-Number: 14266
Gerrit-PatchSet: 1
Gerrit-Owner: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Thu, 19 Sep 2019 18:32:53 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-4057, IMPALA-4050: Fix --webserver interface

Posted by "Thomas Tauber-Marshall (Code Review)" <ge...@cloudera.org>.
Thomas Tauber-Marshall has posted comments on this change. ( http://gerrit.cloudera.org:8080/14266 )

Change subject: IMPALA-4057, IMPALA-4050: Fix --webserver_interface
......................................................................


Patch Set 2: Code-Review+2

Fixed an issue with some SSL tests where webserver_interface needs to be specified to match the hostname from the SSL certificate

Carrying forward


-- 
To view, visit http://gerrit.cloudera.org:8080/14266
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic7e75908426756d73f13a0fa3cfc21fc31da164c
Gerrit-Change-Number: 14266
Gerrit-PatchSet: 2
Gerrit-Owner: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Fri, 20 Sep 2019 16:31:45 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-4057, IMPALA-4050: Fix --webserver interface

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14266 )

Change subject: IMPALA-4057, IMPALA-4050: Fix --webserver_interface
......................................................................


Patch Set 3: Code-Review+2


-- 
To view, visit http://gerrit.cloudera.org:8080/14266
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic7e75908426756d73f13a0fa3cfc21fc31da164c
Gerrit-Change-Number: 14266
Gerrit-PatchSet: 3
Gerrit-Owner: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Fri, 20 Sep 2019 16:32:04 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-4057, IMPALA-4050: Fix --webserver interface

Posted by "Thomas Tauber-Marshall (Code Review)" <ge...@cloudera.org>.
Hello Tim Armstrong, Impala Public Jenkins, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/14266

to look at the new patch set (#2).

Change subject: IMPALA-4057, IMPALA-4050: Fix --webserver_interface
......................................................................

IMPALA-4057, IMPALA-4050: Fix --webserver_interface

This patch fixes two issues with --webserver_interface:
- When --webserver_interface was used start-impala-cluster.py with a
  value that's different from --hostname, minicluster startup would
  appear to fail as liveness is determined by checking for the webui's
  availability at the address specified for --hostname.
- The value of --webserver_interface was applied correctly for the
  catalogd and statestored but not for impalads, due to the way
  ExecEnv constructed the Webserver.
- It is now possible to specify a hostname for webserver_interface
  instead of an IP. The webserver will resolve the hostname.

This patch also upgrades our version of psutil to the latest for the
function 'net_if_addrs'. This requires a few change to our use of
psutil, mostly adding '()' to call functions that previously were
variables.

Testing:
- Added a custom cluster test that finds all available interfaces,
  binds the webserver to one of them, and checks that its only
  available over that interface.

Change-Id: Ic7e75908426756d73f13a0fa3cfc21fc31da164c
---
M be/src/runtime/exec-env.cc
M be/src/util/webserver-test.cc
M be/src/util/webserver.cc
M be/src/util/webserver.h
M infra/python/deps/compiled-requirements.txt
M tests/common/impala_cluster.py
M tests/common/impala_service.py
M tests/common/impala_test_suite.py
M tests/custom_cluster/test_client_ssl.py
M tests/custom_cluster/test_web_pages.py
10 files changed, 112 insertions(+), 64 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/66/14266/2
-- 
To view, visit http://gerrit.cloudera.org:8080/14266
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ic7e75908426756d73f13a0fa3cfc21fc31da164c
Gerrit-Change-Number: 14266
Gerrit-PatchSet: 2
Gerrit-Owner: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>

[Impala-ASF-CR] IMPALA-4057: Fix --webserver interface

Posted by "Tim Armstrong (Code Review)" <ge...@cloudera.org>.
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/14266 )

Change subject: IMPALA-4057: Fix --webserver_interface
......................................................................


Patch Set 1: Code-Review+2


-- 
To view, visit http://gerrit.cloudera.org:8080/14266
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic7e75908426756d73f13a0fa3cfc21fc31da164c
Gerrit-Change-Number: 14266
Gerrit-PatchSet: 1
Gerrit-Owner: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Thu, 19 Sep 2019 23:37:23 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-4057, IMPALA-4050: Fix --webserver interface

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14266 )

Change subject: IMPALA-4057, IMPALA-4050: Fix --webserver_interface
......................................................................


Patch Set 3:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/4979/ DRY_RUN=false


-- 
To view, visit http://gerrit.cloudera.org:8080/14266
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic7e75908426756d73f13a0fa3cfc21fc31da164c
Gerrit-Change-Number: 14266
Gerrit-PatchSet: 3
Gerrit-Owner: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Fri, 20 Sep 2019 16:32:05 +0000
Gerrit-HasComments: No