You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@impala.apache.org by "Quanlong Huang (Code Review)" <ge...@cloudera.org> on 2020/02/10 02:09:19 UTC

[Impala-ASF-CR] IMPALA-9304: Support starting Hive with Ranger in minicluster 

Quanlong Huang has uploaded this change for review. ( http://gerrit.cloudera.org:8080/15189


Change subject: IMPALA-9304: Support starting Hive with Ranger in minicluster
......................................................................

IMPALA-9304: Support starting Hive with Ranger in minicluster

Add a new flag -with_ranger in testdata/bin/run-hive-server.sh to start
Hive with Ranger integration. The relative configuration files are
generated in bin/create-test-configuration.sh using a new varient
ranger_auth in hive-site.xml.py. Only Hive3 is supported.

Current limitation:
Can't use different username in Beeline by the -n option. "select
current_user()" keeps returning my username, while "select
logged_in_user()" can return the username given by -n option but it's
not used in authorization.

Tests:
 - Ran bin/create-test-configuration.sh and verified the generated
   hive-site_ranger_auth.xml contains Ranger configurations.
 - Ran testdata/bin/run-hive-server.sh -with_ranger. Verified column
   masking and row filtering policies took effect in Beeline.

Change-Id: I01e3a195b00a98388244a922a1a79e65146cec42
---
M bin/create-test-configuration.sh
M fe/src/test/resources/hive-site.xml.py
M testdata/bin/run-hive-server.sh
3 files changed, 44 insertions(+), 0 deletions(-)



  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/89/15189/1
-- 
To view, visit http://gerrit.cloudera.org:8080/15189
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I01e3a195b00a98388244a922a1a79e65146cec42
Gerrit-Change-Number: 15189
Gerrit-PatchSet: 1
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>

[Impala-ASF-CR] IMPALA-9304: Support starting Hive with Ranger in minicluster

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15189 )

Change subject: IMPALA-9304: Support starting Hive with Ranger in minicluster
......................................................................


Patch Set 5: Code-Review+2


-- 
To view, visit http://gerrit.cloudera.org:8080/15189
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I01e3a195b00a98388244a922a1a79e65146cec42
Gerrit-Change-Number: 15189
Gerrit-PatchSet: 5
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Comment-Date: Thu, 13 Feb 2020 09:01:53 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-9304: Support starting Hive with Ranger in minicluster

Posted by "Quanlong Huang (Code Review)" <ge...@cloudera.org>.
Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/15189 )

Change subject: IMPALA-9304: Support starting Hive with Ranger in minicluster
......................................................................


Patch Set 3:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/15189/2//COMMIT_MSG
Commit Message:

http://gerrit.cloudera.org:8080/#/c/15189/2//COMMIT_MSG@25
PS2, Line 25: Add
> nit: added
Done



-- 
To view, visit http://gerrit.cloudera.org:8080/15189
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I01e3a195b00a98388244a922a1a79e65146cec42
Gerrit-Change-Number: 15189
Gerrit-PatchSet: 3
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Comment-Date: Wed, 12 Feb 2020 00:25:39 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-9304: Support starting Hive with Ranger in minicluster

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15189 )

Change subject: IMPALA-9304: Support starting Hive with Ranger in minicluster
......................................................................


Patch Set 5:

Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/5331/


-- 
To view, visit http://gerrit.cloudera.org:8080/15189
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I01e3a195b00a98388244a922a1a79e65146cec42
Gerrit-Change-Number: 15189
Gerrit-PatchSet: 5
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Comment-Date: Thu, 13 Feb 2020 21:03:15 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-9304: Support starting Hive with Ranger in minicluster

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15189 )

Change subject: IMPALA-9304: Support starting Hive with Ranger in minicluster
......................................................................


Patch Set 5:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/5331/ DRY_RUN=false


-- 
To view, visit http://gerrit.cloudera.org:8080/15189
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I01e3a195b00a98388244a922a1a79e65146cec42
Gerrit-Change-Number: 15189
Gerrit-PatchSet: 5
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Comment-Date: Thu, 13 Feb 2020 16:11:06 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-9304: Support starting Hive with Ranger in minicluster

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/15189 )

Change subject: IMPALA-9304: Support starting Hive with Ranger in minicluster
......................................................................

IMPALA-9304: Support starting Hive with Ranger in minicluster

Add a new flag -with_ranger in testdata/bin/run-hive-server.sh to start
Hive with Ranger integration. The relative configuration files are
generated in bin/create-test-configuration.sh using a new varient
ranger_auth in hive-site.xml.py. Only Hive3 is supported.

Current limitation:
Can't use different username in Beeline by the -n option. "select
current_user()" keeps returning my username, while "select
logged_in_user()" can return the username given by -n option but it's
not used in authorization.

Tests:
 - Ran bin/create-test-configuration.sh and verified the generated
   hive-site_ranger_auth.xml contains Ranger configurations.
 - Ran testdata/bin/run-hive-server.sh -with_ranger. Verified column
   masking and row filtering policies took effect in Beeline.
 - Added test in test_ranger.py for this mode.

Change-Id: I01e3a195b00a98388244a922a1a79e65146cec42
Reviewed-on: http://gerrit.cloudera.org:8080/15189
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
---
M bin/create-test-configuration.sh
M fe/src/test/resources/hive-site.xml.py
M testdata/bin/run-hive-server.sh
A testdata/workloads/functional-query/queries/QueryTest/hive_ranger_integration.test
M tests/authorization/test_ranger.py
M tests/common/impala_connection.py
M tests/common/skip.py
7 files changed, 93 insertions(+), 5 deletions(-)

Approvals:
  Impala Public Jenkins: Looks good to me, approved; Verified

-- 
To view, visit http://gerrit.cloudera.org:8080/15189
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I01e3a195b00a98388244a922a1a79e65146cec42
Gerrit-Change-Number: 15189
Gerrit-PatchSet: 7
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>

[Impala-ASF-CR] IMPALA-9304: Support starting Hive with Ranger in minicluster

Posted by "Joe McDonnell (Code Review)" <ge...@cloudera.org>.
Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/15189 )

Change subject: IMPALA-9304: Support starting Hive with Ranger in minicluster
......................................................................


Patch Set 3: Code-Review+2

This looks good to me.


-- 
To view, visit http://gerrit.cloudera.org:8080/15189
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I01e3a195b00a98388244a922a1a79e65146cec42
Gerrit-Change-Number: 15189
Gerrit-PatchSet: 3
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Comment-Date: Thu, 13 Feb 2020 00:24:27 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-9304: Support starting Hive with Ranger in minicluster

Posted by "Quanlong Huang (Code Review)" <ge...@cloudera.org>.
Hello Fang-Yu Rao, Joe McDonnell, Csaba Ringhofer, Impala Public Jenkins, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/15189

to look at the new patch set (#2).

Change subject: IMPALA-9304: Support starting Hive with Ranger in minicluster
......................................................................

IMPALA-9304: Support starting Hive with Ranger in minicluster

Add a new flag -with_ranger in testdata/bin/run-hive-server.sh to start
Hive with Ranger integration. The relative configuration files are
generated in bin/create-test-configuration.sh using a new varient
ranger_auth in hive-site.xml.py. Only Hive3 is supported.

Current limitation:
Can't use different username in Beeline by the -n option. "select
current_user()" keeps returning my username, while "select
logged_in_user()" can return the username given by -n option but it's
not used in authorization.

Tests:
 - Ran bin/create-test-configuration.sh and verified the generated
   hive-site_ranger_auth.xml contains Ranger configurations.
 - Ran testdata/bin/run-hive-server.sh -with_ranger. Verified column
   masking and row filtering policies took effect in Beeline.
 - Add test in test_ranger.py for this mode.

Change-Id: I01e3a195b00a98388244a922a1a79e65146cec42
---
M bin/create-test-configuration.sh
M fe/src/test/resources/hive-site.xml.py
M testdata/bin/run-hive-server.sh
A testdata/workloads/functional-query/queries/QueryTest/hive_ranger_integration.test
M tests/authorization/test_ranger.py
M tests/common/impala_connection.py
M tests/common/skip.py
7 files changed, 93 insertions(+), 5 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/89/15189/2
-- 
To view, visit http://gerrit.cloudera.org:8080/15189
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I01e3a195b00a98388244a922a1a79e65146cec42
Gerrit-Change-Number: 15189
Gerrit-PatchSet: 2
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>

[Impala-ASF-CR] IMPALA-9304: Support starting Hive with Ranger in minicluster

Posted by "Quanlong Huang (Code Review)" <ge...@cloudera.org>.
Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/15189 )

Change subject: IMPALA-9304: Support starting Hive with Ranger in minicluster
......................................................................


Patch Set 5:

> Patch Set 5:
> 
> Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/5331/

Failed in a flaky test (IMPALA-9091) again... 
https://jenkins.impala.io/job/ubuntu-16.04-from-scratch/9206


-- 
To view, visit http://gerrit.cloudera.org:8080/15189
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I01e3a195b00a98388244a922a1a79e65146cec42
Gerrit-Change-Number: 15189
Gerrit-PatchSet: 5
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Comment-Date: Thu, 13 Feb 2020 23:33:25 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-9304: Support starting Hive with Ranger in minicluster

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15189 )

Change subject: IMPALA-9304: Support starting Hive with Ranger in minicluster
......................................................................


Patch Set 5:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/5330/ DRY_RUN=false


-- 
To view, visit http://gerrit.cloudera.org:8080/15189
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I01e3a195b00a98388244a922a1a79e65146cec42
Gerrit-Change-Number: 15189
Gerrit-PatchSet: 5
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Comment-Date: Thu, 13 Feb 2020 09:01:54 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-9304: Support starting Hive with Ranger in minicluster

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15189 )

Change subject: IMPALA-9304: Support starting Hive with Ranger in minicluster
......................................................................


Patch Set 2:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/5314/ DRY_RUN=true


-- 
To view, visit http://gerrit.cloudera.org:8080/15189
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I01e3a195b00a98388244a922a1a79e65146cec42
Gerrit-Change-Number: 15189
Gerrit-PatchSet: 2
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Comment-Date: Tue, 11 Feb 2020 03:04:38 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-9304: Support starting Hive with Ranger in minicluster

Posted by "Quanlong Huang (Code Review)" <ge...@cloudera.org>.
Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/15189 )

Change subject: IMPALA-9304: Support starting Hive with Ranger in minicluster
......................................................................


Patch Set 2:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/15189/1//COMMIT_MSG
Commit Message:

http://gerrit.cloudera.org:8080/#/c/15189/1//COMMIT_MSG@20
PS1, Line 20: Tests
> Did you think about adding some kind of custom cluster test to prevent bit 
Good point. Added a test for this.



-- 
To view, visit http://gerrit.cloudera.org:8080/15189
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I01e3a195b00a98388244a922a1a79e65146cec42
Gerrit-Change-Number: 15189
Gerrit-PatchSet: 2
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Comment-Date: Tue, 11 Feb 2020 03:01:18 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-9304: Support starting Hive with Ranger in minicluster

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15189 )

Change subject: IMPALA-9304: Support starting Hive with Ranger in minicluster
......................................................................


Patch Set 2: Verified-1

Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/5314/


-- 
To view, visit http://gerrit.cloudera.org:8080/15189
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I01e3a195b00a98388244a922a1a79e65146cec42
Gerrit-Change-Number: 15189
Gerrit-PatchSet: 2
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Comment-Date: Tue, 11 Feb 2020 07:57:31 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-9304: Support starting Hive with Ranger in minicluster

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15189 )

Change subject: IMPALA-9304: Support starting Hive with Ranger in minicluster
......................................................................


Patch Set 6: Code-Review+2


-- 
To view, visit http://gerrit.cloudera.org:8080/15189
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I01e3a195b00a98388244a922a1a79e65146cec42
Gerrit-Change-Number: 15189
Gerrit-PatchSet: 6
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Comment-Date: Thu, 13 Feb 2020 23:36:29 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-9304: Support starting Hive with Ranger in minicluster

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15189 )

Change subject: IMPALA-9304: Support starting Hive with Ranger in minicluster
......................................................................


Patch Set 1:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/5180/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/15189
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I01e3a195b00a98388244a922a1a79e65146cec42
Gerrit-Change-Number: 15189
Gerrit-PatchSet: 1
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Comment-Date: Mon, 10 Feb 2020 02:54:37 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-9304: Support starting Hive with Ranger in minicluster

Posted by "Csaba Ringhofer (Code Review)" <ge...@cloudera.org>.
Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/15189 )

Change subject: IMPALA-9304: Support starting Hive with Ranger in minicluster
......................................................................


Patch Set 1:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/15189/1//COMMIT_MSG
Commit Message:

http://gerrit.cloudera.org:8080/#/c/15189/1//COMMIT_MSG@20
PS1, Line 20: Tests
Did you think about adding some kind of custom cluster test to prevent bit rot? It would be good to check that updating Ranger/Hive dependencies do not break this.

It could start as a very simple Hive query that needs Ranger, but it may worth to extend it in the future if we need Ranger related interop tests.



-- 
To view, visit http://gerrit.cloudera.org:8080/15189
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I01e3a195b00a98388244a922a1a79e65146cec42
Gerrit-Change-Number: 15189
Gerrit-PatchSet: 1
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Comment-Date: Mon, 10 Feb 2020 12:54:23 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-9304: Support starting Hive with Ranger in minicluster

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15189 )

Change subject: IMPALA-9304: Support starting Hive with Ranger in minicluster
......................................................................


Patch Set 5: Verified-1

Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/5330/


-- 
To view, visit http://gerrit.cloudera.org:8080/15189
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I01e3a195b00a98388244a922a1a79e65146cec42
Gerrit-Change-Number: 15189
Gerrit-PatchSet: 5
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Comment-Date: Thu, 13 Feb 2020 13:50:06 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-9304: Support starting Hive with Ranger in minicluster

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15189 )

Change subject: IMPALA-9304: Support starting Hive with Ranger in minicluster
......................................................................


Patch Set 4: Code-Review+2


-- 
To view, visit http://gerrit.cloudera.org:8080/15189
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I01e3a195b00a98388244a922a1a79e65146cec42
Gerrit-Change-Number: 15189
Gerrit-PatchSet: 4
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Comment-Date: Thu, 13 Feb 2020 00:28:51 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-9304: Support starting Hive with Ranger in minicluster

Posted by "Quanlong Huang (Code Review)" <ge...@cloudera.org>.
Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/15189 )

Change subject: IMPALA-9304: Support starting Hive with Ranger in minicluster
......................................................................


Patch Set 2:

> Patch Set 2: Verified-1
> 
> Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/5314/

The failure is in https://jenkins.impala.io/job/all-build-options-ub1604/5181

 ++ sudo wget -nv https://www-us.apache.org/dist/maven/maven-3/3.5.4/binaries/apache-maven-3.5.4-bin.tar.gz
 failed: Connection timed out.

I think it's not relative.


-- 
To view, visit http://gerrit.cloudera.org:8080/15189
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I01e3a195b00a98388244a922a1a79e65146cec42
Gerrit-Change-Number: 15189
Gerrit-PatchSet: 2
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Comment-Date: Tue, 11 Feb 2020 14:06:41 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-9304: Support starting Hive with Ranger in minicluster

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15189 )

Change subject: IMPALA-9304: Support starting Hive with Ranger in minicluster
......................................................................


Patch Set 6: Verified+1


-- 
To view, visit http://gerrit.cloudera.org:8080/15189
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I01e3a195b00a98388244a922a1a79e65146cec42
Gerrit-Change-Number: 15189
Gerrit-PatchSet: 6
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Comment-Date: Fri, 14 Feb 2020 04:26:14 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-9304: Support starting Hive with Ranger in minicluster

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15189 )

Change subject: IMPALA-9304: Support starting Hive with Ranger in minicluster
......................................................................


Patch Set 4:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/5328/ DRY_RUN=false


-- 
To view, visit http://gerrit.cloudera.org:8080/15189
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I01e3a195b00a98388244a922a1a79e65146cec42
Gerrit-Change-Number: 15189
Gerrit-PatchSet: 4
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Comment-Date: Thu, 13 Feb 2020 00:28:52 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-9304: Support starting Hive with Ranger in minicluster

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15189 )

Change subject: IMPALA-9304: Support starting Hive with Ranger in minicluster
......................................................................


Patch Set 6:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/5337/ DRY_RUN=false


-- 
To view, visit http://gerrit.cloudera.org:8080/15189
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I01e3a195b00a98388244a922a1a79e65146cec42
Gerrit-Change-Number: 15189
Gerrit-PatchSet: 6
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Comment-Date: Thu, 13 Feb 2020 23:36:30 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-9304: Support starting Hive with Ranger in minicluster

Posted by "Csaba Ringhofer (Code Review)" <ge...@cloudera.org>.
Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/15189 )

Change subject: IMPALA-9304: Support starting Hive with Ranger in minicluster
......................................................................


Patch Set 1: Code-Review+1


-- 
To view, visit http://gerrit.cloudera.org:8080/15189
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I01e3a195b00a98388244a922a1a79e65146cec42
Gerrit-Change-Number: 15189
Gerrit-PatchSet: 1
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Comment-Date: Mon, 10 Feb 2020 12:54:28 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-9304: Support starting Hive with Ranger in minicluster

Posted by "Csaba Ringhofer (Code Review)" <ge...@cloudera.org>.
Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/15189 )

Change subject: IMPALA-9304: Support starting Hive with Ranger in minicluster
......................................................................


Patch Set 2: Code-Review+1

(2 comments)

http://gerrit.cloudera.org:8080/#/c/15189/1//COMMIT_MSG
Commit Message:

http://gerrit.cloudera.org:8080/#/c/15189/1//COMMIT_MSG@20
PS1, Line 20: Tests
> Good point. Added a test for this.
Thanks!


http://gerrit.cloudera.org:8080/#/c/15189/2//COMMIT_MSG
Commit Message:

http://gerrit.cloudera.org:8080/#/c/15189/2//COMMIT_MSG@25
PS2, Line 25: Add
nit: added



-- 
To view, visit http://gerrit.cloudera.org:8080/15189
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I01e3a195b00a98388244a922a1a79e65146cec42
Gerrit-Change-Number: 15189
Gerrit-PatchSet: 2
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Comment-Date: Tue, 11 Feb 2020 17:02:08 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-9304: Support starting Hive with Ranger in minicluster

Posted by "Quanlong Huang (Code Review)" <ge...@cloudera.org>.
Hello Fang-Yu Rao, Joe McDonnell, Csaba Ringhofer, Impala Public Jenkins, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/15189

to look at the new patch set (#3).

Change subject: IMPALA-9304: Support starting Hive with Ranger in minicluster
......................................................................

IMPALA-9304: Support starting Hive with Ranger in minicluster

Add a new flag -with_ranger in testdata/bin/run-hive-server.sh to start
Hive with Ranger integration. The relative configuration files are
generated in bin/create-test-configuration.sh using a new varient
ranger_auth in hive-site.xml.py. Only Hive3 is supported.

Current limitation:
Can't use different username in Beeline by the -n option. "select
current_user()" keeps returning my username, while "select
logged_in_user()" can return the username given by -n option but it's
not used in authorization.

Tests:
 - Ran bin/create-test-configuration.sh and verified the generated
   hive-site_ranger_auth.xml contains Ranger configurations.
 - Ran testdata/bin/run-hive-server.sh -with_ranger. Verified column
   masking and row filtering policies took effect in Beeline.
 - Added test in test_ranger.py for this mode.

Change-Id: I01e3a195b00a98388244a922a1a79e65146cec42
---
M bin/create-test-configuration.sh
M fe/src/test/resources/hive-site.xml.py
M testdata/bin/run-hive-server.sh
A testdata/workloads/functional-query/queries/QueryTest/hive_ranger_integration.test
M tests/authorization/test_ranger.py
M tests/common/impala_connection.py
M tests/common/skip.py
7 files changed, 93 insertions(+), 5 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/89/15189/3
-- 
To view, visit http://gerrit.cloudera.org:8080/15189
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I01e3a195b00a98388244a922a1a79e65146cec42
Gerrit-Change-Number: 15189
Gerrit-PatchSet: 3
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>

[Impala-ASF-CR] IMPALA-9304: Support starting Hive with Ranger in minicluster

Posted by "Quanlong Huang (Code Review)" <ge...@cloudera.org>.
Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/15189 )

Change subject: IMPALA-9304: Support starting Hive with Ranger in minicluster
......................................................................


Patch Set 5:

> Patch Set 5: Verified-1
> 
> Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/5330/

Caused by two unrelative failures:
1. https://jenkins.impala.io/job/clang-tidy-ub1604/10263

 ++ sudo wget -nv https://www-us.apache.org/dist/maven/maven-3/3.5.4/binaries/apache-maven-3.5.4-bin.tar.gz
 Unable to establish SSL connection.

2. https://jenkins.impala.io/job/ubuntu-16.04-from-scratch/9199 failed by IMPALA-9152


-- 
To view, visit http://gerrit.cloudera.org:8080/15189
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I01e3a195b00a98388244a922a1a79e65146cec42
Gerrit-Change-Number: 15189
Gerrit-PatchSet: 5
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Comment-Date: Thu, 13 Feb 2020 13:56:40 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-9304: Support starting Hive with Ranger in minicluster

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15189 )

Change subject: IMPALA-9304: Support starting Hive with Ranger in minicluster
......................................................................


Patch Set 2:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/5196/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/15189
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I01e3a195b00a98388244a922a1a79e65146cec42
Gerrit-Change-Number: 15189
Gerrit-PatchSet: 2
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Comment-Date: Tue, 11 Feb 2020 03:47:24 +0000
Gerrit-HasComments: No