[Impala-ASF-CR] IMPALA-10399, IMPALA-11060, IMPALA-11788: Reset Ranger policy repository in an E2E test

["Fang-Yu Rao (Code Review)" <ge...@cloudera.org>]

Hello Quanlong Huang, Csaba Ringhofer, Michael Smith, Impala Public Jenkins, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/19373

to look at the new patch set (#3).

Change subject: IMPALA-10399, IMPALA-11060, IMPALA-11788: Reset Ranger policy repository in an E2E test
......................................................................

IMPALA-10399, IMPALA-11060, IMPALA-11788: Reset Ranger policy repository in an E2E test

test_show_grant_hive_privilege() uses Ranger's REST API to get all the
existing policies from the Ranger server after creating a policy that
grants the LOCK and SELECT privileges on all the tables and columns in
the unique database in order to verify the granted privileges indeed
exist in Ranger's policy repository.

Before this patch, the way we downloaded all the policies from the
Ranger server in test_show_grant_hive_privilege(), however, did not
always work. Specifically, when there were already a lot of existing
policies in Ranger, the policy that granted the LOCK and SELECT
privileges would not be included in the result returned via one single
GET request. We found that to reproduce the issue it suffices to add 300
Ranger policies before adding the policy granting those 2 privileges.

Moreover, we found that even we set the argument 'stream' of
requests.get() to True and used iter_content() to read the response in
chunks, we still could not retrieve the policy added in
test_show_grant_hive_privilege().

As a workaround, this patch resets Ranger's policy repository for Impala
before we create the policy granting those 2 privileges so that this
test will be more resilient to the number of existing policies in the
repository.

Change-Id: Iff56ec03ceeb2912039241ea302f4bb8948d61f8
---
M testdata/bin/create-load-data.sh
A testdata/bin/setup-ranger.sh
M tests/authorization/test_ranger.py
M tests/common/custom_cluster_test_suite.py
4 files changed, 116 insertions(+), 73 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/73/19373/3
-- 
To view, visit http://gerrit.cloudera.org:8080/19373
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Iff56ec03ceeb2912039241ea302f4bb8948d61f8
Gerrit-Change-Number: 19373
Gerrit-PatchSet: 3
Gerrit-Owner: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>