[jira] [Commented] (IMPALA-10694) Improve the error handling in setup-ranger

["ASF subversion and git services (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/IMPALA-10694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17379167#comment-17379167 ] 

ASF subversion and git services commented on IMPALA-10694:
----------------------------------------------------------

Commit 5aa47bf9010278b82f000f0892a6aad0b1383281 in impala's branch refs/heads/master from Fang-Yu Rao
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=5aa47bf ]

IMPALA-10694: Improve the error handling of setup-ranger

We found that setup-ranger will continue the execution if an error
occurs when i) wget was executed to initialize the environment
variables of GROUP_ID_OWNER and GROUP_ID_NON_OWNER, and ii) curl was
executed to upload a revised Ranger policy even though the -e option was
set when create-load-data.sh was executed. This patch improves the error
handling by making setup-ranger exit as soon as an error occurs so that
no test would be run at all in case there is an error.

To exit if an error occurs during wget, we separate the assignment and
the export of the environment variables since the export of an
environment variable will run and succeed even though there is an error
before the export if the assignment and the export are combined. That
is, combining them hides the error.

On the other hand, to exit if an error occurs during curl, we add an
additional -f option so that an error will no longer be silently
ignored.

Testing:
 - Verified that setup-ranger could be successfully executed after this
   patch.
 - Verified that setup-ranger would exit if a URL in setup-ranger is not
   correctly set up or if the 'id' field in policy_4_revised.json does
   not match the URL of the policy to be updated.

Change-Id: I45605d1a7441b734cf80249626638cde3adce28b
Reviewed-on: http://gerrit.cloudera.org:8080/17386
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>


> Improve the error handling in setup-ranger
> ------------------------------------------
>
>                 Key: IMPALA-10694
>                 URL: https://issues.apache.org/jira/browse/IMPALA-10694
>             Project: IMPALA
>          Issue Type: Improvement
>            Reporter: Fang-Yu Rao
>            Assignee: Fang-Yu Rao
>            Priority: Minor
>
> Currently [setup-ranger()|https://github.com/apache/impala/blob/master/testdata/bin/create-load-data.sh#L329-L388] is used to create the Ranger policy repository to perform authorization-related tests. We found that the error handling in {{setup-ranger()}} could be improved.
> For instance, at https://github.com/apache/impala/blob/master/testdata/bin/create-load-data.sh#L385-L387, we update the policy of id equal to 4, which corresponds to the policy of "{{all - database}}". However, it is possible that new default policies are added to the policy repository in newer versions of Ranger. When this occurs, the policy of id equal to 4 may no longer correspond to the policy of "{{all - database}}", resulting in an error returned from the Ranger server. The current implementation of {{setup-ranger()}} would not stop in this case. That is, the pre-commit tests would continue even if the policy repository was not updated correctly, which causes the authorization-related tests to fail later in the pre-commit tests. It would thus be good if we could detect such an error in {{create-load-data.sh}} before any test is run.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org