You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hive.apache.org by "Prasanth Jayachandran (JIRA)" <ji...@apache.org> on 2018/03/27 22:19:00 UTC
[jira] [Comment Edited] (HIVE-19033) Provide an option to purge
LLAP IO cache
[ https://issues.apache.org/jira/browse/HIVE-19033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16416312#comment-16416312 ]
Prasanth Jayachandran edited comment on HIVE-19033 at 3/27/18 10:18 PM:
------------------------------------------------------------------------
Based on offline feedback from [~gopalv] added llap specific commands under HiveCommandOperation to get Hive authorization. Also updated the patch to get use LLAP management API for security.
2 commands are add
1) LLAP cluster commands (only info is implemented in this patch)
2) LLAP cache commands (only purge is implemented in this patch)
"llap cluster -info;" has no authorization, any user can read this information.
"llap cache -purge;" requires users to have admin role.
Some examples:
{code:title=user in non-admin role trying to purge the cache}
0: jdbc:hive2://localhost:10000> set hive.security.authorization.enabled;
+-------------------------------------------+
| set |
+-------------------------------------------+
| hive.security.authorization.enabled=true |
+-------------------------------------------+
1 row selected (0.165 seconds)
0: jdbc:hive2://localhost:10000> llap cache -purge;
Error: Error while processing statement: Permission denied: Principal [name=pjayachandran, type=USER] does not have following privileges for operation LLAP_CACHE [[ADMIN PRIVILEGE] on Object [type=COMMAND_PARAMS, name=[-purge]]] (state=,code=1)
{code}
{code:title=user in admin role trying to purge the cache}
0: jdbc:hive2://localhost:10000> set role admin;
No rows affected (1.019 seconds)
0: jdbc:hive2://localhost:10000> llap cache -purge;
+------------+--------------------+
| hostName | purgedMemoryBytes |
+------------+--------------------+
| localhost | 50429952 |
+------------+--------------------+
{code}
{code:title=when authZ is disabled, user can read cluster info}
0: jdbc:hive2://localhost:10000> set hive.security.authorization.enabled;
+--------------------------------------------+
| set |
+--------------------------------------------+
| hive.security.authorization.enabled=false |
+--------------------------------------------+
1 row selected (0.159 seconds)
0: jdbc:hive2://localhost:10000> llap cluster -info;
+----------------+---------------------------------------+------------+----------+-------------+---------+
| applicationId | workerIdentity | hostname | rpcPort | memory | vcores |
+----------------+---------------------------------------+------------+----------+-------------+---------+
| null | 873b7438-01b1-4974-90e2-1c5631602db9 | localhost | 15001 | 3145728000 | 3 |
+----------------+---------------------------------------+------------+----------+-------------+---------+
{code}
ignore "null" applicationId as this is my local setup which is not deployed via slider or yarn services.
was (Author: prasanth_j):
Based on offline feedback from [~gopalv] added llap specific commands under HiveCommandOperation to get Hive authorization. Also updated the patch to get use LLAP management API for security.
2 commands are add
1) LLAP cluster commands (only info is implemented in this patch)
2) LLAP cache commands (only purge is implemented in this patch)
"llap cluster -info;" has no authorization, any user can read this information.
"llap cache -purge;" requires users to have admin role.
Some examples:
{code:title=user in non-admin role trying to purge the cache}
0: jdbc:hive2://localhost:10000> set hive.security.authorization.enabled;
+-------------------------------------------+
| set |
+-------------------------------------------+
| hive.security.authorization.enabled=true |
+-------------------------------------------+
1 row selected (0.165 seconds)
0: jdbc:hive2://localhost:10000> llap cache -purge;
Error: Error while processing statement: Permission denied: Principal [name=pjayachandran, type=USER] does not have following privileges for operation LLAP_CACHE_PURGE [ADMIN PRIVILEGE on INPUT] (state=,code=1)
{code}
{code:title=user in admin role trying to purge the cache}
0: jdbc:hive2://localhost:10000> set role admin;
No rows affected (1.019 seconds)
0: jdbc:hive2://localhost:10000> llap cache -purge;
+------------+--------------------+
| hostName | purgedMemoryBytes |
+------------+--------------------+
| localhost | 50429952 |
+------------+--------------------+
{code}
{code:title=when authZ is disabled, user can read cluster info}
0: jdbc:hive2://localhost:10000> set hive.security.authorization.enabled;
+--------------------------------------------+
| set |
+--------------------------------------------+
| hive.security.authorization.enabled=false |
+--------------------------------------------+
1 row selected (0.159 seconds)
0: jdbc:hive2://localhost:10000> llap cluster -info;
+----------------+---------------------------------------+------------+----------+-------------+---------+
| applicationId | workerIdentity | hostname | rpcPort | memory | vcores |
+----------------+---------------------------------------+------------+----------+-------------+---------+
| null | 873b7438-01b1-4974-90e2-1c5631602db9 | localhost | 15001 | 3145728000 | 3 |
+----------------+---------------------------------------+------------+----------+-------------+---------+
{code}
ignore "null" applicationId as this is my local setup which is not deployed via slider or yarn services.
> Provide an option to purge LLAP IO cache
> ----------------------------------------
>
> Key: HIVE-19033
> URL: https://issues.apache.org/jira/browse/HIVE-19033
> Project: Hive
> Issue Type: Improvement
> Components: llap
> Affects Versions: 3.0.0
> Reporter: Prasanth Jayachandran
> Assignee: Prasanth Jayachandran
> Priority: Major
> Attachments: HIVE-19033.1.patch
>
>
> Provide an API endpoint that will trigger purging of LLAP IO cache. Also CLI tool to invoke the endpoint of all LLAP daemons.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)