You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hive.apache.org by Justin Workman <ju...@gmail.com> on 2013/08/16 20:39:33 UTC
Hive-0.11 and NPath
I noticed that NPath appears to be included in Hive-0.11 but I can't
find any documents explaining how to use it. Has anyone successfully
used the NPath module and would be willing to share the correct
syntax. I get parse errors or constructor errors anyway I try and use
it.
Thanks
Justin
Sent from my iPhone
column creation issue/glitch
Posted by si...@bt.com.
Hello,
I did something a bit silly today, but I wonder if it needs fixing?
I lamely named a column "timestamp" - well because it was.. Ok I got lots of complaints when I tried to make it timestamp type, and no problems there, but I finally settled on typing it as a string. But, no matter, because when I wrote a query over it I discovered that the parser was getting very huffy.
Now - I guessed instantly what was going on... this is a reserved word and the query parser can't hack it being plonked in a select query.
I wonder if though HIVE should either :
- refuse to make columns with reserve word names and produce an informative error
- produce an informative error when it finds a reserve word out of place
Any thoughts?
Best
Simon
----
Dr. Simon Thompson
Hive Authorization (ROLES AND PRIVILEGES) does not work with
hiveserver2 ?
Posted by Sanjay Subramanian <Sa...@wizecommerce.com>.
0: jdbc:hive2://dev-thdp5:10000> CREATE ROLE sas_role;
No rows affected (0.16 seconds)
0: jdbc:hive2://dev-thdp5:10000> CREATE EXTERNAL TABLE IF NOT EXISTS keyword_impressions_log (date_ STRING,server STRING,impression_id STRING,search_session_id STRING,channel_id INT,visit_id BIGINT,visitor_id BIGINT,app_style STRING,publisher_id INT,ip STRING,keyword_id BIGINT,keyword STRING,node_constraint BIGINT,mfr_constraint BIGINT,seller_constraint BIGINT,other_constraint STRING,continued INT,offset INT,results INT,sort_ INT,ad_nodes STRING,view_ INT,spelling STRING,referrer STRING,internal_ip INT,cat_feat_mode STRING,rules STRING,rb_filter INT,shuffled INT,related_item_results INT,pixeled INT,sr_attr_imps STRING,unranked_ptitle_ids STRING,perpage INT)PARTITIONED BY (header_date_partition STRING) STORED AS INPUTFORMAT "com.hadoop.mapred.DeprecatedLzoTextInputFormat" OUTPUTFORMAT "org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat";
No rows affected (0.468 seconds)
0: jdbc:hive2://dev-thdp5:10000> alter table keyword_impressions_log add if not exists partition (header_date_partition='2013-08-11') location '/user/hive/warehouse/keyword_impressions_log/2013-08-11';
No rows affected (0.438 seconds)
0: jdbc:hive2://dev-thdp5:10000> GRANT SELECT ON TABLE keyword_impressions_log TO ROLE sas_role;
No rows affected (0.403 seconds)
0: jdbc:hive2://dev-thdp5:10000> GRANT ROLE sas_role TO USER hiveuser1;
No rows affected (0.168 seconds)
0: jdbc:hive2://dev-thdp5:10000> show grant role sas_role on table keyword_impressions_log;
No rows affected (0.117 seconds)
The following CLI command shows the role information
hive -e "show grant role sas_role on table keyword_impressions_log;"
Logging initialized using configuration in jar:file:/usr/lib/hive/lib/hive-common-0.10.0-cdh4.3.0.jar!/hive-log4j.properties
Hive history file=/tmp/sasubramanian/hive_job_log_99244cad-b6e4-4c71-9fe1-d4b248b078c3_1882800333.txt
OK
database default
table keyword_impressions_log
principalName sas_role
principalType ROLE
privilege select
grantTime Mon Aug 19 12:24:08 PDT 2013
grantor hive
Time taken: 1.76 seconds
From: Sanjay Subramanian <sa...@wizecommerce.com>>
Reply-To: "user@hive.apache.org<ma...@hive.apache.org>" <us...@hive.apache.org>>
Date: Friday, August 16, 2013 7:24 PM
To: "user@hive.apache.org<ma...@hive.apache.org>" <us...@hive.apache.org>>
Subject: Hive Authorization clarification
Hi guys
I am not getting the expected result from my authorization settings
I am evaluating Hive0.10.0+121
mysql> select * from hive.ROLES;
+---------+-------------+------------+-----------+
| ROLE_ID | CREATE_TIME | OWNER_NAME | ROLE_NAME |
+---------+-------------+------------+-----------+
| 6 | 1376704610 | NULL | sas_role |
+---------+-------------+------------+-----------+
My settings are as follows.
<!-- added by sanjay -->
<property>
<name>hive.server2.authentication</name>
<value>LDAP</value>
</property>
<property>
<name>hive.server2.authentication.ldap.url</name>
<value>ldap://corp.nextag.com</value>
</property>
<property>
<name>hive.security.authorization.enabled</name>
<value>true</value>
<description>enable or disable the hive client authorization</description>
</property>
<property>
<name>hive.security.authorization.createtable.owner.grants</name>
<value>ALL</value>
<description>the privileges automatically granted to the owner whenever a table gets created.
An example like "select,drop" will grant select and drop privilege to the owner of the table</description>
</property>
<property>
<name>hive.security.authorization.createtable.role.grants</name>
<value>sas_role:select</value>
<description>The privileges automatically granted to some roles whenever a table gets created. An example like "roleX,roleY:select;roleZ:create" will grant select privilege to roleX and roleY, and grant create privilege to roleZ whenever a new table created.</description>
</property>
<property>
<name>hive.security.authorization.createtable.group.grants</name>
<value>hiveuser1:select</value>
<description>The privileges automatically granted to some groups whenever a table gets created. An example like "groupX,groupY:select;groupZ:create" will grant select privilege to groupX and groupY, and grant create privilege to groupZ whenever a new table created.</description>
</property>
<property>
<name>hive.security.authorization.createtable.user.grants</name>
<value>hiveuser1:select</value>
<description>The privileges automatically granted to some users whenever a table gets created. An example like "userX,userY:select;userZ:create" will grant select privilege to userX and userY, and grant create privilege to userZ whenever a new table created.</description>
</property>
USECASE STEPS
============
1. Connect as sasubramanian and create table
beeline> !connect jdbc:hive2://dev-thdp5:10000 sanjay.subramanian@wizecommerce.com<ma...@wizecommerce.com> ********* org.apache.hive.jdbc.HiveDriver
Connecting to jdbc:hive2://dev-thdp5:10000
Connected to: Hive (version 0.10.0)
Driver: Hive (version 0.10.0-cdh4.3.0)
Transaction isolation: TRANSACTION_REPEATABLE_READ
0: jdbc:hive2://dev-thdp5:10000> CREATE EXTERNAL TABLE IF NOT EXISTS keyword_impressions_log (date_ STRING,server STRING,impression_id STRING,search_session_id STRING,channel_id INT,visit_id BIGINT,visitor_id BIGINT,app_style STRING,publisher_id INT,ip STRING,keyword_id BIGINT,keyword STRING,node_constraint BIGINT,mfr_constraint BIGINT,seller_constraint BIGINT,other_constraint STRING,continued INT,offset INT,results INT,sort_ INT,ad_nodes STRING,view_ INT,spelling STRING,referrer STRING,internal_ip INT,cat_feat_mode STRING,rules STRING,rb_filter INT,shuffled INT,related_item_results INT,pixeled INT,sr_attr_imps STRING,unranked_ptitle_ids STRING,perpage INT)PARTITIONED BY (header_date_partition STRING) STORED AS INPUTFORMAT "com.hadoop.mapred.DeprecatedLzoTextInputFormat" OUTPUTFORMAT "org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat";
No rows affected (0.251 seconds)
2. Logout of Hive and connect as hiveuser1
beeline> !connect jdbc:hive2://dev-thdp5:10000 hiveuser1@corp.nextag.com<ma...@corp.nextag.com> W!ze2955 org.apache.hive.jdbc.HiveDriver
Connected to: Hive (version 0.10.0)
Driver: Hive (version 0.10.0-cdh4.3.0)
Transaction isolation: TRANSACTION_REPEATABLE_READ
0: jdbc:hive2://dev-thdp5:10000> drop table keyword_impressions_log;
No rows affected (0.397 seconds)
TABLE GETS DROPPED BY hiveuser1 ????? Why ?
<property>
<name>hive.security.authorization.createtable.group.grants</name>
<value>hiveuser1:select</value>
<description>The privileges automatically granted to some groups whenever a table gets created. An example like "groupX,groupY:select;groupZ:create" will grant select privilege to groupX and groupY, and grant create privilege to groupZ whenever a new table created.</description>
</property>
Thanks
sanjay
CONFIDENTIALITY NOTICE
======================
This email message and any attachments are for the exclusive use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message along with any attachments, from your computer system. If you are the intended recipient, please be advised that the content of this message is subject to access, review and disclosure by the sender's Email System Administrator.
CONFIDENTIALITY NOTICE
======================
This email message and any attachments are for the exclusive use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message along with any attachments, from your computer system. If you are the intended recipient, please be advised that the content of this message is subject to access, review and disclosure by the sender's Email System Administrator.
Hive Authorization clarification
Posted by Sanjay Subramanian <Sa...@wizecommerce.com>.
Hi guys
I am not getting the expected result from my authorization settings
I am evaluating Hive0.10.0+121
mysql> select * from hive.ROLES;
+---------+-------------+------------+-----------+
| ROLE_ID | CREATE_TIME | OWNER_NAME | ROLE_NAME |
+---------+-------------+------------+-----------+
| 6 | 1376704610 | NULL | sas_role |
+---------+-------------+------------+-----------+
My settings are as follows.
<!-- added by sanjay -->
<property>
<name>hive.server2.authentication</name>
<value>LDAP</value>
</property>
<property>
<name>hive.server2.authentication.ldap.url</name>
<value>ldap://corp.nextag.com</value>
</property>
<property>
<name>hive.security.authorization.enabled</name>
<value>true</value>
<description>enable or disable the hive client authorization</description>
</property>
<property>
<name>hive.security.authorization.createtable.owner.grants</name>
<value>ALL</value>
<description>the privileges automatically granted to the owner whenever a table gets created.
An example like "select,drop" will grant select and drop privilege to the owner of the table</description>
</property>
<property>
<name>hive.security.authorization.createtable.role.grants</name>
<value>sas_role:select</value>
<description>The privileges automatically granted to some roles whenever a table gets created. An example like "roleX,roleY:select;roleZ:create" will grant select privilege to roleX and roleY, and grant create privilege to roleZ whenever a new table created.</description>
</property>
<property>
<name>hive.security.authorization.createtable.group.grants</name>
<value>hiveuser1:select</value>
<description>The privileges automatically granted to some groups whenever a table gets created. An example like "groupX,groupY:select;groupZ:create" will grant select privilege to groupX and groupY, and grant create privilege to groupZ whenever a new table created.</description>
</property>
<property>
<name>hive.security.authorization.createtable.user.grants</name>
<value>hiveuser1:select</value>
<description>The privileges automatically granted to some users whenever a table gets created. An example like "userX,userY:select;userZ:create" will grant select privilege to userX and userY, and grant create privilege to userZ whenever a new table created.</description>
</property>
USECASE STEPS
============
1. Connect as sasubramanian and create table
beeline> !connect jdbc:hive2://dev-thdp5:10000 sanjay.subramanian@wizecommerce.com ********* org.apache.hive.jdbc.HiveDriver
Connecting to jdbc:hive2://dev-thdp5:10000
Connected to: Hive (version 0.10.0)
Driver: Hive (version 0.10.0-cdh4.3.0)
Transaction isolation: TRANSACTION_REPEATABLE_READ
0: jdbc:hive2://dev-thdp5:10000> CREATE EXTERNAL TABLE IF NOT EXISTS keyword_impressions_log (date_ STRING,server STRING,impression_id STRING,search_session_id STRING,channel_id INT,visit_id BIGINT,visitor_id BIGINT,app_style STRING,publisher_id INT,ip STRING,keyword_id BIGINT,keyword STRING,node_constraint BIGINT,mfr_constraint BIGINT,seller_constraint BIGINT,other_constraint STRING,continued INT,offset INT,results INT,sort_ INT,ad_nodes STRING,view_ INT,spelling STRING,referrer STRING,internal_ip INT,cat_feat_mode STRING,rules STRING,rb_filter INT,shuffled INT,related_item_results INT,pixeled INT,sr_attr_imps STRING,unranked_ptitle_ids STRING,perpage INT)PARTITIONED BY (header_date_partition STRING) STORED AS INPUTFORMAT "com.hadoop.mapred.DeprecatedLzoTextInputFormat" OUTPUTFORMAT "org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat";
No rows affected (0.251 seconds)
2. Logout of Hive and connect as hiveuser1
beeline> !connect jdbc:hive2://dev-thdp5:10000 hiveuser1@corp.nextag.com W!ze2955 org.apache.hive.jdbc.HiveDriver
Connected to: Hive (version 0.10.0)
Driver: Hive (version 0.10.0-cdh4.3.0)
Transaction isolation: TRANSACTION_REPEATABLE_READ
0: jdbc:hive2://dev-thdp5:10000> drop table keyword_impressions_log;
No rows affected (0.397 seconds)
TABLE GETS DROPPED BY hiveuser1 ????? Why ?
<property>
<name>hive.security.authorization.createtable.group.grants</name>
<value>hiveuser1:select</value>
<description>The privileges automatically granted to some groups whenever a table gets created. An example like "groupX,groupY:select;groupZ:create" will grant select privilege to groupX and groupY, and grant create privilege to groupZ whenever a new table created.</description>
</property>
Thanks
sanjay
CONFIDENTIALITY NOTICE
======================
This email message and any attachments are for the exclusive use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message along with any attachments, from your computer system. If you are the intended recipient, please be advised that the content of this message is subject to access, review and disclosure by the sender's Email System Administrator.
Re: SHOW ALL ROLES
Posted by Sanjay Subramanian <Sa...@wizecommerce.com>.
Ok never mind , this will work just fine for me
mysql> select ROLE_NAME from ROLES;
From: Sanjay Subramanian <sa...@wizecommerce.com>>
Reply-To: "user@hive.apache.org<ma...@hive.apache.org>" <us...@hive.apache.org>>
Date: Friday, August 16, 2013 6:34 PM
To: "user@hive.apache.org<ma...@hive.apache.org>" <us...@hive.apache.org>>
Subject: SHOW ALL ROLES
Hi
How do I display all ROLES defined in hive thru CLI ?
Thanks
sanjay
CONFIDENTIALITY NOTICE
======================
This email message and any attachments are for the exclusive use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message along with any attachments, from your computer system. If you are the intended recipient, please be advised that the content of this message is subject to access, review and disclosure by the sender's Email System Administrator.
CONFIDENTIALITY NOTICE
======================
This email message and any attachments are for the exclusive use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message along with any attachments, from your computer system. If you are the intended recipient, please be advised that the content of this message is subject to access, review and disclosure by the sender's Email System Administrator.
SHOW ALL ROLES
Posted by Sanjay Subramanian <Sa...@wizecommerce.com>.
Hi
How do I display all ROLES defined in hive thru CLI ?
Thanks
sanjay
CONFIDENTIALITY NOTICE
======================
This email message and any attachments are for the exclusive use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message along with any attachments, from your computer system. If you are the intended recipient, please be advised that the content of this message is subject to access, review and disclosure by the sender's Email System Administrator.
Re: Hive-0.11 and NPath
Posted by Justin Workman <ju...@gmail.com>.
Very helpful. I now have it working, to a degree. I have a few other
questions but I will post them in another thread.
On Fri, Aug 16, 2013 at 1:18 PM, Edward Capriolo <ed...@gmail.com>wrote:
> Look in the .q files that are in the hive source. Be aware that npath is
> going to be renamed soon https://issues.apache.org/jira/browse/HIVE-5087 .
>
>
> On Fri, Aug 16, 2013 at 3:07 PM, Justin Workman <ju...@gmail.com>wrote:
>
>> That is the syntax I started with, and it does not like it. I get
>> somewhat closer using the aster data type syntax with out using the PATTERN
>> or SYMBOL keywords.
>>
>> Sent from my iPhone
>>
>> On Aug 16, 2013, at 12:58 PM, Nitin Pawar <ni...@gmail.com>
>> wrote:
>>
>> see if the one mentioned on this link works for you
>>
>> https://github.com/hbutani/SQLWindowing/wiki
>>
>>
>> On Sat, Aug 17, 2013 at 12:09 AM, Justin Workman <
>> justinjworkman@gmail.com> wrote:
>>
>>> I noticed that NPath appears to be included in Hive-0.11 but I can't
>>> find any documents explaining how to use it. Has anyone successfully
>>> used the NPath module and would be willing to share the correct
>>> syntax. I get parse errors or constructor errors anyway I try and use
>>> it.
>>>
>>> Thanks
>>> Justin
>>>
>>> Sent from my iPhone
>>>
>>
>>
>>
>> --
>> Nitin Pawar
>>
>>
>
Re: Hive-0.11 and NPath
Posted by Edward Capriolo <ed...@gmail.com>.
Look in the .q files that are in the hive source. Be aware that npath is
going to be renamed soon https://issues.apache.org/jira/browse/HIVE-5087 .
On Fri, Aug 16, 2013 at 3:07 PM, Justin Workman <ju...@gmail.com>wrote:
> That is the syntax I started with, and it does not like it. I get somewhat
> closer using the aster data type syntax with out using the PATTERN or
> SYMBOL keywords.
>
> Sent from my iPhone
>
> On Aug 16, 2013, at 12:58 PM, Nitin Pawar <ni...@gmail.com> wrote:
>
> see if the one mentioned on this link works for you
>
> https://github.com/hbutani/SQLWindowing/wiki
>
>
> On Sat, Aug 17, 2013 at 12:09 AM, Justin Workman <justinjworkman@gmail.com
> > wrote:
>
>> I noticed that NPath appears to be included in Hive-0.11 but I can't
>> find any documents explaining how to use it. Has anyone successfully
>> used the NPath module and would be willing to share the correct
>> syntax. I get parse errors or constructor errors anyway I try and use
>> it.
>>
>> Thanks
>> Justin
>>
>> Sent from my iPhone
>>
>
>
>
> --
> Nitin Pawar
>
>
Re: Hive-0.11 and NPath
Posted by Justin Workman <ju...@gmail.com>.
That is the syntax I started with, and it does not like it. I get somewhat
closer using the aster data type syntax with out using the PATTERN or
SYMBOL keywords.
Sent from my iPhone
On Aug 16, 2013, at 12:58 PM, Nitin Pawar <ni...@gmail.com> wrote:
see if the one mentioned on this link works for you
https://github.com/hbutani/SQLWindowing/wiki
On Sat, Aug 17, 2013 at 12:09 AM, Justin Workman
<ju...@gmail.com>wrote:
> I noticed that NPath appears to be included in Hive-0.11 but I can't
> find any documents explaining how to use it. Has anyone successfully
> used the NPath module and would be willing to share the correct
> syntax. I get parse errors or constructor errors anyway I try and use
> it.
>
> Thanks
> Justin
>
> Sent from my iPhone
>
--
Nitin Pawar
Re: Hive-0.11 and NPath
Posted by Nitin Pawar <ni...@gmail.com>.
see if the one mentioned on this link works for you
https://github.com/hbutani/SQLWindowing/wiki
On Sat, Aug 17, 2013 at 12:09 AM, Justin Workman
<ju...@gmail.com>wrote:
> I noticed that NPath appears to be included in Hive-0.11 but I can't
> find any documents explaining how to use it. Has anyone successfully
> used the NPath module and would be willing to share the correct
> syntax. I get parse errors or constructor errors anyway I try and use
> it.
>
> Thanks
> Justin
>
> Sent from my iPhone
>
--
Nitin Pawar