You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@sentry.apache.org by Anurag <at...@groupon.com> on 2014/06/05 16:43:33 UTC

Re: Apache Sentry works with hive metastore ?

Any response anyone?

Sent from my iPhone
> On Jun 4, 2014, at 3:20 PM, Anurag Tangri <at...@groupon.com> wrote:
> 
> + sentry mailing list
> 
> 
>> On Wed, Jun 4, 2014 at 12:27 AM, Anurag Tangri <at...@groupon.com> wrote:
>> So, I am working on setting this up.
>> 
>> I have HDFS integrated with LDAP and working fine.
>> 
>> I also added ldap params in hive-site along with sentry conf :
>> 
>> ====
>>   <property>
>>     <name>hive.server2.authentication</name>
>>     <value>LDAP</value>
>>   </property>
>>   <property>
>>     <name>hive.server2.authentication.ldap.url</name>
>>     <value>ldap://<ip></value>
>>   </property>
>> 
>> <property>
>>   <name>hive.server2.authentication.ldap.baseDN</name>
>>   <value>dc=a,dc=b</value>
>> </property>
>> 
>> ======
>> 
>> Now when I go to beeline, I type:
>> 
>> /usr/local/lib/hive/bin/beeline -u jdbc:hive2://<host>:10000
>> Beeline version 0.10.0-cdh4.6.0 by Apache Hive
>> 0: jdbc:hive2://<host>:10000> !connect jdbc:hive2://<host>:10000
>> 
>> 
>> It asks for username and password. I give LDAP credentials as:
>> 
>> Connecting to jdbc:hive2://<host>:10000
>> Enter username for jdbc:hive2://<host>:10000: atangri@a.b
>> Enter password for jdbc:hive2://<host>:10000: **********
>> Error: Invalid URL: jdbc:hive2://<host>:10000 (state=08S01,code=0)
>> 1: jdbc:hive2://<host>:10000> 
>> 
>> 
>> Now if I type, hive queries, i see error in hive log:
>> 
>> 2014-06-04 07:15:13,211 WARN  file.HadoopGroupMappingService (HadoopGroupMappingService.java:getGroups(42)) - Unable to obtain groups for atangri@a.b
>> java.io.IOException: No groups found for user atangri@a.b
>>     at org.apache.hadoop.security.Groups.getGroups(Groups.java:105)
>>     at org.apache.sentry.provider.file.HadoopGroupMappingService.getGroups(HadoopGroupMappingService.java:40)
>>     at org.apache.sentry.provider.file.ResourceAuthorizationProvider.doHasAccess(ResourceAuthorizationProvider.java:98)
>>     at org.apache.sentry.provider.file.ResourceAuthorizationProvider.hasAccess(ResourceAuthorizationProvider.java:93)
>>     at org.apache.sentry.binding.hive.authz.HiveAuthzBinding.authorize(HiveAuthzBinding.java:179)
>>     at org.apache.sentry.binding.hive.HiveAuthzBindingHook.filterShowTables(HiveAuthzBindingHook.java:561)
>>     at org.apache.sentry.binding.hive.HiveAuthzBindingHook.postDriverFetch(HiveAuthzBindingHook.java:634)
>>     at org.apache.hadoop.hive.ql.Driver.fireFilterHooks(Driver.java:1455)
>>     at org.apache.hadoop.hive.ql.Driver.getResults(Driver.java:1475)
>>     at org.apache.hive.service.cli.operation.SQLOperation.getNextRowSet(SQLOperation.java:200)
>>     at org.apache.hive.service.cli.operation.OperationManager.getOperationNextRowSet(OperationManager.java:179)
>>     at org.apache.hive.service.cli.session.HiveSessionImpl.fetchResults(HiveSessionImpl.java:468)
>>     at org.apache.hive.service.cli.CLIService.fetchResults(CLIService.java:318)
>>     at org.apache.hive.service.cli.thrift.ThriftCLIService.FetchResults(ThriftCLIService.java:398)
>>     at org.apache.hive.service.cli.thrift.TCLIService$Processor$FetchResults.getResult(TCLIService.java:1613)
>>     at org.apache.hive.service.cli.thrift.TCLIService$Processor$FetchResults.getResult(TCLIService.java:1598)
>>     at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
>>     at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
>>     at org.apache.hive.service.cli.thrift.TSetIpAddressProcessor.process(TSetIpAddressProcessor.java:38)
>>     at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:244)
>>     at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>>     at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>>     at java.lang.Thread.run(Thread.java:722)
>> =======
>> 
>> 
>> I think the issue is my account on LDAP is atangri, not atangri@a.b.
>> 
>> Any thoughts on how to fix this ?
>> 
>> 
>> Thanks,
>> Anurag Tangri
>> 
>> 
>> 
>>  
>> 
>> 
>> =====
>> 2014-06-04 07:15:13,211 WARN  file.HadoopGroupMappingService (HadoopGroupMappingService.java:getGroups(42)) - Unable to obtain groups for atangri@group.on
>> java.io.IOException: No groups found for user atangri@group.on
>> 
>> 
>>> On Tue, Jun 3, 2014 at 12:49 PM, Anurag <at...@groupon.com> wrote:
>>> Thanks Philippe!
>>> 
>>> This answers almost all of my questions.
>>> 
>>> Thanks,
>>> Anurag Tangri
>>> 
>>>> On Jun 3, 2014, at 12:08 PM, Philippe Marseille <ma...@gmail.com> wrote:
>>>> 
>>>> I think the Hive CLI (Command line interface tool) is now deprecated.  see http://blog.cloudera.com/blog/2014/02/migrating-from-hive-cli-to-beeline-a-primer/
>>>> 
>>>> 
>>>> The metastore has nothing to do with it.  The Metastore Daemon is used by all clients.
>>>> 
>>>> The new flow is :
>>>> 
>>>> Beeline -> HiveServer2 -> Metastore Daemon -> Metastore Database (MySql for example).
>>>> 
>>>> The previous flow was :
>>>> 
>>>> Hive CLI (shell ) -> Metastore Daemon -> Metastore Database
>>>> 
>>>> And before the Metastore Daemon ever existed, it was :
>>>> 
>>>> Hive CLI -> Metastore Database
>>>> 
>>>>> On Tuesday, 3 June 2014 13:06:22 UTC-4, Anurag Tangri wrote:
>>>>> Thanks Casey.
>>>>> 
>>>>> This is going to be big shift to ask all our users  to go to beeline from hive.
>>>>> 
>>>>> Is there no way to have consistency between what roles and policies are seen in HUE via hiveserver2 and hive metastore on hive shell ? 
>>>>> Also, is there a reason why hive metastore was excluded from Sentry design ?
>>>>> 
>>>>> 
>>>>> 
>>>>> Thanks,
>>>>> Anurag Tangri
>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>>>> On Tue, Jun 3, 2014 at 9:54 AM, Casey Brotherton <cb...@cloudera.com> wrote:
>>>>>> Hello Anurag, 
>>>>>> 
>>>>>> To use Sentry, as Johndee mentioned, you must use Hiveserver2.  That means you will need to transition users from the hive command line to the beeline command line.
>>>>>> 
>>>>>> This link has more information for CDH4.5
>>>>>> http://www.cloudera.com/content/cloudera-content/cloudera-docs/CDH4/4.5.0/CDH4-Security-Guide/cdh4sg_Sentry.html
>>>>>> 
>>>>>> There is a link to "Securing the Hive Metastore"  which suggests to limit connections using iptables, and granting of permissions.
>>>>>> 
>>>>>> Hope this helps,
>>>>>> Casey
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>>> On Tue, Jun 3, 2014 at 11:45 AM, Anurag Tangri <at...@groupon.com> wrote:
>>>>>>> Hey Philippe,
>>>>>>> No, HUE should be fine as per Johndee's explanation as it can be pointed to hiveserver2.
>>>>>>> 
>>>>>>> By metastore, I mean mysql database that hosts hive metadata.
>>>>>>> 
>>>>>>> This is the database that people go against when they login to hive shell for a hadoop client box.
>>>>>>> 
>>>>>>> 
>>>>>>> Thanks,
>>>>>>> Anurag Tangri
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>>> On Tue, Jun 3, 2014 at 8:11 AM, Philippe Marseille <ma...@gmail.com> wrote:
>>>>>>>> By "Hive Metastore", are you referring to the app in Hue?  
>>>>>>>> 
>>>>>>>> 
>>>>>>>>> On Tuesday, 3 June 2014 00:32:53 UTC-4, Anurag Tangri wrote:
>>>>>>>>> Thanks Johndee.
>>>>>>>>> 
>>>>>>>>> Can you point to some documentation on hive group and rejecting requests from metastore ?
>>>>>>>>> 
>>>>>>>>> Hiveserver2 is mainly for jdbc interface, hue etc.
>>>>>>>>> 
>>>>>>>>> For people going via hive metastore, we will need to enforce same set of roles and policies as via hiveserver2.
>>>>>>>>> 
>>>>>>>>> Thanks,
>>>>>>>>> Anurag Tangri
>>>>>>>>> 
>>>>>>>>>> On Jun 2, 2014, at 7:57 PM, Johndee Cloudera <jo...@cloudera.com> wrote:
>>>>>>>>>> 
>>>>>>>>>> When it comes to hive and sentry you need at least Hive Server 2. Hive Server 2 is what actually runs the Sentry daemon. The Hive Metastore Server handles metadata requests, and in the setup guide is configured to reject all connections from users not in the hive group typically. 
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>>> On Mon, Jun 2, 2014 at 10:15 PM, Anurag Tangri <ta...@gmail.com> wrote:
>>>>>>>>>>> Hi all,
>>>>>>>>>>> From what I read everywhere, sentry is to be used in conjunction with hiveserver2.
>>>>>>>>>>> 
>>>>>>>>>>> Is it true that it can be used with hive metastore ? If yes, Any links explaining how to do so would be great.
>>>>>>>>>>> 
>>>>>>>>>>> Thanks,
>>>>>>>>>>> Anurag Tangri
>>>>>>>>>>> 
>>>>>>>>>>> --
>>>>>>>>>>> 
>>>>>>>>>>> ---
>>>>>>>>>>> You received this message because you are subscribed to the Google Groups "CDH Users" group.
>>>>>>>>>>> To unsubscribe from this group and stop receiving emails from it, send an email to cdh-user+u...@cloudera.org.
>>>>>>>>>>> 
>>>>>>>>>>> For more options, visit https://groups.google.com/a/cloudera.org/d/optout.
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> -- 
>>>>>>>>>> - JRB
>>>>>>>>>> -- 
>>>>>>>>>> 
>>>>>>>>>> --- 
>>>>>>>>>> You received this message because you are subscribed to the Google Groups "CDH Users" group.
>>>>>>>>>> To unsubscribe from this group and stop receiving emails from it, send an email to cdh-user+u...@cloudera.org.
>>>>>>>>>> 
>>>>>>>>>> For more options, visit https://groups.google.com/a/cloudera.org/d/optout.
>>>>>>>> 
>>>>>>>> -- 
>>>>>>>> 
>>>>>>>> --- 
>>>>>>>> You received this message because you are subscribed to the Google Groups "CDH Users" group.
>>>>>>>> To unsubscribe from this group and stop receiving emails from it, send an email to cdh-user+u...@cloudera.org.
>>>>>>>> For more options, visit https://groups.google.com/a/cloudera.org/d/optout.
>>>>>>> 
>>>>>>> -- 
>>>>>>> 
>>>>>>> --- 
>>>>>>> You received this message because you are subscribed to the Google Groups "CDH Users" group.
>>>>>>> To unsubscribe from this group and stop receiving emails from it, send an email to cdh-user+u...@cloudera.org.
>>>>>>> For more options, visit https://groups.google.com/a/cloudera.org/d/optout.
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> -- 
>>>>>> Casey J. Brotherton
>>>>>> Customer Operations Engineer
>>>>>> 
>>>>>> -- 
>>>>>> 
>>>>>> --- 
>>>>>> You received this message because you are subscribed to the Google Groups "CDH Users" group.
>>>>>> To unsubscribe from this group and stop receiving emails from it, send an email to cdh-user+u...@cloudera.org.
>>>>>> For more options, visit https://groups.google.com/a/cloudera.org/d/optout.
>>>> 
>>>> -- 
>>>> 
>>>> --- 
>>>> You received this message because you are subscribed to the Google Groups "CDH Users" group.
>>>> To unsubscribe from this group and stop receiving emails from it, send an email to cdh-user+unsubscribe@cloudera.org.
>>>> For more options, visit https://groups.google.com/a/cloudera.org/d/optout.
> 
> -- 
> 
> --- 
> You received this message because you are subscribed to the Google Groups "CDH Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to cdh-user+unsubscribe@cloudera.org.
> For more options, visit https://groups.google.com/a/cloudera.org/d/optout.

Re: Apache Sentry works with hive metastore ?

Posted by Sravya Tirukkovalur <sr...@cloudera.com>.
Hi Anurag,

Looks like Hive Sever2 authentication itself is not working in your setup.
So even before Sentry authorization happens. And mostly because of domain
name issues. Can you confirm if you are using AD or openLDAP?

Essentially HiveServer2 allows you to setup domain names/ baseDN for AD /
OpenLDAP differently.

For AD, you can use the property "hive.server2.authentication.ldap.Domain" and
for openLDAP you can use "hive.server2.authentication.ldap.baseDN"

Regards,


On Tue, Jun 10, 2014 at 12:58 PM, Anurag Tangri <at...@groupon.com> wrote:

> At this point, I am starting to think Sentry does not support LDAP
> Integration without Kerberos for HiveServer2 as I am not able to get it up
> and keep on getting errors like below:
>
> =========
>
> 2014-06-10 19:50:11,621 DEBUG transport.TSaslServerTransport
> (TSaslServerTransport.java:getTransport(218)) - failed to open server
> transport
> org.apache.thrift.transport.TTransportException: Error validating the login
>     at
>
> org.apache.thrift.transport.TSaslTransport.sendAndThrowMessage(TSaslTransport.java:221)
>     at
> org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:297)
>     at
>
> org.apache.thrift.transport.TSaslServerTransport.open(TSaslServerTransport.java:1)
>     at
>
> org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:216)
>     at
>
> org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:227)
>     at
>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>     at
>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>     at java.lang.Thread.run(Thread.java:722)
> 2014-06-10 19:50:11,621 DEBUG log.LogManager
> (Log4jLoggerAdapter.java:debug(286)) - Operation log assocaited with
> thread: pool-1-thread-2 couldn't be found.
> 2014-06-10 19:50:11,621 DEBUG log.LogDivertAppender
> (Log4jLoggerAdapter.java:debug(286)) -  ---+++=== Dropped log event from
> thread pool-1-thread-2
> 2014-06-10 19:50:11,622 ERROR server.TThreadPoolServer
> (TThreadPoolServer.java:run(253)) - Error occurred during processing of
> message.
> java.lang.RuntimeException:
> org.apache.thrift.transport.TTransportException: Error validating the login
>     at
>
> org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:219)
>     at
>
> org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:227)
>     at
>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>     at
>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>     at java.lang.Thread.run(Thread.java:722)
> Caused by: org.apache.thrift.transport.TTransportException: Error
> validating the login
>     at
>
> org.apache.thrift.transport.TSaslTransport.sendAndThrowMessage(TSaslTransport.java:221)
>     at
> org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:297)
>     at
>
> org.apache.thrift.transport.TSaslServerTransport.open(TSaslServerTransport.java:1)
>     at
>
> org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:216)
>     ... 4 more
>
> ================
>
> It would be nice if Sentry team can confirm this and save me the agony of
> keep on trying to configure something that is not even there :)
>
> Thanks,
> Anurag Tangri
>
>
> On Tue, Jun 10, 2014 at 12:16 AM, Anurag Tangri <at...@groupon.com>
> wrote:
>
> > Hi Prasad,
> > I tried it and the setting hive.server2.authentication.ldap.Domain does
> > not change anything.
> >
> > Thanks,
> > Anurag Tangri
> >
> >
> > On Thu, Jun 5, 2014 at 10:45 AM, Anurag Tangri <at...@groupon.com>
> > wrote:
> >
> >> That's correct Prasad.
> >>
> >> Let me try this and see if it works.
> >>
> >>
> >> Thanks,
> >> Anurag Tangri
> >>
> >>
> >> On Thu, Jun 5, 2014 at 10:35 AM, Prasad Mujumdar <pr...@cloudera.com>
> >> wrote:
> >>
> >>> Hi Anurag,
> >>>
> >>>    If this is active directory, then try setting a config property
> >>> hive.server2.authentication.ldap.Domain to your LDAP domain (a.b).
> Then you
> >>> can login with the short userid atangri.
> >>> Are you using LDAP group mapping in Hadoop  as well ?
> >>>
> >>> thanks
> >>> Prasad
> >>>
> >>>
> >>>
> >>> On Thu, Jun 5, 2014 at 7:43 AM, Anurag <at...@groupon.com> wrote:
> >>>
> >>>>
> >>>> Any response anyone?
> >>>>
> >>>> Sent from my iPhone
> >>>> > On Jun 4, 2014, at 3:20 PM, Anurag Tangri <at...@groupon.com>
> >>>> wrote:
> >>>> >
> >>>> > + sentry mailing list
> >>>> >
> >>>> >
> >>>> >> On Wed, Jun 4, 2014 at 12:27 AM, Anurag Tangri <
> atangri@groupon.com>
> >>>> wrote:
> >>>> >> So, I am working on setting this up.
> >>>> >>
> >>>> >> I have HDFS integrated with LDAP and working fine.
> >>>> >>
> >>>> >> I also added ldap params in hive-site along with sentry conf :
> >>>> >>
> >>>> >> ====
> >>>> >>   <property>
> >>>> >>     <name>hive.server2.authentication</name>
> >>>> >>     <value>LDAP</value>
> >>>> >>   </property>
> >>>> >>   <property>
> >>>> >>     <name>hive.server2.authentication.ldap.url</name>
> >>>> >>     <value>ldap://<ip></value>
> >>>> >>   </property>
> >>>> >>
> >>>> >> <property>
> >>>> >>   <name>hive.server2.authentication.ldap.baseDN</name>
> >>>> >>   <value>dc=a,dc=b</value>
> >>>> >> </property>
> >>>> >>
> >>>> >> ======
> >>>> >>
> >>>> >> Now when I go to beeline, I type:
> >>>> >>
> >>>> >> /usr/local/lib/hive/bin/beeline -u jdbc:hive2://<host>:10000
> >>>> >> Beeline version 0.10.0-cdh4.6.0 by Apache Hive
> >>>> >> 0: jdbc:hive2://<host>:10000> !connect jdbc:hive2://<host>:10000
> >>>> >>
> >>>> >>
> >>>> >> It asks for username and password. I give LDAP credentials as:
> >>>> >>
> >>>> >> Connecting to jdbc:hive2://<host>:10000
> >>>> >> Enter username for jdbc:hive2://<host>:10000: atangri@a.b
> >>>> >> Enter password for jdbc:hive2://<host>:10000: **********
> >>>> >> Error: Invalid URL: jdbc:hive2://<host>:10000 (state=08S01,code=0)
> >>>> >> 1: jdbc:hive2://<host>:10000>
> >>>> >>
> >>>> >>
> >>>> >> Now if I type, hive queries, i see error in hive log:
> >>>> >>
> >>>> >> 2014-06-04 07:15:13,211 WARN  file.HadoopGroupMappingService
> >>>> (HadoopGroupMappingService.java:getGroups(42)) - Unable to obtain
> groups
> >>>> for atangri@a.b
> >>>> >> java.io.IOException: No groups found for user atangri@a.b
> >>>> >>     at org.apache.hadoop.security.Groups.getGroups(Groups.java:105)
> >>>> >>     at
> >>>>
> org.apache.sentry.provider.file.HadoopGroupMappingService.getGroups(HadoopGroupMappingService.java:40)
> >>>> >>     at
> >>>>
> org.apache.sentry.provider.file.ResourceAuthorizationProvider.doHasAccess(ResourceAuthorizationProvider.java:98)
> >>>> >>     at
> >>>>
> org.apache.sentry.provider.file.ResourceAuthorizationProvider.hasAccess(ResourceAuthorizationProvider.java:93)
> >>>> >>     at
> >>>>
> org.apache.sentry.binding.hive.authz.HiveAuthzBinding.authorize(HiveAuthzBinding.java:179)
> >>>> >>     at
> >>>>
> org.apache.sentry.binding.hive.HiveAuthzBindingHook.filterShowTables(HiveAuthzBindingHook.java:561)
> >>>> >>     at
> >>>>
> org.apache.sentry.binding.hive.HiveAuthzBindingHook.postDriverFetch(HiveAuthzBindingHook.java:634)
> >>>> >>     at
> >>>> org.apache.hadoop.hive.ql.Driver.fireFilterHooks(Driver.java:1455)
> >>>> >>     at
> org.apache.hadoop.hive.ql.Driver.getResults(Driver.java:1475)
> >>>> >>     at
> >>>>
> org.apache.hive.service.cli.operation.SQLOperation.getNextRowSet(SQLOperation.java:200)
> >>>> >>     at
> >>>>
> org.apache.hive.service.cli.operation.OperationManager.getOperationNextRowSet(OperationManager.java:179)
> >>>> >>     at
> >>>>
> org.apache.hive.service.cli.session.HiveSessionImpl.fetchResults(HiveSessionImpl.java:468)
> >>>> >>     at
> >>>>
> org.apache.hive.service.cli.CLIService.fetchResults(CLIService.java:318)
> >>>> >>     at
> >>>>
> org.apache.hive.service.cli.thrift.ThriftCLIService.FetchResults(ThriftCLIService.java:398)
> >>>> >>     at
> >>>>
> org.apache.hive.service.cli.thrift.TCLIService$Processor$FetchResults.getResult(TCLIService.java:1613)
> >>>> >>     at
> >>>>
> org.apache.hive.service.cli.thrift.TCLIService$Processor$FetchResults.getResult(TCLIService.java:1598)
> >>>> >>     at
> >>>> org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
> >>>> >>     at
> >>>> org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
> >>>> >>     at
> >>>>
> org.apache.hive.service.cli.thrift.TSetIpAddressProcessor.process(TSetIpAddressProcessor.java:38)
> >>>> >>     at
> >>>>
> org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:244)
> >>>> >>     at
> >>>>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> >>>> >>     at
> >>>>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> >>>> >>     at java.lang.Thread.run(Thread.java:722)
> >>>> >> =======
> >>>> >>
> >>>> >>
> >>>> >> I think the issue is my account on LDAP is atangri, not
> atangri@a.b.
> >>>> >>
> >>>> >> Any thoughts on how to fix this ?
> >>>> >>
> >>>> >>
> >>>> >> Thanks,
> >>>> >> Anurag Tangri
> >>>> >>
> >>>> >>
> >>>> >>
> >>>> >>
> >>>> >>
> >>>> >>
> >>>> >> =====
> >>>> >> 2014-06-04 07:15:13,211 WARN  file.HadoopGroupMappingService
> >>>> (HadoopGroupMappingService.java:getGroups(42)) - Unable to obtain
> groups
> >>>> for atangri@group.on
> >>>> >> java.io.IOException: No groups found for user atangri@group.on
> >>>> >>
> >>>> >>
> >>>> >>> On Tue, Jun 3, 2014 at 12:49 PM, Anurag <at...@groupon.com>
> >>>> wrote:
> >>>> >>> Thanks Philippe!
> >>>> >>>
> >>>> >>> This answers almost all of my questions.
> >>>> >>>
> >>>> >>> Thanks,
> >>>> >>> Anurag Tangri
> >>>> >>>
> >>>> >>>> On Jun 3, 2014, at 12:08 PM, Philippe Marseille <
> >>>> marseilp@gmail.com> wrote:
> >>>> >>>>
> >>>> >>>> I think the Hive CLI (Command line interface tool) is now
> >>>> deprecated.  see
> >>>>
> http://blog.cloudera.com/blog/2014/02/migrating-from-hive-cli-to-beeline-a-primer/
> >>>> >>>>
> >>>> >>>>
> >>>> >>>> The metastore has nothing to do with it.  The Metastore Daemon is
> >>>> used by all clients.
> >>>> >>>>
> >>>> >>>> The new flow is :
> >>>> >>>>
> >>>> >>>> Beeline -> HiveServer2 -> Metastore Daemon -> Metastore Database
> >>>> (MySql for example).
> >>>> >>>>
> >>>> >>>> The previous flow was :
> >>>> >>>>
> >>>> >>>> Hive CLI (shell ) -> Metastore Daemon -> Metastore Database
> >>>> >>>>
> >>>> >>>> And before the Metastore Daemon ever existed, it was :
> >>>> >>>>
> >>>> >>>> Hive CLI -> Metastore Database
> >>>> >>>>
> >>>> >>>>> On Tuesday, 3 June 2014 13:06:22 UTC-4, Anurag Tangri wrote:
> >>>> >>>>> Thanks Casey.
> >>>> >>>>>
> >>>> >>>>> This is going to be big shift to ask all our users  to go to
> >>>> beeline from hive.
> >>>> >>>>>
> >>>> >>>>> Is there no way to have consistency between what roles and
> >>>> policies are seen in HUE via hiveserver2 and hive metastore on hive
> shell ?
> >>>> >>>>> Also, is there a reason why hive metastore was excluded from
> >>>> Sentry design ?
> >>>> >>>>>
> >>>> >>>>>
> >>>> >>>>>
> >>>> >>>>> Thanks,
> >>>> >>>>> Anurag Tangri
> >>>> >>>>>
> >>>> >>>>>
> >>>> >>>>>
> >>>> >>>>>
> >>>> >>>>>> On Tue, Jun 3, 2014 at 9:54 AM, Casey Brotherton <
> >>>> cbrot...@cloudera.com> wrote:
> >>>> >>>>>> Hello Anurag,
> >>>> >>>>>>
> >>>> >>>>>> To use Sentry, as Johndee mentioned, you must use Hiveserver2.
> >>>>  That means you will need to transition users from the hive command
> line to
> >>>> the beeline command line.
> >>>> >>>>>>
> >>>> >>>>>> This link has more information for CDH4.5
> >>>> >>>>>>
> >>>>
> http://www.cloudera.com/content/cloudera-content/cloudera-docs/CDH4/4.5.0/CDH4-Security-Guide/cdh4sg_Sentry.html
> >>>> >>>>>>
> >>>> >>>>>> There is a link to "Securing the Hive Metastore"  which
> suggests
> >>>> to limit connections using iptables, and granting of permissions.
> >>>> >>>>>>
> >>>> >>>>>> Hope this helps,
> >>>> >>>>>> Casey
> >>>> >>>>>>
> >>>> >>>>>>
> >>>> >>>>>>
> >>>> >>>>>>> On Tue, Jun 3, 2014 at 11:45 AM, Anurag Tangri <
> >>>> ata...@groupon.com> wrote:
> >>>> >>>>>>> Hey Philippe,
> >>>> >>>>>>> No, HUE should be fine as per Johndee's explanation as it can
> >>>> be pointed to hiveserver2.
> >>>> >>>>>>>
> >>>> >>>>>>> By metastore, I mean mysql database that hosts hive metadata.
> >>>> >>>>>>>
> >>>> >>>>>>> This is the database that people go against when they login to
> >>>> hive shell for a hadoop client box.
> >>>> >>>>>>>
> >>>> >>>>>>>
> >>>> >>>>>>> Thanks,
> >>>> >>>>>>> Anurag Tangri
> >>>> >>>>>>>
> >>>> >>>>>>>
> >>>> >>>>>>>
> >>>> >>>>>>>> On Tue, Jun 3, 2014 at 8:11 AM, Philippe Marseille <
> >>>> mars...@gmail.com> wrote:
> >>>> >>>>>>>> By "Hive Metastore", are you referring to the app in Hue?
> >>>> >>>>>>>>
> >>>> >>>>>>>>
> >>>> >>>>>>>>> On Tuesday, 3 June 2014 00:32:53 UTC-4, Anurag Tangri wrote:
> >>>> >>>>>>>>> Thanks Johndee.
> >>>> >>>>>>>>>
> >>>> >>>>>>>>> Can you point to some documentation on hive group and
> >>>> rejecting requests from metastore ?
> >>>> >>>>>>>>>
> >>>> >>>>>>>>> Hiveserver2 is mainly for jdbc interface, hue etc.
> >>>> >>>>>>>>>
> >>>> >>>>>>>>> For people going via hive metastore, we will need to enforce
> >>>> same set of roles and policies as via hiveserver2.
> >>>> >>>>>>>>>
> >>>> >>>>>>>>> Thanks,
> >>>> >>>>>>>>> Anurag Tangri
> >>>> >>>>>>>>>
> >>>> >>>>>>>>>> On Jun 2, 2014, at 7:57 PM, Johndee Cloudera <
> >>>> joh...@cloudera.com> wrote:
> >>>> >>>>>>>>>>
> >>>> >>>>>>>>>> When it comes to hive and sentry you need at least Hive
> >>>> Server 2. Hive Server 2 is what actually runs the Sentry daemon. The
> Hive
> >>>> Metastore Server handles metadata requests, and in the setup guide is
> >>>> configured to reject all connections from users not in the hive group
> >>>> typically.
> >>>> >>>>>>>>>>
> >>>> >>>>>>>>>>
> >>>> >>>>>>>>>>> On Mon, Jun 2, 2014 at 10:15 PM, Anurag Tangri <
> >>>> tangri...@gmail.com> wrote:
> >>>> >>>>>>>>>>> Hi all,
> >>>> >>>>>>>>>>> From what I read everywhere, sentry is to be used in
> >>>> conjunction with hiveserver2.
> >>>> >>>>>>>>>>>
> >>>> >>>>>>>>>>> Is it true that it can be used with hive metastore ? If
> >>>> yes, Any links explaining how to do so would be great.
> >>>> >>>>>>>>>>>
> >>>> >>>>>>>>>>> Thanks,
> >>>> >>>>>>>>>>> Anurag Tangri
> >>>> >>>>>>>>>>>
> >>>> >>>>>>>>>>> --
> >>>> >>>>>>>>>>>
> >>>> >>>>>>>>>>> ---
> >>>> >>>>>>>>>>> You received this message because you are subscribed to
> the
> >>>> Google Groups "CDH Users" group.
> >>>> >>>>>>>>>>> To unsubscribe from this group and stop receiving emails
> >>>> from it, send an email to cdh-user+u...@cloudera.org.
> >>>> >>>>>>>>>>>
> >>>> >>>>>>>>>>> For more options, visit
> >>>> https://groups.google.com/a/cloudera.org/d/optout.
> >>>> >>>>>>>>>>
> >>>> >>>>>>>>>>
> >>>> >>>>>>>>>>
> >>>> >>>>>>>>>> --
> >>>> >>>>>>>>>> - JRB
> >>>> >>>>>>>>>> --
> >>>> >>>>>>>>>>
> >>>> >>>>>>>>>> ---
> >>>> >>>>>>>>>> You received this message because you are subscribed to the
> >>>> Google Groups "CDH Users" group.
> >>>> >>>>>>>>>> To unsubscribe from this group and stop receiving emails
> >>>> from it, send an email to cdh-user+u...@cloudera.org.
> >>>> >>>>>>>>>>
> >>>> >>>>>>>>>> For more options, visit
> >>>> https://groups.google.com/a/cloudera.org/d/optout.
> >>>> >>>>>>>>
> >>>> >>>>>>>> --
> >>>> >>>>>>>>
> >>>> >>>>>>>> ---
> >>>> >>>>>>>> You received this message because you are subscribed to the
> >>>> Google Groups "CDH Users" group.
> >>>> >>>>>>>> To unsubscribe from this group and stop receiving emails from
> >>>> it, send an email to cdh-user+u...@cloudera.org.
> >>>> >>>>>>>> For more options, visit
> >>>> https://groups.google.com/a/cloudera.org/d/optout.
> >>>> >>>>>>>
> >>>> >>>>>>> --
> >>>> >>>>>>>
> >>>> >>>>>>> ---
> >>>> >>>>>>> You received this message because you are subscribed to the
> >>>> Google Groups "CDH Users" group.
> >>>> >>>>>>> To unsubscribe from this group and stop receiving emails from
> >>>> it, send an email to cdh-user+u...@cloudera.org.
> >>>> >>>>>>> For more options, visit
> >>>> https://groups.google.com/a/cloudera.org/d/optout.
> >>>> >>>>>>
> >>>> >>>>>>
> >>>> >>>>>>
> >>>> >>>>>> --
> >>>> >>>>>> Casey J. Brotherton
> >>>> >>>>>> Customer Operations Engineer
> >>>> >>>>>>
> >>>> >>>>>> --
> >>>> >>>>>>
> >>>> >>>>>> ---
> >>>> >>>>>> You received this message because you are subscribed to the
> >>>> Google Groups "CDH Users" group.
> >>>> >>>>>> To unsubscribe from this group and stop receiving emails from
> >>>> it, send an email to cdh-user+u...@cloudera.org.
> >>>> >>>>>> For more options, visit
> >>>> https://groups.google.com/a/cloudera.org/d/optout.
> >>>> >>>>
> >>>> >>>> --
> >>>> >>>>
> >>>> >>>> ---
> >>>> >>>> You received this message because you are subscribed to the
> Google
> >>>> Groups "CDH Users" group.
> >>>> >>>> To unsubscribe from this group and stop receiving emails from it,
> >>>> send an email to cdh-user+unsubscribe@cloudera.org.
> >>>> >>>> For more options, visit
> >>>> https://groups.google.com/a/cloudera.org/d/optout.
> >>>> >
> >>>> > --
> >>>> >
> >>>> > ---
> >>>> > You received this message because you are subscribed to the Google
> >>>> Groups "CDH Users" group.
> >>>> > To unsubscribe from this group and stop receiving emails from it,
> >>>> send an email to cdh-user+unsubscribe@cloudera.org.
> >>>> > For more options, visit
> >>>> https://groups.google.com/a/cloudera.org/d/optout.
> >>>>
> >>>
> >>>  --
> >>>
> >>> ---
> >>> You received this message because you are subscribed to the Google
> >>> Groups "CDH Users" group.
> >>> To unsubscribe from this group and stop receiving emails from it, send
> >>> an email to cdh-user+unsubscribe@cloudera.org.
> >>> For more options, visit
> >>> https://groups.google.com/a/cloudera.org/d/optout.
> >>>
> >>
> >>
> >
>



-- 
Sravya Tirukkovalur

Re: Apache Sentry works with hive metastore ?

Posted by Anurag Tangri <at...@groupon.com>.
At this point, I am starting to think Sentry does not support LDAP
Integration without Kerberos for HiveServer2 as I am not able to get it up
and keep on getting errors like below:

=========

2014-06-10 19:50:11,621 DEBUG transport.TSaslServerTransport
(TSaslServerTransport.java:getTransport(218)) - failed to open server
transport
org.apache.thrift.transport.TTransportException: Error validating the login
    at
org.apache.thrift.transport.TSaslTransport.sendAndThrowMessage(TSaslTransport.java:221)
    at
org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:297)
    at
org.apache.thrift.transport.TSaslServerTransport.open(TSaslServerTransport.java:1)
    at
org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:216)
    at
org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:227)
    at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:722)
2014-06-10 19:50:11,621 DEBUG log.LogManager
(Log4jLoggerAdapter.java:debug(286)) - Operation log assocaited with
thread: pool-1-thread-2 couldn't be found.
2014-06-10 19:50:11,621 DEBUG log.LogDivertAppender
(Log4jLoggerAdapter.java:debug(286)) -  ---+++=== Dropped log event from
thread pool-1-thread-2
2014-06-10 19:50:11,622 ERROR server.TThreadPoolServer
(TThreadPoolServer.java:run(253)) - Error occurred during processing of
message.
java.lang.RuntimeException:
org.apache.thrift.transport.TTransportException: Error validating the login
    at
org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:219)
    at
org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:227)
    at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:722)
Caused by: org.apache.thrift.transport.TTransportException: Error
validating the login
    at
org.apache.thrift.transport.TSaslTransport.sendAndThrowMessage(TSaslTransport.java:221)
    at
org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:297)
    at
org.apache.thrift.transport.TSaslServerTransport.open(TSaslServerTransport.java:1)
    at
org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:216)
    ... 4 more

================

It would be nice if Sentry team can confirm this and save me the agony of
keep on trying to configure something that is not even there :)

Thanks,
Anurag Tangri


On Tue, Jun 10, 2014 at 12:16 AM, Anurag Tangri <at...@groupon.com> wrote:

> Hi Prasad,
> I tried it and the setting hive.server2.authentication.ldap.Domain does
> not change anything.
>
> Thanks,
> Anurag Tangri
>
>
> On Thu, Jun 5, 2014 at 10:45 AM, Anurag Tangri <at...@groupon.com>
> wrote:
>
>> That's correct Prasad.
>>
>> Let me try this and see if it works.
>>
>>
>> Thanks,
>> Anurag Tangri
>>
>>
>> On Thu, Jun 5, 2014 at 10:35 AM, Prasad Mujumdar <pr...@cloudera.com>
>> wrote:
>>
>>> Hi Anurag,
>>>
>>>    If this is active directory, then try setting a config property
>>> hive.server2.authentication.ldap.Domain to your LDAP domain (a.b). Then you
>>> can login with the short userid atangri.
>>> Are you using LDAP group mapping in Hadoop  as well ?
>>>
>>> thanks
>>> Prasad
>>>
>>>
>>>
>>> On Thu, Jun 5, 2014 at 7:43 AM, Anurag <at...@groupon.com> wrote:
>>>
>>>>
>>>> Any response anyone?
>>>>
>>>> Sent from my iPhone
>>>> > On Jun 4, 2014, at 3:20 PM, Anurag Tangri <at...@groupon.com>
>>>> wrote:
>>>> >
>>>> > + sentry mailing list
>>>> >
>>>> >
>>>> >> On Wed, Jun 4, 2014 at 12:27 AM, Anurag Tangri <at...@groupon.com>
>>>> wrote:
>>>> >> So, I am working on setting this up.
>>>> >>
>>>> >> I have HDFS integrated with LDAP and working fine.
>>>> >>
>>>> >> I also added ldap params in hive-site along with sentry conf :
>>>> >>
>>>> >> ====
>>>> >>   <property>
>>>> >>     <name>hive.server2.authentication</name>
>>>> >>     <value>LDAP</value>
>>>> >>   </property>
>>>> >>   <property>
>>>> >>     <name>hive.server2.authentication.ldap.url</name>
>>>> >>     <value>ldap://<ip></value>
>>>> >>   </property>
>>>> >>
>>>> >> <property>
>>>> >>   <name>hive.server2.authentication.ldap.baseDN</name>
>>>> >>   <value>dc=a,dc=b</value>
>>>> >> </property>
>>>> >>
>>>> >> ======
>>>> >>
>>>> >> Now when I go to beeline, I type:
>>>> >>
>>>> >> /usr/local/lib/hive/bin/beeline -u jdbc:hive2://<host>:10000
>>>> >> Beeline version 0.10.0-cdh4.6.0 by Apache Hive
>>>> >> 0: jdbc:hive2://<host>:10000> !connect jdbc:hive2://<host>:10000
>>>> >>
>>>> >>
>>>> >> It asks for username and password. I give LDAP credentials as:
>>>> >>
>>>> >> Connecting to jdbc:hive2://<host>:10000
>>>> >> Enter username for jdbc:hive2://<host>:10000: atangri@a.b
>>>> >> Enter password for jdbc:hive2://<host>:10000: **********
>>>> >> Error: Invalid URL: jdbc:hive2://<host>:10000 (state=08S01,code=0)
>>>> >> 1: jdbc:hive2://<host>:10000>
>>>> >>
>>>> >>
>>>> >> Now if I type, hive queries, i see error in hive log:
>>>> >>
>>>> >> 2014-06-04 07:15:13,211 WARN  file.HadoopGroupMappingService
>>>> (HadoopGroupMappingService.java:getGroups(42)) - Unable to obtain groups
>>>> for atangri@a.b
>>>> >> java.io.IOException: No groups found for user atangri@a.b
>>>> >>     at org.apache.hadoop.security.Groups.getGroups(Groups.java:105)
>>>> >>     at
>>>> org.apache.sentry.provider.file.HadoopGroupMappingService.getGroups(HadoopGroupMappingService.java:40)
>>>> >>     at
>>>> org.apache.sentry.provider.file.ResourceAuthorizationProvider.doHasAccess(ResourceAuthorizationProvider.java:98)
>>>> >>     at
>>>> org.apache.sentry.provider.file.ResourceAuthorizationProvider.hasAccess(ResourceAuthorizationProvider.java:93)
>>>> >>     at
>>>> org.apache.sentry.binding.hive.authz.HiveAuthzBinding.authorize(HiveAuthzBinding.java:179)
>>>> >>     at
>>>> org.apache.sentry.binding.hive.HiveAuthzBindingHook.filterShowTables(HiveAuthzBindingHook.java:561)
>>>> >>     at
>>>> org.apache.sentry.binding.hive.HiveAuthzBindingHook.postDriverFetch(HiveAuthzBindingHook.java:634)
>>>> >>     at
>>>> org.apache.hadoop.hive.ql.Driver.fireFilterHooks(Driver.java:1455)
>>>> >>     at org.apache.hadoop.hive.ql.Driver.getResults(Driver.java:1475)
>>>> >>     at
>>>> org.apache.hive.service.cli.operation.SQLOperation.getNextRowSet(SQLOperation.java:200)
>>>> >>     at
>>>> org.apache.hive.service.cli.operation.OperationManager.getOperationNextRowSet(OperationManager.java:179)
>>>> >>     at
>>>> org.apache.hive.service.cli.session.HiveSessionImpl.fetchResults(HiveSessionImpl.java:468)
>>>> >>     at
>>>> org.apache.hive.service.cli.CLIService.fetchResults(CLIService.java:318)
>>>> >>     at
>>>> org.apache.hive.service.cli.thrift.ThriftCLIService.FetchResults(ThriftCLIService.java:398)
>>>> >>     at
>>>> org.apache.hive.service.cli.thrift.TCLIService$Processor$FetchResults.getResult(TCLIService.java:1613)
>>>> >>     at
>>>> org.apache.hive.service.cli.thrift.TCLIService$Processor$FetchResults.getResult(TCLIService.java:1598)
>>>> >>     at
>>>> org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
>>>> >>     at
>>>> org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
>>>> >>     at
>>>> org.apache.hive.service.cli.thrift.TSetIpAddressProcessor.process(TSetIpAddressProcessor.java:38)
>>>> >>     at
>>>> org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:244)
>>>> >>     at
>>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>>>> >>     at
>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>>>> >>     at java.lang.Thread.run(Thread.java:722)
>>>> >> =======
>>>> >>
>>>> >>
>>>> >> I think the issue is my account on LDAP is atangri, not atangri@a.b.
>>>> >>
>>>> >> Any thoughts on how to fix this ?
>>>> >>
>>>> >>
>>>> >> Thanks,
>>>> >> Anurag Tangri
>>>> >>
>>>> >>
>>>> >>
>>>> >>
>>>> >>
>>>> >>
>>>> >> =====
>>>> >> 2014-06-04 07:15:13,211 WARN  file.HadoopGroupMappingService
>>>> (HadoopGroupMappingService.java:getGroups(42)) - Unable to obtain groups
>>>> for atangri@group.on
>>>> >> java.io.IOException: No groups found for user atangri@group.on
>>>> >>
>>>> >>
>>>> >>> On Tue, Jun 3, 2014 at 12:49 PM, Anurag <at...@groupon.com>
>>>> wrote:
>>>> >>> Thanks Philippe!
>>>> >>>
>>>> >>> This answers almost all of my questions.
>>>> >>>
>>>> >>> Thanks,
>>>> >>> Anurag Tangri
>>>> >>>
>>>> >>>> On Jun 3, 2014, at 12:08 PM, Philippe Marseille <
>>>> marseilp@gmail.com> wrote:
>>>> >>>>
>>>> >>>> I think the Hive CLI (Command line interface tool) is now
>>>> deprecated.  see
>>>> http://blog.cloudera.com/blog/2014/02/migrating-from-hive-cli-to-beeline-a-primer/
>>>> >>>>
>>>> >>>>
>>>> >>>> The metastore has nothing to do with it.  The Metastore Daemon is
>>>> used by all clients.
>>>> >>>>
>>>> >>>> The new flow is :
>>>> >>>>
>>>> >>>> Beeline -> HiveServer2 -> Metastore Daemon -> Metastore Database
>>>> (MySql for example).
>>>> >>>>
>>>> >>>> The previous flow was :
>>>> >>>>
>>>> >>>> Hive CLI (shell ) -> Metastore Daemon -> Metastore Database
>>>> >>>>
>>>> >>>> And before the Metastore Daemon ever existed, it was :
>>>> >>>>
>>>> >>>> Hive CLI -> Metastore Database
>>>> >>>>
>>>> >>>>> On Tuesday, 3 June 2014 13:06:22 UTC-4, Anurag Tangri wrote:
>>>> >>>>> Thanks Casey.
>>>> >>>>>
>>>> >>>>> This is going to be big shift to ask all our users  to go to
>>>> beeline from hive.
>>>> >>>>>
>>>> >>>>> Is there no way to have consistency between what roles and
>>>> policies are seen in HUE via hiveserver2 and hive metastore on hive shell ?
>>>> >>>>> Also, is there a reason why hive metastore was excluded from
>>>> Sentry design ?
>>>> >>>>>
>>>> >>>>>
>>>> >>>>>
>>>> >>>>> Thanks,
>>>> >>>>> Anurag Tangri
>>>> >>>>>
>>>> >>>>>
>>>> >>>>>
>>>> >>>>>
>>>> >>>>>> On Tue, Jun 3, 2014 at 9:54 AM, Casey Brotherton <
>>>> cbrot...@cloudera.com> wrote:
>>>> >>>>>> Hello Anurag,
>>>> >>>>>>
>>>> >>>>>> To use Sentry, as Johndee mentioned, you must use Hiveserver2.
>>>>  That means you will need to transition users from the hive command line to
>>>> the beeline command line.
>>>> >>>>>>
>>>> >>>>>> This link has more information for CDH4.5
>>>> >>>>>>
>>>> http://www.cloudera.com/content/cloudera-content/cloudera-docs/CDH4/4.5.0/CDH4-Security-Guide/cdh4sg_Sentry.html
>>>> >>>>>>
>>>> >>>>>> There is a link to "Securing the Hive Metastore"  which suggests
>>>> to limit connections using iptables, and granting of permissions.
>>>> >>>>>>
>>>> >>>>>> Hope this helps,
>>>> >>>>>> Casey
>>>> >>>>>>
>>>> >>>>>>
>>>> >>>>>>
>>>> >>>>>>> On Tue, Jun 3, 2014 at 11:45 AM, Anurag Tangri <
>>>> ata...@groupon.com> wrote:
>>>> >>>>>>> Hey Philippe,
>>>> >>>>>>> No, HUE should be fine as per Johndee's explanation as it can
>>>> be pointed to hiveserver2.
>>>> >>>>>>>
>>>> >>>>>>> By metastore, I mean mysql database that hosts hive metadata.
>>>> >>>>>>>
>>>> >>>>>>> This is the database that people go against when they login to
>>>> hive shell for a hadoop client box.
>>>> >>>>>>>
>>>> >>>>>>>
>>>> >>>>>>> Thanks,
>>>> >>>>>>> Anurag Tangri
>>>> >>>>>>>
>>>> >>>>>>>
>>>> >>>>>>>
>>>> >>>>>>>> On Tue, Jun 3, 2014 at 8:11 AM, Philippe Marseille <
>>>> mars...@gmail.com> wrote:
>>>> >>>>>>>> By "Hive Metastore", are you referring to the app in Hue?
>>>> >>>>>>>>
>>>> >>>>>>>>
>>>> >>>>>>>>> On Tuesday, 3 June 2014 00:32:53 UTC-4, Anurag Tangri wrote:
>>>> >>>>>>>>> Thanks Johndee.
>>>> >>>>>>>>>
>>>> >>>>>>>>> Can you point to some documentation on hive group and
>>>> rejecting requests from metastore ?
>>>> >>>>>>>>>
>>>> >>>>>>>>> Hiveserver2 is mainly for jdbc interface, hue etc.
>>>> >>>>>>>>>
>>>> >>>>>>>>> For people going via hive metastore, we will need to enforce
>>>> same set of roles and policies as via hiveserver2.
>>>> >>>>>>>>>
>>>> >>>>>>>>> Thanks,
>>>> >>>>>>>>> Anurag Tangri
>>>> >>>>>>>>>
>>>> >>>>>>>>>> On Jun 2, 2014, at 7:57 PM, Johndee Cloudera <
>>>> joh...@cloudera.com> wrote:
>>>> >>>>>>>>>>
>>>> >>>>>>>>>> When it comes to hive and sentry you need at least Hive
>>>> Server 2. Hive Server 2 is what actually runs the Sentry daemon. The Hive
>>>> Metastore Server handles metadata requests, and in the setup guide is
>>>> configured to reject all connections from users not in the hive group
>>>> typically.
>>>> >>>>>>>>>>
>>>> >>>>>>>>>>
>>>> >>>>>>>>>>> On Mon, Jun 2, 2014 at 10:15 PM, Anurag Tangri <
>>>> tangri...@gmail.com> wrote:
>>>> >>>>>>>>>>> Hi all,
>>>> >>>>>>>>>>> From what I read everywhere, sentry is to be used in
>>>> conjunction with hiveserver2.
>>>> >>>>>>>>>>>
>>>> >>>>>>>>>>> Is it true that it can be used with hive metastore ? If
>>>> yes, Any links explaining how to do so would be great.
>>>> >>>>>>>>>>>
>>>> >>>>>>>>>>> Thanks,
>>>> >>>>>>>>>>> Anurag Tangri
>>>> >>>>>>>>>>>
>>>> >>>>>>>>>>> --
>>>> >>>>>>>>>>>
>>>> >>>>>>>>>>> ---
>>>> >>>>>>>>>>> You received this message because you are subscribed to the
>>>> Google Groups "CDH Users" group.
>>>> >>>>>>>>>>> To unsubscribe from this group and stop receiving emails
>>>> from it, send an email to cdh-user+u...@cloudera.org.
>>>> >>>>>>>>>>>
>>>> >>>>>>>>>>> For more options, visit
>>>> https://groups.google.com/a/cloudera.org/d/optout.
>>>> >>>>>>>>>>
>>>> >>>>>>>>>>
>>>> >>>>>>>>>>
>>>> >>>>>>>>>> --
>>>> >>>>>>>>>> - JRB
>>>> >>>>>>>>>> --
>>>> >>>>>>>>>>
>>>> >>>>>>>>>> ---
>>>> >>>>>>>>>> You received this message because you are subscribed to the
>>>> Google Groups "CDH Users" group.
>>>> >>>>>>>>>> To unsubscribe from this group and stop receiving emails
>>>> from it, send an email to cdh-user+u...@cloudera.org.
>>>> >>>>>>>>>>
>>>> >>>>>>>>>> For more options, visit
>>>> https://groups.google.com/a/cloudera.org/d/optout.
>>>> >>>>>>>>
>>>> >>>>>>>> --
>>>> >>>>>>>>
>>>> >>>>>>>> ---
>>>> >>>>>>>> You received this message because you are subscribed to the
>>>> Google Groups "CDH Users" group.
>>>> >>>>>>>> To unsubscribe from this group and stop receiving emails from
>>>> it, send an email to cdh-user+u...@cloudera.org.
>>>> >>>>>>>> For more options, visit
>>>> https://groups.google.com/a/cloudera.org/d/optout.
>>>> >>>>>>>
>>>> >>>>>>> --
>>>> >>>>>>>
>>>> >>>>>>> ---
>>>> >>>>>>> You received this message because you are subscribed to the
>>>> Google Groups "CDH Users" group.
>>>> >>>>>>> To unsubscribe from this group and stop receiving emails from
>>>> it, send an email to cdh-user+u...@cloudera.org.
>>>> >>>>>>> For more options, visit
>>>> https://groups.google.com/a/cloudera.org/d/optout.
>>>> >>>>>>
>>>> >>>>>>
>>>> >>>>>>
>>>> >>>>>> --
>>>> >>>>>> Casey J. Brotherton
>>>> >>>>>> Customer Operations Engineer
>>>> >>>>>>
>>>> >>>>>> --
>>>> >>>>>>
>>>> >>>>>> ---
>>>> >>>>>> You received this message because you are subscribed to the
>>>> Google Groups "CDH Users" group.
>>>> >>>>>> To unsubscribe from this group and stop receiving emails from
>>>> it, send an email to cdh-user+u...@cloudera.org.
>>>> >>>>>> For more options, visit
>>>> https://groups.google.com/a/cloudera.org/d/optout.
>>>> >>>>
>>>> >>>> --
>>>> >>>>
>>>> >>>> ---
>>>> >>>> You received this message because you are subscribed to the Google
>>>> Groups "CDH Users" group.
>>>> >>>> To unsubscribe from this group and stop receiving emails from it,
>>>> send an email to cdh-user+unsubscribe@cloudera.org.
>>>> >>>> For more options, visit
>>>> https://groups.google.com/a/cloudera.org/d/optout.
>>>> >
>>>> > --
>>>> >
>>>> > ---
>>>> > You received this message because you are subscribed to the Google
>>>> Groups "CDH Users" group.
>>>> > To unsubscribe from this group and stop receiving emails from it,
>>>> send an email to cdh-user+unsubscribe@cloudera.org.
>>>> > For more options, visit
>>>> https://groups.google.com/a/cloudera.org/d/optout.
>>>>
>>>
>>>  --
>>>
>>> ---
>>> You received this message because you are subscribed to the Google
>>> Groups "CDH Users" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to cdh-user+unsubscribe@cloudera.org.
>>> For more options, visit
>>> https://groups.google.com/a/cloudera.org/d/optout.
>>>
>>
>>
>

Re: Apache Sentry works with hive metastore ?

Posted by Anurag Tangri <at...@groupon.com>.
Hi Prasad,
I tried it and the setting hive.server2.authentication.ldap.Domain does not
change anything.

Thanks,
Anurag Tangri


On Thu, Jun 5, 2014 at 10:45 AM, Anurag Tangri <at...@groupon.com> wrote:

> That's correct Prasad.
>
> Let me try this and see if it works.
>
>
> Thanks,
> Anurag Tangri
>
>
> On Thu, Jun 5, 2014 at 10:35 AM, Prasad Mujumdar <pr...@cloudera.com>
> wrote:
>
>> Hi Anurag,
>>
>>    If this is active directory, then try setting a config property
>> hive.server2.authentication.ldap.Domain to your LDAP domain (a.b). Then you
>> can login with the short userid atangri.
>> Are you using LDAP group mapping in Hadoop  as well ?
>>
>> thanks
>> Prasad
>>
>>
>>
>> On Thu, Jun 5, 2014 at 7:43 AM, Anurag <at...@groupon.com> wrote:
>>
>>>
>>> Any response anyone?
>>>
>>> Sent from my iPhone
>>> > On Jun 4, 2014, at 3:20 PM, Anurag Tangri <at...@groupon.com> wrote:
>>> >
>>> > + sentry mailing list
>>> >
>>> >
>>> >> On Wed, Jun 4, 2014 at 12:27 AM, Anurag Tangri <at...@groupon.com>
>>> wrote:
>>> >> So, I am working on setting this up.
>>> >>
>>> >> I have HDFS integrated with LDAP and working fine.
>>> >>
>>> >> I also added ldap params in hive-site along with sentry conf :
>>> >>
>>> >> ====
>>> >>   <property>
>>> >>     <name>hive.server2.authentication</name>
>>> >>     <value>LDAP</value>
>>> >>   </property>
>>> >>   <property>
>>> >>     <name>hive.server2.authentication.ldap.url</name>
>>> >>     <value>ldap://<ip></value>
>>> >>   </property>
>>> >>
>>> >> <property>
>>> >>   <name>hive.server2.authentication.ldap.baseDN</name>
>>> >>   <value>dc=a,dc=b</value>
>>> >> </property>
>>> >>
>>> >> ======
>>> >>
>>> >> Now when I go to beeline, I type:
>>> >>
>>> >> /usr/local/lib/hive/bin/beeline -u jdbc:hive2://<host>:10000
>>> >> Beeline version 0.10.0-cdh4.6.0 by Apache Hive
>>> >> 0: jdbc:hive2://<host>:10000> !connect jdbc:hive2://<host>:10000
>>> >>
>>> >>
>>> >> It asks for username and password. I give LDAP credentials as:
>>> >>
>>> >> Connecting to jdbc:hive2://<host>:10000
>>> >> Enter username for jdbc:hive2://<host>:10000: atangri@a.b
>>> >> Enter password for jdbc:hive2://<host>:10000: **********
>>> >> Error: Invalid URL: jdbc:hive2://<host>:10000 (state=08S01,code=0)
>>> >> 1: jdbc:hive2://<host>:10000>
>>> >>
>>> >>
>>> >> Now if I type, hive queries, i see error in hive log:
>>> >>
>>> >> 2014-06-04 07:15:13,211 WARN  file.HadoopGroupMappingService
>>> (HadoopGroupMappingService.java:getGroups(42)) - Unable to obtain groups
>>> for atangri@a.b
>>> >> java.io.IOException: No groups found for user atangri@a.b
>>> >>     at org.apache.hadoop.security.Groups.getGroups(Groups.java:105)
>>> >>     at
>>> org.apache.sentry.provider.file.HadoopGroupMappingService.getGroups(HadoopGroupMappingService.java:40)
>>> >>     at
>>> org.apache.sentry.provider.file.ResourceAuthorizationProvider.doHasAccess(ResourceAuthorizationProvider.java:98)
>>> >>     at
>>> org.apache.sentry.provider.file.ResourceAuthorizationProvider.hasAccess(ResourceAuthorizationProvider.java:93)
>>> >>     at
>>> org.apache.sentry.binding.hive.authz.HiveAuthzBinding.authorize(HiveAuthzBinding.java:179)
>>> >>     at
>>> org.apache.sentry.binding.hive.HiveAuthzBindingHook.filterShowTables(HiveAuthzBindingHook.java:561)
>>> >>     at
>>> org.apache.sentry.binding.hive.HiveAuthzBindingHook.postDriverFetch(HiveAuthzBindingHook.java:634)
>>> >>     at
>>> org.apache.hadoop.hive.ql.Driver.fireFilterHooks(Driver.java:1455)
>>> >>     at org.apache.hadoop.hive.ql.Driver.getResults(Driver.java:1475)
>>> >>     at
>>> org.apache.hive.service.cli.operation.SQLOperation.getNextRowSet(SQLOperation.java:200)
>>> >>     at
>>> org.apache.hive.service.cli.operation.OperationManager.getOperationNextRowSet(OperationManager.java:179)
>>> >>     at
>>> org.apache.hive.service.cli.session.HiveSessionImpl.fetchResults(HiveSessionImpl.java:468)
>>> >>     at
>>> org.apache.hive.service.cli.CLIService.fetchResults(CLIService.java:318)
>>> >>     at
>>> org.apache.hive.service.cli.thrift.ThriftCLIService.FetchResults(ThriftCLIService.java:398)
>>> >>     at
>>> org.apache.hive.service.cli.thrift.TCLIService$Processor$FetchResults.getResult(TCLIService.java:1613)
>>> >>     at
>>> org.apache.hive.service.cli.thrift.TCLIService$Processor$FetchResults.getResult(TCLIService.java:1598)
>>> >>     at
>>> org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
>>> >>     at
>>> org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
>>> >>     at
>>> org.apache.hive.service.cli.thrift.TSetIpAddressProcessor.process(TSetIpAddressProcessor.java:38)
>>> >>     at
>>> org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:244)
>>> >>     at
>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>>> >>     at
>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>>> >>     at java.lang.Thread.run(Thread.java:722)
>>> >> =======
>>> >>
>>> >>
>>> >> I think the issue is my account on LDAP is atangri, not atangri@a.b.
>>> >>
>>> >> Any thoughts on how to fix this ?
>>> >>
>>> >>
>>> >> Thanks,
>>> >> Anurag Tangri
>>> >>
>>> >>
>>> >>
>>> >>
>>> >>
>>> >>
>>> >> =====
>>> >> 2014-06-04 07:15:13,211 WARN  file.HadoopGroupMappingService
>>> (HadoopGroupMappingService.java:getGroups(42)) - Unable to obtain groups
>>> for atangri@group.on
>>> >> java.io.IOException: No groups found for user atangri@group.on
>>> >>
>>> >>
>>> >>> On Tue, Jun 3, 2014 at 12:49 PM, Anurag <at...@groupon.com> wrote:
>>> >>> Thanks Philippe!
>>> >>>
>>> >>> This answers almost all of my questions.
>>> >>>
>>> >>> Thanks,
>>> >>> Anurag Tangri
>>> >>>
>>> >>>> On Jun 3, 2014, at 12:08 PM, Philippe Marseille <ma...@gmail.com>
>>> wrote:
>>> >>>>
>>> >>>> I think the Hive CLI (Command line interface tool) is now
>>> deprecated.  see
>>> http://blog.cloudera.com/blog/2014/02/migrating-from-hive-cli-to-beeline-a-primer/
>>> >>>>
>>> >>>>
>>> >>>> The metastore has nothing to do with it.  The Metastore Daemon is
>>> used by all clients.
>>> >>>>
>>> >>>> The new flow is :
>>> >>>>
>>> >>>> Beeline -> HiveServer2 -> Metastore Daemon -> Metastore Database
>>> (MySql for example).
>>> >>>>
>>> >>>> The previous flow was :
>>> >>>>
>>> >>>> Hive CLI (shell ) -> Metastore Daemon -> Metastore Database
>>> >>>>
>>> >>>> And before the Metastore Daemon ever existed, it was :
>>> >>>>
>>> >>>> Hive CLI -> Metastore Database
>>> >>>>
>>> >>>>> On Tuesday, 3 June 2014 13:06:22 UTC-4, Anurag Tangri wrote:
>>> >>>>> Thanks Casey.
>>> >>>>>
>>> >>>>> This is going to be big shift to ask all our users  to go to
>>> beeline from hive.
>>> >>>>>
>>> >>>>> Is there no way to have consistency between what roles and
>>> policies are seen in HUE via hiveserver2 and hive metastore on hive shell ?
>>> >>>>> Also, is there a reason why hive metastore was excluded from
>>> Sentry design ?
>>> >>>>>
>>> >>>>>
>>> >>>>>
>>> >>>>> Thanks,
>>> >>>>> Anurag Tangri
>>> >>>>>
>>> >>>>>
>>> >>>>>
>>> >>>>>
>>> >>>>>> On Tue, Jun 3, 2014 at 9:54 AM, Casey Brotherton <
>>> cbrot...@cloudera.com> wrote:
>>> >>>>>> Hello Anurag,
>>> >>>>>>
>>> >>>>>> To use Sentry, as Johndee mentioned, you must use Hiveserver2.
>>>  That means you will need to transition users from the hive command line to
>>> the beeline command line.
>>> >>>>>>
>>> >>>>>> This link has more information for CDH4.5
>>> >>>>>>
>>> http://www.cloudera.com/content/cloudera-content/cloudera-docs/CDH4/4.5.0/CDH4-Security-Guide/cdh4sg_Sentry.html
>>> >>>>>>
>>> >>>>>> There is a link to "Securing the Hive Metastore"  which suggests
>>> to limit connections using iptables, and granting of permissions.
>>> >>>>>>
>>> >>>>>> Hope this helps,
>>> >>>>>> Casey
>>> >>>>>>
>>> >>>>>>
>>> >>>>>>
>>> >>>>>>> On Tue, Jun 3, 2014 at 11:45 AM, Anurag Tangri <
>>> ata...@groupon.com> wrote:
>>> >>>>>>> Hey Philippe,
>>> >>>>>>> No, HUE should be fine as per Johndee's explanation as it can be
>>> pointed to hiveserver2.
>>> >>>>>>>
>>> >>>>>>> By metastore, I mean mysql database that hosts hive metadata.
>>> >>>>>>>
>>> >>>>>>> This is the database that people go against when they login to
>>> hive shell for a hadoop client box.
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>> Thanks,
>>> >>>>>>> Anurag Tangri
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>>> On Tue, Jun 3, 2014 at 8:11 AM, Philippe Marseille <
>>> mars...@gmail.com> wrote:
>>> >>>>>>>> By "Hive Metastore", are you referring to the app in Hue?
>>> >>>>>>>>
>>> >>>>>>>>
>>> >>>>>>>>> On Tuesday, 3 June 2014 00:32:53 UTC-4, Anurag Tangri wrote:
>>> >>>>>>>>> Thanks Johndee.
>>> >>>>>>>>>
>>> >>>>>>>>> Can you point to some documentation on hive group and
>>> rejecting requests from metastore ?
>>> >>>>>>>>>
>>> >>>>>>>>> Hiveserver2 is mainly for jdbc interface, hue etc.
>>> >>>>>>>>>
>>> >>>>>>>>> For people going via hive metastore, we will need to enforce
>>> same set of roles and policies as via hiveserver2.
>>> >>>>>>>>>
>>> >>>>>>>>> Thanks,
>>> >>>>>>>>> Anurag Tangri
>>> >>>>>>>>>
>>> >>>>>>>>>> On Jun 2, 2014, at 7:57 PM, Johndee Cloudera <
>>> joh...@cloudera.com> wrote:
>>> >>>>>>>>>>
>>> >>>>>>>>>> When it comes to hive and sentry you need at least Hive
>>> Server 2. Hive Server 2 is what actually runs the Sentry daemon. The Hive
>>> Metastore Server handles metadata requests, and in the setup guide is
>>> configured to reject all connections from users not in the hive group
>>> typically.
>>> >>>>>>>>>>
>>> >>>>>>>>>>
>>> >>>>>>>>>>> On Mon, Jun 2, 2014 at 10:15 PM, Anurag Tangri <
>>> tangri...@gmail.com> wrote:
>>> >>>>>>>>>>> Hi all,
>>> >>>>>>>>>>> From what I read everywhere, sentry is to be used in
>>> conjunction with hiveserver2.
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> Is it true that it can be used with hive metastore ? If yes,
>>> Any links explaining how to do so would be great.
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> Thanks,
>>> >>>>>>>>>>> Anurag Tangri
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> --
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> ---
>>> >>>>>>>>>>> You received this message because you are subscribed to the
>>> Google Groups "CDH Users" group.
>>> >>>>>>>>>>> To unsubscribe from this group and stop receiving emails
>>> from it, send an email to cdh-user+u...@cloudera.org.
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> For more options, visit
>>> https://groups.google.com/a/cloudera.org/d/optout.
>>> >>>>>>>>>>
>>> >>>>>>>>>>
>>> >>>>>>>>>>
>>> >>>>>>>>>> --
>>> >>>>>>>>>> - JRB
>>> >>>>>>>>>> --
>>> >>>>>>>>>>
>>> >>>>>>>>>> ---
>>> >>>>>>>>>> You received this message because you are subscribed to the
>>> Google Groups "CDH Users" group.
>>> >>>>>>>>>> To unsubscribe from this group and stop receiving emails from
>>> it, send an email to cdh-user+u...@cloudera.org.
>>> >>>>>>>>>>
>>> >>>>>>>>>> For more options, visit
>>> https://groups.google.com/a/cloudera.org/d/optout.
>>> >>>>>>>>
>>> >>>>>>>> --
>>> >>>>>>>>
>>> >>>>>>>> ---
>>> >>>>>>>> You received this message because you are subscribed to the
>>> Google Groups "CDH Users" group.
>>> >>>>>>>> To unsubscribe from this group and stop receiving emails from
>>> it, send an email to cdh-user+u...@cloudera.org.
>>> >>>>>>>> For more options, visit
>>> https://groups.google.com/a/cloudera.org/d/optout.
>>> >>>>>>>
>>> >>>>>>> --
>>> >>>>>>>
>>> >>>>>>> ---
>>> >>>>>>> You received this message because you are subscribed to the
>>> Google Groups "CDH Users" group.
>>> >>>>>>> To unsubscribe from this group and stop receiving emails from
>>> it, send an email to cdh-user+u...@cloudera.org.
>>> >>>>>>> For more options, visit
>>> https://groups.google.com/a/cloudera.org/d/optout.
>>> >>>>>>
>>> >>>>>>
>>> >>>>>>
>>> >>>>>> --
>>> >>>>>> Casey J. Brotherton
>>> >>>>>> Customer Operations Engineer
>>> >>>>>>
>>> >>>>>> --
>>> >>>>>>
>>> >>>>>> ---
>>> >>>>>> You received this message because you are subscribed to the
>>> Google Groups "CDH Users" group.
>>> >>>>>> To unsubscribe from this group and stop receiving emails from it,
>>> send an email to cdh-user+u...@cloudera.org.
>>> >>>>>> For more options, visit
>>> https://groups.google.com/a/cloudera.org/d/optout.
>>> >>>>
>>> >>>> --
>>> >>>>
>>> >>>> ---
>>> >>>> You received this message because you are subscribed to the Google
>>> Groups "CDH Users" group.
>>> >>>> To unsubscribe from this group and stop receiving emails from it,
>>> send an email to cdh-user+unsubscribe@cloudera.org.
>>> >>>> For more options, visit
>>> https://groups.google.com/a/cloudera.org/d/optout.
>>> >
>>> > --
>>> >
>>> > ---
>>> > You received this message because you are subscribed to the Google
>>> Groups "CDH Users" group.
>>> > To unsubscribe from this group and stop receiving emails from it, send
>>> an email to cdh-user+unsubscribe@cloudera.org.
>>> > For more options, visit
>>> https://groups.google.com/a/cloudera.org/d/optout.
>>>
>>
>>  --
>>
>> ---
>> You received this message because you are subscribed to the Google Groups
>> "CDH Users" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to cdh-user+unsubscribe@cloudera.org.
>> For more options, visit https://groups.google.com/a/cloudera.org/d/optout
>> .
>>
>
>

Re: Apache Sentry works with hive metastore ?

Posted by Anurag Tangri <at...@groupon.com>.
That's correct Prasad.

Let me try this and see if it works.


Thanks,
Anurag Tangri


On Thu, Jun 5, 2014 at 10:35 AM, Prasad Mujumdar <pr...@cloudera.com>
wrote:

> Hi Anurag,
>
>    If this is active directory, then try setting a config property
> hive.server2.authentication.ldap.Domain to your LDAP domain (a.b). Then you
> can login with the short userid atangri.
> Are you using LDAP group mapping in Hadoop  as well ?
>
> thanks
> Prasad
>
>
>
> On Thu, Jun 5, 2014 at 7:43 AM, Anurag <at...@groupon.com> wrote:
>
>>
>> Any response anyone?
>>
>> Sent from my iPhone
>> > On Jun 4, 2014, at 3:20 PM, Anurag Tangri <at...@groupon.com> wrote:
>> >
>> > + sentry mailing list
>> >
>> >
>> >> On Wed, Jun 4, 2014 at 12:27 AM, Anurag Tangri <at...@groupon.com>
>> wrote:
>> >> So, I am working on setting this up.
>> >>
>> >> I have HDFS integrated with LDAP and working fine.
>> >>
>> >> I also added ldap params in hive-site along with sentry conf :
>> >>
>> >> ====
>> >>   <property>
>> >>     <name>hive.server2.authentication</name>
>> >>     <value>LDAP</value>
>> >>   </property>
>> >>   <property>
>> >>     <name>hive.server2.authentication.ldap.url</name>
>> >>     <value>ldap://<ip></value>
>> >>   </property>
>> >>
>> >> <property>
>> >>   <name>hive.server2.authentication.ldap.baseDN</name>
>> >>   <value>dc=a,dc=b</value>
>> >> </property>
>> >>
>> >> ======
>> >>
>> >> Now when I go to beeline, I type:
>> >>
>> >> /usr/local/lib/hive/bin/beeline -u jdbc:hive2://<host>:10000
>> >> Beeline version 0.10.0-cdh4.6.0 by Apache Hive
>> >> 0: jdbc:hive2://<host>:10000> !connect jdbc:hive2://<host>:10000
>> >>
>> >>
>> >> It asks for username and password. I give LDAP credentials as:
>> >>
>> >> Connecting to jdbc:hive2://<host>:10000
>> >> Enter username for jdbc:hive2://<host>:10000: atangri@a.b
>> >> Enter password for jdbc:hive2://<host>:10000: **********
>> >> Error: Invalid URL: jdbc:hive2://<host>:10000 (state=08S01,code=0)
>> >> 1: jdbc:hive2://<host>:10000>
>> >>
>> >>
>> >> Now if I type, hive queries, i see error in hive log:
>> >>
>> >> 2014-06-04 07:15:13,211 WARN  file.HadoopGroupMappingService
>> (HadoopGroupMappingService.java:getGroups(42)) - Unable to obtain groups
>> for atangri@a.b
>> >> java.io.IOException: No groups found for user atangri@a.b
>> >>     at org.apache.hadoop.security.Groups.getGroups(Groups.java:105)
>> >>     at
>> org.apache.sentry.provider.file.HadoopGroupMappingService.getGroups(HadoopGroupMappingService.java:40)
>> >>     at
>> org.apache.sentry.provider.file.ResourceAuthorizationProvider.doHasAccess(ResourceAuthorizationProvider.java:98)
>> >>     at
>> org.apache.sentry.provider.file.ResourceAuthorizationProvider.hasAccess(ResourceAuthorizationProvider.java:93)
>> >>     at
>> org.apache.sentry.binding.hive.authz.HiveAuthzBinding.authorize(HiveAuthzBinding.java:179)
>> >>     at
>> org.apache.sentry.binding.hive.HiveAuthzBindingHook.filterShowTables(HiveAuthzBindingHook.java:561)
>> >>     at
>> org.apache.sentry.binding.hive.HiveAuthzBindingHook.postDriverFetch(HiveAuthzBindingHook.java:634)
>> >>     at
>> org.apache.hadoop.hive.ql.Driver.fireFilterHooks(Driver.java:1455)
>> >>     at org.apache.hadoop.hive.ql.Driver.getResults(Driver.java:1475)
>> >>     at
>> org.apache.hive.service.cli.operation.SQLOperation.getNextRowSet(SQLOperation.java:200)
>> >>     at
>> org.apache.hive.service.cli.operation.OperationManager.getOperationNextRowSet(OperationManager.java:179)
>> >>     at
>> org.apache.hive.service.cli.session.HiveSessionImpl.fetchResults(HiveSessionImpl.java:468)
>> >>     at
>> org.apache.hive.service.cli.CLIService.fetchResults(CLIService.java:318)
>> >>     at
>> org.apache.hive.service.cli.thrift.ThriftCLIService.FetchResults(ThriftCLIService.java:398)
>> >>     at
>> org.apache.hive.service.cli.thrift.TCLIService$Processor$FetchResults.getResult(TCLIService.java:1613)
>> >>     at
>> org.apache.hive.service.cli.thrift.TCLIService$Processor$FetchResults.getResult(TCLIService.java:1598)
>> >>     at
>> org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
>> >>     at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
>> >>     at
>> org.apache.hive.service.cli.thrift.TSetIpAddressProcessor.process(TSetIpAddressProcessor.java:38)
>> >>     at
>> org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:244)
>> >>     at
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>> >>     at
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>> >>     at java.lang.Thread.run(Thread.java:722)
>> >> =======
>> >>
>> >>
>> >> I think the issue is my account on LDAP is atangri, not atangri@a.b.
>> >>
>> >> Any thoughts on how to fix this ?
>> >>
>> >>
>> >> Thanks,
>> >> Anurag Tangri
>> >>
>> >>
>> >>
>> >>
>> >>
>> >>
>> >> =====
>> >> 2014-06-04 07:15:13,211 WARN  file.HadoopGroupMappingService
>> (HadoopGroupMappingService.java:getGroups(42)) - Unable to obtain groups
>> for atangri@group.on
>> >> java.io.IOException: No groups found for user atangri@group.on
>> >>
>> >>
>> >>> On Tue, Jun 3, 2014 at 12:49 PM, Anurag <at...@groupon.com> wrote:
>> >>> Thanks Philippe!
>> >>>
>> >>> This answers almost all of my questions.
>> >>>
>> >>> Thanks,
>> >>> Anurag Tangri
>> >>>
>> >>>> On Jun 3, 2014, at 12:08 PM, Philippe Marseille <ma...@gmail.com>
>> wrote:
>> >>>>
>> >>>> I think the Hive CLI (Command line interface tool) is now
>> deprecated.  see
>> http://blog.cloudera.com/blog/2014/02/migrating-from-hive-cli-to-beeline-a-primer/
>> >>>>
>> >>>>
>> >>>> The metastore has nothing to do with it.  The Metastore Daemon is
>> used by all clients.
>> >>>>
>> >>>> The new flow is :
>> >>>>
>> >>>> Beeline -> HiveServer2 -> Metastore Daemon -> Metastore Database
>> (MySql for example).
>> >>>>
>> >>>> The previous flow was :
>> >>>>
>> >>>> Hive CLI (shell ) -> Metastore Daemon -> Metastore Database
>> >>>>
>> >>>> And before the Metastore Daemon ever existed, it was :
>> >>>>
>> >>>> Hive CLI -> Metastore Database
>> >>>>
>> >>>>> On Tuesday, 3 June 2014 13:06:22 UTC-4, Anurag Tangri wrote:
>> >>>>> Thanks Casey.
>> >>>>>
>> >>>>> This is going to be big shift to ask all our users  to go to
>> beeline from hive.
>> >>>>>
>> >>>>> Is there no way to have consistency between what roles and policies
>> are seen in HUE via hiveserver2 and hive metastore on hive shell ?
>> >>>>> Also, is there a reason why hive metastore was excluded from Sentry
>> design ?
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>> Thanks,
>> >>>>> Anurag Tangri
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>>> On Tue, Jun 3, 2014 at 9:54 AM, Casey Brotherton <
>> cbrot...@cloudera.com> wrote:
>> >>>>>> Hello Anurag,
>> >>>>>>
>> >>>>>> To use Sentry, as Johndee mentioned, you must use Hiveserver2.
>>  That means you will need to transition users from the hive command line to
>> the beeline command line.
>> >>>>>>
>> >>>>>> This link has more information for CDH4.5
>> >>>>>>
>> http://www.cloudera.com/content/cloudera-content/cloudera-docs/CDH4/4.5.0/CDH4-Security-Guide/cdh4sg_Sentry.html
>> >>>>>>
>> >>>>>> There is a link to "Securing the Hive Metastore"  which suggests
>> to limit connections using iptables, and granting of permissions.
>> >>>>>>
>> >>>>>> Hope this helps,
>> >>>>>> Casey
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>>> On Tue, Jun 3, 2014 at 11:45 AM, Anurag Tangri <
>> ata...@groupon.com> wrote:
>> >>>>>>> Hey Philippe,
>> >>>>>>> No, HUE should be fine as per Johndee's explanation as it can be
>> pointed to hiveserver2.
>> >>>>>>>
>> >>>>>>> By metastore, I mean mysql database that hosts hive metadata.
>> >>>>>>>
>> >>>>>>> This is the database that people go against when they login to
>> hive shell for a hadoop client box.
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> Thanks,
>> >>>>>>> Anurag Tangri
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>> On Tue, Jun 3, 2014 at 8:11 AM, Philippe Marseille <
>> mars...@gmail.com> wrote:
>> >>>>>>>> By "Hive Metastore", are you referring to the app in Hue?
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>>>>> On Tuesday, 3 June 2014 00:32:53 UTC-4, Anurag Tangri wrote:
>> >>>>>>>>> Thanks Johndee.
>> >>>>>>>>>
>> >>>>>>>>> Can you point to some documentation on hive group and rejecting
>> requests from metastore ?
>> >>>>>>>>>
>> >>>>>>>>> Hiveserver2 is mainly for jdbc interface, hue etc.
>> >>>>>>>>>
>> >>>>>>>>> For people going via hive metastore, we will need to enforce
>> same set of roles and policies as via hiveserver2.
>> >>>>>>>>>
>> >>>>>>>>> Thanks,
>> >>>>>>>>> Anurag Tangri
>> >>>>>>>>>
>> >>>>>>>>>> On Jun 2, 2014, at 7:57 PM, Johndee Cloudera <
>> joh...@cloudera.com> wrote:
>> >>>>>>>>>>
>> >>>>>>>>>> When it comes to hive and sentry you need at least Hive Server
>> 2. Hive Server 2 is what actually runs the Sentry daemon. The Hive
>> Metastore Server handles metadata requests, and in the setup guide is
>> configured to reject all connections from users not in the hive group
>> typically.
>> >>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>>> On Mon, Jun 2, 2014 at 10:15 PM, Anurag Tangri <
>> tangri...@gmail.com> wrote:
>> >>>>>>>>>>> Hi all,
>> >>>>>>>>>>> From what I read everywhere, sentry is to be used in
>> conjunction with hiveserver2.
>> >>>>>>>>>>>
>> >>>>>>>>>>> Is it true that it can be used with hive metastore ? If yes,
>> Any links explaining how to do so would be great.
>> >>>>>>>>>>>
>> >>>>>>>>>>> Thanks,
>> >>>>>>>>>>> Anurag Tangri
>> >>>>>>>>>>>
>> >>>>>>>>>>> --
>> >>>>>>>>>>>
>> >>>>>>>>>>> ---
>> >>>>>>>>>>> You received this message because you are subscribed to the
>> Google Groups "CDH Users" group.
>> >>>>>>>>>>> To unsubscribe from this group and stop receiving emails from
>> it, send an email to cdh-user+u...@cloudera.org.
>> >>>>>>>>>>>
>> >>>>>>>>>>> For more options, visit
>> https://groups.google.com/a/cloudera.org/d/optout.
>> >>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>> --
>> >>>>>>>>>> - JRB
>> >>>>>>>>>> --
>> >>>>>>>>>>
>> >>>>>>>>>> ---
>> >>>>>>>>>> You received this message because you are subscribed to the
>> Google Groups "CDH Users" group.
>> >>>>>>>>>> To unsubscribe from this group and stop receiving emails from
>> it, send an email to cdh-user+u...@cloudera.org.
>> >>>>>>>>>>
>> >>>>>>>>>> For more options, visit
>> https://groups.google.com/a/cloudera.org/d/optout.
>> >>>>>>>>
>> >>>>>>>> --
>> >>>>>>>>
>> >>>>>>>> ---
>> >>>>>>>> You received this message because you are subscribed to the
>> Google Groups "CDH Users" group.
>> >>>>>>>> To unsubscribe from this group and stop receiving emails from
>> it, send an email to cdh-user+u...@cloudera.org.
>> >>>>>>>> For more options, visit
>> https://groups.google.com/a/cloudera.org/d/optout.
>> >>>>>>>
>> >>>>>>> --
>> >>>>>>>
>> >>>>>>> ---
>> >>>>>>> You received this message because you are subscribed to the
>> Google Groups "CDH Users" group.
>> >>>>>>> To unsubscribe from this group and stop receiving emails from it,
>> send an email to cdh-user+u...@cloudera.org.
>> >>>>>>> For more options, visit
>> https://groups.google.com/a/cloudera.org/d/optout.
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>> --
>> >>>>>> Casey J. Brotherton
>> >>>>>> Customer Operations Engineer
>> >>>>>>
>> >>>>>> --
>> >>>>>>
>> >>>>>> ---
>> >>>>>> You received this message because you are subscribed to the Google
>> Groups "CDH Users" group.
>> >>>>>> To unsubscribe from this group and stop receiving emails from it,
>> send an email to cdh-user+u...@cloudera.org.
>> >>>>>> For more options, visit
>> https://groups.google.com/a/cloudera.org/d/optout.
>> >>>>
>> >>>> --
>> >>>>
>> >>>> ---
>> >>>> You received this message because you are subscribed to the Google
>> Groups "CDH Users" group.
>> >>>> To unsubscribe from this group and stop receiving emails from it,
>> send an email to cdh-user+unsubscribe@cloudera.org.
>> >>>> For more options, visit
>> https://groups.google.com/a/cloudera.org/d/optout.
>> >
>> > --
>> >
>> > ---
>> > You received this message because you are subscribed to the Google
>> Groups "CDH Users" group.
>> > To unsubscribe from this group and stop receiving emails from it, send
>> an email to cdh-user+unsubscribe@cloudera.org.
>> > For more options, visit
>> https://groups.google.com/a/cloudera.org/d/optout.
>>
>
>  --
>
> ---
> You received this message because you are subscribed to the Google Groups
> "CDH Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to cdh-user+unsubscribe@cloudera.org.
> For more options, visit https://groups.google.com/a/cloudera.org/d/optout.
>

Re: Apache Sentry works with hive metastore ?

Posted by Prasad Mujumdar <pr...@cloudera.com>.
Hi Anurag,

   If this is active directory, then try setting a config property
hive.server2.authentication.ldap.Domain to your LDAP domain (a.b). Then you
can login with the short userid atangri.
Are you using LDAP group mapping in Hadoop  as well ?

thanks
Prasad



On Thu, Jun 5, 2014 at 7:43 AM, Anurag <at...@groupon.com> wrote:

>
> Any response anyone?
>
> Sent from my iPhone
> > On Jun 4, 2014, at 3:20 PM, Anurag Tangri <at...@groupon.com> wrote:
> >
> > + sentry mailing list
> >
> >
> >> On Wed, Jun 4, 2014 at 12:27 AM, Anurag Tangri <at...@groupon.com>
> wrote:
> >> So, I am working on setting this up.
> >>
> >> I have HDFS integrated with LDAP and working fine.
> >>
> >> I also added ldap params in hive-site along with sentry conf :
> >>
> >> ====
> >>   <property>
> >>     <name>hive.server2.authentication</name>
> >>     <value>LDAP</value>
> >>   </property>
> >>   <property>
> >>     <name>hive.server2.authentication.ldap.url</name>
> >>     <value>ldap://<ip></value>
> >>   </property>
> >>
> >> <property>
> >>   <name>hive.server2.authentication.ldap.baseDN</name>
> >>   <value>dc=a,dc=b</value>
> >> </property>
> >>
> >> ======
> >>
> >> Now when I go to beeline, I type:
> >>
> >> /usr/local/lib/hive/bin/beeline -u jdbc:hive2://<host>:10000
> >> Beeline version 0.10.0-cdh4.6.0 by Apache Hive
> >> 0: jdbc:hive2://<host>:10000> !connect jdbc:hive2://<host>:10000
> >>
> >>
> >> It asks for username and password. I give LDAP credentials as:
> >>
> >> Connecting to jdbc:hive2://<host>:10000
> >> Enter username for jdbc:hive2://<host>:10000: atangri@a.b
> >> Enter password for jdbc:hive2://<host>:10000: **********
> >> Error: Invalid URL: jdbc:hive2://<host>:10000 (state=08S01,code=0)
> >> 1: jdbc:hive2://<host>:10000>
> >>
> >>
> >> Now if I type, hive queries, i see error in hive log:
> >>
> >> 2014-06-04 07:15:13,211 WARN  file.HadoopGroupMappingService
> (HadoopGroupMappingService.java:getGroups(42)) - Unable to obtain groups
> for atangri@a.b
> >> java.io.IOException: No groups found for user atangri@a.b
> >>     at org.apache.hadoop.security.Groups.getGroups(Groups.java:105)
> >>     at
> org.apache.sentry.provider.file.HadoopGroupMappingService.getGroups(HadoopGroupMappingService.java:40)
> >>     at
> org.apache.sentry.provider.file.ResourceAuthorizationProvider.doHasAccess(ResourceAuthorizationProvider.java:98)
> >>     at
> org.apache.sentry.provider.file.ResourceAuthorizationProvider.hasAccess(ResourceAuthorizationProvider.java:93)
> >>     at
> org.apache.sentry.binding.hive.authz.HiveAuthzBinding.authorize(HiveAuthzBinding.java:179)
> >>     at
> org.apache.sentry.binding.hive.HiveAuthzBindingHook.filterShowTables(HiveAuthzBindingHook.java:561)
> >>     at
> org.apache.sentry.binding.hive.HiveAuthzBindingHook.postDriverFetch(HiveAuthzBindingHook.java:634)
> >>     at
> org.apache.hadoop.hive.ql.Driver.fireFilterHooks(Driver.java:1455)
> >>     at org.apache.hadoop.hive.ql.Driver.getResults(Driver.java:1475)
> >>     at
> org.apache.hive.service.cli.operation.SQLOperation.getNextRowSet(SQLOperation.java:200)
> >>     at
> org.apache.hive.service.cli.operation.OperationManager.getOperationNextRowSet(OperationManager.java:179)
> >>     at
> org.apache.hive.service.cli.session.HiveSessionImpl.fetchResults(HiveSessionImpl.java:468)
> >>     at
> org.apache.hive.service.cli.CLIService.fetchResults(CLIService.java:318)
> >>     at
> org.apache.hive.service.cli.thrift.ThriftCLIService.FetchResults(ThriftCLIService.java:398)
> >>     at
> org.apache.hive.service.cli.thrift.TCLIService$Processor$FetchResults.getResult(TCLIService.java:1613)
> >>     at
> org.apache.hive.service.cli.thrift.TCLIService$Processor$FetchResults.getResult(TCLIService.java:1598)
> >>     at
> org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
> >>     at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
> >>     at
> org.apache.hive.service.cli.thrift.TSetIpAddressProcessor.process(TSetIpAddressProcessor.java:38)
> >>     at
> org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:244)
> >>     at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> >>     at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> >>     at java.lang.Thread.run(Thread.java:722)
> >> =======
> >>
> >>
> >> I think the issue is my account on LDAP is atangri, not atangri@a.b.
> >>
> >> Any thoughts on how to fix this ?
> >>
> >>
> >> Thanks,
> >> Anurag Tangri
> >>
> >>
> >>
> >>
> >>
> >>
> >> =====
> >> 2014-06-04 07:15:13,211 WARN  file.HadoopGroupMappingService
> (HadoopGroupMappingService.java:getGroups(42)) - Unable to obtain groups
> for atangri@group.on
> >> java.io.IOException: No groups found for user atangri@group.on
> >>
> >>
> >>> On Tue, Jun 3, 2014 at 12:49 PM, Anurag <at...@groupon.com> wrote:
> >>> Thanks Philippe!
> >>>
> >>> This answers almost all of my questions.
> >>>
> >>> Thanks,
> >>> Anurag Tangri
> >>>
> >>>> On Jun 3, 2014, at 12:08 PM, Philippe Marseille <ma...@gmail.com>
> wrote:
> >>>>
> >>>> I think the Hive CLI (Command line interface tool) is now deprecated.
>  see
> http://blog.cloudera.com/blog/2014/02/migrating-from-hive-cli-to-beeline-a-primer/
> >>>>
> >>>>
> >>>> The metastore has nothing to do with it.  The Metastore Daemon is
> used by all clients.
> >>>>
> >>>> The new flow is :
> >>>>
> >>>> Beeline -> HiveServer2 -> Metastore Daemon -> Metastore Database
> (MySql for example).
> >>>>
> >>>> The previous flow was :
> >>>>
> >>>> Hive CLI (shell ) -> Metastore Daemon -> Metastore Database
> >>>>
> >>>> And before the Metastore Daemon ever existed, it was :
> >>>>
> >>>> Hive CLI -> Metastore Database
> >>>>
> >>>>> On Tuesday, 3 June 2014 13:06:22 UTC-4, Anurag Tangri wrote:
> >>>>> Thanks Casey.
> >>>>>
> >>>>> This is going to be big shift to ask all our users  to go to beeline
> from hive.
> >>>>>
> >>>>> Is there no way to have consistency between what roles and policies
> are seen in HUE via hiveserver2 and hive metastore on hive shell ?
> >>>>> Also, is there a reason why hive metastore was excluded from Sentry
> design ?
> >>>>>
> >>>>>
> >>>>>
> >>>>> Thanks,
> >>>>> Anurag Tangri
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>> On Tue, Jun 3, 2014 at 9:54 AM, Casey Brotherton <
> cbrot...@cloudera.com> wrote:
> >>>>>> Hello Anurag,
> >>>>>>
> >>>>>> To use Sentry, as Johndee mentioned, you must use Hiveserver2.
>  That means you will need to transition users from the hive command line to
> the beeline command line.
> >>>>>>
> >>>>>> This link has more information for CDH4.5
> >>>>>>
> http://www.cloudera.com/content/cloudera-content/cloudera-docs/CDH4/4.5.0/CDH4-Security-Guide/cdh4sg_Sentry.html
> >>>>>>
> >>>>>> There is a link to "Securing the Hive Metastore"  which suggests to
> limit connections using iptables, and granting of permissions.
> >>>>>>
> >>>>>> Hope this helps,
> >>>>>> Casey
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>> On Tue, Jun 3, 2014 at 11:45 AM, Anurag Tangri <at...@groupon.com>
> wrote:
> >>>>>>> Hey Philippe,
> >>>>>>> No, HUE should be fine as per Johndee's explanation as it can be
> pointed to hiveserver2.
> >>>>>>>
> >>>>>>> By metastore, I mean mysql database that hosts hive metadata.
> >>>>>>>
> >>>>>>> This is the database that people go against when they login to
> hive shell for a hadoop client box.
> >>>>>>>
> >>>>>>>
> >>>>>>> Thanks,
> >>>>>>> Anurag Tangri
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>> On Tue, Jun 3, 2014 at 8:11 AM, Philippe Marseille <
> mars...@gmail.com> wrote:
> >>>>>>>> By "Hive Metastore", are you referring to the app in Hue?
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>> On Tuesday, 3 June 2014 00:32:53 UTC-4, Anurag Tangri wrote:
> >>>>>>>>> Thanks Johndee.
> >>>>>>>>>
> >>>>>>>>> Can you point to some documentation on hive group and rejecting
> requests from metastore ?
> >>>>>>>>>
> >>>>>>>>> Hiveserver2 is mainly for jdbc interface, hue etc.
> >>>>>>>>>
> >>>>>>>>> For people going via hive metastore, we will need to enforce
> same set of roles and policies as via hiveserver2.
> >>>>>>>>>
> >>>>>>>>> Thanks,
> >>>>>>>>> Anurag Tangri
> >>>>>>>>>
> >>>>>>>>>> On Jun 2, 2014, at 7:57 PM, Johndee Cloudera <
> joh...@cloudera.com> wrote:
> >>>>>>>>>>
> >>>>>>>>>> When it comes to hive and sentry you need at least Hive Server
> 2. Hive Server 2 is what actually runs the Sentry daemon. The Hive
> Metastore Server handles metadata requests, and in the setup guide is
> configured to reject all connections from users not in the hive group
> typically.
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>> On Mon, Jun 2, 2014 at 10:15 PM, Anurag Tangri <
> tangri...@gmail.com> wrote:
> >>>>>>>>>>> Hi all,
> >>>>>>>>>>> From what I read everywhere, sentry is to be used in
> conjunction with hiveserver2.
> >>>>>>>>>>>
> >>>>>>>>>>> Is it true that it can be used with hive metastore ? If yes,
> Any links explaining how to do so would be great.
> >>>>>>>>>>>
> >>>>>>>>>>> Thanks,
> >>>>>>>>>>> Anurag Tangri
> >>>>>>>>>>>
> >>>>>>>>>>> --
> >>>>>>>>>>>
> >>>>>>>>>>> ---
> >>>>>>>>>>> You received this message because you are subscribed to the
> Google Groups "CDH Users" group.
> >>>>>>>>>>> To unsubscribe from this group and stop receiving emails from
> it, send an email to cdh-user+u...@cloudera.org.
> >>>>>>>>>>>
> >>>>>>>>>>> For more options, visit
> https://groups.google.com/a/cloudera.org/d/optout.
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> --
> >>>>>>>>>> - JRB
> >>>>>>>>>> --
> >>>>>>>>>>
> >>>>>>>>>> ---
> >>>>>>>>>> You received this message because you are subscribed to the
> Google Groups "CDH Users" group.
> >>>>>>>>>> To unsubscribe from this group and stop receiving emails from
> it, send an email to cdh-user+u...@cloudera.org.
> >>>>>>>>>>
> >>>>>>>>>> For more options, visit
> https://groups.google.com/a/cloudera.org/d/optout.
> >>>>>>>>
> >>>>>>>> --
> >>>>>>>>
> >>>>>>>> ---
> >>>>>>>> You received this message because you are subscribed to the
> Google Groups "CDH Users" group.
> >>>>>>>> To unsubscribe from this group and stop receiving emails from it,
> send an email to cdh-user+u...@cloudera.org.
> >>>>>>>> For more options, visit
> https://groups.google.com/a/cloudera.org/d/optout.
> >>>>>>>
> >>>>>>> --
> >>>>>>>
> >>>>>>> ---
> >>>>>>> You received this message because you are subscribed to the Google
> Groups "CDH Users" group.
> >>>>>>> To unsubscribe from this group and stop receiving emails from it,
> send an email to cdh-user+u...@cloudera.org.
> >>>>>>> For more options, visit
> https://groups.google.com/a/cloudera.org/d/optout.
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> --
> >>>>>> Casey J. Brotherton
> >>>>>> Customer Operations Engineer
> >>>>>>
> >>>>>> --
> >>>>>>
> >>>>>> ---
> >>>>>> You received this message because you are subscribed to the Google
> Groups "CDH Users" group.
> >>>>>> To unsubscribe from this group and stop receiving emails from it,
> send an email to cdh-user+u...@cloudera.org.
> >>>>>> For more options, visit
> https://groups.google.com/a/cloudera.org/d/optout.
> >>>>
> >>>> --
> >>>>
> >>>> ---
> >>>> You received this message because you are subscribed to the Google
> Groups "CDH Users" group.
> >>>> To unsubscribe from this group and stop receiving emails from it,
> send an email to cdh-user+unsubscribe@cloudera.org.
> >>>> For more options, visit
> https://groups.google.com/a/cloudera.org/d/optout.
> >
> > --
> >
> > ---
> > You received this message because you are subscribed to the Google
> Groups "CDH Users" group.
> > To unsubscribe from this group and stop receiving emails from it, send
> an email to cdh-user+unsubscribe@cloudera.org.
> > For more options, visit
> https://groups.google.com/a/cloudera.org/d/optout.
>