You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@drill.apache.org by Ascot Moss <as...@gmail.com> on 2017/08/12 08:11:51 UTC
Drills' Hbase storage plugin
Hi,
I have Hbase with Kerberos enabled, how to configure drill's hbase storage
plugin in order to access kerberos-Hbase?
Regards
Re: Drills' Hbase storage plugin
Posted by Sorabh Hamirwasia <sh...@mapr.com>.
Hi Ascot,
When Drill's HBase storage plugin creates a HBase connection it uses the storage plugin configuration to get all the required configs. The way storage plugin config is formed is based on code I shared with you below or you can look in [1]. If you look into HBaseConfiguration.create() then it first loads all the HBase related configurations provided as (hbase-site.xml / hbase-default.xml) and then override those with Drill's Storage Plugin configs. So the reason of suggesting hbase-site.xml with all proper configuration in classpath was that the security related configurations will be picked up properly from that file and other configuration will be picked from Storage Plugin configuration and hence it should work. Hope it helps.
[1]: https://github.com/apache/drill/blob/master/contrib/storage-hbase/src/main/java/org/apache/drill/exec/store/hbase/HBaseStoragePluginConfig.java#L91
Thanks,
Sorabh
________________________________
From: Ascot Moss <as...@gmail.com>
Sent: Monday, August 21, 2017 12:50:59 AM
To: user@drill.apache.org
Subject: Re: Drills' Hbase storage plugin
Dear Sorabh,
About putting a hbase-site.xml in classpath, can you please explain a bit
more about this point?
Regards
On Mon, Aug 21, 2017 at 3:38 PM, Ascot Moss <as...@gmail.com> wrote:
> Dear Sorabh,
>
> Your reply helps a lot!!
>
> I have copied hbase-site.xml to $DRILL_HOME/conf,
> restarted drillbit, ran below:
>
> kinit
> sqlline -n "jdbc:drill:drillbit=n1.testserver:31010;auth=kerberos" -n "
> u1@MY_DOMAIN.COM"
>
> use hbase;
> +------+---------------------------------------------------+
> | ok | summary |
> +------+---------------------------------------------------+
> | true | Default schema chnaged to [hbase] |
> +------+---------------------------------------------------+
>
>
> show tables;
> (no result, took long time) <=== any idea how to solve it?
>
>
> Regards
>
>
> On Tue, Aug 15, 2017 at 10:14 AM, Sorabh Hamirwasia <sh...@mapr.com>
> wrote:
>
>> Hi Ascot,
>>
>> I am not sure if Kerberos between Hbase storage plugin and Hbase server
>> has been tried before or not. Link in [1] talks about Kerberos setup
>> between Drill Client and Drillbit. However looking into the code for Hbase
>> and link at [2], it looks like if you can put a hbase-site.xml in classpath
>> with corresponding setting, then it should work, since internally Drill's
>> hbase storage plugin creates a HBaseConfiguration instance which loads this
>> file in class-path and set corresponding settings for authentication. You
>> need to create TGT for Drill process user (by running kinit, also
>> information provided in link [2]) which will be running as HBase client to
>> the server.
>>
>>
>> HBaseConnectionKey::getHBaseConf() {
>>
>> return storeConfig.getHBaseConf();
>> }
>>
>>
>> Under HBaseStoragePluginConfig you can find this is how configuration is
>> created:
>>
>>
>> public Configuration getHBaseConf() {
>> if (hbaseConf == null) {
>> hbaseConf = HBaseConfiguration.create();
>> ...
>>
>> ...
>>
>> }
>>
>> ...
>>
>> ...
>>
>> }
>>
>>
>>
>> [1]: https://drill.apache.org/docs/configuring-kerberos-authentication/<
>> https://drill.apache.org/docs/configuring-kerberos-authentication/><
>> https://drill.apache.org/docs/configuring-kerberos-authentication/>
>> [2]: http://hbase.apache.org/0.94/book/security.html
>>
>>
>> Thanks,
>> Sorabh
>>
>
Re: Drills' Hbase storage plugin
Posted by Ascot Moss <as...@gmail.com>.
Dear Sorabh,
About putting a hbase-site.xml in classpath, can you please explain a bit
more about this point?
Regards
On Mon, Aug 21, 2017 at 3:38 PM, Ascot Moss <as...@gmail.com> wrote:
> Dear Sorabh,
>
> Your reply helps a lot!!
>
> I have copied hbase-site.xml to $DRILL_HOME/conf,
> restarted drillbit, ran below:
>
> kinit
> sqlline -n "jdbc:drill:drillbit=n1.testserver:31010;auth=kerberos" -n "
> u1@MY_DOMAIN.COM"
>
> use hbase;
> +------+---------------------------------------------------+
> | ok | summary |
> +------+---------------------------------------------------+
> | true | Default schema chnaged to [hbase] |
> +------+---------------------------------------------------+
>
>
> show tables;
> (no result, took long time) <=== any idea how to solve it?
>
>
> Regards
>
>
> On Tue, Aug 15, 2017 at 10:14 AM, Sorabh Hamirwasia <sh...@mapr.com>
> wrote:
>
>> Hi Ascot,
>>
>> I am not sure if Kerberos between Hbase storage plugin and Hbase server
>> has been tried before or not. Link in [1] talks about Kerberos setup
>> between Drill Client and Drillbit. However looking into the code for Hbase
>> and link at [2], it looks like if you can put a hbase-site.xml in classpath
>> with corresponding setting, then it should work, since internally Drill's
>> hbase storage plugin creates a HBaseConfiguration instance which loads this
>> file in class-path and set corresponding settings for authentication. You
>> need to create TGT for Drill process user (by running kinit, also
>> information provided in link [2]) which will be running as HBase client to
>> the server.
>>
>>
>> HBaseConnectionKey::getHBaseConf() {
>>
>> return storeConfig.getHBaseConf();
>> }
>>
>>
>> Under HBaseStoragePluginConfig you can find this is how configuration is
>> created:
>>
>>
>> public Configuration getHBaseConf() {
>> if (hbaseConf == null) {
>> hbaseConf = HBaseConfiguration.create();
>> ...
>>
>> ...
>>
>> }
>>
>> ...
>>
>> ...
>>
>> }
>>
>>
>>
>> [1]: https://drill.apache.org/docs/configuring-kerberos-authentication/<
>> https://drill.apache.org/docs/configuring-kerberos-authentication/><
>> https://drill.apache.org/docs/configuring-kerberos-authentication/>
>> [2]: http://hbase.apache.org/0.94/book/security.html
>>
>>
>> Thanks,
>> Sorabh
>>
>
Re: Drills' Hbase storage plugin
Posted by Ascot Moss <as...@gmail.com>.
Dear Sorabh,
Your reply helps a lot!!
I have copied hbase-site.xml to $DRILL_HOME/conf,
restarted drillbit, ran below:
kinit
sqlline -n "jdbc:drill:drillbit=n1.testserver:31010;auth=kerberos" -n "
u1@MY_DOMAIN.COM"
use hbase;
+------+---------------------------------------------------+
| ok | summary |
+------+---------------------------------------------------+
| true | Default schema chnaged to [hbase] |
+------+---------------------------------------------------+
show tables;
(no result, took long time) <=== any idea how to solve it?
Regards
On Tue, Aug 15, 2017 at 10:14 AM, Sorabh Hamirwasia <sh...@mapr.com>
wrote:
> Hi Ascot,
>
> I am not sure if Kerberos between Hbase storage plugin and Hbase server
> has been tried before or not. Link in [1] talks about Kerberos setup
> between Drill Client and Drillbit. However looking into the code for Hbase
> and link at [2], it looks like if you can put a hbase-site.xml in classpath
> with corresponding setting, then it should work, since internally Drill's
> hbase storage plugin creates a HBaseConfiguration instance which loads this
> file in class-path and set corresponding settings for authentication. You
> need to create TGT for Drill process user (by running kinit, also
> information provided in link [2]) which will be running as HBase client to
> the server.
>
>
> HBaseConnectionKey::getHBaseConf() {
>
> return storeConfig.getHBaseConf();
> }
>
>
> Under HBaseStoragePluginConfig you can find this is how configuration is
> created:
>
>
> public Configuration getHBaseConf() {
> if (hbaseConf == null) {
> hbaseConf = HBaseConfiguration.create();
> ...
>
> ...
>
> }
>
> ...
>
> ...
>
> }
>
>
>
> [1]: https://drill.apache.org/docs/configuring-kerberos-authentication/<
> https://drill.apache.org/docs/configuring-kerberos-authentication/><http
> s://drill.apache.org/docs/configuring-kerberos-authentication/>
> [2]: http://hbase.apache.org/0.94/book/security.html
>
>
> Thanks,
> Sorabh
>
Re: Drills' Hbase storage plugin
Posted by Sorabh Hamirwasia <sh...@mapr.com>.
Hi Ascot,
I am not sure if Kerberos between Hbase storage plugin and Hbase server has been tried before or not. Link in [1] talks about Kerberos setup between Drill Client and Drillbit. However looking into the code for Hbase and link at [2], it looks like if you can put a hbase-site.xml in classpath with corresponding setting, then it should work, since internally Drill's hbase storage plugin creates a HBaseConfiguration instance which loads this file in class-path and set corresponding settings for authentication. You need to create TGT for Drill process user (by running kinit, also information provided in link [2]) which will be running as HBase client to the server.
HBaseConnectionKey::getHBaseConf() {
return storeConfig.getHBaseConf();
}
Under HBaseStoragePluginConfig you can find this is how configuration is created:
public Configuration getHBaseConf() {
if (hbaseConf == null) {
hbaseConf = HBaseConfiguration.create();
...
...
}
...
...
}
[1]: https://drill.apache.org/docs/configuring-kerberos-authentication/<https://drill.apache.org/docs/configuring-kerberos-authentication/><https://drill.apache.org/docs/configuring-kerberos-authentication/>
[2]: http://hbase.apache.org/0.94/book/security.html
Thanks,
Sorabh
________________________________
From: Ascot Moss <as...@gmail.com>
Sent: Saturday, August 12, 2017 5:01 AM
To: user@drill.apache.org
Subject: Re: Drills' Hbase storage plugin
Does Drill's Hbase Storage support Hbase-Kerberos? I cannot find any link
about this.
https://drill.apache.org/docs/hbase-storage-plugin/
This link does not provide too much detail about Drills' Hbase Storage on
HBase-Kerberos.
On Sat, Aug 12, 2017 at 4:11 PM, Ascot Moss <as...@gmail.com> wrote:
> Hi,
>
> I have Hbase with Kerberos enabled, how to configure drill's hbase storage
> plugin in order to access kerberos-Hbase?
>
> Regards
>
Re: Drills' Hbase storage plugin
Posted by Ascot Moss <as...@gmail.com>.
Does Drill's Hbase Storage support Hbase-Kerberos? I cannot find any link
about this.
https://drill.apache.org/docs/hbase-storage-plugin/
This link does not provide too much detail about Drills' Hbase Storage on
HBase-Kerberos.
On Sat, Aug 12, 2017 at 4:11 PM, Ascot Moss <as...@gmail.com> wrote:
> Hi,
>
> I have Hbase with Kerberos enabled, how to configure drill's hbase storage
> plugin in order to access kerberos-Hbase?
>
> Regards
>