You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by Chris Gent <ch...@bigdatapartnership.com> on 2015/12/18 18:04:53 UTC
doAs with HBase Java API and Apache Ranger
Hi,
We have a webservice that performs reads/writes on HBase tables and have a
requirement to authorize and audit table/column family access using Ranger.
I've configured the reads/writes to be performed under doAs to try to make
this happen but the requests end up being authorized and audit logged as
the service user rather than the requestor.
A snippet of the application code looks like this (doAsUser is the end
user's username):
UserGroupInformation ugi = UserGroupInformation.createProxyUser(doAsUser,
UserGroupInformation.getLoginUser());
try {
ugi.doAs(new PriviledgedExceptionAction<Void>() {
@Override
public Void run() throws Exception {
LOGGER.info("HBase put as user " + ugi.getShortUserName());
table.put(put);
return null;
}
});
Has anyone got experience with the HBase Ranger plugin and/or come across
this problem before and know the best way to solve it?
For reference this is all running with HDP 2.3.2.
Thanks in advance!
--
Chris
--
*NOTICE AND DISCLAIMER*
This email (including attachments) is confidential. If you are not the
intended recipient, notify the sender immediately, delete this email from
your system and do not disclose or use for any purpose.
Business Address: Eagle House, 163 City Road, London, EC1V 1NR. United
Kingdom
Registered Office: Finsgate, 5-7 Cranwood Street, London, EC1V 9EE. United
Kingdom
Big Data Partnership Limited is a company registered in England & Wales
with Company No 7904824
Re: doAs with HBase Java API and Apache Ranger
Posted by Adam Davidson <ad...@bigdatapartnership.com>.
Hi Jerry,
I think we have cracked it now, thanks. I noticed the proxy setting on the
HBase server was setup for the wrong user. It wasn't obvious until I
noticed in some debug logs that the impersonation was being forbidden which
helped me identify the issue. Ranger now reports the correct user in the
audit log, so that side all seems fine.
Regards,
Adam
On Tue, 22 Dec 2015 at 15:00 Jerry He <je...@gmail.com> wrote:
> What is the ConectionCache in your code? Is it org.apache.hadoop.hbase.
> util.ConnectionCache?
>
> As a sample on how to use the ConnectionCache, you can look at this:
>
> https://github.com/apache/hbase/blob/master/hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/RESTServletContainer.java
>
> and
>
> https://github.com/apache/hbase/blob/master/hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/RESTServlet.java
>
> Pay attention to the connectionCache.setEffectiveUser() call. No doAs()
> call is used.
>
> An extra note. You can look at the native HBase audit log in the log
> directory along side with the maser and region server logs.
> If the hbase native audit log shows the same as in Ranger, then you can
> rule out it is the Ranger plug issue.
>
> Hope this helps.
>
> Jerry
>
> On Mon, Dec 21, 2015 at 6:44 AM, Adam Davidson <
> adam.davidson@bigdatapartnership.com> wrote:
>
>> Hi,
>>
>> I'm working alongside Chris on this problem. As Jerry mentioned the user
>> context is obtained when the Connection is created, I've modified the code
>> as below;
>>
>> try {
>> ugi.doAs(new PriviledgedExceptionAction<Void>() {
>> @Override
>> public Void run() throws Exception {
>> LOGGER.info("HBase put as user " +
>> ugi.getShortUserName());
>> connectionCache.getTable(tableName).put(put);
>> return null;
>> }
>> });
>>
>> where the connection cache is initialised at start up using the default
>> UserProvider with a secure Configuration. I believe the server side user
>> proxying has been configured. However, the Ranger audit logs still show
>> the HBase operations being executed by the service user (i.e. the Kerberos
>> principal that runs the rest service). Assuming what I've done is correct,
>> is there anything else you can think of that might be wrong? Is this just a
>> matter for the Ranger team or whomever is responsible for the HBase Ranger
>> plugin?
>>
>> Best Regards,
>> Adam
>>
>> On Sun, 20 Dec 2015 at 03:40 Jerry He <je...@gmail.com> wrote:
>>
>>> To answer your HBase question, user context is obtained when
>>> 'Connection' object is created. 'Table' shares what the 'Connection' has.
>>> Also like Ted mentioned, use ProxyUser needs additional config on the
>>> serer side.
>>>
>>> Jerry
>>>
>>> On Fri, Dec 18, 2015 at 9:37 AM, Ted Yu <yu...@gmail.com> wrote:
>>>
>>>> I talked with a Ranger developer who has read the thread on Ranger
>>>> mailing
>>>> list.
>>>>
>>>> Setting up proxy may require certain steps.
>>>>
>>>> I suggest responding to Bosco's question on the Ranger mailing list (by
>>>> providing related server log, e.g.) - Ranger developers have knowledge
>>>> about HBase.
>>>>
>>>> Cheers
>>>>
>>>> On Fri, Dec 18, 2015 at 9:24 AM, Chris Gent <
>>>> chris.gent@bigdatapartnership.com> wrote:
>>>>
>>>> > Hey Ted,
>>>> >
>>>> > Yeah - they suggested asking over here :-)
>>>> >
>>>> > I think the question is where the user context is set/comes from when
>>>> using
>>>> > the HBase API. It was suggested that it comes when the Table object
>>>> gets
>>>> > created? Or is it right back when the connection is established?
>>>> >
>>>> > --
>>>> > Chris
>>>> >
>>>> >
>>>> >
>>>> > On 18 December 2015 at 17:18, Ted Yu <yu...@gmail.com> wrote:
>>>> >
>>>> > > Have you polled Ranger community with this question ?
>>>> > >
>>>> > > http://ranger.apache.org/mail-lists.html
>>>> > >
>>>> > > Cheers
>>>> > >
>>>> > > On Fri, Dec 18, 2015 at 9:04 AM, Chris Gent <
>>>> > > chris.gent@bigdatapartnership.com> wrote:
>>>> > >
>>>> > > > Hi,
>>>> > > >
>>>> > > > We have a webservice that performs reads/writes on HBase tables
>>>> and
>>>> > have
>>>> > > a
>>>> > > > requirement to authorize and audit table/column family access
>>>> using
>>>> > > Ranger.
>>>> > > >
>>>> > > > I've configured the reads/writes to be performed under doAs to
>>>> try to
>>>> > > make
>>>> > > > this happen but the requests end up being authorized and audit
>>>> logged
>>>> > as
>>>> > > > the service user rather than the requestor.
>>>> > > >
>>>> > > >
>>>> > > > A snippet of the application code looks like this (doAsUser is
>>>> the end
>>>> > > > user's username):
>>>> > > >
>>>> > > >
>>>> > > > UserGroupInformation ugi =
>>>> > UserGroupInformation.createProxyUser(doAsUser,
>>>> > > > UserGroupInformation.getLoginUser());
>>>> > > >
>>>> > > > try {
>>>> > > > ugi.doAs(new PriviledgedExceptionAction<Void>() {
>>>> > > > @Override
>>>> > > > public Void run() throws Exception {
>>>> > > > LOGGER.info("HBase put as user " +
>>>> > > ugi.getShortUserName());
>>>> > > > table.put(put);
>>>> > > > return null;
>>>> > > > }
>>>> > > > });
>>>> > > >
>>>> > > >
>>>> > > > Has anyone got experience with the HBase Ranger plugin and/or come
>>>> > across
>>>> > > > this problem before and know the best way to solve it?
>>>> > > >
>>>> > > > For reference this is all running with HDP 2.3.2.
>>>> > > >
>>>> > > > Thanks in advance!
>>>> > > >
>>>> > > > --
>>>> > > > Chris
>>>> > > >
>>>> > > > --
>>>> > > >
>>>> > > >
>>>> > > > *NOTICE AND DISCLAIMER*
>>>> > > >
>>>> > > > This email (including attachments) is confidential. If you are
>>>> not the
>>>> > > > intended recipient, notify the sender immediately, delete this
>>>> email
>>>> > from
>>>> > > > your system and do not disclose or use for any purpose.
>>>> > > >
>>>> > > > Business Address: Eagle House, 163 City Road, London, EC1V 1NR.
>>>> United
>>>> > > > Kingdom
>>>> > > > Registered Office: Finsgate, 5-7 Cranwood Street, London, EC1V
>>>> 9EE.
>>>> > > United
>>>> > > > Kingdom
>>>> > > > Big Data Partnership Limited is a company registered in England &
>>>> Wales
>>>> > > > with Company No 7904824
>>>> > > >
>>>> > >
>>>> >
>>>> >
>>>> >
>>>> > --
>>>> > *Christopher Gent*
>>>> >
>>>> > *Managing Consultant*
>>>> > Big Data Partnership
>>>> > M: 07795 210205
>>>> > E: chris.gent@bigdatapartnership.com
>>>> >
>>>> > *NOTICE AND DISCLAIMER*
>>>> >
>>>> > This email (including attachments) is confidential. If you are not the
>>>> > intended recipient, notify the sender immediately, delete this email
>>>> from
>>>> > your system and do not disclose or use for any purpose.
>>>> >
>>>> > Business Address: Eagle House, 163 City Road, London, EC1V 1NR. United
>>>> > Kingdom
>>>> > Registered Office: Finsgate, 5-7 Cranwood Street, London, EC1V 9EE.
>>>> United
>>>> > Kingdom
>>>> > Big Data Partnership Limited is a company registered in England &
>>>> Wales
>>>> > with Company No 7904824
>>>> >
>>>> > --
>>>> >
>>>> >
>>>> > *NOTICE AND DISCLAIMER*
>>>> >
>>>> > This email (including attachments) is confidential. If you are not the
>>>> > intended recipient, notify the sender immediately, delete this email
>>>> from
>>>> > your system and do not disclose or use for any purpose.
>>>> >
>>>> > Business Address: Eagle House, 163 City Road, London, EC1V 1NR. United
>>>> > Kingdom
>>>> > Registered Office: Finsgate, 5-7 Cranwood Street, London, EC1V 9EE.
>>>> United
>>>> > Kingdom
>>>> > Big Data Partnership Limited is a company registered in England &
>>>> Wales
>>>> > with Company No 7904824
>>>> >
>>>>
>>>
>>>
>> *NOTICE AND DISCLAIMER*
>>
> This email (including attachments) is confidential. If you are not the
>> intended recipient, notify the sender immediately, delete this email from
>> your system and do not disclose or use for any purpose.
>>
>> Business Address: Eagle House, 163 City Road, London, EC1V 1NR. United
>> Kingdom
>> Registered Office: Finsgate, 5-7 Cranwood Street, London, EC1V 9EE.
>> United Kingdom
>> Big Data Partnership Limited is a company registered in England & Wales
>> with Company No 7904824
>>
>
--
*NOTICE AND DISCLAIMER*
This email (including attachments) is confidential. If you are not the
intended recipient, notify the sender immediately, delete this email from
your system and do not disclose or use for any purpose.
Business Address: Eagle House, 163 City Road, London, EC1V 1NR. United
Kingdom
Registered Office: Finsgate, 5-7 Cranwood Street, London, EC1V 9EE. United
Kingdom
Big Data Partnership Limited is a company registered in England & Wales
with Company No 7904824
Re: doAs with HBase Java API and Apache Ranger
Posted by Jerry He <je...@gmail.com>.
What is the ConectionCache in your code? Is it org.apache.hadoop.hbase.
util.ConnectionCache?
As a sample on how to use the ConnectionCache, you can look at this:
https://github.com/apache/hbase/blob/master/hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/RESTServletContainer.java
and
https://github.com/apache/hbase/blob/master/hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/RESTServlet.java
Pay attention to the connectionCache.setEffectiveUser() call. No doAs()
call is used.
An extra note. You can look at the native HBase audit log in the log
directory along side with the maser and region server logs.
If the hbase native audit log shows the same as in Ranger, then you can
rule out it is the Ranger plug issue.
Hope this helps.
Jerry
On Mon, Dec 21, 2015 at 6:44 AM, Adam Davidson <
adam.davidson@bigdatapartnership.com> wrote:
> Hi,
>
> I'm working alongside Chris on this problem. As Jerry mentioned the user
> context is obtained when the Connection is created, I've modified the code
> as below;
>
> try {
> ugi.doAs(new PriviledgedExceptionAction<Void>() {
> @Override
> public Void run() throws Exception {
> LOGGER.info("HBase put as user " + ugi.getShortUserName());
> connectionCache.getTable(tableName).put(put);
> return null;
> }
> });
>
> where the connection cache is initialised at start up using the default
> UserProvider with a secure Configuration. I believe the server side user
> proxying has been configured. However, the Ranger audit logs still show
> the HBase operations being executed by the service user (i.e. the Kerberos
> principal that runs the rest service). Assuming what I've done is correct,
> is there anything else you can think of that might be wrong? Is this just a
> matter for the Ranger team or whomever is responsible for the HBase Ranger
> plugin?
>
> Best Regards,
> Adam
>
> On Sun, 20 Dec 2015 at 03:40 Jerry He <je...@gmail.com> wrote:
>
>> To answer your HBase question, user context is obtained when 'Connection'
>> object is created. 'Table' shares what the 'Connection' has.
>> Also like Ted mentioned, use ProxyUser needs additional config on the
>> serer side.
>>
>> Jerry
>>
>> On Fri, Dec 18, 2015 at 9:37 AM, Ted Yu <yu...@gmail.com> wrote:
>>
>>> I talked with a Ranger developer who has read the thread on Ranger
>>> mailing
>>> list.
>>>
>>> Setting up proxy may require certain steps.
>>>
>>> I suggest responding to Bosco's question on the Ranger mailing list (by
>>> providing related server log, e.g.) - Ranger developers have knowledge
>>> about HBase.
>>>
>>> Cheers
>>>
>>> On Fri, Dec 18, 2015 at 9:24 AM, Chris Gent <
>>> chris.gent@bigdatapartnership.com> wrote:
>>>
>>> > Hey Ted,
>>> >
>>> > Yeah - they suggested asking over here :-)
>>> >
>>> > I think the question is where the user context is set/comes from when
>>> using
>>> > the HBase API. It was suggested that it comes when the Table object
>>> gets
>>> > created? Or is it right back when the connection is established?
>>> >
>>> > --
>>> > Chris
>>> >
>>> >
>>> >
>>> > On 18 December 2015 at 17:18, Ted Yu <yu...@gmail.com> wrote:
>>> >
>>> > > Have you polled Ranger community with this question ?
>>> > >
>>> > > http://ranger.apache.org/mail-lists.html
>>> > >
>>> > > Cheers
>>> > >
>>> > > On Fri, Dec 18, 2015 at 9:04 AM, Chris Gent <
>>> > > chris.gent@bigdatapartnership.com> wrote:
>>> > >
>>> > > > Hi,
>>> > > >
>>> > > > We have a webservice that performs reads/writes on HBase tables and
>>> > have
>>> > > a
>>> > > > requirement to authorize and audit table/column family access using
>>> > > Ranger.
>>> > > >
>>> > > > I've configured the reads/writes to be performed under doAs to try
>>> to
>>> > > make
>>> > > > this happen but the requests end up being authorized and audit
>>> logged
>>> > as
>>> > > > the service user rather than the requestor.
>>> > > >
>>> > > >
>>> > > > A snippet of the application code looks like this (doAsUser is the
>>> end
>>> > > > user's username):
>>> > > >
>>> > > >
>>> > > > UserGroupInformation ugi =
>>> > UserGroupInformation.createProxyUser(doAsUser,
>>> > > > UserGroupInformation.getLoginUser());
>>> > > >
>>> > > > try {
>>> > > > ugi.doAs(new PriviledgedExceptionAction<Void>() {
>>> > > > @Override
>>> > > > public Void run() throws Exception {
>>> > > > LOGGER.info("HBase put as user " +
>>> > > ugi.getShortUserName());
>>> > > > table.put(put);
>>> > > > return null;
>>> > > > }
>>> > > > });
>>> > > >
>>> > > >
>>> > > > Has anyone got experience with the HBase Ranger plugin and/or come
>>> > across
>>> > > > this problem before and know the best way to solve it?
>>> > > >
>>> > > > For reference this is all running with HDP 2.3.2.
>>> > > >
>>> > > > Thanks in advance!
>>> > > >
>>> > > > --
>>> > > > Chris
>>> > > >
>>> > > > --
>>> > > >
>>> > > >
>>> > > > *NOTICE AND DISCLAIMER*
>>> > > >
>>> > > > This email (including attachments) is confidential. If you are not
>>> the
>>> > > > intended recipient, notify the sender immediately, delete this
>>> email
>>> > from
>>> > > > your system and do not disclose or use for any purpose.
>>> > > >
>>> > > > Business Address: Eagle House, 163 City Road, London, EC1V 1NR.
>>> United
>>> > > > Kingdom
>>> > > > Registered Office: Finsgate, 5-7 Cranwood Street, London, EC1V 9EE.
>>> > > United
>>> > > > Kingdom
>>> > > > Big Data Partnership Limited is a company registered in England &
>>> Wales
>>> > > > with Company No 7904824
>>> > > >
>>> > >
>>> >
>>> >
>>> >
>>> > --
>>> > *Christopher Gent*
>>> >
>>> > *Managing Consultant*
>>> > Big Data Partnership
>>> > M: 07795 210205
>>> > E: chris.gent@bigdatapartnership.com
>>> >
>>> > *NOTICE AND DISCLAIMER*
>>> >
>>> > This email (including attachments) is confidential. If you are not the
>>> > intended recipient, notify the sender immediately, delete this email
>>> from
>>> > your system and do not disclose or use for any purpose.
>>> >
>>> > Business Address: Eagle House, 163 City Road, London, EC1V 1NR. United
>>> > Kingdom
>>> > Registered Office: Finsgate, 5-7 Cranwood Street, London, EC1V 9EE.
>>> United
>>> > Kingdom
>>> > Big Data Partnership Limited is a company registered in England & Wales
>>> > with Company No 7904824
>>> >
>>> > --
>>> >
>>> >
>>> > *NOTICE AND DISCLAIMER*
>>> >
>>> > This email (including attachments) is confidential. If you are not the
>>> > intended recipient, notify the sender immediately, delete this email
>>> from
>>> > your system and do not disclose or use for any purpose.
>>> >
>>> > Business Address: Eagle House, 163 City Road, London, EC1V 1NR. United
>>> > Kingdom
>>> > Registered Office: Finsgate, 5-7 Cranwood Street, London, EC1V 9EE.
>>> United
>>> > Kingdom
>>> > Big Data Partnership Limited is a company registered in England & Wales
>>> > with Company No 7904824
>>> >
>>>
>>
>>
> *NOTICE AND DISCLAIMER*
>
> This email (including attachments) is confidential. If you are not the
> intended recipient, notify the sender immediately, delete this email from
> your system and do not disclose or use for any purpose.
>
> Business Address: Eagle House, 163 City Road, London, EC1V 1NR. United
> Kingdom
> Registered Office: Finsgate, 5-7 Cranwood Street, London, EC1V 9EE. United
> Kingdom
> Big Data Partnership Limited is a company registered in England & Wales
> with Company No 7904824
>
Re: doAs with HBase Java API and Apache Ranger
Posted by Adam Davidson <ad...@bigdatapartnership.com>.
Hi,
I'm working alongside Chris on this problem. As Jerry mentioned the user
context is obtained when the Connection is created, I've modified the code
as below;
try {
ugi.doAs(new PriviledgedExceptionAction<Void>() {
@Override
public Void run() throws Exception {
LOGGER.info("HBase put as user " + ugi.getShortUserName());
connectionCache.getTable(tableName).put(put);
return null;
}
});
where the connection cache is initialised at start up using the default
UserProvider with a secure Configuration. I believe the server side user
proxying has been configured. However, the Ranger audit logs still show
the HBase operations being executed by the service user (i.e. the Kerberos
principal that runs the rest service). Assuming what I've done is correct,
is there anything else you can think of that might be wrong? Is this just a
matter for the Ranger team or whomever is responsible for the HBase Ranger
plugin?
Best Regards,
Adam
On Sun, 20 Dec 2015 at 03:40 Jerry He <je...@gmail.com> wrote:
> To answer your HBase question, user context is obtained when 'Connection'
> object is created. 'Table' shares what the 'Connection' has.
> Also like Ted mentioned, use ProxyUser needs additional config on the
> serer side.
>
> Jerry
>
> On Fri, Dec 18, 2015 at 9:37 AM, Ted Yu <yu...@gmail.com> wrote:
>
>> I talked with a Ranger developer who has read the thread on Ranger mailing
>> list.
>>
>> Setting up proxy may require certain steps.
>>
>> I suggest responding to Bosco's question on the Ranger mailing list (by
>> providing related server log, e.g.) - Ranger developers have knowledge
>> about HBase.
>>
>> Cheers
>>
>> On Fri, Dec 18, 2015 at 9:24 AM, Chris Gent <
>> chris.gent@bigdatapartnership.com> wrote:
>>
>> > Hey Ted,
>> >
>> > Yeah - they suggested asking over here :-)
>> >
>> > I think the question is where the user context is set/comes from when
>> using
>> > the HBase API. It was suggested that it comes when the Table object gets
>> > created? Or is it right back when the connection is established?
>> >
>> > --
>> > Chris
>> >
>> >
>> >
>> > On 18 December 2015 at 17:18, Ted Yu <yu...@gmail.com> wrote:
>> >
>> > > Have you polled Ranger community with this question ?
>> > >
>> > > http://ranger.apache.org/mail-lists.html
>> > >
>> > > Cheers
>> > >
>> > > On Fri, Dec 18, 2015 at 9:04 AM, Chris Gent <
>> > > chris.gent@bigdatapartnership.com> wrote:
>> > >
>> > > > Hi,
>> > > >
>> > > > We have a webservice that performs reads/writes on HBase tables and
>> > have
>> > > a
>> > > > requirement to authorize and audit table/column family access using
>> > > Ranger.
>> > > >
>> > > > I've configured the reads/writes to be performed under doAs to try
>> to
>> > > make
>> > > > this happen but the requests end up being authorized and audit
>> logged
>> > as
>> > > > the service user rather than the requestor.
>> > > >
>> > > >
>> > > > A snippet of the application code looks like this (doAsUser is the
>> end
>> > > > user's username):
>> > > >
>> > > >
>> > > > UserGroupInformation ugi =
>> > UserGroupInformation.createProxyUser(doAsUser,
>> > > > UserGroupInformation.getLoginUser());
>> > > >
>> > > > try {
>> > > > ugi.doAs(new PriviledgedExceptionAction<Void>() {
>> > > > @Override
>> > > > public Void run() throws Exception {
>> > > > LOGGER.info("HBase put as user " +
>> > > ugi.getShortUserName());
>> > > > table.put(put);
>> > > > return null;
>> > > > }
>> > > > });
>> > > >
>> > > >
>> > > > Has anyone got experience with the HBase Ranger plugin and/or come
>> > across
>> > > > this problem before and know the best way to solve it?
>> > > >
>> > > > For reference this is all running with HDP 2.3.2.
>> > > >
>> > > > Thanks in advance!
>> > > >
>> > > > --
>> > > > Chris
>> > > >
>> > > > --
>> > > >
>> > > >
>> > > > *NOTICE AND DISCLAIMER*
>> > > >
>> > > > This email (including attachments) is confidential. If you are not
>> the
>> > > > intended recipient, notify the sender immediately, delete this email
>> > from
>> > > > your system and do not disclose or use for any purpose.
>> > > >
>> > > > Business Address: Eagle House, 163 City Road, London, EC1V 1NR.
>> United
>> > > > Kingdom
>> > > > Registered Office: Finsgate, 5-7 Cranwood Street, London, EC1V 9EE.
>> > > United
>> > > > Kingdom
>> > > > Big Data Partnership Limited is a company registered in England &
>> Wales
>> > > > with Company No 7904824
>> > > >
>> > >
>> >
>> >
>> >
>> > --
>> > *Christopher Gent*
>> >
>> > *Managing Consultant*
>> > Big Data Partnership
>> > M: 07795 210205
>> > E: chris.gent@bigdatapartnership.com
>> >
>> > *NOTICE AND DISCLAIMER*
>> >
>> > This email (including attachments) is confidential. If you are not the
>> > intended recipient, notify the sender immediately, delete this email
>> from
>> > your system and do not disclose or use for any purpose.
>> >
>> > Business Address: Eagle House, 163 City Road, London, EC1V 1NR. United
>> > Kingdom
>> > Registered Office: Finsgate, 5-7 Cranwood Street, London, EC1V 9EE.
>> United
>> > Kingdom
>> > Big Data Partnership Limited is a company registered in England & Wales
>> > with Company No 7904824
>> >
>> > --
>> >
>> >
>> > *NOTICE AND DISCLAIMER*
>> >
>> > This email (including attachments) is confidential. If you are not the
>> > intended recipient, notify the sender immediately, delete this email
>> from
>> > your system and do not disclose or use for any purpose.
>> >
>> > Business Address: Eagle House, 163 City Road, London, EC1V 1NR. United
>> > Kingdom
>> > Registered Office: Finsgate, 5-7 Cranwood Street, London, EC1V 9EE.
>> United
>> > Kingdom
>> > Big Data Partnership Limited is a company registered in England & Wales
>> > with Company No 7904824
>> >
>>
>
>
--
*NOTICE AND DISCLAIMER*
This email (including attachments) is confidential. If you are not the
intended recipient, notify the sender immediately, delete this email from
your system and do not disclose or use for any purpose.
Business Address: Eagle House, 163 City Road, London, EC1V 1NR. United
Kingdom
Registered Office: Finsgate, 5-7 Cranwood Street, London, EC1V 9EE. United
Kingdom
Big Data Partnership Limited is a company registered in England & Wales
with Company No 7904824
Re: doAs with HBase Java API and Apache Ranger
Posted by Jerry He <je...@gmail.com>.
To answer your HBase question, user context is obtained when 'Connection'
object is created. 'Table' shares what the 'Connection' has.
Also like Ted mentioned, use ProxyUser needs additional config on the serer
side.
Jerry
On Fri, Dec 18, 2015 at 9:37 AM, Ted Yu <yu...@gmail.com> wrote:
> I talked with a Ranger developer who has read the thread on Ranger mailing
> list.
>
> Setting up proxy may require certain steps.
>
> I suggest responding to Bosco's question on the Ranger mailing list (by
> providing related server log, e.g.) - Ranger developers have knowledge
> about HBase.
>
> Cheers
>
> On Fri, Dec 18, 2015 at 9:24 AM, Chris Gent <
> chris.gent@bigdatapartnership.com> wrote:
>
> > Hey Ted,
> >
> > Yeah - they suggested asking over here :-)
> >
> > I think the question is where the user context is set/comes from when
> using
> > the HBase API. It was suggested that it comes when the Table object gets
> > created? Or is it right back when the connection is established?
> >
> > --
> > Chris
> >
> >
> >
> > On 18 December 2015 at 17:18, Ted Yu <yu...@gmail.com> wrote:
> >
> > > Have you polled Ranger community with this question ?
> > >
> > > http://ranger.apache.org/mail-lists.html
> > >
> > > Cheers
> > >
> > > On Fri, Dec 18, 2015 at 9:04 AM, Chris Gent <
> > > chris.gent@bigdatapartnership.com> wrote:
> > >
> > > > Hi,
> > > >
> > > > We have a webservice that performs reads/writes on HBase tables and
> > have
> > > a
> > > > requirement to authorize and audit table/column family access using
> > > Ranger.
> > > >
> > > > I've configured the reads/writes to be performed under doAs to try to
> > > make
> > > > this happen but the requests end up being authorized and audit logged
> > as
> > > > the service user rather than the requestor.
> > > >
> > > >
> > > > A snippet of the application code looks like this (doAsUser is the
> end
> > > > user's username):
> > > >
> > > >
> > > > UserGroupInformation ugi =
> > UserGroupInformation.createProxyUser(doAsUser,
> > > > UserGroupInformation.getLoginUser());
> > > >
> > > > try {
> > > > ugi.doAs(new PriviledgedExceptionAction<Void>() {
> > > > @Override
> > > > public Void run() throws Exception {
> > > > LOGGER.info("HBase put as user " +
> > > ugi.getShortUserName());
> > > > table.put(put);
> > > > return null;
> > > > }
> > > > });
> > > >
> > > >
> > > > Has anyone got experience with the HBase Ranger plugin and/or come
> > across
> > > > this problem before and know the best way to solve it?
> > > >
> > > > For reference this is all running with HDP 2.3.2.
> > > >
> > > > Thanks in advance!
> > > >
> > > > --
> > > > Chris
> > > >
> > > > --
> > > >
> > > >
> > > > *NOTICE AND DISCLAIMER*
> > > >
> > > > This email (including attachments) is confidential. If you are not
> the
> > > > intended recipient, notify the sender immediately, delete this email
> > from
> > > > your system and do not disclose or use for any purpose.
> > > >
> > > > Business Address: Eagle House, 163 City Road, London, EC1V 1NR.
> United
> > > > Kingdom
> > > > Registered Office: Finsgate, 5-7 Cranwood Street, London, EC1V 9EE.
> > > United
> > > > Kingdom
> > > > Big Data Partnership Limited is a company registered in England &
> Wales
> > > > with Company No 7904824
> > > >
> > >
> >
> >
> >
> > --
> > *Christopher Gent*
> >
> > *Managing Consultant*
> > Big Data Partnership
> > M: 07795 210205
> > E: chris.gent@bigdatapartnership.com
> >
> > *NOTICE AND DISCLAIMER*
> >
> > This email (including attachments) is confidential. If you are not the
> > intended recipient, notify the sender immediately, delete this email from
> > your system and do not disclose or use for any purpose.
> >
> > Business Address: Eagle House, 163 City Road, London, EC1V 1NR. United
> > Kingdom
> > Registered Office: Finsgate, 5-7 Cranwood Street, London, EC1V 9EE.
> United
> > Kingdom
> > Big Data Partnership Limited is a company registered in England & Wales
> > with Company No 7904824
> >
> > --
> >
> >
> > *NOTICE AND DISCLAIMER*
> >
> > This email (including attachments) is confidential. If you are not the
> > intended recipient, notify the sender immediately, delete this email from
> > your system and do not disclose or use for any purpose.
> >
> > Business Address: Eagle House, 163 City Road, London, EC1V 1NR. United
> > Kingdom
> > Registered Office: Finsgate, 5-7 Cranwood Street, London, EC1V 9EE.
> United
> > Kingdom
> > Big Data Partnership Limited is a company registered in England & Wales
> > with Company No 7904824
> >
>
Re: doAs with HBase Java API and Apache Ranger
Posted by Ted Yu <yu...@gmail.com>.
I talked with a Ranger developer who has read the thread on Ranger mailing
list.
Setting up proxy may require certain steps.
I suggest responding to Bosco's question on the Ranger mailing list (by
providing related server log, e.g.) - Ranger developers have knowledge
about HBase.
Cheers
On Fri, Dec 18, 2015 at 9:24 AM, Chris Gent <
chris.gent@bigdatapartnership.com> wrote:
> Hey Ted,
>
> Yeah - they suggested asking over here :-)
>
> I think the question is where the user context is set/comes from when using
> the HBase API. It was suggested that it comes when the Table object gets
> created? Or is it right back when the connection is established?
>
> --
> Chris
>
>
>
> On 18 December 2015 at 17:18, Ted Yu <yu...@gmail.com> wrote:
>
> > Have you polled Ranger community with this question ?
> >
> > http://ranger.apache.org/mail-lists.html
> >
> > Cheers
> >
> > On Fri, Dec 18, 2015 at 9:04 AM, Chris Gent <
> > chris.gent@bigdatapartnership.com> wrote:
> >
> > > Hi,
> > >
> > > We have a webservice that performs reads/writes on HBase tables and
> have
> > a
> > > requirement to authorize and audit table/column family access using
> > Ranger.
> > >
> > > I've configured the reads/writes to be performed under doAs to try to
> > make
> > > this happen but the requests end up being authorized and audit logged
> as
> > > the service user rather than the requestor.
> > >
> > >
> > > A snippet of the application code looks like this (doAsUser is the end
> > > user's username):
> > >
> > >
> > > UserGroupInformation ugi =
> UserGroupInformation.createProxyUser(doAsUser,
> > > UserGroupInformation.getLoginUser());
> > >
> > > try {
> > > ugi.doAs(new PriviledgedExceptionAction<Void>() {
> > > @Override
> > > public Void run() throws Exception {
> > > LOGGER.info("HBase put as user " +
> > ugi.getShortUserName());
> > > table.put(put);
> > > return null;
> > > }
> > > });
> > >
> > >
> > > Has anyone got experience with the HBase Ranger plugin and/or come
> across
> > > this problem before and know the best way to solve it?
> > >
> > > For reference this is all running with HDP 2.3.2.
> > >
> > > Thanks in advance!
> > >
> > > --
> > > Chris
> > >
> > > --
> > >
> > >
> > > *NOTICE AND DISCLAIMER*
> > >
> > > This email (including attachments) is confidential. If you are not the
> > > intended recipient, notify the sender immediately, delete this email
> from
> > > your system and do not disclose or use for any purpose.
> > >
> > > Business Address: Eagle House, 163 City Road, London, EC1V 1NR. United
> > > Kingdom
> > > Registered Office: Finsgate, 5-7 Cranwood Street, London, EC1V 9EE.
> > United
> > > Kingdom
> > > Big Data Partnership Limited is a company registered in England & Wales
> > > with Company No 7904824
> > >
> >
>
>
>
> --
> *Christopher Gent*
>
> *Managing Consultant*
> Big Data Partnership
> M: 07795 210205
> E: chris.gent@bigdatapartnership.com
>
> *NOTICE AND DISCLAIMER*
>
> This email (including attachments) is confidential. If you are not the
> intended recipient, notify the sender immediately, delete this email from
> your system and do not disclose or use for any purpose.
>
> Business Address: Eagle House, 163 City Road, London, EC1V 1NR. United
> Kingdom
> Registered Office: Finsgate, 5-7 Cranwood Street, London, EC1V 9EE. United
> Kingdom
> Big Data Partnership Limited is a company registered in England & Wales
> with Company No 7904824
>
> --
>
>
> *NOTICE AND DISCLAIMER*
>
> This email (including attachments) is confidential. If you are not the
> intended recipient, notify the sender immediately, delete this email from
> your system and do not disclose or use for any purpose.
>
> Business Address: Eagle House, 163 City Road, London, EC1V 1NR. United
> Kingdom
> Registered Office: Finsgate, 5-7 Cranwood Street, London, EC1V 9EE. United
> Kingdom
> Big Data Partnership Limited is a company registered in England & Wales
> with Company No 7904824
>
Re: doAs with HBase Java API and Apache Ranger
Posted by Chris Gent <ch...@bigdatapartnership.com>.
Hey Ted,
Yeah - they suggested asking over here :-)
I think the question is where the user context is set/comes from when using
the HBase API. It was suggested that it comes when the Table object gets
created? Or is it right back when the connection is established?
--
Chris
On 18 December 2015 at 17:18, Ted Yu <yu...@gmail.com> wrote:
> Have you polled Ranger community with this question ?
>
> http://ranger.apache.org/mail-lists.html
>
> Cheers
>
> On Fri, Dec 18, 2015 at 9:04 AM, Chris Gent <
> chris.gent@bigdatapartnership.com> wrote:
>
> > Hi,
> >
> > We have a webservice that performs reads/writes on HBase tables and have
> a
> > requirement to authorize and audit table/column family access using
> Ranger.
> >
> > I've configured the reads/writes to be performed under doAs to try to
> make
> > this happen but the requests end up being authorized and audit logged as
> > the service user rather than the requestor.
> >
> >
> > A snippet of the application code looks like this (doAsUser is the end
> > user's username):
> >
> >
> > UserGroupInformation ugi = UserGroupInformation.createProxyUser(doAsUser,
> > UserGroupInformation.getLoginUser());
> >
> > try {
> > ugi.doAs(new PriviledgedExceptionAction<Void>() {
> > @Override
> > public Void run() throws Exception {
> > LOGGER.info("HBase put as user " +
> ugi.getShortUserName());
> > table.put(put);
> > return null;
> > }
> > });
> >
> >
> > Has anyone got experience with the HBase Ranger plugin and/or come across
> > this problem before and know the best way to solve it?
> >
> > For reference this is all running with HDP 2.3.2.
> >
> > Thanks in advance!
> >
> > --
> > Chris
> >
> > --
> >
> >
> > *NOTICE AND DISCLAIMER*
> >
> > This email (including attachments) is confidential. If you are not the
> > intended recipient, notify the sender immediately, delete this email from
> > your system and do not disclose or use for any purpose.
> >
> > Business Address: Eagle House, 163 City Road, London, EC1V 1NR. United
> > Kingdom
> > Registered Office: Finsgate, 5-7 Cranwood Street, London, EC1V 9EE.
> United
> > Kingdom
> > Big Data Partnership Limited is a company registered in England & Wales
> > with Company No 7904824
> >
>
--
*Christopher Gent*
*Managing Consultant*
Big Data Partnership
M: 07795 210205
E: chris.gent@bigdatapartnership.com
*NOTICE AND DISCLAIMER*
This email (including attachments) is confidential. If you are not the
intended recipient, notify the sender immediately, delete this email from
your system and do not disclose or use for any purpose.
Business Address: Eagle House, 163 City Road, London, EC1V 1NR. United
Kingdom
Registered Office: Finsgate, 5-7 Cranwood Street, London, EC1V 9EE. United
Kingdom
Big Data Partnership Limited is a company registered in England & Wales
with Company No 7904824
--
*NOTICE AND DISCLAIMER*
This email (including attachments) is confidential. If you are not the
intended recipient, notify the sender immediately, delete this email from
your system and do not disclose or use for any purpose.
Business Address: Eagle House, 163 City Road, London, EC1V 1NR. United
Kingdom
Registered Office: Finsgate, 5-7 Cranwood Street, London, EC1V 9EE. United
Kingdom
Big Data Partnership Limited is a company registered in England & Wales
with Company No 7904824
Re: doAs with HBase Java API and Apache Ranger
Posted by Ted Yu <yu...@gmail.com>.
Have you polled Ranger community with this question ?
http://ranger.apache.org/mail-lists.html
Cheers
On Fri, Dec 18, 2015 at 9:04 AM, Chris Gent <
chris.gent@bigdatapartnership.com> wrote:
> Hi,
>
> We have a webservice that performs reads/writes on HBase tables and have a
> requirement to authorize and audit table/column family access using Ranger.
>
> I've configured the reads/writes to be performed under doAs to try to make
> this happen but the requests end up being authorized and audit logged as
> the service user rather than the requestor.
>
>
> A snippet of the application code looks like this (doAsUser is the end
> user's username):
>
>
> UserGroupInformation ugi = UserGroupInformation.createProxyUser(doAsUser,
> UserGroupInformation.getLoginUser());
>
> try {
> ugi.doAs(new PriviledgedExceptionAction<Void>() {
> @Override
> public Void run() throws Exception {
> LOGGER.info("HBase put as user " + ugi.getShortUserName());
> table.put(put);
> return null;
> }
> });
>
>
> Has anyone got experience with the HBase Ranger plugin and/or come across
> this problem before and know the best way to solve it?
>
> For reference this is all running with HDP 2.3.2.
>
> Thanks in advance!
>
> --
> Chris
>
> --
>
>
> *NOTICE AND DISCLAIMER*
>
> This email (including attachments) is confidential. If you are not the
> intended recipient, notify the sender immediately, delete this email from
> your system and do not disclose or use for any purpose.
>
> Business Address: Eagle House, 163 City Road, London, EC1V 1NR. United
> Kingdom
> Registered Office: Finsgate, 5-7 Cranwood Street, London, EC1V 9EE. United
> Kingdom
> Big Data Partnership Limited is a company registered in England & Wales
> with Company No 7904824
>