You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by Suresh Subbiah <su...@gmail.com> on 2015/10/11 00:58:58 UTC
Unexpected behaviour when VisibilityController coprocessor is used
Hi,
When I run the following script from hbase shell the last scan returns no
rows
create 'visibilityTest', 'f1'
put 'visibilityTest', 'r1', 'f1:c1', 'value1'
deleteall 'visibilityTest', 'r1'
put 'visibilityTest', 'r1', 'f1:c1', 'value2'
scan 'visibilityTest'
*hbase(main):013:0> scan 'visibilityTest'*
*ROW COLUMN+CELL
*
*0 row(s) in 0.0100 seconds*
However if I run
scan 'visibilityTest' , {RAW=>TRUE}
I see that the second row that I put is indeed there and has a timestamp
value higher that the previous delete
*hbase(main):014:0> scan 'visibilityTest', {RAW=>TRUE}*
*ROW COLUMN+CELL
*
* r1 column=f1:, timestamp=1444516578296,
type=DeleteFamily *
* r1 column=f1:c1, timestamp=1444516647655, value=value2
*
*1 row(s) in 0.0110 seconds*
This is on hbase 0.98.6. Problem is seen only when hbase-site.xml has
these lines. No other coprocessors were used during this test.
<property>
<name>hbase.coprocessor.region.classes</name>
<value>org.apache.hadoop.hbase.security.visibility.VisibilityController</value>
</property>
<property>
<name>hbase.coprocessor.master.classes</name>
<value>org.apache.hadoop.hbase.security.visibility.VisibilityController</value>
</property>
<property>
<name>hfile.format.version</name>
<value>3</value>
</property>
Any suggestions of what I may be doing incorrectly? Or is this a bug?
Thank you
Suresh
Re: Unexpected behaviour when VisibilityController coprocessor is used
Posted by Anoop John <an...@gmail.com>.
Its our pleasure to help you guys. No problem. Do write back with issues
if u find any. :-)
Anoop
On Tuesday, October 13, 2015, Suresh Subbiah <su...@gmail.com>
wrote:
> Thank you Anoop, Ram, Ted.
> Yes it is likely due to https://issues.apache.org/jira/browse/HBASE-13734
>
> Narendra Goyal from our team verified a short while ago that 1.0.2 indeed
> does not have this problem.
> This is for the Trafodion SQL engine. We have some transactional coprocs
> that run on 1.0.0 but are yet to be tested with 1.0.2.
> We will do that now.
>
> Thank you very much for your help. At the Trafodion incubation user dlist
> we try to learn from you'll and be as responsive and helpful. Thank you
for
> being such an excellent role model.
>
> BTW we were able to use the suggestions from the previous round of email
> exchanges and use HFile utility to see the tags associated with a
> visibility label.
>
> Thank you
> Suresh
>
>
>
> On Mon, Oct 12, 2015 at 10:43 PM, ramkrishna vasudevan <
> ramkrishna.s.vasudevan@gmail.com> wrote:
>
>> I tried it on the latest trunk and this issue is not there. So as Anoop
>> said the latest version of 0.98 should be solving this problem.
>> @Suresh
>> Let us know if you still find the issue in later versions of 0.98 and we
>> can work on it to solve the problem.
>>
>> Regards
>> Ram
>>
>> On Tue, Oct 13, 2015 at 9:09 AM, Anoop John <an...@gmail.com>
wrote:
>>
>> > Yes as such there is not mandatory to use AC along with VC. It can be
>> used
>> > alone..
>> > I believe u r getting the bug HBASE-13734. This is fixed in 98.13
only.
>> > Just change ur version from 98.6 to 98.13 and test once. Let us know
>> how
>> > is it then.
>> >
>> > -Anoop-
>> >
>> > On Tue, Oct 13, 2015 at 9:01 AM, ramkrishna vasudevan <
>> > ramkrishna.s.vasudevan@gmail.com> wrote:
>> >
>> > > I think, even with only configuring VisibilityController there should
>> not
>> > > be a different behaviour, considering the fact that there are no
>> > visibility
>> > > labels. With just VisibilityController configured and doing puts and
>> > scans
>> > > using super user let me check what is happening.
>> > >
>> > > Regards
>> > > Ram
>> > >
>> > > On Tue, Oct 13, 2015 at 8:47 AM, Anoop John <an...@gmail.com>
>> > wrote:
>> > >
>> > > > Hi Suresh
>> > > > You said abt doing test as an HBase super user. You mean even
>> when
>> > > scan
>> > > > is issues as a super user, u are not getting the rows back?
>> > > >
>> > > > -Anoop-
>> > > >
>> > > > On Tue, Oct 13, 2015 at 4:06 AM, Ted Yu <yu...@gmail.com>
wrote:
>> > > >
>> > > > > Convention is to put AccessController ahead of
VisibilityController
>> > in
>> > > > > hbase-site.xml
>> > > > >
>> > > > > Took a quick pass over region server log but haven't found much
>> yet.
>> > > > >
>> > > > > FYI
>> > > > >
>> > > > > On Mon, Oct 12, 2015 at 3:28 PM, Suresh Subbiah <
>> > > > > suresh.subbiah60@gmail.com>
>> > > > > wrote:
>> > > > >
>> > > > > > Hi Ted,
>> > > > > >
>> > > > > > Thank you. Yes HDFS cluster has also been kerberized. BTW, this
>> is
>> > a
>> > > > > > "cluster" with only one node.
>> > > > > >
>> > > > > > Master hbase-site.xml, RS hbase-site.ml and RS log for the time
>> > > > interval
>> > > > > > test was run is attached
>> > > > > >
>> > > > > > http://pastebin.com/zuqCC4xG
>> > > > > > http://pastebin.com/88Wx0KDf
>> > > > > > http://pastebin.com/QZqihN1W
>> > > > > >
>> > > > > > Will try deploying 1.1.2 next.
>> > > > > >
>> > > > > > Thanks
>> > > > > > Suresh
>> > > > > >
>> > > > > >
>> > > > > >
>> > > > > > On Mon, Oct 12, 2015 at 3:46 PM, Ted Yu <yu...@gmail.com>
>> > wrote:
>> > > > > >
>> > > > > > > bq. cluster enabled for secure HBase with kerberos
>> > > > > > >
>> > > > > > > I assume your hdfs cluster has also been kerberized.
>> > > > > > >
>> > > > > > > Please pastebin the complete hbase-site.xml
>> > > > > > >
>> > > > > > > Please turn on DEBUG logging and pastebin the region server
log
>> > > which
>> > > > > > hosts
>> > > > > > > visibilityTest
>> > > > > > >
>> > > > > > > BTW if possible, can you deploy 1.1.2 ?
>> > > > > > >
>> > > > > > > Cheers
>> > > > > > >
>> > > > > > > On Mon, Oct 12, 2015 at 1:14 PM, Suresh Subbiah <
>> > > > > > > suresh.subbiah60@gmail.com>
>> > > > > > > wrote:
>> > > > > > >
>> > > > > > > > Hi Ted,
>> > > > > > > >
>> > > > > > > > I understand that using VisibilityController on an
unsercure
>> > > > cluster
>> > > > > is
>> > > > > > > of
>> > > > > > > > limited value. I am still in the early stages of my task. I
>> am
>> > > > logged
>> > > > > > in
>> > > > > > > as
>> > > > > > > > HBase super user and was simply checking if rows could be
>> > > accessed.
>> > > > > > > >
>> > > > > > > > With my colleague's help we did get the cluster enabled for
>> > > secure
>> > > > > > HBase
>> > > > > > > > with kerberos. I repeated the test to get the same result.
>> Our
>> > > > > cluster
>> > > > > > is
>> > > > > > > > on 1.0. Do you think I may be doing something incorrectly?
>> What
>> > > > > > > information
>> > > > > > > > can I send to help ensure that I have not made a mistake.
>> > > > > > > >
>> > > > > > > > Thanks
>> > > > > > > > Suresh
>> > > > > > > >
>> > > > > > > > hbase shell
>> > > > > > > > 15/10/12 14:35:09 INFO Configuration.deprecation:
>> > > hadoop.native.lib
>> > > > > is
>> > > > > > > > deprecated. Instead, use io.native.lib.available
>> > > > > > > > HBase Shell; enter 'help<RETURN>' for list of supported
>> > commands.
>> > > > > > > > Type "exit<RETURN>" to leave the HBase Shell
>> > > > > > > > Version 1.0.0-cdh5.4.4, rUnknown, Mon Jul 6 16:59:55 PDT
>> 2015
>> > > > > > > >
>> > > > > > > > hbase(main):001:0> create 'visibilityTest', 'f1'
>> > > > > > > > 0 row(s) in 0.7780 seconds
>> > > > > > > >
>> > > > > > > > => Hbase::Table - visibilityTest
>> > > > > > > > hbase(main):002:0> put 'visibilityTest', 'r1', 'f1:c1',
>> > 'value1'
>> > > > > > > > 0 row(s) in 0.1300 seconds
>> > > > > > > >
>> > > > > > > > hbase(main):003:0> deleteall 'visibilityTest', 'r1'
>> > > > > > > > 0 row(s) in 0.0330 seconds
>> > > > > > > >
>> > > > > > > > hbase(main):004:0> put 'visibilityTest', 'r1', 'f1:c1',
>> > 'value2'
>> > > > > > > > 0 row(s) in 0.0150 seconds
>> > > > > > > >
>> > > > > > > > hbase(main):005:0> scan 'visibilityTest'
>> > > > > > > > ROW COLUMN+CELL
>> > > > > > > >
>> > > > > > > > 0 row(s) in 0.0550 seconds
>> > > > > > > >
>> > > > > > > > hbase(main):006:0> scan 'visibilityTest', {RAW=>TRUE}
>> > > > > > > > ROW COLUMN+CELL
>> > > > > > > >
>> > > > > > > > r1 column=f1:, timestamp=1444660561138,
>> > > > > > > > type=DeleteFamily
>> > > > > > > > r1 column=f1:c1,
timestamp=1444660576868,
>> > > > > > value=value2
>> > > > > > > >
>> > > > > > > > 1 row(s) in 0.0370 seconds
>> > > > > > > >
>> > > > > > > > -----------------------------------------------------
>> > > > > > > > <property>
>> > > > > > > > <name>hbase.coprocessor.master.classes</name>
>> > > > > > > >
>> > > > > > > >
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>>
<value>org.apache.hadoop.hbase.security.visibility.VisibilityController,org.apache.hadoop.hbase.security.access.AccessController</value>
>> > > > > > > > </property>
>> > > > > > > >
>> > > > > > > > <property>
>> > > > > > > > <name>hbase.coprocessor.region.classes</name>
>> > > > > > > >
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>>
<value>org.apache.hadoop.hbase.security.visibility.VisibilityController,org.apache.hadoop.hbase.security.token.TokenProvider,org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint,org.apache.hadoop.hbase.security.access.AccessController</value>
>> > > > > > > > </property>
>> > > > > > > >
>> > > > > > > > --------------------------------------------------------
>> > > > > > > >
>> > > > > > > >
>> > > > > > > >
>> > > > > > > >
>> > > > > > > > On Sat, Oct 10, 2015 at 9:51 PM, Ted Yu <
yuzhihong@gmail.com
>> >
>> > > > wrote:
>> > > > > > > >
>> > > > > > > > > To my understanding, VisibilityController is used in a
>> secure
>> > > > > > cluster.
>> > > > > > > > > Without security, how do you enforce that only select
>> user(s)
>> > > can
>> > > > > > > access
>> > > > > > > > > certain cells ?
>> > > > > > > > >
>> > > > > > > > > Please see the following sections in refguide:
>> > > > > > > > >
>> > > > > > > > >
>> http://hbase.apache.org/book.html#hbase.secure.configuration
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>>
http://hbase.apache.org/book.html#_server_side_configuration_for_simple_user_access_operation
>> > > > > > > > >
>> > > > > > > > > On Sat, Oct 10, 2015 at 7:40 PM, Suresh Subbiah <
>> > > > > > > > > suresh.subbiah60@gmail.com>
>> > > > > > > > > wrote:
>> > > > > > > > >
>> > > > > > > > > > Hi Ted,
>> > > > > > > > > >
>> > > > > > > > > > Thank you for your response.
>> > > > > > > > > > I found a machine with HBase 1.0.0 and tried the script
>> > with
>> > > > all
>> > > > > 6
>> > > > > > > > > coprocs
>> > > > > > > > > > you listed (2 in master, and 4 in RS). I still do not
see
>> > the
>> > > > row
>> > > > > > > after
>> > > > > > > > > the
>> > > > > > > > > > second scan.
>> > > > > > > > > >
>> > > > > > > > > > However my cluster is not secure enabled I think. Is
that
>> > > > > > necessary?
>> > > > > > > I
>> > > > > > > > am
>> > > > > > > > > > not sure how to do that, though I can ask other members
>> of
>> > my
>> > > > > team
>> > > > > > > and
>> > > > > > > > > try
>> > > > > > > > > > it if that will help.
>> > > > > > > > > >
>> > > > > > > > > > It will be ideal if we could get this to work on a 1.0
>> > based
>> > > > > > version.
>> > > > > > > > > > Moving to 1.1 will take more time since we have some
>> > > > > dependencies.
>> > > > > > > > > >
>> > > > > > > > > > Thank you
>> > > > > > > > > > Suresh
>> > > > > > > > > >
>> > > > > > > > > > 15/10/10 19:20:44 INFO Configuration.deprecation:
>> > > > > hadoop.native.lib
>> > > > > > > is
>> > > > > > > > > > deprecated. Instead, use io.native.lib.available
>> > > > > > > > > > HBase Shell; enter 'help<RETURN>' for list of supported
>> > > > commands.
>> > > > > > > > > > Type "exit<RETURN>" to leave the HBase Shell
>> > > > > > > > > > Version 1.0.0-cdh5.4.4, rUnknown, Mon Jul 6 16:59:55
PDT
>> > > 2015
>> > > > > > > > > >
>> > > > > > > > > > *hbase(main):001:0> create 'visibilityTest', 'f1' *
>> > > > > > > > > > *0 row(s) in 0.5460 seconds*
>> > > > > > > > > >
>> > > > > > > > > > *=> Hbase::Table - visibilityTest*
>> > > > > > > > > > *hbase(main):002:0> put 'visibilityTest', 'r1',
'f1:c1',
>> > > > > 'value1' *
>> > > > > > > > > > *0 row(s) in 0.0670 seconds*
>> > > > > > > > > >
>> > > > > > > > > > *hbase(main):003:0> deleteall 'visibilityTest', 'r1' *
>> > > > > > > > > > *0 row(s) in 0.0090 seconds*
>> > > > > > > > > >
>> > > > > > > > > > *hbase(main):004:0> put 'visibilityTest', 'r1',
'f1:c1',
>> > > > > 'value2'*
>> > > > > > > > > > *0 row(s) in 0.0040 seconds*
>> > > > > > > > > >
>> > > > > > > > > > *hbase(main):005:0> scan 'visibilityTest'*
>> > > > > > > > > > *ROW COLUMN+CELL
>> > > > > > > > > > *
>> > > > > > > > > > *0 row(s) in 0.0160 seconds*
>> > > > > > > > > >
>> > > > > > > > > > *hbase(main):006:0> scan 'visibilityTest', {RAW=>TRUE}*
>> > > > > > > > > > *ROW COLUMN+CELL
>> > > > > > > > > > *
>> > > > > > > > > > * r1 column=f1:,
>> timestamp=1444530064056,
>> > > > > > > > > > type=DeleteFamily *
>> > > > > > > > > > * r1 column=f1:c1,
>> > timestamp=1444530064084,
>> > > > > > > > > value=value2
>> > > > > > > > > > *
>> > > > > > > > > > *1 row(s) in 0.0580 seconds*
>> > > > > > > > > >
>> > > > > > > > > > *hbase(main):007:0> exit*
>> > > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > > On Sat, Oct 10, 2015 at 7:26 PM, Ted Yu <
>> > yuzhihong@gmail.com
>> > > >
>> > > > > > wrote:
>> > > > > > > > > >
>> > > > > > > > > > > I tried the sequence of commands from your example
on a
>> > > > secure
>> > > > > > > 1.1.2
>> > > > > > > > > > > cluster with the following config:
>> > > > > > > > > > >
>> > > > > > > > > > > <property>
>> > > > > > > > > > > <name>hbase.coprocessor.master.classes</name>
>> > > > > > > > > > >
>> > > > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > >
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>>
<value>org.apache.hadoop.hbase.security.access.AccessController,org.apache.hadoop.hbase.security.visibility.VisibilityController</value>
>> > > > > > > > > > > </property>
>> > > > > > > > > > > <property>
>> > > > > > > > > > > <name>hbase.coprocessor.region.classes</name>
>> > > > > > > > > > >
>> > > > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > >
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>>
<value>org.apache.hadoop.hbase.security.token.TokenProvider,org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint,org.apache.hadoop.hbase.security.access.AccessController,org.apache.hadoop.hbase.security.visibility.VisibilityController</value>
>> > > > > > > > > > > </property>
>> > > > > > > > > > >
>> > > > > > > > > > > I got:
>> > > > > > > > > > >
>> > > > > > > > > > > hbase(main):005:0> scan 'visibilityTest'
>> > > > > > > > > > > ROW
>> COLUMN+CELL
>> > > > > > > > > > > r1
>> > column=f1:c1,
>> > > > > > > > > > > timestamp=1444522994981, value=value2
>> > > > > > > > > > > 1 row(s) in 0.1020 seconds
>> > > > > > > > > > >
>> > > > > > > > > > > Can you try again with 0.98.15 release whose vote
>> passed
>> > > > Friday
>> > > > > > to
>> > > > > > > > see
>> > > > > > > > > if
>> > > > > > > > > > > what you observed can be reproduced ?
>> > > > > > > > > > >
>> > > > > > > > > > > Cheers
>> > > > > > > > > > >
>> > > > > > > > > > > On Sat, Oct 10, 2015 at 3:58 PM, Suresh Subbiah <
>> > > > > > > > > > > suresh.subbiah60@gmail.com>
>> > > > > > > > > > > wrote:
>> > > > > > > > > > >
>> > > > > > > > > > > > Hi,
>> > > > > > > > > > > >
>> > > > > > > > > > > > When I run the following script from hbase shell
the
>> > last
>> > > > > scan
>> > > > > > > > > returns
>> > > > > > > > > > no
>> > > > > > > > > > > > rows
>> > > > > > > > > > > >
>> > > > > > > > > > > > create 'visibilityTest', 'f1'
>> > > > > > > > > > > > put 'visibilityTest', 'r1', 'f1:c1', 'value1'
>> > > > > > > > > > > > deleteall 'visibilityTest', 'r1'
>> > > > > > > > > > > > put 'visibilityTest', 'r1', 'f1:c1', 'value2'
>> > > > > > > > > > > > scan 'visibilityTest'
>> > > > > > > > > > > >
>> > > > > > > > > > > > *hbase(main):013:0> scan 'visibilityTest'*
>> > > > > > > > > > > > *ROW COLUMN+CELL
>> > > > > > > > > > > > *
>> > > > > > > > > > > > *0 row(s) in 0.0100 seconds*
>> > > > > > > > > > > >
>> > > > > > > > > > > >
>> > > > > > > > > > > > However if I run
>> > > > > > > > > > > > scan 'visibilityTest' , {RAW=>TRUE}
>> > > > > > > > > > > >
>> > > > > > > > > > > > I see that the second row that I put is indeed
there
>> > and
>> > > > has
>> > > > > a
>> > > > > > > > > > timestamp
>> > > > > > > > > > > > value higher that the previous delete
>> > > > > > > > > > > >
>> > > > > > > > > > > > *hbase(main):014:0> scan 'visibilityTest',
>> {RAW=>TRUE}*
>> > > > > > > > > > > > *ROW COLUMN+CELL
>> > > > > > > > > > > > *
>> > > > > > > > > > > > * r1 column=f1:,
>> > > timestamp=1444516578296,
>> > > > > > > > > > > > type=DeleteFamily *
>> > > > > > > > > > > > * r1 column=f1:c1,
>> > > > timestamp=1444516647655,
>> > > > > > > > > > > value=value2
>> > > > > > > > > > > > *
>> > > > > > > > > > > > *1 row(s) in 0.0110 seconds*
>> > > > > > > > > > > >
>> > > > > > > > > > > >
>> > > > > > > > > > > >
>> > > > > > > > > > > > This is on hbase 0.98.6. Problem is seen only when
>> > > > > > > hbase-site.xml
>> > > > > > > > > has
>> > > > > > > > > > > > these lines. No other coprocessors were used during
>> > this
>> > > > > test.
>> > > > > > > > > > > >
>> > > > > > > > > > > > <property>
>> > > > > > > > > > > > <name>hbase.coprocessor.region.classes</name>
>> > > > > > > > > > > >
>> > > > > > > > > > > >
>> > > > > > > > > > > >
>> > > > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > >
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>>
<value>org.apache.hadoop.hbase.security.visibility.VisibilityController</value>
>> > > > > > > > > > > > </property>
>> > > > > > > > > > > > <property>
>> > > > > > > > > > > > <name>hbase.coprocessor.master.classes</name>
>> > > > > > > > > > > >
>> > > > > > > > > > > >
>> > > > > > > > > > > >
>> > > > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > >
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>>
<value>org.apache.hadoop.hbase.security.visibility.VisibilityController</value>
>> > > > > > > > > > > > </property>
>> > > > > > > > > > > > <property>
>> > > > > > > > > > > > <name>hfile.format.version</name>
>> > > > > > > > > > > > <value>3</value>
>> > > > > > > > > > > > </property>
>> > > > > > > > > > > >
>> > > > > > > > > > > > Any suggestions of what I may be doing incorrectly?
>> Or
>> > is
>> > > > > this
>> > > > > > a
>> > > > > > > > bug?
>> > > > > > > > > > > >
>> > > > > > > > > > > > Thank you
>> > > > > > > > > > > > Suresh
>> > > > > > > > > > > >
>> > > > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > >
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>>
>
Re: Unexpected behaviour when VisibilityController coprocessor is used
Posted by Suresh Subbiah <su...@gmail.com>.
Thank you Anoop, Ram, Ted.
Yes it is likely due to https://issues.apache.org/jira/browse/HBASE-13734
Narendra Goyal from our team verified a short while ago that 1.0.2 indeed
does not have this problem.
This is for the Trafodion SQL engine. We have some transactional coprocs
that run on 1.0.0 but are yet to be tested with 1.0.2.
We will do that now.
Thank you very much for your help. At the Trafodion incubation user dlist
we try to learn from you'll and be as responsive and helpful. Thank you for
being such an excellent role model.
BTW we were able to use the suggestions from the previous round of email
exchanges and use HFile utility to see the tags associated with a
visibility label.
Thank you
Suresh
On Mon, Oct 12, 2015 at 10:43 PM, ramkrishna vasudevan <
ramkrishna.s.vasudevan@gmail.com> wrote:
> I tried it on the latest trunk and this issue is not there. So as Anoop
> said the latest version of 0.98 should be solving this problem.
> @Suresh
> Let us know if you still find the issue in later versions of 0.98 and we
> can work on it to solve the problem.
>
> Regards
> Ram
>
> On Tue, Oct 13, 2015 at 9:09 AM, Anoop John <an...@gmail.com> wrote:
>
> > Yes as such there is not mandatory to use AC along with VC. It can be
> used
> > alone..
> > I believe u r getting the bug HBASE-13734. This is fixed in 98.13 only.
> > Just change ur version from 98.6 to 98.13 and test once. Let us know
> how
> > is it then.
> >
> > -Anoop-
> >
> > On Tue, Oct 13, 2015 at 9:01 AM, ramkrishna vasudevan <
> > ramkrishna.s.vasudevan@gmail.com> wrote:
> >
> > > I think, even with only configuring VisibilityController there should
> not
> > > be a different behaviour, considering the fact that there are no
> > visibility
> > > labels. With just VisibilityController configured and doing puts and
> > scans
> > > using super user let me check what is happening.
> > >
> > > Regards
> > > Ram
> > >
> > > On Tue, Oct 13, 2015 at 8:47 AM, Anoop John <an...@gmail.com>
> > wrote:
> > >
> > > > Hi Suresh
> > > > You said abt doing test as an HBase super user. You mean even
> when
> > > scan
> > > > is issues as a super user, u are not getting the rows back?
> > > >
> > > > -Anoop-
> > > >
> > > > On Tue, Oct 13, 2015 at 4:06 AM, Ted Yu <yu...@gmail.com> wrote:
> > > >
> > > > > Convention is to put AccessController ahead of VisibilityController
> > in
> > > > > hbase-site.xml
> > > > >
> > > > > Took a quick pass over region server log but haven't found much
> yet.
> > > > >
> > > > > FYI
> > > > >
> > > > > On Mon, Oct 12, 2015 at 3:28 PM, Suresh Subbiah <
> > > > > suresh.subbiah60@gmail.com>
> > > > > wrote:
> > > > >
> > > > > > Hi Ted,
> > > > > >
> > > > > > Thank you. Yes HDFS cluster has also been kerberized. BTW, this
> is
> > a
> > > > > > "cluster" with only one node.
> > > > > >
> > > > > > Master hbase-site.xml, RS hbase-site.ml and RS log for the time
> > > > interval
> > > > > > test was run is attached
> > > > > >
> > > > > > http://pastebin.com/zuqCC4xG
> > > > > > http://pastebin.com/88Wx0KDf
> > > > > > http://pastebin.com/QZqihN1W
> > > > > >
> > > > > > Will try deploying 1.1.2 next.
> > > > > >
> > > > > > Thanks
> > > > > > Suresh
> > > > > >
> > > > > >
> > > > > >
> > > > > > On Mon, Oct 12, 2015 at 3:46 PM, Ted Yu <yu...@gmail.com>
> > wrote:
> > > > > >
> > > > > > > bq. cluster enabled for secure HBase with kerberos
> > > > > > >
> > > > > > > I assume your hdfs cluster has also been kerberized.
> > > > > > >
> > > > > > > Please pastebin the complete hbase-site.xml
> > > > > > >
> > > > > > > Please turn on DEBUG logging and pastebin the region server log
> > > which
> > > > > > hosts
> > > > > > > visibilityTest
> > > > > > >
> > > > > > > BTW if possible, can you deploy 1.1.2 ?
> > > > > > >
> > > > > > > Cheers
> > > > > > >
> > > > > > > On Mon, Oct 12, 2015 at 1:14 PM, Suresh Subbiah <
> > > > > > > suresh.subbiah60@gmail.com>
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Hi Ted,
> > > > > > > >
> > > > > > > > I understand that using VisibilityController on an unsercure
> > > > cluster
> > > > > is
> > > > > > > of
> > > > > > > > limited value. I am still in the early stages of my task. I
> am
> > > > logged
> > > > > > in
> > > > > > > as
> > > > > > > > HBase super user and was simply checking if rows could be
> > > accessed.
> > > > > > > >
> > > > > > > > With my colleague's help we did get the cluster enabled for
> > > secure
> > > > > > HBase
> > > > > > > > with kerberos. I repeated the test to get the same result.
> Our
> > > > > cluster
> > > > > > is
> > > > > > > > on 1.0. Do you think I may be doing something incorrectly?
> What
> > > > > > > information
> > > > > > > > can I send to help ensure that I have not made a mistake.
> > > > > > > >
> > > > > > > > Thanks
> > > > > > > > Suresh
> > > > > > > >
> > > > > > > > hbase shell
> > > > > > > > 15/10/12 14:35:09 INFO Configuration.deprecation:
> > > hadoop.native.lib
> > > > > is
> > > > > > > > deprecated. Instead, use io.native.lib.available
> > > > > > > > HBase Shell; enter 'help<RETURN>' for list of supported
> > commands.
> > > > > > > > Type "exit<RETURN>" to leave the HBase Shell
> > > > > > > > Version 1.0.0-cdh5.4.4, rUnknown, Mon Jul 6 16:59:55 PDT
> 2015
> > > > > > > >
> > > > > > > > hbase(main):001:0> create 'visibilityTest', 'f1'
> > > > > > > > 0 row(s) in 0.7780 seconds
> > > > > > > >
> > > > > > > > => Hbase::Table - visibilityTest
> > > > > > > > hbase(main):002:0> put 'visibilityTest', 'r1', 'f1:c1',
> > 'value1'
> > > > > > > > 0 row(s) in 0.1300 seconds
> > > > > > > >
> > > > > > > > hbase(main):003:0> deleteall 'visibilityTest', 'r1'
> > > > > > > > 0 row(s) in 0.0330 seconds
> > > > > > > >
> > > > > > > > hbase(main):004:0> put 'visibilityTest', 'r1', 'f1:c1',
> > 'value2'
> > > > > > > > 0 row(s) in 0.0150 seconds
> > > > > > > >
> > > > > > > > hbase(main):005:0> scan 'visibilityTest'
> > > > > > > > ROW COLUMN+CELL
> > > > > > > >
> > > > > > > > 0 row(s) in 0.0550 seconds
> > > > > > > >
> > > > > > > > hbase(main):006:0> scan 'visibilityTest', {RAW=>TRUE}
> > > > > > > > ROW COLUMN+CELL
> > > > > > > >
> > > > > > > > r1 column=f1:, timestamp=1444660561138,
> > > > > > > > type=DeleteFamily
> > > > > > > > r1 column=f1:c1, timestamp=1444660576868,
> > > > > > value=value2
> > > > > > > >
> > > > > > > > 1 row(s) in 0.0370 seconds
> > > > > > > >
> > > > > > > > -----------------------------------------------------
> > > > > > > > <property>
> > > > > > > > <name>hbase.coprocessor.master.classes</name>
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> <value>org.apache.hadoop.hbase.security.visibility.VisibilityController,org.apache.hadoop.hbase.security.access.AccessController</value>
> > > > > > > > </property>
> > > > > > > >
> > > > > > > > <property>
> > > > > > > > <name>hbase.coprocessor.region.classes</name>
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> <value>org.apache.hadoop.hbase.security.visibility.VisibilityController,org.apache.hadoop.hbase.security.token.TokenProvider,org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint,org.apache.hadoop.hbase.security.access.AccessController</value>
> > > > > > > > </property>
> > > > > > > >
> > > > > > > > --------------------------------------------------------
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > On Sat, Oct 10, 2015 at 9:51 PM, Ted Yu <yuzhihong@gmail.com
> >
> > > > wrote:
> > > > > > > >
> > > > > > > > > To my understanding, VisibilityController is used in a
> secure
> > > > > > cluster.
> > > > > > > > > Without security, how do you enforce that only select
> user(s)
> > > can
> > > > > > > access
> > > > > > > > > certain cells ?
> > > > > > > > >
> > > > > > > > > Please see the following sections in refguide:
> > > > > > > > >
> > > > > > > > >
> http://hbase.apache.org/book.html#hbase.secure.configuration
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> http://hbase.apache.org/book.html#_server_side_configuration_for_simple_user_access_operation
> > > > > > > > >
> > > > > > > > > On Sat, Oct 10, 2015 at 7:40 PM, Suresh Subbiah <
> > > > > > > > > suresh.subbiah60@gmail.com>
> > > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > > Hi Ted,
> > > > > > > > > >
> > > > > > > > > > Thank you for your response.
> > > > > > > > > > I found a machine with HBase 1.0.0 and tried the script
> > with
> > > > all
> > > > > 6
> > > > > > > > > coprocs
> > > > > > > > > > you listed (2 in master, and 4 in RS). I still do not see
> > the
> > > > row
> > > > > > > after
> > > > > > > > > the
> > > > > > > > > > second scan.
> > > > > > > > > >
> > > > > > > > > > However my cluster is not secure enabled I think. Is that
> > > > > > necessary?
> > > > > > > I
> > > > > > > > am
> > > > > > > > > > not sure how to do that, though I can ask other members
> of
> > my
> > > > > team
> > > > > > > and
> > > > > > > > > try
> > > > > > > > > > it if that will help.
> > > > > > > > > >
> > > > > > > > > > It will be ideal if we could get this to work on a 1.0
> > based
> > > > > > version.
> > > > > > > > > > Moving to 1.1 will take more time since we have some
> > > > > dependencies.
> > > > > > > > > >
> > > > > > > > > > Thank you
> > > > > > > > > > Suresh
> > > > > > > > > >
> > > > > > > > > > 15/10/10 19:20:44 INFO Configuration.deprecation:
> > > > > hadoop.native.lib
> > > > > > > is
> > > > > > > > > > deprecated. Instead, use io.native.lib.available
> > > > > > > > > > HBase Shell; enter 'help<RETURN>' for list of supported
> > > > commands.
> > > > > > > > > > Type "exit<RETURN>" to leave the HBase Shell
> > > > > > > > > > Version 1.0.0-cdh5.4.4, rUnknown, Mon Jul 6 16:59:55 PDT
> > > 2015
> > > > > > > > > >
> > > > > > > > > > *hbase(main):001:0> create 'visibilityTest', 'f1' *
> > > > > > > > > > *0 row(s) in 0.5460 seconds*
> > > > > > > > > >
> > > > > > > > > > *=> Hbase::Table - visibilityTest*
> > > > > > > > > > *hbase(main):002:0> put 'visibilityTest', 'r1', 'f1:c1',
> > > > > 'value1' *
> > > > > > > > > > *0 row(s) in 0.0670 seconds*
> > > > > > > > > >
> > > > > > > > > > *hbase(main):003:0> deleteall 'visibilityTest', 'r1' *
> > > > > > > > > > *0 row(s) in 0.0090 seconds*
> > > > > > > > > >
> > > > > > > > > > *hbase(main):004:0> put 'visibilityTest', 'r1', 'f1:c1',
> > > > > 'value2'*
> > > > > > > > > > *0 row(s) in 0.0040 seconds*
> > > > > > > > > >
> > > > > > > > > > *hbase(main):005:0> scan 'visibilityTest'*
> > > > > > > > > > *ROW COLUMN+CELL
> > > > > > > > > > *
> > > > > > > > > > *0 row(s) in 0.0160 seconds*
> > > > > > > > > >
> > > > > > > > > > *hbase(main):006:0> scan 'visibilityTest', {RAW=>TRUE}*
> > > > > > > > > > *ROW COLUMN+CELL
> > > > > > > > > > *
> > > > > > > > > > * r1 column=f1:,
> timestamp=1444530064056,
> > > > > > > > > > type=DeleteFamily *
> > > > > > > > > > * r1 column=f1:c1,
> > timestamp=1444530064084,
> > > > > > > > > value=value2
> > > > > > > > > > *
> > > > > > > > > > *1 row(s) in 0.0580 seconds*
> > > > > > > > > >
> > > > > > > > > > *hbase(main):007:0> exit*
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > On Sat, Oct 10, 2015 at 7:26 PM, Ted Yu <
> > yuzhihong@gmail.com
> > > >
> > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > > I tried the sequence of commands from your example on a
> > > > secure
> > > > > > > 1.1.2
> > > > > > > > > > > cluster with the following config:
> > > > > > > > > > >
> > > > > > > > > > > <property>
> > > > > > > > > > > <name>hbase.coprocessor.master.classes</name>
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> <value>org.apache.hadoop.hbase.security.access.AccessController,org.apache.hadoop.hbase.security.visibility.VisibilityController</value>
> > > > > > > > > > > </property>
> > > > > > > > > > > <property>
> > > > > > > > > > > <name>hbase.coprocessor.region.classes</name>
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> <value>org.apache.hadoop.hbase.security.token.TokenProvider,org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint,org.apache.hadoop.hbase.security.access.AccessController,org.apache.hadoop.hbase.security.visibility.VisibilityController</value>
> > > > > > > > > > > </property>
> > > > > > > > > > >
> > > > > > > > > > > I got:
> > > > > > > > > > >
> > > > > > > > > > > hbase(main):005:0> scan 'visibilityTest'
> > > > > > > > > > > ROW
> COLUMN+CELL
> > > > > > > > > > > r1
> > column=f1:c1,
> > > > > > > > > > > timestamp=1444522994981, value=value2
> > > > > > > > > > > 1 row(s) in 0.1020 seconds
> > > > > > > > > > >
> > > > > > > > > > > Can you try again with 0.98.15 release whose vote
> passed
> > > > Friday
> > > > > > to
> > > > > > > > see
> > > > > > > > > if
> > > > > > > > > > > what you observed can be reproduced ?
> > > > > > > > > > >
> > > > > > > > > > > Cheers
> > > > > > > > > > >
> > > > > > > > > > > On Sat, Oct 10, 2015 at 3:58 PM, Suresh Subbiah <
> > > > > > > > > > > suresh.subbiah60@gmail.com>
> > > > > > > > > > > wrote:
> > > > > > > > > > >
> > > > > > > > > > > > Hi,
> > > > > > > > > > > >
> > > > > > > > > > > > When I run the following script from hbase shell the
> > last
> > > > > scan
> > > > > > > > > returns
> > > > > > > > > > no
> > > > > > > > > > > > rows
> > > > > > > > > > > >
> > > > > > > > > > > > create 'visibilityTest', 'f1'
> > > > > > > > > > > > put 'visibilityTest', 'r1', 'f1:c1', 'value1'
> > > > > > > > > > > > deleteall 'visibilityTest', 'r1'
> > > > > > > > > > > > put 'visibilityTest', 'r1', 'f1:c1', 'value2'
> > > > > > > > > > > > scan 'visibilityTest'
> > > > > > > > > > > >
> > > > > > > > > > > > *hbase(main):013:0> scan 'visibilityTest'*
> > > > > > > > > > > > *ROW COLUMN+CELL
> > > > > > > > > > > > *
> > > > > > > > > > > > *0 row(s) in 0.0100 seconds*
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > However if I run
> > > > > > > > > > > > scan 'visibilityTest' , {RAW=>TRUE}
> > > > > > > > > > > >
> > > > > > > > > > > > I see that the second row that I put is indeed there
> > and
> > > > has
> > > > > a
> > > > > > > > > > timestamp
> > > > > > > > > > > > value higher that the previous delete
> > > > > > > > > > > >
> > > > > > > > > > > > *hbase(main):014:0> scan 'visibilityTest',
> {RAW=>TRUE}*
> > > > > > > > > > > > *ROW COLUMN+CELL
> > > > > > > > > > > > *
> > > > > > > > > > > > * r1 column=f1:,
> > > timestamp=1444516578296,
> > > > > > > > > > > > type=DeleteFamily *
> > > > > > > > > > > > * r1 column=f1:c1,
> > > > timestamp=1444516647655,
> > > > > > > > > > > value=value2
> > > > > > > > > > > > *
> > > > > > > > > > > > *1 row(s) in 0.0110 seconds*
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > This is on hbase 0.98.6. Problem is seen only when
> > > > > > > hbase-site.xml
> > > > > > > > > has
> > > > > > > > > > > > these lines. No other coprocessors were used during
> > this
> > > > > test.
> > > > > > > > > > > >
> > > > > > > > > > > > <property>
> > > > > > > > > > > > <name>hbase.coprocessor.region.classes</name>
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> <value>org.apache.hadoop.hbase.security.visibility.VisibilityController</value>
> > > > > > > > > > > > </property>
> > > > > > > > > > > > <property>
> > > > > > > > > > > > <name>hbase.coprocessor.master.classes</name>
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> <value>org.apache.hadoop.hbase.security.visibility.VisibilityController</value>
> > > > > > > > > > > > </property>
> > > > > > > > > > > > <property>
> > > > > > > > > > > > <name>hfile.format.version</name>
> > > > > > > > > > > > <value>3</value>
> > > > > > > > > > > > </property>
> > > > > > > > > > > >
> > > > > > > > > > > > Any suggestions of what I may be doing incorrectly?
> Or
> > is
> > > > > this
> > > > > > a
> > > > > > > > bug?
> > > > > > > > > > > >
> > > > > > > > > > > > Thank you
> > > > > > > > > > > > Suresh
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
Re: Unexpected behaviour when VisibilityController coprocessor is used
Posted by ramkrishna vasudevan <ra...@gmail.com>.
I tried it on the latest trunk and this issue is not there. So as Anoop
said the latest version of 0.98 should be solving this problem.
@Suresh
Let us know if you still find the issue in later versions of 0.98 and we
can work on it to solve the problem.
Regards
Ram
On Tue, Oct 13, 2015 at 9:09 AM, Anoop John <an...@gmail.com> wrote:
> Yes as such there is not mandatory to use AC along with VC. It can be used
> alone..
> I believe u r getting the bug HBASE-13734. This is fixed in 98.13 only.
> Just change ur version from 98.6 to 98.13 and test once. Let us know how
> is it then.
>
> -Anoop-
>
> On Tue, Oct 13, 2015 at 9:01 AM, ramkrishna vasudevan <
> ramkrishna.s.vasudevan@gmail.com> wrote:
>
> > I think, even with only configuring VisibilityController there should not
> > be a different behaviour, considering the fact that there are no
> visibility
> > labels. With just VisibilityController configured and doing puts and
> scans
> > using super user let me check what is happening.
> >
> > Regards
> > Ram
> >
> > On Tue, Oct 13, 2015 at 8:47 AM, Anoop John <an...@gmail.com>
> wrote:
> >
> > > Hi Suresh
> > > You said abt doing test as an HBase super user. You mean even when
> > scan
> > > is issues as a super user, u are not getting the rows back?
> > >
> > > -Anoop-
> > >
> > > On Tue, Oct 13, 2015 at 4:06 AM, Ted Yu <yu...@gmail.com> wrote:
> > >
> > > > Convention is to put AccessController ahead of VisibilityController
> in
> > > > hbase-site.xml
> > > >
> > > > Took a quick pass over region server log but haven't found much yet.
> > > >
> > > > FYI
> > > >
> > > > On Mon, Oct 12, 2015 at 3:28 PM, Suresh Subbiah <
> > > > suresh.subbiah60@gmail.com>
> > > > wrote:
> > > >
> > > > > Hi Ted,
> > > > >
> > > > > Thank you. Yes HDFS cluster has also been kerberized. BTW, this is
> a
> > > > > "cluster" with only one node.
> > > > >
> > > > > Master hbase-site.xml, RS hbase-site.ml and RS log for the time
> > > interval
> > > > > test was run is attached
> > > > >
> > > > > http://pastebin.com/zuqCC4xG
> > > > > http://pastebin.com/88Wx0KDf
> > > > > http://pastebin.com/QZqihN1W
> > > > >
> > > > > Will try deploying 1.1.2 next.
> > > > >
> > > > > Thanks
> > > > > Suresh
> > > > >
> > > > >
> > > > >
> > > > > On Mon, Oct 12, 2015 at 3:46 PM, Ted Yu <yu...@gmail.com>
> wrote:
> > > > >
> > > > > > bq. cluster enabled for secure HBase with kerberos
> > > > > >
> > > > > > I assume your hdfs cluster has also been kerberized.
> > > > > >
> > > > > > Please pastebin the complete hbase-site.xml
> > > > > >
> > > > > > Please turn on DEBUG logging and pastebin the region server log
> > which
> > > > > hosts
> > > > > > visibilityTest
> > > > > >
> > > > > > BTW if possible, can you deploy 1.1.2 ?
> > > > > >
> > > > > > Cheers
> > > > > >
> > > > > > On Mon, Oct 12, 2015 at 1:14 PM, Suresh Subbiah <
> > > > > > suresh.subbiah60@gmail.com>
> > > > > > wrote:
> > > > > >
> > > > > > > Hi Ted,
> > > > > > >
> > > > > > > I understand that using VisibilityController on an unsercure
> > > cluster
> > > > is
> > > > > > of
> > > > > > > limited value. I am still in the early stages of my task. I am
> > > logged
> > > > > in
> > > > > > as
> > > > > > > HBase super user and was simply checking if rows could be
> > accessed.
> > > > > > >
> > > > > > > With my colleague's help we did get the cluster enabled for
> > secure
> > > > > HBase
> > > > > > > with kerberos. I repeated the test to get the same result. Our
> > > > cluster
> > > > > is
> > > > > > > on 1.0. Do you think I may be doing something incorrectly? What
> > > > > > information
> > > > > > > can I send to help ensure that I have not made a mistake.
> > > > > > >
> > > > > > > Thanks
> > > > > > > Suresh
> > > > > > >
> > > > > > > hbase shell
> > > > > > > 15/10/12 14:35:09 INFO Configuration.deprecation:
> > hadoop.native.lib
> > > > is
> > > > > > > deprecated. Instead, use io.native.lib.available
> > > > > > > HBase Shell; enter 'help<RETURN>' for list of supported
> commands.
> > > > > > > Type "exit<RETURN>" to leave the HBase Shell
> > > > > > > Version 1.0.0-cdh5.4.4, rUnknown, Mon Jul 6 16:59:55 PDT 2015
> > > > > > >
> > > > > > > hbase(main):001:0> create 'visibilityTest', 'f1'
> > > > > > > 0 row(s) in 0.7780 seconds
> > > > > > >
> > > > > > > => Hbase::Table - visibilityTest
> > > > > > > hbase(main):002:0> put 'visibilityTest', 'r1', 'f1:c1',
> 'value1'
> > > > > > > 0 row(s) in 0.1300 seconds
> > > > > > >
> > > > > > > hbase(main):003:0> deleteall 'visibilityTest', 'r1'
> > > > > > > 0 row(s) in 0.0330 seconds
> > > > > > >
> > > > > > > hbase(main):004:0> put 'visibilityTest', 'r1', 'f1:c1',
> 'value2'
> > > > > > > 0 row(s) in 0.0150 seconds
> > > > > > >
> > > > > > > hbase(main):005:0> scan 'visibilityTest'
> > > > > > > ROW COLUMN+CELL
> > > > > > >
> > > > > > > 0 row(s) in 0.0550 seconds
> > > > > > >
> > > > > > > hbase(main):006:0> scan 'visibilityTest', {RAW=>TRUE}
> > > > > > > ROW COLUMN+CELL
> > > > > > >
> > > > > > > r1 column=f1:, timestamp=1444660561138,
> > > > > > > type=DeleteFamily
> > > > > > > r1 column=f1:c1, timestamp=1444660576868,
> > > > > value=value2
> > > > > > >
> > > > > > > 1 row(s) in 0.0370 seconds
> > > > > > >
> > > > > > > -----------------------------------------------------
> > > > > > > <property>
> > > > > > > <name>hbase.coprocessor.master.classes</name>
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> <value>org.apache.hadoop.hbase.security.visibility.VisibilityController,org.apache.hadoop.hbase.security.access.AccessController</value>
> > > > > > > </property>
> > > > > > >
> > > > > > > <property>
> > > > > > > <name>hbase.coprocessor.region.classes</name>
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> <value>org.apache.hadoop.hbase.security.visibility.VisibilityController,org.apache.hadoop.hbase.security.token.TokenProvider,org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint,org.apache.hadoop.hbase.security.access.AccessController</value>
> > > > > > > </property>
> > > > > > >
> > > > > > > --------------------------------------------------------
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > On Sat, Oct 10, 2015 at 9:51 PM, Ted Yu <yu...@gmail.com>
> > > wrote:
> > > > > > >
> > > > > > > > To my understanding, VisibilityController is used in a secure
> > > > > cluster.
> > > > > > > > Without security, how do you enforce that only select user(s)
> > can
> > > > > > access
> > > > > > > > certain cells ?
> > > > > > > >
> > > > > > > > Please see the following sections in refguide:
> > > > > > > >
> > > > > > > > http://hbase.apache.org/book.html#hbase.secure.configuration
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> http://hbase.apache.org/book.html#_server_side_configuration_for_simple_user_access_operation
> > > > > > > >
> > > > > > > > On Sat, Oct 10, 2015 at 7:40 PM, Suresh Subbiah <
> > > > > > > > suresh.subbiah60@gmail.com>
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > > > Hi Ted,
> > > > > > > > >
> > > > > > > > > Thank you for your response.
> > > > > > > > > I found a machine with HBase 1.0.0 and tried the script
> with
> > > all
> > > > 6
> > > > > > > > coprocs
> > > > > > > > > you listed (2 in master, and 4 in RS). I still do not see
> the
> > > row
> > > > > > after
> > > > > > > > the
> > > > > > > > > second scan.
> > > > > > > > >
> > > > > > > > > However my cluster is not secure enabled I think. Is that
> > > > > necessary?
> > > > > > I
> > > > > > > am
> > > > > > > > > not sure how to do that, though I can ask other members of
> my
> > > > team
> > > > > > and
> > > > > > > > try
> > > > > > > > > it if that will help.
> > > > > > > > >
> > > > > > > > > It will be ideal if we could get this to work on a 1.0
> based
> > > > > version.
> > > > > > > > > Moving to 1.1 will take more time since we have some
> > > > dependencies.
> > > > > > > > >
> > > > > > > > > Thank you
> > > > > > > > > Suresh
> > > > > > > > >
> > > > > > > > > 15/10/10 19:20:44 INFO Configuration.deprecation:
> > > > hadoop.native.lib
> > > > > > is
> > > > > > > > > deprecated. Instead, use io.native.lib.available
> > > > > > > > > HBase Shell; enter 'help<RETURN>' for list of supported
> > > commands.
> > > > > > > > > Type "exit<RETURN>" to leave the HBase Shell
> > > > > > > > > Version 1.0.0-cdh5.4.4, rUnknown, Mon Jul 6 16:59:55 PDT
> > 2015
> > > > > > > > >
> > > > > > > > > *hbase(main):001:0> create 'visibilityTest', 'f1' *
> > > > > > > > > *0 row(s) in 0.5460 seconds*
> > > > > > > > >
> > > > > > > > > *=> Hbase::Table - visibilityTest*
> > > > > > > > > *hbase(main):002:0> put 'visibilityTest', 'r1', 'f1:c1',
> > > > 'value1' *
> > > > > > > > > *0 row(s) in 0.0670 seconds*
> > > > > > > > >
> > > > > > > > > *hbase(main):003:0> deleteall 'visibilityTest', 'r1' *
> > > > > > > > > *0 row(s) in 0.0090 seconds*
> > > > > > > > >
> > > > > > > > > *hbase(main):004:0> put 'visibilityTest', 'r1', 'f1:c1',
> > > > 'value2'*
> > > > > > > > > *0 row(s) in 0.0040 seconds*
> > > > > > > > >
> > > > > > > > > *hbase(main):005:0> scan 'visibilityTest'*
> > > > > > > > > *ROW COLUMN+CELL
> > > > > > > > > *
> > > > > > > > > *0 row(s) in 0.0160 seconds*
> > > > > > > > >
> > > > > > > > > *hbase(main):006:0> scan 'visibilityTest', {RAW=>TRUE}*
> > > > > > > > > *ROW COLUMN+CELL
> > > > > > > > > *
> > > > > > > > > * r1 column=f1:, timestamp=1444530064056,
> > > > > > > > > type=DeleteFamily *
> > > > > > > > > * r1 column=f1:c1,
> timestamp=1444530064084,
> > > > > > > > value=value2
> > > > > > > > > *
> > > > > > > > > *1 row(s) in 0.0580 seconds*
> > > > > > > > >
> > > > > > > > > *hbase(main):007:0> exit*
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > On Sat, Oct 10, 2015 at 7:26 PM, Ted Yu <
> yuzhihong@gmail.com
> > >
> > > > > wrote:
> > > > > > > > >
> > > > > > > > > > I tried the sequence of commands from your example on a
> > > secure
> > > > > > 1.1.2
> > > > > > > > > > cluster with the following config:
> > > > > > > > > >
> > > > > > > > > > <property>
> > > > > > > > > > <name>hbase.coprocessor.master.classes</name>
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> <value>org.apache.hadoop.hbase.security.access.AccessController,org.apache.hadoop.hbase.security.visibility.VisibilityController</value>
> > > > > > > > > > </property>
> > > > > > > > > > <property>
> > > > > > > > > > <name>hbase.coprocessor.region.classes</name>
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> <value>org.apache.hadoop.hbase.security.token.TokenProvider,org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint,org.apache.hadoop.hbase.security.access.AccessController,org.apache.hadoop.hbase.security.visibility.VisibilityController</value>
> > > > > > > > > > </property>
> > > > > > > > > >
> > > > > > > > > > I got:
> > > > > > > > > >
> > > > > > > > > > hbase(main):005:0> scan 'visibilityTest'
> > > > > > > > > > ROW COLUMN+CELL
> > > > > > > > > > r1
> column=f1:c1,
> > > > > > > > > > timestamp=1444522994981, value=value2
> > > > > > > > > > 1 row(s) in 0.1020 seconds
> > > > > > > > > >
> > > > > > > > > > Can you try again with 0.98.15 release whose vote passed
> > > Friday
> > > > > to
> > > > > > > see
> > > > > > > > if
> > > > > > > > > > what you observed can be reproduced ?
> > > > > > > > > >
> > > > > > > > > > Cheers
> > > > > > > > > >
> > > > > > > > > > On Sat, Oct 10, 2015 at 3:58 PM, Suresh Subbiah <
> > > > > > > > > > suresh.subbiah60@gmail.com>
> > > > > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > > Hi,
> > > > > > > > > > >
> > > > > > > > > > > When I run the following script from hbase shell the
> last
> > > > scan
> > > > > > > > returns
> > > > > > > > > no
> > > > > > > > > > > rows
> > > > > > > > > > >
> > > > > > > > > > > create 'visibilityTest', 'f1'
> > > > > > > > > > > put 'visibilityTest', 'r1', 'f1:c1', 'value1'
> > > > > > > > > > > deleteall 'visibilityTest', 'r1'
> > > > > > > > > > > put 'visibilityTest', 'r1', 'f1:c1', 'value2'
> > > > > > > > > > > scan 'visibilityTest'
> > > > > > > > > > >
> > > > > > > > > > > *hbase(main):013:0> scan 'visibilityTest'*
> > > > > > > > > > > *ROW COLUMN+CELL
> > > > > > > > > > > *
> > > > > > > > > > > *0 row(s) in 0.0100 seconds*
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > However if I run
> > > > > > > > > > > scan 'visibilityTest' , {RAW=>TRUE}
> > > > > > > > > > >
> > > > > > > > > > > I see that the second row that I put is indeed there
> and
> > > has
> > > > a
> > > > > > > > > timestamp
> > > > > > > > > > > value higher that the previous delete
> > > > > > > > > > >
> > > > > > > > > > > *hbase(main):014:0> scan 'visibilityTest', {RAW=>TRUE}*
> > > > > > > > > > > *ROW COLUMN+CELL
> > > > > > > > > > > *
> > > > > > > > > > > * r1 column=f1:,
> > timestamp=1444516578296,
> > > > > > > > > > > type=DeleteFamily *
> > > > > > > > > > > * r1 column=f1:c1,
> > > timestamp=1444516647655,
> > > > > > > > > > value=value2
> > > > > > > > > > > *
> > > > > > > > > > > *1 row(s) in 0.0110 seconds*
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > This is on hbase 0.98.6. Problem is seen only when
> > > > > > hbase-site.xml
> > > > > > > > has
> > > > > > > > > > > these lines. No other coprocessors were used during
> this
> > > > test.
> > > > > > > > > > >
> > > > > > > > > > > <property>
> > > > > > > > > > > <name>hbase.coprocessor.region.classes</name>
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> <value>org.apache.hadoop.hbase.security.visibility.VisibilityController</value>
> > > > > > > > > > > </property>
> > > > > > > > > > > <property>
> > > > > > > > > > > <name>hbase.coprocessor.master.classes</name>
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> <value>org.apache.hadoop.hbase.security.visibility.VisibilityController</value>
> > > > > > > > > > > </property>
> > > > > > > > > > > <property>
> > > > > > > > > > > <name>hfile.format.version</name>
> > > > > > > > > > > <value>3</value>
> > > > > > > > > > > </property>
> > > > > > > > > > >
> > > > > > > > > > > Any suggestions of what I may be doing incorrectly? Or
> is
> > > > this
> > > > > a
> > > > > > > bug?
> > > > > > > > > > >
> > > > > > > > > > > Thank you
> > > > > > > > > > > Suresh
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
Re: Unexpected behaviour when VisibilityController coprocessor is used
Posted by Anoop John <an...@gmail.com>.
Yes as such there is not mandatory to use AC along with VC. It can be used
alone..
I believe u r getting the bug HBASE-13734. This is fixed in 98.13 only.
Just change ur version from 98.6 to 98.13 and test once. Let us know how
is it then.
-Anoop-
On Tue, Oct 13, 2015 at 9:01 AM, ramkrishna vasudevan <
ramkrishna.s.vasudevan@gmail.com> wrote:
> I think, even with only configuring VisibilityController there should not
> be a different behaviour, considering the fact that there are no visibility
> labels. With just VisibilityController configured and doing puts and scans
> using super user let me check what is happening.
>
> Regards
> Ram
>
> On Tue, Oct 13, 2015 at 8:47 AM, Anoop John <an...@gmail.com> wrote:
>
> > Hi Suresh
> > You said abt doing test as an HBase super user. You mean even when
> scan
> > is issues as a super user, u are not getting the rows back?
> >
> > -Anoop-
> >
> > On Tue, Oct 13, 2015 at 4:06 AM, Ted Yu <yu...@gmail.com> wrote:
> >
> > > Convention is to put AccessController ahead of VisibilityController in
> > > hbase-site.xml
> > >
> > > Took a quick pass over region server log but haven't found much yet.
> > >
> > > FYI
> > >
> > > On Mon, Oct 12, 2015 at 3:28 PM, Suresh Subbiah <
> > > suresh.subbiah60@gmail.com>
> > > wrote:
> > >
> > > > Hi Ted,
> > > >
> > > > Thank you. Yes HDFS cluster has also been kerberized. BTW, this is a
> > > > "cluster" with only one node.
> > > >
> > > > Master hbase-site.xml, RS hbase-site.ml and RS log for the time
> > interval
> > > > test was run is attached
> > > >
> > > > http://pastebin.com/zuqCC4xG
> > > > http://pastebin.com/88Wx0KDf
> > > > http://pastebin.com/QZqihN1W
> > > >
> > > > Will try deploying 1.1.2 next.
> > > >
> > > > Thanks
> > > > Suresh
> > > >
> > > >
> > > >
> > > > On Mon, Oct 12, 2015 at 3:46 PM, Ted Yu <yu...@gmail.com> wrote:
> > > >
> > > > > bq. cluster enabled for secure HBase with kerberos
> > > > >
> > > > > I assume your hdfs cluster has also been kerberized.
> > > > >
> > > > > Please pastebin the complete hbase-site.xml
> > > > >
> > > > > Please turn on DEBUG logging and pastebin the region server log
> which
> > > > hosts
> > > > > visibilityTest
> > > > >
> > > > > BTW if possible, can you deploy 1.1.2 ?
> > > > >
> > > > > Cheers
> > > > >
> > > > > On Mon, Oct 12, 2015 at 1:14 PM, Suresh Subbiah <
> > > > > suresh.subbiah60@gmail.com>
> > > > > wrote:
> > > > >
> > > > > > Hi Ted,
> > > > > >
> > > > > > I understand that using VisibilityController on an unsercure
> > cluster
> > > is
> > > > > of
> > > > > > limited value. I am still in the early stages of my task. I am
> > logged
> > > > in
> > > > > as
> > > > > > HBase super user and was simply checking if rows could be
> accessed.
> > > > > >
> > > > > > With my colleague's help we did get the cluster enabled for
> secure
> > > > HBase
> > > > > > with kerberos. I repeated the test to get the same result. Our
> > > cluster
> > > > is
> > > > > > on 1.0. Do you think I may be doing something incorrectly? What
> > > > > information
> > > > > > can I send to help ensure that I have not made a mistake.
> > > > > >
> > > > > > Thanks
> > > > > > Suresh
> > > > > >
> > > > > > hbase shell
> > > > > > 15/10/12 14:35:09 INFO Configuration.deprecation:
> hadoop.native.lib
> > > is
> > > > > > deprecated. Instead, use io.native.lib.available
> > > > > > HBase Shell; enter 'help<RETURN>' for list of supported commands.
> > > > > > Type "exit<RETURN>" to leave the HBase Shell
> > > > > > Version 1.0.0-cdh5.4.4, rUnknown, Mon Jul 6 16:59:55 PDT 2015
> > > > > >
> > > > > > hbase(main):001:0> create 'visibilityTest', 'f1'
> > > > > > 0 row(s) in 0.7780 seconds
> > > > > >
> > > > > > => Hbase::Table - visibilityTest
> > > > > > hbase(main):002:0> put 'visibilityTest', 'r1', 'f1:c1', 'value1'
> > > > > > 0 row(s) in 0.1300 seconds
> > > > > >
> > > > > > hbase(main):003:0> deleteall 'visibilityTest', 'r1'
> > > > > > 0 row(s) in 0.0330 seconds
> > > > > >
> > > > > > hbase(main):004:0> put 'visibilityTest', 'r1', 'f1:c1', 'value2'
> > > > > > 0 row(s) in 0.0150 seconds
> > > > > >
> > > > > > hbase(main):005:0> scan 'visibilityTest'
> > > > > > ROW COLUMN+CELL
> > > > > >
> > > > > > 0 row(s) in 0.0550 seconds
> > > > > >
> > > > > > hbase(main):006:0> scan 'visibilityTest', {RAW=>TRUE}
> > > > > > ROW COLUMN+CELL
> > > > > >
> > > > > > r1 column=f1:, timestamp=1444660561138,
> > > > > > type=DeleteFamily
> > > > > > r1 column=f1:c1, timestamp=1444660576868,
> > > > value=value2
> > > > > >
> > > > > > 1 row(s) in 0.0370 seconds
> > > > > >
> > > > > > -----------------------------------------------------
> > > > > > <property>
> > > > > > <name>hbase.coprocessor.master.classes</name>
> > > > > >
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> <value>org.apache.hadoop.hbase.security.visibility.VisibilityController,org.apache.hadoop.hbase.security.access.AccessController</value>
> > > > > > </property>
> > > > > >
> > > > > > <property>
> > > > > > <name>hbase.coprocessor.region.classes</name>
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> <value>org.apache.hadoop.hbase.security.visibility.VisibilityController,org.apache.hadoop.hbase.security.token.TokenProvider,org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint,org.apache.hadoop.hbase.security.access.AccessController</value>
> > > > > > </property>
> > > > > >
> > > > > > --------------------------------------------------------
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > On Sat, Oct 10, 2015 at 9:51 PM, Ted Yu <yu...@gmail.com>
> > wrote:
> > > > > >
> > > > > > > To my understanding, VisibilityController is used in a secure
> > > > cluster.
> > > > > > > Without security, how do you enforce that only select user(s)
> can
> > > > > access
> > > > > > > certain cells ?
> > > > > > >
> > > > > > > Please see the following sections in refguide:
> > > > > > >
> > > > > > > http://hbase.apache.org/book.html#hbase.secure.configuration
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> http://hbase.apache.org/book.html#_server_side_configuration_for_simple_user_access_operation
> > > > > > >
> > > > > > > On Sat, Oct 10, 2015 at 7:40 PM, Suresh Subbiah <
> > > > > > > suresh.subbiah60@gmail.com>
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Hi Ted,
> > > > > > > >
> > > > > > > > Thank you for your response.
> > > > > > > > I found a machine with HBase 1.0.0 and tried the script with
> > all
> > > 6
> > > > > > > coprocs
> > > > > > > > you listed (2 in master, and 4 in RS). I still do not see the
> > row
> > > > > after
> > > > > > > the
> > > > > > > > second scan.
> > > > > > > >
> > > > > > > > However my cluster is not secure enabled I think. Is that
> > > > necessary?
> > > > > I
> > > > > > am
> > > > > > > > not sure how to do that, though I can ask other members of my
> > > team
> > > > > and
> > > > > > > try
> > > > > > > > it if that will help.
> > > > > > > >
> > > > > > > > It will be ideal if we could get this to work on a 1.0 based
> > > > version.
> > > > > > > > Moving to 1.1 will take more time since we have some
> > > dependencies.
> > > > > > > >
> > > > > > > > Thank you
> > > > > > > > Suresh
> > > > > > > >
> > > > > > > > 15/10/10 19:20:44 INFO Configuration.deprecation:
> > > hadoop.native.lib
> > > > > is
> > > > > > > > deprecated. Instead, use io.native.lib.available
> > > > > > > > HBase Shell; enter 'help<RETURN>' for list of supported
> > commands.
> > > > > > > > Type "exit<RETURN>" to leave the HBase Shell
> > > > > > > > Version 1.0.0-cdh5.4.4, rUnknown, Mon Jul 6 16:59:55 PDT
> 2015
> > > > > > > >
> > > > > > > > *hbase(main):001:0> create 'visibilityTest', 'f1' *
> > > > > > > > *0 row(s) in 0.5460 seconds*
> > > > > > > >
> > > > > > > > *=> Hbase::Table - visibilityTest*
> > > > > > > > *hbase(main):002:0> put 'visibilityTest', 'r1', 'f1:c1',
> > > 'value1' *
> > > > > > > > *0 row(s) in 0.0670 seconds*
> > > > > > > >
> > > > > > > > *hbase(main):003:0> deleteall 'visibilityTest', 'r1' *
> > > > > > > > *0 row(s) in 0.0090 seconds*
> > > > > > > >
> > > > > > > > *hbase(main):004:0> put 'visibilityTest', 'r1', 'f1:c1',
> > > 'value2'*
> > > > > > > > *0 row(s) in 0.0040 seconds*
> > > > > > > >
> > > > > > > > *hbase(main):005:0> scan 'visibilityTest'*
> > > > > > > > *ROW COLUMN+CELL
> > > > > > > > *
> > > > > > > > *0 row(s) in 0.0160 seconds*
> > > > > > > >
> > > > > > > > *hbase(main):006:0> scan 'visibilityTest', {RAW=>TRUE}*
> > > > > > > > *ROW COLUMN+CELL
> > > > > > > > *
> > > > > > > > * r1 column=f1:, timestamp=1444530064056,
> > > > > > > > type=DeleteFamily *
> > > > > > > > * r1 column=f1:c1, timestamp=1444530064084,
> > > > > > > value=value2
> > > > > > > > *
> > > > > > > > *1 row(s) in 0.0580 seconds*
> > > > > > > >
> > > > > > > > *hbase(main):007:0> exit*
> > > > > > > >
> > > > > > > >
> > > > > > > > On Sat, Oct 10, 2015 at 7:26 PM, Ted Yu <yuzhihong@gmail.com
> >
> > > > wrote:
> > > > > > > >
> > > > > > > > > I tried the sequence of commands from your example on a
> > secure
> > > > > 1.1.2
> > > > > > > > > cluster with the following config:
> > > > > > > > >
> > > > > > > > > <property>
> > > > > > > > > <name>hbase.coprocessor.master.classes</name>
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> <value>org.apache.hadoop.hbase.security.access.AccessController,org.apache.hadoop.hbase.security.visibility.VisibilityController</value>
> > > > > > > > > </property>
> > > > > > > > > <property>
> > > > > > > > > <name>hbase.coprocessor.region.classes</name>
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> <value>org.apache.hadoop.hbase.security.token.TokenProvider,org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint,org.apache.hadoop.hbase.security.access.AccessController,org.apache.hadoop.hbase.security.visibility.VisibilityController</value>
> > > > > > > > > </property>
> > > > > > > > >
> > > > > > > > > I got:
> > > > > > > > >
> > > > > > > > > hbase(main):005:0> scan 'visibilityTest'
> > > > > > > > > ROW COLUMN+CELL
> > > > > > > > > r1 column=f1:c1,
> > > > > > > > > timestamp=1444522994981, value=value2
> > > > > > > > > 1 row(s) in 0.1020 seconds
> > > > > > > > >
> > > > > > > > > Can you try again with 0.98.15 release whose vote passed
> > Friday
> > > > to
> > > > > > see
> > > > > > > if
> > > > > > > > > what you observed can be reproduced ?
> > > > > > > > >
> > > > > > > > > Cheers
> > > > > > > > >
> > > > > > > > > On Sat, Oct 10, 2015 at 3:58 PM, Suresh Subbiah <
> > > > > > > > > suresh.subbiah60@gmail.com>
> > > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > > Hi,
> > > > > > > > > >
> > > > > > > > > > When I run the following script from hbase shell the last
> > > scan
> > > > > > > returns
> > > > > > > > no
> > > > > > > > > > rows
> > > > > > > > > >
> > > > > > > > > > create 'visibilityTest', 'f1'
> > > > > > > > > > put 'visibilityTest', 'r1', 'f1:c1', 'value1'
> > > > > > > > > > deleteall 'visibilityTest', 'r1'
> > > > > > > > > > put 'visibilityTest', 'r1', 'f1:c1', 'value2'
> > > > > > > > > > scan 'visibilityTest'
> > > > > > > > > >
> > > > > > > > > > *hbase(main):013:0> scan 'visibilityTest'*
> > > > > > > > > > *ROW COLUMN+CELL
> > > > > > > > > > *
> > > > > > > > > > *0 row(s) in 0.0100 seconds*
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > However if I run
> > > > > > > > > > scan 'visibilityTest' , {RAW=>TRUE}
> > > > > > > > > >
> > > > > > > > > > I see that the second row that I put is indeed there and
> > has
> > > a
> > > > > > > > timestamp
> > > > > > > > > > value higher that the previous delete
> > > > > > > > > >
> > > > > > > > > > *hbase(main):014:0> scan 'visibilityTest', {RAW=>TRUE}*
> > > > > > > > > > *ROW COLUMN+CELL
> > > > > > > > > > *
> > > > > > > > > > * r1 column=f1:,
> timestamp=1444516578296,
> > > > > > > > > > type=DeleteFamily *
> > > > > > > > > > * r1 column=f1:c1,
> > timestamp=1444516647655,
> > > > > > > > > value=value2
> > > > > > > > > > *
> > > > > > > > > > *1 row(s) in 0.0110 seconds*
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > This is on hbase 0.98.6. Problem is seen only when
> > > > > hbase-site.xml
> > > > > > > has
> > > > > > > > > > these lines. No other coprocessors were used during this
> > > test.
> > > > > > > > > >
> > > > > > > > > > <property>
> > > > > > > > > > <name>hbase.coprocessor.region.classes</name>
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> <value>org.apache.hadoop.hbase.security.visibility.VisibilityController</value>
> > > > > > > > > > </property>
> > > > > > > > > > <property>
> > > > > > > > > > <name>hbase.coprocessor.master.classes</name>
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> <value>org.apache.hadoop.hbase.security.visibility.VisibilityController</value>
> > > > > > > > > > </property>
> > > > > > > > > > <property>
> > > > > > > > > > <name>hfile.format.version</name>
> > > > > > > > > > <value>3</value>
> > > > > > > > > > </property>
> > > > > > > > > >
> > > > > > > > > > Any suggestions of what I may be doing incorrectly? Or is
> > > this
> > > > a
> > > > > > bug?
> > > > > > > > > >
> > > > > > > > > > Thank you
> > > > > > > > > > Suresh
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
Re: Unexpected behaviour when VisibilityController coprocessor is used
Posted by ramkrishna vasudevan <ra...@gmail.com>.
I think, even with only configuring VisibilityController there should not
be a different behaviour, considering the fact that there are no visibility
labels. With just VisibilityController configured and doing puts and scans
using super user let me check what is happening.
Regards
Ram
On Tue, Oct 13, 2015 at 8:47 AM, Anoop John <an...@gmail.com> wrote:
> Hi Suresh
> You said abt doing test as an HBase super user. You mean even when scan
> is issues as a super user, u are not getting the rows back?
>
> -Anoop-
>
> On Tue, Oct 13, 2015 at 4:06 AM, Ted Yu <yu...@gmail.com> wrote:
>
> > Convention is to put AccessController ahead of VisibilityController in
> > hbase-site.xml
> >
> > Took a quick pass over region server log but haven't found much yet.
> >
> > FYI
> >
> > On Mon, Oct 12, 2015 at 3:28 PM, Suresh Subbiah <
> > suresh.subbiah60@gmail.com>
> > wrote:
> >
> > > Hi Ted,
> > >
> > > Thank you. Yes HDFS cluster has also been kerberized. BTW, this is a
> > > "cluster" with only one node.
> > >
> > > Master hbase-site.xml, RS hbase-site.ml and RS log for the time
> interval
> > > test was run is attached
> > >
> > > http://pastebin.com/zuqCC4xG
> > > http://pastebin.com/88Wx0KDf
> > > http://pastebin.com/QZqihN1W
> > >
> > > Will try deploying 1.1.2 next.
> > >
> > > Thanks
> > > Suresh
> > >
> > >
> > >
> > > On Mon, Oct 12, 2015 at 3:46 PM, Ted Yu <yu...@gmail.com> wrote:
> > >
> > > > bq. cluster enabled for secure HBase with kerberos
> > > >
> > > > I assume your hdfs cluster has also been kerberized.
> > > >
> > > > Please pastebin the complete hbase-site.xml
> > > >
> > > > Please turn on DEBUG logging and pastebin the region server log which
> > > hosts
> > > > visibilityTest
> > > >
> > > > BTW if possible, can you deploy 1.1.2 ?
> > > >
> > > > Cheers
> > > >
> > > > On Mon, Oct 12, 2015 at 1:14 PM, Suresh Subbiah <
> > > > suresh.subbiah60@gmail.com>
> > > > wrote:
> > > >
> > > > > Hi Ted,
> > > > >
> > > > > I understand that using VisibilityController on an unsercure
> cluster
> > is
> > > > of
> > > > > limited value. I am still in the early stages of my task. I am
> logged
> > > in
> > > > as
> > > > > HBase super user and was simply checking if rows could be accessed.
> > > > >
> > > > > With my colleague's help we did get the cluster enabled for secure
> > > HBase
> > > > > with kerberos. I repeated the test to get the same result. Our
> > cluster
> > > is
> > > > > on 1.0. Do you think I may be doing something incorrectly? What
> > > > information
> > > > > can I send to help ensure that I have not made a mistake.
> > > > >
> > > > > Thanks
> > > > > Suresh
> > > > >
> > > > > hbase shell
> > > > > 15/10/12 14:35:09 INFO Configuration.deprecation: hadoop.native.lib
> > is
> > > > > deprecated. Instead, use io.native.lib.available
> > > > > HBase Shell; enter 'help<RETURN>' for list of supported commands.
> > > > > Type "exit<RETURN>" to leave the HBase Shell
> > > > > Version 1.0.0-cdh5.4.4, rUnknown, Mon Jul 6 16:59:55 PDT 2015
> > > > >
> > > > > hbase(main):001:0> create 'visibilityTest', 'f1'
> > > > > 0 row(s) in 0.7780 seconds
> > > > >
> > > > > => Hbase::Table - visibilityTest
> > > > > hbase(main):002:0> put 'visibilityTest', 'r1', 'f1:c1', 'value1'
> > > > > 0 row(s) in 0.1300 seconds
> > > > >
> > > > > hbase(main):003:0> deleteall 'visibilityTest', 'r1'
> > > > > 0 row(s) in 0.0330 seconds
> > > > >
> > > > > hbase(main):004:0> put 'visibilityTest', 'r1', 'f1:c1', 'value2'
> > > > > 0 row(s) in 0.0150 seconds
> > > > >
> > > > > hbase(main):005:0> scan 'visibilityTest'
> > > > > ROW COLUMN+CELL
> > > > >
> > > > > 0 row(s) in 0.0550 seconds
> > > > >
> > > > > hbase(main):006:0> scan 'visibilityTest', {RAW=>TRUE}
> > > > > ROW COLUMN+CELL
> > > > >
> > > > > r1 column=f1:, timestamp=1444660561138,
> > > > > type=DeleteFamily
> > > > > r1 column=f1:c1, timestamp=1444660576868,
> > > value=value2
> > > > >
> > > > > 1 row(s) in 0.0370 seconds
> > > > >
> > > > > -----------------------------------------------------
> > > > > <property>
> > > > > <name>hbase.coprocessor.master.classes</name>
> > > > >
> > > > >
> > > > >
> > > >
> > >
> >
> <value>org.apache.hadoop.hbase.security.visibility.VisibilityController,org.apache.hadoop.hbase.security.access.AccessController</value>
> > > > > </property>
> > > > >
> > > > > <property>
> > > > > <name>hbase.coprocessor.region.classes</name>
> > > > >
> > > > >
> > > >
> > >
> >
> <value>org.apache.hadoop.hbase.security.visibility.VisibilityController,org.apache.hadoop.hbase.security.token.TokenProvider,org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint,org.apache.hadoop.hbase.security.access.AccessController</value>
> > > > > </property>
> > > > >
> > > > > --------------------------------------------------------
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > On Sat, Oct 10, 2015 at 9:51 PM, Ted Yu <yu...@gmail.com>
> wrote:
> > > > >
> > > > > > To my understanding, VisibilityController is used in a secure
> > > cluster.
> > > > > > Without security, how do you enforce that only select user(s) can
> > > > access
> > > > > > certain cells ?
> > > > > >
> > > > > > Please see the following sections in refguide:
> > > > > >
> > > > > > http://hbase.apache.org/book.html#hbase.secure.configuration
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> http://hbase.apache.org/book.html#_server_side_configuration_for_simple_user_access_operation
> > > > > >
> > > > > > On Sat, Oct 10, 2015 at 7:40 PM, Suresh Subbiah <
> > > > > > suresh.subbiah60@gmail.com>
> > > > > > wrote:
> > > > > >
> > > > > > > Hi Ted,
> > > > > > >
> > > > > > > Thank you for your response.
> > > > > > > I found a machine with HBase 1.0.0 and tried the script with
> all
> > 6
> > > > > > coprocs
> > > > > > > you listed (2 in master, and 4 in RS). I still do not see the
> row
> > > > after
> > > > > > the
> > > > > > > second scan.
> > > > > > >
> > > > > > > However my cluster is not secure enabled I think. Is that
> > > necessary?
> > > > I
> > > > > am
> > > > > > > not sure how to do that, though I can ask other members of my
> > team
> > > > and
> > > > > > try
> > > > > > > it if that will help.
> > > > > > >
> > > > > > > It will be ideal if we could get this to work on a 1.0 based
> > > version.
> > > > > > > Moving to 1.1 will take more time since we have some
> > dependencies.
> > > > > > >
> > > > > > > Thank you
> > > > > > > Suresh
> > > > > > >
> > > > > > > 15/10/10 19:20:44 INFO Configuration.deprecation:
> > hadoop.native.lib
> > > > is
> > > > > > > deprecated. Instead, use io.native.lib.available
> > > > > > > HBase Shell; enter 'help<RETURN>' for list of supported
> commands.
> > > > > > > Type "exit<RETURN>" to leave the HBase Shell
> > > > > > > Version 1.0.0-cdh5.4.4, rUnknown, Mon Jul 6 16:59:55 PDT 2015
> > > > > > >
> > > > > > > *hbase(main):001:0> create 'visibilityTest', 'f1' *
> > > > > > > *0 row(s) in 0.5460 seconds*
> > > > > > >
> > > > > > > *=> Hbase::Table - visibilityTest*
> > > > > > > *hbase(main):002:0> put 'visibilityTest', 'r1', 'f1:c1',
> > 'value1' *
> > > > > > > *0 row(s) in 0.0670 seconds*
> > > > > > >
> > > > > > > *hbase(main):003:0> deleteall 'visibilityTest', 'r1' *
> > > > > > > *0 row(s) in 0.0090 seconds*
> > > > > > >
> > > > > > > *hbase(main):004:0> put 'visibilityTest', 'r1', 'f1:c1',
> > 'value2'*
> > > > > > > *0 row(s) in 0.0040 seconds*
> > > > > > >
> > > > > > > *hbase(main):005:0> scan 'visibilityTest'*
> > > > > > > *ROW COLUMN+CELL
> > > > > > > *
> > > > > > > *0 row(s) in 0.0160 seconds*
> > > > > > >
> > > > > > > *hbase(main):006:0> scan 'visibilityTest', {RAW=>TRUE}*
> > > > > > > *ROW COLUMN+CELL
> > > > > > > *
> > > > > > > * r1 column=f1:, timestamp=1444530064056,
> > > > > > > type=DeleteFamily *
> > > > > > > * r1 column=f1:c1, timestamp=1444530064084,
> > > > > > value=value2
> > > > > > > *
> > > > > > > *1 row(s) in 0.0580 seconds*
> > > > > > >
> > > > > > > *hbase(main):007:0> exit*
> > > > > > >
> > > > > > >
> > > > > > > On Sat, Oct 10, 2015 at 7:26 PM, Ted Yu <yu...@gmail.com>
> > > wrote:
> > > > > > >
> > > > > > > > I tried the sequence of commands from your example on a
> secure
> > > > 1.1.2
> > > > > > > > cluster with the following config:
> > > > > > > >
> > > > > > > > <property>
> > > > > > > > <name>hbase.coprocessor.master.classes</name>
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> <value>org.apache.hadoop.hbase.security.access.AccessController,org.apache.hadoop.hbase.security.visibility.VisibilityController</value>
> > > > > > > > </property>
> > > > > > > > <property>
> > > > > > > > <name>hbase.coprocessor.region.classes</name>
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> <value>org.apache.hadoop.hbase.security.token.TokenProvider,org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint,org.apache.hadoop.hbase.security.access.AccessController,org.apache.hadoop.hbase.security.visibility.VisibilityController</value>
> > > > > > > > </property>
> > > > > > > >
> > > > > > > > I got:
> > > > > > > >
> > > > > > > > hbase(main):005:0> scan 'visibilityTest'
> > > > > > > > ROW COLUMN+CELL
> > > > > > > > r1 column=f1:c1,
> > > > > > > > timestamp=1444522994981, value=value2
> > > > > > > > 1 row(s) in 0.1020 seconds
> > > > > > > >
> > > > > > > > Can you try again with 0.98.15 release whose vote passed
> Friday
> > > to
> > > > > see
> > > > > > if
> > > > > > > > what you observed can be reproduced ?
> > > > > > > >
> > > > > > > > Cheers
> > > > > > > >
> > > > > > > > On Sat, Oct 10, 2015 at 3:58 PM, Suresh Subbiah <
> > > > > > > > suresh.subbiah60@gmail.com>
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > > > Hi,
> > > > > > > > >
> > > > > > > > > When I run the following script from hbase shell the last
> > scan
> > > > > > returns
> > > > > > > no
> > > > > > > > > rows
> > > > > > > > >
> > > > > > > > > create 'visibilityTest', 'f1'
> > > > > > > > > put 'visibilityTest', 'r1', 'f1:c1', 'value1'
> > > > > > > > > deleteall 'visibilityTest', 'r1'
> > > > > > > > > put 'visibilityTest', 'r1', 'f1:c1', 'value2'
> > > > > > > > > scan 'visibilityTest'
> > > > > > > > >
> > > > > > > > > *hbase(main):013:0> scan 'visibilityTest'*
> > > > > > > > > *ROW COLUMN+CELL
> > > > > > > > > *
> > > > > > > > > *0 row(s) in 0.0100 seconds*
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > However if I run
> > > > > > > > > scan 'visibilityTest' , {RAW=>TRUE}
> > > > > > > > >
> > > > > > > > > I see that the second row that I put is indeed there and
> has
> > a
> > > > > > > timestamp
> > > > > > > > > value higher that the previous delete
> > > > > > > > >
> > > > > > > > > *hbase(main):014:0> scan 'visibilityTest', {RAW=>TRUE}*
> > > > > > > > > *ROW COLUMN+CELL
> > > > > > > > > *
> > > > > > > > > * r1 column=f1:, timestamp=1444516578296,
> > > > > > > > > type=DeleteFamily *
> > > > > > > > > * r1 column=f1:c1,
> timestamp=1444516647655,
> > > > > > > > value=value2
> > > > > > > > > *
> > > > > > > > > *1 row(s) in 0.0110 seconds*
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > This is on hbase 0.98.6. Problem is seen only when
> > > > hbase-site.xml
> > > > > > has
> > > > > > > > > these lines. No other coprocessors were used during this
> > test.
> > > > > > > > >
> > > > > > > > > <property>
> > > > > > > > > <name>hbase.coprocessor.region.classes</name>
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> <value>org.apache.hadoop.hbase.security.visibility.VisibilityController</value>
> > > > > > > > > </property>
> > > > > > > > > <property>
> > > > > > > > > <name>hbase.coprocessor.master.classes</name>
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> <value>org.apache.hadoop.hbase.security.visibility.VisibilityController</value>
> > > > > > > > > </property>
> > > > > > > > > <property>
> > > > > > > > > <name>hfile.format.version</name>
> > > > > > > > > <value>3</value>
> > > > > > > > > </property>
> > > > > > > > >
> > > > > > > > > Any suggestions of what I may be doing incorrectly? Or is
> > this
> > > a
> > > > > bug?
> > > > > > > > >
> > > > > > > > > Thank you
> > > > > > > > > Suresh
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
Re: Unexpected behaviour when VisibilityController coprocessor is used
Posted by Anoop John <an...@gmail.com>.
Hi Suresh
You said abt doing test as an HBase super user. You mean even when scan
is issues as a super user, u are not getting the rows back?
-Anoop-
On Tue, Oct 13, 2015 at 4:06 AM, Ted Yu <yu...@gmail.com> wrote:
> Convention is to put AccessController ahead of VisibilityController in
> hbase-site.xml
>
> Took a quick pass over region server log but haven't found much yet.
>
> FYI
>
> On Mon, Oct 12, 2015 at 3:28 PM, Suresh Subbiah <
> suresh.subbiah60@gmail.com>
> wrote:
>
> > Hi Ted,
> >
> > Thank you. Yes HDFS cluster has also been kerberized. BTW, this is a
> > "cluster" with only one node.
> >
> > Master hbase-site.xml, RS hbase-site.ml and RS log for the time interval
> > test was run is attached
> >
> > http://pastebin.com/zuqCC4xG
> > http://pastebin.com/88Wx0KDf
> > http://pastebin.com/QZqihN1W
> >
> > Will try deploying 1.1.2 next.
> >
> > Thanks
> > Suresh
> >
> >
> >
> > On Mon, Oct 12, 2015 at 3:46 PM, Ted Yu <yu...@gmail.com> wrote:
> >
> > > bq. cluster enabled for secure HBase with kerberos
> > >
> > > I assume your hdfs cluster has also been kerberized.
> > >
> > > Please pastebin the complete hbase-site.xml
> > >
> > > Please turn on DEBUG logging and pastebin the region server log which
> > hosts
> > > visibilityTest
> > >
> > > BTW if possible, can you deploy 1.1.2 ?
> > >
> > > Cheers
> > >
> > > On Mon, Oct 12, 2015 at 1:14 PM, Suresh Subbiah <
> > > suresh.subbiah60@gmail.com>
> > > wrote:
> > >
> > > > Hi Ted,
> > > >
> > > > I understand that using VisibilityController on an unsercure cluster
> is
> > > of
> > > > limited value. I am still in the early stages of my task. I am logged
> > in
> > > as
> > > > HBase super user and was simply checking if rows could be accessed.
> > > >
> > > > With my colleague's help we did get the cluster enabled for secure
> > HBase
> > > > with kerberos. I repeated the test to get the same result. Our
> cluster
> > is
> > > > on 1.0. Do you think I may be doing something incorrectly? What
> > > information
> > > > can I send to help ensure that I have not made a mistake.
> > > >
> > > > Thanks
> > > > Suresh
> > > >
> > > > hbase shell
> > > > 15/10/12 14:35:09 INFO Configuration.deprecation: hadoop.native.lib
> is
> > > > deprecated. Instead, use io.native.lib.available
> > > > HBase Shell; enter 'help<RETURN>' for list of supported commands.
> > > > Type "exit<RETURN>" to leave the HBase Shell
> > > > Version 1.0.0-cdh5.4.4, rUnknown, Mon Jul 6 16:59:55 PDT 2015
> > > >
> > > > hbase(main):001:0> create 'visibilityTest', 'f1'
> > > > 0 row(s) in 0.7780 seconds
> > > >
> > > > => Hbase::Table - visibilityTest
> > > > hbase(main):002:0> put 'visibilityTest', 'r1', 'f1:c1', 'value1'
> > > > 0 row(s) in 0.1300 seconds
> > > >
> > > > hbase(main):003:0> deleteall 'visibilityTest', 'r1'
> > > > 0 row(s) in 0.0330 seconds
> > > >
> > > > hbase(main):004:0> put 'visibilityTest', 'r1', 'f1:c1', 'value2'
> > > > 0 row(s) in 0.0150 seconds
> > > >
> > > > hbase(main):005:0> scan 'visibilityTest'
> > > > ROW COLUMN+CELL
> > > >
> > > > 0 row(s) in 0.0550 seconds
> > > >
> > > > hbase(main):006:0> scan 'visibilityTest', {RAW=>TRUE}
> > > > ROW COLUMN+CELL
> > > >
> > > > r1 column=f1:, timestamp=1444660561138,
> > > > type=DeleteFamily
> > > > r1 column=f1:c1, timestamp=1444660576868,
> > value=value2
> > > >
> > > > 1 row(s) in 0.0370 seconds
> > > >
> > > > -----------------------------------------------------
> > > > <property>
> > > > <name>hbase.coprocessor.master.classes</name>
> > > >
> > > >
> > > >
> > >
> >
> <value>org.apache.hadoop.hbase.security.visibility.VisibilityController,org.apache.hadoop.hbase.security.access.AccessController</value>
> > > > </property>
> > > >
> > > > <property>
> > > > <name>hbase.coprocessor.region.classes</name>
> > > >
> > > >
> > >
> >
> <value>org.apache.hadoop.hbase.security.visibility.VisibilityController,org.apache.hadoop.hbase.security.token.TokenProvider,org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint,org.apache.hadoop.hbase.security.access.AccessController</value>
> > > > </property>
> > > >
> > > > --------------------------------------------------------
> > > >
> > > >
> > > >
> > > >
> > > > On Sat, Oct 10, 2015 at 9:51 PM, Ted Yu <yu...@gmail.com> wrote:
> > > >
> > > > > To my understanding, VisibilityController is used in a secure
> > cluster.
> > > > > Without security, how do you enforce that only select user(s) can
> > > access
> > > > > certain cells ?
> > > > >
> > > > > Please see the following sections in refguide:
> > > > >
> > > > > http://hbase.apache.org/book.html#hbase.secure.configuration
> > > > >
> > > > >
> > > >
> > >
> >
> http://hbase.apache.org/book.html#_server_side_configuration_for_simple_user_access_operation
> > > > >
> > > > > On Sat, Oct 10, 2015 at 7:40 PM, Suresh Subbiah <
> > > > > suresh.subbiah60@gmail.com>
> > > > > wrote:
> > > > >
> > > > > > Hi Ted,
> > > > > >
> > > > > > Thank you for your response.
> > > > > > I found a machine with HBase 1.0.0 and tried the script with all
> 6
> > > > > coprocs
> > > > > > you listed (2 in master, and 4 in RS). I still do not see the row
> > > after
> > > > > the
> > > > > > second scan.
> > > > > >
> > > > > > However my cluster is not secure enabled I think. Is that
> > necessary?
> > > I
> > > > am
> > > > > > not sure how to do that, though I can ask other members of my
> team
> > > and
> > > > > try
> > > > > > it if that will help.
> > > > > >
> > > > > > It will be ideal if we could get this to work on a 1.0 based
> > version.
> > > > > > Moving to 1.1 will take more time since we have some
> dependencies.
> > > > > >
> > > > > > Thank you
> > > > > > Suresh
> > > > > >
> > > > > > 15/10/10 19:20:44 INFO Configuration.deprecation:
> hadoop.native.lib
> > > is
> > > > > > deprecated. Instead, use io.native.lib.available
> > > > > > HBase Shell; enter 'help<RETURN>' for list of supported commands.
> > > > > > Type "exit<RETURN>" to leave the HBase Shell
> > > > > > Version 1.0.0-cdh5.4.4, rUnknown, Mon Jul 6 16:59:55 PDT 2015
> > > > > >
> > > > > > *hbase(main):001:0> create 'visibilityTest', 'f1' *
> > > > > > *0 row(s) in 0.5460 seconds*
> > > > > >
> > > > > > *=> Hbase::Table - visibilityTest*
> > > > > > *hbase(main):002:0> put 'visibilityTest', 'r1', 'f1:c1',
> 'value1' *
> > > > > > *0 row(s) in 0.0670 seconds*
> > > > > >
> > > > > > *hbase(main):003:0> deleteall 'visibilityTest', 'r1' *
> > > > > > *0 row(s) in 0.0090 seconds*
> > > > > >
> > > > > > *hbase(main):004:0> put 'visibilityTest', 'r1', 'f1:c1',
> 'value2'*
> > > > > > *0 row(s) in 0.0040 seconds*
> > > > > >
> > > > > > *hbase(main):005:0> scan 'visibilityTest'*
> > > > > > *ROW COLUMN+CELL
> > > > > > *
> > > > > > *0 row(s) in 0.0160 seconds*
> > > > > >
> > > > > > *hbase(main):006:0> scan 'visibilityTest', {RAW=>TRUE}*
> > > > > > *ROW COLUMN+CELL
> > > > > > *
> > > > > > * r1 column=f1:, timestamp=1444530064056,
> > > > > > type=DeleteFamily *
> > > > > > * r1 column=f1:c1, timestamp=1444530064084,
> > > > > value=value2
> > > > > > *
> > > > > > *1 row(s) in 0.0580 seconds*
> > > > > >
> > > > > > *hbase(main):007:0> exit*
> > > > > >
> > > > > >
> > > > > > On Sat, Oct 10, 2015 at 7:26 PM, Ted Yu <yu...@gmail.com>
> > wrote:
> > > > > >
> > > > > > > I tried the sequence of commands from your example on a secure
> > > 1.1.2
> > > > > > > cluster with the following config:
> > > > > > >
> > > > > > > <property>
> > > > > > > <name>hbase.coprocessor.master.classes</name>
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> <value>org.apache.hadoop.hbase.security.access.AccessController,org.apache.hadoop.hbase.security.visibility.VisibilityController</value>
> > > > > > > </property>
> > > > > > > <property>
> > > > > > > <name>hbase.coprocessor.region.classes</name>
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> <value>org.apache.hadoop.hbase.security.token.TokenProvider,org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint,org.apache.hadoop.hbase.security.access.AccessController,org.apache.hadoop.hbase.security.visibility.VisibilityController</value>
> > > > > > > </property>
> > > > > > >
> > > > > > > I got:
> > > > > > >
> > > > > > > hbase(main):005:0> scan 'visibilityTest'
> > > > > > > ROW COLUMN+CELL
> > > > > > > r1 column=f1:c1,
> > > > > > > timestamp=1444522994981, value=value2
> > > > > > > 1 row(s) in 0.1020 seconds
> > > > > > >
> > > > > > > Can you try again with 0.98.15 release whose vote passed Friday
> > to
> > > > see
> > > > > if
> > > > > > > what you observed can be reproduced ?
> > > > > > >
> > > > > > > Cheers
> > > > > > >
> > > > > > > On Sat, Oct 10, 2015 at 3:58 PM, Suresh Subbiah <
> > > > > > > suresh.subbiah60@gmail.com>
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Hi,
> > > > > > > >
> > > > > > > > When I run the following script from hbase shell the last
> scan
> > > > > returns
> > > > > > no
> > > > > > > > rows
> > > > > > > >
> > > > > > > > create 'visibilityTest', 'f1'
> > > > > > > > put 'visibilityTest', 'r1', 'f1:c1', 'value1'
> > > > > > > > deleteall 'visibilityTest', 'r1'
> > > > > > > > put 'visibilityTest', 'r1', 'f1:c1', 'value2'
> > > > > > > > scan 'visibilityTest'
> > > > > > > >
> > > > > > > > *hbase(main):013:0> scan 'visibilityTest'*
> > > > > > > > *ROW COLUMN+CELL
> > > > > > > > *
> > > > > > > > *0 row(s) in 0.0100 seconds*
> > > > > > > >
> > > > > > > >
> > > > > > > > However if I run
> > > > > > > > scan 'visibilityTest' , {RAW=>TRUE}
> > > > > > > >
> > > > > > > > I see that the second row that I put is indeed there and has
> a
> > > > > > timestamp
> > > > > > > > value higher that the previous delete
> > > > > > > >
> > > > > > > > *hbase(main):014:0> scan 'visibilityTest', {RAW=>TRUE}*
> > > > > > > > *ROW COLUMN+CELL
> > > > > > > > *
> > > > > > > > * r1 column=f1:, timestamp=1444516578296,
> > > > > > > > type=DeleteFamily *
> > > > > > > > * r1 column=f1:c1, timestamp=1444516647655,
> > > > > > > value=value2
> > > > > > > > *
> > > > > > > > *1 row(s) in 0.0110 seconds*
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > This is on hbase 0.98.6. Problem is seen only when
> > > hbase-site.xml
> > > > > has
> > > > > > > > these lines. No other coprocessors were used during this
> test.
> > > > > > > >
> > > > > > > > <property>
> > > > > > > > <name>hbase.coprocessor.region.classes</name>
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> <value>org.apache.hadoop.hbase.security.visibility.VisibilityController</value>
> > > > > > > > </property>
> > > > > > > > <property>
> > > > > > > > <name>hbase.coprocessor.master.classes</name>
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> <value>org.apache.hadoop.hbase.security.visibility.VisibilityController</value>
> > > > > > > > </property>
> > > > > > > > <property>
> > > > > > > > <name>hfile.format.version</name>
> > > > > > > > <value>3</value>
> > > > > > > > </property>
> > > > > > > >
> > > > > > > > Any suggestions of what I may be doing incorrectly? Or is
> this
> > a
> > > > bug?
> > > > > > > >
> > > > > > > > Thank you
> > > > > > > > Suresh
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
Re: Unexpected behaviour when VisibilityController coprocessor is used
Posted by Ted Yu <yu...@gmail.com>.
Convention is to put AccessController ahead of VisibilityController in
hbase-site.xml
Took a quick pass over region server log but haven't found much yet.
FYI
On Mon, Oct 12, 2015 at 3:28 PM, Suresh Subbiah <su...@gmail.com>
wrote:
> Hi Ted,
>
> Thank you. Yes HDFS cluster has also been kerberized. BTW, this is a
> "cluster" with only one node.
>
> Master hbase-site.xml, RS hbase-site.ml and RS log for the time interval
> test was run is attached
>
> http://pastebin.com/zuqCC4xG
> http://pastebin.com/88Wx0KDf
> http://pastebin.com/QZqihN1W
>
> Will try deploying 1.1.2 next.
>
> Thanks
> Suresh
>
>
>
> On Mon, Oct 12, 2015 at 3:46 PM, Ted Yu <yu...@gmail.com> wrote:
>
> > bq. cluster enabled for secure HBase with kerberos
> >
> > I assume your hdfs cluster has also been kerberized.
> >
> > Please pastebin the complete hbase-site.xml
> >
> > Please turn on DEBUG logging and pastebin the region server log which
> hosts
> > visibilityTest
> >
> > BTW if possible, can you deploy 1.1.2 ?
> >
> > Cheers
> >
> > On Mon, Oct 12, 2015 at 1:14 PM, Suresh Subbiah <
> > suresh.subbiah60@gmail.com>
> > wrote:
> >
> > > Hi Ted,
> > >
> > > I understand that using VisibilityController on an unsercure cluster is
> > of
> > > limited value. I am still in the early stages of my task. I am logged
> in
> > as
> > > HBase super user and was simply checking if rows could be accessed.
> > >
> > > With my colleague's help we did get the cluster enabled for secure
> HBase
> > > with kerberos. I repeated the test to get the same result. Our cluster
> is
> > > on 1.0. Do you think I may be doing something incorrectly? What
> > information
> > > can I send to help ensure that I have not made a mistake.
> > >
> > > Thanks
> > > Suresh
> > >
> > > hbase shell
> > > 15/10/12 14:35:09 INFO Configuration.deprecation: hadoop.native.lib is
> > > deprecated. Instead, use io.native.lib.available
> > > HBase Shell; enter 'help<RETURN>' for list of supported commands.
> > > Type "exit<RETURN>" to leave the HBase Shell
> > > Version 1.0.0-cdh5.4.4, rUnknown, Mon Jul 6 16:59:55 PDT 2015
> > >
> > > hbase(main):001:0> create 'visibilityTest', 'f1'
> > > 0 row(s) in 0.7780 seconds
> > >
> > > => Hbase::Table - visibilityTest
> > > hbase(main):002:0> put 'visibilityTest', 'r1', 'f1:c1', 'value1'
> > > 0 row(s) in 0.1300 seconds
> > >
> > > hbase(main):003:0> deleteall 'visibilityTest', 'r1'
> > > 0 row(s) in 0.0330 seconds
> > >
> > > hbase(main):004:0> put 'visibilityTest', 'r1', 'f1:c1', 'value2'
> > > 0 row(s) in 0.0150 seconds
> > >
> > > hbase(main):005:0> scan 'visibilityTest'
> > > ROW COLUMN+CELL
> > >
> > > 0 row(s) in 0.0550 seconds
> > >
> > > hbase(main):006:0> scan 'visibilityTest', {RAW=>TRUE}
> > > ROW COLUMN+CELL
> > >
> > > r1 column=f1:, timestamp=1444660561138,
> > > type=DeleteFamily
> > > r1 column=f1:c1, timestamp=1444660576868,
> value=value2
> > >
> > > 1 row(s) in 0.0370 seconds
> > >
> > > -----------------------------------------------------
> > > <property>
> > > <name>hbase.coprocessor.master.classes</name>
> > >
> > >
> > >
> >
> <value>org.apache.hadoop.hbase.security.visibility.VisibilityController,org.apache.hadoop.hbase.security.access.AccessController</value>
> > > </property>
> > >
> > > <property>
> > > <name>hbase.coprocessor.region.classes</name>
> > >
> > >
> >
> <value>org.apache.hadoop.hbase.security.visibility.VisibilityController,org.apache.hadoop.hbase.security.token.TokenProvider,org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint,org.apache.hadoop.hbase.security.access.AccessController</value>
> > > </property>
> > >
> > > --------------------------------------------------------
> > >
> > >
> > >
> > >
> > > On Sat, Oct 10, 2015 at 9:51 PM, Ted Yu <yu...@gmail.com> wrote:
> > >
> > > > To my understanding, VisibilityController is used in a secure
> cluster.
> > > > Without security, how do you enforce that only select user(s) can
> > access
> > > > certain cells ?
> > > >
> > > > Please see the following sections in refguide:
> > > >
> > > > http://hbase.apache.org/book.html#hbase.secure.configuration
> > > >
> > > >
> > >
> >
> http://hbase.apache.org/book.html#_server_side_configuration_for_simple_user_access_operation
> > > >
> > > > On Sat, Oct 10, 2015 at 7:40 PM, Suresh Subbiah <
> > > > suresh.subbiah60@gmail.com>
> > > > wrote:
> > > >
> > > > > Hi Ted,
> > > > >
> > > > > Thank you for your response.
> > > > > I found a machine with HBase 1.0.0 and tried the script with all 6
> > > > coprocs
> > > > > you listed (2 in master, and 4 in RS). I still do not see the row
> > after
> > > > the
> > > > > second scan.
> > > > >
> > > > > However my cluster is not secure enabled I think. Is that
> necessary?
> > I
> > > am
> > > > > not sure how to do that, though I can ask other members of my team
> > and
> > > > try
> > > > > it if that will help.
> > > > >
> > > > > It will be ideal if we could get this to work on a 1.0 based
> version.
> > > > > Moving to 1.1 will take more time since we have some dependencies.
> > > > >
> > > > > Thank you
> > > > > Suresh
> > > > >
> > > > > 15/10/10 19:20:44 INFO Configuration.deprecation: hadoop.native.lib
> > is
> > > > > deprecated. Instead, use io.native.lib.available
> > > > > HBase Shell; enter 'help<RETURN>' for list of supported commands.
> > > > > Type "exit<RETURN>" to leave the HBase Shell
> > > > > Version 1.0.0-cdh5.4.4, rUnknown, Mon Jul 6 16:59:55 PDT 2015
> > > > >
> > > > > *hbase(main):001:0> create 'visibilityTest', 'f1' *
> > > > > *0 row(s) in 0.5460 seconds*
> > > > >
> > > > > *=> Hbase::Table - visibilityTest*
> > > > > *hbase(main):002:0> put 'visibilityTest', 'r1', 'f1:c1', 'value1' *
> > > > > *0 row(s) in 0.0670 seconds*
> > > > >
> > > > > *hbase(main):003:0> deleteall 'visibilityTest', 'r1' *
> > > > > *0 row(s) in 0.0090 seconds*
> > > > >
> > > > > *hbase(main):004:0> put 'visibilityTest', 'r1', 'f1:c1', 'value2'*
> > > > > *0 row(s) in 0.0040 seconds*
> > > > >
> > > > > *hbase(main):005:0> scan 'visibilityTest'*
> > > > > *ROW COLUMN+CELL
> > > > > *
> > > > > *0 row(s) in 0.0160 seconds*
> > > > >
> > > > > *hbase(main):006:0> scan 'visibilityTest', {RAW=>TRUE}*
> > > > > *ROW COLUMN+CELL
> > > > > *
> > > > > * r1 column=f1:, timestamp=1444530064056,
> > > > > type=DeleteFamily *
> > > > > * r1 column=f1:c1, timestamp=1444530064084,
> > > > value=value2
> > > > > *
> > > > > *1 row(s) in 0.0580 seconds*
> > > > >
> > > > > *hbase(main):007:0> exit*
> > > > >
> > > > >
> > > > > On Sat, Oct 10, 2015 at 7:26 PM, Ted Yu <yu...@gmail.com>
> wrote:
> > > > >
> > > > > > I tried the sequence of commands from your example on a secure
> > 1.1.2
> > > > > > cluster with the following config:
> > > > > >
> > > > > > <property>
> > > > > > <name>hbase.coprocessor.master.classes</name>
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> <value>org.apache.hadoop.hbase.security.access.AccessController,org.apache.hadoop.hbase.security.visibility.VisibilityController</value>
> > > > > > </property>
> > > > > > <property>
> > > > > > <name>hbase.coprocessor.region.classes</name>
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> <value>org.apache.hadoop.hbase.security.token.TokenProvider,org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint,org.apache.hadoop.hbase.security.access.AccessController,org.apache.hadoop.hbase.security.visibility.VisibilityController</value>
> > > > > > </property>
> > > > > >
> > > > > > I got:
> > > > > >
> > > > > > hbase(main):005:0> scan 'visibilityTest'
> > > > > > ROW COLUMN+CELL
> > > > > > r1 column=f1:c1,
> > > > > > timestamp=1444522994981, value=value2
> > > > > > 1 row(s) in 0.1020 seconds
> > > > > >
> > > > > > Can you try again with 0.98.15 release whose vote passed Friday
> to
> > > see
> > > > if
> > > > > > what you observed can be reproduced ?
> > > > > >
> > > > > > Cheers
> > > > > >
> > > > > > On Sat, Oct 10, 2015 at 3:58 PM, Suresh Subbiah <
> > > > > > suresh.subbiah60@gmail.com>
> > > > > > wrote:
> > > > > >
> > > > > > > Hi,
> > > > > > >
> > > > > > > When I run the following script from hbase shell the last scan
> > > > returns
> > > > > no
> > > > > > > rows
> > > > > > >
> > > > > > > create 'visibilityTest', 'f1'
> > > > > > > put 'visibilityTest', 'r1', 'f1:c1', 'value1'
> > > > > > > deleteall 'visibilityTest', 'r1'
> > > > > > > put 'visibilityTest', 'r1', 'f1:c1', 'value2'
> > > > > > > scan 'visibilityTest'
> > > > > > >
> > > > > > > *hbase(main):013:0> scan 'visibilityTest'*
> > > > > > > *ROW COLUMN+CELL
> > > > > > > *
> > > > > > > *0 row(s) in 0.0100 seconds*
> > > > > > >
> > > > > > >
> > > > > > > However if I run
> > > > > > > scan 'visibilityTest' , {RAW=>TRUE}
> > > > > > >
> > > > > > > I see that the second row that I put is indeed there and has a
> > > > > timestamp
> > > > > > > value higher that the previous delete
> > > > > > >
> > > > > > > *hbase(main):014:0> scan 'visibilityTest', {RAW=>TRUE}*
> > > > > > > *ROW COLUMN+CELL
> > > > > > > *
> > > > > > > * r1 column=f1:, timestamp=1444516578296,
> > > > > > > type=DeleteFamily *
> > > > > > > * r1 column=f1:c1, timestamp=1444516647655,
> > > > > > value=value2
> > > > > > > *
> > > > > > > *1 row(s) in 0.0110 seconds*
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > This is on hbase 0.98.6. Problem is seen only when
> > hbase-site.xml
> > > > has
> > > > > > > these lines. No other coprocessors were used during this test.
> > > > > > >
> > > > > > > <property>
> > > > > > > <name>hbase.coprocessor.region.classes</name>
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> <value>org.apache.hadoop.hbase.security.visibility.VisibilityController</value>
> > > > > > > </property>
> > > > > > > <property>
> > > > > > > <name>hbase.coprocessor.master.classes</name>
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> <value>org.apache.hadoop.hbase.security.visibility.VisibilityController</value>
> > > > > > > </property>
> > > > > > > <property>
> > > > > > > <name>hfile.format.version</name>
> > > > > > > <value>3</value>
> > > > > > > </property>
> > > > > > >
> > > > > > > Any suggestions of what I may be doing incorrectly? Or is this
> a
> > > bug?
> > > > > > >
> > > > > > > Thank you
> > > > > > > Suresh
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
Re: Unexpected behaviour when VisibilityController coprocessor is used
Posted by Suresh Subbiah <su...@gmail.com>.
Hi Ted,
Thank you. Yes HDFS cluster has also been kerberized. BTW, this is a
"cluster" with only one node.
Master hbase-site.xml, RS hbase-site.ml and RS log for the time interval
test was run is attached
http://pastebin.com/zuqCC4xG
http://pastebin.com/88Wx0KDf
http://pastebin.com/QZqihN1W
Will try deploying 1.1.2 next.
Thanks
Suresh
On Mon, Oct 12, 2015 at 3:46 PM, Ted Yu <yu...@gmail.com> wrote:
> bq. cluster enabled for secure HBase with kerberos
>
> I assume your hdfs cluster has also been kerberized.
>
> Please pastebin the complete hbase-site.xml
>
> Please turn on DEBUG logging and pastebin the region server log which hosts
> visibilityTest
>
> BTW if possible, can you deploy 1.1.2 ?
>
> Cheers
>
> On Mon, Oct 12, 2015 at 1:14 PM, Suresh Subbiah <
> suresh.subbiah60@gmail.com>
> wrote:
>
> > Hi Ted,
> >
> > I understand that using VisibilityController on an unsercure cluster is
> of
> > limited value. I am still in the early stages of my task. I am logged in
> as
> > HBase super user and was simply checking if rows could be accessed.
> >
> > With my colleague's help we did get the cluster enabled for secure HBase
> > with kerberos. I repeated the test to get the same result. Our cluster is
> > on 1.0. Do you think I may be doing something incorrectly? What
> information
> > can I send to help ensure that I have not made a mistake.
> >
> > Thanks
> > Suresh
> >
> > hbase shell
> > 15/10/12 14:35:09 INFO Configuration.deprecation: hadoop.native.lib is
> > deprecated. Instead, use io.native.lib.available
> > HBase Shell; enter 'help<RETURN>' for list of supported commands.
> > Type "exit<RETURN>" to leave the HBase Shell
> > Version 1.0.0-cdh5.4.4, rUnknown, Mon Jul 6 16:59:55 PDT 2015
> >
> > hbase(main):001:0> create 'visibilityTest', 'f1'
> > 0 row(s) in 0.7780 seconds
> >
> > => Hbase::Table - visibilityTest
> > hbase(main):002:0> put 'visibilityTest', 'r1', 'f1:c1', 'value1'
> > 0 row(s) in 0.1300 seconds
> >
> > hbase(main):003:0> deleteall 'visibilityTest', 'r1'
> > 0 row(s) in 0.0330 seconds
> >
> > hbase(main):004:0> put 'visibilityTest', 'r1', 'f1:c1', 'value2'
> > 0 row(s) in 0.0150 seconds
> >
> > hbase(main):005:0> scan 'visibilityTest'
> > ROW COLUMN+CELL
> >
> > 0 row(s) in 0.0550 seconds
> >
> > hbase(main):006:0> scan 'visibilityTest', {RAW=>TRUE}
> > ROW COLUMN+CELL
> >
> > r1 column=f1:, timestamp=1444660561138,
> > type=DeleteFamily
> > r1 column=f1:c1, timestamp=1444660576868, value=value2
> >
> > 1 row(s) in 0.0370 seconds
> >
> > -----------------------------------------------------
> > <property>
> > <name>hbase.coprocessor.master.classes</name>
> >
> >
> >
> <value>org.apache.hadoop.hbase.security.visibility.VisibilityController,org.apache.hadoop.hbase.security.access.AccessController</value>
> > </property>
> >
> > <property>
> > <name>hbase.coprocessor.region.classes</name>
> >
> >
> <value>org.apache.hadoop.hbase.security.visibility.VisibilityController,org.apache.hadoop.hbase.security.token.TokenProvider,org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint,org.apache.hadoop.hbase.security.access.AccessController</value>
> > </property>
> >
> > --------------------------------------------------------
> >
> >
> >
> >
> > On Sat, Oct 10, 2015 at 9:51 PM, Ted Yu <yu...@gmail.com> wrote:
> >
> > > To my understanding, VisibilityController is used in a secure cluster.
> > > Without security, how do you enforce that only select user(s) can
> access
> > > certain cells ?
> > >
> > > Please see the following sections in refguide:
> > >
> > > http://hbase.apache.org/book.html#hbase.secure.configuration
> > >
> > >
> >
> http://hbase.apache.org/book.html#_server_side_configuration_for_simple_user_access_operation
> > >
> > > On Sat, Oct 10, 2015 at 7:40 PM, Suresh Subbiah <
> > > suresh.subbiah60@gmail.com>
> > > wrote:
> > >
> > > > Hi Ted,
> > > >
> > > > Thank you for your response.
> > > > I found a machine with HBase 1.0.0 and tried the script with all 6
> > > coprocs
> > > > you listed (2 in master, and 4 in RS). I still do not see the row
> after
> > > the
> > > > second scan.
> > > >
> > > > However my cluster is not secure enabled I think. Is that necessary?
> I
> > am
> > > > not sure how to do that, though I can ask other members of my team
> and
> > > try
> > > > it if that will help.
> > > >
> > > > It will be ideal if we could get this to work on a 1.0 based version.
> > > > Moving to 1.1 will take more time since we have some dependencies.
> > > >
> > > > Thank you
> > > > Suresh
> > > >
> > > > 15/10/10 19:20:44 INFO Configuration.deprecation: hadoop.native.lib
> is
> > > > deprecated. Instead, use io.native.lib.available
> > > > HBase Shell; enter 'help<RETURN>' for list of supported commands.
> > > > Type "exit<RETURN>" to leave the HBase Shell
> > > > Version 1.0.0-cdh5.4.4, rUnknown, Mon Jul 6 16:59:55 PDT 2015
> > > >
> > > > *hbase(main):001:0> create 'visibilityTest', 'f1' *
> > > > *0 row(s) in 0.5460 seconds*
> > > >
> > > > *=> Hbase::Table - visibilityTest*
> > > > *hbase(main):002:0> put 'visibilityTest', 'r1', 'f1:c1', 'value1' *
> > > > *0 row(s) in 0.0670 seconds*
> > > >
> > > > *hbase(main):003:0> deleteall 'visibilityTest', 'r1' *
> > > > *0 row(s) in 0.0090 seconds*
> > > >
> > > > *hbase(main):004:0> put 'visibilityTest', 'r1', 'f1:c1', 'value2'*
> > > > *0 row(s) in 0.0040 seconds*
> > > >
> > > > *hbase(main):005:0> scan 'visibilityTest'*
> > > > *ROW COLUMN+CELL
> > > > *
> > > > *0 row(s) in 0.0160 seconds*
> > > >
> > > > *hbase(main):006:0> scan 'visibilityTest', {RAW=>TRUE}*
> > > > *ROW COLUMN+CELL
> > > > *
> > > > * r1 column=f1:, timestamp=1444530064056,
> > > > type=DeleteFamily *
> > > > * r1 column=f1:c1, timestamp=1444530064084,
> > > value=value2
> > > > *
> > > > *1 row(s) in 0.0580 seconds*
> > > >
> > > > *hbase(main):007:0> exit*
> > > >
> > > >
> > > > On Sat, Oct 10, 2015 at 7:26 PM, Ted Yu <yu...@gmail.com> wrote:
> > > >
> > > > > I tried the sequence of commands from your example on a secure
> 1.1.2
> > > > > cluster with the following config:
> > > > >
> > > > > <property>
> > > > > <name>hbase.coprocessor.master.classes</name>
> > > > >
> > > > >
> > > >
> > >
> >
> <value>org.apache.hadoop.hbase.security.access.AccessController,org.apache.hadoop.hbase.security.visibility.VisibilityController</value>
> > > > > </property>
> > > > > <property>
> > > > > <name>hbase.coprocessor.region.classes</name>
> > > > >
> > > > >
> > > >
> > >
> >
> <value>org.apache.hadoop.hbase.security.token.TokenProvider,org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint,org.apache.hadoop.hbase.security.access.AccessController,org.apache.hadoop.hbase.security.visibility.VisibilityController</value>
> > > > > </property>
> > > > >
> > > > > I got:
> > > > >
> > > > > hbase(main):005:0> scan 'visibilityTest'
> > > > > ROW COLUMN+CELL
> > > > > r1 column=f1:c1,
> > > > > timestamp=1444522994981, value=value2
> > > > > 1 row(s) in 0.1020 seconds
> > > > >
> > > > > Can you try again with 0.98.15 release whose vote passed Friday to
> > see
> > > if
> > > > > what you observed can be reproduced ?
> > > > >
> > > > > Cheers
> > > > >
> > > > > On Sat, Oct 10, 2015 at 3:58 PM, Suresh Subbiah <
> > > > > suresh.subbiah60@gmail.com>
> > > > > wrote:
> > > > >
> > > > > > Hi,
> > > > > >
> > > > > > When I run the following script from hbase shell the last scan
> > > returns
> > > > no
> > > > > > rows
> > > > > >
> > > > > > create 'visibilityTest', 'f1'
> > > > > > put 'visibilityTest', 'r1', 'f1:c1', 'value1'
> > > > > > deleteall 'visibilityTest', 'r1'
> > > > > > put 'visibilityTest', 'r1', 'f1:c1', 'value2'
> > > > > > scan 'visibilityTest'
> > > > > >
> > > > > > *hbase(main):013:0> scan 'visibilityTest'*
> > > > > > *ROW COLUMN+CELL
> > > > > > *
> > > > > > *0 row(s) in 0.0100 seconds*
> > > > > >
> > > > > >
> > > > > > However if I run
> > > > > > scan 'visibilityTest' , {RAW=>TRUE}
> > > > > >
> > > > > > I see that the second row that I put is indeed there and has a
> > > > timestamp
> > > > > > value higher that the previous delete
> > > > > >
> > > > > > *hbase(main):014:0> scan 'visibilityTest', {RAW=>TRUE}*
> > > > > > *ROW COLUMN+CELL
> > > > > > *
> > > > > > * r1 column=f1:, timestamp=1444516578296,
> > > > > > type=DeleteFamily *
> > > > > > * r1 column=f1:c1, timestamp=1444516647655,
> > > > > value=value2
> > > > > > *
> > > > > > *1 row(s) in 0.0110 seconds*
> > > > > >
> > > > > >
> > > > > >
> > > > > > This is on hbase 0.98.6. Problem is seen only when
> hbase-site.xml
> > > has
> > > > > > these lines. No other coprocessors were used during this test.
> > > > > >
> > > > > > <property>
> > > > > > <name>hbase.coprocessor.region.classes</name>
> > > > > >
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> <value>org.apache.hadoop.hbase.security.visibility.VisibilityController</value>
> > > > > > </property>
> > > > > > <property>
> > > > > > <name>hbase.coprocessor.master.classes</name>
> > > > > >
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> <value>org.apache.hadoop.hbase.security.visibility.VisibilityController</value>
> > > > > > </property>
> > > > > > <property>
> > > > > > <name>hfile.format.version</name>
> > > > > > <value>3</value>
> > > > > > </property>
> > > > > >
> > > > > > Any suggestions of what I may be doing incorrectly? Or is this a
> > bug?
> > > > > >
> > > > > > Thank you
> > > > > > Suresh
> > > > > >
> > > > >
> > > >
> > >
> >
>
Re: Unexpected behaviour when VisibilityController coprocessor is used
Posted by Ted Yu <yu...@gmail.com>.
bq. cluster enabled for secure HBase with kerberos
I assume your hdfs cluster has also been kerberized.
Please pastebin the complete hbase-site.xml
Please turn on DEBUG logging and pastebin the region server log which hosts
visibilityTest
BTW if possible, can you deploy 1.1.2 ?
Cheers
On Mon, Oct 12, 2015 at 1:14 PM, Suresh Subbiah <su...@gmail.com>
wrote:
> Hi Ted,
>
> I understand that using VisibilityController on an unsercure cluster is of
> limited value. I am still in the early stages of my task. I am logged in as
> HBase super user and was simply checking if rows could be accessed.
>
> With my colleague's help we did get the cluster enabled for secure HBase
> with kerberos. I repeated the test to get the same result. Our cluster is
> on 1.0. Do you think I may be doing something incorrectly? What information
> can I send to help ensure that I have not made a mistake.
>
> Thanks
> Suresh
>
> hbase shell
> 15/10/12 14:35:09 INFO Configuration.deprecation: hadoop.native.lib is
> deprecated. Instead, use io.native.lib.available
> HBase Shell; enter 'help<RETURN>' for list of supported commands.
> Type "exit<RETURN>" to leave the HBase Shell
> Version 1.0.0-cdh5.4.4, rUnknown, Mon Jul 6 16:59:55 PDT 2015
>
> hbase(main):001:0> create 'visibilityTest', 'f1'
> 0 row(s) in 0.7780 seconds
>
> => Hbase::Table - visibilityTest
> hbase(main):002:0> put 'visibilityTest', 'r1', 'f1:c1', 'value1'
> 0 row(s) in 0.1300 seconds
>
> hbase(main):003:0> deleteall 'visibilityTest', 'r1'
> 0 row(s) in 0.0330 seconds
>
> hbase(main):004:0> put 'visibilityTest', 'r1', 'f1:c1', 'value2'
> 0 row(s) in 0.0150 seconds
>
> hbase(main):005:0> scan 'visibilityTest'
> ROW COLUMN+CELL
>
> 0 row(s) in 0.0550 seconds
>
> hbase(main):006:0> scan 'visibilityTest', {RAW=>TRUE}
> ROW COLUMN+CELL
>
> r1 column=f1:, timestamp=1444660561138,
> type=DeleteFamily
> r1 column=f1:c1, timestamp=1444660576868, value=value2
>
> 1 row(s) in 0.0370 seconds
>
> -----------------------------------------------------
> <property>
> <name>hbase.coprocessor.master.classes</name>
>
>
> <value>org.apache.hadoop.hbase.security.visibility.VisibilityController,org.apache.hadoop.hbase.security.access.AccessController</value>
> </property>
>
> <property>
> <name>hbase.coprocessor.region.classes</name>
>
> <value>org.apache.hadoop.hbase.security.visibility.VisibilityController,org.apache.hadoop.hbase.security.token.TokenProvider,org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint,org.apache.hadoop.hbase.security.access.AccessController</value>
> </property>
>
> --------------------------------------------------------
>
>
>
>
> On Sat, Oct 10, 2015 at 9:51 PM, Ted Yu <yu...@gmail.com> wrote:
>
> > To my understanding, VisibilityController is used in a secure cluster.
> > Without security, how do you enforce that only select user(s) can access
> > certain cells ?
> >
> > Please see the following sections in refguide:
> >
> > http://hbase.apache.org/book.html#hbase.secure.configuration
> >
> >
> http://hbase.apache.org/book.html#_server_side_configuration_for_simple_user_access_operation
> >
> > On Sat, Oct 10, 2015 at 7:40 PM, Suresh Subbiah <
> > suresh.subbiah60@gmail.com>
> > wrote:
> >
> > > Hi Ted,
> > >
> > > Thank you for your response.
> > > I found a machine with HBase 1.0.0 and tried the script with all 6
> > coprocs
> > > you listed (2 in master, and 4 in RS). I still do not see the row after
> > the
> > > second scan.
> > >
> > > However my cluster is not secure enabled I think. Is that necessary? I
> am
> > > not sure how to do that, though I can ask other members of my team and
> > try
> > > it if that will help.
> > >
> > > It will be ideal if we could get this to work on a 1.0 based version.
> > > Moving to 1.1 will take more time since we have some dependencies.
> > >
> > > Thank you
> > > Suresh
> > >
> > > 15/10/10 19:20:44 INFO Configuration.deprecation: hadoop.native.lib is
> > > deprecated. Instead, use io.native.lib.available
> > > HBase Shell; enter 'help<RETURN>' for list of supported commands.
> > > Type "exit<RETURN>" to leave the HBase Shell
> > > Version 1.0.0-cdh5.4.4, rUnknown, Mon Jul 6 16:59:55 PDT 2015
> > >
> > > *hbase(main):001:0> create 'visibilityTest', 'f1' *
> > > *0 row(s) in 0.5460 seconds*
> > >
> > > *=> Hbase::Table - visibilityTest*
> > > *hbase(main):002:0> put 'visibilityTest', 'r1', 'f1:c1', 'value1' *
> > > *0 row(s) in 0.0670 seconds*
> > >
> > > *hbase(main):003:0> deleteall 'visibilityTest', 'r1' *
> > > *0 row(s) in 0.0090 seconds*
> > >
> > > *hbase(main):004:0> put 'visibilityTest', 'r1', 'f1:c1', 'value2'*
> > > *0 row(s) in 0.0040 seconds*
> > >
> > > *hbase(main):005:0> scan 'visibilityTest'*
> > > *ROW COLUMN+CELL
> > > *
> > > *0 row(s) in 0.0160 seconds*
> > >
> > > *hbase(main):006:0> scan 'visibilityTest', {RAW=>TRUE}*
> > > *ROW COLUMN+CELL
> > > *
> > > * r1 column=f1:, timestamp=1444530064056,
> > > type=DeleteFamily *
> > > * r1 column=f1:c1, timestamp=1444530064084,
> > value=value2
> > > *
> > > *1 row(s) in 0.0580 seconds*
> > >
> > > *hbase(main):007:0> exit*
> > >
> > >
> > > On Sat, Oct 10, 2015 at 7:26 PM, Ted Yu <yu...@gmail.com> wrote:
> > >
> > > > I tried the sequence of commands from your example on a secure 1.1.2
> > > > cluster with the following config:
> > > >
> > > > <property>
> > > > <name>hbase.coprocessor.master.classes</name>
> > > >
> > > >
> > >
> >
> <value>org.apache.hadoop.hbase.security.access.AccessController,org.apache.hadoop.hbase.security.visibility.VisibilityController</value>
> > > > </property>
> > > > <property>
> > > > <name>hbase.coprocessor.region.classes</name>
> > > >
> > > >
> > >
> >
> <value>org.apache.hadoop.hbase.security.token.TokenProvider,org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint,org.apache.hadoop.hbase.security.access.AccessController,org.apache.hadoop.hbase.security.visibility.VisibilityController</value>
> > > > </property>
> > > >
> > > > I got:
> > > >
> > > > hbase(main):005:0> scan 'visibilityTest'
> > > > ROW COLUMN+CELL
> > > > r1 column=f1:c1,
> > > > timestamp=1444522994981, value=value2
> > > > 1 row(s) in 0.1020 seconds
> > > >
> > > > Can you try again with 0.98.15 release whose vote passed Friday to
> see
> > if
> > > > what you observed can be reproduced ?
> > > >
> > > > Cheers
> > > >
> > > > On Sat, Oct 10, 2015 at 3:58 PM, Suresh Subbiah <
> > > > suresh.subbiah60@gmail.com>
> > > > wrote:
> > > >
> > > > > Hi,
> > > > >
> > > > > When I run the following script from hbase shell the last scan
> > returns
> > > no
> > > > > rows
> > > > >
> > > > > create 'visibilityTest', 'f1'
> > > > > put 'visibilityTest', 'r1', 'f1:c1', 'value1'
> > > > > deleteall 'visibilityTest', 'r1'
> > > > > put 'visibilityTest', 'r1', 'f1:c1', 'value2'
> > > > > scan 'visibilityTest'
> > > > >
> > > > > *hbase(main):013:0> scan 'visibilityTest'*
> > > > > *ROW COLUMN+CELL
> > > > > *
> > > > > *0 row(s) in 0.0100 seconds*
> > > > >
> > > > >
> > > > > However if I run
> > > > > scan 'visibilityTest' , {RAW=>TRUE}
> > > > >
> > > > > I see that the second row that I put is indeed there and has a
> > > timestamp
> > > > > value higher that the previous delete
> > > > >
> > > > > *hbase(main):014:0> scan 'visibilityTest', {RAW=>TRUE}*
> > > > > *ROW COLUMN+CELL
> > > > > *
> > > > > * r1 column=f1:, timestamp=1444516578296,
> > > > > type=DeleteFamily *
> > > > > * r1 column=f1:c1, timestamp=1444516647655,
> > > > value=value2
> > > > > *
> > > > > *1 row(s) in 0.0110 seconds*
> > > > >
> > > > >
> > > > >
> > > > > This is on hbase 0.98.6. Problem is seen only when hbase-site.xml
> > has
> > > > > these lines. No other coprocessors were used during this test.
> > > > >
> > > > > <property>
> > > > > <name>hbase.coprocessor.region.classes</name>
> > > > >
> > > > >
> > > > >
> > > >
> > >
> >
> <value>org.apache.hadoop.hbase.security.visibility.VisibilityController</value>
> > > > > </property>
> > > > > <property>
> > > > > <name>hbase.coprocessor.master.classes</name>
> > > > >
> > > > >
> > > > >
> > > >
> > >
> >
> <value>org.apache.hadoop.hbase.security.visibility.VisibilityController</value>
> > > > > </property>
> > > > > <property>
> > > > > <name>hfile.format.version</name>
> > > > > <value>3</value>
> > > > > </property>
> > > > >
> > > > > Any suggestions of what I may be doing incorrectly? Or is this a
> bug?
> > > > >
> > > > > Thank you
> > > > > Suresh
> > > > >
> > > >
> > >
> >
>
Re: Unexpected behaviour when VisibilityController coprocessor is used
Posted by Suresh Subbiah <su...@gmail.com>.
Hi Ted,
I understand that using VisibilityController on an unsercure cluster is of
limited value. I am still in the early stages of my task. I am logged in as
HBase super user and was simply checking if rows could be accessed.
With my colleague's help we did get the cluster enabled for secure HBase
with kerberos. I repeated the test to get the same result. Our cluster is
on 1.0. Do you think I may be doing something incorrectly? What information
can I send to help ensure that I have not made a mistake.
Thanks
Suresh
hbase shell
15/10/12 14:35:09 INFO Configuration.deprecation: hadoop.native.lib is
deprecated. Instead, use io.native.lib.available
HBase Shell; enter 'help<RETURN>' for list of supported commands.
Type "exit<RETURN>" to leave the HBase Shell
Version 1.0.0-cdh5.4.4, rUnknown, Mon Jul 6 16:59:55 PDT 2015
hbase(main):001:0> create 'visibilityTest', 'f1'
0 row(s) in 0.7780 seconds
=> Hbase::Table - visibilityTest
hbase(main):002:0> put 'visibilityTest', 'r1', 'f1:c1', 'value1'
0 row(s) in 0.1300 seconds
hbase(main):003:0> deleteall 'visibilityTest', 'r1'
0 row(s) in 0.0330 seconds
hbase(main):004:0> put 'visibilityTest', 'r1', 'f1:c1', 'value2'
0 row(s) in 0.0150 seconds
hbase(main):005:0> scan 'visibilityTest'
ROW COLUMN+CELL
0 row(s) in 0.0550 seconds
hbase(main):006:0> scan 'visibilityTest', {RAW=>TRUE}
ROW COLUMN+CELL
r1 column=f1:, timestamp=1444660561138,
type=DeleteFamily
r1 column=f1:c1, timestamp=1444660576868, value=value2
1 row(s) in 0.0370 seconds
-----------------------------------------------------
<property>
<name>hbase.coprocessor.master.classes</name>
<value>org.apache.hadoop.hbase.security.visibility.VisibilityController,org.apache.hadoop.hbase.security.access.AccessController</value>
</property>
<property>
<name>hbase.coprocessor.region.classes</name>
<value>org.apache.hadoop.hbase.security.visibility.VisibilityController,org.apache.hadoop.hbase.security.token.TokenProvider,org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint,org.apache.hadoop.hbase.security.access.AccessController</value>
</property>
--------------------------------------------------------
On Sat, Oct 10, 2015 at 9:51 PM, Ted Yu <yu...@gmail.com> wrote:
> To my understanding, VisibilityController is used in a secure cluster.
> Without security, how do you enforce that only select user(s) can access
> certain cells ?
>
> Please see the following sections in refguide:
>
> http://hbase.apache.org/book.html#hbase.secure.configuration
>
> http://hbase.apache.org/book.html#_server_side_configuration_for_simple_user_access_operation
>
> On Sat, Oct 10, 2015 at 7:40 PM, Suresh Subbiah <
> suresh.subbiah60@gmail.com>
> wrote:
>
> > Hi Ted,
> >
> > Thank you for your response.
> > I found a machine with HBase 1.0.0 and tried the script with all 6
> coprocs
> > you listed (2 in master, and 4 in RS). I still do not see the row after
> the
> > second scan.
> >
> > However my cluster is not secure enabled I think. Is that necessary? I am
> > not sure how to do that, though I can ask other members of my team and
> try
> > it if that will help.
> >
> > It will be ideal if we could get this to work on a 1.0 based version.
> > Moving to 1.1 will take more time since we have some dependencies.
> >
> > Thank you
> > Suresh
> >
> > 15/10/10 19:20:44 INFO Configuration.deprecation: hadoop.native.lib is
> > deprecated. Instead, use io.native.lib.available
> > HBase Shell; enter 'help<RETURN>' for list of supported commands.
> > Type "exit<RETURN>" to leave the HBase Shell
> > Version 1.0.0-cdh5.4.4, rUnknown, Mon Jul 6 16:59:55 PDT 2015
> >
> > *hbase(main):001:0> create 'visibilityTest', 'f1' *
> > *0 row(s) in 0.5460 seconds*
> >
> > *=> Hbase::Table - visibilityTest*
> > *hbase(main):002:0> put 'visibilityTest', 'r1', 'f1:c1', 'value1' *
> > *0 row(s) in 0.0670 seconds*
> >
> > *hbase(main):003:0> deleteall 'visibilityTest', 'r1' *
> > *0 row(s) in 0.0090 seconds*
> >
> > *hbase(main):004:0> put 'visibilityTest', 'r1', 'f1:c1', 'value2'*
> > *0 row(s) in 0.0040 seconds*
> >
> > *hbase(main):005:0> scan 'visibilityTest'*
> > *ROW COLUMN+CELL
> > *
> > *0 row(s) in 0.0160 seconds*
> >
> > *hbase(main):006:0> scan 'visibilityTest', {RAW=>TRUE}*
> > *ROW COLUMN+CELL
> > *
> > * r1 column=f1:, timestamp=1444530064056,
> > type=DeleteFamily *
> > * r1 column=f1:c1, timestamp=1444530064084,
> value=value2
> > *
> > *1 row(s) in 0.0580 seconds*
> >
> > *hbase(main):007:0> exit*
> >
> >
> > On Sat, Oct 10, 2015 at 7:26 PM, Ted Yu <yu...@gmail.com> wrote:
> >
> > > I tried the sequence of commands from your example on a secure 1.1.2
> > > cluster with the following config:
> > >
> > > <property>
> > > <name>hbase.coprocessor.master.classes</name>
> > >
> > >
> >
> <value>org.apache.hadoop.hbase.security.access.AccessController,org.apache.hadoop.hbase.security.visibility.VisibilityController</value>
> > > </property>
> > > <property>
> > > <name>hbase.coprocessor.region.classes</name>
> > >
> > >
> >
> <value>org.apache.hadoop.hbase.security.token.TokenProvider,org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint,org.apache.hadoop.hbase.security.access.AccessController,org.apache.hadoop.hbase.security.visibility.VisibilityController</value>
> > > </property>
> > >
> > > I got:
> > >
> > > hbase(main):005:0> scan 'visibilityTest'
> > > ROW COLUMN+CELL
> > > r1 column=f1:c1,
> > > timestamp=1444522994981, value=value2
> > > 1 row(s) in 0.1020 seconds
> > >
> > > Can you try again with 0.98.15 release whose vote passed Friday to see
> if
> > > what you observed can be reproduced ?
> > >
> > > Cheers
> > >
> > > On Sat, Oct 10, 2015 at 3:58 PM, Suresh Subbiah <
> > > suresh.subbiah60@gmail.com>
> > > wrote:
> > >
> > > > Hi,
> > > >
> > > > When I run the following script from hbase shell the last scan
> returns
> > no
> > > > rows
> > > >
> > > > create 'visibilityTest', 'f1'
> > > > put 'visibilityTest', 'r1', 'f1:c1', 'value1'
> > > > deleteall 'visibilityTest', 'r1'
> > > > put 'visibilityTest', 'r1', 'f1:c1', 'value2'
> > > > scan 'visibilityTest'
> > > >
> > > > *hbase(main):013:0> scan 'visibilityTest'*
> > > > *ROW COLUMN+CELL
> > > > *
> > > > *0 row(s) in 0.0100 seconds*
> > > >
> > > >
> > > > However if I run
> > > > scan 'visibilityTest' , {RAW=>TRUE}
> > > >
> > > > I see that the second row that I put is indeed there and has a
> > timestamp
> > > > value higher that the previous delete
> > > >
> > > > *hbase(main):014:0> scan 'visibilityTest', {RAW=>TRUE}*
> > > > *ROW COLUMN+CELL
> > > > *
> > > > * r1 column=f1:, timestamp=1444516578296,
> > > > type=DeleteFamily *
> > > > * r1 column=f1:c1, timestamp=1444516647655,
> > > value=value2
> > > > *
> > > > *1 row(s) in 0.0110 seconds*
> > > >
> > > >
> > > >
> > > > This is on hbase 0.98.6. Problem is seen only when hbase-site.xml
> has
> > > > these lines. No other coprocessors were used during this test.
> > > >
> > > > <property>
> > > > <name>hbase.coprocessor.region.classes</name>
> > > >
> > > >
> > > >
> > >
> >
> <value>org.apache.hadoop.hbase.security.visibility.VisibilityController</value>
> > > > </property>
> > > > <property>
> > > > <name>hbase.coprocessor.master.classes</name>
> > > >
> > > >
> > > >
> > >
> >
> <value>org.apache.hadoop.hbase.security.visibility.VisibilityController</value>
> > > > </property>
> > > > <property>
> > > > <name>hfile.format.version</name>
> > > > <value>3</value>
> > > > </property>
> > > >
> > > > Any suggestions of what I may be doing incorrectly? Or is this a bug?
> > > >
> > > > Thank you
> > > > Suresh
> > > >
> > >
> >
>
Re: Unexpected behaviour when VisibilityController coprocessor is used
Posted by Ted Yu <yu...@gmail.com>.
To my understanding, VisibilityController is used in a secure cluster.
Without security, how do you enforce that only select user(s) can access
certain cells ?
Please see the following sections in refguide:
http://hbase.apache.org/book.html#hbase.secure.configuration
http://hbase.apache.org/book.html#_server_side_configuration_for_simple_user_access_operation
On Sat, Oct 10, 2015 at 7:40 PM, Suresh Subbiah <su...@gmail.com>
wrote:
> Hi Ted,
>
> Thank you for your response.
> I found a machine with HBase 1.0.0 and tried the script with all 6 coprocs
> you listed (2 in master, and 4 in RS). I still do not see the row after the
> second scan.
>
> However my cluster is not secure enabled I think. Is that necessary? I am
> not sure how to do that, though I can ask other members of my team and try
> it if that will help.
>
> It will be ideal if we could get this to work on a 1.0 based version.
> Moving to 1.1 will take more time since we have some dependencies.
>
> Thank you
> Suresh
>
> 15/10/10 19:20:44 INFO Configuration.deprecation: hadoop.native.lib is
> deprecated. Instead, use io.native.lib.available
> HBase Shell; enter 'help<RETURN>' for list of supported commands.
> Type "exit<RETURN>" to leave the HBase Shell
> Version 1.0.0-cdh5.4.4, rUnknown, Mon Jul 6 16:59:55 PDT 2015
>
> *hbase(main):001:0> create 'visibilityTest', 'f1' *
> *0 row(s) in 0.5460 seconds*
>
> *=> Hbase::Table - visibilityTest*
> *hbase(main):002:0> put 'visibilityTest', 'r1', 'f1:c1', 'value1' *
> *0 row(s) in 0.0670 seconds*
>
> *hbase(main):003:0> deleteall 'visibilityTest', 'r1' *
> *0 row(s) in 0.0090 seconds*
>
> *hbase(main):004:0> put 'visibilityTest', 'r1', 'f1:c1', 'value2'*
> *0 row(s) in 0.0040 seconds*
>
> *hbase(main):005:0> scan 'visibilityTest'*
> *ROW COLUMN+CELL
> *
> *0 row(s) in 0.0160 seconds*
>
> *hbase(main):006:0> scan 'visibilityTest', {RAW=>TRUE}*
> *ROW COLUMN+CELL
> *
> * r1 column=f1:, timestamp=1444530064056,
> type=DeleteFamily *
> * r1 column=f1:c1, timestamp=1444530064084, value=value2
> *
> *1 row(s) in 0.0580 seconds*
>
> *hbase(main):007:0> exit*
>
>
> On Sat, Oct 10, 2015 at 7:26 PM, Ted Yu <yu...@gmail.com> wrote:
>
> > I tried the sequence of commands from your example on a secure 1.1.2
> > cluster with the following config:
> >
> > <property>
> > <name>hbase.coprocessor.master.classes</name>
> >
> >
> <value>org.apache.hadoop.hbase.security.access.AccessController,org.apache.hadoop.hbase.security.visibility.VisibilityController</value>
> > </property>
> > <property>
> > <name>hbase.coprocessor.region.classes</name>
> >
> >
> <value>org.apache.hadoop.hbase.security.token.TokenProvider,org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint,org.apache.hadoop.hbase.security.access.AccessController,org.apache.hadoop.hbase.security.visibility.VisibilityController</value>
> > </property>
> >
> > I got:
> >
> > hbase(main):005:0> scan 'visibilityTest'
> > ROW COLUMN+CELL
> > r1 column=f1:c1,
> > timestamp=1444522994981, value=value2
> > 1 row(s) in 0.1020 seconds
> >
> > Can you try again with 0.98.15 release whose vote passed Friday to see if
> > what you observed can be reproduced ?
> >
> > Cheers
> >
> > On Sat, Oct 10, 2015 at 3:58 PM, Suresh Subbiah <
> > suresh.subbiah60@gmail.com>
> > wrote:
> >
> > > Hi,
> > >
> > > When I run the following script from hbase shell the last scan returns
> no
> > > rows
> > >
> > > create 'visibilityTest', 'f1'
> > > put 'visibilityTest', 'r1', 'f1:c1', 'value1'
> > > deleteall 'visibilityTest', 'r1'
> > > put 'visibilityTest', 'r1', 'f1:c1', 'value2'
> > > scan 'visibilityTest'
> > >
> > > *hbase(main):013:0> scan 'visibilityTest'*
> > > *ROW COLUMN+CELL
> > > *
> > > *0 row(s) in 0.0100 seconds*
> > >
> > >
> > > However if I run
> > > scan 'visibilityTest' , {RAW=>TRUE}
> > >
> > > I see that the second row that I put is indeed there and has a
> timestamp
> > > value higher that the previous delete
> > >
> > > *hbase(main):014:0> scan 'visibilityTest', {RAW=>TRUE}*
> > > *ROW COLUMN+CELL
> > > *
> > > * r1 column=f1:, timestamp=1444516578296,
> > > type=DeleteFamily *
> > > * r1 column=f1:c1, timestamp=1444516647655,
> > value=value2
> > > *
> > > *1 row(s) in 0.0110 seconds*
> > >
> > >
> > >
> > > This is on hbase 0.98.6. Problem is seen only when hbase-site.xml has
> > > these lines. No other coprocessors were used during this test.
> > >
> > > <property>
> > > <name>hbase.coprocessor.region.classes</name>
> > >
> > >
> > >
> >
> <value>org.apache.hadoop.hbase.security.visibility.VisibilityController</value>
> > > </property>
> > > <property>
> > > <name>hbase.coprocessor.master.classes</name>
> > >
> > >
> > >
> >
> <value>org.apache.hadoop.hbase.security.visibility.VisibilityController</value>
> > > </property>
> > > <property>
> > > <name>hfile.format.version</name>
> > > <value>3</value>
> > > </property>
> > >
> > > Any suggestions of what I may be doing incorrectly? Or is this a bug?
> > >
> > > Thank you
> > > Suresh
> > >
> >
>
Re: Unexpected behaviour when VisibilityController coprocessor is used
Posted by Suresh Subbiah <su...@gmail.com>.
Hi Ted,
Thank you for your response.
I found a machine with HBase 1.0.0 and tried the script with all 6 coprocs
you listed (2 in master, and 4 in RS). I still do not see the row after the
second scan.
However my cluster is not secure enabled I think. Is that necessary? I am
not sure how to do that, though I can ask other members of my team and try
it if that will help.
It will be ideal if we could get this to work on a 1.0 based version.
Moving to 1.1 will take more time since we have some dependencies.
Thank you
Suresh
15/10/10 19:20:44 INFO Configuration.deprecation: hadoop.native.lib is
deprecated. Instead, use io.native.lib.available
HBase Shell; enter 'help<RETURN>' for list of supported commands.
Type "exit<RETURN>" to leave the HBase Shell
Version 1.0.0-cdh5.4.4, rUnknown, Mon Jul 6 16:59:55 PDT 2015
*hbase(main):001:0> create 'visibilityTest', 'f1' *
*0 row(s) in 0.5460 seconds*
*=> Hbase::Table - visibilityTest*
*hbase(main):002:0> put 'visibilityTest', 'r1', 'f1:c1', 'value1' *
*0 row(s) in 0.0670 seconds*
*hbase(main):003:0> deleteall 'visibilityTest', 'r1' *
*0 row(s) in 0.0090 seconds*
*hbase(main):004:0> put 'visibilityTest', 'r1', 'f1:c1', 'value2'*
*0 row(s) in 0.0040 seconds*
*hbase(main):005:0> scan 'visibilityTest'*
*ROW COLUMN+CELL
*
*0 row(s) in 0.0160 seconds*
*hbase(main):006:0> scan 'visibilityTest', {RAW=>TRUE}*
*ROW COLUMN+CELL
*
* r1 column=f1:, timestamp=1444530064056,
type=DeleteFamily *
* r1 column=f1:c1, timestamp=1444530064084, value=value2
*
*1 row(s) in 0.0580 seconds*
*hbase(main):007:0> exit*
On Sat, Oct 10, 2015 at 7:26 PM, Ted Yu <yu...@gmail.com> wrote:
> I tried the sequence of commands from your example on a secure 1.1.2
> cluster with the following config:
>
> <property>
> <name>hbase.coprocessor.master.classes</name>
>
> <value>org.apache.hadoop.hbase.security.access.AccessController,org.apache.hadoop.hbase.security.visibility.VisibilityController</value>
> </property>
> <property>
> <name>hbase.coprocessor.region.classes</name>
>
> <value>org.apache.hadoop.hbase.security.token.TokenProvider,org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint,org.apache.hadoop.hbase.security.access.AccessController,org.apache.hadoop.hbase.security.visibility.VisibilityController</value>
> </property>
>
> I got:
>
> hbase(main):005:0> scan 'visibilityTest'
> ROW COLUMN+CELL
> r1 column=f1:c1,
> timestamp=1444522994981, value=value2
> 1 row(s) in 0.1020 seconds
>
> Can you try again with 0.98.15 release whose vote passed Friday to see if
> what you observed can be reproduced ?
>
> Cheers
>
> On Sat, Oct 10, 2015 at 3:58 PM, Suresh Subbiah <
> suresh.subbiah60@gmail.com>
> wrote:
>
> > Hi,
> >
> > When I run the following script from hbase shell the last scan returns no
> > rows
> >
> > create 'visibilityTest', 'f1'
> > put 'visibilityTest', 'r1', 'f1:c1', 'value1'
> > deleteall 'visibilityTest', 'r1'
> > put 'visibilityTest', 'r1', 'f1:c1', 'value2'
> > scan 'visibilityTest'
> >
> > *hbase(main):013:0> scan 'visibilityTest'*
> > *ROW COLUMN+CELL
> > *
> > *0 row(s) in 0.0100 seconds*
> >
> >
> > However if I run
> > scan 'visibilityTest' , {RAW=>TRUE}
> >
> > I see that the second row that I put is indeed there and has a timestamp
> > value higher that the previous delete
> >
> > *hbase(main):014:0> scan 'visibilityTest', {RAW=>TRUE}*
> > *ROW COLUMN+CELL
> > *
> > * r1 column=f1:, timestamp=1444516578296,
> > type=DeleteFamily *
> > * r1 column=f1:c1, timestamp=1444516647655,
> value=value2
> > *
> > *1 row(s) in 0.0110 seconds*
> >
> >
> >
> > This is on hbase 0.98.6. Problem is seen only when hbase-site.xml has
> > these lines. No other coprocessors were used during this test.
> >
> > <property>
> > <name>hbase.coprocessor.region.classes</name>
> >
> >
> >
> <value>org.apache.hadoop.hbase.security.visibility.VisibilityController</value>
> > </property>
> > <property>
> > <name>hbase.coprocessor.master.classes</name>
> >
> >
> >
> <value>org.apache.hadoop.hbase.security.visibility.VisibilityController</value>
> > </property>
> > <property>
> > <name>hfile.format.version</name>
> > <value>3</value>
> > </property>
> >
> > Any suggestions of what I may be doing incorrectly? Or is this a bug?
> >
> > Thank you
> > Suresh
> >
>
Re: Unexpected behaviour when VisibilityController coprocessor is used
Posted by Ted Yu <yu...@gmail.com>.
I tried the sequence of commands from your example on a secure 1.1.2
cluster with the following config:
<property>
<name>hbase.coprocessor.master.classes</name>
<value>org.apache.hadoop.hbase.security.access.AccessController,org.apache.hadoop.hbase.security.visibility.VisibilityController</value>
</property>
<property>
<name>hbase.coprocessor.region.classes</name>
<value>org.apache.hadoop.hbase.security.token.TokenProvider,org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint,org.apache.hadoop.hbase.security.access.AccessController,org.apache.hadoop.hbase.security.visibility.VisibilityController</value>
</property>
I got:
hbase(main):005:0> scan 'visibilityTest'
ROW COLUMN+CELL
r1 column=f1:c1,
timestamp=1444522994981, value=value2
1 row(s) in 0.1020 seconds
Can you try again with 0.98.15 release whose vote passed Friday to see if
what you observed can be reproduced ?
Cheers
On Sat, Oct 10, 2015 at 3:58 PM, Suresh Subbiah <su...@gmail.com>
wrote:
> Hi,
>
> When I run the following script from hbase shell the last scan returns no
> rows
>
> create 'visibilityTest', 'f1'
> put 'visibilityTest', 'r1', 'f1:c1', 'value1'
> deleteall 'visibilityTest', 'r1'
> put 'visibilityTest', 'r1', 'f1:c1', 'value2'
> scan 'visibilityTest'
>
> *hbase(main):013:0> scan 'visibilityTest'*
> *ROW COLUMN+CELL
> *
> *0 row(s) in 0.0100 seconds*
>
>
> However if I run
> scan 'visibilityTest' , {RAW=>TRUE}
>
> I see that the second row that I put is indeed there and has a timestamp
> value higher that the previous delete
>
> *hbase(main):014:0> scan 'visibilityTest', {RAW=>TRUE}*
> *ROW COLUMN+CELL
> *
> * r1 column=f1:, timestamp=1444516578296,
> type=DeleteFamily *
> * r1 column=f1:c1, timestamp=1444516647655, value=value2
> *
> *1 row(s) in 0.0110 seconds*
>
>
>
> This is on hbase 0.98.6. Problem is seen only when hbase-site.xml has
> these lines. No other coprocessors were used during this test.
>
> <property>
> <name>hbase.coprocessor.region.classes</name>
>
>
> <value>org.apache.hadoop.hbase.security.visibility.VisibilityController</value>
> </property>
> <property>
> <name>hbase.coprocessor.master.classes</name>
>
>
> <value>org.apache.hadoop.hbase.security.visibility.VisibilityController</value>
> </property>
> <property>
> <name>hfile.format.version</name>
> <value>3</value>
> </property>
>
> Any suggestions of what I may be doing incorrectly? Or is this a bug?
>
> Thank you
> Suresh
>