You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by Stuti Awasthi <st...@hcl.com> on 2011/08/29 09:38:43 UTC
Scan not working properly on composite keys
Hi Friends,
I have a query regarding scan operation on composite keys.
I have table as follows :
admin/user1 column=info:email, timestamp=1314594930540, value=abc
admin/ user1 column=info:mobile, timestamp=1314594940686, value=123456
admin/ user1 column=info:name, timestamp=1314594915091, value=def
admin/user2 column=info:email, timestamp=1314597627865, value=def
admin/ user2 column=info:mobile, timestamp=1314597636663, value=67890
admin/ user2 column=info:name, timestamp=1314597614585, value=pqr
viewer/user3 column=info:email, timestamp=1314601194373, value=pqr
viewer/ user3 column=info:mobile, timestamp=1314601256479, value=12345
viewer/ user3 column=info:name, timestamp=1314601207662, value=xyz
I want to find all the records of "user2". For this I tried following query but it is giving me all records of user3 :
hbase(main):005:0> scan 'users', {STARTROW=>'user2'}
ROW COLUMN+CELL
viewer/user3 column=info:email, timestamp=1314601194373, value=satyendrap@hcl.com
viewer/ user3 column=info:mobile, timestamp=1314601256479, value=37498734893
viewer/ user3 column=info:name, timestamp=1314601207662, value=satyendra pandey
1 row(s) in 0.0330 seconds
I also tried various queries for this but not getting correct result. I am definitely missing something. Please help
Thanks & Regards
Stuti Awasthi
Sr Specialist
________________________________
::DISCLAIMER::
-----------------------------------------------------------------------------------------------------------------------
The contents of this e-mail and any attachment(s) are confidential and intended for the named recipient(s) only.
It shall not attach any liability on the originator or HCL or its affiliates. Any views or opinions presented in
this email are solely those of the author and may not necessarily reflect the opinions of HCL or its affiliates.
Any form of reproduction, dissemination, copying, disclosure, modification, distribution and / or publication of
this message without the prior written consent of the author of this e-mail is strictly prohibited. If you have
received this email in error please delete it and notify the sender immediately. Before opening any mail and
attachments please check them for viruses and defect.
-----------------------------------------------------------------------------------------------------------------------
Re: Scan not working properly on composite keys
Posted by lars hofhansl <lh...@yahoo.com>.
Nope :)
Sorting the map starting with the row-key is fundamental to HBase.
If you need to search for a suffix of a key that will need to scan the entire table.
You can always create another table with the right key structure for your search and have it point back to your table (i.e. the key of the main table is stored as a value of this table). See: http://hbase.apache.org/book/secondary.indexes.html
-- Lars
----- Original Message -----
From: Stuti Awasthi <st...@hcl.com>
To: "user@hbase.apache.org" <us...@hbase.apache.org>
Cc:
Sent: Monday, August 29, 2011 8:49 AM
Subject: RE: Scan not working properly on composite keys
Hi,
Sure, I tried also by prefixing with admin, and it was working fine. I was just wondering that is there any way that we can get the information from Hbase using some part of Composite keys or wild card characters ?
Thanks
-----Original Message-----
From: Buttler, David [mailto:buttler1@llnl.gov]
Sent: Monday, August 29, 2011 8:46 PM
To: user@hbase.apache.org
Subject: RE: Scan not working properly on composite keys
The scan started at "viewer/user3" because that is the next key after "user2". If your keys have a prefix, like admin or viewer, then you need to prepend that to your start row as well. Hbase has no magic where it can intuit your intent.
Dave
-----Original Message-----
From: Stuti Awasthi [mailto:stutiawasthi@hcl.com]
Sent: Monday, August 29, 2011 12:39 AM
To: user@hbase.apache.org
Subject: Scan not working properly on composite keys
Hi Friends,
I have a query regarding scan operation on composite keys.
I have table as follows :
admin/user1 column=info:email, timestamp=1314594930540, value=abc
admin/ user1 column=info:mobile, timestamp=1314594940686, value=123456
admin/ user1 column=info:name, timestamp=1314594915091, value=def
admin/user2 column=info:email, timestamp=1314597627865, value=def
admin/ user2 column=info:mobile, timestamp=1314597636663, value=67890
admin/ user2 column=info:name, timestamp=1314597614585, value=pqr
viewer/user3 column=info:email, timestamp=1314601194373, value=pqr
viewer/ user3 column=info:mobile, timestamp=1314601256479, value=12345
viewer/ user3 column=info:name, timestamp=1314601207662, value=xyz
I want to find all the records of "user2". For this I tried following query but it is giving me all records of user3 :
hbase(main):005:0> scan 'users', {STARTROW=>'user2'}
ROW COLUMN+CELL
viewer/user3 column=info:email, timestamp=1314601194373, value=satyendrap@hcl.com
viewer/ user3 column=info:mobile, timestamp=1314601256479, value=37498734893
viewer/ user3 column=info:name, timestamp=1314601207662, value=satyendra pandey
1 row(s) in 0.0330 seconds
I also tried various queries for this but not getting correct result. I am definitely missing something. Please help
Thanks & Regards
Stuti Awasthi
Sr Specialist
________________________________
::DISCLAIMER::
-----------------------------------------------------------------------------------------------------------------------
The contents of this e-mail and any attachment(s) are confidential and intended for the named recipient(s) only.
It shall not attach any liability on the originator or HCL or its affiliates. Any views or opinions presented in this email are solely those of the author and may not necessarily reflect the opinions of HCL or its affiliates.
Any form of reproduction, dissemination, copying, disclosure, modification, distribution and / or publication of this message without the prior written consent of the author of this e-mail is strictly prohibited. If you have received this email in error please delete it and notify the sender immediately. Before opening any mail and attachments please check them for viruses and defect.
-----------------------------------------------------------------------------------------------------------------------
RE: Scan not working properly on composite keys
Posted by Stuti Awasthi <st...@hcl.com>.
Hi Ian,
Thanks for explaining this. :)
-----Original Message-----
From: Ian Varley [mailto:ivarley@salesforce.com]
Sent: Monday, August 29, 2011 9:31 PM
To: user@hbase.apache.org
Subject: Re: Scan not working properly on composite keys
Stuti,
The rows are physically sorted on disk (and in memory) according to the row keys you define, and that's the only way that HBase can access them (unlike a relational database, it doesn't have built in indexes that allow you to access rows by something other than their physical sort order). So having a general wildcard lookup feature wouldn't be practical; in reality, it would have to scan ever row in the table to see which ones match your pattern.
That said, you could build a little wrapper on top of your access code as follows (pseudocode):
function getRowsByUser(string user) {
prefixes = "admin", "viewer", "etc";
for each prefix in prefixes {
result.add(getRowsFromHBase(prefix + user));
}
return result;
}
That way, at least your scan is linear in the number of prefixes--it just does one constant-time seek to the start of the rows for the given user, for each of the prefixes you know about.
You can't expect functionality like this to be built in to HBase, because to HBase, row keys are simply byte arrays, nothing more. It does nothing to enforce structure on them, that's entirely your application's job.
Ian
On Aug 29, 2011, at 10:49 AM, Stuti Awasthi wrote:
> Hi,
>
> Sure, I tried also by prefixing with admin, and it was working fine. I was just wondering that is there any way that we can get the information from Hbase using some part of Composite keys or wild card characters ?
>
> Thanks
>
> -----Original Message-----
> From: Buttler, David [mailto:buttler1@llnl.gov]
> Sent: Monday, August 29, 2011 8:46 PM
> To: user@hbase.apache.org
> Subject: RE: Scan not working properly on composite keys
>
> The scan started at "viewer/user3" because that is the next key after "user2". If your keys have a prefix, like admin or viewer, then you need to prepend that to your start row as well. Hbase has no magic where it can intuit your intent.
>
> Dave
>
> -----Original Message-----
> From: Stuti Awasthi [mailto:stutiawasthi@hcl.com]
> Sent: Monday, August 29, 2011 12:39 AM
> To: user@hbase.apache.org
> Subject: Scan not working properly on composite keys
>
> Hi Friends,
>
> I have a query regarding scan operation on composite keys.
>
> I have table as follows :
>
> admin/user1 column=info:email, timestamp=1314594930540, value=abc
> admin/ user1 column=info:mobile, timestamp=1314594940686, value=123456
> admin/ user1 column=info:name, timestamp=1314594915091, value=def
> admin/user2 column=info:email, timestamp=1314597627865, value=def
> admin/ user2 column=info:mobile, timestamp=1314597636663, value=67890
> admin/ user2 column=info:name, timestamp=1314597614585, value=pqr
> viewer/user3 column=info:email, timestamp=1314601194373, value=pqr
> viewer/ user3 column=info:mobile, timestamp=1314601256479, value=12345
> viewer/ user3 column=info:name, timestamp=1314601207662, value=xyz
>
> I want to find all the records of "user2". For this I tried following query but it is giving me all records of user3 :
>
> hbase(main):005:0> scan 'users', {STARTROW=>'user2'}
> ROW COLUMN+CELL
> viewer/user3 column=info:email, timestamp=1314601194373, value=satyendrap@hcl.com
> viewer/ user3 column=info:mobile, timestamp=1314601256479, value=37498734893
> viewer/ user3 column=info:name, timestamp=1314601207662, value=satyendra pandey
> 1 row(s) in 0.0330 seconds
>
> I also tried various queries for this but not getting correct result. I am definitely missing something. Please help
>
> Thanks & Regards
> Stuti Awasthi
> Sr Specialist
>
>
> ________________________________
> ::DISCLAIMER::
> -----------------------------------------------------------------------------------------------------------------------
>
> The contents of this e-mail and any attachment(s) are confidential and intended for the named recipient(s) only.
> It shall not attach any liability on the originator or HCL or its affiliates. Any views or opinions presented in this email are solely those of the author and may not necessarily reflect the opinions of HCL or its affiliates.
> Any form of reproduction, dissemination, copying, disclosure, modification, distribution and / or publication of this message without the prior written consent of the author of this e-mail is strictly prohibited. If you have received this email in error please delete it and notify the sender immediately. Before opening any mail and attachments please check them for viruses and defect.
>
> -----------------------------------------------------------------------------------------------------------------------
Re: Scan not working properly on composite keys
Posted by Ian Varley <iv...@salesforce.com>.
Stuti,
The rows are physically sorted on disk (and in memory) according to the row keys you define, and that's the only way that HBase can access them (unlike a relational database, it doesn't have built in indexes that allow you to access rows by something other than their physical sort order). So having a general wildcard lookup feature wouldn't be practical; in reality, it would have to scan ever row in the table to see which ones match your pattern.
That said, you could build a little wrapper on top of your access code as follows (pseudocode):
function getRowsByUser(string user) {
prefixes = "admin", "viewer", "etc";
for each prefix in prefixes {
result.add(getRowsFromHBase(prefix + user));
}
return result;
}
That way, at least your scan is linear in the number of prefixes--it just does one constant-time seek to the start of the rows for the given user, for each of the prefixes you know about.
You can't expect functionality like this to be built in to HBase, because to HBase, row keys are simply byte arrays, nothing more. It does nothing to enforce structure on them, that's entirely your application's job.
Ian
On Aug 29, 2011, at 10:49 AM, Stuti Awasthi wrote:
> Hi,
>
> Sure, I tried also by prefixing with admin, and it was working fine. I was just wondering that is there any way that we can get the information from Hbase using some part of Composite keys or wild card characters ?
>
> Thanks
>
> -----Original Message-----
> From: Buttler, David [mailto:buttler1@llnl.gov]
> Sent: Monday, August 29, 2011 8:46 PM
> To: user@hbase.apache.org
> Subject: RE: Scan not working properly on composite keys
>
> The scan started at "viewer/user3" because that is the next key after "user2". If your keys have a prefix, like admin or viewer, then you need to prepend that to your start row as well. Hbase has no magic where it can intuit your intent.
>
> Dave
>
> -----Original Message-----
> From: Stuti Awasthi [mailto:stutiawasthi@hcl.com]
> Sent: Monday, August 29, 2011 12:39 AM
> To: user@hbase.apache.org
> Subject: Scan not working properly on composite keys
>
> Hi Friends,
>
> I have a query regarding scan operation on composite keys.
>
> I have table as follows :
>
> admin/user1 column=info:email, timestamp=1314594930540, value=abc
> admin/ user1 column=info:mobile, timestamp=1314594940686, value=123456
> admin/ user1 column=info:name, timestamp=1314594915091, value=def
> admin/user2 column=info:email, timestamp=1314597627865, value=def
> admin/ user2 column=info:mobile, timestamp=1314597636663, value=67890
> admin/ user2 column=info:name, timestamp=1314597614585, value=pqr
> viewer/user3 column=info:email, timestamp=1314601194373, value=pqr
> viewer/ user3 column=info:mobile, timestamp=1314601256479, value=12345
> viewer/ user3 column=info:name, timestamp=1314601207662, value=xyz
>
> I want to find all the records of "user2". For this I tried following query but it is giving me all records of user3 :
>
> hbase(main):005:0> scan 'users', {STARTROW=>'user2'}
> ROW COLUMN+CELL
> viewer/user3 column=info:email, timestamp=1314601194373, value=satyendrap@hcl.com
> viewer/ user3 column=info:mobile, timestamp=1314601256479, value=37498734893
> viewer/ user3 column=info:name, timestamp=1314601207662, value=satyendra pandey
> 1 row(s) in 0.0330 seconds
>
> I also tried various queries for this but not getting correct result. I am definitely missing something. Please help
>
> Thanks & Regards
> Stuti Awasthi
> Sr Specialist
>
>
> ________________________________
> ::DISCLAIMER::
> -----------------------------------------------------------------------------------------------------------------------
>
> The contents of this e-mail and any attachment(s) are confidential and intended for the named recipient(s) only.
> It shall not attach any liability on the originator or HCL or its affiliates. Any views or opinions presented in this email are solely those of the author and may not necessarily reflect the opinions of HCL or its affiliates.
> Any form of reproduction, dissemination, copying, disclosure, modification, distribution and / or publication of this message without the prior written consent of the author of this e-mail is strictly prohibited. If you have received this email in error please delete it and notify the sender immediately. Before opening any mail and attachments please check them for viruses and defect.
>
> -----------------------------------------------------------------------------------------------------------------------
RE: Scan not working properly on composite keys
Posted by Stuti Awasthi <st...@hcl.com>.
Hi,
Sure, I tried also by prefixing with admin, and it was working fine. I was just wondering that is there any way that we can get the information from Hbase using some part of Composite keys or wild card characters ?
Thanks
-----Original Message-----
From: Buttler, David [mailto:buttler1@llnl.gov]
Sent: Monday, August 29, 2011 8:46 PM
To: user@hbase.apache.org
Subject: RE: Scan not working properly on composite keys
The scan started at "viewer/user3" because that is the next key after "user2". If your keys have a prefix, like admin or viewer, then you need to prepend that to your start row as well. Hbase has no magic where it can intuit your intent.
Dave
-----Original Message-----
From: Stuti Awasthi [mailto:stutiawasthi@hcl.com]
Sent: Monday, August 29, 2011 12:39 AM
To: user@hbase.apache.org
Subject: Scan not working properly on composite keys
Hi Friends,
I have a query regarding scan operation on composite keys.
I have table as follows :
admin/user1 column=info:email, timestamp=1314594930540, value=abc
admin/ user1 column=info:mobile, timestamp=1314594940686, value=123456
admin/ user1 column=info:name, timestamp=1314594915091, value=def
admin/user2 column=info:email, timestamp=1314597627865, value=def
admin/ user2 column=info:mobile, timestamp=1314597636663, value=67890
admin/ user2 column=info:name, timestamp=1314597614585, value=pqr
viewer/user3 column=info:email, timestamp=1314601194373, value=pqr
viewer/ user3 column=info:mobile, timestamp=1314601256479, value=12345
viewer/ user3 column=info:name, timestamp=1314601207662, value=xyz
I want to find all the records of "user2". For this I tried following query but it is giving me all records of user3 :
hbase(main):005:0> scan 'users', {STARTROW=>'user2'}
ROW COLUMN+CELL
viewer/user3 column=info:email, timestamp=1314601194373, value=satyendrap@hcl.com
viewer/ user3 column=info:mobile, timestamp=1314601256479, value=37498734893
viewer/ user3 column=info:name, timestamp=1314601207662, value=satyendra pandey
1 row(s) in 0.0330 seconds
I also tried various queries for this but not getting correct result. I am definitely missing something. Please help
Thanks & Regards
Stuti Awasthi
Sr Specialist
________________________________
::DISCLAIMER::
-----------------------------------------------------------------------------------------------------------------------
The contents of this e-mail and any attachment(s) are confidential and intended for the named recipient(s) only.
It shall not attach any liability on the originator or HCL or its affiliates. Any views or opinions presented in this email are solely those of the author and may not necessarily reflect the opinions of HCL or its affiliates.
Any form of reproduction, dissemination, copying, disclosure, modification, distribution and / or publication of this message without the prior written consent of the author of this e-mail is strictly prohibited. If you have received this email in error please delete it and notify the sender immediately. Before opening any mail and attachments please check them for viruses and defect.
-----------------------------------------------------------------------------------------------------------------------
RE: Scan not working properly on composite keys
Posted by "Buttler, David" <bu...@llnl.gov>.
The scan started at "viewer/user3" because that is the next key after "user2". If your keys have a prefix, like admin or viewer, then you need to prepend that to your start row as well. Hbase has no magic where it can intuit your intent.
Dave
-----Original Message-----
From: Stuti Awasthi [mailto:stutiawasthi@hcl.com]
Sent: Monday, August 29, 2011 12:39 AM
To: user@hbase.apache.org
Subject: Scan not working properly on composite keys
Hi Friends,
I have a query regarding scan operation on composite keys.
I have table as follows :
admin/user1 column=info:email, timestamp=1314594930540, value=abc
admin/ user1 column=info:mobile, timestamp=1314594940686, value=123456
admin/ user1 column=info:name, timestamp=1314594915091, value=def
admin/user2 column=info:email, timestamp=1314597627865, value=def
admin/ user2 column=info:mobile, timestamp=1314597636663, value=67890
admin/ user2 column=info:name, timestamp=1314597614585, value=pqr
viewer/user3 column=info:email, timestamp=1314601194373, value=pqr
viewer/ user3 column=info:mobile, timestamp=1314601256479, value=12345
viewer/ user3 column=info:name, timestamp=1314601207662, value=xyz
I want to find all the records of "user2". For this I tried following query but it is giving me all records of user3 :
hbase(main):005:0> scan 'users', {STARTROW=>'user2'}
ROW COLUMN+CELL
viewer/user3 column=info:email, timestamp=1314601194373, value=satyendrap@hcl.com
viewer/ user3 column=info:mobile, timestamp=1314601256479, value=37498734893
viewer/ user3 column=info:name, timestamp=1314601207662, value=satyendra pandey
1 row(s) in 0.0330 seconds
I also tried various queries for this but not getting correct result. I am definitely missing something. Please help
Thanks & Regards
Stuti Awasthi
Sr Specialist
________________________________
::DISCLAIMER::
-----------------------------------------------------------------------------------------------------------------------
The contents of this e-mail and any attachment(s) are confidential and intended for the named recipient(s) only.
It shall not attach any liability on the originator or HCL or its affiliates. Any views or opinions presented in
this email are solely those of the author and may not necessarily reflect the opinions of HCL or its affiliates.
Any form of reproduction, dissemination, copying, disclosure, modification, distribution and / or publication of
this message without the prior written consent of the author of this e-mail is strictly prohibited. If you have
received this email in error please delete it and notify the sender immediately. Before opening any mail and
attachments please check them for viruses and defect.
-----------------------------------------------------------------------------------------------------------------------