You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by Andrzej <bo...@wp.pl> on 2017/09/20 10:33:06 UTC
PrefixFilter and Scanner
PrefixFilter - takes a single argument, a prefix of a row key. It
returns only those key-values present in a row that start with the
specified row prefix
Previously I thought, that if we have rows:
Addr1Hash1
Addr1Hash2
Addr1Hash3
Addr2Hash1
Addr2Hash2
Addr2Hash3
If we call
shared_ptr<Scan> scan = make_shared<Scan>("Addr2");
shared_ptr<ResultScanner> scanner = table->Scan(*scan);
that Scanner go to "Addr2Hash1"
and if we apply make_shared<Scan>(""), Scanner got to first row of table.
But is needed PrefixFilter? How it apply with Scanner?
Re: Is possible store BINARY data in row: string?
Posted by Esteban Gutierrez <es...@cloudera.com>.
Hello Andrzej,
HBase doesn't really care what encoding you use for the key and the value.
This issue is most likely happing on the C++ client or in your application.
If you can reproduce this issue systematically with the C++ client please
feel free to open an upstream JIRA for the HBase project:
https://issues.apache.org/jira/HBASE
Thanks,
esteban.
--
Cloudera, Inc.
On Wed, Sep 27, 2017 at 11:49 AM, Andrzej <bo...@wp.pl> wrote:
> I have big problem:
> I have row: address 36 bytes + hasz20 bytes (binary with zeros!) + 4 bytes
> index + 1 byte type.
> I write:
> 304d494e454478434f494e42415345303030303030303030303030303030
> 303030300000982051fd1e4ba744bbbe680e1fee14677ba1a3c3540bf7b1
> cdb606e857233e0e0000000000
>
> and I read:
> 304d494e454478434f494e42415345303030303030303030303030303030
> 303030300020203c636f310000000000000000000000000000004f494e42
> 41534530303030303030303030
>
> address=0MINEDxCOINBASE0000000000000000000 hash=0c00680000000000000000000
> 00000000159a670000000000000005109227972 xio=32767, type=3
>
>
> only address is correct!
> address end with zero byte. Strings with zeros are not correct stored?
> I store with:
>
> auto put = std::make_unique<Put>(row);
> for (int i = 0; i<qualifiers.size(); i++)
> put->AddColumn(families[i], qualifiers[i], values[i]);
> getTable(hTable)->Put(*std::move(put));
>
> and read with:
> return string: result->Row()
>
Re: Is possible store BINARY data in row: string?
Posted by Ted Yu <yu...@gmail.com>.
In simple-client, we have this code:
void ValidateResult(const Result &result, const std::string &row) {
CHECK(!result.IsEmpty());
CHECK_EQ(result.Row(), row);
Are you able to reproduce what you described by modifying simple-client
(which results in assertion failure) ?
On Wed, Sep 27, 2017 at 11:49 AM, Andrzej <bo...@wp.pl> wrote:
> I have big problem:
> I have row: address 36 bytes + hasz20 bytes (binary with zeros!) + 4 bytes
> index + 1 byte type.
> I write:
> 304d494e454478434f494e42415345303030303030303030303030303030
> 303030300000982051fd1e4ba744bbbe680e1fee14677ba1a3c3540bf7b1
> cdb606e857233e0e0000000000
>
> and I read:
> 304d494e454478434f494e42415345303030303030303030303030303030
> 303030300020203c636f310000000000000000000000000000004f494e42
> 41534530303030303030303030
>
> address=0MINEDxCOINBASE0000000000000000000 hash=0c00680000000000000000000
> 00000000159a670000000000000005109227972 xio=32767, type=3
>
>
> only address is correct!
> address end with zero byte. Strings with zeros are not correct stored?
> I store with:
>
> auto put = std::make_unique<Put>(row);
> for (int i = 0; i<qualifiers.size(); i++)
> put->AddColumn(families[i], qualifiers[i], values[i]);
> getTable(hTable)->Put(*std::move(put));
>
> and read with:
> return string: result->Row()
>
Is possible store BINARY data in row: string?
Posted by Andrzej <bo...@wp.pl>.
I have big problem:
I have row: address 36 bytes + hasz20 bytes (binary with zeros!) + 4
bytes index + 1 byte type.
I write:
304d494e454478434f494e42415345303030303030303030303030303030303030300000982051fd1e4ba744bbbe680e1fee14677ba1a3c3540bf7b1cdb606e857233e0e0000000000
and I read:
304d494e454478434f494e42415345303030303030303030303030303030303030300020203c636f310000000000000000000000000000004f494e4241534530303030303030303030
address=0MINEDxCOINBASE0000000000000000000
hash=0c0068000000000000000000000000000159a670000000000000005109227972
xio=32767, type=3
only address is correct!
address end with zero byte. Strings with zeros are not correct stored?
I store with:
auto put = std::make_unique<Put>(row);
for (int i = 0; i<qualifiers.size(); i++)
put->AddColumn(families[i], qualifiers[i], values[i]);
getTable(hTable)->Put(*std::move(put));
and read with:
return string: result->Row()
Re: Unable to connect http://localhost:16010/
Posted by Ted Yu <yu...@gmail.com>.
Java 9 support is not fully in place in hbase.
Suggest using Java 8 to unblock you.
On Sat, Sep 23, 2017 at 8:13 AM, Andrzej <bo...@wp.pl> wrote:
> W dniu 23.09.2017 o 17:00, Ted Yu pisze:
>
>> Please inspect hbase-root-master-andrzej-VirtualBox.out (h
>> base-root-master-andrzej-VirtualBox.log) to see if there was any fatal
>> error.
>>
>
> Problem can be because I use new 9-th version of Java?
> I use hadoop-2.8.1.tar.gz and hbase-1.3.1-bin.tar.gz
>
> 2017-09-23 17:11:21,750 ERROR [main] master.HMasterCommandLine: Master
> exiting
> java.lang.RuntimeException: Failed construction of Master: class
> org.apache.hadoop.hbase.master.HMasterCommandLine$LocalHMasterUnexpected
> version format: 9-internal
> at org.apache.hadoop.hbase.util.JVMClusterUtil.createMasterThre
> ad(JVMClusterUtil.java:143)
> at org.apache.hadoop.hbase.LocalHBaseCluster.addMaster(LocalHBa
> seCluster.java:220)
> at org.apache.hadoop.hbase.LocalHBaseCluster.<init>(LocalHBaseC
> luster.java:155)
> at org.apache.hadoop.hbase.master.HMasterCommandLine.startMaste
> r(HMasterCommandLine.java:222)
> at org.apache.hadoop.hbase.master.HMasterCommandLine.run(HMaste
> rCommandLine.java:137)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
> at org.apache.hadoop.hbase.util.ServerCommandLine.doMain(Server
> CommandLine.java:126)
> at org.apache.hadoop.hbase.master.HMaster.main(HMaster.java:2610)
> Caused by: java.lang.ExceptionInInitializerError
> at org.apache.hadoop.hbase.ipc.IPCUtil.<init>(IPCUtil.java:75)
> at org.apache.hadoop.hbase.ipc.RpcServer.<init>(RpcServer.java:
> 2185)
> at org.apache.hadoop.hbase.regionserver.RSRpcServices.<init>(
> RSRpcServices.java:994)
> at org.apache.hadoop.hbase.master.MasterRpcServices.<init>(
> MasterRpcServices.java:233)
> at org.apache.hadoop.hbase.master.HMaster.createRpcServices(
> HMaster.java:587)
> at org.apache.hadoop.hbase.regionserver.HRegionServer.<init>(
> HRegionServer.java:543)
> at org.apache.hadoop.hbase.master.HMaster.<init>(HMaster.java:427)
> at org.apache.hadoop.hbase.master.HMasterCommandLine$LocalHMast
> er.<init>(HMasterCommandLine.java:312)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(java.
> base@9-internal/Native Method)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance(java.
> base@9-internal/NativeConstructorAccessorImpl.java:62)
> at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(
> java.base@9-internal/DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(java.base@9-intern
> al/Constructor.java:453)
> at org.apache.hadoop.hbase.util.JVMClusterUtil.createMasterThre
> ad(JVMClusterUtil.java:139)
> ... 7 more
> Caused by: java.lang.RuntimeException: Unexpected version format:
> 9-internal
> at org.apache.hadoop.hbase.util.ClassSize.<clinit>(ClassSize.ja
> va:119)
> ... 20 more
>
>
Re: Unable to connect http://localhost:16010/
Posted by Andrzej <bo...@wp.pl>.
W dniu 23.09.2017 o 17:00, Ted Yu pisze:
> Please inspect hbase-root-master-andrzej-VirtualBox.out (h
> base-root-master-andrzej-VirtualBox.log) to see if there was any fatal
> error.
Problem can be because I use new 9-th version of Java?
I use hadoop-2.8.1.tar.gz and hbase-1.3.1-bin.tar.gz
2017-09-23 17:11:21,750 ERROR [main] master.HMasterCommandLine: Master
exiting
java.lang.RuntimeException: Failed construction of Master: class
org.apache.hadoop.hbase.master.HMasterCommandLine$LocalHMasterUnexpected
version format: 9-internal
at
org.apache.hadoop.hbase.util.JVMClusterUtil.createMasterThread(JVMClusterUtil.java:143)
at
org.apache.hadoop.hbase.LocalHBaseCluster.addMaster(LocalHBaseCluster.java:220)
at
org.apache.hadoop.hbase.LocalHBaseCluster.<init>(LocalHBaseCluster.java:155)
at
org.apache.hadoop.hbase.master.HMasterCommandLine.startMaster(HMasterCommandLine.java:222)
at
org.apache.hadoop.hbase.master.HMasterCommandLine.run(HMasterCommandLine.java:137)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at
org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:126)
at org.apache.hadoop.hbase.master.HMaster.main(HMaster.java:2610)
Caused by: java.lang.ExceptionInInitializerError
at org.apache.hadoop.hbase.ipc.IPCUtil.<init>(IPCUtil.java:75)
at org.apache.hadoop.hbase.ipc.RpcServer.<init>(RpcServer.java:2185)
at
org.apache.hadoop.hbase.regionserver.RSRpcServices.<init>(RSRpcServices.java:994)
at
org.apache.hadoop.hbase.master.MasterRpcServices.<init>(MasterRpcServices.java:233)
at
org.apache.hadoop.hbase.master.HMaster.createRpcServices(HMaster.java:587)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.<init>(HRegionServer.java:543)
at org.apache.hadoop.hbase.master.HMaster.<init>(HMaster.java:427)
at
org.apache.hadoop.hbase.master.HMasterCommandLine$LocalHMaster.<init>(HMasterCommandLine.java:312)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance0(java.base@9-internal/Native
Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(java.base@9-internal/NativeConstructorAccessorImpl.java:62)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(java.base@9-internal/DelegatingConstructorAccessorImpl.java:45)
at
java.lang.reflect.Constructor.newInstance(java.base@9-internal/Constructor.java:453)
at
org.apache.hadoop.hbase.util.JVMClusterUtil.createMasterThread(JVMClusterUtil.java:139)
... 7 more
Caused by: java.lang.RuntimeException: Unexpected version format: 9-internal
at org.apache.hadoop.hbase.util.ClassSize.<clinit>(ClassSize.java:119)
... 20 more
Re: Unable to connect http://localhost:16010/
Posted by Ted Yu <yu...@gmail.com>.
You can use 'ps' command to see if master / region server is running.
Please inspect hbase-root-master-andrzej-VirtualBox.out (h
base-root-master-andrzej-VirtualBox.log) to see if there was any fatal
error.
On Sat, Sep 23, 2017 at 7:49 AM, Andrzej <bo...@wp.pl> wrote:
> In new Linux I install Hadoop and http://localhost:50070/ is ok,
> http://localhost:8088/cluster is ok.
> Nect I try install HBase by https://www.tutorialspoint.com
> /hbase/hbase_installation.htm
>
> I do
> sudo /usr/local/HBase/bin/start-hbase.sh
> next
> I check http://localhost:16010 or 60010
> but not connected
>
> sudo /usr/local/HBase/bin/start-hbase.sh gitve me no errors:
> $ sudo /usr/local/HBase/bin/start-hbase.sh
> starting master, logging to /usr/local/HBase/bin/../logs/h
> base-root-master-andrzej-VirtualBox.out
>
Unable to connect http://localhost:16010/
Posted by Andrzej <bo...@wp.pl>.
In new Linux I install Hadoop and http://localhost:50070/ is ok,
http://localhost:8088/cluster is ok.
Nect I try install HBase by
https://www.tutorialspoint.com/hbase/hbase_installation.htm
I do
sudo /usr/local/HBase/bin/start-hbase.sh
next
I check http://localhost:16010 or 60010
but not connected
sudo /usr/local/HBase/bin/start-hbase.sh gitve me no errors:
$ sudo /usr/local/HBase/bin/start-hbase.sh
starting master, logging to
/usr/local/HBase/bin/../logs/hbase-root-master-andrzej-VirtualBox.out
Re: PrefixFilter and Scanner
Posted by Ted Yu <yu...@gmail.com>.
Please take a look at ./hbase-native-client/src/hbase/client/filter-test.cc
:
get_one.SetFilter(FilterFactory::ColumnPrefixFilter("foo_"));
get_two.SetFilter(FilterFactory::ColumnPrefixFilter("column_"));
FYI
On Wed, Sep 20, 2017 at 3:33 AM, Andrzej <bo...@wp.pl> wrote:
> PrefixFilter - takes a single argument, a prefix of a row key. It returns
> only those key-values present in a row that start with the specified row
> prefix
>
> Previously I thought, that if we have rows:
> Addr1Hash1
> Addr1Hash2
> Addr1Hash3
> Addr2Hash1
> Addr2Hash2
> Addr2Hash3
>
> If we call
> shared_ptr<Scan> scan = make_shared<Scan>("Addr2");
> shared_ptr<ResultScanner> scanner = table->Scan(*scan);
>
> that Scanner go to "Addr2Hash1"
> and if we apply make_shared<Scan>(""), Scanner got to first row of table.
> But is needed PrefixFilter? How it apply with Scanner?
>