You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by Alex Baranov <al...@gmail.com> on 2010/03/18 07:25:34 UTC
Re: Adding filter to scan at remote client causes
UnknownScannerException
To give more clarity, I'm using *not custom* filter, but "standard"
SingleColumnValueFilter. So it's not related to classpath issues.
Any help is very appreciated!
Thanks,
Alex.
On Wed, Mar 17, 2010 at 6:00 PM, Alex Baranov <al...@gmail.com>wrote:
> Hello guys,
>
> I've got a problem while adding a filter to scanner in a client app which
> runs on the remote (not the one from HBase cluster) box. The same code works
> well and scan result is fetched very quickly if I run the client on the same
> box where HBase master resides. If I comment out adding filter then the
> scanner returns results. But with filter it keeps showing me the error
> below.
>
> I'm using HBase 0.20.3 on both ends.
>
> On the mailing list I saw that problems like this can arise when using
> different versions of HBase on server and on client, but this is not the
> case. Also the error like this can show up when it takes a lot of time to
> initialize scanner (lease time by default is 1 min), but I assume this is
> also not the case since without adding filter I got results very quickly.
>
> Does anyone have an idea what is going on?
>
> - in log on remote client side:
>
> Exception in thread "main"
> org.apache.hadoop.hbase.client.RetriesExhaustedException: Trying to contact
> region server 10.210.71.80:39207 for region xxxxx,,1267450079067, row
> 'yyyyy', but failed after 10 attempts.
> Exceptions:
> java.io.IOException: Call to /10.210.71.80:39207 failed on local
> exception: java.io.EOFException
> java.io.IOException: Call to /10.210.71.80:39207 failed on local
> exception: java.io.EOFException
> java.io.IOException: Call to /10.210.71.80:39207 failed on local
> exception: java.io.EOFException
> java.io.IOException: Call to /10.210.71.80:39207 failed on local
> exception: java.io.EOFException
> java.io.IOException: Call to /10.210.71.80:39207 failed on local
> exception: java.io.EOFException
> java.io.IOException: Call to /10.210.71.80:39207 failed on local
> exception: java.io.EOFException
> java.io.IOException: Call to /10.210.71.80:39207 failed on local
> exception: java.io.EOFException
> java.io.IOException: Call to /10.210.71.80:39207 failed on local
> exception: java.io.EOFException
> java.io.IOException: Call to /10.210.71.80:39207 failed on local
> exception: java.io.EOFException
> java.io.IOException: Call to /10.210.71.80:39207 failed on local
> exception: java.io.EOFException
>
> at
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getRegionServerWithRetries(HConnectionManager.java:1002)
> at
> org.apache.hadoop.hbase.client.HTable$ClientScanner.nextScanner(HTable.java:1931)
> at
> org.apache.hadoop.hbase.client.HTable$ClientScanner.initialize(HTable.java:1851)
> at org.apache.hadoop.hbase.client.HTable.getScanner(HTable.java:372)
> at
> com.sematext.sa.client.AggregatesAccessor.getResult(AggregatesAccessor.java:74)
> at com.sematext.sa.client.ClientExample.main(ClientExample.java:41)
>
>
> - in HBase master log:
>
> 2010-03-17 12:37:45,068 ERROR
> org.apache.hadoop.hbase.regionserver.HRegionServer:
> org.apache.hadoop.hbase.UnknownScannerException: Name: -1
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.next(HRegionServer.java:1877)
> at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:657)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:915)
>
> - in HBase region server log/out: nothing
>
> Thank you in advance.
> Alex.
>
Re: Adding filter to scan at remote client causes
UnknownScannerException
Posted by Jean-Daniel Cryans <jd...@apache.org>.
Alex,
Good job on finding out your issue, which boils down to our mistake as
hbase devs. 0.20.3 included fixes for the filters and changed their
readFields/write behavior. This should have either 1) not be committed
or 2) we should have bumped the RPC version.
I ran cross-version tests before releasing 0.20.3 but I did not verify
filters. This could probably be automated with our EC2 tests we are
planning (wink Andrew Purtell), eg running all our tests with
different versions of clients and servers.
J-D
On Mon, Mar 22, 2010 at 5:17 AM, Alex Baranov <al...@gmail.com> wrote:
> Fond the mistake. It was mine, sorry.
>
> The error was caused by using HBase 0.20.2 version jar on client end (and
> 0.20.3 on the server end). Despite I put proper version in classpath in
> "java -jar" command (I copied here the java run command previosly), in my
> client app jar the manifest file had a link to hbase 0.20.2 version jar.
> (Btw, this happend because I'm using maven in development and once back I
> added dependency to the HBase 0.20.2 since it was (and *is now*) the only
> one available in public maven repos).
>
>
> Thank you for support: your check make me clean-up and re-double-check
> everything.
> Alex.
>
> On Mon, Mar 22, 2010 at 11:31 AM, Alex Baranov <al...@gmail.com>wrote:
>
>> It hangs for some time. I'm not using any contribs.
>>
>> Thanks for the help,
>> Alex.
>>
>>
>> On Fri, Mar 19, 2010 at 11:29 PM, Jean-Daniel Cryans <jd...@apache.org>wrote:
>>
>>> Alex,
>>>
>>> I tried your code from a remote machine to a pseudo-distributed setup
>>> and it worked well (on trunk, didn't have a 0.20.3 setup around). When
>>> the call fails, doesn't it return right away or it hangs for some
>>> time? Also are you using any contrib?
>>>
>>> Thx
>>>
>>> J-D
>>>
>>> On Fri, Mar 19, 2010 at 5:28 AM, Alex Baranov <al...@gmail.com>
>>> wrote:
>>> > Hello J-D,
>>> >
>>> > Thanks for helping me out!
>>> >
>>> > Here is the code that works if I run it on machine that has HBase master
>>> on
>>> > it and doesn't work on "remote" client box:
>>> >
>>> > // CODE BEGINS
>>> >
>>> > HBaseConfiguration conf = new HBaseConfiguration();
>>> > HTable hTable = new HTable(conf, "agg9");
>>> >
>>> > Scan scan = new Scan();
>>> > scan.setStartRow(Bytes.toBytes("qf|byday_bytype_|14656__|"));
>>> >
>>> > FilterList filters = new
>>> FilterList(FilterList.Operator.MUST_PASS_ALL);
>>> >
>>> > SingleColumnValueFilter filter = new
>>> > SingleColumnValueFilter(Bytes.toBytes("agg"), Bytes.toBytes("count"),
>>> > CompareFilter.CompareOp.GREATER, Bytes.toBytes(35));
>>> > filters.addFilter(filter);
>>> >
>>> > InclusiveStopFilter stopFilter = new
>>> > InclusiveStopFilter(Bytes.toBytes("qf|byday_|14739_|"));
>>> > filters.addFilter(stopFilter);
>>> > scan.setFilter(filters);
>>> >
>>> > ResultScanner rs = hTable.getScanner(scan);
>>> > Result next = rs.next();
>>> > int readCount = 0;
>>> > while (next != null && readCount < 40) {
>>> > System.out.println("Row key: " + Bytes.toString(next.getRow()));
>>> > System.out.println("count: " +
>>> > Bytes.toInt(next.getValue(Bytes.toBytes("agg"),
>>> Bytes.toBytes("count"))));
>>> > next = rs.next();
>>> > readCount++;
>>> > }
>>> >
>>> > // CODE ENDS
>>> >
>>> > If I comment the line
>>> > filters.addFilter(filter);
>>> > then the code works on remote client box as well.
>>> >
>>> >
>>> > Client fails with the exception I provided previously. The bigger master
>>> log
>>> > (this is the log till the end and started when I ran the client code):
>>> >
>>> > 2010-03-19 12:15:02,098 INFO
>>> org.apache.hadoop.hbase.master.ServerManager: 1
>>> > region servers, 0 dead, average load 27.0
>>> > 2010-03-19 12:15:41,369 INFO org.apache.hadoop.hbase.master.BaseScanner:
>>> > RegionManager.metaScanner scanning meta region {server:
>>> 10.210.71.80:39207,
>>> > regionname: .META.,,1, startKey: <>}
>>> > 2010-03-19 12:15:41,398 INFO org.apache.hadoop.hbase.master.BaseScanner:
>>> > RegionManager.metaScanner scan of 25 row(s) of meta region {server:
>>> > 10.210.71.80:39207, regionname: .META.,,1, startKey: <>} complete
>>> > 2010-03-19 12:15:41,398 INFO org.apache.hadoop.hbase.master.BaseScanner:
>>> All
>>> > 1 .META. region(s) scanned
>>> > 2010-03-19 12:15:41,548 INFO org.apache.hadoop.hbase.master.BaseScanner:
>>> > RegionManager.rootScanner scanning meta region {server:
>>> 10.210.71.80:39207,
>>> > regionname: -ROOT-,,0, startKey: <>}
>>> > 2010-03-19 12:15:41,549 INFO org.apache.hadoop.hbase.master.BaseScanner:
>>> > RegionManager.rootScanner scan of 1 row(s) of meta region {server:
>>> > 10.210.71.80:39207, regionname: -ROOT-,,0, startKey: <>} complete
>>> > 2010-03-19 12:15:45,398 DEBUG
>>> > org.apache.hadoop.hbase.io.hfile.LruBlockCache: Cache Stats: Sizes:
>>> > Total=43.474342MB (45586152), Free=156.21317MB (163801368),
>>> Max=199.6875MB
>>> > (209387520), Counts: Blocks=684, Access=90114, Hit=80498, Miss=9616,
>>> > Evictions=0, Evicted=0, Ratios: Hit Ratio=89.32906985282898%, Miss
>>> > Ratio=10.670927911996841%, Evicted/Run=NaN
>>> > 2010-03-19 12:16:02,108 INFO
>>> org.apache.hadoop.hbase.master.ServerManager: 1
>>> > region servers, 0 dead, average load 27.0
>>> > 2010-03-19 12:16:41,378 INFO org.apache.hadoop.hbase.master.BaseScanner:
>>> > RegionManager.metaScanner scanning meta region {server:
>>> 10.210.71.80:39207,
>>> > regionname: .META.,,1, startKey: <>}
>>> > 2010-03-19 12:16:41,461 INFO org.apache.hadoop.hbase.master.BaseScanner:
>>> > RegionManager.metaScanner scan of 25 row(s) of meta region {server:
>>> > 10.210.71.80:39207, regionname: .META.,,1, startKey: <>} complete
>>> > 2010-03-19 12:16:41,461 INFO org.apache.hadoop.hbase.master.BaseScanner:
>>> All
>>> > 1 .META. region(s) scanned
>>> > 2010-03-19 12:16:41,558 INFO org.apache.hadoop.hbase.master.BaseScanner:
>>> > RegionManager.rootScanner scanning meta region {server:
>>> 10.210.71.80:39207,
>>> > regionname: -ROOT-,,0, startKey: <>}
>>> > 2010-03-19 12:16:41,559 INFO org.apache.hadoop.hbase.master.BaseScanner:
>>> > RegionManager.rootScanner scan of 1 row(s) of meta region {server:
>>> > 10.210.71.80:39207, regionname: -ROOT-,,0, startKey: <>} complete
>>> > 2010-03-19 12:16:45,398 DEBUG
>>> > org.apache.hadoop.hbase.io.hfile.LruBlockCache: Cache Stats: Sizes:
>>> > Total=43.474342MB (45586152), Free=156.21317MB (163801368),
>>> Max=199.6875MB
>>> > (209387520), Counts: Blocks=684, Access=90116, Hit=80500, Miss=9616,
>>> > Evictions=0, Evicted=0, Ratios: Hit Ratio=89.32930827140808%, Miss
>>> > Ratio=10.67069098353386%, Evicted/Run=NaN
>>> > 2010-03-19 12:16:59,872 ERROR
>>> > org.apache.hadoop.hbase.regionserver.HRegionServer:
>>> > org.apache.hadoop.hbase.UnknownScannerException: Name: -1
>>> > at
>>> >
>>> org.apache.hadoop.hbase.regionserver.HRegionServer.next(HRegionServer.java:1877)
>>> > at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source)
>>> > at
>>> >
>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>> > at java.lang.reflect.Method.invoke(Method.java:597)
>>> > at
>>> > org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:657)
>>> > at
>>> >
>>> org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:915)
>>> > 2010-03-19 12:16:59,874 INFO org.apache.hadoop.ipc.HBaseServer: IPC
>>> Server
>>> > handler 16 on 39207, call next(-1, 1) from 10.241.27.112:38002: error:
>>> > org.apache.hadoop.hbase.UnknownScannerException: Name: -1
>>> > org.apache.hadoop.hbase.UnknownScannerException: Name: -1
>>> > at
>>> >
>>> org.apache.hadoop.hbase.regionserver.HRegionServer.next(HRegionServer.java:1877)
>>> > at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source)
>>> > at
>>> >
>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>> > at java.lang.reflect.Method.invoke(Method.java:597)
>>> > at
>>> > org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:657)
>>> > at
>>> >
>>> org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:915)
>>> > 2010-03-19 12:17:02,118 INFO
>>> org.apache.hadoop.hbase.master.ServerManager: 1
>>> > region servers, 0 dead, average load 27.0
>>> > 2010-03-19 12:17:41,388 INFO org.apache.hadoop.hbase.master.BaseScanner:
>>> > RegionManager.metaScanner scanning meta region {server:
>>> 10.210.71.80:39207,
>>> > regionname: .META.,,1, startKey: <>}
>>> > 2010-03-19 12:17:41,471 INFO org.apache.hadoop.hbase.master.BaseScanner:
>>> > RegionManager.metaScanner scan of 25 row(s) of meta region {server:
>>> > 10.210.71.80:39207, regionname: .META.,,1, startKey: <>} complete
>>> > 2010-03-19 12:17:41,471 INFO org.apache.hadoop.hbase.master.BaseScanner:
>>> All
>>> > 1 .META. region(s) scanned
>>> > 2010-03-19 12:17:41,568 INFO org.apache.hadoop.hbase.master.BaseScanner:
>>> > RegionManager.rootScanner scanning meta region {server:
>>> 10.210.71.80:39207,
>>> > regionname: -ROOT-,,0, startKey: <>}
>>> > 2010-03-19 12:17:41,569 INFO org.apache.hadoop.hbase.master.BaseScanner:
>>> > RegionManager.rootScanner scan of 1 row(s) of meta region {server:
>>> > 10.210.71.80:39207, regionname: -ROOT-,,0, startKey: <>} complete
>>> > 2010-03-19 12:17:45,398 DEBUG
>>> > org.apache.hadoop.hbase.io.hfile.LruBlockCache: Cache Stats: Sizes:
>>> > Total=43.474342MB (45586152), Free=156.21317MB (163801368),
>>> Max=199.6875MB
>>> > (209387520), Counts: Blocks=684, Access=90118, Hit=80502, Miss=9616,
>>> > Evictions=0, Evicted=0, Ratios: Hit Ratio=89.32954668998718%, Miss
>>> > Ratio=10.670454055070877%, Evicted/Run=NaN
>>> > 2010-03-19 12:18:02,128 INFO
>>> org.apache.hadoop.hbase.master.ServerManager: 1
>>> > region servers, 0 dead, average load 27.0
>>> >
>>> > There is nothing in region log:
>>> >
>>> > ubuntu@xxx:/mnt/hbase-0.20.3/logs$ ls -l | grep region
>>> > -rw-r--r-- 1 ubuntu ubuntu 2032 2010-02-24 05:51
>>> > hbase-ubuntu-regionserver-domU-12-31-39-09-40-A2.log
>>> > -rw-r--r-- 1 ubuntu ubuntu 0 2010-02-24 05:51
>>> > hbase-ubuntu-regionserver-domU-12-31-39-09-40-A2.out
>>> > -rw-r--r-- 1 ubuntu ubuntu 0 2010-02-04 16:04
>>> > hbase-ubuntu-regionserver-domU-12-31-39-09-40-A2.out.1
>>> > -rw-r--r-- 1 ubuntu ubuntu 0 2010-02-04 15:03
>>> > hbase-ubuntu-regionserver-domU-12-31-39-09-40-A2.out.2
>>> > -rw-r--r-- 1 ubuntu ubuntu 0 2010-02-04 14:37
>>> > hbase-ubuntu-regionserver-domU-12-31-39-09-40-A2.out.3
>>> > -rw-r--r-- 1 ubuntu ubuntu 0 2010-02-04 08:15
>>> > hbase-ubuntu-regionserver-domU-12-31-39-09-40-A2.out.4
>>> > -rw-r--r-- 1 ubuntu ubuntu 0 2010-02-04 07:56
>>> > hbase-ubuntu-regionserver-domU-12-31-39-09-40-A2.out.5
>>> >
>>> > Also nothing in .out logs:
>>> >
>>> > ubuntu@xxx:/mnt/hbase-0.20.3/logs$ ls -l | grep \.out
>>> > -rw-r--r-- 1 ubuntu ubuntu 324 2010-02-24 05:51
>>> > hbase-ubuntu-master-domU-12-31-39-09-40-A2.out
>>> > -rw-r--r-- 1 ubuntu ubuntu 0 2010-02-24 05:50
>>> > hbase-ubuntu-master-domU-12-31-39-09-40-A2.out.1
>>> > -rw-r--r-- 1 ubuntu ubuntu 0 2010-02-04 16:01
>>> > hbase-ubuntu-master-domU-12-31-39-09-40-A2.out.2
>>> > -rw-r--r-- 1 ubuntu ubuntu 0 2010-02-04 14:56
>>> > hbase-ubuntu-master-domU-12-31-39-09-40-A2.out.3
>>> > -rw-r--r-- 1 ubuntu ubuntu 0 2010-02-04 14:30
>>> > hbase-ubuntu-master-domU-12-31-39-09-40-A2.out.4
>>> > -rw-r--r-- 1 ubuntu ubuntu 0 2010-02-04 08:07
>>> > hbase-ubuntu-master-domU-12-31-39-09-40-A2.out.5
>>> > -rw-r--r-- 1 ubuntu ubuntu 0 2010-02-24 05:51
>>> > hbase-ubuntu-regionserver-domU-12-31-39-09-40-A2.out
>>> > -rw-r--r-- 1 ubuntu ubuntu 0 2010-02-04 16:04
>>> > hbase-ubuntu-regionserver-domU-12-31-39-09-40-A2.out.1
>>> > -rw-r--r-- 1 ubuntu ubuntu 0 2010-02-04 15:03
>>> > hbase-ubuntu-regionserver-domU-12-31-39-09-40-A2.out.2
>>> > -rw-r--r-- 1 ubuntu ubuntu 0 2010-02-04 14:37
>>> > hbase-ubuntu-regionserver-domU-12-31-39-09-40-A2.out.3
>>> > -rw-r--r-- 1 ubuntu ubuntu 0 2010-02-04 08:15
>>> > hbase-ubuntu-regionserver-domU-12-31-39-09-40-A2.out.4
>>> > -rw-r--r-- 1 ubuntu ubuntu 0 2010-02-04 07:56
>>> > hbase-ubuntu-regionserver-domU-12-31-39-09-40-A2.out.5
>>> > -rw-r--r-- 1 ubuntu ubuntu 0 2010-02-24 05:51
>>> > hbase-ubuntu-zookeeper-domU-12-31-39-09-40-A2.out
>>> > -rw-r--r-- 1 ubuntu ubuntu 0 2010-02-04 16:04
>>> > hbase-ubuntu-zookeeper-domU-12-31-39-09-40-A2.out.1
>>> > -rw-r--r-- 1 ubuntu ubuntu 0 2010-02-04 15:03
>>> > hbase-ubuntu-zookeeper-domU-12-31-39-09-40-A2.out.2
>>> > -rw-r--r-- 1 ubuntu ubuntu 0 2010-02-04 14:37
>>> > hbase-ubuntu-zookeeper-domU-12-31-39-09-40-A2.out.3
>>> > -rw-r--r-- 1 ubuntu ubuntu 0 2010-02-04 08:15
>>> > hbase-ubuntu-zookeeper-domU-12-31-39-09-40-A2.out.4
>>> > -rw-r--r-- 1 ubuntu ubuntu 0 2010-02-04 07:56
>>> > hbase-ubuntu-zookeeper-domU-12-31-39-09-40-A2.out.5
>>> >
>>> > Is there anything else I can provide you?
>>> >
>>> > The command I'm using to run the client is the following (you can see
>>> that
>>> > 0.20.3 version of HBase is used and also other versions you might be
>>> > interested in):
>>> > java -cp
>>> >
>>> commons-cli-1.2.jar:commons-logging-1.1.1.jar:hadoop-0.20.1-core.jar:hbase-0.20.3.jar:hbase-0.20.3-test.jar:log4j-1.2.15.jar:test-1.0-SNAPSHOT.jar:zookeeper-3.2.2.jar
>>> > com.foo.bar.client.ClientExample
>>> >
>>> > Thank you for your help!
>>> > Alex
>>> >
>>> > On Thu, Mar 18, 2010 at 7:13 PM, Jean-Daniel Cryans <
>>> jdcryans@apache.org>wrote:
>>> >
>>> >> Alex,
>>> >>
>>> >> Is there anything else in the region server logs before that like
>>> >> lease expirations? Can we see a much bigger log? Also is there
>>> >> anything in the .out file? Can you post a snippet of the code you are
>>> >> using?
>>> >>
>>> >> Thx
>>> >>
>>> >> J-D
>>> >>
>>> >> On Wed, Mar 17, 2010 at 11:25 PM, Alex Baranov <
>>> alex.baranov.v@gmail.com>
>>> >> wrote:
>>> >> > To give more clarity, I'm using *not custom* filter, but "standard"
>>> >> > SingleColumnValueFilter. So it's not related to classpath issues.
>>> >> >
>>> >> > Any help is very appreciated!
>>> >> >
>>> >> > Thanks,
>>> >> > Alex.
>>> >> >
>>> >> > On Wed, Mar 17, 2010 at 6:00 PM, Alex Baranov <
>>> alex.baranov.v@gmail.com
>>> >> >wrote:
>>> >> >
>>> >> >> Hello guys,
>>> >> >>
>>> >> >> I've got a problem while adding a filter to scanner in a client app
>>> >> which
>>> >> >> runs on the remote (not the one from HBase cluster) box. The same
>>> code
>>> >> works
>>> >> >> well and scan result is fetched very quickly if I run the client on
>>> the
>>> >> same
>>> >> >> box where HBase master resides. If I comment out adding filter then
>>> the
>>> >> >> scanner returns results. But with filter it keeps showing me the
>>> error
>>> >> >> below.
>>> >> >>
>>> >> >> I'm using HBase 0.20.3 on both ends.
>>> >> >>
>>> >> >> On the mailing list I saw that problems like this can arise when
>>> using
>>> >> >> different versions of HBase on server and on client, but this is not
>>> the
>>> >> >> case. Also the error like this can show up when it takes a lot of
>>> time
>>> >> to
>>> >> >> initialize scanner (lease time by default is 1 min), but I assume
>>> this
>>> >> is
>>> >> >> also not the case since without adding filter I got results very
>>> >> quickly.
>>> >> >>
>>> >> >> Does anyone have an idea what is going on?
>>> >> >>
>>> >> >> - in log on remote client side:
>>> >> >>
>>> >> >> Exception in thread "main"
>>> >> >> org.apache.hadoop.hbase.client.RetriesExhaustedException: Trying to
>>> >> contact
>>> >> >> region server 10.210.71.80:39207 for region xxxxx,,1267450079067,
>>> row
>>> >> >> 'yyyyy', but failed after 10 attempts.
>>> >> >> Exceptions:
>>> >> >> java.io.IOException: Call to /10.210.71.80:39207 failed on local
>>> >> >> exception: java.io.EOFException
>>> >> >> java.io.IOException: Call to /10.210.71.80:39207 failed on local
>>> >> >> exception: java.io.EOFException
>>> >> >> java.io.IOException: Call to /10.210.71.80:39207 failed on local
>>> >> >> exception: java.io.EOFException
>>> >> >> java.io.IOException: Call to /10.210.71.80:39207 failed on local
>>> >> >> exception: java.io.EOFException
>>> >> >> java.io.IOException: Call to /10.210.71.80:39207 failed on local
>>> >> >> exception: java.io.EOFException
>>> >> >> java.io.IOException: Call to /10.210.71.80:39207 failed on local
>>> >> >> exception: java.io.EOFException
>>> >> >> java.io.IOException: Call to /10.210.71.80:39207 failed on local
>>> >> >> exception: java.io.EOFException
>>> >> >> java.io.IOException: Call to /10.210.71.80:39207 failed on local
>>> >> >> exception: java.io.EOFException
>>> >> >> java.io.IOException: Call to /10.210.71.80:39207 failed on local
>>> >> >> exception: java.io.EOFException
>>> >> >> java.io.IOException: Call to /10.210.71.80:39207 failed on local
>>> >> >> exception: java.io.EOFException
>>> >> >>
>>> >> >> at
>>> >> >>
>>> >>
>>> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getRegionServerWithRetries(HConnectionManager.java:1002)
>>> >> >> at
>>> >> >>
>>> >>
>>> org.apache.hadoop.hbase.client.HTable$ClientScanner.nextScanner(HTable.java:1931)
>>> >> >> at
>>> >> >>
>>> >>
>>> org.apache.hadoop.hbase.client.HTable$ClientScanner.initialize(HTable.java:1851)
>>> >> >> at
>>> org.apache.hadoop.hbase.client.HTable.getScanner(HTable.java:372)
>>> >> >> at
>>> >> >>
>>> >>
>>> com.sematext.sa.client.AggregatesAccessor.getResult(AggregatesAccessor.java:74)
>>> >> >> at
>>> com.sematext.sa.client.ClientExample.main(ClientExample.java:41)
>>> >> >>
>>> >> >>
>>> >> >> - in HBase master log:
>>> >> >>
>>> >> >> 2010-03-17 12:37:45,068 ERROR
>>> >> >> org.apache.hadoop.hbase.regionserver.HRegionServer:
>>> >> >> org.apache.hadoop.hbase.UnknownScannerException: Name: -1
>>> >> >> at
>>> >> >>
>>> >>
>>> org.apache.hadoop.hbase.regionserver.HRegionServer.next(HRegionServer.java:1877)
>>> >> >> at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown
>>> Source)
>>> >> >> at
>>> >> >>
>>> >>
>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>> >> >> at java.lang.reflect.Method.invoke(Method.java:597)
>>> >> >> at
>>> >> >> org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:657)
>>> >> >> at
>>> >> >>
>>> >>
>>> org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:915)
>>> >> >>
>>> >> >> - in HBase region server log/out: nothing
>>> >> >>
>>> >> >> Thank you in advance.
>>> >> >> Alex.
>>> >> >>
>>> >> >
>>> >>
>>> >
>>>
>>
>>
>
Re: Adding filter to scan at remote client causes
UnknownScannerException
Posted by Alex Baranov <al...@gmail.com>.
Fond the mistake. It was mine, sorry.
The error was caused by using HBase 0.20.2 version jar on client end (and
0.20.3 on the server end). Despite I put proper version in classpath in
"java -jar" command (I copied here the java run command previosly), in my
client app jar the manifest file had a link to hbase 0.20.2 version jar.
(Btw, this happend because I'm using maven in development and once back I
added dependency to the HBase 0.20.2 since it was (and *is now*) the only
one available in public maven repos).
Thank you for support: your check make me clean-up and re-double-check
everything.
Alex.
On Mon, Mar 22, 2010 at 11:31 AM, Alex Baranov <al...@gmail.com>wrote:
> It hangs for some time. I'm not using any contribs.
>
> Thanks for the help,
> Alex.
>
>
> On Fri, Mar 19, 2010 at 11:29 PM, Jean-Daniel Cryans <jd...@apache.org>wrote:
>
>> Alex,
>>
>> I tried your code from a remote machine to a pseudo-distributed setup
>> and it worked well (on trunk, didn't have a 0.20.3 setup around). When
>> the call fails, doesn't it return right away or it hangs for some
>> time? Also are you using any contrib?
>>
>> Thx
>>
>> J-D
>>
>> On Fri, Mar 19, 2010 at 5:28 AM, Alex Baranov <al...@gmail.com>
>> wrote:
>> > Hello J-D,
>> >
>> > Thanks for helping me out!
>> >
>> > Here is the code that works if I run it on machine that has HBase master
>> on
>> > it and doesn't work on "remote" client box:
>> >
>> > // CODE BEGINS
>> >
>> > HBaseConfiguration conf = new HBaseConfiguration();
>> > HTable hTable = new HTable(conf, "agg9");
>> >
>> > Scan scan = new Scan();
>> > scan.setStartRow(Bytes.toBytes("qf|byday_bytype_|14656__|"));
>> >
>> > FilterList filters = new
>> FilterList(FilterList.Operator.MUST_PASS_ALL);
>> >
>> > SingleColumnValueFilter filter = new
>> > SingleColumnValueFilter(Bytes.toBytes("agg"), Bytes.toBytes("count"),
>> > CompareFilter.CompareOp.GREATER, Bytes.toBytes(35));
>> > filters.addFilter(filter);
>> >
>> > InclusiveStopFilter stopFilter = new
>> > InclusiveStopFilter(Bytes.toBytes("qf|byday_|14739_|"));
>> > filters.addFilter(stopFilter);
>> > scan.setFilter(filters);
>> >
>> > ResultScanner rs = hTable.getScanner(scan);
>> > Result next = rs.next();
>> > int readCount = 0;
>> > while (next != null && readCount < 40) {
>> > System.out.println("Row key: " + Bytes.toString(next.getRow()));
>> > System.out.println("count: " +
>> > Bytes.toInt(next.getValue(Bytes.toBytes("agg"),
>> Bytes.toBytes("count"))));
>> > next = rs.next();
>> > readCount++;
>> > }
>> >
>> > // CODE ENDS
>> >
>> > If I comment the line
>> > filters.addFilter(filter);
>> > then the code works on remote client box as well.
>> >
>> >
>> > Client fails with the exception I provided previously. The bigger master
>> log
>> > (this is the log till the end and started when I ran the client code):
>> >
>> > 2010-03-19 12:15:02,098 INFO
>> org.apache.hadoop.hbase.master.ServerManager: 1
>> > region servers, 0 dead, average load 27.0
>> > 2010-03-19 12:15:41,369 INFO org.apache.hadoop.hbase.master.BaseScanner:
>> > RegionManager.metaScanner scanning meta region {server:
>> 10.210.71.80:39207,
>> > regionname: .META.,,1, startKey: <>}
>> > 2010-03-19 12:15:41,398 INFO org.apache.hadoop.hbase.master.BaseScanner:
>> > RegionManager.metaScanner scan of 25 row(s) of meta region {server:
>> > 10.210.71.80:39207, regionname: .META.,,1, startKey: <>} complete
>> > 2010-03-19 12:15:41,398 INFO org.apache.hadoop.hbase.master.BaseScanner:
>> All
>> > 1 .META. region(s) scanned
>> > 2010-03-19 12:15:41,548 INFO org.apache.hadoop.hbase.master.BaseScanner:
>> > RegionManager.rootScanner scanning meta region {server:
>> 10.210.71.80:39207,
>> > regionname: -ROOT-,,0, startKey: <>}
>> > 2010-03-19 12:15:41,549 INFO org.apache.hadoop.hbase.master.BaseScanner:
>> > RegionManager.rootScanner scan of 1 row(s) of meta region {server:
>> > 10.210.71.80:39207, regionname: -ROOT-,,0, startKey: <>} complete
>> > 2010-03-19 12:15:45,398 DEBUG
>> > org.apache.hadoop.hbase.io.hfile.LruBlockCache: Cache Stats: Sizes:
>> > Total=43.474342MB (45586152), Free=156.21317MB (163801368),
>> Max=199.6875MB
>> > (209387520), Counts: Blocks=684, Access=90114, Hit=80498, Miss=9616,
>> > Evictions=0, Evicted=0, Ratios: Hit Ratio=89.32906985282898%, Miss
>> > Ratio=10.670927911996841%, Evicted/Run=NaN
>> > 2010-03-19 12:16:02,108 INFO
>> org.apache.hadoop.hbase.master.ServerManager: 1
>> > region servers, 0 dead, average load 27.0
>> > 2010-03-19 12:16:41,378 INFO org.apache.hadoop.hbase.master.BaseScanner:
>> > RegionManager.metaScanner scanning meta region {server:
>> 10.210.71.80:39207,
>> > regionname: .META.,,1, startKey: <>}
>> > 2010-03-19 12:16:41,461 INFO org.apache.hadoop.hbase.master.BaseScanner:
>> > RegionManager.metaScanner scan of 25 row(s) of meta region {server:
>> > 10.210.71.80:39207, regionname: .META.,,1, startKey: <>} complete
>> > 2010-03-19 12:16:41,461 INFO org.apache.hadoop.hbase.master.BaseScanner:
>> All
>> > 1 .META. region(s) scanned
>> > 2010-03-19 12:16:41,558 INFO org.apache.hadoop.hbase.master.BaseScanner:
>> > RegionManager.rootScanner scanning meta region {server:
>> 10.210.71.80:39207,
>> > regionname: -ROOT-,,0, startKey: <>}
>> > 2010-03-19 12:16:41,559 INFO org.apache.hadoop.hbase.master.BaseScanner:
>> > RegionManager.rootScanner scan of 1 row(s) of meta region {server:
>> > 10.210.71.80:39207, regionname: -ROOT-,,0, startKey: <>} complete
>> > 2010-03-19 12:16:45,398 DEBUG
>> > org.apache.hadoop.hbase.io.hfile.LruBlockCache: Cache Stats: Sizes:
>> > Total=43.474342MB (45586152), Free=156.21317MB (163801368),
>> Max=199.6875MB
>> > (209387520), Counts: Blocks=684, Access=90116, Hit=80500, Miss=9616,
>> > Evictions=0, Evicted=0, Ratios: Hit Ratio=89.32930827140808%, Miss
>> > Ratio=10.67069098353386%, Evicted/Run=NaN
>> > 2010-03-19 12:16:59,872 ERROR
>> > org.apache.hadoop.hbase.regionserver.HRegionServer:
>> > org.apache.hadoop.hbase.UnknownScannerException: Name: -1
>> > at
>> >
>> org.apache.hadoop.hbase.regionserver.HRegionServer.next(HRegionServer.java:1877)
>> > at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source)
>> > at
>> >
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>> > at java.lang.reflect.Method.invoke(Method.java:597)
>> > at
>> > org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:657)
>> > at
>> >
>> org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:915)
>> > 2010-03-19 12:16:59,874 INFO org.apache.hadoop.ipc.HBaseServer: IPC
>> Server
>> > handler 16 on 39207, call next(-1, 1) from 10.241.27.112:38002: error:
>> > org.apache.hadoop.hbase.UnknownScannerException: Name: -1
>> > org.apache.hadoop.hbase.UnknownScannerException: Name: -1
>> > at
>> >
>> org.apache.hadoop.hbase.regionserver.HRegionServer.next(HRegionServer.java:1877)
>> > at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source)
>> > at
>> >
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>> > at java.lang.reflect.Method.invoke(Method.java:597)
>> > at
>> > org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:657)
>> > at
>> >
>> org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:915)
>> > 2010-03-19 12:17:02,118 INFO
>> org.apache.hadoop.hbase.master.ServerManager: 1
>> > region servers, 0 dead, average load 27.0
>> > 2010-03-19 12:17:41,388 INFO org.apache.hadoop.hbase.master.BaseScanner:
>> > RegionManager.metaScanner scanning meta region {server:
>> 10.210.71.80:39207,
>> > regionname: .META.,,1, startKey: <>}
>> > 2010-03-19 12:17:41,471 INFO org.apache.hadoop.hbase.master.BaseScanner:
>> > RegionManager.metaScanner scan of 25 row(s) of meta region {server:
>> > 10.210.71.80:39207, regionname: .META.,,1, startKey: <>} complete
>> > 2010-03-19 12:17:41,471 INFO org.apache.hadoop.hbase.master.BaseScanner:
>> All
>> > 1 .META. region(s) scanned
>> > 2010-03-19 12:17:41,568 INFO org.apache.hadoop.hbase.master.BaseScanner:
>> > RegionManager.rootScanner scanning meta region {server:
>> 10.210.71.80:39207,
>> > regionname: -ROOT-,,0, startKey: <>}
>> > 2010-03-19 12:17:41,569 INFO org.apache.hadoop.hbase.master.BaseScanner:
>> > RegionManager.rootScanner scan of 1 row(s) of meta region {server:
>> > 10.210.71.80:39207, regionname: -ROOT-,,0, startKey: <>} complete
>> > 2010-03-19 12:17:45,398 DEBUG
>> > org.apache.hadoop.hbase.io.hfile.LruBlockCache: Cache Stats: Sizes:
>> > Total=43.474342MB (45586152), Free=156.21317MB (163801368),
>> Max=199.6875MB
>> > (209387520), Counts: Blocks=684, Access=90118, Hit=80502, Miss=9616,
>> > Evictions=0, Evicted=0, Ratios: Hit Ratio=89.32954668998718%, Miss
>> > Ratio=10.670454055070877%, Evicted/Run=NaN
>> > 2010-03-19 12:18:02,128 INFO
>> org.apache.hadoop.hbase.master.ServerManager: 1
>> > region servers, 0 dead, average load 27.0
>> >
>> > There is nothing in region log:
>> >
>> > ubuntu@xxx:/mnt/hbase-0.20.3/logs$ ls -l | grep region
>> > -rw-r--r-- 1 ubuntu ubuntu 2032 2010-02-24 05:51
>> > hbase-ubuntu-regionserver-domU-12-31-39-09-40-A2.log
>> > -rw-r--r-- 1 ubuntu ubuntu 0 2010-02-24 05:51
>> > hbase-ubuntu-regionserver-domU-12-31-39-09-40-A2.out
>> > -rw-r--r-- 1 ubuntu ubuntu 0 2010-02-04 16:04
>> > hbase-ubuntu-regionserver-domU-12-31-39-09-40-A2.out.1
>> > -rw-r--r-- 1 ubuntu ubuntu 0 2010-02-04 15:03
>> > hbase-ubuntu-regionserver-domU-12-31-39-09-40-A2.out.2
>> > -rw-r--r-- 1 ubuntu ubuntu 0 2010-02-04 14:37
>> > hbase-ubuntu-regionserver-domU-12-31-39-09-40-A2.out.3
>> > -rw-r--r-- 1 ubuntu ubuntu 0 2010-02-04 08:15
>> > hbase-ubuntu-regionserver-domU-12-31-39-09-40-A2.out.4
>> > -rw-r--r-- 1 ubuntu ubuntu 0 2010-02-04 07:56
>> > hbase-ubuntu-regionserver-domU-12-31-39-09-40-A2.out.5
>> >
>> > Also nothing in .out logs:
>> >
>> > ubuntu@xxx:/mnt/hbase-0.20.3/logs$ ls -l | grep \.out
>> > -rw-r--r-- 1 ubuntu ubuntu 324 2010-02-24 05:51
>> > hbase-ubuntu-master-domU-12-31-39-09-40-A2.out
>> > -rw-r--r-- 1 ubuntu ubuntu 0 2010-02-24 05:50
>> > hbase-ubuntu-master-domU-12-31-39-09-40-A2.out.1
>> > -rw-r--r-- 1 ubuntu ubuntu 0 2010-02-04 16:01
>> > hbase-ubuntu-master-domU-12-31-39-09-40-A2.out.2
>> > -rw-r--r-- 1 ubuntu ubuntu 0 2010-02-04 14:56
>> > hbase-ubuntu-master-domU-12-31-39-09-40-A2.out.3
>> > -rw-r--r-- 1 ubuntu ubuntu 0 2010-02-04 14:30
>> > hbase-ubuntu-master-domU-12-31-39-09-40-A2.out.4
>> > -rw-r--r-- 1 ubuntu ubuntu 0 2010-02-04 08:07
>> > hbase-ubuntu-master-domU-12-31-39-09-40-A2.out.5
>> > -rw-r--r-- 1 ubuntu ubuntu 0 2010-02-24 05:51
>> > hbase-ubuntu-regionserver-domU-12-31-39-09-40-A2.out
>> > -rw-r--r-- 1 ubuntu ubuntu 0 2010-02-04 16:04
>> > hbase-ubuntu-regionserver-domU-12-31-39-09-40-A2.out.1
>> > -rw-r--r-- 1 ubuntu ubuntu 0 2010-02-04 15:03
>> > hbase-ubuntu-regionserver-domU-12-31-39-09-40-A2.out.2
>> > -rw-r--r-- 1 ubuntu ubuntu 0 2010-02-04 14:37
>> > hbase-ubuntu-regionserver-domU-12-31-39-09-40-A2.out.3
>> > -rw-r--r-- 1 ubuntu ubuntu 0 2010-02-04 08:15
>> > hbase-ubuntu-regionserver-domU-12-31-39-09-40-A2.out.4
>> > -rw-r--r-- 1 ubuntu ubuntu 0 2010-02-04 07:56
>> > hbase-ubuntu-regionserver-domU-12-31-39-09-40-A2.out.5
>> > -rw-r--r-- 1 ubuntu ubuntu 0 2010-02-24 05:51
>> > hbase-ubuntu-zookeeper-domU-12-31-39-09-40-A2.out
>> > -rw-r--r-- 1 ubuntu ubuntu 0 2010-02-04 16:04
>> > hbase-ubuntu-zookeeper-domU-12-31-39-09-40-A2.out.1
>> > -rw-r--r-- 1 ubuntu ubuntu 0 2010-02-04 15:03
>> > hbase-ubuntu-zookeeper-domU-12-31-39-09-40-A2.out.2
>> > -rw-r--r-- 1 ubuntu ubuntu 0 2010-02-04 14:37
>> > hbase-ubuntu-zookeeper-domU-12-31-39-09-40-A2.out.3
>> > -rw-r--r-- 1 ubuntu ubuntu 0 2010-02-04 08:15
>> > hbase-ubuntu-zookeeper-domU-12-31-39-09-40-A2.out.4
>> > -rw-r--r-- 1 ubuntu ubuntu 0 2010-02-04 07:56
>> > hbase-ubuntu-zookeeper-domU-12-31-39-09-40-A2.out.5
>> >
>> > Is there anything else I can provide you?
>> >
>> > The command I'm using to run the client is the following (you can see
>> that
>> > 0.20.3 version of HBase is used and also other versions you might be
>> > interested in):
>> > java -cp
>> >
>> commons-cli-1.2.jar:commons-logging-1.1.1.jar:hadoop-0.20.1-core.jar:hbase-0.20.3.jar:hbase-0.20.3-test.jar:log4j-1.2.15.jar:test-1.0-SNAPSHOT.jar:zookeeper-3.2.2.jar
>> > com.foo.bar.client.ClientExample
>> >
>> > Thank you for your help!
>> > Alex
>> >
>> > On Thu, Mar 18, 2010 at 7:13 PM, Jean-Daniel Cryans <
>> jdcryans@apache.org>wrote:
>> >
>> >> Alex,
>> >>
>> >> Is there anything else in the region server logs before that like
>> >> lease expirations? Can we see a much bigger log? Also is there
>> >> anything in the .out file? Can you post a snippet of the code you are
>> >> using?
>> >>
>> >> Thx
>> >>
>> >> J-D
>> >>
>> >> On Wed, Mar 17, 2010 at 11:25 PM, Alex Baranov <
>> alex.baranov.v@gmail.com>
>> >> wrote:
>> >> > To give more clarity, I'm using *not custom* filter, but "standard"
>> >> > SingleColumnValueFilter. So it's not related to classpath issues.
>> >> >
>> >> > Any help is very appreciated!
>> >> >
>> >> > Thanks,
>> >> > Alex.
>> >> >
>> >> > On Wed, Mar 17, 2010 at 6:00 PM, Alex Baranov <
>> alex.baranov.v@gmail.com
>> >> >wrote:
>> >> >
>> >> >> Hello guys,
>> >> >>
>> >> >> I've got a problem while adding a filter to scanner in a client app
>> >> which
>> >> >> runs on the remote (not the one from HBase cluster) box. The same
>> code
>> >> works
>> >> >> well and scan result is fetched very quickly if I run the client on
>> the
>> >> same
>> >> >> box where HBase master resides. If I comment out adding filter then
>> the
>> >> >> scanner returns results. But with filter it keeps showing me the
>> error
>> >> >> below.
>> >> >>
>> >> >> I'm using HBase 0.20.3 on both ends.
>> >> >>
>> >> >> On the mailing list I saw that problems like this can arise when
>> using
>> >> >> different versions of HBase on server and on client, but this is not
>> the
>> >> >> case. Also the error like this can show up when it takes a lot of
>> time
>> >> to
>> >> >> initialize scanner (lease time by default is 1 min), but I assume
>> this
>> >> is
>> >> >> also not the case since without adding filter I got results very
>> >> quickly.
>> >> >>
>> >> >> Does anyone have an idea what is going on?
>> >> >>
>> >> >> - in log on remote client side:
>> >> >>
>> >> >> Exception in thread "main"
>> >> >> org.apache.hadoop.hbase.client.RetriesExhaustedException: Trying to
>> >> contact
>> >> >> region server 10.210.71.80:39207 for region xxxxx,,1267450079067,
>> row
>> >> >> 'yyyyy', but failed after 10 attempts.
>> >> >> Exceptions:
>> >> >> java.io.IOException: Call to /10.210.71.80:39207 failed on local
>> >> >> exception: java.io.EOFException
>> >> >> java.io.IOException: Call to /10.210.71.80:39207 failed on local
>> >> >> exception: java.io.EOFException
>> >> >> java.io.IOException: Call to /10.210.71.80:39207 failed on local
>> >> >> exception: java.io.EOFException
>> >> >> java.io.IOException: Call to /10.210.71.80:39207 failed on local
>> >> >> exception: java.io.EOFException
>> >> >> java.io.IOException: Call to /10.210.71.80:39207 failed on local
>> >> >> exception: java.io.EOFException
>> >> >> java.io.IOException: Call to /10.210.71.80:39207 failed on local
>> >> >> exception: java.io.EOFException
>> >> >> java.io.IOException: Call to /10.210.71.80:39207 failed on local
>> >> >> exception: java.io.EOFException
>> >> >> java.io.IOException: Call to /10.210.71.80:39207 failed on local
>> >> >> exception: java.io.EOFException
>> >> >> java.io.IOException: Call to /10.210.71.80:39207 failed on local
>> >> >> exception: java.io.EOFException
>> >> >> java.io.IOException: Call to /10.210.71.80:39207 failed on local
>> >> >> exception: java.io.EOFException
>> >> >>
>> >> >> at
>> >> >>
>> >>
>> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getRegionServerWithRetries(HConnectionManager.java:1002)
>> >> >> at
>> >> >>
>> >>
>> org.apache.hadoop.hbase.client.HTable$ClientScanner.nextScanner(HTable.java:1931)
>> >> >> at
>> >> >>
>> >>
>> org.apache.hadoop.hbase.client.HTable$ClientScanner.initialize(HTable.java:1851)
>> >> >> at
>> org.apache.hadoop.hbase.client.HTable.getScanner(HTable.java:372)
>> >> >> at
>> >> >>
>> >>
>> com.sematext.sa.client.AggregatesAccessor.getResult(AggregatesAccessor.java:74)
>> >> >> at
>> com.sematext.sa.client.ClientExample.main(ClientExample.java:41)
>> >> >>
>> >> >>
>> >> >> - in HBase master log:
>> >> >>
>> >> >> 2010-03-17 12:37:45,068 ERROR
>> >> >> org.apache.hadoop.hbase.regionserver.HRegionServer:
>> >> >> org.apache.hadoop.hbase.UnknownScannerException: Name: -1
>> >> >> at
>> >> >>
>> >>
>> org.apache.hadoop.hbase.regionserver.HRegionServer.next(HRegionServer.java:1877)
>> >> >> at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown
>> Source)
>> >> >> at
>> >> >>
>> >>
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>> >> >> at java.lang.reflect.Method.invoke(Method.java:597)
>> >> >> at
>> >> >> org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:657)
>> >> >> at
>> >> >>
>> >>
>> org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:915)
>> >> >>
>> >> >> - in HBase region server log/out: nothing
>> >> >>
>> >> >> Thank you in advance.
>> >> >> Alex.
>> >> >>
>> >> >
>> >>
>> >
>>
>
>
Re: Adding filter to scan at remote client causes
UnknownScannerException
Posted by Alex Baranov <al...@gmail.com>.
It hangs for some time. I'm not using any contribs.
Thanks for the help,
Alex.
On Fri, Mar 19, 2010 at 11:29 PM, Jean-Daniel Cryans <jd...@apache.org>wrote:
> Alex,
>
> I tried your code from a remote machine to a pseudo-distributed setup
> and it worked well (on trunk, didn't have a 0.20.3 setup around). When
> the call fails, doesn't it return right away or it hangs for some
> time? Also are you using any contrib?
>
> Thx
>
> J-D
>
> On Fri, Mar 19, 2010 at 5:28 AM, Alex Baranov <al...@gmail.com>
> wrote:
> > Hello J-D,
> >
> > Thanks for helping me out!
> >
> > Here is the code that works if I run it on machine that has HBase master
> on
> > it and doesn't work on "remote" client box:
> >
> > // CODE BEGINS
> >
> > HBaseConfiguration conf = new HBaseConfiguration();
> > HTable hTable = new HTable(conf, "agg9");
> >
> > Scan scan = new Scan();
> > scan.setStartRow(Bytes.toBytes("qf|byday_bytype_|14656__|"));
> >
> > FilterList filters = new
> FilterList(FilterList.Operator.MUST_PASS_ALL);
> >
> > SingleColumnValueFilter filter = new
> > SingleColumnValueFilter(Bytes.toBytes("agg"), Bytes.toBytes("count"),
> > CompareFilter.CompareOp.GREATER, Bytes.toBytes(35));
> > filters.addFilter(filter);
> >
> > InclusiveStopFilter stopFilter = new
> > InclusiveStopFilter(Bytes.toBytes("qf|byday_|14739_|"));
> > filters.addFilter(stopFilter);
> > scan.setFilter(filters);
> >
> > ResultScanner rs = hTable.getScanner(scan);
> > Result next = rs.next();
> > int readCount = 0;
> > while (next != null && readCount < 40) {
> > System.out.println("Row key: " + Bytes.toString(next.getRow()));
> > System.out.println("count: " +
> > Bytes.toInt(next.getValue(Bytes.toBytes("agg"),
> Bytes.toBytes("count"))));
> > next = rs.next();
> > readCount++;
> > }
> >
> > // CODE ENDS
> >
> > If I comment the line
> > filters.addFilter(filter);
> > then the code works on remote client box as well.
> >
> >
> > Client fails with the exception I provided previously. The bigger master
> log
> > (this is the log till the end and started when I ran the client code):
> >
> > 2010-03-19 12:15:02,098 INFO
> org.apache.hadoop.hbase.master.ServerManager: 1
> > region servers, 0 dead, average load 27.0
> > 2010-03-19 12:15:41,369 INFO org.apache.hadoop.hbase.master.BaseScanner:
> > RegionManager.metaScanner scanning meta region {server:
> 10.210.71.80:39207,
> > regionname: .META.,,1, startKey: <>}
> > 2010-03-19 12:15:41,398 INFO org.apache.hadoop.hbase.master.BaseScanner:
> > RegionManager.metaScanner scan of 25 row(s) of meta region {server:
> > 10.210.71.80:39207, regionname: .META.,,1, startKey: <>} complete
> > 2010-03-19 12:15:41,398 INFO org.apache.hadoop.hbase.master.BaseScanner:
> All
> > 1 .META. region(s) scanned
> > 2010-03-19 12:15:41,548 INFO org.apache.hadoop.hbase.master.BaseScanner:
> > RegionManager.rootScanner scanning meta region {server:
> 10.210.71.80:39207,
> > regionname: -ROOT-,,0, startKey: <>}
> > 2010-03-19 12:15:41,549 INFO org.apache.hadoop.hbase.master.BaseScanner:
> > RegionManager.rootScanner scan of 1 row(s) of meta region {server:
> > 10.210.71.80:39207, regionname: -ROOT-,,0, startKey: <>} complete
> > 2010-03-19 12:15:45,398 DEBUG
> > org.apache.hadoop.hbase.io.hfile.LruBlockCache: Cache Stats: Sizes:
> > Total=43.474342MB (45586152), Free=156.21317MB (163801368),
> Max=199.6875MB
> > (209387520), Counts: Blocks=684, Access=90114, Hit=80498, Miss=9616,
> > Evictions=0, Evicted=0, Ratios: Hit Ratio=89.32906985282898%, Miss
> > Ratio=10.670927911996841%, Evicted/Run=NaN
> > 2010-03-19 12:16:02,108 INFO
> org.apache.hadoop.hbase.master.ServerManager: 1
> > region servers, 0 dead, average load 27.0
> > 2010-03-19 12:16:41,378 INFO org.apache.hadoop.hbase.master.BaseScanner:
> > RegionManager.metaScanner scanning meta region {server:
> 10.210.71.80:39207,
> > regionname: .META.,,1, startKey: <>}
> > 2010-03-19 12:16:41,461 INFO org.apache.hadoop.hbase.master.BaseScanner:
> > RegionManager.metaScanner scan of 25 row(s) of meta region {server:
> > 10.210.71.80:39207, regionname: .META.,,1, startKey: <>} complete
> > 2010-03-19 12:16:41,461 INFO org.apache.hadoop.hbase.master.BaseScanner:
> All
> > 1 .META. region(s) scanned
> > 2010-03-19 12:16:41,558 INFO org.apache.hadoop.hbase.master.BaseScanner:
> > RegionManager.rootScanner scanning meta region {server:
> 10.210.71.80:39207,
> > regionname: -ROOT-,,0, startKey: <>}
> > 2010-03-19 12:16:41,559 INFO org.apache.hadoop.hbase.master.BaseScanner:
> > RegionManager.rootScanner scan of 1 row(s) of meta region {server:
> > 10.210.71.80:39207, regionname: -ROOT-,,0, startKey: <>} complete
> > 2010-03-19 12:16:45,398 DEBUG
> > org.apache.hadoop.hbase.io.hfile.LruBlockCache: Cache Stats: Sizes:
> > Total=43.474342MB (45586152), Free=156.21317MB (163801368),
> Max=199.6875MB
> > (209387520), Counts: Blocks=684, Access=90116, Hit=80500, Miss=9616,
> > Evictions=0, Evicted=0, Ratios: Hit Ratio=89.32930827140808%, Miss
> > Ratio=10.67069098353386%, Evicted/Run=NaN
> > 2010-03-19 12:16:59,872 ERROR
> > org.apache.hadoop.hbase.regionserver.HRegionServer:
> > org.apache.hadoop.hbase.UnknownScannerException: Name: -1
> > at
> >
> org.apache.hadoop.hbase.regionserver.HRegionServer.next(HRegionServer.java:1877)
> > at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source)
> > at
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> > at java.lang.reflect.Method.invoke(Method.java:597)
> > at
> > org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:657)
> > at
> > org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:915)
> > 2010-03-19 12:16:59,874 INFO org.apache.hadoop.ipc.HBaseServer: IPC
> Server
> > handler 16 on 39207, call next(-1, 1) from 10.241.27.112:38002: error:
> > org.apache.hadoop.hbase.UnknownScannerException: Name: -1
> > org.apache.hadoop.hbase.UnknownScannerException: Name: -1
> > at
> >
> org.apache.hadoop.hbase.regionserver.HRegionServer.next(HRegionServer.java:1877)
> > at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source)
> > at
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> > at java.lang.reflect.Method.invoke(Method.java:597)
> > at
> > org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:657)
> > at
> > org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:915)
> > 2010-03-19 12:17:02,118 INFO
> org.apache.hadoop.hbase.master.ServerManager: 1
> > region servers, 0 dead, average load 27.0
> > 2010-03-19 12:17:41,388 INFO org.apache.hadoop.hbase.master.BaseScanner:
> > RegionManager.metaScanner scanning meta region {server:
> 10.210.71.80:39207,
> > regionname: .META.,,1, startKey: <>}
> > 2010-03-19 12:17:41,471 INFO org.apache.hadoop.hbase.master.BaseScanner:
> > RegionManager.metaScanner scan of 25 row(s) of meta region {server:
> > 10.210.71.80:39207, regionname: .META.,,1, startKey: <>} complete
> > 2010-03-19 12:17:41,471 INFO org.apache.hadoop.hbase.master.BaseScanner:
> All
> > 1 .META. region(s) scanned
> > 2010-03-19 12:17:41,568 INFO org.apache.hadoop.hbase.master.BaseScanner:
> > RegionManager.rootScanner scanning meta region {server:
> 10.210.71.80:39207,
> > regionname: -ROOT-,,0, startKey: <>}
> > 2010-03-19 12:17:41,569 INFO org.apache.hadoop.hbase.master.BaseScanner:
> > RegionManager.rootScanner scan of 1 row(s) of meta region {server:
> > 10.210.71.80:39207, regionname: -ROOT-,,0, startKey: <>} complete
> > 2010-03-19 12:17:45,398 DEBUG
> > org.apache.hadoop.hbase.io.hfile.LruBlockCache: Cache Stats: Sizes:
> > Total=43.474342MB (45586152), Free=156.21317MB (163801368),
> Max=199.6875MB
> > (209387520), Counts: Blocks=684, Access=90118, Hit=80502, Miss=9616,
> > Evictions=0, Evicted=0, Ratios: Hit Ratio=89.32954668998718%, Miss
> > Ratio=10.670454055070877%, Evicted/Run=NaN
> > 2010-03-19 12:18:02,128 INFO
> org.apache.hadoop.hbase.master.ServerManager: 1
> > region servers, 0 dead, average load 27.0
> >
> > There is nothing in region log:
> >
> > ubuntu@xxx:/mnt/hbase-0.20.3/logs$ ls -l | grep region
> > -rw-r--r-- 1 ubuntu ubuntu 2032 2010-02-24 05:51
> > hbase-ubuntu-regionserver-domU-12-31-39-09-40-A2.log
> > -rw-r--r-- 1 ubuntu ubuntu 0 2010-02-24 05:51
> > hbase-ubuntu-regionserver-domU-12-31-39-09-40-A2.out
> > -rw-r--r-- 1 ubuntu ubuntu 0 2010-02-04 16:04
> > hbase-ubuntu-regionserver-domU-12-31-39-09-40-A2.out.1
> > -rw-r--r-- 1 ubuntu ubuntu 0 2010-02-04 15:03
> > hbase-ubuntu-regionserver-domU-12-31-39-09-40-A2.out.2
> > -rw-r--r-- 1 ubuntu ubuntu 0 2010-02-04 14:37
> > hbase-ubuntu-regionserver-domU-12-31-39-09-40-A2.out.3
> > -rw-r--r-- 1 ubuntu ubuntu 0 2010-02-04 08:15
> > hbase-ubuntu-regionserver-domU-12-31-39-09-40-A2.out.4
> > -rw-r--r-- 1 ubuntu ubuntu 0 2010-02-04 07:56
> > hbase-ubuntu-regionserver-domU-12-31-39-09-40-A2.out.5
> >
> > Also nothing in .out logs:
> >
> > ubuntu@xxx:/mnt/hbase-0.20.3/logs$ ls -l | grep \.out
> > -rw-r--r-- 1 ubuntu ubuntu 324 2010-02-24 05:51
> > hbase-ubuntu-master-domU-12-31-39-09-40-A2.out
> > -rw-r--r-- 1 ubuntu ubuntu 0 2010-02-24 05:50
> > hbase-ubuntu-master-domU-12-31-39-09-40-A2.out.1
> > -rw-r--r-- 1 ubuntu ubuntu 0 2010-02-04 16:01
> > hbase-ubuntu-master-domU-12-31-39-09-40-A2.out.2
> > -rw-r--r-- 1 ubuntu ubuntu 0 2010-02-04 14:56
> > hbase-ubuntu-master-domU-12-31-39-09-40-A2.out.3
> > -rw-r--r-- 1 ubuntu ubuntu 0 2010-02-04 14:30
> > hbase-ubuntu-master-domU-12-31-39-09-40-A2.out.4
> > -rw-r--r-- 1 ubuntu ubuntu 0 2010-02-04 08:07
> > hbase-ubuntu-master-domU-12-31-39-09-40-A2.out.5
> > -rw-r--r-- 1 ubuntu ubuntu 0 2010-02-24 05:51
> > hbase-ubuntu-regionserver-domU-12-31-39-09-40-A2.out
> > -rw-r--r-- 1 ubuntu ubuntu 0 2010-02-04 16:04
> > hbase-ubuntu-regionserver-domU-12-31-39-09-40-A2.out.1
> > -rw-r--r-- 1 ubuntu ubuntu 0 2010-02-04 15:03
> > hbase-ubuntu-regionserver-domU-12-31-39-09-40-A2.out.2
> > -rw-r--r-- 1 ubuntu ubuntu 0 2010-02-04 14:37
> > hbase-ubuntu-regionserver-domU-12-31-39-09-40-A2.out.3
> > -rw-r--r-- 1 ubuntu ubuntu 0 2010-02-04 08:15
> > hbase-ubuntu-regionserver-domU-12-31-39-09-40-A2.out.4
> > -rw-r--r-- 1 ubuntu ubuntu 0 2010-02-04 07:56
> > hbase-ubuntu-regionserver-domU-12-31-39-09-40-A2.out.5
> > -rw-r--r-- 1 ubuntu ubuntu 0 2010-02-24 05:51
> > hbase-ubuntu-zookeeper-domU-12-31-39-09-40-A2.out
> > -rw-r--r-- 1 ubuntu ubuntu 0 2010-02-04 16:04
> > hbase-ubuntu-zookeeper-domU-12-31-39-09-40-A2.out.1
> > -rw-r--r-- 1 ubuntu ubuntu 0 2010-02-04 15:03
> > hbase-ubuntu-zookeeper-domU-12-31-39-09-40-A2.out.2
> > -rw-r--r-- 1 ubuntu ubuntu 0 2010-02-04 14:37
> > hbase-ubuntu-zookeeper-domU-12-31-39-09-40-A2.out.3
> > -rw-r--r-- 1 ubuntu ubuntu 0 2010-02-04 08:15
> > hbase-ubuntu-zookeeper-domU-12-31-39-09-40-A2.out.4
> > -rw-r--r-- 1 ubuntu ubuntu 0 2010-02-04 07:56
> > hbase-ubuntu-zookeeper-domU-12-31-39-09-40-A2.out.5
> >
> > Is there anything else I can provide you?
> >
> > The command I'm using to run the client is the following (you can see
> that
> > 0.20.3 version of HBase is used and also other versions you might be
> > interested in):
> > java -cp
> >
> commons-cli-1.2.jar:commons-logging-1.1.1.jar:hadoop-0.20.1-core.jar:hbase-0.20.3.jar:hbase-0.20.3-test.jar:log4j-1.2.15.jar:test-1.0-SNAPSHOT.jar:zookeeper-3.2.2.jar
> > com.foo.bar.client.ClientExample
> >
> > Thank you for your help!
> > Alex
> >
> > On Thu, Mar 18, 2010 at 7:13 PM, Jean-Daniel Cryans <jdcryans@apache.org
> >wrote:
> >
> >> Alex,
> >>
> >> Is there anything else in the region server logs before that like
> >> lease expirations? Can we see a much bigger log? Also is there
> >> anything in the .out file? Can you post a snippet of the code you are
> >> using?
> >>
> >> Thx
> >>
> >> J-D
> >>
> >> On Wed, Mar 17, 2010 at 11:25 PM, Alex Baranov <
> alex.baranov.v@gmail.com>
> >> wrote:
> >> > To give more clarity, I'm using *not custom* filter, but "standard"
> >> > SingleColumnValueFilter. So it's not related to classpath issues.
> >> >
> >> > Any help is very appreciated!
> >> >
> >> > Thanks,
> >> > Alex.
> >> >
> >> > On Wed, Mar 17, 2010 at 6:00 PM, Alex Baranov <
> alex.baranov.v@gmail.com
> >> >wrote:
> >> >
> >> >> Hello guys,
> >> >>
> >> >> I've got a problem while adding a filter to scanner in a client app
> >> which
> >> >> runs on the remote (not the one from HBase cluster) box. The same
> code
> >> works
> >> >> well and scan result is fetched very quickly if I run the client on
> the
> >> same
> >> >> box where HBase master resides. If I comment out adding filter then
> the
> >> >> scanner returns results. But with filter it keeps showing me the
> error
> >> >> below.
> >> >>
> >> >> I'm using HBase 0.20.3 on both ends.
> >> >>
> >> >> On the mailing list I saw that problems like this can arise when
> using
> >> >> different versions of HBase on server and on client, but this is not
> the
> >> >> case. Also the error like this can show up when it takes a lot of
> time
> >> to
> >> >> initialize scanner (lease time by default is 1 min), but I assume
> this
> >> is
> >> >> also not the case since without adding filter I got results very
> >> quickly.
> >> >>
> >> >> Does anyone have an idea what is going on?
> >> >>
> >> >> - in log on remote client side:
> >> >>
> >> >> Exception in thread "main"
> >> >> org.apache.hadoop.hbase.client.RetriesExhaustedException: Trying to
> >> contact
> >> >> region server 10.210.71.80:39207 for region xxxxx,,1267450079067,
> row
> >> >> 'yyyyy', but failed after 10 attempts.
> >> >> Exceptions:
> >> >> java.io.IOException: Call to /10.210.71.80:39207 failed on local
> >> >> exception: java.io.EOFException
> >> >> java.io.IOException: Call to /10.210.71.80:39207 failed on local
> >> >> exception: java.io.EOFException
> >> >> java.io.IOException: Call to /10.210.71.80:39207 failed on local
> >> >> exception: java.io.EOFException
> >> >> java.io.IOException: Call to /10.210.71.80:39207 failed on local
> >> >> exception: java.io.EOFException
> >> >> java.io.IOException: Call to /10.210.71.80:39207 failed on local
> >> >> exception: java.io.EOFException
> >> >> java.io.IOException: Call to /10.210.71.80:39207 failed on local
> >> >> exception: java.io.EOFException
> >> >> java.io.IOException: Call to /10.210.71.80:39207 failed on local
> >> >> exception: java.io.EOFException
> >> >> java.io.IOException: Call to /10.210.71.80:39207 failed on local
> >> >> exception: java.io.EOFException
> >> >> java.io.IOException: Call to /10.210.71.80:39207 failed on local
> >> >> exception: java.io.EOFException
> >> >> java.io.IOException: Call to /10.210.71.80:39207 failed on local
> >> >> exception: java.io.EOFException
> >> >>
> >> >> at
> >> >>
> >>
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getRegionServerWithRetries(HConnectionManager.java:1002)
> >> >> at
> >> >>
> >>
> org.apache.hadoop.hbase.client.HTable$ClientScanner.nextScanner(HTable.java:1931)
> >> >> at
> >> >>
> >>
> org.apache.hadoop.hbase.client.HTable$ClientScanner.initialize(HTable.java:1851)
> >> >> at
> org.apache.hadoop.hbase.client.HTable.getScanner(HTable.java:372)
> >> >> at
> >> >>
> >>
> com.sematext.sa.client.AggregatesAccessor.getResult(AggregatesAccessor.java:74)
> >> >> at
> com.sematext.sa.client.ClientExample.main(ClientExample.java:41)
> >> >>
> >> >>
> >> >> - in HBase master log:
> >> >>
> >> >> 2010-03-17 12:37:45,068 ERROR
> >> >> org.apache.hadoop.hbase.regionserver.HRegionServer:
> >> >> org.apache.hadoop.hbase.UnknownScannerException: Name: -1
> >> >> at
> >> >>
> >>
> org.apache.hadoop.hbase.regionserver.HRegionServer.next(HRegionServer.java:1877)
> >> >> at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown
> Source)
> >> >> at
> >> >>
> >>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> >> >> at java.lang.reflect.Method.invoke(Method.java:597)
> >> >> at
> >> >> org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:657)
> >> >> at
> >> >>
> >>
> org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:915)
> >> >>
> >> >> - in HBase region server log/out: nothing
> >> >>
> >> >> Thank you in advance.
> >> >> Alex.
> >> >>
> >> >
> >>
> >
>
Re: Adding filter to scan at remote client causes
UnknownScannerException
Posted by Jean-Daniel Cryans <jd...@apache.org>.
Alex,
I tried your code from a remote machine to a pseudo-distributed setup
and it worked well (on trunk, didn't have a 0.20.3 setup around). When
the call fails, doesn't it return right away or it hangs for some
time? Also are you using any contrib?
Thx
J-D
On Fri, Mar 19, 2010 at 5:28 AM, Alex Baranov <al...@gmail.com> wrote:
> Hello J-D,
>
> Thanks for helping me out!
>
> Here is the code that works if I run it on machine that has HBase master on
> it and doesn't work on "remote" client box:
>
> // CODE BEGINS
>
> HBaseConfiguration conf = new HBaseConfiguration();
> HTable hTable = new HTable(conf, "agg9");
>
> Scan scan = new Scan();
> scan.setStartRow(Bytes.toBytes("qf|byday_bytype_|14656__|"));
>
> FilterList filters = new FilterList(FilterList.Operator.MUST_PASS_ALL);
>
> SingleColumnValueFilter filter = new
> SingleColumnValueFilter(Bytes.toBytes("agg"), Bytes.toBytes("count"),
> CompareFilter.CompareOp.GREATER, Bytes.toBytes(35));
> filters.addFilter(filter);
>
> InclusiveStopFilter stopFilter = new
> InclusiveStopFilter(Bytes.toBytes("qf|byday_|14739_|"));
> filters.addFilter(stopFilter);
> scan.setFilter(filters);
>
> ResultScanner rs = hTable.getScanner(scan);
> Result next = rs.next();
> int readCount = 0;
> while (next != null && readCount < 40) {
> System.out.println("Row key: " + Bytes.toString(next.getRow()));
> System.out.println("count: " +
> Bytes.toInt(next.getValue(Bytes.toBytes("agg"), Bytes.toBytes("count"))));
> next = rs.next();
> readCount++;
> }
>
> // CODE ENDS
>
> If I comment the line
> filters.addFilter(filter);
> then the code works on remote client box as well.
>
>
> Client fails with the exception I provided previously. The bigger master log
> (this is the log till the end and started when I ran the client code):
>
> 2010-03-19 12:15:02,098 INFO org.apache.hadoop.hbase.master.ServerManager: 1
> region servers, 0 dead, average load 27.0
> 2010-03-19 12:15:41,369 INFO org.apache.hadoop.hbase.master.BaseScanner:
> RegionManager.metaScanner scanning meta region {server: 10.210.71.80:39207,
> regionname: .META.,,1, startKey: <>}
> 2010-03-19 12:15:41,398 INFO org.apache.hadoop.hbase.master.BaseScanner:
> RegionManager.metaScanner scan of 25 row(s) of meta region {server:
> 10.210.71.80:39207, regionname: .META.,,1, startKey: <>} complete
> 2010-03-19 12:15:41,398 INFO org.apache.hadoop.hbase.master.BaseScanner: All
> 1 .META. region(s) scanned
> 2010-03-19 12:15:41,548 INFO org.apache.hadoop.hbase.master.BaseScanner:
> RegionManager.rootScanner scanning meta region {server: 10.210.71.80:39207,
> regionname: -ROOT-,,0, startKey: <>}
> 2010-03-19 12:15:41,549 INFO org.apache.hadoop.hbase.master.BaseScanner:
> RegionManager.rootScanner scan of 1 row(s) of meta region {server:
> 10.210.71.80:39207, regionname: -ROOT-,,0, startKey: <>} complete
> 2010-03-19 12:15:45,398 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Cache Stats: Sizes:
> Total=43.474342MB (45586152), Free=156.21317MB (163801368), Max=199.6875MB
> (209387520), Counts: Blocks=684, Access=90114, Hit=80498, Miss=9616,
> Evictions=0, Evicted=0, Ratios: Hit Ratio=89.32906985282898%, Miss
> Ratio=10.670927911996841%, Evicted/Run=NaN
> 2010-03-19 12:16:02,108 INFO org.apache.hadoop.hbase.master.ServerManager: 1
> region servers, 0 dead, average load 27.0
> 2010-03-19 12:16:41,378 INFO org.apache.hadoop.hbase.master.BaseScanner:
> RegionManager.metaScanner scanning meta region {server: 10.210.71.80:39207,
> regionname: .META.,,1, startKey: <>}
> 2010-03-19 12:16:41,461 INFO org.apache.hadoop.hbase.master.BaseScanner:
> RegionManager.metaScanner scan of 25 row(s) of meta region {server:
> 10.210.71.80:39207, regionname: .META.,,1, startKey: <>} complete
> 2010-03-19 12:16:41,461 INFO org.apache.hadoop.hbase.master.BaseScanner: All
> 1 .META. region(s) scanned
> 2010-03-19 12:16:41,558 INFO org.apache.hadoop.hbase.master.BaseScanner:
> RegionManager.rootScanner scanning meta region {server: 10.210.71.80:39207,
> regionname: -ROOT-,,0, startKey: <>}
> 2010-03-19 12:16:41,559 INFO org.apache.hadoop.hbase.master.BaseScanner:
> RegionManager.rootScanner scan of 1 row(s) of meta region {server:
> 10.210.71.80:39207, regionname: -ROOT-,,0, startKey: <>} complete
> 2010-03-19 12:16:45,398 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Cache Stats: Sizes:
> Total=43.474342MB (45586152), Free=156.21317MB (163801368), Max=199.6875MB
> (209387520), Counts: Blocks=684, Access=90116, Hit=80500, Miss=9616,
> Evictions=0, Evicted=0, Ratios: Hit Ratio=89.32930827140808%, Miss
> Ratio=10.67069098353386%, Evicted/Run=NaN
> 2010-03-19 12:16:59,872 ERROR
> org.apache.hadoop.hbase.regionserver.HRegionServer:
> org.apache.hadoop.hbase.UnknownScannerException: Name: -1
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.next(HRegionServer.java:1877)
> at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:657)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:915)
> 2010-03-19 12:16:59,874 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server
> handler 16 on 39207, call next(-1, 1) from 10.241.27.112:38002: error:
> org.apache.hadoop.hbase.UnknownScannerException: Name: -1
> org.apache.hadoop.hbase.UnknownScannerException: Name: -1
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.next(HRegionServer.java:1877)
> at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:657)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:915)
> 2010-03-19 12:17:02,118 INFO org.apache.hadoop.hbase.master.ServerManager: 1
> region servers, 0 dead, average load 27.0
> 2010-03-19 12:17:41,388 INFO org.apache.hadoop.hbase.master.BaseScanner:
> RegionManager.metaScanner scanning meta region {server: 10.210.71.80:39207,
> regionname: .META.,,1, startKey: <>}
> 2010-03-19 12:17:41,471 INFO org.apache.hadoop.hbase.master.BaseScanner:
> RegionManager.metaScanner scan of 25 row(s) of meta region {server:
> 10.210.71.80:39207, regionname: .META.,,1, startKey: <>} complete
> 2010-03-19 12:17:41,471 INFO org.apache.hadoop.hbase.master.BaseScanner: All
> 1 .META. region(s) scanned
> 2010-03-19 12:17:41,568 INFO org.apache.hadoop.hbase.master.BaseScanner:
> RegionManager.rootScanner scanning meta region {server: 10.210.71.80:39207,
> regionname: -ROOT-,,0, startKey: <>}
> 2010-03-19 12:17:41,569 INFO org.apache.hadoop.hbase.master.BaseScanner:
> RegionManager.rootScanner scan of 1 row(s) of meta region {server:
> 10.210.71.80:39207, regionname: -ROOT-,,0, startKey: <>} complete
> 2010-03-19 12:17:45,398 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Cache Stats: Sizes:
> Total=43.474342MB (45586152), Free=156.21317MB (163801368), Max=199.6875MB
> (209387520), Counts: Blocks=684, Access=90118, Hit=80502, Miss=9616,
> Evictions=0, Evicted=0, Ratios: Hit Ratio=89.32954668998718%, Miss
> Ratio=10.670454055070877%, Evicted/Run=NaN
> 2010-03-19 12:18:02,128 INFO org.apache.hadoop.hbase.master.ServerManager: 1
> region servers, 0 dead, average load 27.0
>
> There is nothing in region log:
>
> ubuntu@xxx:/mnt/hbase-0.20.3/logs$ ls -l | grep region
> -rw-r--r-- 1 ubuntu ubuntu 2032 2010-02-24 05:51
> hbase-ubuntu-regionserver-domU-12-31-39-09-40-A2.log
> -rw-r--r-- 1 ubuntu ubuntu 0 2010-02-24 05:51
> hbase-ubuntu-regionserver-domU-12-31-39-09-40-A2.out
> -rw-r--r-- 1 ubuntu ubuntu 0 2010-02-04 16:04
> hbase-ubuntu-regionserver-domU-12-31-39-09-40-A2.out.1
> -rw-r--r-- 1 ubuntu ubuntu 0 2010-02-04 15:03
> hbase-ubuntu-regionserver-domU-12-31-39-09-40-A2.out.2
> -rw-r--r-- 1 ubuntu ubuntu 0 2010-02-04 14:37
> hbase-ubuntu-regionserver-domU-12-31-39-09-40-A2.out.3
> -rw-r--r-- 1 ubuntu ubuntu 0 2010-02-04 08:15
> hbase-ubuntu-regionserver-domU-12-31-39-09-40-A2.out.4
> -rw-r--r-- 1 ubuntu ubuntu 0 2010-02-04 07:56
> hbase-ubuntu-regionserver-domU-12-31-39-09-40-A2.out.5
>
> Also nothing in .out logs:
>
> ubuntu@xxx:/mnt/hbase-0.20.3/logs$ ls -l | grep \.out
> -rw-r--r-- 1 ubuntu ubuntu 324 2010-02-24 05:51
> hbase-ubuntu-master-domU-12-31-39-09-40-A2.out
> -rw-r--r-- 1 ubuntu ubuntu 0 2010-02-24 05:50
> hbase-ubuntu-master-domU-12-31-39-09-40-A2.out.1
> -rw-r--r-- 1 ubuntu ubuntu 0 2010-02-04 16:01
> hbase-ubuntu-master-domU-12-31-39-09-40-A2.out.2
> -rw-r--r-- 1 ubuntu ubuntu 0 2010-02-04 14:56
> hbase-ubuntu-master-domU-12-31-39-09-40-A2.out.3
> -rw-r--r-- 1 ubuntu ubuntu 0 2010-02-04 14:30
> hbase-ubuntu-master-domU-12-31-39-09-40-A2.out.4
> -rw-r--r-- 1 ubuntu ubuntu 0 2010-02-04 08:07
> hbase-ubuntu-master-domU-12-31-39-09-40-A2.out.5
> -rw-r--r-- 1 ubuntu ubuntu 0 2010-02-24 05:51
> hbase-ubuntu-regionserver-domU-12-31-39-09-40-A2.out
> -rw-r--r-- 1 ubuntu ubuntu 0 2010-02-04 16:04
> hbase-ubuntu-regionserver-domU-12-31-39-09-40-A2.out.1
> -rw-r--r-- 1 ubuntu ubuntu 0 2010-02-04 15:03
> hbase-ubuntu-regionserver-domU-12-31-39-09-40-A2.out.2
> -rw-r--r-- 1 ubuntu ubuntu 0 2010-02-04 14:37
> hbase-ubuntu-regionserver-domU-12-31-39-09-40-A2.out.3
> -rw-r--r-- 1 ubuntu ubuntu 0 2010-02-04 08:15
> hbase-ubuntu-regionserver-domU-12-31-39-09-40-A2.out.4
> -rw-r--r-- 1 ubuntu ubuntu 0 2010-02-04 07:56
> hbase-ubuntu-regionserver-domU-12-31-39-09-40-A2.out.5
> -rw-r--r-- 1 ubuntu ubuntu 0 2010-02-24 05:51
> hbase-ubuntu-zookeeper-domU-12-31-39-09-40-A2.out
> -rw-r--r-- 1 ubuntu ubuntu 0 2010-02-04 16:04
> hbase-ubuntu-zookeeper-domU-12-31-39-09-40-A2.out.1
> -rw-r--r-- 1 ubuntu ubuntu 0 2010-02-04 15:03
> hbase-ubuntu-zookeeper-domU-12-31-39-09-40-A2.out.2
> -rw-r--r-- 1 ubuntu ubuntu 0 2010-02-04 14:37
> hbase-ubuntu-zookeeper-domU-12-31-39-09-40-A2.out.3
> -rw-r--r-- 1 ubuntu ubuntu 0 2010-02-04 08:15
> hbase-ubuntu-zookeeper-domU-12-31-39-09-40-A2.out.4
> -rw-r--r-- 1 ubuntu ubuntu 0 2010-02-04 07:56
> hbase-ubuntu-zookeeper-domU-12-31-39-09-40-A2.out.5
>
> Is there anything else I can provide you?
>
> The command I'm using to run the client is the following (you can see that
> 0.20.3 version of HBase is used and also other versions you might be
> interested in):
> java -cp
> commons-cli-1.2.jar:commons-logging-1.1.1.jar:hadoop-0.20.1-core.jar:hbase-0.20.3.jar:hbase-0.20.3-test.jar:log4j-1.2.15.jar:test-1.0-SNAPSHOT.jar:zookeeper-3.2.2.jar
> com.foo.bar.client.ClientExample
>
> Thank you for your help!
> Alex
>
> On Thu, Mar 18, 2010 at 7:13 PM, Jean-Daniel Cryans <jd...@apache.org>wrote:
>
>> Alex,
>>
>> Is there anything else in the region server logs before that like
>> lease expirations? Can we see a much bigger log? Also is there
>> anything in the .out file? Can you post a snippet of the code you are
>> using?
>>
>> Thx
>>
>> J-D
>>
>> On Wed, Mar 17, 2010 at 11:25 PM, Alex Baranov <al...@gmail.com>
>> wrote:
>> > To give more clarity, I'm using *not custom* filter, but "standard"
>> > SingleColumnValueFilter. So it's not related to classpath issues.
>> >
>> > Any help is very appreciated!
>> >
>> > Thanks,
>> > Alex.
>> >
>> > On Wed, Mar 17, 2010 at 6:00 PM, Alex Baranov <alex.baranov.v@gmail.com
>> >wrote:
>> >
>> >> Hello guys,
>> >>
>> >> I've got a problem while adding a filter to scanner in a client app
>> which
>> >> runs on the remote (not the one from HBase cluster) box. The same code
>> works
>> >> well and scan result is fetched very quickly if I run the client on the
>> same
>> >> box where HBase master resides. If I comment out adding filter then the
>> >> scanner returns results. But with filter it keeps showing me the error
>> >> below.
>> >>
>> >> I'm using HBase 0.20.3 on both ends.
>> >>
>> >> On the mailing list I saw that problems like this can arise when using
>> >> different versions of HBase on server and on client, but this is not the
>> >> case. Also the error like this can show up when it takes a lot of time
>> to
>> >> initialize scanner (lease time by default is 1 min), but I assume this
>> is
>> >> also not the case since without adding filter I got results very
>> quickly.
>> >>
>> >> Does anyone have an idea what is going on?
>> >>
>> >> - in log on remote client side:
>> >>
>> >> Exception in thread "main"
>> >> org.apache.hadoop.hbase.client.RetriesExhaustedException: Trying to
>> contact
>> >> region server 10.210.71.80:39207 for region xxxxx,,1267450079067, row
>> >> 'yyyyy', but failed after 10 attempts.
>> >> Exceptions:
>> >> java.io.IOException: Call to /10.210.71.80:39207 failed on local
>> >> exception: java.io.EOFException
>> >> java.io.IOException: Call to /10.210.71.80:39207 failed on local
>> >> exception: java.io.EOFException
>> >> java.io.IOException: Call to /10.210.71.80:39207 failed on local
>> >> exception: java.io.EOFException
>> >> java.io.IOException: Call to /10.210.71.80:39207 failed on local
>> >> exception: java.io.EOFException
>> >> java.io.IOException: Call to /10.210.71.80:39207 failed on local
>> >> exception: java.io.EOFException
>> >> java.io.IOException: Call to /10.210.71.80:39207 failed on local
>> >> exception: java.io.EOFException
>> >> java.io.IOException: Call to /10.210.71.80:39207 failed on local
>> >> exception: java.io.EOFException
>> >> java.io.IOException: Call to /10.210.71.80:39207 failed on local
>> >> exception: java.io.EOFException
>> >> java.io.IOException: Call to /10.210.71.80:39207 failed on local
>> >> exception: java.io.EOFException
>> >> java.io.IOException: Call to /10.210.71.80:39207 failed on local
>> >> exception: java.io.EOFException
>> >>
>> >> at
>> >>
>> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getRegionServerWithRetries(HConnectionManager.java:1002)
>> >> at
>> >>
>> org.apache.hadoop.hbase.client.HTable$ClientScanner.nextScanner(HTable.java:1931)
>> >> at
>> >>
>> org.apache.hadoop.hbase.client.HTable$ClientScanner.initialize(HTable.java:1851)
>> >> at org.apache.hadoop.hbase.client.HTable.getScanner(HTable.java:372)
>> >> at
>> >>
>> com.sematext.sa.client.AggregatesAccessor.getResult(AggregatesAccessor.java:74)
>> >> at com.sematext.sa.client.ClientExample.main(ClientExample.java:41)
>> >>
>> >>
>> >> - in HBase master log:
>> >>
>> >> 2010-03-17 12:37:45,068 ERROR
>> >> org.apache.hadoop.hbase.regionserver.HRegionServer:
>> >> org.apache.hadoop.hbase.UnknownScannerException: Name: -1
>> >> at
>> >>
>> org.apache.hadoop.hbase.regionserver.HRegionServer.next(HRegionServer.java:1877)
>> >> at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source)
>> >> at
>> >>
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>> >> at java.lang.reflect.Method.invoke(Method.java:597)
>> >> at
>> >> org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:657)
>> >> at
>> >>
>> org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:915)
>> >>
>> >> - in HBase region server log/out: nothing
>> >>
>> >> Thank you in advance.
>> >> Alex.
>> >>
>> >
>>
>
Re: Adding filter to scan at remote client causes
UnknownScannerException
Posted by Alex Baranov <al...@gmail.com>.
Hello J-D,
Thanks for helping me out!
Here is the code that works if I run it on machine that has HBase master on
it and doesn't work on "remote" client box:
// CODE BEGINS
HBaseConfiguration conf = new HBaseConfiguration();
HTable hTable = new HTable(conf, "agg9");
Scan scan = new Scan();
scan.setStartRow(Bytes.toBytes("qf|byday_bytype_|14656__|"));
FilterList filters = new FilterList(FilterList.Operator.MUST_PASS_ALL);
SingleColumnValueFilter filter = new
SingleColumnValueFilter(Bytes.toBytes("agg"), Bytes.toBytes("count"),
CompareFilter.CompareOp.GREATER, Bytes.toBytes(35));
filters.addFilter(filter);
InclusiveStopFilter stopFilter = new
InclusiveStopFilter(Bytes.toBytes("qf|byday_|14739_|"));
filters.addFilter(stopFilter);
scan.setFilter(filters);
ResultScanner rs = hTable.getScanner(scan);
Result next = rs.next();
int readCount = 0;
while (next != null && readCount < 40) {
System.out.println("Row key: " + Bytes.toString(next.getRow()));
System.out.println("count: " +
Bytes.toInt(next.getValue(Bytes.toBytes("agg"), Bytes.toBytes("count"))));
next = rs.next();
readCount++;
}
// CODE ENDS
If I comment the line
filters.addFilter(filter);
then the code works on remote client box as well.
Client fails with the exception I provided previously. The bigger master log
(this is the log till the end and started when I ran the client code):
2010-03-19 12:15:02,098 INFO org.apache.hadoop.hbase.master.ServerManager: 1
region servers, 0 dead, average load 27.0
2010-03-19 12:15:41,369 INFO org.apache.hadoop.hbase.master.BaseScanner:
RegionManager.metaScanner scanning meta region {server: 10.210.71.80:39207,
regionname: .META.,,1, startKey: <>}
2010-03-19 12:15:41,398 INFO org.apache.hadoop.hbase.master.BaseScanner:
RegionManager.metaScanner scan of 25 row(s) of meta region {server:
10.210.71.80:39207, regionname: .META.,,1, startKey: <>} complete
2010-03-19 12:15:41,398 INFO org.apache.hadoop.hbase.master.BaseScanner: All
1 .META. region(s) scanned
2010-03-19 12:15:41,548 INFO org.apache.hadoop.hbase.master.BaseScanner:
RegionManager.rootScanner scanning meta region {server: 10.210.71.80:39207,
regionname: -ROOT-,,0, startKey: <>}
2010-03-19 12:15:41,549 INFO org.apache.hadoop.hbase.master.BaseScanner:
RegionManager.rootScanner scan of 1 row(s) of meta region {server:
10.210.71.80:39207, regionname: -ROOT-,,0, startKey: <>} complete
2010-03-19 12:15:45,398 DEBUG
org.apache.hadoop.hbase.io.hfile.LruBlockCache: Cache Stats: Sizes:
Total=43.474342MB (45586152), Free=156.21317MB (163801368), Max=199.6875MB
(209387520), Counts: Blocks=684, Access=90114, Hit=80498, Miss=9616,
Evictions=0, Evicted=0, Ratios: Hit Ratio=89.32906985282898%, Miss
Ratio=10.670927911996841%, Evicted/Run=NaN
2010-03-19 12:16:02,108 INFO org.apache.hadoop.hbase.master.ServerManager: 1
region servers, 0 dead, average load 27.0
2010-03-19 12:16:41,378 INFO org.apache.hadoop.hbase.master.BaseScanner:
RegionManager.metaScanner scanning meta region {server: 10.210.71.80:39207,
regionname: .META.,,1, startKey: <>}
2010-03-19 12:16:41,461 INFO org.apache.hadoop.hbase.master.BaseScanner:
RegionManager.metaScanner scan of 25 row(s) of meta region {server:
10.210.71.80:39207, regionname: .META.,,1, startKey: <>} complete
2010-03-19 12:16:41,461 INFO org.apache.hadoop.hbase.master.BaseScanner: All
1 .META. region(s) scanned
2010-03-19 12:16:41,558 INFO org.apache.hadoop.hbase.master.BaseScanner:
RegionManager.rootScanner scanning meta region {server: 10.210.71.80:39207,
regionname: -ROOT-,,0, startKey: <>}
2010-03-19 12:16:41,559 INFO org.apache.hadoop.hbase.master.BaseScanner:
RegionManager.rootScanner scan of 1 row(s) of meta region {server:
10.210.71.80:39207, regionname: -ROOT-,,0, startKey: <>} complete
2010-03-19 12:16:45,398 DEBUG
org.apache.hadoop.hbase.io.hfile.LruBlockCache: Cache Stats: Sizes:
Total=43.474342MB (45586152), Free=156.21317MB (163801368), Max=199.6875MB
(209387520), Counts: Blocks=684, Access=90116, Hit=80500, Miss=9616,
Evictions=0, Evicted=0, Ratios: Hit Ratio=89.32930827140808%, Miss
Ratio=10.67069098353386%, Evicted/Run=NaN
2010-03-19 12:16:59,872 ERROR
org.apache.hadoop.hbase.regionserver.HRegionServer:
org.apache.hadoop.hbase.UnknownScannerException: Name: -1
at
org.apache.hadoop.hbase.regionserver.HRegionServer.next(HRegionServer.java:1877)
at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:657)
at
org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:915)
2010-03-19 12:16:59,874 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server
handler 16 on 39207, call next(-1, 1) from 10.241.27.112:38002: error:
org.apache.hadoop.hbase.UnknownScannerException: Name: -1
org.apache.hadoop.hbase.UnknownScannerException: Name: -1
at
org.apache.hadoop.hbase.regionserver.HRegionServer.next(HRegionServer.java:1877)
at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:657)
at
org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:915)
2010-03-19 12:17:02,118 INFO org.apache.hadoop.hbase.master.ServerManager: 1
region servers, 0 dead, average load 27.0
2010-03-19 12:17:41,388 INFO org.apache.hadoop.hbase.master.BaseScanner:
RegionManager.metaScanner scanning meta region {server: 10.210.71.80:39207,
regionname: .META.,,1, startKey: <>}
2010-03-19 12:17:41,471 INFO org.apache.hadoop.hbase.master.BaseScanner:
RegionManager.metaScanner scan of 25 row(s) of meta region {server:
10.210.71.80:39207, regionname: .META.,,1, startKey: <>} complete
2010-03-19 12:17:41,471 INFO org.apache.hadoop.hbase.master.BaseScanner: All
1 .META. region(s) scanned
2010-03-19 12:17:41,568 INFO org.apache.hadoop.hbase.master.BaseScanner:
RegionManager.rootScanner scanning meta region {server: 10.210.71.80:39207,
regionname: -ROOT-,,0, startKey: <>}
2010-03-19 12:17:41,569 INFO org.apache.hadoop.hbase.master.BaseScanner:
RegionManager.rootScanner scan of 1 row(s) of meta region {server:
10.210.71.80:39207, regionname: -ROOT-,,0, startKey: <>} complete
2010-03-19 12:17:45,398 DEBUG
org.apache.hadoop.hbase.io.hfile.LruBlockCache: Cache Stats: Sizes:
Total=43.474342MB (45586152), Free=156.21317MB (163801368), Max=199.6875MB
(209387520), Counts: Blocks=684, Access=90118, Hit=80502, Miss=9616,
Evictions=0, Evicted=0, Ratios: Hit Ratio=89.32954668998718%, Miss
Ratio=10.670454055070877%, Evicted/Run=NaN
2010-03-19 12:18:02,128 INFO org.apache.hadoop.hbase.master.ServerManager: 1
region servers, 0 dead, average load 27.0
There is nothing in region log:
ubuntu@xxx:/mnt/hbase-0.20.3/logs$ ls -l | grep region
-rw-r--r-- 1 ubuntu ubuntu 2032 2010-02-24 05:51
hbase-ubuntu-regionserver-domU-12-31-39-09-40-A2.log
-rw-r--r-- 1 ubuntu ubuntu 0 2010-02-24 05:51
hbase-ubuntu-regionserver-domU-12-31-39-09-40-A2.out
-rw-r--r-- 1 ubuntu ubuntu 0 2010-02-04 16:04
hbase-ubuntu-regionserver-domU-12-31-39-09-40-A2.out.1
-rw-r--r-- 1 ubuntu ubuntu 0 2010-02-04 15:03
hbase-ubuntu-regionserver-domU-12-31-39-09-40-A2.out.2
-rw-r--r-- 1 ubuntu ubuntu 0 2010-02-04 14:37
hbase-ubuntu-regionserver-domU-12-31-39-09-40-A2.out.3
-rw-r--r-- 1 ubuntu ubuntu 0 2010-02-04 08:15
hbase-ubuntu-regionserver-domU-12-31-39-09-40-A2.out.4
-rw-r--r-- 1 ubuntu ubuntu 0 2010-02-04 07:56
hbase-ubuntu-regionserver-domU-12-31-39-09-40-A2.out.5
Also nothing in .out logs:
ubuntu@xxx:/mnt/hbase-0.20.3/logs$ ls -l | grep \.out
-rw-r--r-- 1 ubuntu ubuntu 324 2010-02-24 05:51
hbase-ubuntu-master-domU-12-31-39-09-40-A2.out
-rw-r--r-- 1 ubuntu ubuntu 0 2010-02-24 05:50
hbase-ubuntu-master-domU-12-31-39-09-40-A2.out.1
-rw-r--r-- 1 ubuntu ubuntu 0 2010-02-04 16:01
hbase-ubuntu-master-domU-12-31-39-09-40-A2.out.2
-rw-r--r-- 1 ubuntu ubuntu 0 2010-02-04 14:56
hbase-ubuntu-master-domU-12-31-39-09-40-A2.out.3
-rw-r--r-- 1 ubuntu ubuntu 0 2010-02-04 14:30
hbase-ubuntu-master-domU-12-31-39-09-40-A2.out.4
-rw-r--r-- 1 ubuntu ubuntu 0 2010-02-04 08:07
hbase-ubuntu-master-domU-12-31-39-09-40-A2.out.5
-rw-r--r-- 1 ubuntu ubuntu 0 2010-02-24 05:51
hbase-ubuntu-regionserver-domU-12-31-39-09-40-A2.out
-rw-r--r-- 1 ubuntu ubuntu 0 2010-02-04 16:04
hbase-ubuntu-regionserver-domU-12-31-39-09-40-A2.out.1
-rw-r--r-- 1 ubuntu ubuntu 0 2010-02-04 15:03
hbase-ubuntu-regionserver-domU-12-31-39-09-40-A2.out.2
-rw-r--r-- 1 ubuntu ubuntu 0 2010-02-04 14:37
hbase-ubuntu-regionserver-domU-12-31-39-09-40-A2.out.3
-rw-r--r-- 1 ubuntu ubuntu 0 2010-02-04 08:15
hbase-ubuntu-regionserver-domU-12-31-39-09-40-A2.out.4
-rw-r--r-- 1 ubuntu ubuntu 0 2010-02-04 07:56
hbase-ubuntu-regionserver-domU-12-31-39-09-40-A2.out.5
-rw-r--r-- 1 ubuntu ubuntu 0 2010-02-24 05:51
hbase-ubuntu-zookeeper-domU-12-31-39-09-40-A2.out
-rw-r--r-- 1 ubuntu ubuntu 0 2010-02-04 16:04
hbase-ubuntu-zookeeper-domU-12-31-39-09-40-A2.out.1
-rw-r--r-- 1 ubuntu ubuntu 0 2010-02-04 15:03
hbase-ubuntu-zookeeper-domU-12-31-39-09-40-A2.out.2
-rw-r--r-- 1 ubuntu ubuntu 0 2010-02-04 14:37
hbase-ubuntu-zookeeper-domU-12-31-39-09-40-A2.out.3
-rw-r--r-- 1 ubuntu ubuntu 0 2010-02-04 08:15
hbase-ubuntu-zookeeper-domU-12-31-39-09-40-A2.out.4
-rw-r--r-- 1 ubuntu ubuntu 0 2010-02-04 07:56
hbase-ubuntu-zookeeper-domU-12-31-39-09-40-A2.out.5
Is there anything else I can provide you?
The command I'm using to run the client is the following (you can see that
0.20.3 version of HBase is used and also other versions you might be
interested in):
java -cp
commons-cli-1.2.jar:commons-logging-1.1.1.jar:hadoop-0.20.1-core.jar:hbase-0.20.3.jar:hbase-0.20.3-test.jar:log4j-1.2.15.jar:test-1.0-SNAPSHOT.jar:zookeeper-3.2.2.jar
com.foo.bar.client.ClientExample
Thank you for your help!
Alex
On Thu, Mar 18, 2010 at 7:13 PM, Jean-Daniel Cryans <jd...@apache.org>wrote:
> Alex,
>
> Is there anything else in the region server logs before that like
> lease expirations? Can we see a much bigger log? Also is there
> anything in the .out file? Can you post a snippet of the code you are
> using?
>
> Thx
>
> J-D
>
> On Wed, Mar 17, 2010 at 11:25 PM, Alex Baranov <al...@gmail.com>
> wrote:
> > To give more clarity, I'm using *not custom* filter, but "standard"
> > SingleColumnValueFilter. So it's not related to classpath issues.
> >
> > Any help is very appreciated!
> >
> > Thanks,
> > Alex.
> >
> > On Wed, Mar 17, 2010 at 6:00 PM, Alex Baranov <alex.baranov.v@gmail.com
> >wrote:
> >
> >> Hello guys,
> >>
> >> I've got a problem while adding a filter to scanner in a client app
> which
> >> runs on the remote (not the one from HBase cluster) box. The same code
> works
> >> well and scan result is fetched very quickly if I run the client on the
> same
> >> box where HBase master resides. If I comment out adding filter then the
> >> scanner returns results. But with filter it keeps showing me the error
> >> below.
> >>
> >> I'm using HBase 0.20.3 on both ends.
> >>
> >> On the mailing list I saw that problems like this can arise when using
> >> different versions of HBase on server and on client, but this is not the
> >> case. Also the error like this can show up when it takes a lot of time
> to
> >> initialize scanner (lease time by default is 1 min), but I assume this
> is
> >> also not the case since without adding filter I got results very
> quickly.
> >>
> >> Does anyone have an idea what is going on?
> >>
> >> - in log on remote client side:
> >>
> >> Exception in thread "main"
> >> org.apache.hadoop.hbase.client.RetriesExhaustedException: Trying to
> contact
> >> region server 10.210.71.80:39207 for region xxxxx,,1267450079067, row
> >> 'yyyyy', but failed after 10 attempts.
> >> Exceptions:
> >> java.io.IOException: Call to /10.210.71.80:39207 failed on local
> >> exception: java.io.EOFException
> >> java.io.IOException: Call to /10.210.71.80:39207 failed on local
> >> exception: java.io.EOFException
> >> java.io.IOException: Call to /10.210.71.80:39207 failed on local
> >> exception: java.io.EOFException
> >> java.io.IOException: Call to /10.210.71.80:39207 failed on local
> >> exception: java.io.EOFException
> >> java.io.IOException: Call to /10.210.71.80:39207 failed on local
> >> exception: java.io.EOFException
> >> java.io.IOException: Call to /10.210.71.80:39207 failed on local
> >> exception: java.io.EOFException
> >> java.io.IOException: Call to /10.210.71.80:39207 failed on local
> >> exception: java.io.EOFException
> >> java.io.IOException: Call to /10.210.71.80:39207 failed on local
> >> exception: java.io.EOFException
> >> java.io.IOException: Call to /10.210.71.80:39207 failed on local
> >> exception: java.io.EOFException
> >> java.io.IOException: Call to /10.210.71.80:39207 failed on local
> >> exception: java.io.EOFException
> >>
> >> at
> >>
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getRegionServerWithRetries(HConnectionManager.java:1002)
> >> at
> >>
> org.apache.hadoop.hbase.client.HTable$ClientScanner.nextScanner(HTable.java:1931)
> >> at
> >>
> org.apache.hadoop.hbase.client.HTable$ClientScanner.initialize(HTable.java:1851)
> >> at org.apache.hadoop.hbase.client.HTable.getScanner(HTable.java:372)
> >> at
> >>
> com.sematext.sa.client.AggregatesAccessor.getResult(AggregatesAccessor.java:74)
> >> at com.sematext.sa.client.ClientExample.main(ClientExample.java:41)
> >>
> >>
> >> - in HBase master log:
> >>
> >> 2010-03-17 12:37:45,068 ERROR
> >> org.apache.hadoop.hbase.regionserver.HRegionServer:
> >> org.apache.hadoop.hbase.UnknownScannerException: Name: -1
> >> at
> >>
> org.apache.hadoop.hbase.regionserver.HRegionServer.next(HRegionServer.java:1877)
> >> at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source)
> >> at
> >>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> >> at java.lang.reflect.Method.invoke(Method.java:597)
> >> at
> >> org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:657)
> >> at
> >>
> org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:915)
> >>
> >> - in HBase region server log/out: nothing
> >>
> >> Thank you in advance.
> >> Alex.
> >>
> >
>
Re: Adding filter to scan at remote client causes
UnknownScannerException
Posted by Jean-Daniel Cryans <jd...@apache.org>.
Alex,
Is there anything else in the region server logs before that like
lease expirations? Can we see a much bigger log? Also is there
anything in the .out file? Can you post a snippet of the code you are
using?
Thx
J-D
On Wed, Mar 17, 2010 at 11:25 PM, Alex Baranov <al...@gmail.com> wrote:
> To give more clarity, I'm using *not custom* filter, but "standard"
> SingleColumnValueFilter. So it's not related to classpath issues.
>
> Any help is very appreciated!
>
> Thanks,
> Alex.
>
> On Wed, Mar 17, 2010 at 6:00 PM, Alex Baranov <al...@gmail.com>wrote:
>
>> Hello guys,
>>
>> I've got a problem while adding a filter to scanner in a client app which
>> runs on the remote (not the one from HBase cluster) box. The same code works
>> well and scan result is fetched very quickly if I run the client on the same
>> box where HBase master resides. If I comment out adding filter then the
>> scanner returns results. But with filter it keeps showing me the error
>> below.
>>
>> I'm using HBase 0.20.3 on both ends.
>>
>> On the mailing list I saw that problems like this can arise when using
>> different versions of HBase on server and on client, but this is not the
>> case. Also the error like this can show up when it takes a lot of time to
>> initialize scanner (lease time by default is 1 min), but I assume this is
>> also not the case since without adding filter I got results very quickly.
>>
>> Does anyone have an idea what is going on?
>>
>> - in log on remote client side:
>>
>> Exception in thread "main"
>> org.apache.hadoop.hbase.client.RetriesExhaustedException: Trying to contact
>> region server 10.210.71.80:39207 for region xxxxx,,1267450079067, row
>> 'yyyyy', but failed after 10 attempts.
>> Exceptions:
>> java.io.IOException: Call to /10.210.71.80:39207 failed on local
>> exception: java.io.EOFException
>> java.io.IOException: Call to /10.210.71.80:39207 failed on local
>> exception: java.io.EOFException
>> java.io.IOException: Call to /10.210.71.80:39207 failed on local
>> exception: java.io.EOFException
>> java.io.IOException: Call to /10.210.71.80:39207 failed on local
>> exception: java.io.EOFException
>> java.io.IOException: Call to /10.210.71.80:39207 failed on local
>> exception: java.io.EOFException
>> java.io.IOException: Call to /10.210.71.80:39207 failed on local
>> exception: java.io.EOFException
>> java.io.IOException: Call to /10.210.71.80:39207 failed on local
>> exception: java.io.EOFException
>> java.io.IOException: Call to /10.210.71.80:39207 failed on local
>> exception: java.io.EOFException
>> java.io.IOException: Call to /10.210.71.80:39207 failed on local
>> exception: java.io.EOFException
>> java.io.IOException: Call to /10.210.71.80:39207 failed on local
>> exception: java.io.EOFException
>>
>> at
>> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getRegionServerWithRetries(HConnectionManager.java:1002)
>> at
>> org.apache.hadoop.hbase.client.HTable$ClientScanner.nextScanner(HTable.java:1931)
>> at
>> org.apache.hadoop.hbase.client.HTable$ClientScanner.initialize(HTable.java:1851)
>> at org.apache.hadoop.hbase.client.HTable.getScanner(HTable.java:372)
>> at
>> com.sematext.sa.client.AggregatesAccessor.getResult(AggregatesAccessor.java:74)
>> at com.sematext.sa.client.ClientExample.main(ClientExample.java:41)
>>
>>
>> - in HBase master log:
>>
>> 2010-03-17 12:37:45,068 ERROR
>> org.apache.hadoop.hbase.regionserver.HRegionServer:
>> org.apache.hadoop.hbase.UnknownScannerException: Name: -1
>> at
>> org.apache.hadoop.hbase.regionserver.HRegionServer.next(HRegionServer.java:1877)
>> at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source)
>> at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>> at java.lang.reflect.Method.invoke(Method.java:597)
>> at
>> org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:657)
>> at
>> org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:915)
>>
>> - in HBase region server log/out: nothing
>>
>> Thank you in advance.
>> Alex.
>>
>