You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hbase.apache.org by "PeterL in (JIRA)" <ji...@apache.org> on 2008/08/13 13:34:48 UTC
[jira] Created: (HBASE-827) Failed to open scanner in shell after
series of action executed
Failed to open scanner in shell after series of action executed
---------------------------------------------------------------
Key: HBASE-827
URL: https://issues.apache.org/jira/browse/HBASE-827
Project: Hadoop HBase
Issue Type: Bug
Affects Versions: 0.3.0
Environment: Java runtime version : 1.6.0_06
Hbase version : occured in both 0.2.0 RC2 & 0.3.0 trunk (svn 685494)
Hbase configuration: run in localhost without hadoop
Reporter: PeterL in
Background introduction :
I was intent to implement a large queue-liked structure based on HBase.
Different from normal queue structure, in my structure, a pop action means 'randomly' pick one elements(which is key/value pair from a row) in queue, and decrease the queueSize of that queue.
For performance consideration, I implemented pop action by following steps :
1.get the queue size
2.random a number between 0~queueSize
3.take step2 as a qualifier, get the value of that element, and put the last element's value to the original qualifier.
4.decrease the queue size. (I didn't delete the last elements since storage space is not my primary consideration)
When the queueSize and number of pop action become large, hbase failed to open a scanner. (Even the data of database is correct)
My program can be split into few steps:
1.create a large queue.
2.pop until the queue empty
3.keep popping the queue
4.delete the queue ( delete a row with deleteAll )
5.create a small queue with same name (i.e: same row)
After create a small queue, I checked every key/value in the hbase and found it is correct. But when I tried to open a scanner, it got Unknown Exception as follows:
In shell :
NativeException: org.apache.hadoop.hbase.client.RetriesExhaustedException: Trying to contact region server 127.0.0.1:54141 for region pleaseCrash,,1218621377152, row '', but failed after 3 attempts.
Exceptions:
java.net.SocketTimeoutException: timed out waiting for rpc response
java.net.SocketTimeoutException: timed out waiting for rpc response
java.net.SocketTimeoutException: timed out waiting for rpc response
from org/apache/hadoop/hbase/client/HConnectionManager.java:873:in `getRegionServerWithRetries'
from org/apache/hadoop/hbase/client/HTable.java:1415:in `nextScanner'
from org/apache/hadoop/hbase/client/HTable.java:1360:in `<init>'
from org/apache/hadoop/hbase/client/HTable.java:985:in `getScanner'
from org/apache/hadoop/hbase/client/HTable.java:779:in `getScanner'
from org/apache/hadoop/hbase/client/HTable.java:706:in `getScanner'
from sun/reflect/NativeMethodAccessorImpl.java:-2:in `invoke0'
from sun/reflect/NativeMethodAccessorImpl.java:39:in `invoke'
from sun/reflect/DelegatingMethodAccessorImpl.java:25:in `invoke'
from java/lang/reflect/Method.java:597:in `invoke'
from org/jruby/javasupport/JavaMethod.java:250:in `invokeWithExceptionHandling'
from org/jruby/javasupport/JavaMethod.java:219:in `invoke'
from org/jruby/javasupport/JavaClass.java:416:in `execute'
from org/jruby/internal/runtime/methods/SimpleCallbackMethod.java:67:in `call'
from org/jruby/internal/runtime/methods/DynamicMethod.java:78:in `call'
from org/jruby/runtime/CallSite.java:155:in `cacheAndCall'
... 128 levels...
from ruby.home.itspeter.hbase_minus_0_dot_3_dot_0_minus_dev.bin.hirbInvokermethod__23$RUBY$startOpt:-1:in `call'
from org/jruby/internal/runtime/methods/DynamicMethod.java:74:in `call'
from org/jruby/internal/runtime/methods/CompiledMethod.java:48:in `call'
from org/jruby/runtime/CallSite.java:123:in `cacheAndCall'
from org/jruby/runtime/CallSite.java:298:in `call'
from ruby/home/itspeter/hbase_minus_0_dot_3_dot_0_minus_dev/bin//home/itspeter/hbase-0.3.0-dev/bin/../bin/hirb.rb:350:in `__file__'
from ruby/home/itspeter/hbase_minus_0_dot_3_dot_0_minus_dev/bin//home/itspeter/hbase-0.3.0-dev/bin/../bin/hirb.rb:-1:in `__file__'
from ruby/home/itspeter/hbase_minus_0_dot_3_dot_0_minus_dev/bin//home/itspeter/hbase-0.3.0-dev/bin/../bin/hirb.rb:-1:in `load'
from org/jruby/Ruby.java:512:in `runScript'
from org/jruby/Ruby.java:432:in `runNormally'
from org/jruby/Ruby.java:312:in `runFromMain'
from org/jruby/Main.java:144:in `run'
from org/jruby/Main.java:89:in `run'
from org/jruby/Main.java:80:in `main'
from /home/itspeter/hbase-0.3.0-dev/bin/../bin/hirb.rb:271:in `scan'
from (hbase):3:in `binding'
Side effects :
After scan got time out, I discovered that I also failed to disable the table (But not every time), somehow more strange is that table will be dropped successfully even disable failed (but not every time).
System Resources:
Hbase will rob the cpu...
In my platform(4-core), cpu usage will raise from 10% -> 50% ..100% -> 200% -> 300% ..
Procedure to reproduce :
Run the attached java file.
Try to type "scan 'pleaseCrash'" in hbase shell after program finished.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-827) Failed to open scanner in shell after
series of action executed
Posted by "PeterL in (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
PeterL in updated HBASE-827:
----------------------------
Attachment: MyClient.java
Code to reproduce problem
> Failed to open scanner in shell after series of action executed
> ---------------------------------------------------------------
>
> Key: HBASE-827
> URL: https://issues.apache.org/jira/browse/HBASE-827
> Project: Hadoop HBase
> Issue Type: Bug
> Affects Versions: 0.3.0
> Environment: Java runtime version : 1.6.0_06
> Hbase version : occured in both 0.2.0 RC2 & 0.3.0 trunk (svn 685494)
> Hbase configuration: run in localhost without hadoop
> Reporter: PeterL in
> Attachments: MyClient.java
>
>
> Background introduction :
> I was intent to implement a large queue-liked structure based on HBase.
> Different from normal queue structure, in my structure, a pop action means 'randomly' pick one elements(which is key/value pair from a row) in queue, and decrease the queueSize of that queue.
> For performance consideration, I implemented pop action by following steps :
> 1.get the queue size
> 2.random a number between 0~queueSize
> 3.take step2 as a qualifier, get the value of that element, and put the last element's value to the original qualifier.
> 4.decrease the queue size. (I didn't delete the last elements since storage space is not my primary consideration)
> When the queueSize and number of pop action become large, hbase failed to open a scanner. (Even the data of database is correct)
> My program can be split into few steps:
> 1.create a large queue.
> 2.pop until the queue empty
> 3.keep popping the queue
> 4.delete the queue ( delete a row with deleteAll )
> 5.create a small queue with same name (i.e: same row)
> After create a small queue, I checked every key/value in the hbase and found it is correct. But when I tried to open a scanner, it got Unknown Exception as follows:
> In shell :
> NativeException: org.apache.hadoop.hbase.client.RetriesExhaustedException: Trying to contact region server 127.0.0.1:54141 for region pleaseCrash,,1218621377152, row '', but failed after 3 attempts.
> Exceptions:
> java.net.SocketTimeoutException: timed out waiting for rpc response
> java.net.SocketTimeoutException: timed out waiting for rpc response
> java.net.SocketTimeoutException: timed out waiting for rpc response
> from org/apache/hadoop/hbase/client/HConnectionManager.java:873:in `getRegionServerWithRetries'
> from org/apache/hadoop/hbase/client/HTable.java:1415:in `nextScanner'
> from org/apache/hadoop/hbase/client/HTable.java:1360:in `<init>'
> from org/apache/hadoop/hbase/client/HTable.java:985:in `getScanner'
> from org/apache/hadoop/hbase/client/HTable.java:779:in `getScanner'
> from org/apache/hadoop/hbase/client/HTable.java:706:in `getScanner'
> from sun/reflect/NativeMethodAccessorImpl.java:-2:in `invoke0'
> from sun/reflect/NativeMethodAccessorImpl.java:39:in `invoke'
> from sun/reflect/DelegatingMethodAccessorImpl.java:25:in `invoke'
> from java/lang/reflect/Method.java:597:in `invoke'
> from org/jruby/javasupport/JavaMethod.java:250:in `invokeWithExceptionHandling'
> from org/jruby/javasupport/JavaMethod.java:219:in `invoke'
> from org/jruby/javasupport/JavaClass.java:416:in `execute'
> from org/jruby/internal/runtime/methods/SimpleCallbackMethod.java:67:in `call'
> from org/jruby/internal/runtime/methods/DynamicMethod.java:78:in `call'
> from org/jruby/runtime/CallSite.java:155:in `cacheAndCall'
> ... 128 levels...
> from ruby.home.itspeter.hbase_minus_0_dot_3_dot_0_minus_dev.bin.hirbInvokermethod__23$RUBY$startOpt:-1:in `call'
> from org/jruby/internal/runtime/methods/DynamicMethod.java:74:in `call'
> from org/jruby/internal/runtime/methods/CompiledMethod.java:48:in `call'
> from org/jruby/runtime/CallSite.java:123:in `cacheAndCall'
> from org/jruby/runtime/CallSite.java:298:in `call'
> from ruby/home/itspeter/hbase_minus_0_dot_3_dot_0_minus_dev/bin//home/itspeter/hbase-0.3.0-dev/bin/../bin/hirb.rb:350:in `__file__'
> from ruby/home/itspeter/hbase_minus_0_dot_3_dot_0_minus_dev/bin//home/itspeter/hbase-0.3.0-dev/bin/../bin/hirb.rb:-1:in `__file__'
> from ruby/home/itspeter/hbase_minus_0_dot_3_dot_0_minus_dev/bin//home/itspeter/hbase-0.3.0-dev/bin/../bin/hirb.rb:-1:in `load'
> from org/jruby/Ruby.java:512:in `runScript'
> from org/jruby/Ruby.java:432:in `runNormally'
> from org/jruby/Ruby.java:312:in `runFromMain'
> from org/jruby/Main.java:144:in `run'
> from org/jruby/Main.java:89:in `run'
> from org/jruby/Main.java:80:in `main'
> from /home/itspeter/hbase-0.3.0-dev/bin/../bin/hirb.rb:271:in `scan'
> from (hbase):3:in `binding'
> Side effects :
> After scan got time out, I discovered that I also failed to disable the table (But not every time), somehow more strange is that table will be dropped successfully even disable failed (but not every time).
> System Resources:
> Hbase will rob the cpu...
> In my platform(4-core), cpu usage will raise from 10% -> 50% ..100% -> 200% -> 300% ..
> Procedure to reproduce :
> Run the attached java file.
> Try to type "scan 'pleaseCrash'" in hbase shell after program finished.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-827) Failed to open scanner in shell after
series of action executed
Posted by "PeterL in (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
PeterL in updated HBASE-827:
----------------------------
Attachment: jstack.log
> Failed to open scanner in shell after series of action executed
> ---------------------------------------------------------------
>
> Key: HBASE-827
> URL: https://issues.apache.org/jira/browse/HBASE-827
> Project: Hadoop HBase
> Issue Type: Bug
> Affects Versions: 0.3.0
> Environment: Java runtime version : 1.6.0_06
> Hbase version : occured in both 0.2.0 RC2 & 0.3.0 trunk (svn 685494)
> Hbase configuration: run in localhost without hadoop
> Reporter: PeterL in
> Attachments: jstack.log, MyClient.java
>
>
> Background introduction :
> I was intent to implement a large queue-liked structure based on HBase.
> Different from normal queue structure, in my structure, a pop action means 'randomly' pick one elements(which is key/value pair from a row) in queue, and decrease the queueSize of that queue.
> For performance consideration, I implemented pop action by following steps :
> 1.get the queue size
> 2.random a number between 0~queueSize
> 3.take step2 as a qualifier, get the value of that element, and put the last element's value to the original qualifier.
> 4.decrease the queue size. (I didn't delete the last elements since storage space is not my primary consideration)
> When the queueSize and number of pop action become large, hbase failed to open a scanner. (Even the data of database is correct)
> My program can be split into few steps:
> 1.create a large queue.
> 2.pop until the queue empty
> 3.keep popping the queue
> 4.delete the queue ( delete a row with deleteAll )
> 5.create a small queue with same name (i.e: same row)
> After create a small queue, I checked every key/value in the hbase and found it is correct. But when I tried to open a scanner, it got Unknown Exception as follows:
> In shell :
> NativeException: org.apache.hadoop.hbase.client.RetriesExhaustedException: Trying to contact region server 127.0.0.1:54141 for region pleaseCrash,,1218621377152, row '', but failed after 3 attempts.
> Exceptions:
> java.net.SocketTimeoutException: timed out waiting for rpc response
> java.net.SocketTimeoutException: timed out waiting for rpc response
> java.net.SocketTimeoutException: timed out waiting for rpc response
> from org/apache/hadoop/hbase/client/HConnectionManager.java:873:in `getRegionServerWithRetries'
> from org/apache/hadoop/hbase/client/HTable.java:1415:in `nextScanner'
> from org/apache/hadoop/hbase/client/HTable.java:1360:in `<init>'
> from org/apache/hadoop/hbase/client/HTable.java:985:in `getScanner'
> from org/apache/hadoop/hbase/client/HTable.java:779:in `getScanner'
> from org/apache/hadoop/hbase/client/HTable.java:706:in `getScanner'
> from sun/reflect/NativeMethodAccessorImpl.java:-2:in `invoke0'
> from sun/reflect/NativeMethodAccessorImpl.java:39:in `invoke'
> from sun/reflect/DelegatingMethodAccessorImpl.java:25:in `invoke'
> from java/lang/reflect/Method.java:597:in `invoke'
> from org/jruby/javasupport/JavaMethod.java:250:in `invokeWithExceptionHandling'
> from org/jruby/javasupport/JavaMethod.java:219:in `invoke'
> from org/jruby/javasupport/JavaClass.java:416:in `execute'
> from org/jruby/internal/runtime/methods/SimpleCallbackMethod.java:67:in `call'
> from org/jruby/internal/runtime/methods/DynamicMethod.java:78:in `call'
> from org/jruby/runtime/CallSite.java:155:in `cacheAndCall'
> ... 128 levels...
> from ruby.home.itspeter.hbase_minus_0_dot_3_dot_0_minus_dev.bin.hirbInvokermethod__23$RUBY$startOpt:-1:in `call'
> from org/jruby/internal/runtime/methods/DynamicMethod.java:74:in `call'
> from org/jruby/internal/runtime/methods/CompiledMethod.java:48:in `call'
> from org/jruby/runtime/CallSite.java:123:in `cacheAndCall'
> from org/jruby/runtime/CallSite.java:298:in `call'
> from ruby/home/itspeter/hbase_minus_0_dot_3_dot_0_minus_dev/bin//home/itspeter/hbase-0.3.0-dev/bin/../bin/hirb.rb:350:in `__file__'
> from ruby/home/itspeter/hbase_minus_0_dot_3_dot_0_minus_dev/bin//home/itspeter/hbase-0.3.0-dev/bin/../bin/hirb.rb:-1:in `__file__'
> from ruby/home/itspeter/hbase_minus_0_dot_3_dot_0_minus_dev/bin//home/itspeter/hbase-0.3.0-dev/bin/../bin/hirb.rb:-1:in `load'
> from org/jruby/Ruby.java:512:in `runScript'
> from org/jruby/Ruby.java:432:in `runNormally'
> from org/jruby/Ruby.java:312:in `runFromMain'
> from org/jruby/Main.java:144:in `run'
> from org/jruby/Main.java:89:in `run'
> from org/jruby/Main.java:80:in `main'
> from /home/itspeter/hbase-0.3.0-dev/bin/../bin/hirb.rb:271:in `scan'
> from (hbase):3:in `binding'
> Side effects :
> After scan got time out, I discovered that I also failed to disable the table (But not every time), somehow more strange is that table will be dropped successfully even disable failed (but not every time).
> System Resources:
> Hbase will rob the cpu...
> In my platform(4-core), cpu usage will raise from 10% -> 50% ..100% -> 200% -> 300% ..
> Procedure to reproduce :
> Run the attached java file.
> Try to type "scan 'pleaseCrash'" in hbase shell after program finished.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-827) Failed to open scanner in shell after
series of action executed
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12623228#action_12623228 ]
stack commented on HBASE-827:
-----------------------------
(Thanks for digging in Rong-en)
I set the timeouts and retries down from the default in the shell to try and make it more 'responsive' (one thing having a program hang round but another asking a human do the same).
'@configuration' is the global HBaseConfiguration for the hirb shell and it seems easy enough changing the shell configuration to be something else:
{code}
durruti:~/Documents/checkouts/hbase/trunk stack$ ./bin/hbase shell
HBase Shell; enter 'help<RETURN>' for list of supported commands.
Version: 0.3.0-dev, r686335, Fri Aug 15 16:42:18 PDT 2008
hbase(main):001:0> @configuration.get('hbase.client.retries.number')
=> "3"
hbase(main):003:0> @configuration.setInt('hbase.client.retries.number', 10)
hbase(main):005:0> @configuration.get('hbase.client.retries.number')
=> "10"
{code}
> Failed to open scanner in shell after series of action executed
> ---------------------------------------------------------------
>
> Key: HBASE-827
> URL: https://issues.apache.org/jira/browse/HBASE-827
> Project: Hadoop HBase
> Issue Type: Bug
> Affects Versions: 0.3.0
> Environment: Java runtime version : 1.6.0_06
> Hbase version : occured in both 0.2.0 RC2 & 0.3.0 trunk (svn 685494)
> Hbase configuration: run in localhost without hadoop
> Reporter: PeterL in
> Attachments: hbase.log, jstack.log, MyClient.java
>
>
> Background introduction :
> I was intent to implement a large queue-liked structure based on HBase.
> Different from normal queue structure, in my structure, a pop action means 'randomly' pick one elements(which is key/value pair from a row) in queue, and decrease the queueSize of that queue.
> For performance consideration, I implemented pop action by following steps :
> 1.get the queue size
> 2.random a number between 0~queueSize
> 3.take step2 as a qualifier, get the value of that element, and put the last element's value to the original qualifier.
> 4.decrease the queue size. (I didn't delete the last elements since storage space is not my primary consideration)
> When the queueSize and number of pop action become large, hbase failed to open a scanner. (Even the data of database is correct)
> My program can be split into few steps:
> 1.create a large queue.
> 2.pop until the queue empty
> 3.keep popping the queue
> 4.delete the queue ( delete a row with deleteAll )
> 5.create a small queue with same name (i.e: same row)
> After create a small queue, I checked every key/value in the hbase and found it is correct. But when I tried to open a scanner, it got Unknown Exception as follows:
> In shell :
> NativeException: org.apache.hadoop.hbase.client.RetriesExhaustedException: Trying to contact region server 127.0.0.1:54141 for region pleaseCrash,,1218621377152, row '', but failed after 3 attempts.
> Exceptions:
> java.net.SocketTimeoutException: timed out waiting for rpc response
> java.net.SocketTimeoutException: timed out waiting for rpc response
> java.net.SocketTimeoutException: timed out waiting for rpc response
> from org/apache/hadoop/hbase/client/HConnectionManager.java:873:in `getRegionServerWithRetries'
> from org/apache/hadoop/hbase/client/HTable.java:1415:in `nextScanner'
> from org/apache/hadoop/hbase/client/HTable.java:1360:in `<init>'
> from org/apache/hadoop/hbase/client/HTable.java:985:in `getScanner'
> from org/apache/hadoop/hbase/client/HTable.java:779:in `getScanner'
> from org/apache/hadoop/hbase/client/HTable.java:706:in `getScanner'
> from sun/reflect/NativeMethodAccessorImpl.java:-2:in `invoke0'
> from sun/reflect/NativeMethodAccessorImpl.java:39:in `invoke'
> from sun/reflect/DelegatingMethodAccessorImpl.java:25:in `invoke'
> from java/lang/reflect/Method.java:597:in `invoke'
> from org/jruby/javasupport/JavaMethod.java:250:in `invokeWithExceptionHandling'
> from org/jruby/javasupport/JavaMethod.java:219:in `invoke'
> from org/jruby/javasupport/JavaClass.java:416:in `execute'
> from org/jruby/internal/runtime/methods/SimpleCallbackMethod.java:67:in `call'
> from org/jruby/internal/runtime/methods/DynamicMethod.java:78:in `call'
> from org/jruby/runtime/CallSite.java:155:in `cacheAndCall'
> ... 128 levels...
> from ruby.home.itspeter.hbase_minus_0_dot_3_dot_0_minus_dev.bin.hirbInvokermethod__23$RUBY$startOpt:-1:in `call'
> from org/jruby/internal/runtime/methods/DynamicMethod.java:74:in `call'
> from org/jruby/internal/runtime/methods/CompiledMethod.java:48:in `call'
> from org/jruby/runtime/CallSite.java:123:in `cacheAndCall'
> from org/jruby/runtime/CallSite.java:298:in `call'
> from ruby/home/itspeter/hbase_minus_0_dot_3_dot_0_minus_dev/bin//home/itspeter/hbase-0.3.0-dev/bin/../bin/hirb.rb:350:in `__file__'
> from ruby/home/itspeter/hbase_minus_0_dot_3_dot_0_minus_dev/bin//home/itspeter/hbase-0.3.0-dev/bin/../bin/hirb.rb:-1:in `__file__'
> from ruby/home/itspeter/hbase_minus_0_dot_3_dot_0_minus_dev/bin//home/itspeter/hbase-0.3.0-dev/bin/../bin/hirb.rb:-1:in `load'
> from org/jruby/Ruby.java:512:in `runScript'
> from org/jruby/Ruby.java:432:in `runNormally'
> from org/jruby/Ruby.java:312:in `runFromMain'
> from org/jruby/Main.java:144:in `run'
> from org/jruby/Main.java:89:in `run'
> from org/jruby/Main.java:80:in `main'
> from /home/itspeter/hbase-0.3.0-dev/bin/../bin/hirb.rb:271:in `scan'
> from (hbase):3:in `binding'
> Side effects :
> After scan got time out, I discovered that I also failed to disable the table (But not every time), somehow more strange is that table will be dropped successfully even disable failed (but not every time).
> System Resources:
> Hbase will rob the cpu...
> In my platform(4-core), cpu usage will raise from 10% -> 50% ..100% -> 200% -> 300% ..
> Procedure to reproduce :
> Run the attached java file.
> Try to type "scan 'pleaseCrash'" in hbase shell after program finished.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Issue Comment Edited: (HBASE-827) Failed to open scanner in
shell after series of action executed
Posted by "PeterL in (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12622179#action_12622179 ]
itspeter edited comment on HBASE-827 at 8/13/08 4:39 AM:
----------------------------------------------------------
Code to reproduce problem : MyClient.java
jstack log : jstack.log
Hbase log : hbase.log (From empty database to scanner failed)
was (Author: itspeter):
Code to reproduce problem : MyClient.java
> Failed to open scanner in shell after series of action executed
> ---------------------------------------------------------------
>
> Key: HBASE-827
> URL: https://issues.apache.org/jira/browse/HBASE-827
> Project: Hadoop HBase
> Issue Type: Bug
> Affects Versions: 0.3.0
> Environment: Java runtime version : 1.6.0_06
> Hbase version : occured in both 0.2.0 RC2 & 0.3.0 trunk (svn 685494)
> Hbase configuration: run in localhost without hadoop
> Reporter: PeterL in
> Attachments: jstack.log, MyClient.java
>
>
> Background introduction :
> I was intent to implement a large queue-liked structure based on HBase.
> Different from normal queue structure, in my structure, a pop action means 'randomly' pick one elements(which is key/value pair from a row) in queue, and decrease the queueSize of that queue.
> For performance consideration, I implemented pop action by following steps :
> 1.get the queue size
> 2.random a number between 0~queueSize
> 3.take step2 as a qualifier, get the value of that element, and put the last element's value to the original qualifier.
> 4.decrease the queue size. (I didn't delete the last elements since storage space is not my primary consideration)
> When the queueSize and number of pop action become large, hbase failed to open a scanner. (Even the data of database is correct)
> My program can be split into few steps:
> 1.create a large queue.
> 2.pop until the queue empty
> 3.keep popping the queue
> 4.delete the queue ( delete a row with deleteAll )
> 5.create a small queue with same name (i.e: same row)
> After create a small queue, I checked every key/value in the hbase and found it is correct. But when I tried to open a scanner, it got Unknown Exception as follows:
> In shell :
> NativeException: org.apache.hadoop.hbase.client.RetriesExhaustedException: Trying to contact region server 127.0.0.1:54141 for region pleaseCrash,,1218621377152, row '', but failed after 3 attempts.
> Exceptions:
> java.net.SocketTimeoutException: timed out waiting for rpc response
> java.net.SocketTimeoutException: timed out waiting for rpc response
> java.net.SocketTimeoutException: timed out waiting for rpc response
> from org/apache/hadoop/hbase/client/HConnectionManager.java:873:in `getRegionServerWithRetries'
> from org/apache/hadoop/hbase/client/HTable.java:1415:in `nextScanner'
> from org/apache/hadoop/hbase/client/HTable.java:1360:in `<init>'
> from org/apache/hadoop/hbase/client/HTable.java:985:in `getScanner'
> from org/apache/hadoop/hbase/client/HTable.java:779:in `getScanner'
> from org/apache/hadoop/hbase/client/HTable.java:706:in `getScanner'
> from sun/reflect/NativeMethodAccessorImpl.java:-2:in `invoke0'
> from sun/reflect/NativeMethodAccessorImpl.java:39:in `invoke'
> from sun/reflect/DelegatingMethodAccessorImpl.java:25:in `invoke'
> from java/lang/reflect/Method.java:597:in `invoke'
> from org/jruby/javasupport/JavaMethod.java:250:in `invokeWithExceptionHandling'
> from org/jruby/javasupport/JavaMethod.java:219:in `invoke'
> from org/jruby/javasupport/JavaClass.java:416:in `execute'
> from org/jruby/internal/runtime/methods/SimpleCallbackMethod.java:67:in `call'
> from org/jruby/internal/runtime/methods/DynamicMethod.java:78:in `call'
> from org/jruby/runtime/CallSite.java:155:in `cacheAndCall'
> ... 128 levels...
> from ruby.home.itspeter.hbase_minus_0_dot_3_dot_0_minus_dev.bin.hirbInvokermethod__23$RUBY$startOpt:-1:in `call'
> from org/jruby/internal/runtime/methods/DynamicMethod.java:74:in `call'
> from org/jruby/internal/runtime/methods/CompiledMethod.java:48:in `call'
> from org/jruby/runtime/CallSite.java:123:in `cacheAndCall'
> from org/jruby/runtime/CallSite.java:298:in `call'
> from ruby/home/itspeter/hbase_minus_0_dot_3_dot_0_minus_dev/bin//home/itspeter/hbase-0.3.0-dev/bin/../bin/hirb.rb:350:in `__file__'
> from ruby/home/itspeter/hbase_minus_0_dot_3_dot_0_minus_dev/bin//home/itspeter/hbase-0.3.0-dev/bin/../bin/hirb.rb:-1:in `__file__'
> from ruby/home/itspeter/hbase_minus_0_dot_3_dot_0_minus_dev/bin//home/itspeter/hbase-0.3.0-dev/bin/../bin/hirb.rb:-1:in `load'
> from org/jruby/Ruby.java:512:in `runScript'
> from org/jruby/Ruby.java:432:in `runNormally'
> from org/jruby/Ruby.java:312:in `runFromMain'
> from org/jruby/Main.java:144:in `run'
> from org/jruby/Main.java:89:in `run'
> from org/jruby/Main.java:80:in `main'
> from /home/itspeter/hbase-0.3.0-dev/bin/../bin/hirb.rb:271:in `scan'
> from (hbase):3:in `binding'
> Side effects :
> After scan got time out, I discovered that I also failed to disable the table (But not every time), somehow more strange is that table will be dropped successfully even disable failed (but not every time).
> System Resources:
> Hbase will rob the cpu...
> In my platform(4-core), cpu usage will raise from 10% -> 50% ..100% -> 200% -> 300% ..
> Procedure to reproduce :
> Run the attached java file.
> Try to type "scan 'pleaseCrash'" in hbase shell after program finished.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (HBASE-827) Failed to open scanner in shell after
series of action executed
Posted by "PeterL in (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
PeterL in resolved HBASE-827.
-----------------------------
Resolution: Invalid
Thanks for Rong-en and stack's solution : )
> Failed to open scanner in shell after series of action executed
> ---------------------------------------------------------------
>
> Key: HBASE-827
> URL: https://issues.apache.org/jira/browse/HBASE-827
> Project: Hadoop HBase
> Issue Type: Bug
> Affects Versions: 0.3.0
> Environment: Java runtime version : 1.6.0_06
> Hbase version : occured in both 0.2.0 RC2 & 0.3.0 trunk (svn 685494)
> Hbase configuration: run in localhost without hadoop
> Reporter: PeterL in
> Attachments: hbase.log, jstack.log, MyClient.java
>
>
> Background introduction :
> I was intent to implement a large queue-liked structure based on HBase.
> Different from normal queue structure, in my structure, a pop action means 'randomly' pick one elements(which is key/value pair from a row) in queue, and decrease the queueSize of that queue.
> For performance consideration, I implemented pop action by following steps :
> 1.get the queue size
> 2.random a number between 0~queueSize
> 3.take step2 as a qualifier, get the value of that element, and put the last element's value to the original qualifier.
> 4.decrease the queue size. (I didn't delete the last elements since storage space is not my primary consideration)
> When the queueSize and number of pop action become large, hbase failed to open a scanner. (Even the data of database is correct)
> My program can be split into few steps:
> 1.create a large queue.
> 2.pop until the queue empty
> 3.keep popping the queue
> 4.delete the queue ( delete a row with deleteAll )
> 5.create a small queue with same name (i.e: same row)
> After create a small queue, I checked every key/value in the hbase and found it is correct. But when I tried to open a scanner, it got Unknown Exception as follows:
> In shell :
> NativeException: org.apache.hadoop.hbase.client.RetriesExhaustedException: Trying to contact region server 127.0.0.1:54141 for region pleaseCrash,,1218621377152, row '', but failed after 3 attempts.
> Exceptions:
> java.net.SocketTimeoutException: timed out waiting for rpc response
> java.net.SocketTimeoutException: timed out waiting for rpc response
> java.net.SocketTimeoutException: timed out waiting for rpc response
> from org/apache/hadoop/hbase/client/HConnectionManager.java:873:in `getRegionServerWithRetries'
> from org/apache/hadoop/hbase/client/HTable.java:1415:in `nextScanner'
> from org/apache/hadoop/hbase/client/HTable.java:1360:in `<init>'
> from org/apache/hadoop/hbase/client/HTable.java:985:in `getScanner'
> from org/apache/hadoop/hbase/client/HTable.java:779:in `getScanner'
> from org/apache/hadoop/hbase/client/HTable.java:706:in `getScanner'
> from sun/reflect/NativeMethodAccessorImpl.java:-2:in `invoke0'
> from sun/reflect/NativeMethodAccessorImpl.java:39:in `invoke'
> from sun/reflect/DelegatingMethodAccessorImpl.java:25:in `invoke'
> from java/lang/reflect/Method.java:597:in `invoke'
> from org/jruby/javasupport/JavaMethod.java:250:in `invokeWithExceptionHandling'
> from org/jruby/javasupport/JavaMethod.java:219:in `invoke'
> from org/jruby/javasupport/JavaClass.java:416:in `execute'
> from org/jruby/internal/runtime/methods/SimpleCallbackMethod.java:67:in `call'
> from org/jruby/internal/runtime/methods/DynamicMethod.java:78:in `call'
> from org/jruby/runtime/CallSite.java:155:in `cacheAndCall'
> ... 128 levels...
> from ruby.home.itspeter.hbase_minus_0_dot_3_dot_0_minus_dev.bin.hirbInvokermethod__23$RUBY$startOpt:-1:in `call'
> from org/jruby/internal/runtime/methods/DynamicMethod.java:74:in `call'
> from org/jruby/internal/runtime/methods/CompiledMethod.java:48:in `call'
> from org/jruby/runtime/CallSite.java:123:in `cacheAndCall'
> from org/jruby/runtime/CallSite.java:298:in `call'
> from ruby/home/itspeter/hbase_minus_0_dot_3_dot_0_minus_dev/bin//home/itspeter/hbase-0.3.0-dev/bin/../bin/hirb.rb:350:in `__file__'
> from ruby/home/itspeter/hbase_minus_0_dot_3_dot_0_minus_dev/bin//home/itspeter/hbase-0.3.0-dev/bin/../bin/hirb.rb:-1:in `__file__'
> from ruby/home/itspeter/hbase_minus_0_dot_3_dot_0_minus_dev/bin//home/itspeter/hbase-0.3.0-dev/bin/../bin/hirb.rb:-1:in `load'
> from org/jruby/Ruby.java:512:in `runScript'
> from org/jruby/Ruby.java:432:in `runNormally'
> from org/jruby/Ruby.java:312:in `runFromMain'
> from org/jruby/Main.java:144:in `run'
> from org/jruby/Main.java:89:in `run'
> from org/jruby/Main.java:80:in `main'
> from /home/itspeter/hbase-0.3.0-dev/bin/../bin/hirb.rb:271:in `scan'
> from (hbase):3:in `binding'
> Side effects :
> After scan got time out, I discovered that I also failed to disable the table (But not every time), somehow more strange is that table will be dropped successfully even disable failed (but not every time).
> System Resources:
> Hbase will rob the cpu...
> In my platform(4-core), cpu usage will raise from 10% -> 50% ..100% -> 200% -> 300% ..
> Procedure to reproduce :
> Run the attached java file.
> Try to type "scan 'pleaseCrash'" in hbase shell after program finished.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-827) Failed to open scanner in shell after
series of action executed
Posted by "PeterL in (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
PeterL in updated HBASE-827:
----------------------------
Attachment: hbase.log
> Failed to open scanner in shell after series of action executed
> ---------------------------------------------------------------
>
> Key: HBASE-827
> URL: https://issues.apache.org/jira/browse/HBASE-827
> Project: Hadoop HBase
> Issue Type: Bug
> Affects Versions: 0.3.0
> Environment: Java runtime version : 1.6.0_06
> Hbase version : occured in both 0.2.0 RC2 & 0.3.0 trunk (svn 685494)
> Hbase configuration: run in localhost without hadoop
> Reporter: PeterL in
> Attachments: hbase.log, jstack.log, MyClient.java
>
>
> Background introduction :
> I was intent to implement a large queue-liked structure based on HBase.
> Different from normal queue structure, in my structure, a pop action means 'randomly' pick one elements(which is key/value pair from a row) in queue, and decrease the queueSize of that queue.
> For performance consideration, I implemented pop action by following steps :
> 1.get the queue size
> 2.random a number between 0~queueSize
> 3.take step2 as a qualifier, get the value of that element, and put the last element's value to the original qualifier.
> 4.decrease the queue size. (I didn't delete the last elements since storage space is not my primary consideration)
> When the queueSize and number of pop action become large, hbase failed to open a scanner. (Even the data of database is correct)
> My program can be split into few steps:
> 1.create a large queue.
> 2.pop until the queue empty
> 3.keep popping the queue
> 4.delete the queue ( delete a row with deleteAll )
> 5.create a small queue with same name (i.e: same row)
> After create a small queue, I checked every key/value in the hbase and found it is correct. But when I tried to open a scanner, it got Unknown Exception as follows:
> In shell :
> NativeException: org.apache.hadoop.hbase.client.RetriesExhaustedException: Trying to contact region server 127.0.0.1:54141 for region pleaseCrash,,1218621377152, row '', but failed after 3 attempts.
> Exceptions:
> java.net.SocketTimeoutException: timed out waiting for rpc response
> java.net.SocketTimeoutException: timed out waiting for rpc response
> java.net.SocketTimeoutException: timed out waiting for rpc response
> from org/apache/hadoop/hbase/client/HConnectionManager.java:873:in `getRegionServerWithRetries'
> from org/apache/hadoop/hbase/client/HTable.java:1415:in `nextScanner'
> from org/apache/hadoop/hbase/client/HTable.java:1360:in `<init>'
> from org/apache/hadoop/hbase/client/HTable.java:985:in `getScanner'
> from org/apache/hadoop/hbase/client/HTable.java:779:in `getScanner'
> from org/apache/hadoop/hbase/client/HTable.java:706:in `getScanner'
> from sun/reflect/NativeMethodAccessorImpl.java:-2:in `invoke0'
> from sun/reflect/NativeMethodAccessorImpl.java:39:in `invoke'
> from sun/reflect/DelegatingMethodAccessorImpl.java:25:in `invoke'
> from java/lang/reflect/Method.java:597:in `invoke'
> from org/jruby/javasupport/JavaMethod.java:250:in `invokeWithExceptionHandling'
> from org/jruby/javasupport/JavaMethod.java:219:in `invoke'
> from org/jruby/javasupport/JavaClass.java:416:in `execute'
> from org/jruby/internal/runtime/methods/SimpleCallbackMethod.java:67:in `call'
> from org/jruby/internal/runtime/methods/DynamicMethod.java:78:in `call'
> from org/jruby/runtime/CallSite.java:155:in `cacheAndCall'
> ... 128 levels...
> from ruby.home.itspeter.hbase_minus_0_dot_3_dot_0_minus_dev.bin.hirbInvokermethod__23$RUBY$startOpt:-1:in `call'
> from org/jruby/internal/runtime/methods/DynamicMethod.java:74:in `call'
> from org/jruby/internal/runtime/methods/CompiledMethod.java:48:in `call'
> from org/jruby/runtime/CallSite.java:123:in `cacheAndCall'
> from org/jruby/runtime/CallSite.java:298:in `call'
> from ruby/home/itspeter/hbase_minus_0_dot_3_dot_0_minus_dev/bin//home/itspeter/hbase-0.3.0-dev/bin/../bin/hirb.rb:350:in `__file__'
> from ruby/home/itspeter/hbase_minus_0_dot_3_dot_0_minus_dev/bin//home/itspeter/hbase-0.3.0-dev/bin/../bin/hirb.rb:-1:in `__file__'
> from ruby/home/itspeter/hbase_minus_0_dot_3_dot_0_minus_dev/bin//home/itspeter/hbase-0.3.0-dev/bin/../bin/hirb.rb:-1:in `load'
> from org/jruby/Ruby.java:512:in `runScript'
> from org/jruby/Ruby.java:432:in `runNormally'
> from org/jruby/Ruby.java:312:in `runFromMain'
> from org/jruby/Main.java:144:in `run'
> from org/jruby/Main.java:89:in `run'
> from org/jruby/Main.java:80:in `main'
> from /home/itspeter/hbase-0.3.0-dev/bin/../bin/hirb.rb:271:in `scan'
> from (hbase):3:in `binding'
> Side effects :
> After scan got time out, I discovered that I also failed to disable the table (But not every time), somehow more strange is that table will be dropped successfully even disable failed (but not every time).
> System Resources:
> Hbase will rob the cpu...
> In my platform(4-core), cpu usage will raise from 10% -> 50% ..100% -> 200% -> 300% ..
> Procedure to reproduce :
> Run the attached java file.
> Try to type "scan 'pleaseCrash'" in hbase shell after program finished.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-827) Failed to open scanner in shell after
series of action executed
Posted by "Rong-En Fan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12623175#action_12623175 ]
Rong-En Fan commented on HBASE-827:
-----------------------------------
I grab the MyClient.java and latest trunk (r686572) to try it out. So, the example program do the following things:
1. insert 40000 columns under one col family
2. insert a number into the same col family to record the total # of cols in this family
3. repeat 100,000 times, do read one column value, modify it, insert back
4. delete everything
5. repeat 1 but with much smaller columns to insert
Then I try to get a scanner from java client:
table.getScanner(new String[]{"insPool:"});
The table in question has only few columns but lots of deleted records (about 140,000). The scanner needs some time to finish it's job. The line above requires 240 seconds to finish (that's 4 mins).
However, in hirb.rb, we have
@configuration.setInt("hbase.client.retries.number", 3)
@configuration.setInt("ipc.client.connect.max.retries", 3)
These are smaller than hbase-default.xml, and this is why scan shell fails.
> Failed to open scanner in shell after series of action executed
> ---------------------------------------------------------------
>
> Key: HBASE-827
> URL: https://issues.apache.org/jira/browse/HBASE-827
> Project: Hadoop HBase
> Issue Type: Bug
> Affects Versions: 0.3.0
> Environment: Java runtime version : 1.6.0_06
> Hbase version : occured in both 0.2.0 RC2 & 0.3.0 trunk (svn 685494)
> Hbase configuration: run in localhost without hadoop
> Reporter: PeterL in
> Attachments: hbase.log, jstack.log, MyClient.java
>
>
> Background introduction :
> I was intent to implement a large queue-liked structure based on HBase.
> Different from normal queue structure, in my structure, a pop action means 'randomly' pick one elements(which is key/value pair from a row) in queue, and decrease the queueSize of that queue.
> For performance consideration, I implemented pop action by following steps :
> 1.get the queue size
> 2.random a number between 0~queueSize
> 3.take step2 as a qualifier, get the value of that element, and put the last element's value to the original qualifier.
> 4.decrease the queue size. (I didn't delete the last elements since storage space is not my primary consideration)
> When the queueSize and number of pop action become large, hbase failed to open a scanner. (Even the data of database is correct)
> My program can be split into few steps:
> 1.create a large queue.
> 2.pop until the queue empty
> 3.keep popping the queue
> 4.delete the queue ( delete a row with deleteAll )
> 5.create a small queue with same name (i.e: same row)
> After create a small queue, I checked every key/value in the hbase and found it is correct. But when I tried to open a scanner, it got Unknown Exception as follows:
> In shell :
> NativeException: org.apache.hadoop.hbase.client.RetriesExhaustedException: Trying to contact region server 127.0.0.1:54141 for region pleaseCrash,,1218621377152, row '', but failed after 3 attempts.
> Exceptions:
> java.net.SocketTimeoutException: timed out waiting for rpc response
> java.net.SocketTimeoutException: timed out waiting for rpc response
> java.net.SocketTimeoutException: timed out waiting for rpc response
> from org/apache/hadoop/hbase/client/HConnectionManager.java:873:in `getRegionServerWithRetries'
> from org/apache/hadoop/hbase/client/HTable.java:1415:in `nextScanner'
> from org/apache/hadoop/hbase/client/HTable.java:1360:in `<init>'
> from org/apache/hadoop/hbase/client/HTable.java:985:in `getScanner'
> from org/apache/hadoop/hbase/client/HTable.java:779:in `getScanner'
> from org/apache/hadoop/hbase/client/HTable.java:706:in `getScanner'
> from sun/reflect/NativeMethodAccessorImpl.java:-2:in `invoke0'
> from sun/reflect/NativeMethodAccessorImpl.java:39:in `invoke'
> from sun/reflect/DelegatingMethodAccessorImpl.java:25:in `invoke'
> from java/lang/reflect/Method.java:597:in `invoke'
> from org/jruby/javasupport/JavaMethod.java:250:in `invokeWithExceptionHandling'
> from org/jruby/javasupport/JavaMethod.java:219:in `invoke'
> from org/jruby/javasupport/JavaClass.java:416:in `execute'
> from org/jruby/internal/runtime/methods/SimpleCallbackMethod.java:67:in `call'
> from org/jruby/internal/runtime/methods/DynamicMethod.java:78:in `call'
> from org/jruby/runtime/CallSite.java:155:in `cacheAndCall'
> ... 128 levels...
> from ruby.home.itspeter.hbase_minus_0_dot_3_dot_0_minus_dev.bin.hirbInvokermethod__23$RUBY$startOpt:-1:in `call'
> from org/jruby/internal/runtime/methods/DynamicMethod.java:74:in `call'
> from org/jruby/internal/runtime/methods/CompiledMethod.java:48:in `call'
> from org/jruby/runtime/CallSite.java:123:in `cacheAndCall'
> from org/jruby/runtime/CallSite.java:298:in `call'
> from ruby/home/itspeter/hbase_minus_0_dot_3_dot_0_minus_dev/bin//home/itspeter/hbase-0.3.0-dev/bin/../bin/hirb.rb:350:in `__file__'
> from ruby/home/itspeter/hbase_minus_0_dot_3_dot_0_minus_dev/bin//home/itspeter/hbase-0.3.0-dev/bin/../bin/hirb.rb:-1:in `__file__'
> from ruby/home/itspeter/hbase_minus_0_dot_3_dot_0_minus_dev/bin//home/itspeter/hbase-0.3.0-dev/bin/../bin/hirb.rb:-1:in `load'
> from org/jruby/Ruby.java:512:in `runScript'
> from org/jruby/Ruby.java:432:in `runNormally'
> from org/jruby/Ruby.java:312:in `runFromMain'
> from org/jruby/Main.java:144:in `run'
> from org/jruby/Main.java:89:in `run'
> from org/jruby/Main.java:80:in `main'
> from /home/itspeter/hbase-0.3.0-dev/bin/../bin/hirb.rb:271:in `scan'
> from (hbase):3:in `binding'
> Side effects :
> After scan got time out, I discovered that I also failed to disable the table (But not every time), somehow more strange is that table will be dropped successfully even disable failed (but not every time).
> System Resources:
> Hbase will rob the cpu...
> In my platform(4-core), cpu usage will raise from 10% -> 50% ..100% -> 200% -> 300% ..
> Procedure to reproduce :
> Run the attached java file.
> Try to type "scan 'pleaseCrash'" in hbase shell after program finished.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Issue Comment Edited: (HBASE-827) Failed to open scanner in
shell after series of action executed
Posted by "PeterL in (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12622179#action_12622179 ]
itspeter edited comment on HBASE-827 at 8/13/08 4:37 AM:
----------------------------------------------------------
Code to reproduce problem : MyClient.java
was (Author: itspeter):
Code to reproduce problem
> Failed to open scanner in shell after series of action executed
> ---------------------------------------------------------------
>
> Key: HBASE-827
> URL: https://issues.apache.org/jira/browse/HBASE-827
> Project: Hadoop HBase
> Issue Type: Bug
> Affects Versions: 0.3.0
> Environment: Java runtime version : 1.6.0_06
> Hbase version : occured in both 0.2.0 RC2 & 0.3.0 trunk (svn 685494)
> Hbase configuration: run in localhost without hadoop
> Reporter: PeterL in
> Attachments: MyClient.java
>
>
> Background introduction :
> I was intent to implement a large queue-liked structure based on HBase.
> Different from normal queue structure, in my structure, a pop action means 'randomly' pick one elements(which is key/value pair from a row) in queue, and decrease the queueSize of that queue.
> For performance consideration, I implemented pop action by following steps :
> 1.get the queue size
> 2.random a number between 0~queueSize
> 3.take step2 as a qualifier, get the value of that element, and put the last element's value to the original qualifier.
> 4.decrease the queue size. (I didn't delete the last elements since storage space is not my primary consideration)
> When the queueSize and number of pop action become large, hbase failed to open a scanner. (Even the data of database is correct)
> My program can be split into few steps:
> 1.create a large queue.
> 2.pop until the queue empty
> 3.keep popping the queue
> 4.delete the queue ( delete a row with deleteAll )
> 5.create a small queue with same name (i.e: same row)
> After create a small queue, I checked every key/value in the hbase and found it is correct. But when I tried to open a scanner, it got Unknown Exception as follows:
> In shell :
> NativeException: org.apache.hadoop.hbase.client.RetriesExhaustedException: Trying to contact region server 127.0.0.1:54141 for region pleaseCrash,,1218621377152, row '', but failed after 3 attempts.
> Exceptions:
> java.net.SocketTimeoutException: timed out waiting for rpc response
> java.net.SocketTimeoutException: timed out waiting for rpc response
> java.net.SocketTimeoutException: timed out waiting for rpc response
> from org/apache/hadoop/hbase/client/HConnectionManager.java:873:in `getRegionServerWithRetries'
> from org/apache/hadoop/hbase/client/HTable.java:1415:in `nextScanner'
> from org/apache/hadoop/hbase/client/HTable.java:1360:in `<init>'
> from org/apache/hadoop/hbase/client/HTable.java:985:in `getScanner'
> from org/apache/hadoop/hbase/client/HTable.java:779:in `getScanner'
> from org/apache/hadoop/hbase/client/HTable.java:706:in `getScanner'
> from sun/reflect/NativeMethodAccessorImpl.java:-2:in `invoke0'
> from sun/reflect/NativeMethodAccessorImpl.java:39:in `invoke'
> from sun/reflect/DelegatingMethodAccessorImpl.java:25:in `invoke'
> from java/lang/reflect/Method.java:597:in `invoke'
> from org/jruby/javasupport/JavaMethod.java:250:in `invokeWithExceptionHandling'
> from org/jruby/javasupport/JavaMethod.java:219:in `invoke'
> from org/jruby/javasupport/JavaClass.java:416:in `execute'
> from org/jruby/internal/runtime/methods/SimpleCallbackMethod.java:67:in `call'
> from org/jruby/internal/runtime/methods/DynamicMethod.java:78:in `call'
> from org/jruby/runtime/CallSite.java:155:in `cacheAndCall'
> ... 128 levels...
> from ruby.home.itspeter.hbase_minus_0_dot_3_dot_0_minus_dev.bin.hirbInvokermethod__23$RUBY$startOpt:-1:in `call'
> from org/jruby/internal/runtime/methods/DynamicMethod.java:74:in `call'
> from org/jruby/internal/runtime/methods/CompiledMethod.java:48:in `call'
> from org/jruby/runtime/CallSite.java:123:in `cacheAndCall'
> from org/jruby/runtime/CallSite.java:298:in `call'
> from ruby/home/itspeter/hbase_minus_0_dot_3_dot_0_minus_dev/bin//home/itspeter/hbase-0.3.0-dev/bin/../bin/hirb.rb:350:in `__file__'
> from ruby/home/itspeter/hbase_minus_0_dot_3_dot_0_minus_dev/bin//home/itspeter/hbase-0.3.0-dev/bin/../bin/hirb.rb:-1:in `__file__'
> from ruby/home/itspeter/hbase_minus_0_dot_3_dot_0_minus_dev/bin//home/itspeter/hbase-0.3.0-dev/bin/../bin/hirb.rb:-1:in `load'
> from org/jruby/Ruby.java:512:in `runScript'
> from org/jruby/Ruby.java:432:in `runNormally'
> from org/jruby/Ruby.java:312:in `runFromMain'
> from org/jruby/Main.java:144:in `run'
> from org/jruby/Main.java:89:in `run'
> from org/jruby/Main.java:80:in `main'
> from /home/itspeter/hbase-0.3.0-dev/bin/../bin/hirb.rb:271:in `scan'
> from (hbase):3:in `binding'
> Side effects :
> After scan got time out, I discovered that I also failed to disable the table (But not every time), somehow more strange is that table will be dropped successfully even disable failed (but not every time).
> System Resources:
> Hbase will rob the cpu...
> In my platform(4-core), cpu usage will raise from 10% -> 50% ..100% -> 200% -> 300% ..
> Procedure to reproduce :
> Run the attached java file.
> Try to type "scan 'pleaseCrash'" in hbase shell after program finished.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.