You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hbase.apache.org by "Alex Newman (JIRA)" <ji...@apache.org> on 2009/04/30 19:40:30 UTC

[jira] Created: (HBASE-1359) After a large truncating table HBase becomes unresponsive

After  a large truncating table HBase becomes unresponsive
----------------------------------------------------------

                 Key: HBASE-1359
                 URL: https://issues.apache.org/jira/browse/HBASE-1359
             Project: Hadoop HBase
          Issue Type: Bug
         Environment: linux  r769665
            Reporter: Alex Newman


If you see **** I removed and ip or something for security reasons

Once I truncate the table, hbase freaks out for about 10 seconds and all the thrift servers die.

Thrift server log:
2009-04-02 12:09:08,971 INFO org.apache.hadoop.ipc.HBaseClass: Retrying connect 
to server: /*****:60020. Already tried 0 time(s).
You see this a bunch of times and then it times out

The hbase shell
nhbase(main):001:0> truncate 't2'
09/04/30 13:01:08 INFO zookeeper.ZooKeeperWrapper: Quorum servers: 
****
Truncating t2; it may take a while
Disabling table...
09/04/30 13:01:19 INFO client.HBaseAdmin: Disabled t2
0 row(s) in 10.3417 seconds
Dropping table...
09/04/30 13:01:19 INFO client.HBaseAdmin: Deleted t2
0 row(s) in 0.1592 seconds
Creating table...
0 row(s) in 14.7567 seconds
hbase(main):002:0> lsit
NameError: undefined local variable or method `lsit' for #<Object:0x3bbe9a50>
        from (hbase):3
hbase(main):003:0> lsit
NameError: undefined local variable or method `lsit' for #<Object:0x3bbe9a50>
        from (hbase):4
hbase(main):004:0> list
NativeException: java.lang.NullPointerException: null
        from org/apache/hadoop/hbase/client/HConnectionManager.java:344:in `proc
essRow' 
      from org/apache/hadoop/hbase/client/MetaScanner.java:64:in `metaScan'
        from org/apache/hadoop/hbase/client/MetaScanner.java:29:in `metaScan'
        from org/apache/hadoop/hbase/client/HConnectionManager.java:351:in `list
Tables' 
        from org/apache/hadoop/hbase/client/HBaseAdmin.java:121:in `listTables'
        from sun/reflect/NativeMethodAccessorImpl.java:-2:in `invoke0'
        from sun/reflect/NativeMethodAccessorImpl.java:39:in `invoke'
        from sun/reflect/DelegatingMethodAccessorImpl.java:25:in `invoke'
        from java/lang/reflect/Method.java:597:in `invoke'
        from org/jruby/javasupport/JavaMethod.java:298:in `invokeWithExceptionHa
ndling' 
        from org/jruby/javasupport/JavaMethod.java:259:in `invoke'
        from org/jruby/java/invokers/InstanceMethodInvoker.java:36:in `call'
        from org/jruby/runtime/callsite/CachingCallSite.java:260:in `cacheAndCal
l'
        from org/jruby/runtime/callsite/CachingCallSite.java:75:in `call'
        from org/jruby/ast/CallNoArgNode.java:61:in `interpret'
        from org/jruby/ast/ForNode.java:101:in `interpret'
... 113 levels...
        from org/jruby/internal/runtime/methods/DynamicMethod.java:226:in `call'
        from org/jruby/internal/runtime/methods/CompiledMethod.java:216:in `call
'
        from org/jruby/internal/runtime/methods/CompiledMethod.java:71:in `call'
        from org/jruby/runtime/callsite/CachingCallSite.java:260:in `cacheAndCal
l'
        from org/jruby/runtime/callsite/CachingCallSite.java:75:in `call'
        from home/fds/ts/hadoop/hbase/bin/$_dot_dot_/bin/hirb.rb:441:in `__file_
_'
        from home/fds/ts/hadoop/hbase/bin/$_dot_dot_/bin/hirb.rb:-1:in `__file__
'
        from home/fds/ts/hadoop/hbase/bin/$_dot_dot_/bin/hirb.rb:-1:in `load'
        from org/jruby/Ruby.java:564:in `runScript'
        from org/jruby/Ruby.java:467:in `runNormally'
        from org/jruby/Ruby.java:340:in `runFromMain'
        from org/jruby/Main.java:214:in `run'
        from org/jruby/Main.java:100:in `run'
        from org/jruby/Main.java:84:in `main'
        from /home/fds/ts/hadoop/hbase/bin/../bin/hirb.rb:300:in `list'
        from (hbase):5hbase(main):005:0>
hbase(main):006:0* 



-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Resolved: (HBASE-1359) After a large truncating table HBase becomes unresponsive

Posted by "stack (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HBASE-1359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

stack resolved HBASE-1359.
--------------------------

    Resolution: Fixed

Resolving at Alex's prompting.

> After  a large truncating table HBase becomes unresponsive
> ----------------------------------------------------------
>
>                 Key: HBASE-1359
>                 URL: https://issues.apache.org/jira/browse/HBASE-1359
>             Project: Hadoop HBase
>          Issue Type: Bug
>         Environment: linux  r769665
>            Reporter: Alex Newman
>            Assignee: Alex Newman
>            Priority: Critical
>             Fix For: 0.20.0
>
>
> If you see **** I removed and ip or something for security reasons
> Once I truncate the table, hbase freaks out for about 10 seconds and all the thrift servers die.
> Thrift server log:
> 2009-04-02 12:09:08,971 INFO org.apache.hadoop.ipc.HBaseClass: Retrying connect 
> to server: /*****:60020. Already tried 0 time(s).
> You see this a bunch of times and then it times out
> The hbase shell
> nhbase(main):001:0> truncate 't2'
> 09/04/30 13:01:08 INFO zookeeper.ZooKeeperWrapper: Quorum servers: 
> ****
> Truncating t2; it may take a while
> Disabling table...
> 09/04/30 13:01:19 INFO client.HBaseAdmin: Disabled t2
> 0 row(s) in 10.3417 seconds
> Dropping table...
> 09/04/30 13:01:19 INFO client.HBaseAdmin: Deleted t2
> 0 row(s) in 0.1592 seconds
> Creating table...
> 0 row(s) in 14.7567 seconds
> hbase(main):002:0> lsit
> NameError: undefined local variable or method `lsit' for #<Object:0x3bbe9a50>
>         from (hbase):3
> hbase(main):003:0> lsit
> NameError: undefined local variable or method `lsit' for #<Object:0x3bbe9a50>
>         from (hbase):4
> hbase(main):004:0> list
> NativeException: java.lang.NullPointerException: null
>         from org/apache/hadoop/hbase/client/HConnectionManager.java:344:in `proc
> essRow' 
>       from org/apache/hadoop/hbase/client/MetaScanner.java:64:in `metaScan'
>         from org/apache/hadoop/hbase/client/MetaScanner.java:29:in `metaScan'
>         from org/apache/hadoop/hbase/client/HConnectionManager.java:351:in `list
> Tables' 
>         from org/apache/hadoop/hbase/client/HBaseAdmin.java:121:in `listTables'
>         from sun/reflect/NativeMethodAccessorImpl.java:-2:in `invoke0'
>         from sun/reflect/NativeMethodAccessorImpl.java:39:in `invoke'
>         from sun/reflect/DelegatingMethodAccessorImpl.java:25:in `invoke'
>         from java/lang/reflect/Method.java:597:in `invoke'
>         from org/jruby/javasupport/JavaMethod.java:298:in `invokeWithExceptionHa
> ndling' 
>         from org/jruby/javasupport/JavaMethod.java:259:in `invoke'
>         from org/jruby/java/invokers/InstanceMethodInvoker.java:36:in `call'
>         from org/jruby/runtime/callsite/CachingCallSite.java:260:in `cacheAndCal
> l'
>         from org/jruby/runtime/callsite/CachingCallSite.java:75:in `call'
>         from org/jruby/ast/CallNoArgNode.java:61:in `interpret'
>         from org/jruby/ast/ForNode.java:101:in `interpret'
> ... 113 levels...
>         from org/jruby/internal/runtime/methods/DynamicMethod.java:226:in `call'
>         from org/jruby/internal/runtime/methods/CompiledMethod.java:216:in `call
> '
>         from org/jruby/internal/runtime/methods/CompiledMethod.java:71:in `call'
>         from org/jruby/runtime/callsite/CachingCallSite.java:260:in `cacheAndCal
> l'
>         from org/jruby/runtime/callsite/CachingCallSite.java:75:in `call'
>         from home/fds/ts/hadoop/hbase/bin/$_dot_dot_/bin/hirb.rb:441:in `__file_
> _'
>         from home/fds/ts/hadoop/hbase/bin/$_dot_dot_/bin/hirb.rb:-1:in `__file__
> '
>         from home/fds/ts/hadoop/hbase/bin/$_dot_dot_/bin/hirb.rb:-1:in `load'
>         from org/jruby/Ruby.java:564:in `runScript'
>         from org/jruby/Ruby.java:467:in `runNormally'
>         from org/jruby/Ruby.java:340:in `runFromMain'
>         from org/jruby/Main.java:214:in `run'
>         from org/jruby/Main.java:100:in `run'
>         from org/jruby/Main.java:84:in `main'
>         from /home/fds/ts/hadoop/hbase/bin/../bin/hirb.rb:300:in `list'
>         from (hbase):5hbase(main):005:0>
> hbase(main):006:0* 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (HBASE-1359) After a large truncating table HBase becomes unresponsive

Posted by "Alex Newman (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HBASE-1359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12730387#action_12730387 ] 

Alex Newman commented on HBASE-1359:
------------------------------------

This works a bit better but the table seems to never disable even still.

> After  a large truncating table HBase becomes unresponsive
> ----------------------------------------------------------
>
>                 Key: HBASE-1359
>                 URL: https://issues.apache.org/jira/browse/HBASE-1359
>             Project: Hadoop HBase
>          Issue Type: Bug
>         Environment: linux  r769665
>            Reporter: Alex Newman
>            Assignee: Alex Newman
>
> If you see **** I removed and ip or something for security reasons
> Once I truncate the table, hbase freaks out for about 10 seconds and all the thrift servers die.
> Thrift server log:
> 2009-04-02 12:09:08,971 INFO org.apache.hadoop.ipc.HBaseClass: Retrying connect 
> to server: /*****:60020. Already tried 0 time(s).
> You see this a bunch of times and then it times out
> The hbase shell
> nhbase(main):001:0> truncate 't2'
> 09/04/30 13:01:08 INFO zookeeper.ZooKeeperWrapper: Quorum servers: 
> ****
> Truncating t2; it may take a while
> Disabling table...
> 09/04/30 13:01:19 INFO client.HBaseAdmin: Disabled t2
> 0 row(s) in 10.3417 seconds
> Dropping table...
> 09/04/30 13:01:19 INFO client.HBaseAdmin: Deleted t2
> 0 row(s) in 0.1592 seconds
> Creating table...
> 0 row(s) in 14.7567 seconds
> hbase(main):002:0> lsit
> NameError: undefined local variable or method `lsit' for #<Object:0x3bbe9a50>
>         from (hbase):3
> hbase(main):003:0> lsit
> NameError: undefined local variable or method `lsit' for #<Object:0x3bbe9a50>
>         from (hbase):4
> hbase(main):004:0> list
> NativeException: java.lang.NullPointerException: null
>         from org/apache/hadoop/hbase/client/HConnectionManager.java:344:in `proc
> essRow' 
>       from org/apache/hadoop/hbase/client/MetaScanner.java:64:in `metaScan'
>         from org/apache/hadoop/hbase/client/MetaScanner.java:29:in `metaScan'
>         from org/apache/hadoop/hbase/client/HConnectionManager.java:351:in `list
> Tables' 
>         from org/apache/hadoop/hbase/client/HBaseAdmin.java:121:in `listTables'
>         from sun/reflect/NativeMethodAccessorImpl.java:-2:in `invoke0'
>         from sun/reflect/NativeMethodAccessorImpl.java:39:in `invoke'
>         from sun/reflect/DelegatingMethodAccessorImpl.java:25:in `invoke'
>         from java/lang/reflect/Method.java:597:in `invoke'
>         from org/jruby/javasupport/JavaMethod.java:298:in `invokeWithExceptionHa
> ndling' 
>         from org/jruby/javasupport/JavaMethod.java:259:in `invoke'
>         from org/jruby/java/invokers/InstanceMethodInvoker.java:36:in `call'
>         from org/jruby/runtime/callsite/CachingCallSite.java:260:in `cacheAndCal
> l'
>         from org/jruby/runtime/callsite/CachingCallSite.java:75:in `call'
>         from org/jruby/ast/CallNoArgNode.java:61:in `interpret'
>         from org/jruby/ast/ForNode.java:101:in `interpret'
> ... 113 levels...
>         from org/jruby/internal/runtime/methods/DynamicMethod.java:226:in `call'
>         from org/jruby/internal/runtime/methods/CompiledMethod.java:216:in `call
> '
>         from org/jruby/internal/runtime/methods/CompiledMethod.java:71:in `call'
>         from org/jruby/runtime/callsite/CachingCallSite.java:260:in `cacheAndCal
> l'
>         from org/jruby/runtime/callsite/CachingCallSite.java:75:in `call'
>         from home/fds/ts/hadoop/hbase/bin/$_dot_dot_/bin/hirb.rb:441:in `__file_
> _'
>         from home/fds/ts/hadoop/hbase/bin/$_dot_dot_/bin/hirb.rb:-1:in `__file__
> '
>         from home/fds/ts/hadoop/hbase/bin/$_dot_dot_/bin/hirb.rb:-1:in `load'
>         from org/jruby/Ruby.java:564:in `runScript'
>         from org/jruby/Ruby.java:467:in `runNormally'
>         from org/jruby/Ruby.java:340:in `runFromMain'
>         from org/jruby/Main.java:214:in `run'
>         from org/jruby/Main.java:100:in `run'
>         from org/jruby/Main.java:84:in `main'
>         from /home/fds/ts/hadoop/hbase/bin/../bin/hirb.rb:300:in `list'
>         from (hbase):5hbase(main):005:0>
> hbase(main):006:0* 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (HBASE-1359) After a large truncating table HBase becomes unresponsive

Posted by "stack (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HBASE-1359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12732337#action_12732337 ] 

stack commented on HBASE-1359:
------------------------------

Please update Alex and try again.  I think things should be a good bit better now after hbase-1583 went in.  

> After  a large truncating table HBase becomes unresponsive
> ----------------------------------------------------------
>
>                 Key: HBASE-1359
>                 URL: https://issues.apache.org/jira/browse/HBASE-1359
>             Project: Hadoop HBase
>          Issue Type: Bug
>         Environment: linux  r769665
>            Reporter: Alex Newman
>            Assignee: Alex Newman
>            Priority: Critical
>             Fix For: 0.20.0
>
>
> If you see **** I removed and ip or something for security reasons
> Once I truncate the table, hbase freaks out for about 10 seconds and all the thrift servers die.
> Thrift server log:
> 2009-04-02 12:09:08,971 INFO org.apache.hadoop.ipc.HBaseClass: Retrying connect 
> to server: /*****:60020. Already tried 0 time(s).
> You see this a bunch of times and then it times out
> The hbase shell
> nhbase(main):001:0> truncate 't2'
> 09/04/30 13:01:08 INFO zookeeper.ZooKeeperWrapper: Quorum servers: 
> ****
> Truncating t2; it may take a while
> Disabling table...
> 09/04/30 13:01:19 INFO client.HBaseAdmin: Disabled t2
> 0 row(s) in 10.3417 seconds
> Dropping table...
> 09/04/30 13:01:19 INFO client.HBaseAdmin: Deleted t2
> 0 row(s) in 0.1592 seconds
> Creating table...
> 0 row(s) in 14.7567 seconds
> hbase(main):002:0> lsit
> NameError: undefined local variable or method `lsit' for #<Object:0x3bbe9a50>
>         from (hbase):3
> hbase(main):003:0> lsit
> NameError: undefined local variable or method `lsit' for #<Object:0x3bbe9a50>
>         from (hbase):4
> hbase(main):004:0> list
> NativeException: java.lang.NullPointerException: null
>         from org/apache/hadoop/hbase/client/HConnectionManager.java:344:in `proc
> essRow' 
>       from org/apache/hadoop/hbase/client/MetaScanner.java:64:in `metaScan'
>         from org/apache/hadoop/hbase/client/MetaScanner.java:29:in `metaScan'
>         from org/apache/hadoop/hbase/client/HConnectionManager.java:351:in `list
> Tables' 
>         from org/apache/hadoop/hbase/client/HBaseAdmin.java:121:in `listTables'
>         from sun/reflect/NativeMethodAccessorImpl.java:-2:in `invoke0'
>         from sun/reflect/NativeMethodAccessorImpl.java:39:in `invoke'
>         from sun/reflect/DelegatingMethodAccessorImpl.java:25:in `invoke'
>         from java/lang/reflect/Method.java:597:in `invoke'
>         from org/jruby/javasupport/JavaMethod.java:298:in `invokeWithExceptionHa
> ndling' 
>         from org/jruby/javasupport/JavaMethod.java:259:in `invoke'
>         from org/jruby/java/invokers/InstanceMethodInvoker.java:36:in `call'
>         from org/jruby/runtime/callsite/CachingCallSite.java:260:in `cacheAndCal
> l'
>         from org/jruby/runtime/callsite/CachingCallSite.java:75:in `call'
>         from org/jruby/ast/CallNoArgNode.java:61:in `interpret'
>         from org/jruby/ast/ForNode.java:101:in `interpret'
> ... 113 levels...
>         from org/jruby/internal/runtime/methods/DynamicMethod.java:226:in `call'
>         from org/jruby/internal/runtime/methods/CompiledMethod.java:216:in `call
> '
>         from org/jruby/internal/runtime/methods/CompiledMethod.java:71:in `call'
>         from org/jruby/runtime/callsite/CachingCallSite.java:260:in `cacheAndCal
> l'
>         from org/jruby/runtime/callsite/CachingCallSite.java:75:in `call'
>         from home/fds/ts/hadoop/hbase/bin/$_dot_dot_/bin/hirb.rb:441:in `__file_
> _'
>         from home/fds/ts/hadoop/hbase/bin/$_dot_dot_/bin/hirb.rb:-1:in `__file__
> '
>         from home/fds/ts/hadoop/hbase/bin/$_dot_dot_/bin/hirb.rb:-1:in `load'
>         from org/jruby/Ruby.java:564:in `runScript'
>         from org/jruby/Ruby.java:467:in `runNormally'
>         from org/jruby/Ruby.java:340:in `runFromMain'
>         from org/jruby/Main.java:214:in `run'
>         from org/jruby/Main.java:100:in `run'
>         from org/jruby/Main.java:84:in `main'
>         from /home/fds/ts/hadoop/hbase/bin/../bin/hirb.rb:300:in `list'
>         from (hbase):5hbase(main):005:0>
> hbase(main):006:0* 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (HBASE-1359) After a large truncating table HBase becomes unresponsive

Posted by "Alex Newman (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HBASE-1359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12731528#action_12731528 ] 

Alex Newman commented on HBASE-1359:
------------------------------------

Oooh I am totally wrong that broke the table, even after a restart list and simple commands don't work
09/07/15 11:50:07 ERROR client.HConnectionManager$TableServers: Result=keyvalues={t5,054050\x7CKRX\x7C1236088243,1247564171866/historian:open/1247614903643/Put/vlen=45}
Note this table shouldn't even exist.

> After  a large truncating table HBase becomes unresponsive
> ----------------------------------------------------------
>
>                 Key: HBASE-1359
>                 URL: https://issues.apache.org/jira/browse/HBASE-1359
>             Project: Hadoop HBase
>          Issue Type: Bug
>         Environment: linux  r769665
>            Reporter: Alex Newman
>            Assignee: Alex Newman
>            Priority: Critical
>             Fix For: 0.20.0
>
>
> If you see **** I removed and ip or something for security reasons
> Once I truncate the table, hbase freaks out for about 10 seconds and all the thrift servers die.
> Thrift server log:
> 2009-04-02 12:09:08,971 INFO org.apache.hadoop.ipc.HBaseClass: Retrying connect 
> to server: /*****:60020. Already tried 0 time(s).
> You see this a bunch of times and then it times out
> The hbase shell
> nhbase(main):001:0> truncate 't2'
> 09/04/30 13:01:08 INFO zookeeper.ZooKeeperWrapper: Quorum servers: 
> ****
> Truncating t2; it may take a while
> Disabling table...
> 09/04/30 13:01:19 INFO client.HBaseAdmin: Disabled t2
> 0 row(s) in 10.3417 seconds
> Dropping table...
> 09/04/30 13:01:19 INFO client.HBaseAdmin: Deleted t2
> 0 row(s) in 0.1592 seconds
> Creating table...
> 0 row(s) in 14.7567 seconds
> hbase(main):002:0> lsit
> NameError: undefined local variable or method `lsit' for #<Object:0x3bbe9a50>
>         from (hbase):3
> hbase(main):003:0> lsit
> NameError: undefined local variable or method `lsit' for #<Object:0x3bbe9a50>
>         from (hbase):4
> hbase(main):004:0> list
> NativeException: java.lang.NullPointerException: null
>         from org/apache/hadoop/hbase/client/HConnectionManager.java:344:in `proc
> essRow' 
>       from org/apache/hadoop/hbase/client/MetaScanner.java:64:in `metaScan'
>         from org/apache/hadoop/hbase/client/MetaScanner.java:29:in `metaScan'
>         from org/apache/hadoop/hbase/client/HConnectionManager.java:351:in `list
> Tables' 
>         from org/apache/hadoop/hbase/client/HBaseAdmin.java:121:in `listTables'
>         from sun/reflect/NativeMethodAccessorImpl.java:-2:in `invoke0'
>         from sun/reflect/NativeMethodAccessorImpl.java:39:in `invoke'
>         from sun/reflect/DelegatingMethodAccessorImpl.java:25:in `invoke'
>         from java/lang/reflect/Method.java:597:in `invoke'
>         from org/jruby/javasupport/JavaMethod.java:298:in `invokeWithExceptionHa
> ndling' 
>         from org/jruby/javasupport/JavaMethod.java:259:in `invoke'
>         from org/jruby/java/invokers/InstanceMethodInvoker.java:36:in `call'
>         from org/jruby/runtime/callsite/CachingCallSite.java:260:in `cacheAndCal
> l'
>         from org/jruby/runtime/callsite/CachingCallSite.java:75:in `call'
>         from org/jruby/ast/CallNoArgNode.java:61:in `interpret'
>         from org/jruby/ast/ForNode.java:101:in `interpret'
> ... 113 levels...
>         from org/jruby/internal/runtime/methods/DynamicMethod.java:226:in `call'
>         from org/jruby/internal/runtime/methods/CompiledMethod.java:216:in `call
> '
>         from org/jruby/internal/runtime/methods/CompiledMethod.java:71:in `call'
>         from org/jruby/runtime/callsite/CachingCallSite.java:260:in `cacheAndCal
> l'
>         from org/jruby/runtime/callsite/CachingCallSite.java:75:in `call'
>         from home/fds/ts/hadoop/hbase/bin/$_dot_dot_/bin/hirb.rb:441:in `__file_
> _'
>         from home/fds/ts/hadoop/hbase/bin/$_dot_dot_/bin/hirb.rb:-1:in `__file__
> '
>         from home/fds/ts/hadoop/hbase/bin/$_dot_dot_/bin/hirb.rb:-1:in `load'
>         from org/jruby/Ruby.java:564:in `runScript'
>         from org/jruby/Ruby.java:467:in `runNormally'
>         from org/jruby/Ruby.java:340:in `runFromMain'
>         from org/jruby/Main.java:214:in `run'
>         from org/jruby/Main.java:100:in `run'
>         from org/jruby/Main.java:84:in `main'
>         from /home/fds/ts/hadoop/hbase/bin/../bin/hirb.rb:300:in `list'
>         from (hbase):5hbase(main):005:0>
> hbase(main):006:0* 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (HBASE-1359) After a large truncating table HBase becomes unresponsive

Posted by "Alex Newman (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HBASE-1359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12731517#action_12731517 ] 

Alex Newman commented on HBASE-1359:
------------------------------------

Disable and drop now work so I would say this is not a blocking issue. This is a rather large table maybe 40/50 regions.


$date; echo "disable 't5'"| .../hbase shell ;date
Wed Jul 15 11:36:21 EDT 2009
HBase Shell; enter 'help<RETURN>' for list of supported commands.
Version: 0.20.0-dev, r794282, Wed Jul 15 10:29:12 EDT 2009
disable 't5'
09/07/15 11:36:24 DEBUG client.HConnectionManager: Created new HBASE_INSTANCES
09/07/15 11:36:24 DEBUG client.HConnectionManager$TableServers: Got ZooKeeper event, state: SyncConnected, type: None, path: null
09/07/15 11:36:24 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/master got **************:60000
09/07/15 11:36:49 DEBUG client.HConnectionManager: Created new HBASE_INSTANCES
09/07/15 11:36:49 DEBUG client.HConnectionManager$TableServers: Got ZooKeeper event, state: SyncConnected, type: None, path: null
09/07/15 11:36:49 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got **************:60020
09/07/15 11:36:49 DEBUG client.HConnectionManager$TableServers: Found ROOT at **************:60020
09/07/15 11:36:50 DEBUG client.HConnectionManager$TableServers: Cached location address: **************:60020, regioninfo: REGION => {NAME => '.META.,,1', STARTKEY => '', ENDKEY => '', ENCODED => 1028785192, TABLE => {{NAME => '.META.', IS_META => 'true', MEMSTORE_FLUSHSIZE => '16384', FAMILIES => [{NAME => 'historian', COMPRESSION => 'NONE', VERSIONS => '2147483647', TTL => '604800', BLOCKSIZE => '8192', IN_MEMORY => 'false', BLOCKCACHE => 'false'}, {NAME => 'info', COMPRESSION => 'NONE', VERSIONS => '10', TTL => '2147483647', BLOCKSIZE => '8192', IN_MEMORY => 'false', BLOCKCACHE => 'true'}]}}


:$date; echo "drop 't5'"| /home/fds/ts/hadoop/hbase/bin/hbase shell ;date Wed Jul 15 11:37:39 EDT 2009
HBase Shell; enter 'help<RETURN>' for list of supported commands.
Version: 0.20.0-dev, r794282, Wed Jul 15 10:29:12 EDT 2009
drop't5'
09/07/15 11:37:42 DEBUG client.HConnectionManager: Created new HBASE_INSTANCES
09/07/15 11:37:42 DEBUG client.HConnectionManager$TableServers: Got ZooKeeper event, state: SyncConnected, type: None, path: null
09/07/15 11:37:42 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/master got **************:60000
09/07/15 11:37:43 DEBUG client.HConnectionManager: Created new HBASE_INSTANCES
09/07/15 11:37:43 DEBUG client.HConnectionManager$TableServers: Got ZooKeeper event, state: SyncConnected, type: None, path: null
09/07/15 11:37:43 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got **************:60020
09/07/15 11:37:43 DEBUG client.HConnectionManager$TableServers: Found ROOT at **************:60020
09/07/15 11:37:44 DEBUG client.HConnectionManager$TableServers: Cached location address: **************:60020, regioninfo: REGION => {NAME => '.META.,,1', STARTKEY => '', ENDKEY => '', ENCODED => 1028785192, TABLE => {{NAME => '.META.', IS_META => 'true', MEMSTORE_FLUSHSIZE => '16384', FAMILIES => [{NAME => 'historian', COMPRESSION => 'NONE', VERSIONS => '2147483647', TTL => '604800', BLOCKSIZE => '8192', IN_MEMORY => 'false', BLOCKCACHE => 'false'}, {NAME => 'info', COMPRESSION => 'NONE', VERSIONS => '10', TTL => '2147483647', BLOCKSIZE => '8192', IN_MEMORY => 'false', BLOCKCACHE => 'true'}]}}
09/07/15 11:37:47 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got **************:60020
09/07/15 11:37:47 DEBUG client.HConnectionManager$TableServers: Found ROOT at **************:60020
09/07/15 11:37:47 DEBUG client.HConnectionManager$TableServers: Cached location address: **************:60020, regioninfo: REGION => {NAME => '.META.,,1', STARTKEY => '', ENDKEY => '', ENCODED => 1028785192, TABLE => {{NAME => '.META.', IS_META => 'true', MEMSTORE_FLUSHSIZE => '16384', FAMILIES => [{NAME => 'historian', COMPRESSION => 'NONE', VERSIONS => '2147483647', TTL => '604800', BLOCKSIZE => '8192', IN_MEMORY => 'false', BLOCKCACHE => 'false'}, {NAME => 'info', COMPRESSION => 'NONE', VERSIONS => '10', TTL => '2147483647', BLOCKSIZE => '8192', IN_MEMORY => 'false', BLOCKCACHE => 'true'}]}}
09/07/15 11:39:37 DEBUG zookeeper.ZooKeeperWrapper: Closed connection with ZooKeeper
09/07/15 11:39:37 INFO client.HBaseAdmin: Deleted t5
0 row(s) in 3.3580 seconds
0 row(s) in 118.0930 seconds
Wed Jul 15 11:39:42 EDT 2009

09/07/15 11:36:53 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got **************:60020
09/07/15 11:36:54 DEBUG client.HConnectionManager$TableServers: Found ROOT at **************:60020
09/07/15 11:36:54 DEBUG client.HConnectionManager$TableServers: Cached location address: **************:60020, regioninfo: REGION => {NAME => '.META.,,1', STARTKEY => '', ENDKEY => '', ENCODED => 1028785192, TABLE => {{NAME => '.META.', IS_META => 'true', MEMSTORE_FLUSHSIZE => '16384', FAMILIES => [{NAME => 'historian', COMPRESSION => 'NONE', VERSIONS => '2147483647', TTL => '604800', BLOCKSIZE => '8192', IN_MEMORY => 'false', BLOCKCACHE => 'false'}, {NAME => 'info', COMPRESSION => 'NONE', VERSIONS => '10', TTL => '2147483647', BLOCKSIZE => '8192', IN_MEMORY => 'false', BLOCKCACHE => 'true'}]}}
09/07/15 11:37:04 DEBUG client.HBaseAdmin: Sleep. Waiting for all regions to be disabled from t5
09/07/15 11:37:06 DEBUG client.HBaseAdmin: Wake. Waiting for all regions to be disabled from t5
09/07/15 11:37:06 DEBUG client.HConnectionManager$TableServers: Cache hit for row <> in tableName .META.: location server **************:60020, location region name .META.,,1
09/07/15 11:37:09 DEBUG client.HConnectionManager$TableServers: Cache hit for row <> in tableName .META.: location server **************60020, location region name .META.,,1
09/07/15 11:37:16 INFO client.HBaseAdmin: Disabled t5
0 row(s) in 51.0380 seconds
Wed Jul 15 11:37:16 EDT 2009





> After  a large truncating table HBase becomes unresponsive
> ----------------------------------------------------------
>
>                 Key: HBASE-1359
>                 URL: https://issues.apache.org/jira/browse/HBASE-1359
>             Project: Hadoop HBase
>          Issue Type: Bug
>         Environment: linux  r769665
>            Reporter: Alex Newman
>            Assignee: Alex Newman
>            Priority: Critical
>             Fix For: 0.20.0
>
>
> If you see **** I removed and ip or something for security reasons
> Once I truncate the table, hbase freaks out for about 10 seconds and all the thrift servers die.
> Thrift server log:
> 2009-04-02 12:09:08,971 INFO org.apache.hadoop.ipc.HBaseClass: Retrying connect 
> to server: /*****:60020. Already tried 0 time(s).
> You see this a bunch of times and then it times out
> The hbase shell
> nhbase(main):001:0> truncate 't2'
> 09/04/30 13:01:08 INFO zookeeper.ZooKeeperWrapper: Quorum servers: 
> ****
> Truncating t2; it may take a while
> Disabling table...
> 09/04/30 13:01:19 INFO client.HBaseAdmin: Disabled t2
> 0 row(s) in 10.3417 seconds
> Dropping table...
> 09/04/30 13:01:19 INFO client.HBaseAdmin: Deleted t2
> 0 row(s) in 0.1592 seconds
> Creating table...
> 0 row(s) in 14.7567 seconds
> hbase(main):002:0> lsit
> NameError: undefined local variable or method `lsit' for #<Object:0x3bbe9a50>
>         from (hbase):3
> hbase(main):003:0> lsit
> NameError: undefined local variable or method `lsit' for #<Object:0x3bbe9a50>
>         from (hbase):4
> hbase(main):004:0> list
> NativeException: java.lang.NullPointerException: null
>         from org/apache/hadoop/hbase/client/HConnectionManager.java:344:in `proc
> essRow' 
>       from org/apache/hadoop/hbase/client/MetaScanner.java:64:in `metaScan'
>         from org/apache/hadoop/hbase/client/MetaScanner.java:29:in `metaScan'
>         from org/apache/hadoop/hbase/client/HConnectionManager.java:351:in `list
> Tables' 
>         from org/apache/hadoop/hbase/client/HBaseAdmin.java:121:in `listTables'
>         from sun/reflect/NativeMethodAccessorImpl.java:-2:in `invoke0'
>         from sun/reflect/NativeMethodAccessorImpl.java:39:in `invoke'
>         from sun/reflect/DelegatingMethodAccessorImpl.java:25:in `invoke'
>         from java/lang/reflect/Method.java:597:in `invoke'
>         from org/jruby/javasupport/JavaMethod.java:298:in `invokeWithExceptionHa
> ndling' 
>         from org/jruby/javasupport/JavaMethod.java:259:in `invoke'
>         from org/jruby/java/invokers/InstanceMethodInvoker.java:36:in `call'
>         from org/jruby/runtime/callsite/CachingCallSite.java:260:in `cacheAndCal
> l'
>         from org/jruby/runtime/callsite/CachingCallSite.java:75:in `call'
>         from org/jruby/ast/CallNoArgNode.java:61:in `interpret'
>         from org/jruby/ast/ForNode.java:101:in `interpret'
> ... 113 levels...
>         from org/jruby/internal/runtime/methods/DynamicMethod.java:226:in `call'
>         from org/jruby/internal/runtime/methods/CompiledMethod.java:216:in `call
> '
>         from org/jruby/internal/runtime/methods/CompiledMethod.java:71:in `call'
>         from org/jruby/runtime/callsite/CachingCallSite.java:260:in `cacheAndCal
> l'
>         from org/jruby/runtime/callsite/CachingCallSite.java:75:in `call'
>         from home/fds/ts/hadoop/hbase/bin/$_dot_dot_/bin/hirb.rb:441:in `__file_
> _'
>         from home/fds/ts/hadoop/hbase/bin/$_dot_dot_/bin/hirb.rb:-1:in `__file__
> '
>         from home/fds/ts/hadoop/hbase/bin/$_dot_dot_/bin/hirb.rb:-1:in `load'
>         from org/jruby/Ruby.java:564:in `runScript'
>         from org/jruby/Ruby.java:467:in `runNormally'
>         from org/jruby/Ruby.java:340:in `runFromMain'
>         from org/jruby/Main.java:214:in `run'
>         from org/jruby/Main.java:100:in `run'
>         from org/jruby/Main.java:84:in `main'
>         from /home/fds/ts/hadoop/hbase/bin/../bin/hirb.rb:300:in `list'
>         from (hbase):5hbase(main):005:0>
> hbase(main):006:0* 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Assigned: (HBASE-1359) After a large truncating table HBase becomes unresponsive

Posted by "Alex Newman (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HBASE-1359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Alex Newman reassigned HBASE-1359:
----------------------------------

    Assignee: Alex Newman

> After  a large truncating table HBase becomes unresponsive
> ----------------------------------------------------------
>
>                 Key: HBASE-1359
>                 URL: https://issues.apache.org/jira/browse/HBASE-1359
>             Project: Hadoop HBase
>          Issue Type: Bug
>         Environment: linux  r769665
>            Reporter: Alex Newman
>            Assignee: Alex Newman
>
> If you see **** I removed and ip or something for security reasons
> Once I truncate the table, hbase freaks out for about 10 seconds and all the thrift servers die.
> Thrift server log:
> 2009-04-02 12:09:08,971 INFO org.apache.hadoop.ipc.HBaseClass: Retrying connect 
> to server: /*****:60020. Already tried 0 time(s).
> You see this a bunch of times and then it times out
> The hbase shell
> nhbase(main):001:0> truncate 't2'
> 09/04/30 13:01:08 INFO zookeeper.ZooKeeperWrapper: Quorum servers: 
> ****
> Truncating t2; it may take a while
> Disabling table...
> 09/04/30 13:01:19 INFO client.HBaseAdmin: Disabled t2
> 0 row(s) in 10.3417 seconds
> Dropping table...
> 09/04/30 13:01:19 INFO client.HBaseAdmin: Deleted t2
> 0 row(s) in 0.1592 seconds
> Creating table...
> 0 row(s) in 14.7567 seconds
> hbase(main):002:0> lsit
> NameError: undefined local variable or method `lsit' for #<Object:0x3bbe9a50>
>         from (hbase):3
> hbase(main):003:0> lsit
> NameError: undefined local variable or method `lsit' for #<Object:0x3bbe9a50>
>         from (hbase):4
> hbase(main):004:0> list
> NativeException: java.lang.NullPointerException: null
>         from org/apache/hadoop/hbase/client/HConnectionManager.java:344:in `proc
> essRow' 
>       from org/apache/hadoop/hbase/client/MetaScanner.java:64:in `metaScan'
>         from org/apache/hadoop/hbase/client/MetaScanner.java:29:in `metaScan'
>         from org/apache/hadoop/hbase/client/HConnectionManager.java:351:in `list
> Tables' 
>         from org/apache/hadoop/hbase/client/HBaseAdmin.java:121:in `listTables'
>         from sun/reflect/NativeMethodAccessorImpl.java:-2:in `invoke0'
>         from sun/reflect/NativeMethodAccessorImpl.java:39:in `invoke'
>         from sun/reflect/DelegatingMethodAccessorImpl.java:25:in `invoke'
>         from java/lang/reflect/Method.java:597:in `invoke'
>         from org/jruby/javasupport/JavaMethod.java:298:in `invokeWithExceptionHa
> ndling' 
>         from org/jruby/javasupport/JavaMethod.java:259:in `invoke'
>         from org/jruby/java/invokers/InstanceMethodInvoker.java:36:in `call'
>         from org/jruby/runtime/callsite/CachingCallSite.java:260:in `cacheAndCal
> l'
>         from org/jruby/runtime/callsite/CachingCallSite.java:75:in `call'
>         from org/jruby/ast/CallNoArgNode.java:61:in `interpret'
>         from org/jruby/ast/ForNode.java:101:in `interpret'
> ... 113 levels...
>         from org/jruby/internal/runtime/methods/DynamicMethod.java:226:in `call'
>         from org/jruby/internal/runtime/methods/CompiledMethod.java:216:in `call
> '
>         from org/jruby/internal/runtime/methods/CompiledMethod.java:71:in `call'
>         from org/jruby/runtime/callsite/CachingCallSite.java:260:in `cacheAndCal
> l'
>         from org/jruby/runtime/callsite/CachingCallSite.java:75:in `call'
>         from home/fds/ts/hadoop/hbase/bin/$_dot_dot_/bin/hirb.rb:441:in `__file_
> _'
>         from home/fds/ts/hadoop/hbase/bin/$_dot_dot_/bin/hirb.rb:-1:in `__file__
> '
>         from home/fds/ts/hadoop/hbase/bin/$_dot_dot_/bin/hirb.rb:-1:in `load'
>         from org/jruby/Ruby.java:564:in `runScript'
>         from org/jruby/Ruby.java:467:in `runNormally'
>         from org/jruby/Ruby.java:340:in `runFromMain'
>         from org/jruby/Main.java:214:in `run'
>         from org/jruby/Main.java:100:in `run'
>         from org/jruby/Main.java:84:in `main'
>         from /home/fds/ts/hadoop/hbase/bin/../bin/hirb.rb:300:in `list'
>         from (hbase):5hbase(main):005:0>
> hbase(main):006:0* 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (HBASE-1359) After a large truncating table HBase becomes unresponsive

Posted by "Alex Newman (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HBASE-1359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12733345#action_12733345 ] 

Alex Newman commented on HBASE-1359:
------------------------------------

I just dropped a +1000 region table and everything worked fine. I say close it out.

> After  a large truncating table HBase becomes unresponsive
> ----------------------------------------------------------
>
>                 Key: HBASE-1359
>                 URL: https://issues.apache.org/jira/browse/HBASE-1359
>             Project: Hadoop HBase
>          Issue Type: Bug
>         Environment: linux  r769665
>            Reporter: Alex Newman
>            Assignee: Alex Newman
>            Priority: Critical
>             Fix For: 0.20.0
>
>
> If you see **** I removed and ip or something for security reasons
> Once I truncate the table, hbase freaks out for about 10 seconds and all the thrift servers die.
> Thrift server log:
> 2009-04-02 12:09:08,971 INFO org.apache.hadoop.ipc.HBaseClass: Retrying connect 
> to server: /*****:60020. Already tried 0 time(s).
> You see this a bunch of times and then it times out
> The hbase shell
> nhbase(main):001:0> truncate 't2'
> 09/04/30 13:01:08 INFO zookeeper.ZooKeeperWrapper: Quorum servers: 
> ****
> Truncating t2; it may take a while
> Disabling table...
> 09/04/30 13:01:19 INFO client.HBaseAdmin: Disabled t2
> 0 row(s) in 10.3417 seconds
> Dropping table...
> 09/04/30 13:01:19 INFO client.HBaseAdmin: Deleted t2
> 0 row(s) in 0.1592 seconds
> Creating table...
> 0 row(s) in 14.7567 seconds
> hbase(main):002:0> lsit
> NameError: undefined local variable or method `lsit' for #<Object:0x3bbe9a50>
>         from (hbase):3
> hbase(main):003:0> lsit
> NameError: undefined local variable or method `lsit' for #<Object:0x3bbe9a50>
>         from (hbase):4
> hbase(main):004:0> list
> NativeException: java.lang.NullPointerException: null
>         from org/apache/hadoop/hbase/client/HConnectionManager.java:344:in `proc
> essRow' 
>       from org/apache/hadoop/hbase/client/MetaScanner.java:64:in `metaScan'
>         from org/apache/hadoop/hbase/client/MetaScanner.java:29:in `metaScan'
>         from org/apache/hadoop/hbase/client/HConnectionManager.java:351:in `list
> Tables' 
>         from org/apache/hadoop/hbase/client/HBaseAdmin.java:121:in `listTables'
>         from sun/reflect/NativeMethodAccessorImpl.java:-2:in `invoke0'
>         from sun/reflect/NativeMethodAccessorImpl.java:39:in `invoke'
>         from sun/reflect/DelegatingMethodAccessorImpl.java:25:in `invoke'
>         from java/lang/reflect/Method.java:597:in `invoke'
>         from org/jruby/javasupport/JavaMethod.java:298:in `invokeWithExceptionHa
> ndling' 
>         from org/jruby/javasupport/JavaMethod.java:259:in `invoke'
>         from org/jruby/java/invokers/InstanceMethodInvoker.java:36:in `call'
>         from org/jruby/runtime/callsite/CachingCallSite.java:260:in `cacheAndCal
> l'
>         from org/jruby/runtime/callsite/CachingCallSite.java:75:in `call'
>         from org/jruby/ast/CallNoArgNode.java:61:in `interpret'
>         from org/jruby/ast/ForNode.java:101:in `interpret'
> ... 113 levels...
>         from org/jruby/internal/runtime/methods/DynamicMethod.java:226:in `call'
>         from org/jruby/internal/runtime/methods/CompiledMethod.java:216:in `call
> '
>         from org/jruby/internal/runtime/methods/CompiledMethod.java:71:in `call'
>         from org/jruby/runtime/callsite/CachingCallSite.java:260:in `cacheAndCal
> l'
>         from org/jruby/runtime/callsite/CachingCallSite.java:75:in `call'
>         from home/fds/ts/hadoop/hbase/bin/$_dot_dot_/bin/hirb.rb:441:in `__file_
> _'
>         from home/fds/ts/hadoop/hbase/bin/$_dot_dot_/bin/hirb.rb:-1:in `__file__
> '
>         from home/fds/ts/hadoop/hbase/bin/$_dot_dot_/bin/hirb.rb:-1:in `load'
>         from org/jruby/Ruby.java:564:in `runScript'
>         from org/jruby/Ruby.java:467:in `runNormally'
>         from org/jruby/Ruby.java:340:in `runFromMain'
>         from org/jruby/Main.java:214:in `run'
>         from org/jruby/Main.java:100:in `run'
>         from org/jruby/Main.java:84:in `main'
>         from /home/fds/ts/hadoop/hbase/bin/../bin/hirb.rb:300:in `list'
>         from (hbase):5hbase(main):005:0>
> hbase(main):006:0* 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (HBASE-1359) After a large truncating table HBase becomes unresponsive

Posted by "stack (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HBASE-1359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12733371#action_12733371 ] 

stack commented on HBASE-1359:
------------------------------

Work in hbase-1583 fixed this.

> After  a large truncating table HBase becomes unresponsive
> ----------------------------------------------------------
>
>                 Key: HBASE-1359
>                 URL: https://issues.apache.org/jira/browse/HBASE-1359
>             Project: Hadoop HBase
>          Issue Type: Bug
>         Environment: linux  r769665
>            Reporter: Alex Newman
>            Assignee: Alex Newman
>            Priority: Critical
>             Fix For: 0.20.0
>
>
> If you see **** I removed and ip or something for security reasons
> Once I truncate the table, hbase freaks out for about 10 seconds and all the thrift servers die.
> Thrift server log:
> 2009-04-02 12:09:08,971 INFO org.apache.hadoop.ipc.HBaseClass: Retrying connect 
> to server: /*****:60020. Already tried 0 time(s).
> You see this a bunch of times and then it times out
> The hbase shell
> nhbase(main):001:0> truncate 't2'
> 09/04/30 13:01:08 INFO zookeeper.ZooKeeperWrapper: Quorum servers: 
> ****
> Truncating t2; it may take a while
> Disabling table...
> 09/04/30 13:01:19 INFO client.HBaseAdmin: Disabled t2
> 0 row(s) in 10.3417 seconds
> Dropping table...
> 09/04/30 13:01:19 INFO client.HBaseAdmin: Deleted t2
> 0 row(s) in 0.1592 seconds
> Creating table...
> 0 row(s) in 14.7567 seconds
> hbase(main):002:0> lsit
> NameError: undefined local variable or method `lsit' for #<Object:0x3bbe9a50>
>         from (hbase):3
> hbase(main):003:0> lsit
> NameError: undefined local variable or method `lsit' for #<Object:0x3bbe9a50>
>         from (hbase):4
> hbase(main):004:0> list
> NativeException: java.lang.NullPointerException: null
>         from org/apache/hadoop/hbase/client/HConnectionManager.java:344:in `proc
> essRow' 
>       from org/apache/hadoop/hbase/client/MetaScanner.java:64:in `metaScan'
>         from org/apache/hadoop/hbase/client/MetaScanner.java:29:in `metaScan'
>         from org/apache/hadoop/hbase/client/HConnectionManager.java:351:in `list
> Tables' 
>         from org/apache/hadoop/hbase/client/HBaseAdmin.java:121:in `listTables'
>         from sun/reflect/NativeMethodAccessorImpl.java:-2:in `invoke0'
>         from sun/reflect/NativeMethodAccessorImpl.java:39:in `invoke'
>         from sun/reflect/DelegatingMethodAccessorImpl.java:25:in `invoke'
>         from java/lang/reflect/Method.java:597:in `invoke'
>         from org/jruby/javasupport/JavaMethod.java:298:in `invokeWithExceptionHa
> ndling' 
>         from org/jruby/javasupport/JavaMethod.java:259:in `invoke'
>         from org/jruby/java/invokers/InstanceMethodInvoker.java:36:in `call'
>         from org/jruby/runtime/callsite/CachingCallSite.java:260:in `cacheAndCal
> l'
>         from org/jruby/runtime/callsite/CachingCallSite.java:75:in `call'
>         from org/jruby/ast/CallNoArgNode.java:61:in `interpret'
>         from org/jruby/ast/ForNode.java:101:in `interpret'
> ... 113 levels...
>         from org/jruby/internal/runtime/methods/DynamicMethod.java:226:in `call'
>         from org/jruby/internal/runtime/methods/CompiledMethod.java:216:in `call
> '
>         from org/jruby/internal/runtime/methods/CompiledMethod.java:71:in `call'
>         from org/jruby/runtime/callsite/CachingCallSite.java:260:in `cacheAndCal
> l'
>         from org/jruby/runtime/callsite/CachingCallSite.java:75:in `call'
>         from home/fds/ts/hadoop/hbase/bin/$_dot_dot_/bin/hirb.rb:441:in `__file_
> _'
>         from home/fds/ts/hadoop/hbase/bin/$_dot_dot_/bin/hirb.rb:-1:in `__file__
> '
>         from home/fds/ts/hadoop/hbase/bin/$_dot_dot_/bin/hirb.rb:-1:in `load'
>         from org/jruby/Ruby.java:564:in `runScript'
>         from org/jruby/Ruby.java:467:in `runNormally'
>         from org/jruby/Ruby.java:340:in `runFromMain'
>         from org/jruby/Main.java:214:in `run'
>         from org/jruby/Main.java:100:in `run'
>         from org/jruby/Main.java:84:in `main'
>         from /home/fds/ts/hadoop/hbase/bin/../bin/hirb.rb:300:in `list'
>         from (hbase):5hbase(main):005:0>
> hbase(main):006:0* 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Updated: (HBASE-1359) After a large truncating table HBase becomes unresponsive

Posted by "stack (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HBASE-1359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

stack updated HBASE-1359:
-------------------------

         Priority: Critical  (was: Major)
    Fix Version/s: 0.20.0

Pulling into 0.20.0.  Have to be able to disable tables.

posix, is it unreponsive on the shell?

> After  a large truncating table HBase becomes unresponsive
> ----------------------------------------------------------
>
>                 Key: HBASE-1359
>                 URL: https://issues.apache.org/jira/browse/HBASE-1359
>             Project: Hadoop HBase
>          Issue Type: Bug
>         Environment: linux  r769665
>            Reporter: Alex Newman
>            Assignee: Alex Newman
>            Priority: Critical
>             Fix For: 0.20.0
>
>
> If you see **** I removed and ip or something for security reasons
> Once I truncate the table, hbase freaks out for about 10 seconds and all the thrift servers die.
> Thrift server log:
> 2009-04-02 12:09:08,971 INFO org.apache.hadoop.ipc.HBaseClass: Retrying connect 
> to server: /*****:60020. Already tried 0 time(s).
> You see this a bunch of times and then it times out
> The hbase shell
> nhbase(main):001:0> truncate 't2'
> 09/04/30 13:01:08 INFO zookeeper.ZooKeeperWrapper: Quorum servers: 
> ****
> Truncating t2; it may take a while
> Disabling table...
> 09/04/30 13:01:19 INFO client.HBaseAdmin: Disabled t2
> 0 row(s) in 10.3417 seconds
> Dropping table...
> 09/04/30 13:01:19 INFO client.HBaseAdmin: Deleted t2
> 0 row(s) in 0.1592 seconds
> Creating table...
> 0 row(s) in 14.7567 seconds
> hbase(main):002:0> lsit
> NameError: undefined local variable or method `lsit' for #<Object:0x3bbe9a50>
>         from (hbase):3
> hbase(main):003:0> lsit
> NameError: undefined local variable or method `lsit' for #<Object:0x3bbe9a50>
>         from (hbase):4
> hbase(main):004:0> list
> NativeException: java.lang.NullPointerException: null
>         from org/apache/hadoop/hbase/client/HConnectionManager.java:344:in `proc
> essRow' 
>       from org/apache/hadoop/hbase/client/MetaScanner.java:64:in `metaScan'
>         from org/apache/hadoop/hbase/client/MetaScanner.java:29:in `metaScan'
>         from org/apache/hadoop/hbase/client/HConnectionManager.java:351:in `list
> Tables' 
>         from org/apache/hadoop/hbase/client/HBaseAdmin.java:121:in `listTables'
>         from sun/reflect/NativeMethodAccessorImpl.java:-2:in `invoke0'
>         from sun/reflect/NativeMethodAccessorImpl.java:39:in `invoke'
>         from sun/reflect/DelegatingMethodAccessorImpl.java:25:in `invoke'
>         from java/lang/reflect/Method.java:597:in `invoke'
>         from org/jruby/javasupport/JavaMethod.java:298:in `invokeWithExceptionHa
> ndling' 
>         from org/jruby/javasupport/JavaMethod.java:259:in `invoke'
>         from org/jruby/java/invokers/InstanceMethodInvoker.java:36:in `call'
>         from org/jruby/runtime/callsite/CachingCallSite.java:260:in `cacheAndCal
> l'
>         from org/jruby/runtime/callsite/CachingCallSite.java:75:in `call'
>         from org/jruby/ast/CallNoArgNode.java:61:in `interpret'
>         from org/jruby/ast/ForNode.java:101:in `interpret'
> ... 113 levels...
>         from org/jruby/internal/runtime/methods/DynamicMethod.java:226:in `call'
>         from org/jruby/internal/runtime/methods/CompiledMethod.java:216:in `call
> '
>         from org/jruby/internal/runtime/methods/CompiledMethod.java:71:in `call'
>         from org/jruby/runtime/callsite/CachingCallSite.java:260:in `cacheAndCal
> l'
>         from org/jruby/runtime/callsite/CachingCallSite.java:75:in `call'
>         from home/fds/ts/hadoop/hbase/bin/$_dot_dot_/bin/hirb.rb:441:in `__file_
> _'
>         from home/fds/ts/hadoop/hbase/bin/$_dot_dot_/bin/hirb.rb:-1:in `__file__
> '
>         from home/fds/ts/hadoop/hbase/bin/$_dot_dot_/bin/hirb.rb:-1:in `load'
>         from org/jruby/Ruby.java:564:in `runScript'
>         from org/jruby/Ruby.java:467:in `runNormally'
>         from org/jruby/Ruby.java:340:in `runFromMain'
>         from org/jruby/Main.java:214:in `run'
>         from org/jruby/Main.java:100:in `run'
>         from org/jruby/Main.java:84:in `main'
>         from /home/fds/ts/hadoop/hbase/bin/../bin/hirb.rb:300:in `list'
>         from (hbase):5hbase(main):005:0>
> hbase(main):006:0* 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (HBASE-1359) After a large truncating table HBase becomes unresponsive

Posted by "Jonathan Gray (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HBASE-1359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12733346#action_12733346 ] 

Jonathan Gray commented on HBASE-1359:
--------------------------------------

+1 on closing as fixed by HBASE-1583

> After  a large truncating table HBase becomes unresponsive
> ----------------------------------------------------------
>
>                 Key: HBASE-1359
>                 URL: https://issues.apache.org/jira/browse/HBASE-1359
>             Project: Hadoop HBase
>          Issue Type: Bug
>         Environment: linux  r769665
>            Reporter: Alex Newman
>            Assignee: Alex Newman
>            Priority: Critical
>             Fix For: 0.20.0
>
>
> If you see **** I removed and ip or something for security reasons
> Once I truncate the table, hbase freaks out for about 10 seconds and all the thrift servers die.
> Thrift server log:
> 2009-04-02 12:09:08,971 INFO org.apache.hadoop.ipc.HBaseClass: Retrying connect 
> to server: /*****:60020. Already tried 0 time(s).
> You see this a bunch of times and then it times out
> The hbase shell
> nhbase(main):001:0> truncate 't2'
> 09/04/30 13:01:08 INFO zookeeper.ZooKeeperWrapper: Quorum servers: 
> ****
> Truncating t2; it may take a while
> Disabling table...
> 09/04/30 13:01:19 INFO client.HBaseAdmin: Disabled t2
> 0 row(s) in 10.3417 seconds
> Dropping table...
> 09/04/30 13:01:19 INFO client.HBaseAdmin: Deleted t2
> 0 row(s) in 0.1592 seconds
> Creating table...
> 0 row(s) in 14.7567 seconds
> hbase(main):002:0> lsit
> NameError: undefined local variable or method `lsit' for #<Object:0x3bbe9a50>
>         from (hbase):3
> hbase(main):003:0> lsit
> NameError: undefined local variable or method `lsit' for #<Object:0x3bbe9a50>
>         from (hbase):4
> hbase(main):004:0> list
> NativeException: java.lang.NullPointerException: null
>         from org/apache/hadoop/hbase/client/HConnectionManager.java:344:in `proc
> essRow' 
>       from org/apache/hadoop/hbase/client/MetaScanner.java:64:in `metaScan'
>         from org/apache/hadoop/hbase/client/MetaScanner.java:29:in `metaScan'
>         from org/apache/hadoop/hbase/client/HConnectionManager.java:351:in `list
> Tables' 
>         from org/apache/hadoop/hbase/client/HBaseAdmin.java:121:in `listTables'
>         from sun/reflect/NativeMethodAccessorImpl.java:-2:in `invoke0'
>         from sun/reflect/NativeMethodAccessorImpl.java:39:in `invoke'
>         from sun/reflect/DelegatingMethodAccessorImpl.java:25:in `invoke'
>         from java/lang/reflect/Method.java:597:in `invoke'
>         from org/jruby/javasupport/JavaMethod.java:298:in `invokeWithExceptionHa
> ndling' 
>         from org/jruby/javasupport/JavaMethod.java:259:in `invoke'
>         from org/jruby/java/invokers/InstanceMethodInvoker.java:36:in `call'
>         from org/jruby/runtime/callsite/CachingCallSite.java:260:in `cacheAndCal
> l'
>         from org/jruby/runtime/callsite/CachingCallSite.java:75:in `call'
>         from org/jruby/ast/CallNoArgNode.java:61:in `interpret'
>         from org/jruby/ast/ForNode.java:101:in `interpret'
> ... 113 levels...
>         from org/jruby/internal/runtime/methods/DynamicMethod.java:226:in `call'
>         from org/jruby/internal/runtime/methods/CompiledMethod.java:216:in `call
> '
>         from org/jruby/internal/runtime/methods/CompiledMethod.java:71:in `call'
>         from org/jruby/runtime/callsite/CachingCallSite.java:260:in `cacheAndCal
> l'
>         from org/jruby/runtime/callsite/CachingCallSite.java:75:in `call'
>         from home/fds/ts/hadoop/hbase/bin/$_dot_dot_/bin/hirb.rb:441:in `__file_
> _'
>         from home/fds/ts/hadoop/hbase/bin/$_dot_dot_/bin/hirb.rb:-1:in `__file__
> '
>         from home/fds/ts/hadoop/hbase/bin/$_dot_dot_/bin/hirb.rb:-1:in `load'
>         from org/jruby/Ruby.java:564:in `runScript'
>         from org/jruby/Ruby.java:467:in `runNormally'
>         from org/jruby/Ruby.java:340:in `runFromMain'
>         from org/jruby/Main.java:214:in `run'
>         from org/jruby/Main.java:100:in `run'
>         from org/jruby/Main.java:84:in `main'
>         from /home/fds/ts/hadoop/hbase/bin/../bin/hirb.rb:300:in `list'
>         from (hbase):5hbase(main):005:0>
> hbase(main):006:0* 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (HBASE-1359) After a large truncating table HBase becomes unresponsive

Posted by "stack (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HBASE-1359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12731607#action_12731607 ] 

stack commented on HBASE-1359:
------------------------------

Eventually I got this on repeated disablings:

{code}
09/07/15 18:15:46 [main] DEBUG client.HConnectionManager$TableServers: Cache hit for row <> in tableName .META.: location server 208.76.44.141:60020, location region name .META.,,1
09/07/15 18:15:47 [main] ERROR client.HConnectionManager$TableServers: Result=keyvalues={TestTable,\x00\x05\x05\x05\x01\x08\x01\x06\x04\x09,1247619405666/info:server/1247679801551/Put/vlen=19, TestTable,\x00\x05\x05\x05\x01\x08\x01\x06\x04\x09,1247619405666/info:serverstartcode/1247679801551/Put/vlen=8}
NativeException: java.lang.NullPointerException: null
        from org/apache/hadoop/hbase/util/Writables.java:74:in `getWritable'
        from org/apache/hadoop/hbase/util/Writables.java:118:in `getHRegionInfo'
        from org/apache/hadoop/hbase/client/HConnectionManager.java:356:in `processRow'
...
{code}

> After  a large truncating table HBase becomes unresponsive
> ----------------------------------------------------------
>
>                 Key: HBASE-1359
>                 URL: https://issues.apache.org/jira/browse/HBASE-1359
>             Project: Hadoop HBase
>          Issue Type: Bug
>         Environment: linux  r769665
>            Reporter: Alex Newman
>            Assignee: Alex Newman
>            Priority: Critical
>             Fix For: 0.20.0
>
>
> If you see **** I removed and ip or something for security reasons
> Once I truncate the table, hbase freaks out for about 10 seconds and all the thrift servers die.
> Thrift server log:
> 2009-04-02 12:09:08,971 INFO org.apache.hadoop.ipc.HBaseClass: Retrying connect 
> to server: /*****:60020. Already tried 0 time(s).
> You see this a bunch of times and then it times out
> The hbase shell
> nhbase(main):001:0> truncate 't2'
> 09/04/30 13:01:08 INFO zookeeper.ZooKeeperWrapper: Quorum servers: 
> ****
> Truncating t2; it may take a while
> Disabling table...
> 09/04/30 13:01:19 INFO client.HBaseAdmin: Disabled t2
> 0 row(s) in 10.3417 seconds
> Dropping table...
> 09/04/30 13:01:19 INFO client.HBaseAdmin: Deleted t2
> 0 row(s) in 0.1592 seconds
> Creating table...
> 0 row(s) in 14.7567 seconds
> hbase(main):002:0> lsit
> NameError: undefined local variable or method `lsit' for #<Object:0x3bbe9a50>
>         from (hbase):3
> hbase(main):003:0> lsit
> NameError: undefined local variable or method `lsit' for #<Object:0x3bbe9a50>
>         from (hbase):4
> hbase(main):004:0> list
> NativeException: java.lang.NullPointerException: null
>         from org/apache/hadoop/hbase/client/HConnectionManager.java:344:in `proc
> essRow' 
>       from org/apache/hadoop/hbase/client/MetaScanner.java:64:in `metaScan'
>         from org/apache/hadoop/hbase/client/MetaScanner.java:29:in `metaScan'
>         from org/apache/hadoop/hbase/client/HConnectionManager.java:351:in `list
> Tables' 
>         from org/apache/hadoop/hbase/client/HBaseAdmin.java:121:in `listTables'
>         from sun/reflect/NativeMethodAccessorImpl.java:-2:in `invoke0'
>         from sun/reflect/NativeMethodAccessorImpl.java:39:in `invoke'
>         from sun/reflect/DelegatingMethodAccessorImpl.java:25:in `invoke'
>         from java/lang/reflect/Method.java:597:in `invoke'
>         from org/jruby/javasupport/JavaMethod.java:298:in `invokeWithExceptionHa
> ndling' 
>         from org/jruby/javasupport/JavaMethod.java:259:in `invoke'
>         from org/jruby/java/invokers/InstanceMethodInvoker.java:36:in `call'
>         from org/jruby/runtime/callsite/CachingCallSite.java:260:in `cacheAndCal
> l'
>         from org/jruby/runtime/callsite/CachingCallSite.java:75:in `call'
>         from org/jruby/ast/CallNoArgNode.java:61:in `interpret'
>         from org/jruby/ast/ForNode.java:101:in `interpret'
> ... 113 levels...
>         from org/jruby/internal/runtime/methods/DynamicMethod.java:226:in `call'
>         from org/jruby/internal/runtime/methods/CompiledMethod.java:216:in `call
> '
>         from org/jruby/internal/runtime/methods/CompiledMethod.java:71:in `call'
>         from org/jruby/runtime/callsite/CachingCallSite.java:260:in `cacheAndCal
> l'
>         from org/jruby/runtime/callsite/CachingCallSite.java:75:in `call'
>         from home/fds/ts/hadoop/hbase/bin/$_dot_dot_/bin/hirb.rb:441:in `__file_
> _'
>         from home/fds/ts/hadoop/hbase/bin/$_dot_dot_/bin/hirb.rb:-1:in `__file__
> '
>         from home/fds/ts/hadoop/hbase/bin/$_dot_dot_/bin/hirb.rb:-1:in `load'
>         from org/jruby/Ruby.java:564:in `runScript'
>         from org/jruby/Ruby.java:467:in `runNormally'
>         from org/jruby/Ruby.java:340:in `runFromMain'
>         from org/jruby/Main.java:214:in `run'
>         from org/jruby/Main.java:100:in `run'
>         from org/jruby/Main.java:84:in `main'
>         from /home/fds/ts/hadoop/hbase/bin/../bin/hirb.rb:300:in `list'
>         from (hbase):5hbase(main):005:0>
> hbase(main):006:0* 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (HBASE-1359) After a large truncating table HBase becomes unresponsive

Posted by "Alex Newman (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HBASE-1359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12732546#action_12732546 ] 

Alex Newman commented on HBASE-1359:
------------------------------------

Rerunning my test.

On Jul 17, 2009 12:41am, "stack (JIRA)" <ji...@apache.org> wrote:









































































































> After  a large truncating table HBase becomes unresponsive
> ----------------------------------------------------------
>
>                 Key: HBASE-1359
>                 URL: https://issues.apache.org/jira/browse/HBASE-1359
>             Project: Hadoop HBase
>          Issue Type: Bug
>         Environment: linux  r769665
>            Reporter: Alex Newman
>            Assignee: Alex Newman
>            Priority: Critical
>             Fix For: 0.20.0
>
>
> If you see **** I removed and ip or something for security reasons
> Once I truncate the table, hbase freaks out for about 10 seconds and all the thrift servers die.
> Thrift server log:
> 2009-04-02 12:09:08,971 INFO org.apache.hadoop.ipc.HBaseClass: Retrying connect 
> to server: /*****:60020. Already tried 0 time(s).
> You see this a bunch of times and then it times out
> The hbase shell
> nhbase(main):001:0> truncate 't2'
> 09/04/30 13:01:08 INFO zookeeper.ZooKeeperWrapper: Quorum servers: 
> ****
> Truncating t2; it may take a while
> Disabling table...
> 09/04/30 13:01:19 INFO client.HBaseAdmin: Disabled t2
> 0 row(s) in 10.3417 seconds
> Dropping table...
> 09/04/30 13:01:19 INFO client.HBaseAdmin: Deleted t2
> 0 row(s) in 0.1592 seconds
> Creating table...
> 0 row(s) in 14.7567 seconds
> hbase(main):002:0> lsit
> NameError: undefined local variable or method `lsit' for #<Object:0x3bbe9a50>
>         from (hbase):3
> hbase(main):003:0> lsit
> NameError: undefined local variable or method `lsit' for #<Object:0x3bbe9a50>
>         from (hbase):4
> hbase(main):004:0> list
> NativeException: java.lang.NullPointerException: null
>         from org/apache/hadoop/hbase/client/HConnectionManager.java:344:in `proc
> essRow' 
>       from org/apache/hadoop/hbase/client/MetaScanner.java:64:in `metaScan'
>         from org/apache/hadoop/hbase/client/MetaScanner.java:29:in `metaScan'
>         from org/apache/hadoop/hbase/client/HConnectionManager.java:351:in `list
> Tables' 
>         from org/apache/hadoop/hbase/client/HBaseAdmin.java:121:in `listTables'
>         from sun/reflect/NativeMethodAccessorImpl.java:-2:in `invoke0'
>         from sun/reflect/NativeMethodAccessorImpl.java:39:in `invoke'
>         from sun/reflect/DelegatingMethodAccessorImpl.java:25:in `invoke'
>         from java/lang/reflect/Method.java:597:in `invoke'
>         from org/jruby/javasupport/JavaMethod.java:298:in `invokeWithExceptionHa
> ndling' 
>         from org/jruby/javasupport/JavaMethod.java:259:in `invoke'
>         from org/jruby/java/invokers/InstanceMethodInvoker.java:36:in `call'
>         from org/jruby/runtime/callsite/CachingCallSite.java:260:in `cacheAndCal
> l'
>         from org/jruby/runtime/callsite/CachingCallSite.java:75:in `call'
>         from org/jruby/ast/CallNoArgNode.java:61:in `interpret'
>         from org/jruby/ast/ForNode.java:101:in `interpret'
> ... 113 levels...
>         from org/jruby/internal/runtime/methods/DynamicMethod.java:226:in `call'
>         from org/jruby/internal/runtime/methods/CompiledMethod.java:216:in `call
> '
>         from org/jruby/internal/runtime/methods/CompiledMethod.java:71:in `call'
>         from org/jruby/runtime/callsite/CachingCallSite.java:260:in `cacheAndCal
> l'
>         from org/jruby/runtime/callsite/CachingCallSite.java:75:in `call'
>         from home/fds/ts/hadoop/hbase/bin/$_dot_dot_/bin/hirb.rb:441:in `__file_
> _'
>         from home/fds/ts/hadoop/hbase/bin/$_dot_dot_/bin/hirb.rb:-1:in `__file__
> '
>         from home/fds/ts/hadoop/hbase/bin/$_dot_dot_/bin/hirb.rb:-1:in `load'
>         from org/jruby/Ruby.java:564:in `runScript'
>         from org/jruby/Ruby.java:467:in `runNormally'
>         from org/jruby/Ruby.java:340:in `runFromMain'
>         from org/jruby/Main.java:214:in `run'
>         from org/jruby/Main.java:100:in `run'
>         from org/jruby/Main.java:84:in `main'
>         from /home/fds/ts/hadoop/hbase/bin/../bin/hirb.rb:300:in `list'
>         from (hbase):5hbase(main):005:0>
> hbase(main):006:0* 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (HBASE-1359) After a large truncating table HBase becomes unresponsive

Posted by "stack (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HBASE-1359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12731542#action_12731542 ] 

stack commented on HBASE-1359:
------------------------------

I just tried disable on table of 1500 regions:

{code}
09/07/15 16:17:27 [main] DEBUG client.HConnectionManager$TableServers: Cache hit for row <> in tableName .META.: location server 208.76.44.141:60020, location region name .META.,,1
NativeException: org.apache.hadoop.hbase.RegionException: Retries exhausted, it took too long to wait for the table TestTable to be disabled.
{code}

Doing it a few times, each time it gets above.

> After  a large truncating table HBase becomes unresponsive
> ----------------------------------------------------------
>
>                 Key: HBASE-1359
>                 URL: https://issues.apache.org/jira/browse/HBASE-1359
>             Project: Hadoop HBase
>          Issue Type: Bug
>         Environment: linux  r769665
>            Reporter: Alex Newman
>            Assignee: Alex Newman
>            Priority: Critical
>             Fix For: 0.20.0
>
>
> If you see **** I removed and ip or something for security reasons
> Once I truncate the table, hbase freaks out for about 10 seconds and all the thrift servers die.
> Thrift server log:
> 2009-04-02 12:09:08,971 INFO org.apache.hadoop.ipc.HBaseClass: Retrying connect 
> to server: /*****:60020. Already tried 0 time(s).
> You see this a bunch of times and then it times out
> The hbase shell
> nhbase(main):001:0> truncate 't2'
> 09/04/30 13:01:08 INFO zookeeper.ZooKeeperWrapper: Quorum servers: 
> ****
> Truncating t2; it may take a while
> Disabling table...
> 09/04/30 13:01:19 INFO client.HBaseAdmin: Disabled t2
> 0 row(s) in 10.3417 seconds
> Dropping table...
> 09/04/30 13:01:19 INFO client.HBaseAdmin: Deleted t2
> 0 row(s) in 0.1592 seconds
> Creating table...
> 0 row(s) in 14.7567 seconds
> hbase(main):002:0> lsit
> NameError: undefined local variable or method `lsit' for #<Object:0x3bbe9a50>
>         from (hbase):3
> hbase(main):003:0> lsit
> NameError: undefined local variable or method `lsit' for #<Object:0x3bbe9a50>
>         from (hbase):4
> hbase(main):004:0> list
> NativeException: java.lang.NullPointerException: null
>         from org/apache/hadoop/hbase/client/HConnectionManager.java:344:in `proc
> essRow' 
>       from org/apache/hadoop/hbase/client/MetaScanner.java:64:in `metaScan'
>         from org/apache/hadoop/hbase/client/MetaScanner.java:29:in `metaScan'
>         from org/apache/hadoop/hbase/client/HConnectionManager.java:351:in `list
> Tables' 
>         from org/apache/hadoop/hbase/client/HBaseAdmin.java:121:in `listTables'
>         from sun/reflect/NativeMethodAccessorImpl.java:-2:in `invoke0'
>         from sun/reflect/NativeMethodAccessorImpl.java:39:in `invoke'
>         from sun/reflect/DelegatingMethodAccessorImpl.java:25:in `invoke'
>         from java/lang/reflect/Method.java:597:in `invoke'
>         from org/jruby/javasupport/JavaMethod.java:298:in `invokeWithExceptionHa
> ndling' 
>         from org/jruby/javasupport/JavaMethod.java:259:in `invoke'
>         from org/jruby/java/invokers/InstanceMethodInvoker.java:36:in `call'
>         from org/jruby/runtime/callsite/CachingCallSite.java:260:in `cacheAndCal
> l'
>         from org/jruby/runtime/callsite/CachingCallSite.java:75:in `call'
>         from org/jruby/ast/CallNoArgNode.java:61:in `interpret'
>         from org/jruby/ast/ForNode.java:101:in `interpret'
> ... 113 levels...
>         from org/jruby/internal/runtime/methods/DynamicMethod.java:226:in `call'
>         from org/jruby/internal/runtime/methods/CompiledMethod.java:216:in `call
> '
>         from org/jruby/internal/runtime/methods/CompiledMethod.java:71:in `call'
>         from org/jruby/runtime/callsite/CachingCallSite.java:260:in `cacheAndCal
> l'
>         from org/jruby/runtime/callsite/CachingCallSite.java:75:in `call'
>         from home/fds/ts/hadoop/hbase/bin/$_dot_dot_/bin/hirb.rb:441:in `__file_
> _'
>         from home/fds/ts/hadoop/hbase/bin/$_dot_dot_/bin/hirb.rb:-1:in `__file__
> '
>         from home/fds/ts/hadoop/hbase/bin/$_dot_dot_/bin/hirb.rb:-1:in `load'
>         from org/jruby/Ruby.java:564:in `runScript'
>         from org/jruby/Ruby.java:467:in `runNormally'
>         from org/jruby/Ruby.java:340:in `runFromMain'
>         from org/jruby/Main.java:214:in `run'
>         from org/jruby/Main.java:100:in `run'
>         from org/jruby/Main.java:84:in `main'
>         from /home/fds/ts/hadoop/hbase/bin/../bin/hirb.rb:300:in `list'
>         from (hbase):5hbase(main):005:0>
> hbase(main):006:0* 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.