You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by "Ananth T. Sarathy" <an...@gmail.com> on 2009/10/20 22:32:37 UTC
Waiting forever on scanner iterator
I am running this code where
getScannerForAllRows(columns) just does return table.getScanner(columns);
and the table has setScannerCaching(100);
But it spins forever after getting the iterator. Why would that be? How can
I speed it up?
HBaseTableDataManagerImpl htdmni = new HBaseTableDataManagerImpl(
"GS_Applications");
String[] columns = { "Files:Name" };
log.info("Getting all Rows with Files");
Scanner s = htdmni.getScannerForAllRows(columns);
log.info("Got all Rows with Files");
log.info("Getting Iterator");
Iterator<RowResult> iter = s.iterator();
log.info("Got Iterator");
while (iter.hasNext())
{
log.info("Getting next Row");
RowResult rr = iter.next();
Ananth T Sarathy
Re: Waiting forever on scanner iterator
Posted by "Ananth T. Sarathy" <an...@gmail.com>.
solrry...
http://pastebin.com/m5aa78a09
Ananth T Sarathy
On Wed, Oct 21, 2009 at 2:41 PM, Ananth T. Sarathy <
ananth.t.sarathy@gmail.com> wrote:
> here's what I get
>
> Heap
> par new generation total 19136K, used 13714K [0x00002aaaafc40000,
> 0x00002aaab1100000, 0x00002aaab4f70000)
> eden space 17024K, 79% used [0x00002aaaafc40000, 0x00002aaab09816d8,
> 0x00002aaab0ce0000)
> from space 2112K, 6% used [0x00002aaab0ef0000, 0x00002aaab0f13470,
> 0x00002aaab1100000)
> to space 2112K, 0% used [0x00002aaab0ce0000, 0x00002aaab0ce0000,
> 0x00002aaab0ef0000)
> concurrent mark-sweep generation total 63872K, used 25097K
> [0x00002aaab4f70000, 0x00002aaab8dd0000, 0x00002aaaee440000)
> concurrent-mark-sweep perm gen total 30220K, used 18263K
> [0x00002aaaee440000, 0x00002aaaf01c3000, 0x00002aaaf3840000)
>
> 2009-10-21 14:40:58
> Full thread dump Java HotSpot(TM) 64-Bit Server VM (14.2-b01 mixed mode):
>
> "pool-1-thread-1" prio=10 tid=0x00002aaafc01a000 nid=0x49f waiting on
> condition [0x0000000042e56000]
> java.lang.Thread.State: TIMED_WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x00002aaab6559490> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at
> java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:198)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:1963)
> at java.util.concurrent.DelayQueue.take(DelayQueue.java:164)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:583)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:576)
> at
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
> at java.lang.Thread.run(Thread.java:619)
>
> "LruBlockCache.EvictionThread" daemon prio=10 tid=0x00002aaafc01f800
> nid=0x49e in Object.wait() [0x0000000042d55000]
> java.lang.Thread.State: WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> - waiting on <0x00002aaab6519890> (a
> org.apache.hadoop.hbase.io.hfile.LruBlockCache$EvictionThread)
> at java.lang.Object.wait(Object.java:485)
> at
> org.apache.hadoop.hbase.io.hfile.LruBlockCache$EvictionThread.run(LruBlockCache.java:512)
> - locked <0x00002aaab6519890> (a
> org.apache.hadoop.hbase.io.hfile.LruBlockCache$EvictionThread)
>
> "IPC Server handler 9 on 60020" daemon prio=10 tid=0x00002aaaf7c12000
> nid=0x49d waiting on condition [0x0000000042c54000]
> java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x00002aaab6434798> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at
> java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
> at
> java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:901)
>
> "IPC Server handler 8 on 60020" daemon prio=10 tid=0x00002aaaf7c0f800
> nid=0x49c waiting on condition [0x0000000042b53000]
> java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x00002aaab6434798> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at
> java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
> at
> java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:901)
>
> "IPC Server handler 7 on 60020" daemon prio=10 tid=0x00002aaaf7c0d800
> nid=0x49b waiting on condition [0x0000000042a52000]
> java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x00002aaab6434798> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at
> java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
> at
> java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:901)
>
> "IPC Server handler 6 on 60020" daemon prio=10 tid=0x00002aaafd08b800
> nid=0x49a waiting on condition [0x0000000042951000]
> java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x00002aaab6434798> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at
> java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
> at
> java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:901)
>
> "IPC Server handler 5 on 60020" daemon prio=10 tid=0x00002aaafd089800
> nid=0x499 waiting on condition [0x0000000042850000]
> java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x00002aaab6434798> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at
> java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
> at
> java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:901)
>
> "IPC Server handler 4 on 60020" daemon prio=10 tid=0x00002aaafd087000
> nid=0x498 waiting on condition [0x000000004274f000]
> java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x00002aaab6434798> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at
> java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
> at
> java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:901)
>
> "IPC Server handler 3 on 60020" daemon prio=10 tid=0x00002aaafd085000
> nid=0x497 waiting on condition [0x000000004264e000]
> java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x00002aaab6434798> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at
> java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
> at
> java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:901)
>
> "IPC Server handler 2 on 60020" daemon prio=10 tid=0x00002aaaf7adf800
> nid=0x496 waiting on condition [0x000000004254d000]
> java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x00002aaab6434798> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at
> java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
> at
> java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:901)
>
> "IPC Server handler 1 on 60020" daemon prio=10 tid=0x00002aaaf7adb800
> nid=0x495 waiting on condition [0x000000004244c000]
> java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x00002aaab6434798> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at
> java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
> at
> java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:901)
>
> "IPC Server handler 0 on 60020" daemon prio=10 tid=0x00002aaaf7adb000
> nid=0x494 waiting on condition [0x000000004234b000]
> java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x00002aaab6434798> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at
> java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
> at
> java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:901)
>
> "IPC Server listener on 60020" daemon prio=10 tid=0x00002aaaf7de9800
> nid=0x493 runnable [0x000000004224a000]
> java.lang.Thread.State: RUNNABLE
> at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
> at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:215)
> at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:65)
> at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
> - locked <0x00002aaab643b548> (a sun.nio.ch.Util$1)
> - locked <0x00002aaab643b530> (a
> java.util.Collections$UnmodifiableSet)
> - locked <0x00002aaab63ea7e0> (a sun.nio.ch.EPollSelectorImpl)
> at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
> at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:84)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Listener.run(HBaseServer.java:308)
>
> "IPC Server Responder" daemon prio=10 tid=0x00002aaaf7de9000 nid=0x492
> runnable [0x0000000042149000]
> java.lang.Thread.State: RUNNABLE
> at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
> at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:215)
> at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:65)
> at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
> - locked <0x00002aaab643b458> (a sun.nio.ch.Util$1)
> - locked <0x00002aaab643b440> (a
> java.util.Collections$UnmodifiableSet)
> - locked <0x00002aaab63ea768> (a sun.nio.ch.EPollSelectorImpl)
> at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Responder.run(HBaseServer.java:477)
>
> "regionserver/10.244.9.171:60020.safeMode" daemon prio=10
> tid=0x00002aaaf7c84000 nid=0x491 in Object.wait() [0x0000000042048000]
> java.lang.Thread.State: WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> - waiting on <0x00002aaab63911f8> (a
> java.util.concurrent.atomic.AtomicBoolean)
> at java.lang.Object.wait(Object.java:485)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer$CompactionLimitThread.run(HRegionServer.java:902)
> - locked <0x00002aaab63911f8> (a
> java.util.concurrent.atomic.AtomicBoolean)
>
> "Timer-0" daemon prio=10 tid=0x00002aaaf7de4000 nid=0x490 in Object.wait()
> [0x0000000041f47000]
> java.lang.Thread.State: TIMED_WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> - waiting on <0x00002aaab6558e60> (a java.util.TaskQueue)
> at java.util.TimerThread.mainLoop(Timer.java:509)
> - locked <0x00002aaab6558e60> (a java.util.TaskQueue)
> at java.util.TimerThread.run(Timer.java:462)
>
> "853527367@qtp0-1 - Acceptor0 SelectChannelConnector@0.0.0.0:60030"
> prio=10 tid=0x00002aaafcd3a800 nid=0x48f runnable [0x0000000041e46000]
> java.lang.Thread.State: RUNNABLE
> at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
> at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:215)
> at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:65)
> at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
> - locked <0x00002aaab651bf38> (a sun.nio.ch.Util$1)
> - locked <0x00002aaab651bf50> (a
> java.util.Collections$UnmodifiableSet)
> - locked <0x00002aaab6620c60> (a sun.nio.ch.EPollSelectorImpl)
> at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
> at
> org.mortbay.io.nio.SelectorManager$SelectSet.doSelect(SelectorManager.java:429)
> at
> org.mortbay.io.nio.SelectorManager.doSelect(SelectorManager.java:185)
> at
> org.mortbay.jetty.nio.SelectChannelConnector.accept(SelectChannelConnector.java:124)
> at
> org.mortbay.jetty.AbstractConnector$Acceptor.run(AbstractConnector.java:707)
> at
> org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:522)
>
> "1096304647@qtp0-0" prio=10 tid=0x00002aaaf7d5b800 nid=0x48e in
> Object.wait() [0x0000000041d45000]
> java.lang.Thread.State: TIMED_WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> - waiting on <0x00002aaab653acd0> (a
> org.mortbay.thread.QueuedThreadPool$PoolThread)
> at
> org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:565)
> - locked <0x00002aaab653acd0> (a
> org.mortbay.thread.QueuedThreadPool$PoolThread)
>
> "regionserver/10.244.9.171:60020.leaseChecker" prio=10
> tid=0x00002aaafc16a000 nid=0x48d waiting on condition [0x0000000041c44000]
> java.lang.Thread.State: TIMED_WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x00002aaab6434518> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at
> java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:198)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:1963)
> at java.util.concurrent.DelayQueue.poll(DelayQueue.java:209)
> at org.apache.hadoop.hbase.Leases.run(Leases.java:81)
>
> "regionserver/10.244.9.171:60020.majorCompactionChecker" daemon prio=10
> tid=0x00002aaafc168000 nid=0x48c waiting on condition [0x0000000041b43000]
> java.lang.Thread.State: TIMED_WAITING (sleeping)
> at java.lang.Thread.sleep(Native Method)
> at org.apache.hadoop.hbase.util.Sleeper.sleep(Sleeper.java:74)
> at org.apache.hadoop.hbase.Chore.run(Chore.java:76)
>
> "regionserver/10.244.9.171:60020.worker" daemon prio=10
> tid=0x00002aaafc228000 nid=0x48b waiting on condition [0x0000000041a42000]
> java.lang.Thread.State: TIMED_WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x00002aaab63a3b68> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at
> java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:198)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:1963)
> at
> java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:395)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer$Worker.run(HRegionServer.java:1459)
> at java.lang.Thread.run(Thread.java:619)
>
> "regionserver/10.244.9.171:60020.compactor" daemon prio=10
> tid=0x00002aaafc226000 nid=0x48a waiting on condition [0x0000000041941000]
> java.lang.Thread.State: TIMED_WAITING (sleeping)
> at java.lang.Thread.sleep(Native Method)
> at
> org.apache.hadoop.hbase.regionserver.CompactSplitThread.run(CompactSplitThread.java:79)
>
> "regionserver/10.244.9.171:60020.cacheFlusher" daemon prio=10
> tid=0x00002aaafc225000 nid=0x489 waiting on condition [0x0000000041840000]
> java.lang.Thread.State: TIMED_WAITING (sleeping)
> at java.lang.Thread.sleep(Native Method)
> at
> org.apache.hadoop.hbase.regionserver.MemStoreFlusher.run(MemStoreFlusher.java:137)
>
> "regionserver/10.244.9.171:60020.logFlusher" daemon prio=10
> tid=0x00002aaafc212000 nid=0x488 waiting on condition [0x000000004173f000]
> java.lang.Thread.State: TIMED_WAITING (sleeping)
> at java.lang.Thread.sleep(Native Method)
> at org.apache.hadoop.hbase.util.Sleeper.sleep(Sleeper.java:74)
> at org.apache.hadoop.hbase.Chore.run(Chore.java:76)
>
> "regionserver/10.244.9.171:60020.logRoller" daemon prio=10
> tid=0x00002aaafc211800 nid=0x487 in Object.wait() [0x000000004153d000]
> java.lang.Thread.State: TIMED_WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> - waiting on <0x00002aaab63913f0> (a
> java.util.concurrent.atomic.AtomicBoolean)
> at
> org.apache.hadoop.hbase.regionserver.LogRoller.run(LogRoller.java:65)
> - locked <0x00002aaab63913f0> (a
> java.util.concurrent.atomic.AtomicBoolean)
>
> "MultiThreadedHttpConnectionManager cleanup" daemon prio=10
> tid=0x00002aaaf7e14000 nid=0x486 in Object.wait() [0x000000004163e000]
> java.lang.Thread.State: TIMED_WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> - waiting on <0x00002aaab65446e0> (a
> java.lang.ref.ReferenceQueue$Lock)
> at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:118)
> - locked <0x00002aaab65446e0> (a java.lang.ref.ReferenceQueue$Lock)
> at
> org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$ReferenceQueueThread.run(MultiThreadedHttpConnectionManager.java:1082)
>
> "IPC Client (47) connection to /10.244.9.171:60000 from an unknown user"
> daemon prio=10 tid=0x00002aaaf80b6800 nid=0x46d in Object.wait()
> [0x000000004143c000]
> java.lang.Thread.State: TIMED_WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> - waiting on <0x00002aaab642ba08> (a
> org.apache.hadoop.hbase.ipc.HBaseClient$Connection)
> at
> org.apache.hadoop.hbase.ipc.HBaseClient$Connection.waitForWork(HBaseClient.java:405)
> - locked <0x00002aaab642ba08> (a
> org.apache.hadoop.hbase.ipc.HBaseClient$Connection)
> at
> org.apache.hadoop.hbase.ipc.HBaseClient$Connection.run(HBaseClient.java:448)
>
> "DestroyJavaVM" prio=10 tid=0x00002aaaf80b4800 nid=0x455 waiting on
> condition [0x0000000000000000]
> java.lang.Thread.State: RUNNABLE
>
> "regionserver/10.244.9.171:60020" prio=10 tid=0x00002aaaf80b3800 nid=0x467
> waiting on condition [0x000000004133b000]
> java.lang.Thread.State: TIMED_WAITING (sleeping)
> at java.lang.Thread.sleep(Native Method)
> at org.apache.hadoop.hbase.util.Sleeper.sleep(Sleeper.java:74)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:596)
> at java.lang.Thread.run(Thread.java:619)
>
> "main-EventThread" daemon prio=10 tid=0x00002aaaf80ab800 nid=0x466 waiting
> on condition [0x000000004123a000]
> java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x00002aaab6391598> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at
> java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
> at
> java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
> at
> org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:376)
>
> "main-SendThread" daemon prio=10 tid=0x00002aaaf80ab000 nid=0x465 runnable
> [0x0000000041139000]
> java.lang.Thread.State: RUNNABLE
> at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
> at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:215)
> at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:65)
> at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
> - locked <0x00002aaab6418908> (a sun.nio.ch.Util$1)
> - locked <0x00002aaab64188f0> (a
> java.util.Collections$UnmodifiableSet)
> - locked <0x00002aaab63fb0a0> (a sun.nio.ch.EPollSelectorImpl)
> at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
> at
> org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:873)
>
> "Low Memory Detector" daemon prio=10 tid=0x00002aaaf7858800 nid=0x463
> runnable [0x0000000000000000]
> java.lang.Thread.State: RUNNABLE
>
> "CompilerThread1" daemon prio=10 tid=0x00002aaaf7856000 nid=0x462 waiting
> on condition [0x0000000000000000]
> java.lang.Thread.State: RUNNABLE
>
> "CompilerThread0" daemon prio=10 tid=0x00002aaaf7852800 nid=0x461 waiting
> on condition [0x0000000000000000]
> java.lang.Thread.State: RUNNABLE
>
> "Signal Dispatcher" daemon prio=10 tid=0x00002aaaf76b9800 nid=0x460 waiting
> on condition [0x0000000000000000]
> java.lang.Thread.State: RUNNABLE
>
> "Surrogate Locker Thread (CMS)" daemon prio=10 tid=0x00002aaaf76b7800
> nid=0x45f waiting on condition [0x0000000000000000]
> java.lang.Thread.State: RUNNABLE
>
> "Finalizer" daemon prio=10 tid=0x00002aaaf7695800 nid=0x45e in
> Object.wait() [0x0000000040a32000]
> java.lang.Thread.State: WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> - waiting on <0x00002aaab6396438> (a
> java.lang.ref.ReferenceQueue$Lock)
> at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:118)
> - locked <0x00002aaab6396438> (a java.lang.ref.ReferenceQueue$Lock)
> at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:134)
> at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:159)
>
> "Reference Handler" daemon prio=10 tid=0x00002aaaf7693800 nid=0x45c in
> Object.wait() [0x0000000040931000]
> java.lang.Thread.State: WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> - waiting on <0x00002aaab63910d8> (a java.lang.ref.Reference$Lock)
> at java.lang.Object.wait(Object.java:485)
> at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:116)
> - locked <0x00002aaab63910d8> (a java.lang.ref.Reference$Lock)
>
> "VM Thread" prio=10 tid=0x00002aaaf768d000 nid=0x45b runnable
>
> "Gang worker#0 (Parallel GC Threads)" prio=10 tid=0x0000000040119000
> nid=0x456 runnable
>
> "Gang worker#1 (Parallel GC Threads)" prio=10 tid=0x000000004011b000
> nid=0x457 runnable
>
> "Gang worker#2 (Parallel GC Threads)" prio=10 tid=0x000000004011c800
> nid=0x458 runnable
>
> "Gang worker#3 (Parallel GC Threads)" prio=10 tid=0x000000004011e800
> nid=0x459 runnable
>
> "Concurrent Mark-Sweep GC Thread" prio=10 tid=0x00002aaaf3d9d800 nid=0x45a
> runnable
> "VM Periodic Task Thread" prio=10 tid=0x00002aaaf785b000 nid=0x464 waiting
> on condition
>
> JNI global references: 806
>
> Heap
> par new generation total 19136K, used 14022K [0x00002aaaafc40000,
> 0x00002aaab1100000, 0x00002aaab4f70000)
> eden space 17024K, 81% used [0x00002aaaafc40000, 0x00002aaab09ce548,
> 0x00002aaab0ce0000)
> from space 2112K, 6% used [0x00002aaab0ef0000, 0x00002aaab0f13470,
> 0x00002aaab1100000)
> to space 2112K, 0% used [0x00002aaab0ce0000, 0x00002aaab0ce0000,
> 0x00002aaab0ef0000)
> concurrent mark-sweep generation total 63872K, used 25097K
> [0x00002aaab4f70000, 0x00002aaab8dd0000, 0x00002aaaee440000)
> concurrent-mark-sweep perm gen total 30220K, used 18263K
> [0x00002aaaee440000, 0x00002aaaf01c3000, 0x00002aaaf3840000)
>
>
>
> Ananth T Sarathy
>
>
>
> On Wed, Oct 21, 2009 at 1:24 PM, stack <st...@duboce.net> wrote:
>
>> If thats all you get, a thread dump on 10.242.71.191:60020 would help ($
>> kill -QUIT RS_PID). Thread dump will be in the RS .out file. Do a few.
>> Paste to pastebin.
>> Thanks Ananth,
>> St.Ack
>>
>> On Wed, Oct 21, 2009 at 8:05 AM, Ananth T. Sarathy <
>> ananth.t.sarathy@gmail.com> wrote:
>>
>> > yeah,
>> > I just don't understand why getScanner("Column:X") returns the iterator
>> > and
>> > process them yet getScanner("Column:Y") just spins and spins, yet
>> Column:Y
>> > is a much denser result.
>> >
>> > When I load from shell
>> >
>> > *Version: 0.20.0, r810752, Thu Sep 3 00:06:18 PDT 2009
>> > hbase(main):001:0> count 'GS_Applications'
>> > 09/10/21 11:04:48 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode
>> > /hbase/root-region-server got 10.244.9.171:60020
>> > 09/10/21 11:04:48 DEBUG client.HConnectionManager$ClientZKWatcher: Got
>> > ZooKeeper event, state: SyncConnected, type: None, path: null
>> > 09/10/21 11:04:48 DEBUG client.HConnectionManager$TableServers: Found
>> ROOT
>> > at 10.244.9.171:60020
>> > 09/10/21 11:04:48 DEBUG client.HConnectionManager$TableServers: Cached
>> > location address: 10.245.82.160:60020, regioninfo: REGION => {NAME =>
>> > '.META.,,1', STARTKEY => '', ENDKEY => '', ENCODED => 1028785192, TABLE
>> =>
>> > {{NAME => '.META.', IS_META => 'true', MEMSTORE_FLUSHSIZE => '16384',
>> > FAMILIES => [{NAME => 'historian', VERSIONS => '2147483647', COMPRESSION
>> =>
>> > 'NONE', TTL => '604800', BLOCKSIZE => '8192', IN_MEMORY => 'false',
>> > BLOCKCACHE => 'false'}, {NAME => 'info', VERSIONS => '10', COMPRESSION
>> =>
>> > 'NONE', TTL => '2147483647', BLOCKSIZE => '8192', IN_MEMORY => 'false',
>> > BLOCKCACHE => 'false'}]}}
>> > 09/10/21 11:04:49 DEBUG client.HConnectionManager$TableServers: Cached
>> > location address: 10.242.71.191:60020, regioninfo: REGION => {NAME =>
>> > 'GS_Applications,,1255020109210', STARTKEY => '', ENDKEY => '', ENCODED
>> =>
>> > 1732076772, TABLE => {{NAME => 'GS_Applications', FAMILIES => [{NAME =>
>> > 'Application', COMPRESSION => 'NONE', VERSIONS => '1', TTL =>
>> '2147483647',
>> > BLOCKSIZE => '65536', IN_MEMORY => 'false', BLOCKCACHE => 'true'}, {NAME
>> =>
>> > 'BinaryData', COMPRESSION => 'NONE', VERSIONS => '1', TTL =>
>> '2147483647',
>> > BLOCKSIZE => '65536', IN_MEMORY => 'false', BLOCKCACHE => 'true'}, {NAME
>> =>
>> > 'BinaryRetrieval', COMPRESSION => 'NONE', VERSIONS => '1', TTL =>
>> > '2147483647', BLOCKSIZE => '65536', IN_MEMORY => 'false', BLOCKCACHE =>
>> > 'true'}, {NAME => 'Files', COMPRESSION => 'NONE', VERSIONS => '1', TTL
>> =>
>> > '2147483647', BLOCKSIZE => '65536', IN_MEMORY => 'false', BLOCKCACHE =>
>> > 'true'}, {NAME => 'Info', COMPRESSION => 'NONE', VERSIONS => '1', TTL =>
>> > '2147483647', BLOCKSIZE => '65536', IN_MEMORY => 'false', BLOCKCACHE =>
>> > 'true'}, {NAME => 'Network', COMPRESSION => 'NONE', VERSIONS => '1', TTL
>> =>
>> > '2147483647', BLOCKSIZE => '65536', IN_MEMORY => 'false', BLOCKCACHE =>
>> > 'true'}, {NAME => 'Registry', VERSIONS => '1', COMPRESSION => 'NONE',
>> TTL
>> > =>
>> > '2147483647', BLOCKSIZE => '65536', IN_MEMORY => 'false', BLOCKCACHE =>
>> > 'true'}]}}
>> > 09/10/21 11:04:49 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode
>> > /hbase/root-region-server got 10.244.9.171:60020
>> > 09/10/21 11:04:49 DEBUG client.HConnectionManager$TableServers: Found
>> ROOT
>> > at 10.244.9.171:60020
>> > 09/10/21 11:04:49 DEBUG client.HConnectionManager$TableServers: Cached
>> > location address: 10.245.82.160:60020, regioninfo: REGION => {NAME =>
>> > '.META.,,1', STARTKEY => '', ENDKEY => '', ENCODED => 1028785192, TABLE
>> =>
>> > {{NAME => '.META.', IS_META => 'true', MEMSTORE_FLUSHSIZE => '16384',
>> > FAMILIES => [{NAME => 'historian', VERSIONS => '2147483647', COMPRESSION
>> =>
>> > 'NONE', TTL => '604800', BLOCKSIZE => '8192', IN_MEMORY => 'false',
>> > BLOCKCACHE => 'false'}, {NAME => 'info', VERSIONS => '10', COMPRESSION
>> =>
>> > 'NONE', TTL => '2147483647', BLOCKSIZE => '8192', IN_MEMORY => 'false',
>> > BLOCKCACHE => 'false'}]}}
>> > 09/10/21 11:04:50 DEBUG client.HTable$ClientScanner: Creating scanner
>> over
>> > GS_Applications starting at key ''
>> > 09/10/21 11:04:50 DEBUG client.HTable$ClientScanner: Advancing internal
>> > scanner to startKey at ''
>> > 09/10/21 11:04:50 DEBUG client.HConnectionManager$TableServers: Cache
>> hit
>> > for row <> in tableName GS_Applications: location server
>> > 10.242.71.191:60020,
>> > location region name GS_Applications,,1255020109210
>> >
>> >
>> > *Ananth T Sarathy
>> >
>> >
>> > On Wed, Oct 21, 2009 at 11:00 AM, stack <st...@duboce.net> wrote:
>> >
>> > > In both cases you are doing a full table scan?
>> > >
>> > > Try from shell with DEBUG enable. You'll see the regions being
>> loaded.
>> > > May
>> > > help you narrow in on problem region or at least on problem
>> regionserver.
>> > >
>> > > St.Ack
>> > >
>> > > On Wed, Oct 21, 2009 at 7:19 AM, Ananth T. Sarathy <
>> > > ananth.t.sarathy@gmail.com> wrote:
>> > >
>> > > > Anyone have any further thoughts on this?
>> > > > Ananth T Sarathy
>> > > >
>> > > >
>> > > > On Tue, Oct 20, 2009 at 6:37 PM, Ananth T. Sarathy <
>> > > > ananth.t.sarathy@gmail.com> wrote:
>> > > >
>> > > > > Well that's not the case. Every Row has that column. In fact the
>> > > second
>> > > > > snippet i sent is with a column with many less rows. (1k vs 25k)
>> but
>> > > > comes
>> > > > > back pretty quickly.
>> > > > >
>> > > > > By forever, I mean i have watched my logs do nothing for a half
>> hour
>> > > > before
>> > > > > giving up.
>> > > > >
>> > > > >
>> > > > > Ananth T Sarathy
>> > > > >
>> > > > >
>> > > > >
>> > > > > On Tue, Oct 20, 2009 at 5:03 PM, Ryan Rawson <ry...@gmail.com>
>> > > wrote:
>> > > > >
>> > > > >> If you are asking for a column that is very sparse and doesnt
>> exist,
>> > > > >> it will cause HBase to read through the entire region to find 100
>> > > > >> matching rows. This could take a while, you said 'forever', but
>> > could
>> > > > >> you quantify that?
>> > > > >>
>> > > > >> On Tue, Oct 20, 2009 at 1:58 PM, Jean-Daniel Cryans <
>> > > > jdcryans@apache.org>
>> > > > >> wrote:
>> > > > >> > Scanner pre-fetching is always faster, so something must be
>> wrong
>> > > with
>> > > > >> > your region server. Check the logs, top, etc
>> > > > >> >
>> > > > >> > WRT to row size, it's pretty much a matter of how many bytes
>> you
>> > > have
>> > > > >> > in each column and sum them up (plus some overhead with the
>> keys).
>> > > > >> >
>> > > > >> > You want filters, check the filter package in the javadoc.
>> > > > >> >
>> > > > >> > J-D
>> > > > >> >
>> > > > >> > On Tue, Oct 20, 2009 at 1:52 PM, Ananth T. Sarathy
>> > > > >> > <an...@gmail.com> wrote:
>> > > > >> >> Ok, but how come
>> > > > >> >> when I run a similiar call (with less returned rows 1000 vs
>> 25k
>> > in
>> > > > the
>> > > > >> >> previous one) it runs through the iterator very quickly? (See
>> > > Below)
>> > > > >> >>
>> > > > >> >> Also, how do I determine the row size? It's just text data,
>> and
>> > > > really
>> > > > >> not
>> > > > >> >> much.
>> > > > >> >>
>> > > > >> >> Finally, is there a way to query for rows that do not have a
>> > > column?
>> > > > >> (Ie all
>> > > > >> >> rows without Files:path1)
>> > > > >> >>
>> > > > >> >> HBaseTableDataManagerImpl htdmni = new
>> > > > >> HBaseTableDataManagerImpl(
>> > > > >> >> "GS_Applications");
>> > > > >> >>
>> > > > >> >> String[] columns = { "Files:path1" };
>> > > > >> >> log.info("Getting all Rows with Files");
>> > > > >> >> Scanner s = htdmni.getScannerForAllRows(columns);
>> > > > >> >> log.info("Got all Rows with Files");
>> > > > >> >>
>> > > > >> >> Iterator<RowResult> iter = s.iterator();
>> > > > >> >> out
>> > > > >> >>
>> > > > >> >>
>> > > > >>
>> > > >
>> > >
>> >
>> .write("Application_Full_Name,Version,Application_installer_name,Operating
>> > > > >> >> System, Application_platform
>> > > > >> >>
>> > > > >>
>> > > >
>> > >
>> >
>> ,Application_sub_category,md5Hash,Sha1Hash,Sha256Hash,filepath,fileName,modified,size,operation\n");
>> > > > >> >> out.write("<BR>");
>> > > > >> >> while (iter.hasNext())
>> > > > >> >> {
>> > > > >> >>
>> > > > >> >> Ananth T Sarathy
>> > > > >> >>
>> > > > >> >>
>> > > > >> >> On Tue, Oct 20, 2009 at 4:44 PM, Jean-Daniel Cryans <
>> > > > >> jdcryans@apache.org>wrote:
>> > > > >> >>
>> > > > >> >>> If you have a very slow data source (S3), then it fetches 100
>> > row
>> > > > >> >>> before coming back to your client with all of them and that
>> can
>> > > take
>> > > > a
>> > > > >> >>> lot of time. Also make sure that 100 of your rows can fit in
>> a
>> > > > region
>> > > > >> >>> server's memory. How big is each row?
>> > > > >> >>>
>> > > > >> >>> J-D
>> > > > >> >>>
>> > > > >> >>> On Tue, Oct 20, 2009 at 1:32 PM, Ananth T. Sarathy
>> > > > >> >>> <an...@gmail.com> wrote:
>> > > > >> >>> > I am running this code where
>> > > > >> >>> >
>> > > > >> >>> > getScannerForAllRows(columns) just does return
>> > > > >> table.getScanner(columns);
>> > > > >> >>> >
>> > > > >> >>> > and the table has setScannerCaching(100);
>> > > > >> >>> >
>> > > > >> >>> > But it spins forever after getting the iterator. Why would
>> > that
>> > > > be?
>> > > > >> How
>> > > > >> >>> can
>> > > > >> >>> > I speed it up?
>> > > > >> >>> >
>> > > > >> >>> > HBaseTableDataManagerImpl htdmni = new
>> > > > >> HBaseTableDataManagerImpl(
>> > > > >> >>> > "GS_Applications");
>> > > > >> >>> >
>> > > > >> >>> > String[] columns = { "Files:Name" };
>> > > > >> >>> > log.info("Getting all Rows with Files");
>> > > > >> >>> > Scanner s = htdmni.getScannerForAllRows(columns);
>> > > > >> >>> > log.info("Got all Rows with Files");
>> > > > >> >>> > log.info("Getting Iterator");
>> > > > >> >>> >
>> > > > >> >>> > Iterator<RowResult> iter = s.iterator();
>> > > > >> >>> > log.info("Got Iterator");
>> > > > >> >>> >
>> > > > >> >>> > while (iter.hasNext())
>> > > > >> >>> > {
>> > > > >> >>> > log.info("Getting next Row");
>> > > > >> >>> > RowResult rr = iter.next();
>> > > > >> >>> >
>> > > > >> >>> >
>> > > > >> >>> > Ananth T Sarathy
>> > > > >> >>> >
>> > > > >> >>>
>> > > > >> >>
>> > > > >> >
>> > > > >>
>> > > > >
>> > > > >
>> > > >
>> > >
>> >
>>
>
>
Re: Waiting forever on scanner iterator
Posted by "Ananth T. Sarathy" <an...@gmail.com>.
here's what I get
Heap
par new generation total 19136K, used 13714K [0x00002aaaafc40000,
0x00002aaab1100000, 0x00002aaab4f70000)
eden space 17024K, 79% used [0x00002aaaafc40000, 0x00002aaab09816d8,
0x00002aaab0ce0000)
from space 2112K, 6% used [0x00002aaab0ef0000, 0x00002aaab0f13470,
0x00002aaab1100000)
to space 2112K, 0% used [0x00002aaab0ce0000, 0x00002aaab0ce0000,
0x00002aaab0ef0000)
concurrent mark-sweep generation total 63872K, used 25097K
[0x00002aaab4f70000, 0x00002aaab8dd0000, 0x00002aaaee440000)
concurrent-mark-sweep perm gen total 30220K, used 18263K
[0x00002aaaee440000, 0x00002aaaf01c3000, 0x00002aaaf3840000)
2009-10-21 14:40:58
Full thread dump Java HotSpot(TM) 64-Bit Server VM (14.2-b01 mixed mode):
"pool-1-thread-1" prio=10 tid=0x00002aaafc01a000 nid=0x49f waiting on
condition [0x0000000042e56000]
java.lang.Thread.State: TIMED_WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00002aaab6559490> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:198)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:1963)
at java.util.concurrent.DelayQueue.take(DelayQueue.java:164)
at
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:583)
at
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:576)
at
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java.lang.Thread.run(Thread.java:619)
"LruBlockCache.EvictionThread" daemon prio=10 tid=0x00002aaafc01f800
nid=0x49e in Object.wait() [0x0000000042d55000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x00002aaab6519890> (a
org.apache.hadoop.hbase.io.hfile.LruBlockCache$EvictionThread)
at java.lang.Object.wait(Object.java:485)
at
org.apache.hadoop.hbase.io.hfile.LruBlockCache$EvictionThread.run(LruBlockCache.java:512)
- locked <0x00002aaab6519890> (a
org.apache.hadoop.hbase.io.hfile.LruBlockCache$EvictionThread)
"IPC Server handler 9 on 60020" daemon prio=10 tid=0x00002aaaf7c12000
nid=0x49d waiting on condition [0x0000000042c54000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00002aaab6434798> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at
org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:901)
"IPC Server handler 8 on 60020" daemon prio=10 tid=0x00002aaaf7c0f800
nid=0x49c waiting on condition [0x0000000042b53000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00002aaab6434798> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at
org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:901)
"IPC Server handler 7 on 60020" daemon prio=10 tid=0x00002aaaf7c0d800
nid=0x49b waiting on condition [0x0000000042a52000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00002aaab6434798> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at
org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:901)
"IPC Server handler 6 on 60020" daemon prio=10 tid=0x00002aaafd08b800
nid=0x49a waiting on condition [0x0000000042951000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00002aaab6434798> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at
org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:901)
"IPC Server handler 5 on 60020" daemon prio=10 tid=0x00002aaafd089800
nid=0x499 waiting on condition [0x0000000042850000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00002aaab6434798> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at
org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:901)
"IPC Server handler 4 on 60020" daemon prio=10 tid=0x00002aaafd087000
nid=0x498 waiting on condition [0x000000004274f000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00002aaab6434798> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at
org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:901)
"IPC Server handler 3 on 60020" daemon prio=10 tid=0x00002aaafd085000
nid=0x497 waiting on condition [0x000000004264e000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00002aaab6434798> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at
org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:901)
"IPC Server handler 2 on 60020" daemon prio=10 tid=0x00002aaaf7adf800
nid=0x496 waiting on condition [0x000000004254d000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00002aaab6434798> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at
org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:901)
"IPC Server handler 1 on 60020" daemon prio=10 tid=0x00002aaaf7adb800
nid=0x495 waiting on condition [0x000000004244c000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00002aaab6434798> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at
org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:901)
"IPC Server handler 0 on 60020" daemon prio=10 tid=0x00002aaaf7adb000
nid=0x494 waiting on condition [0x000000004234b000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00002aaab6434798> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at
org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:901)
"IPC Server listener on 60020" daemon prio=10 tid=0x00002aaaf7de9800
nid=0x493 runnable [0x000000004224a000]
java.lang.Thread.State: RUNNABLE
at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:215)
at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:65)
at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
- locked <0x00002aaab643b548> (a sun.nio.ch.Util$1)
- locked <0x00002aaab643b530> (a
java.util.Collections$UnmodifiableSet)
- locked <0x00002aaab63ea7e0> (a sun.nio.ch.EPollSelectorImpl)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:84)
at
org.apache.hadoop.hbase.ipc.HBaseServer$Listener.run(HBaseServer.java:308)
"IPC Server Responder" daemon prio=10 tid=0x00002aaaf7de9000 nid=0x492
runnable [0x0000000042149000]
java.lang.Thread.State: RUNNABLE
at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:215)
at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:65)
at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
- locked <0x00002aaab643b458> (a sun.nio.ch.Util$1)
- locked <0x00002aaab643b440> (a
java.util.Collections$UnmodifiableSet)
- locked <0x00002aaab63ea768> (a sun.nio.ch.EPollSelectorImpl)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
at
org.apache.hadoop.hbase.ipc.HBaseServer$Responder.run(HBaseServer.java:477)
"regionserver/10.244.9.171:60020.safeMode" daemon prio=10
tid=0x00002aaaf7c84000 nid=0x491 in Object.wait() [0x0000000042048000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x00002aaab63911f8> (a
java.util.concurrent.atomic.AtomicBoolean)
at java.lang.Object.wait(Object.java:485)
at
org.apache.hadoop.hbase.regionserver.HRegionServer$CompactionLimitThread.run(HRegionServer.java:902)
- locked <0x00002aaab63911f8> (a
java.util.concurrent.atomic.AtomicBoolean)
"Timer-0" daemon prio=10 tid=0x00002aaaf7de4000 nid=0x490 in Object.wait()
[0x0000000041f47000]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x00002aaab6558e60> (a java.util.TaskQueue)
at java.util.TimerThread.mainLoop(Timer.java:509)
- locked <0x00002aaab6558e60> (a java.util.TaskQueue)
at java.util.TimerThread.run(Timer.java:462)
"853527367@qtp0-1 - Acceptor0 SelectChannelConnector@0.0.0.0:60030" prio=10
tid=0x00002aaafcd3a800 nid=0x48f runnable [0x0000000041e46000]
java.lang.Thread.State: RUNNABLE
at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:215)
at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:65)
at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
- locked <0x00002aaab651bf38> (a sun.nio.ch.Util$1)
- locked <0x00002aaab651bf50> (a
java.util.Collections$UnmodifiableSet)
- locked <0x00002aaab6620c60> (a sun.nio.ch.EPollSelectorImpl)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
at
org.mortbay.io.nio.SelectorManager$SelectSet.doSelect(SelectorManager.java:429)
at
org.mortbay.io.nio.SelectorManager.doSelect(SelectorManager.java:185)
at
org.mortbay.jetty.nio.SelectChannelConnector.accept(SelectChannelConnector.java:124)
at
org.mortbay.jetty.AbstractConnector$Acceptor.run(AbstractConnector.java:707)
at
org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:522)
"1096304647@qtp0-0" prio=10 tid=0x00002aaaf7d5b800 nid=0x48e in
Object.wait() [0x0000000041d45000]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x00002aaab653acd0> (a
org.mortbay.thread.QueuedThreadPool$PoolThread)
at
org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:565)
- locked <0x00002aaab653acd0> (a
org.mortbay.thread.QueuedThreadPool$PoolThread)
"regionserver/10.244.9.171:60020.leaseChecker" prio=10
tid=0x00002aaafc16a000 nid=0x48d waiting on condition [0x0000000041c44000]
java.lang.Thread.State: TIMED_WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00002aaab6434518> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:198)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:1963)
at java.util.concurrent.DelayQueue.poll(DelayQueue.java:209)
at org.apache.hadoop.hbase.Leases.run(Leases.java:81)
"regionserver/10.244.9.171:60020.majorCompactionChecker" daemon prio=10
tid=0x00002aaafc168000 nid=0x48c waiting on condition [0x0000000041b43000]
java.lang.Thread.State: TIMED_WAITING (sleeping)
at java.lang.Thread.sleep(Native Method)
at org.apache.hadoop.hbase.util.Sleeper.sleep(Sleeper.java:74)
at org.apache.hadoop.hbase.Chore.run(Chore.java:76)
"regionserver/10.244.9.171:60020.worker" daemon prio=10
tid=0x00002aaafc228000 nid=0x48b waiting on condition [0x0000000041a42000]
java.lang.Thread.State: TIMED_WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00002aaab63a3b68> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:198)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:1963)
at
java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:395)
at
org.apache.hadoop.hbase.regionserver.HRegionServer$Worker.run(HRegionServer.java:1459)
at java.lang.Thread.run(Thread.java:619)
"regionserver/10.244.9.171:60020.compactor" daemon prio=10
tid=0x00002aaafc226000 nid=0x48a waiting on condition [0x0000000041941000]
java.lang.Thread.State: TIMED_WAITING (sleeping)
at java.lang.Thread.sleep(Native Method)
at
org.apache.hadoop.hbase.regionserver.CompactSplitThread.run(CompactSplitThread.java:79)
"regionserver/10.244.9.171:60020.cacheFlusher" daemon prio=10
tid=0x00002aaafc225000 nid=0x489 waiting on condition [0x0000000041840000]
java.lang.Thread.State: TIMED_WAITING (sleeping)
at java.lang.Thread.sleep(Native Method)
at
org.apache.hadoop.hbase.regionserver.MemStoreFlusher.run(MemStoreFlusher.java:137)
"regionserver/10.244.9.171:60020.logFlusher" daemon prio=10
tid=0x00002aaafc212000 nid=0x488 waiting on condition [0x000000004173f000]
java.lang.Thread.State: TIMED_WAITING (sleeping)
at java.lang.Thread.sleep(Native Method)
at org.apache.hadoop.hbase.util.Sleeper.sleep(Sleeper.java:74)
at org.apache.hadoop.hbase.Chore.run(Chore.java:76)
"regionserver/10.244.9.171:60020.logRoller" daemon prio=10
tid=0x00002aaafc211800 nid=0x487 in Object.wait() [0x000000004153d000]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x00002aaab63913f0> (a
java.util.concurrent.atomic.AtomicBoolean)
at
org.apache.hadoop.hbase.regionserver.LogRoller.run(LogRoller.java:65)
- locked <0x00002aaab63913f0> (a
java.util.concurrent.atomic.AtomicBoolean)
"MultiThreadedHttpConnectionManager cleanup" daemon prio=10
tid=0x00002aaaf7e14000 nid=0x486 in Object.wait() [0x000000004163e000]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x00002aaab65446e0> (a
java.lang.ref.ReferenceQueue$Lock)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:118)
- locked <0x00002aaab65446e0> (a java.lang.ref.ReferenceQueue$Lock)
at
org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$ReferenceQueueThread.run(MultiThreadedHttpConnectionManager.java:1082)
"IPC Client (47) connection to /10.244.9.171:60000 from an unknown user"
daemon prio=10 tid=0x00002aaaf80b6800 nid=0x46d in Object.wait()
[0x000000004143c000]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x00002aaab642ba08> (a
org.apache.hadoop.hbase.ipc.HBaseClient$Connection)
at
org.apache.hadoop.hbase.ipc.HBaseClient$Connection.waitForWork(HBaseClient.java:405)
- locked <0x00002aaab642ba08> (a
org.apache.hadoop.hbase.ipc.HBaseClient$Connection)
at
org.apache.hadoop.hbase.ipc.HBaseClient$Connection.run(HBaseClient.java:448)
"DestroyJavaVM" prio=10 tid=0x00002aaaf80b4800 nid=0x455 waiting on
condition [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
"regionserver/10.244.9.171:60020" prio=10 tid=0x00002aaaf80b3800 nid=0x467
waiting on condition [0x000000004133b000]
java.lang.Thread.State: TIMED_WAITING (sleeping)
at java.lang.Thread.sleep(Native Method)
at org.apache.hadoop.hbase.util.Sleeper.sleep(Sleeper.java:74)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:596)
at java.lang.Thread.run(Thread.java:619)
"main-EventThread" daemon prio=10 tid=0x00002aaaf80ab800 nid=0x466 waiting
on condition [0x000000004123a000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00002aaab6391598> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at
org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:376)
"main-SendThread" daemon prio=10 tid=0x00002aaaf80ab000 nid=0x465 runnable
[0x0000000041139000]
java.lang.Thread.State: RUNNABLE
at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:215)
at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:65)
at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
- locked <0x00002aaab6418908> (a sun.nio.ch.Util$1)
- locked <0x00002aaab64188f0> (a
java.util.Collections$UnmodifiableSet)
- locked <0x00002aaab63fb0a0> (a sun.nio.ch.EPollSelectorImpl)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
at
org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:873)
"Low Memory Detector" daemon prio=10 tid=0x00002aaaf7858800 nid=0x463
runnable [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
"CompilerThread1" daemon prio=10 tid=0x00002aaaf7856000 nid=0x462 waiting on
condition [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
"CompilerThread0" daemon prio=10 tid=0x00002aaaf7852800 nid=0x461 waiting on
condition [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
"Signal Dispatcher" daemon prio=10 tid=0x00002aaaf76b9800 nid=0x460 waiting
on condition [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
"Surrogate Locker Thread (CMS)" daemon prio=10 tid=0x00002aaaf76b7800
nid=0x45f waiting on condition [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
"Finalizer" daemon prio=10 tid=0x00002aaaf7695800 nid=0x45e in Object.wait()
[0x0000000040a32000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x00002aaab6396438> (a
java.lang.ref.ReferenceQueue$Lock)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:118)
- locked <0x00002aaab6396438> (a java.lang.ref.ReferenceQueue$Lock)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:134)
at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:159)
"Reference Handler" daemon prio=10 tid=0x00002aaaf7693800 nid=0x45c in
Object.wait() [0x0000000040931000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x00002aaab63910d8> (a java.lang.ref.Reference$Lock)
at java.lang.Object.wait(Object.java:485)
at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:116)
- locked <0x00002aaab63910d8> (a java.lang.ref.Reference$Lock)
"VM Thread" prio=10 tid=0x00002aaaf768d000 nid=0x45b runnable
"Gang worker#0 (Parallel GC Threads)" prio=10 tid=0x0000000040119000
nid=0x456 runnable
"Gang worker#1 (Parallel GC Threads)" prio=10 tid=0x000000004011b000
nid=0x457 runnable
"Gang worker#2 (Parallel GC Threads)" prio=10 tid=0x000000004011c800
nid=0x458 runnable
"Gang worker#3 (Parallel GC Threads)" prio=10 tid=0x000000004011e800
nid=0x459 runnable
"Concurrent Mark-Sweep GC Thread" prio=10 tid=0x00002aaaf3d9d800 nid=0x45a
runnable
"VM Periodic Task Thread" prio=10 tid=0x00002aaaf785b000 nid=0x464 waiting
on condition
JNI global references: 806
Heap
par new generation total 19136K, used 14022K [0x00002aaaafc40000,
0x00002aaab1100000, 0x00002aaab4f70000)
eden space 17024K, 81% used [0x00002aaaafc40000, 0x00002aaab09ce548,
0x00002aaab0ce0000)
from space 2112K, 6% used [0x00002aaab0ef0000, 0x00002aaab0f13470,
0x00002aaab1100000)
to space 2112K, 0% used [0x00002aaab0ce0000, 0x00002aaab0ce0000,
0x00002aaab0ef0000)
concurrent mark-sweep generation total 63872K, used 25097K
[0x00002aaab4f70000, 0x00002aaab8dd0000, 0x00002aaaee440000)
concurrent-mark-sweep perm gen total 30220K, used 18263K
[0x00002aaaee440000, 0x00002aaaf01c3000, 0x00002aaaf3840000)
Ananth T Sarathy
On Wed, Oct 21, 2009 at 1:24 PM, stack <st...@duboce.net> wrote:
> If thats all you get, a thread dump on 10.242.71.191:60020 would help ($
> kill -QUIT RS_PID). Thread dump will be in the RS .out file. Do a few.
> Paste to pastebin.
> Thanks Ananth,
> St.Ack
>
> On Wed, Oct 21, 2009 at 8:05 AM, Ananth T. Sarathy <
> ananth.t.sarathy@gmail.com> wrote:
>
> > yeah,
> > I just don't understand why getScanner("Column:X") returns the iterator
> > and
> > process them yet getScanner("Column:Y") just spins and spins, yet
> Column:Y
> > is a much denser result.
> >
> > When I load from shell
> >
> > *Version: 0.20.0, r810752, Thu Sep 3 00:06:18 PDT 2009
> > hbase(main):001:0> count 'GS_Applications'
> > 09/10/21 11:04:48 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode
> > /hbase/root-region-server got 10.244.9.171:60020
> > 09/10/21 11:04:48 DEBUG client.HConnectionManager$ClientZKWatcher: Got
> > ZooKeeper event, state: SyncConnected, type: None, path: null
> > 09/10/21 11:04:48 DEBUG client.HConnectionManager$TableServers: Found
> ROOT
> > at 10.244.9.171:60020
> > 09/10/21 11:04:48 DEBUG client.HConnectionManager$TableServers: Cached
> > location address: 10.245.82.160:60020, regioninfo: REGION => {NAME =>
> > '.META.,,1', STARTKEY => '', ENDKEY => '', ENCODED => 1028785192, TABLE
> =>
> > {{NAME => '.META.', IS_META => 'true', MEMSTORE_FLUSHSIZE => '16384',
> > FAMILIES => [{NAME => 'historian', VERSIONS => '2147483647', COMPRESSION
> =>
> > 'NONE', TTL => '604800', BLOCKSIZE => '8192', IN_MEMORY => 'false',
> > BLOCKCACHE => 'false'}, {NAME => 'info', VERSIONS => '10', COMPRESSION =>
> > 'NONE', TTL => '2147483647', BLOCKSIZE => '8192', IN_MEMORY => 'false',
> > BLOCKCACHE => 'false'}]}}
> > 09/10/21 11:04:49 DEBUG client.HConnectionManager$TableServers: Cached
> > location address: 10.242.71.191:60020, regioninfo: REGION => {NAME =>
> > 'GS_Applications,,1255020109210', STARTKEY => '', ENDKEY => '', ENCODED
> =>
> > 1732076772, TABLE => {{NAME => 'GS_Applications', FAMILIES => [{NAME =>
> > 'Application', COMPRESSION => 'NONE', VERSIONS => '1', TTL =>
> '2147483647',
> > BLOCKSIZE => '65536', IN_MEMORY => 'false', BLOCKCACHE => 'true'}, {NAME
> =>
> > 'BinaryData', COMPRESSION => 'NONE', VERSIONS => '1', TTL =>
> '2147483647',
> > BLOCKSIZE => '65536', IN_MEMORY => 'false', BLOCKCACHE => 'true'}, {NAME
> =>
> > 'BinaryRetrieval', COMPRESSION => 'NONE', VERSIONS => '1', TTL =>
> > '2147483647', BLOCKSIZE => '65536', IN_MEMORY => 'false', BLOCKCACHE =>
> > 'true'}, {NAME => 'Files', COMPRESSION => 'NONE', VERSIONS => '1', TTL =>
> > '2147483647', BLOCKSIZE => '65536', IN_MEMORY => 'false', BLOCKCACHE =>
> > 'true'}, {NAME => 'Info', COMPRESSION => 'NONE', VERSIONS => '1', TTL =>
> > '2147483647', BLOCKSIZE => '65536', IN_MEMORY => 'false', BLOCKCACHE =>
> > 'true'}, {NAME => 'Network', COMPRESSION => 'NONE', VERSIONS => '1', TTL
> =>
> > '2147483647', BLOCKSIZE => '65536', IN_MEMORY => 'false', BLOCKCACHE =>
> > 'true'}, {NAME => 'Registry', VERSIONS => '1', COMPRESSION => 'NONE', TTL
> > =>
> > '2147483647', BLOCKSIZE => '65536', IN_MEMORY => 'false', BLOCKCACHE =>
> > 'true'}]}}
> > 09/10/21 11:04:49 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode
> > /hbase/root-region-server got 10.244.9.171:60020
> > 09/10/21 11:04:49 DEBUG client.HConnectionManager$TableServers: Found
> ROOT
> > at 10.244.9.171:60020
> > 09/10/21 11:04:49 DEBUG client.HConnectionManager$TableServers: Cached
> > location address: 10.245.82.160:60020, regioninfo: REGION => {NAME =>
> > '.META.,,1', STARTKEY => '', ENDKEY => '', ENCODED => 1028785192, TABLE
> =>
> > {{NAME => '.META.', IS_META => 'true', MEMSTORE_FLUSHSIZE => '16384',
> > FAMILIES => [{NAME => 'historian', VERSIONS => '2147483647', COMPRESSION
> =>
> > 'NONE', TTL => '604800', BLOCKSIZE => '8192', IN_MEMORY => 'false',
> > BLOCKCACHE => 'false'}, {NAME => 'info', VERSIONS => '10', COMPRESSION =>
> > 'NONE', TTL => '2147483647', BLOCKSIZE => '8192', IN_MEMORY => 'false',
> > BLOCKCACHE => 'false'}]}}
> > 09/10/21 11:04:50 DEBUG client.HTable$ClientScanner: Creating scanner
> over
> > GS_Applications starting at key ''
> > 09/10/21 11:04:50 DEBUG client.HTable$ClientScanner: Advancing internal
> > scanner to startKey at ''
> > 09/10/21 11:04:50 DEBUG client.HConnectionManager$TableServers: Cache hit
> > for row <> in tableName GS_Applications: location server
> > 10.242.71.191:60020,
> > location region name GS_Applications,,1255020109210
> >
> >
> > *Ananth T Sarathy
> >
> >
> > On Wed, Oct 21, 2009 at 11:00 AM, stack <st...@duboce.net> wrote:
> >
> > > In both cases you are doing a full table scan?
> > >
> > > Try from shell with DEBUG enable. You'll see the regions being loaded.
> > > May
> > > help you narrow in on problem region or at least on problem
> regionserver.
> > >
> > > St.Ack
> > >
> > > On Wed, Oct 21, 2009 at 7:19 AM, Ananth T. Sarathy <
> > > ananth.t.sarathy@gmail.com> wrote:
> > >
> > > > Anyone have any further thoughts on this?
> > > > Ananth T Sarathy
> > > >
> > > >
> > > > On Tue, Oct 20, 2009 at 6:37 PM, Ananth T. Sarathy <
> > > > ananth.t.sarathy@gmail.com> wrote:
> > > >
> > > > > Well that's not the case. Every Row has that column. In fact the
> > > second
> > > > > snippet i sent is with a column with many less rows. (1k vs 25k)
> but
> > > > comes
> > > > > back pretty quickly.
> > > > >
> > > > > By forever, I mean i have watched my logs do nothing for a half
> hour
> > > > before
> > > > > giving up.
> > > > >
> > > > >
> > > > > Ananth T Sarathy
> > > > >
> > > > >
> > > > >
> > > > > On Tue, Oct 20, 2009 at 5:03 PM, Ryan Rawson <ry...@gmail.com>
> > > wrote:
> > > > >
> > > > >> If you are asking for a column that is very sparse and doesnt
> exist,
> > > > >> it will cause HBase to read through the entire region to find 100
> > > > >> matching rows. This could take a while, you said 'forever', but
> > could
> > > > >> you quantify that?
> > > > >>
> > > > >> On Tue, Oct 20, 2009 at 1:58 PM, Jean-Daniel Cryans <
> > > > jdcryans@apache.org>
> > > > >> wrote:
> > > > >> > Scanner pre-fetching is always faster, so something must be
> wrong
> > > with
> > > > >> > your region server. Check the logs, top, etc
> > > > >> >
> > > > >> > WRT to row size, it's pretty much a matter of how many bytes you
> > > have
> > > > >> > in each column and sum them up (plus some overhead with the
> keys).
> > > > >> >
> > > > >> > You want filters, check the filter package in the javadoc.
> > > > >> >
> > > > >> > J-D
> > > > >> >
> > > > >> > On Tue, Oct 20, 2009 at 1:52 PM, Ananth T. Sarathy
> > > > >> > <an...@gmail.com> wrote:
> > > > >> >> Ok, but how come
> > > > >> >> when I run a similiar call (with less returned rows 1000 vs 25k
> > in
> > > > the
> > > > >> >> previous one) it runs through the iterator very quickly? (See
> > > Below)
> > > > >> >>
> > > > >> >> Also, how do I determine the row size? It's just text data, and
> > > > really
> > > > >> not
> > > > >> >> much.
> > > > >> >>
> > > > >> >> Finally, is there a way to query for rows that do not have a
> > > column?
> > > > >> (Ie all
> > > > >> >> rows without Files:path1)
> > > > >> >>
> > > > >> >> HBaseTableDataManagerImpl htdmni = new
> > > > >> HBaseTableDataManagerImpl(
> > > > >> >> "GS_Applications");
> > > > >> >>
> > > > >> >> String[] columns = { "Files:path1" };
> > > > >> >> log.info("Getting all Rows with Files");
> > > > >> >> Scanner s = htdmni.getScannerForAllRows(columns);
> > > > >> >> log.info("Got all Rows with Files");
> > > > >> >>
> > > > >> >> Iterator<RowResult> iter = s.iterator();
> > > > >> >> out
> > > > >> >>
> > > > >> >>
> > > > >>
> > > >
> > >
> >
> .write("Application_Full_Name,Version,Application_installer_name,Operating
> > > > >> >> System, Application_platform
> > > > >> >>
> > > > >>
> > > >
> > >
> >
> ,Application_sub_category,md5Hash,Sha1Hash,Sha256Hash,filepath,fileName,modified,size,operation\n");
> > > > >> >> out.write("<BR>");
> > > > >> >> while (iter.hasNext())
> > > > >> >> {
> > > > >> >>
> > > > >> >> Ananth T Sarathy
> > > > >> >>
> > > > >> >>
> > > > >> >> On Tue, Oct 20, 2009 at 4:44 PM, Jean-Daniel Cryans <
> > > > >> jdcryans@apache.org>wrote:
> > > > >> >>
> > > > >> >>> If you have a very slow data source (S3), then it fetches 100
> > row
> > > > >> >>> before coming back to your client with all of them and that
> can
> > > take
> > > > a
> > > > >> >>> lot of time. Also make sure that 100 of your rows can fit in a
> > > > region
> > > > >> >>> server's memory. How big is each row?
> > > > >> >>>
> > > > >> >>> J-D
> > > > >> >>>
> > > > >> >>> On Tue, Oct 20, 2009 at 1:32 PM, Ananth T. Sarathy
> > > > >> >>> <an...@gmail.com> wrote:
> > > > >> >>> > I am running this code where
> > > > >> >>> >
> > > > >> >>> > getScannerForAllRows(columns) just does return
> > > > >> table.getScanner(columns);
> > > > >> >>> >
> > > > >> >>> > and the table has setScannerCaching(100);
> > > > >> >>> >
> > > > >> >>> > But it spins forever after getting the iterator. Why would
> > that
> > > > be?
> > > > >> How
> > > > >> >>> can
> > > > >> >>> > I speed it up?
> > > > >> >>> >
> > > > >> >>> > HBaseTableDataManagerImpl htdmni = new
> > > > >> HBaseTableDataManagerImpl(
> > > > >> >>> > "GS_Applications");
> > > > >> >>> >
> > > > >> >>> > String[] columns = { "Files:Name" };
> > > > >> >>> > log.info("Getting all Rows with Files");
> > > > >> >>> > Scanner s = htdmni.getScannerForAllRows(columns);
> > > > >> >>> > log.info("Got all Rows with Files");
> > > > >> >>> > log.info("Getting Iterator");
> > > > >> >>> >
> > > > >> >>> > Iterator<RowResult> iter = s.iterator();
> > > > >> >>> > log.info("Got Iterator");
> > > > >> >>> >
> > > > >> >>> > while (iter.hasNext())
> > > > >> >>> > {
> > > > >> >>> > log.info("Getting next Row");
> > > > >> >>> > RowResult rr = iter.next();
> > > > >> >>> >
> > > > >> >>> >
> > > > >> >>> > Ananth T Sarathy
> > > > >> >>> >
> > > > >> >>>
> > > > >> >>
> > > > >> >
> > > > >>
> > > > >
> > > > >
> > > >
> > >
> >
>
Re: Waiting forever on scanner iterator
Posted by stack <st...@duboce.net>.
If thats all you get, a thread dump on 10.242.71.191:60020 would help ($
kill -QUIT RS_PID). Thread dump will be in the RS .out file. Do a few.
Paste to pastebin.
Thanks Ananth,
St.Ack
On Wed, Oct 21, 2009 at 8:05 AM, Ananth T. Sarathy <
ananth.t.sarathy@gmail.com> wrote:
> yeah,
> I just don't understand why getScanner("Column:X") returns the iterator
> and
> process them yet getScanner("Column:Y") just spins and spins, yet Column:Y
> is a much denser result.
>
> When I load from shell
>
> *Version: 0.20.0, r810752, Thu Sep 3 00:06:18 PDT 2009
> hbase(main):001:0> count 'GS_Applications'
> 09/10/21 11:04:48 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode
> /hbase/root-region-server got 10.244.9.171:60020
> 09/10/21 11:04:48 DEBUG client.HConnectionManager$ClientZKWatcher: Got
> ZooKeeper event, state: SyncConnected, type: None, path: null
> 09/10/21 11:04:48 DEBUG client.HConnectionManager$TableServers: Found ROOT
> at 10.244.9.171:60020
> 09/10/21 11:04:48 DEBUG client.HConnectionManager$TableServers: Cached
> location address: 10.245.82.160:60020, regioninfo: REGION => {NAME =>
> '.META.,,1', STARTKEY => '', ENDKEY => '', ENCODED => 1028785192, TABLE =>
> {{NAME => '.META.', IS_META => 'true', MEMSTORE_FLUSHSIZE => '16384',
> FAMILIES => [{NAME => 'historian', VERSIONS => '2147483647', COMPRESSION =>
> 'NONE', TTL => '604800', BLOCKSIZE => '8192', IN_MEMORY => 'false',
> BLOCKCACHE => 'false'}, {NAME => 'info', VERSIONS => '10', COMPRESSION =>
> 'NONE', TTL => '2147483647', BLOCKSIZE => '8192', IN_MEMORY => 'false',
> BLOCKCACHE => 'false'}]}}
> 09/10/21 11:04:49 DEBUG client.HConnectionManager$TableServers: Cached
> location address: 10.242.71.191:60020, regioninfo: REGION => {NAME =>
> 'GS_Applications,,1255020109210', STARTKEY => '', ENDKEY => '', ENCODED =>
> 1732076772, TABLE => {{NAME => 'GS_Applications', FAMILIES => [{NAME =>
> 'Application', COMPRESSION => 'NONE', VERSIONS => '1', TTL => '2147483647',
> BLOCKSIZE => '65536', IN_MEMORY => 'false', BLOCKCACHE => 'true'}, {NAME =>
> 'BinaryData', COMPRESSION => 'NONE', VERSIONS => '1', TTL => '2147483647',
> BLOCKSIZE => '65536', IN_MEMORY => 'false', BLOCKCACHE => 'true'}, {NAME =>
> 'BinaryRetrieval', COMPRESSION => 'NONE', VERSIONS => '1', TTL =>
> '2147483647', BLOCKSIZE => '65536', IN_MEMORY => 'false', BLOCKCACHE =>
> 'true'}, {NAME => 'Files', COMPRESSION => 'NONE', VERSIONS => '1', TTL =>
> '2147483647', BLOCKSIZE => '65536', IN_MEMORY => 'false', BLOCKCACHE =>
> 'true'}, {NAME => 'Info', COMPRESSION => 'NONE', VERSIONS => '1', TTL =>
> '2147483647', BLOCKSIZE => '65536', IN_MEMORY => 'false', BLOCKCACHE =>
> 'true'}, {NAME => 'Network', COMPRESSION => 'NONE', VERSIONS => '1', TTL =>
> '2147483647', BLOCKSIZE => '65536', IN_MEMORY => 'false', BLOCKCACHE =>
> 'true'}, {NAME => 'Registry', VERSIONS => '1', COMPRESSION => 'NONE', TTL
> =>
> '2147483647', BLOCKSIZE => '65536', IN_MEMORY => 'false', BLOCKCACHE =>
> 'true'}]}}
> 09/10/21 11:04:49 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode
> /hbase/root-region-server got 10.244.9.171:60020
> 09/10/21 11:04:49 DEBUG client.HConnectionManager$TableServers: Found ROOT
> at 10.244.9.171:60020
> 09/10/21 11:04:49 DEBUG client.HConnectionManager$TableServers: Cached
> location address: 10.245.82.160:60020, regioninfo: REGION => {NAME =>
> '.META.,,1', STARTKEY => '', ENDKEY => '', ENCODED => 1028785192, TABLE =>
> {{NAME => '.META.', IS_META => 'true', MEMSTORE_FLUSHSIZE => '16384',
> FAMILIES => [{NAME => 'historian', VERSIONS => '2147483647', COMPRESSION =>
> 'NONE', TTL => '604800', BLOCKSIZE => '8192', IN_MEMORY => 'false',
> BLOCKCACHE => 'false'}, {NAME => 'info', VERSIONS => '10', COMPRESSION =>
> 'NONE', TTL => '2147483647', BLOCKSIZE => '8192', IN_MEMORY => 'false',
> BLOCKCACHE => 'false'}]}}
> 09/10/21 11:04:50 DEBUG client.HTable$ClientScanner: Creating scanner over
> GS_Applications starting at key ''
> 09/10/21 11:04:50 DEBUG client.HTable$ClientScanner: Advancing internal
> scanner to startKey at ''
> 09/10/21 11:04:50 DEBUG client.HConnectionManager$TableServers: Cache hit
> for row <> in tableName GS_Applications: location server
> 10.242.71.191:60020,
> location region name GS_Applications,,1255020109210
>
>
> *Ananth T Sarathy
>
>
> On Wed, Oct 21, 2009 at 11:00 AM, stack <st...@duboce.net> wrote:
>
> > In both cases you are doing a full table scan?
> >
> > Try from shell with DEBUG enable. You'll see the regions being loaded.
> > May
> > help you narrow in on problem region or at least on problem regionserver.
> >
> > St.Ack
> >
> > On Wed, Oct 21, 2009 at 7:19 AM, Ananth T. Sarathy <
> > ananth.t.sarathy@gmail.com> wrote:
> >
> > > Anyone have any further thoughts on this?
> > > Ananth T Sarathy
> > >
> > >
> > > On Tue, Oct 20, 2009 at 6:37 PM, Ananth T. Sarathy <
> > > ananth.t.sarathy@gmail.com> wrote:
> > >
> > > > Well that's not the case. Every Row has that column. In fact the
> > second
> > > > snippet i sent is with a column with many less rows. (1k vs 25k) but
> > > comes
> > > > back pretty quickly.
> > > >
> > > > By forever, I mean i have watched my logs do nothing for a half hour
> > > before
> > > > giving up.
> > > >
> > > >
> > > > Ananth T Sarathy
> > > >
> > > >
> > > >
> > > > On Tue, Oct 20, 2009 at 5:03 PM, Ryan Rawson <ry...@gmail.com>
> > wrote:
> > > >
> > > >> If you are asking for a column that is very sparse and doesnt exist,
> > > >> it will cause HBase to read through the entire region to find 100
> > > >> matching rows. This could take a while, you said 'forever', but
> could
> > > >> you quantify that?
> > > >>
> > > >> On Tue, Oct 20, 2009 at 1:58 PM, Jean-Daniel Cryans <
> > > jdcryans@apache.org>
> > > >> wrote:
> > > >> > Scanner pre-fetching is always faster, so something must be wrong
> > with
> > > >> > your region server. Check the logs, top, etc
> > > >> >
> > > >> > WRT to row size, it's pretty much a matter of how many bytes you
> > have
> > > >> > in each column and sum them up (plus some overhead with the keys).
> > > >> >
> > > >> > You want filters, check the filter package in the javadoc.
> > > >> >
> > > >> > J-D
> > > >> >
> > > >> > On Tue, Oct 20, 2009 at 1:52 PM, Ananth T. Sarathy
> > > >> > <an...@gmail.com> wrote:
> > > >> >> Ok, but how come
> > > >> >> when I run a similiar call (with less returned rows 1000 vs 25k
> in
> > > the
> > > >> >> previous one) it runs through the iterator very quickly? (See
> > Below)
> > > >> >>
> > > >> >> Also, how do I determine the row size? It's just text data, and
> > > really
> > > >> not
> > > >> >> much.
> > > >> >>
> > > >> >> Finally, is there a way to query for rows that do not have a
> > column?
> > > >> (Ie all
> > > >> >> rows without Files:path1)
> > > >> >>
> > > >> >> HBaseTableDataManagerImpl htdmni = new
> > > >> HBaseTableDataManagerImpl(
> > > >> >> "GS_Applications");
> > > >> >>
> > > >> >> String[] columns = { "Files:path1" };
> > > >> >> log.info("Getting all Rows with Files");
> > > >> >> Scanner s = htdmni.getScannerForAllRows(columns);
> > > >> >> log.info("Got all Rows with Files");
> > > >> >>
> > > >> >> Iterator<RowResult> iter = s.iterator();
> > > >> >> out
> > > >> >>
> > > >> >>
> > > >>
> > >
> >
> .write("Application_Full_Name,Version,Application_installer_name,Operating
> > > >> >> System, Application_platform
> > > >> >>
> > > >>
> > >
> >
> ,Application_sub_category,md5Hash,Sha1Hash,Sha256Hash,filepath,fileName,modified,size,operation\n");
> > > >> >> out.write("<BR>");
> > > >> >> while (iter.hasNext())
> > > >> >> {
> > > >> >>
> > > >> >> Ananth T Sarathy
> > > >> >>
> > > >> >>
> > > >> >> On Tue, Oct 20, 2009 at 4:44 PM, Jean-Daniel Cryans <
> > > >> jdcryans@apache.org>wrote:
> > > >> >>
> > > >> >>> If you have a very slow data source (S3), then it fetches 100
> row
> > > >> >>> before coming back to your client with all of them and that can
> > take
> > > a
> > > >> >>> lot of time. Also make sure that 100 of your rows can fit in a
> > > region
> > > >> >>> server's memory. How big is each row?
> > > >> >>>
> > > >> >>> J-D
> > > >> >>>
> > > >> >>> On Tue, Oct 20, 2009 at 1:32 PM, Ananth T. Sarathy
> > > >> >>> <an...@gmail.com> wrote:
> > > >> >>> > I am running this code where
> > > >> >>> >
> > > >> >>> > getScannerForAllRows(columns) just does return
> > > >> table.getScanner(columns);
> > > >> >>> >
> > > >> >>> > and the table has setScannerCaching(100);
> > > >> >>> >
> > > >> >>> > But it spins forever after getting the iterator. Why would
> that
> > > be?
> > > >> How
> > > >> >>> can
> > > >> >>> > I speed it up?
> > > >> >>> >
> > > >> >>> > HBaseTableDataManagerImpl htdmni = new
> > > >> HBaseTableDataManagerImpl(
> > > >> >>> > "GS_Applications");
> > > >> >>> >
> > > >> >>> > String[] columns = { "Files:Name" };
> > > >> >>> > log.info("Getting all Rows with Files");
> > > >> >>> > Scanner s = htdmni.getScannerForAllRows(columns);
> > > >> >>> > log.info("Got all Rows with Files");
> > > >> >>> > log.info("Getting Iterator");
> > > >> >>> >
> > > >> >>> > Iterator<RowResult> iter = s.iterator();
> > > >> >>> > log.info("Got Iterator");
> > > >> >>> >
> > > >> >>> > while (iter.hasNext())
> > > >> >>> > {
> > > >> >>> > log.info("Getting next Row");
> > > >> >>> > RowResult rr = iter.next();
> > > >> >>> >
> > > >> >>> >
> > > >> >>> > Ananth T Sarathy
> > > >> >>> >
> > > >> >>>
> > > >> >>
> > > >> >
> > > >>
> > > >
> > > >
> > >
> >
>
Re: Waiting forever on scanner iterator
Posted by "Ananth T. Sarathy" <an...@gmail.com>.
yeah,
I just don't understand why getScanner("Column:X") returns the iterator and
process them yet getScanner("Column:Y") just spins and spins, yet Column:Y
is a much denser result.
When I load from shell
*Version: 0.20.0, r810752, Thu Sep 3 00:06:18 PDT 2009
hbase(main):001:0> count 'GS_Applications'
09/10/21 11:04:48 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode
/hbase/root-region-server got 10.244.9.171:60020
09/10/21 11:04:48 DEBUG client.HConnectionManager$ClientZKWatcher: Got
ZooKeeper event, state: SyncConnected, type: None, path: null
09/10/21 11:04:48 DEBUG client.HConnectionManager$TableServers: Found ROOT
at 10.244.9.171:60020
09/10/21 11:04:48 DEBUG client.HConnectionManager$TableServers: Cached
location address: 10.245.82.160:60020, regioninfo: REGION => {NAME =>
'.META.,,1', STARTKEY => '', ENDKEY => '', ENCODED => 1028785192, TABLE =>
{{NAME => '.META.', IS_META => 'true', MEMSTORE_FLUSHSIZE => '16384',
FAMILIES => [{NAME => 'historian', VERSIONS => '2147483647', COMPRESSION =>
'NONE', TTL => '604800', BLOCKSIZE => '8192', IN_MEMORY => 'false',
BLOCKCACHE => 'false'}, {NAME => 'info', VERSIONS => '10', COMPRESSION =>
'NONE', TTL => '2147483647', BLOCKSIZE => '8192', IN_MEMORY => 'false',
BLOCKCACHE => 'false'}]}}
09/10/21 11:04:49 DEBUG client.HConnectionManager$TableServers: Cached
location address: 10.242.71.191:60020, regioninfo: REGION => {NAME =>
'GS_Applications,,1255020109210', STARTKEY => '', ENDKEY => '', ENCODED =>
1732076772, TABLE => {{NAME => 'GS_Applications', FAMILIES => [{NAME =>
'Application', COMPRESSION => 'NONE', VERSIONS => '1', TTL => '2147483647',
BLOCKSIZE => '65536', IN_MEMORY => 'false', BLOCKCACHE => 'true'}, {NAME =>
'BinaryData', COMPRESSION => 'NONE', VERSIONS => '1', TTL => '2147483647',
BLOCKSIZE => '65536', IN_MEMORY => 'false', BLOCKCACHE => 'true'}, {NAME =>
'BinaryRetrieval', COMPRESSION => 'NONE', VERSIONS => '1', TTL =>
'2147483647', BLOCKSIZE => '65536', IN_MEMORY => 'false', BLOCKCACHE =>
'true'}, {NAME => 'Files', COMPRESSION => 'NONE', VERSIONS => '1', TTL =>
'2147483647', BLOCKSIZE => '65536', IN_MEMORY => 'false', BLOCKCACHE =>
'true'}, {NAME => 'Info', COMPRESSION => 'NONE', VERSIONS => '1', TTL =>
'2147483647', BLOCKSIZE => '65536', IN_MEMORY => 'false', BLOCKCACHE =>
'true'}, {NAME => 'Network', COMPRESSION => 'NONE', VERSIONS => '1', TTL =>
'2147483647', BLOCKSIZE => '65536', IN_MEMORY => 'false', BLOCKCACHE =>
'true'}, {NAME => 'Registry', VERSIONS => '1', COMPRESSION => 'NONE', TTL =>
'2147483647', BLOCKSIZE => '65536', IN_MEMORY => 'false', BLOCKCACHE =>
'true'}]}}
09/10/21 11:04:49 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode
/hbase/root-region-server got 10.244.9.171:60020
09/10/21 11:04:49 DEBUG client.HConnectionManager$TableServers: Found ROOT
at 10.244.9.171:60020
09/10/21 11:04:49 DEBUG client.HConnectionManager$TableServers: Cached
location address: 10.245.82.160:60020, regioninfo: REGION => {NAME =>
'.META.,,1', STARTKEY => '', ENDKEY => '', ENCODED => 1028785192, TABLE =>
{{NAME => '.META.', IS_META => 'true', MEMSTORE_FLUSHSIZE => '16384',
FAMILIES => [{NAME => 'historian', VERSIONS => '2147483647', COMPRESSION =>
'NONE', TTL => '604800', BLOCKSIZE => '8192', IN_MEMORY => 'false',
BLOCKCACHE => 'false'}, {NAME => 'info', VERSIONS => '10', COMPRESSION =>
'NONE', TTL => '2147483647', BLOCKSIZE => '8192', IN_MEMORY => 'false',
BLOCKCACHE => 'false'}]}}
09/10/21 11:04:50 DEBUG client.HTable$ClientScanner: Creating scanner over
GS_Applications starting at key ''
09/10/21 11:04:50 DEBUG client.HTable$ClientScanner: Advancing internal
scanner to startKey at ''
09/10/21 11:04:50 DEBUG client.HConnectionManager$TableServers: Cache hit
for row <> in tableName GS_Applications: location server 10.242.71.191:60020,
location region name GS_Applications,,1255020109210
*Ananth T Sarathy
On Wed, Oct 21, 2009 at 11:00 AM, stack <st...@duboce.net> wrote:
> In both cases you are doing a full table scan?
>
> Try from shell with DEBUG enable. You'll see the regions being loaded.
> May
> help you narrow in on problem region or at least on problem regionserver.
>
> St.Ack
>
> On Wed, Oct 21, 2009 at 7:19 AM, Ananth T. Sarathy <
> ananth.t.sarathy@gmail.com> wrote:
>
> > Anyone have any further thoughts on this?
> > Ananth T Sarathy
> >
> >
> > On Tue, Oct 20, 2009 at 6:37 PM, Ananth T. Sarathy <
> > ananth.t.sarathy@gmail.com> wrote:
> >
> > > Well that's not the case. Every Row has that column. In fact the
> second
> > > snippet i sent is with a column with many less rows. (1k vs 25k) but
> > comes
> > > back pretty quickly.
> > >
> > > By forever, I mean i have watched my logs do nothing for a half hour
> > before
> > > giving up.
> > >
> > >
> > > Ananth T Sarathy
> > >
> > >
> > >
> > > On Tue, Oct 20, 2009 at 5:03 PM, Ryan Rawson <ry...@gmail.com>
> wrote:
> > >
> > >> If you are asking for a column that is very sparse and doesnt exist,
> > >> it will cause HBase to read through the entire region to find 100
> > >> matching rows. This could take a while, you said 'forever', but could
> > >> you quantify that?
> > >>
> > >> On Tue, Oct 20, 2009 at 1:58 PM, Jean-Daniel Cryans <
> > jdcryans@apache.org>
> > >> wrote:
> > >> > Scanner pre-fetching is always faster, so something must be wrong
> with
> > >> > your region server. Check the logs, top, etc
> > >> >
> > >> > WRT to row size, it's pretty much a matter of how many bytes you
> have
> > >> > in each column and sum them up (plus some overhead with the keys).
> > >> >
> > >> > You want filters, check the filter package in the javadoc.
> > >> >
> > >> > J-D
> > >> >
> > >> > On Tue, Oct 20, 2009 at 1:52 PM, Ananth T. Sarathy
> > >> > <an...@gmail.com> wrote:
> > >> >> Ok, but how come
> > >> >> when I run a similiar call (with less returned rows 1000 vs 25k in
> > the
> > >> >> previous one) it runs through the iterator very quickly? (See
> Below)
> > >> >>
> > >> >> Also, how do I determine the row size? It's just text data, and
> > really
> > >> not
> > >> >> much.
> > >> >>
> > >> >> Finally, is there a way to query for rows that do not have a
> column?
> > >> (Ie all
> > >> >> rows without Files:path1)
> > >> >>
> > >> >> HBaseTableDataManagerImpl htdmni = new
> > >> HBaseTableDataManagerImpl(
> > >> >> "GS_Applications");
> > >> >>
> > >> >> String[] columns = { "Files:path1" };
> > >> >> log.info("Getting all Rows with Files");
> > >> >> Scanner s = htdmni.getScannerForAllRows(columns);
> > >> >> log.info("Got all Rows with Files");
> > >> >>
> > >> >> Iterator<RowResult> iter = s.iterator();
> > >> >> out
> > >> >>
> > >> >>
> > >>
> >
> .write("Application_Full_Name,Version,Application_installer_name,Operating
> > >> >> System, Application_platform
> > >> >>
> > >>
> >
> ,Application_sub_category,md5Hash,Sha1Hash,Sha256Hash,filepath,fileName,modified,size,operation\n");
> > >> >> out.write("<BR>");
> > >> >> while (iter.hasNext())
> > >> >> {
> > >> >>
> > >> >> Ananth T Sarathy
> > >> >>
> > >> >>
> > >> >> On Tue, Oct 20, 2009 at 4:44 PM, Jean-Daniel Cryans <
> > >> jdcryans@apache.org>wrote:
> > >> >>
> > >> >>> If you have a very slow data source (S3), then it fetches 100 row
> > >> >>> before coming back to your client with all of them and that can
> take
> > a
> > >> >>> lot of time. Also make sure that 100 of your rows can fit in a
> > region
> > >> >>> server's memory. How big is each row?
> > >> >>>
> > >> >>> J-D
> > >> >>>
> > >> >>> On Tue, Oct 20, 2009 at 1:32 PM, Ananth T. Sarathy
> > >> >>> <an...@gmail.com> wrote:
> > >> >>> > I am running this code where
> > >> >>> >
> > >> >>> > getScannerForAllRows(columns) just does return
> > >> table.getScanner(columns);
> > >> >>> >
> > >> >>> > and the table has setScannerCaching(100);
> > >> >>> >
> > >> >>> > But it spins forever after getting the iterator. Why would that
> > be?
> > >> How
> > >> >>> can
> > >> >>> > I speed it up?
> > >> >>> >
> > >> >>> > HBaseTableDataManagerImpl htdmni = new
> > >> HBaseTableDataManagerImpl(
> > >> >>> > "GS_Applications");
> > >> >>> >
> > >> >>> > String[] columns = { "Files:Name" };
> > >> >>> > log.info("Getting all Rows with Files");
> > >> >>> > Scanner s = htdmni.getScannerForAllRows(columns);
> > >> >>> > log.info("Got all Rows with Files");
> > >> >>> > log.info("Getting Iterator");
> > >> >>> >
> > >> >>> > Iterator<RowResult> iter = s.iterator();
> > >> >>> > log.info("Got Iterator");
> > >> >>> >
> > >> >>> > while (iter.hasNext())
> > >> >>> > {
> > >> >>> > log.info("Getting next Row");
> > >> >>> > RowResult rr = iter.next();
> > >> >>> >
> > >> >>> >
> > >> >>> > Ananth T Sarathy
> > >> >>> >
> > >> >>>
> > >> >>
> > >> >
> > >>
> > >
> > >
> >
>
Re: Waiting forever on scanner iterator
Posted by stack <st...@duboce.net>.
In both cases you are doing a full table scan?
Try from shell with DEBUG enable. You'll see the regions being loaded. May
help you narrow in on problem region or at least on problem regionserver.
St.Ack
On Wed, Oct 21, 2009 at 7:19 AM, Ananth T. Sarathy <
ananth.t.sarathy@gmail.com> wrote:
> Anyone have any further thoughts on this?
> Ananth T Sarathy
>
>
> On Tue, Oct 20, 2009 at 6:37 PM, Ananth T. Sarathy <
> ananth.t.sarathy@gmail.com> wrote:
>
> > Well that's not the case. Every Row has that column. In fact the second
> > snippet i sent is with a column with many less rows. (1k vs 25k) but
> comes
> > back pretty quickly.
> >
> > By forever, I mean i have watched my logs do nothing for a half hour
> before
> > giving up.
> >
> >
> > Ananth T Sarathy
> >
> >
> >
> > On Tue, Oct 20, 2009 at 5:03 PM, Ryan Rawson <ry...@gmail.com> wrote:
> >
> >> If you are asking for a column that is very sparse and doesnt exist,
> >> it will cause HBase to read through the entire region to find 100
> >> matching rows. This could take a while, you said 'forever', but could
> >> you quantify that?
> >>
> >> On Tue, Oct 20, 2009 at 1:58 PM, Jean-Daniel Cryans <
> jdcryans@apache.org>
> >> wrote:
> >> > Scanner pre-fetching is always faster, so something must be wrong with
> >> > your region server. Check the logs, top, etc
> >> >
> >> > WRT to row size, it's pretty much a matter of how many bytes you have
> >> > in each column and sum them up (plus some overhead with the keys).
> >> >
> >> > You want filters, check the filter package in the javadoc.
> >> >
> >> > J-D
> >> >
> >> > On Tue, Oct 20, 2009 at 1:52 PM, Ananth T. Sarathy
> >> > <an...@gmail.com> wrote:
> >> >> Ok, but how come
> >> >> when I run a similiar call (with less returned rows 1000 vs 25k in
> the
> >> >> previous one) it runs through the iterator very quickly? (See Below)
> >> >>
> >> >> Also, how do I determine the row size? It's just text data, and
> really
> >> not
> >> >> much.
> >> >>
> >> >> Finally, is there a way to query for rows that do not have a column?
> >> (Ie all
> >> >> rows without Files:path1)
> >> >>
> >> >> HBaseTableDataManagerImpl htdmni = new
> >> HBaseTableDataManagerImpl(
> >> >> "GS_Applications");
> >> >>
> >> >> String[] columns = { "Files:path1" };
> >> >> log.info("Getting all Rows with Files");
> >> >> Scanner s = htdmni.getScannerForAllRows(columns);
> >> >> log.info("Got all Rows with Files");
> >> >>
> >> >> Iterator<RowResult> iter = s.iterator();
> >> >> out
> >> >>
> >> >>
> >>
> .write("Application_Full_Name,Version,Application_installer_name,Operating
> >> >> System, Application_platform
> >> >>
> >>
> ,Application_sub_category,md5Hash,Sha1Hash,Sha256Hash,filepath,fileName,modified,size,operation\n");
> >> >> out.write("<BR>");
> >> >> while (iter.hasNext())
> >> >> {
> >> >>
> >> >> Ananth T Sarathy
> >> >>
> >> >>
> >> >> On Tue, Oct 20, 2009 at 4:44 PM, Jean-Daniel Cryans <
> >> jdcryans@apache.org>wrote:
> >> >>
> >> >>> If you have a very slow data source (S3), then it fetches 100 row
> >> >>> before coming back to your client with all of them and that can take
> a
> >> >>> lot of time. Also make sure that 100 of your rows can fit in a
> region
> >> >>> server's memory. How big is each row?
> >> >>>
> >> >>> J-D
> >> >>>
> >> >>> On Tue, Oct 20, 2009 at 1:32 PM, Ananth T. Sarathy
> >> >>> <an...@gmail.com> wrote:
> >> >>> > I am running this code where
> >> >>> >
> >> >>> > getScannerForAllRows(columns) just does return
> >> table.getScanner(columns);
> >> >>> >
> >> >>> > and the table has setScannerCaching(100);
> >> >>> >
> >> >>> > But it spins forever after getting the iterator. Why would that
> be?
> >> How
> >> >>> can
> >> >>> > I speed it up?
> >> >>> >
> >> >>> > HBaseTableDataManagerImpl htdmni = new
> >> HBaseTableDataManagerImpl(
> >> >>> > "GS_Applications");
> >> >>> >
> >> >>> > String[] columns = { "Files:Name" };
> >> >>> > log.info("Getting all Rows with Files");
> >> >>> > Scanner s = htdmni.getScannerForAllRows(columns);
> >> >>> > log.info("Got all Rows with Files");
> >> >>> > log.info("Getting Iterator");
> >> >>> >
> >> >>> > Iterator<RowResult> iter = s.iterator();
> >> >>> > log.info("Got Iterator");
> >> >>> >
> >> >>> > while (iter.hasNext())
> >> >>> > {
> >> >>> > log.info("Getting next Row");
> >> >>> > RowResult rr = iter.next();
> >> >>> >
> >> >>> >
> >> >>> > Ananth T Sarathy
> >> >>> >
> >> >>>
> >> >>
> >> >
> >>
> >
> >
>
Re: Waiting forever on scanner iterator
Posted by Jean-Daniel Cryans <jd...@apache.org>.
As I said yesterday, you can check the logs, top, etc. One particular
thing of interest would be a jstack of your region server's process
while it's scanning and not returning.
J-D
On Wed, Oct 21, 2009 at 7:19 AM, Ananth T. Sarathy
<an...@gmail.com> wrote:
> Anyone have any further thoughts on this?
> Ananth T Sarathy
>
>
> On Tue, Oct 20, 2009 at 6:37 PM, Ananth T. Sarathy <
> ananth.t.sarathy@gmail.com> wrote:
>
>> Well that's not the case. Every Row has that column. In fact the second
>> snippet i sent is with a column with many less rows. (1k vs 25k) but comes
>> back pretty quickly.
>>
>> By forever, I mean i have watched my logs do nothing for a half hour before
>> giving up.
>>
>>
>> Ananth T Sarathy
>>
>>
>>
>> On Tue, Oct 20, 2009 at 5:03 PM, Ryan Rawson <ry...@gmail.com> wrote:
>>
>>> If you are asking for a column that is very sparse and doesnt exist,
>>> it will cause HBase to read through the entire region to find 100
>>> matching rows. This could take a while, you said 'forever', but could
>>> you quantify that?
>>>
>>> On Tue, Oct 20, 2009 at 1:58 PM, Jean-Daniel Cryans <jd...@apache.org>
>>> wrote:
>>> > Scanner pre-fetching is always faster, so something must be wrong with
>>> > your region server. Check the logs, top, etc
>>> >
>>> > WRT to row size, it's pretty much a matter of how many bytes you have
>>> > in each column and sum them up (plus some overhead with the keys).
>>> >
>>> > You want filters, check the filter package in the javadoc.
>>> >
>>> > J-D
>>> >
>>> > On Tue, Oct 20, 2009 at 1:52 PM, Ananth T. Sarathy
>>> > <an...@gmail.com> wrote:
>>> >> Ok, but how come
>>> >> when I run a similiar call (with less returned rows 1000 vs 25k in the
>>> >> previous one) it runs through the iterator very quickly? (See Below)
>>> >>
>>> >> Also, how do I determine the row size? It's just text data, and really
>>> not
>>> >> much.
>>> >>
>>> >> Finally, is there a way to query for rows that do not have a column?
>>> (Ie all
>>> >> rows without Files:path1)
>>> >>
>>> >> HBaseTableDataManagerImpl htdmni = new
>>> HBaseTableDataManagerImpl(
>>> >> "GS_Applications");
>>> >>
>>> >> String[] columns = { "Files:path1" };
>>> >> log.info("Getting all Rows with Files");
>>> >> Scanner s = htdmni.getScannerForAllRows(columns);
>>> >> log.info("Got all Rows with Files");
>>> >>
>>> >> Iterator<RowResult> iter = s.iterator();
>>> >> out
>>> >>
>>> >>
>>> .write("Application_Full_Name,Version,Application_installer_name,Operating
>>> >> System, Application_platform
>>> >>
>>> ,Application_sub_category,md5Hash,Sha1Hash,Sha256Hash,filepath,fileName,modified,size,operation\n");
>>> >> out.write("<BR>");
>>> >> while (iter.hasNext())
>>> >> {
>>> >>
>>> >> Ananth T Sarathy
>>> >>
>>> >>
>>> >> On Tue, Oct 20, 2009 at 4:44 PM, Jean-Daniel Cryans <
>>> jdcryans@apache.org>wrote:
>>> >>
>>> >>> If you have a very slow data source (S3), then it fetches 100 row
>>> >>> before coming back to your client with all of them and that can take a
>>> >>> lot of time. Also make sure that 100 of your rows can fit in a region
>>> >>> server's memory. How big is each row?
>>> >>>
>>> >>> J-D
>>> >>>
>>> >>> On Tue, Oct 20, 2009 at 1:32 PM, Ananth T. Sarathy
>>> >>> <an...@gmail.com> wrote:
>>> >>> > I am running this code where
>>> >>> >
>>> >>> > getScannerForAllRows(columns) just does return
>>> table.getScanner(columns);
>>> >>> >
>>> >>> > and the table has setScannerCaching(100);
>>> >>> >
>>> >>> > But it spins forever after getting the iterator. Why would that be?
>>> How
>>> >>> can
>>> >>> > I speed it up?
>>> >>> >
>>> >>> > HBaseTableDataManagerImpl htdmni = new
>>> HBaseTableDataManagerImpl(
>>> >>> > "GS_Applications");
>>> >>> >
>>> >>> > String[] columns = { "Files:Name" };
>>> >>> > log.info("Getting all Rows with Files");
>>> >>> > Scanner s = htdmni.getScannerForAllRows(columns);
>>> >>> > log.info("Got all Rows with Files");
>>> >>> > log.info("Getting Iterator");
>>> >>> >
>>> >>> > Iterator<RowResult> iter = s.iterator();
>>> >>> > log.info("Got Iterator");
>>> >>> >
>>> >>> > while (iter.hasNext())
>>> >>> > {
>>> >>> > log.info("Getting next Row");
>>> >>> > RowResult rr = iter.next();
>>> >>> >
>>> >>> >
>>> >>> > Ananth T Sarathy
>>> >>> >
>>> >>>
>>> >>
>>> >
>>>
>>
>>
>
Re: Waiting forever on scanner iterator
Posted by "Ananth T. Sarathy" <an...@gmail.com>.
Anyone have any further thoughts on this?
Ananth T Sarathy
On Tue, Oct 20, 2009 at 6:37 PM, Ananth T. Sarathy <
ananth.t.sarathy@gmail.com> wrote:
> Well that's not the case. Every Row has that column. In fact the second
> snippet i sent is with a column with many less rows. (1k vs 25k) but comes
> back pretty quickly.
>
> By forever, I mean i have watched my logs do nothing for a half hour before
> giving up.
>
>
> Ananth T Sarathy
>
>
>
> On Tue, Oct 20, 2009 at 5:03 PM, Ryan Rawson <ry...@gmail.com> wrote:
>
>> If you are asking for a column that is very sparse and doesnt exist,
>> it will cause HBase to read through the entire region to find 100
>> matching rows. This could take a while, you said 'forever', but could
>> you quantify that?
>>
>> On Tue, Oct 20, 2009 at 1:58 PM, Jean-Daniel Cryans <jd...@apache.org>
>> wrote:
>> > Scanner pre-fetching is always faster, so something must be wrong with
>> > your region server. Check the logs, top, etc
>> >
>> > WRT to row size, it's pretty much a matter of how many bytes you have
>> > in each column and sum them up (plus some overhead with the keys).
>> >
>> > You want filters, check the filter package in the javadoc.
>> >
>> > J-D
>> >
>> > On Tue, Oct 20, 2009 at 1:52 PM, Ananth T. Sarathy
>> > <an...@gmail.com> wrote:
>> >> Ok, but how come
>> >> when I run a similiar call (with less returned rows 1000 vs 25k in the
>> >> previous one) it runs through the iterator very quickly? (See Below)
>> >>
>> >> Also, how do I determine the row size? It's just text data, and really
>> not
>> >> much.
>> >>
>> >> Finally, is there a way to query for rows that do not have a column?
>> (Ie all
>> >> rows without Files:path1)
>> >>
>> >> HBaseTableDataManagerImpl htdmni = new
>> HBaseTableDataManagerImpl(
>> >> "GS_Applications");
>> >>
>> >> String[] columns = { "Files:path1" };
>> >> log.info("Getting all Rows with Files");
>> >> Scanner s = htdmni.getScannerForAllRows(columns);
>> >> log.info("Got all Rows with Files");
>> >>
>> >> Iterator<RowResult> iter = s.iterator();
>> >> out
>> >>
>> >>
>> .write("Application_Full_Name,Version,Application_installer_name,Operating
>> >> System, Application_platform
>> >>
>> ,Application_sub_category,md5Hash,Sha1Hash,Sha256Hash,filepath,fileName,modified,size,operation\n");
>> >> out.write("<BR>");
>> >> while (iter.hasNext())
>> >> {
>> >>
>> >> Ananth T Sarathy
>> >>
>> >>
>> >> On Tue, Oct 20, 2009 at 4:44 PM, Jean-Daniel Cryans <
>> jdcryans@apache.org>wrote:
>> >>
>> >>> If you have a very slow data source (S3), then it fetches 100 row
>> >>> before coming back to your client with all of them and that can take a
>> >>> lot of time. Also make sure that 100 of your rows can fit in a region
>> >>> server's memory. How big is each row?
>> >>>
>> >>> J-D
>> >>>
>> >>> On Tue, Oct 20, 2009 at 1:32 PM, Ananth T. Sarathy
>> >>> <an...@gmail.com> wrote:
>> >>> > I am running this code where
>> >>> >
>> >>> > getScannerForAllRows(columns) just does return
>> table.getScanner(columns);
>> >>> >
>> >>> > and the table has setScannerCaching(100);
>> >>> >
>> >>> > But it spins forever after getting the iterator. Why would that be?
>> How
>> >>> can
>> >>> > I speed it up?
>> >>> >
>> >>> > HBaseTableDataManagerImpl htdmni = new
>> HBaseTableDataManagerImpl(
>> >>> > "GS_Applications");
>> >>> >
>> >>> > String[] columns = { "Files:Name" };
>> >>> > log.info("Getting all Rows with Files");
>> >>> > Scanner s = htdmni.getScannerForAllRows(columns);
>> >>> > log.info("Got all Rows with Files");
>> >>> > log.info("Getting Iterator");
>> >>> >
>> >>> > Iterator<RowResult> iter = s.iterator();
>> >>> > log.info("Got Iterator");
>> >>> >
>> >>> > while (iter.hasNext())
>> >>> > {
>> >>> > log.info("Getting next Row");
>> >>> > RowResult rr = iter.next();
>> >>> >
>> >>> >
>> >>> > Ananth T Sarathy
>> >>> >
>> >>>
>> >>
>> >
>>
>
>
Re: Waiting forever on scanner iterator
Posted by "Ananth T. Sarathy" <an...@gmail.com>.
Well that's not the case. Every Row has that column. In fact the second
snippet i sent is with a column with many less rows. (1k vs 25k) but comes
back pretty quickly.
By forever, I mean i have watched my logs do nothing for a half hour before
giving up.
Ananth T Sarathy
On Tue, Oct 20, 2009 at 5:03 PM, Ryan Rawson <ry...@gmail.com> wrote:
> If you are asking for a column that is very sparse and doesnt exist,
> it will cause HBase to read through the entire region to find 100
> matching rows. This could take a while, you said 'forever', but could
> you quantify that?
>
> On Tue, Oct 20, 2009 at 1:58 PM, Jean-Daniel Cryans <jd...@apache.org>
> wrote:
> > Scanner pre-fetching is always faster, so something must be wrong with
> > your region server. Check the logs, top, etc
> >
> > WRT to row size, it's pretty much a matter of how many bytes you have
> > in each column and sum them up (plus some overhead with the keys).
> >
> > You want filters, check the filter package in the javadoc.
> >
> > J-D
> >
> > On Tue, Oct 20, 2009 at 1:52 PM, Ananth T. Sarathy
> > <an...@gmail.com> wrote:
> >> Ok, but how come
> >> when I run a similiar call (with less returned rows 1000 vs 25k in the
> >> previous one) it runs through the iterator very quickly? (See Below)
> >>
> >> Also, how do I determine the row size? It's just text data, and really
> not
> >> much.
> >>
> >> Finally, is there a way to query for rows that do not have a column? (Ie
> all
> >> rows without Files:path1)
> >>
> >> HBaseTableDataManagerImpl htdmni = new HBaseTableDataManagerImpl(
> >> "GS_Applications");
> >>
> >> String[] columns = { "Files:path1" };
> >> log.info("Getting all Rows with Files");
> >> Scanner s = htdmni.getScannerForAllRows(columns);
> >> log.info("Got all Rows with Files");
> >>
> >> Iterator<RowResult> iter = s.iterator();
> >> out
> >>
> >>
> .write("Application_Full_Name,Version,Application_installer_name,Operating
> >> System, Application_platform
> >>
> ,Application_sub_category,md5Hash,Sha1Hash,Sha256Hash,filepath,fileName,modified,size,operation\n");
> >> out.write("<BR>");
> >> while (iter.hasNext())
> >> {
> >>
> >> Ananth T Sarathy
> >>
> >>
> >> On Tue, Oct 20, 2009 at 4:44 PM, Jean-Daniel Cryans <
> jdcryans@apache.org>wrote:
> >>
> >>> If you have a very slow data source (S3), then it fetches 100 row
> >>> before coming back to your client with all of them and that can take a
> >>> lot of time. Also make sure that 100 of your rows can fit in a region
> >>> server's memory. How big is each row?
> >>>
> >>> J-D
> >>>
> >>> On Tue, Oct 20, 2009 at 1:32 PM, Ananth T. Sarathy
> >>> <an...@gmail.com> wrote:
> >>> > I am running this code where
> >>> >
> >>> > getScannerForAllRows(columns) just does return
> table.getScanner(columns);
> >>> >
> >>> > and the table has setScannerCaching(100);
> >>> >
> >>> > But it spins forever after getting the iterator. Why would that be?
> How
> >>> can
> >>> > I speed it up?
> >>> >
> >>> > HBaseTableDataManagerImpl htdmni = new
> HBaseTableDataManagerImpl(
> >>> > "GS_Applications");
> >>> >
> >>> > String[] columns = { "Files:Name" };
> >>> > log.info("Getting all Rows with Files");
> >>> > Scanner s = htdmni.getScannerForAllRows(columns);
> >>> > log.info("Got all Rows with Files");
> >>> > log.info("Getting Iterator");
> >>> >
> >>> > Iterator<RowResult> iter = s.iterator();
> >>> > log.info("Got Iterator");
> >>> >
> >>> > while (iter.hasNext())
> >>> > {
> >>> > log.info("Getting next Row");
> >>> > RowResult rr = iter.next();
> >>> >
> >>> >
> >>> > Ananth T Sarathy
> >>> >
> >>>
> >>
> >
>
Re: Waiting forever on scanner iterator
Posted by Ryan Rawson <ry...@gmail.com>.
If you are asking for a column that is very sparse and doesnt exist,
it will cause HBase to read through the entire region to find 100
matching rows. This could take a while, you said 'forever', but could
you quantify that?
On Tue, Oct 20, 2009 at 1:58 PM, Jean-Daniel Cryans <jd...@apache.org> wrote:
> Scanner pre-fetching is always faster, so something must be wrong with
> your region server. Check the logs, top, etc
>
> WRT to row size, it's pretty much a matter of how many bytes you have
> in each column and sum them up (plus some overhead with the keys).
>
> You want filters, check the filter package in the javadoc.
>
> J-D
>
> On Tue, Oct 20, 2009 at 1:52 PM, Ananth T. Sarathy
> <an...@gmail.com> wrote:
>> Ok, but how come
>> when I run a similiar call (with less returned rows 1000 vs 25k in the
>> previous one) it runs through the iterator very quickly? (See Below)
>>
>> Also, how do I determine the row size? It's just text data, and really not
>> much.
>>
>> Finally, is there a way to query for rows that do not have a column? (Ie all
>> rows without Files:path1)
>>
>> HBaseTableDataManagerImpl htdmni = new HBaseTableDataManagerImpl(
>> "GS_Applications");
>>
>> String[] columns = { "Files:path1" };
>> log.info("Getting all Rows with Files");
>> Scanner s = htdmni.getScannerForAllRows(columns);
>> log.info("Got all Rows with Files");
>>
>> Iterator<RowResult> iter = s.iterator();
>> out
>>
>> .write("Application_Full_Name,Version,Application_installer_name,Operating
>> System, Application_platform
>> ,Application_sub_category,md5Hash,Sha1Hash,Sha256Hash,filepath,fileName,modified,size,operation\n");
>> out.write("<BR>");
>> while (iter.hasNext())
>> {
>>
>> Ananth T Sarathy
>>
>>
>> On Tue, Oct 20, 2009 at 4:44 PM, Jean-Daniel Cryans <jd...@apache.org>wrote:
>>
>>> If you have a very slow data source (S3), then it fetches 100 row
>>> before coming back to your client with all of them and that can take a
>>> lot of time. Also make sure that 100 of your rows can fit in a region
>>> server's memory. How big is each row?
>>>
>>> J-D
>>>
>>> On Tue, Oct 20, 2009 at 1:32 PM, Ananth T. Sarathy
>>> <an...@gmail.com> wrote:
>>> > I am running this code where
>>> >
>>> > getScannerForAllRows(columns) just does return table.getScanner(columns);
>>> >
>>> > and the table has setScannerCaching(100);
>>> >
>>> > But it spins forever after getting the iterator. Why would that be? How
>>> can
>>> > I speed it up?
>>> >
>>> > HBaseTableDataManagerImpl htdmni = new HBaseTableDataManagerImpl(
>>> > "GS_Applications");
>>> >
>>> > String[] columns = { "Files:Name" };
>>> > log.info("Getting all Rows with Files");
>>> > Scanner s = htdmni.getScannerForAllRows(columns);
>>> > log.info("Got all Rows with Files");
>>> > log.info("Getting Iterator");
>>> >
>>> > Iterator<RowResult> iter = s.iterator();
>>> > log.info("Got Iterator");
>>> >
>>> > while (iter.hasNext())
>>> > {
>>> > log.info("Getting next Row");
>>> > RowResult rr = iter.next();
>>> >
>>> >
>>> > Ananth T Sarathy
>>> >
>>>
>>
>
Re: Waiting forever on scanner iterator
Posted by Jean-Daniel Cryans <jd...@apache.org>.
Scanner pre-fetching is always faster, so something must be wrong with
your region server. Check the logs, top, etc
WRT to row size, it's pretty much a matter of how many bytes you have
in each column and sum them up (plus some overhead with the keys).
You want filters, check the filter package in the javadoc.
J-D
On Tue, Oct 20, 2009 at 1:52 PM, Ananth T. Sarathy
<an...@gmail.com> wrote:
> Ok, but how come
> when I run a similiar call (with less returned rows 1000 vs 25k in the
> previous one) it runs through the iterator very quickly? (See Below)
>
> Also, how do I determine the row size? It's just text data, and really not
> much.
>
> Finally, is there a way to query for rows that do not have a column? (Ie all
> rows without Files:path1)
>
> HBaseTableDataManagerImpl htdmni = new HBaseTableDataManagerImpl(
> "GS_Applications");
>
> String[] columns = { "Files:path1" };
> log.info("Getting all Rows with Files");
> Scanner s = htdmni.getScannerForAllRows(columns);
> log.info("Got all Rows with Files");
>
> Iterator<RowResult> iter = s.iterator();
> out
>
> .write("Application_Full_Name,Version,Application_installer_name,Operating
> System, Application_platform
> ,Application_sub_category,md5Hash,Sha1Hash,Sha256Hash,filepath,fileName,modified,size,operation\n");
> out.write("<BR>");
> while (iter.hasNext())
> {
>
> Ananth T Sarathy
>
>
> On Tue, Oct 20, 2009 at 4:44 PM, Jean-Daniel Cryans <jd...@apache.org>wrote:
>
>> If you have a very slow data source (S3), then it fetches 100 row
>> before coming back to your client with all of them and that can take a
>> lot of time. Also make sure that 100 of your rows can fit in a region
>> server's memory. How big is each row?
>>
>> J-D
>>
>> On Tue, Oct 20, 2009 at 1:32 PM, Ananth T. Sarathy
>> <an...@gmail.com> wrote:
>> > I am running this code where
>> >
>> > getScannerForAllRows(columns) just does return table.getScanner(columns);
>> >
>> > and the table has setScannerCaching(100);
>> >
>> > But it spins forever after getting the iterator. Why would that be? How
>> can
>> > I speed it up?
>> >
>> > HBaseTableDataManagerImpl htdmni = new HBaseTableDataManagerImpl(
>> > "GS_Applications");
>> >
>> > String[] columns = { "Files:Name" };
>> > log.info("Getting all Rows with Files");
>> > Scanner s = htdmni.getScannerForAllRows(columns);
>> > log.info("Got all Rows with Files");
>> > log.info("Getting Iterator");
>> >
>> > Iterator<RowResult> iter = s.iterator();
>> > log.info("Got Iterator");
>> >
>> > while (iter.hasNext())
>> > {
>> > log.info("Getting next Row");
>> > RowResult rr = iter.next();
>> >
>> >
>> > Ananth T Sarathy
>> >
>>
>
Re: Waiting forever on scanner iterator
Posted by "Ananth T. Sarathy" <an...@gmail.com>.
Ok, but how come
when I run a similiar call (with less returned rows 1000 vs 25k in the
previous one) it runs through the iterator very quickly? (See Below)
Also, how do I determine the row size? It's just text data, and really not
much.
Finally, is there a way to query for rows that do not have a column? (Ie all
rows without Files:path1)
HBaseTableDataManagerImpl htdmni = new HBaseTableDataManagerImpl(
"GS_Applications");
String[] columns = { "Files:path1" };
log.info("Getting all Rows with Files");
Scanner s = htdmni.getScannerForAllRows(columns);
log.info("Got all Rows with Files");
Iterator<RowResult> iter = s.iterator();
out
.write("Application_Full_Name,Version,Application_installer_name,Operating
System, Application_platform
,Application_sub_category,md5Hash,Sha1Hash,Sha256Hash,filepath,fileName,modified,size,operation\n");
out.write("<BR>");
while (iter.hasNext())
{
Ananth T Sarathy
On Tue, Oct 20, 2009 at 4:44 PM, Jean-Daniel Cryans <jd...@apache.org>wrote:
> If you have a very slow data source (S3), then it fetches 100 row
> before coming back to your client with all of them and that can take a
> lot of time. Also make sure that 100 of your rows can fit in a region
> server's memory. How big is each row?
>
> J-D
>
> On Tue, Oct 20, 2009 at 1:32 PM, Ananth T. Sarathy
> <an...@gmail.com> wrote:
> > I am running this code where
> >
> > getScannerForAllRows(columns) just does return table.getScanner(columns);
> >
> > and the table has setScannerCaching(100);
> >
> > But it spins forever after getting the iterator. Why would that be? How
> can
> > I speed it up?
> >
> > HBaseTableDataManagerImpl htdmni = new HBaseTableDataManagerImpl(
> > "GS_Applications");
> >
> > String[] columns = { "Files:Name" };
> > log.info("Getting all Rows with Files");
> > Scanner s = htdmni.getScannerForAllRows(columns);
> > log.info("Got all Rows with Files");
> > log.info("Getting Iterator");
> >
> > Iterator<RowResult> iter = s.iterator();
> > log.info("Got Iterator");
> >
> > while (iter.hasNext())
> > {
> > log.info("Getting next Row");
> > RowResult rr = iter.next();
> >
> >
> > Ananth T Sarathy
> >
>
Re: Waiting forever on scanner iterator
Posted by Jean-Daniel Cryans <jd...@apache.org>.
If you have a very slow data source (S3), then it fetches 100 row
before coming back to your client with all of them and that can take a
lot of time. Also make sure that 100 of your rows can fit in a region
server's memory. How big is each row?
J-D
On Tue, Oct 20, 2009 at 1:32 PM, Ananth T. Sarathy
<an...@gmail.com> wrote:
> I am running this code where
>
> getScannerForAllRows(columns) just does return table.getScanner(columns);
>
> and the table has setScannerCaching(100);
>
> But it spins forever after getting the iterator. Why would that be? How can
> I speed it up?
>
> HBaseTableDataManagerImpl htdmni = new HBaseTableDataManagerImpl(
> "GS_Applications");
>
> String[] columns = { "Files:Name" };
> log.info("Getting all Rows with Files");
> Scanner s = htdmni.getScannerForAllRows(columns);
> log.info("Got all Rows with Files");
> log.info("Getting Iterator");
>
> Iterator<RowResult> iter = s.iterator();
> log.info("Got Iterator");
>
> while (iter.hasNext())
> {
> log.info("Getting next Row");
> RowResult rr = iter.next();
>
>
> Ananth T Sarathy
>