You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flume.apache.org by "Roshan Naik (JIRA)" <ji...@apache.org> on 2013/07/24 04:58:48 UTC
[jira] [Commented] (FLUME-2134) AsyncHbase Sink bugfix plus tests
errors on Windows
[ https://issues.apache.org/jira/browse/FLUME-2134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13717916#comment-13717916 ]
Roshan Naik commented on FLUME-2134:
------------------------------------
The Async HBase sink uses conf.get("hbase.zookeeper.quorum") to get the zookeeper quorum info... which on windows is only returning "localhost". HBase folks advised the use of ZKConfig.getZKQuorumServersString(conf) instead. I confirmed it returns the right value "localhost:port#" on both Windows and Linux.
> AsyncHbase Sink bugfix plus tests errors on Windows
> ---------------------------------------------------
>
> Key: FLUME-2134
> URL: https://issues.apache.org/jira/browse/FLUME-2134
> Project: Flume
> Issue Type: Bug
> Components: Sinks+Sources, Test, Windows
> Reporter: Roshan Naik
> Assignee: Roshan Naik
> Labels: hbase, sink, test, windows
>
> Problem appears to be inabilty to handle windows paths.
> {code}
> 13/07/23 17:16:35 INFO server.PrepRequestProcessor: Got user-level KeeperException when processing sessionid:0x1400e08d22b0000 type:delete cxid:0x13 zxid:0xb txntype:-1 reqpath:n/a Error Path:/hbase/backup-masters/192.168.50.128,55884,1374624994459 Error:KeeperErrorCode = NoNode for /hbase/backup-masters/192.168.50.128,55884,1374624994459
> 13/07/23 17:16:35 WARN zookeeper.RecoverableZooKeeper: Node /hbase/backup-masters/192.168.50.128,55884,1374624994459 already deleted, and this is not a retry
> 13/07/23 17:16:35 INFO master.ActiveMasterManager: Master=192.168.50.128,55884,1374624994459
> 13/07/23 17:16:35 ERROR master.HMaster: Unhandled exception. Starting shutdown.
> java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative path in absolute URI: file://C:%5CUsers%5CROSHAN~1%5CAppData%5CLocal%5CTemp%5C1374624993647-0%5Chbase.tmp
> at org.apache.hadoop.fs.Path.initialize(Path.java:148)
> at org.apache.hadoop.fs.Path.<init>(Path.java:71)
> at org.apache.hadoop.fs.Path.<init>(Path.java:50)
> at org.apache.hadoop.hbase.master.MasterFileSystem.<init>(MasterFileSystem.java:111)
> at org.apache.hadoop.hbase.master.HMaster.finishInitialization(HMaster.java:532)
> at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:391)
> at java.lang.Thread.run(Thread.java:722)
> Caused by: java.net.URISyntaxException: Relative path in absolute URI: file://C:%5CUsers%5CROSHAN~1%5CAppData%5CLocal%5CTemp%5C1374624993647-0%5Chbase.tmp
> at java.net.URI.checkPath(URI.java:1804)
> at java.net.URI.<init>(URI.java:752)
> at org.apache.hadoop.fs.Path.initialize(Path.java:145)
> ... 6 more
> 13/07/23 17:16:35 INFO master.HMaster: Aborting
> 13/07/23 17:16:35 INFO ipc.HBaseServer: Stopping server on 55884
> 13/07/23 17:16:35 INFO ipc.HBaseServer: Stopping IPC Server listener on 55884
> 13/07/23 17:16:35 INFO ipc.HBaseServer: Stopping IPC Server Responder
> 13/07/23 17:16:35 INFO ipc.HBaseServer: Stopping IPC Server Responder
> 13/07/23 17:16:35 INFO ipc.HBaseServer: IPC Server handler 0 on 55884: exiting
> 13/07/23 17:16:35 INFO ipc.HBaseServer: IPC Server handler 1 on 55884: exiting
> 13/07/23 17:16:35 INFO ipc.HBaseServer: IPC Server handler 2 on 55884: exiting
> 13/07/23 17:16:35 INFO ipc.HBaseServer: IPC Server handler 3 on 55884: exiting
> 13/07/23 17:16:35 INFO ipc.HBaseServer: IPC Server handler 4 on 55884: exiting
> 13/07/23 17:16:35 INFO ipc.HBaseServer: IPC Server handler 5 on 55884: exiting
> 13/07/23 17:16:35 INFO ipc.HBaseServer: IPC Server handler 6 on 55884: exiting
> 13/07/23 17:16:35 INFO ipc.HBaseServer: IPC Server handler 7 on 55884: exiting
> 13/07/23 17:16:35 INFO ipc.HBaseServer: IPC Server handler 8 on 55884: exiting
> 13/07/23 17:16:35 INFO ipc.HBaseServer: IPC Server handler 9 on 55884: exiting
> 13/07/23 17:16:35 INFO ipc.HBaseServer: REPL IPC Server handler 0 on 55884: exiting
> 13/07/23 17:16:35 INFO ipc.HBaseServer: REPL IPC Server handler 1 on 55884: exiting
> 13/07/23 17:16:35 INFO ipc.HBaseServer: REPL IPC Server handler 2 on 55884: exiting
> 13/07/23 17:16:35 INFO server.PrepRequestProcessor: Processed session termination for sessionid: 0x1400e08d22b0000
> 13/07/23 17:16:35 INFO server.NIOServerCnxn: Closed socket connection for client /192.168.50.128:55911 which had sessionid 0x1400e08d22b0000
> 13/07/23 17:16:35 INFO zookeeper.ZooKeeper: Session: 0x1400e08d22b0000 closed
> 13/07/23 17:16:35 INFO master.HMaster: HMaster main thread exiting
> 13/07/23 17:16:35 INFO zookeeper.ClientCnxn: EventThread shut down
> 13/07/23 17:16:35 INFO regionserver.ShutdownHook: Installed shutdown hook thread: Shutdownhook:RegionServer:0;192.168.50.128,55912,1374624994999
> 13/07/23 17:16:35 INFO zookeeper.ZooKeeper: Initiating client connection, connectString=0.0.0.0:2181 sessionTimeout=180000 watcher=regionserver:55912
> 13/07/23 17:16:35 INFO zookeeper.RecoverableZooKeeper: The identifier of this process is 5824@WIN-6AKNOG7MFK5
> 13/07/23 17:16:35 INFO zookeeper.ClientCnxn: Opening socket connection to server 0.0.0.0/0.0.0.0:2181. Will not attempt to authenticate using SASL (unknown error)
> 13/07/23 17:16:35 INFO server.NIOServerCnxnFactory: Accepted socket connection from /192.168.50.128:55939
> 13/07/23 17:16:35 INFO zookeeper.ClientCnxn: Socket connection established to 0.0.0.0/0.0.0.0:2181, initiating session
> 13/07/23 17:16:35 INFO server.ZooKeeperServer: Client attempting to establish new session at /192.168.50.128:55939
> 13/07/23 17:16:35 INFO zookeeper.ClientCnxn: Session establishment complete on server 0.0.0.0/0.0.0.0:2181, sessionid = 0x1400e08d22b0001, negotiated timeout = 40000
> 13/07/23 17:16:35 INFO server.ZooKeeperServer: Established session 0x1400e08d22b0001 with negotiated timeout 40000 for client /192.168.50.128:55939
> 13/07/23 17:19:55 ERROR hbase.MiniHBaseCluster: Error starting cluster
> java.lang.RuntimeException: Master not initialized after 200 seconds
> at org.apache.hadoop.hbase.util.JVMClusterUtil.startup(JVMClusterUtil.java:206)
> at org.apache.hadoop.hbase.LocalHBaseCluster.startup(LocalHBaseCluster.java:420)
> at org.apache.hadoop.hbase.MiniHBaseCluster.init(MiniHBaseCluster.java:216)
> at org.apache.hadoop.hbase.MiniHBaseCluster.<init>(MiniHBaseCluster.java:86)
> at org.apache.hadoop.hbase.MiniHBaseCluster.<init>(MiniHBaseCluster.java:77)
> at org.apache.hadoop.hbase.MiniHBaseCluster.<init>(MiniHBaseCluster.java:64)
> at org.apache.flume.sink.hbase.TestAsyncHBaseSink.setUp(TestAsyncHBaseSink.java:123)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:601)
> at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
> at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
> at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
> at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:27)
> at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
> at org.junit.runner.JUnitCore.run(JUnitCore.java:157)
> at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:77)
> at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:195)
> at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:63)
> 13/07/23 17:19:55 WARN zookeeper.RecoverableZooKeeper: Possibly transient ZooKeeper exception: org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /hbase/shutdown
> 13/07/23 17:19:55 INFO util.RetryCounter: Sleeping 2000ms before retry #1...
> 13/07/23 17:19:57 WARN zookeeper.RecoverableZooKeeper: Possibly transient ZooKeeper exception: org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /hbase/shutdown
> 13/07/23 17:19:57 INFO util.RetryCounter: Sleeping 4000ms before retry #2...
> 13/07/23 17:20:01 WARN zookeeper.RecoverableZooKeeper: Possibly transient ZooKeeper exception: org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /hbase/shutdown
> 13/07/23 17:20:01 INFO util.RetryCounter: Sleeping 8000ms before retry #3...
> 13/07/23 17:20:09 WARN zookeeper.RecoverableZooKeeper: Possibly transient ZooKeeper exception: org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /hbase/shutdown
> 13/07/23 17:20:09 ERROR zookeeper.RecoverableZooKeeper: ZooKeeper delete failed after 3 retries
> 13/07/23 17:20:09 ERROR master.HMaster: ZooKeeper exception trying to set cluster as down in ZK
> org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /hbase/shutdown
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
> at org.apache.zookeeper.ZooKeeper.delete(ZooKeeper.java:873)
> at org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.delete(RecoverableZooKeeper.java:133)
> at org.apache.hadoop.hbase.zookeeper.ZKUtil.deleteNode(ZKUtil.java:1197)
> at org.apache.hadoop.hbase.zookeeper.ZKUtil.deleteNode(ZKUtil.java:1186)
> at org.apache.hadoop.hbase.zookeeper.ClusterStatusTracker.setClusterDown(ClusterStatusTracker.java:80)
> at org.apache.hadoop.hbase.master.HMaster.shutdown(HMaster.java:1672)
> at org.apache.hadoop.hbase.util.JVMClusterUtil.shutdown(JVMClusterUtil.java:234)
> at org.apache.hadoop.hbase.LocalHBaseCluster.shutdown(LocalHBaseCluster.java:427)
> at org.apache.hadoop.hbase.MiniHBaseCluster.shutdown(MiniHBaseCluster.java:492)
> at org.apache.hadoop.hbase.MiniHBaseCluster.init(MiniHBaseCluster.java:222)
> at org.apache.hadoop.hbase.MiniHBaseCluster.<init>(MiniHBaseCluster.java:86)
> at org.apache.hadoop.hbase.MiniHBaseCluster.<init>(MiniHBaseCluster.java:77)
> at org.apache.hadoop.hbase.MiniHBaseCluster.<init>(MiniHBaseCluster.java:64)
> at org.apache.flume.sink.hbase.TestAsyncHBaseSink.setUp(TestAsyncHBaseSink.java:123)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:601)
> at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
> at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
> at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
> at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:27)
> at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
> at org.junit.runner.JUnitCore.run(JUnitCore.java:157)
> at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:77)
> at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:195)
> at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:63)
> 13/07/23 17:20:09 INFO regionserver.ShutdownHook: Shutdown hook starting; hbase.shutdown.hook=true; fsShutdownHook=Thread[Thread-19,5,main]
> 13/07/23 17:20:09 ERROR server.NIOServerCnxnFactory: Thread Thread[Shutdownhook:RegionServer:0;192.168.50.128,55912,1374624994999,5,main] died
> java.lang.NullPointerException
> at org.apache.hadoop.hbase.regionserver.HRegionServer.stop(HRegionServer.java:1810)
> at org.apache.hadoop.hbase.regionserver.ShutdownHook$ShutdownHookThread.run(ShutdownHook.java:113)
> java.lang.NullPointerException
> at org.apache.hadoop.hbase.regionserver.HRegionServer.stop(HRegionServer.java:1810)
> at org.apache.hadoop.hbase.util.JVMClusterUtil.shutdown(JVMClusterUtil.java:241)
> at org.apache.hadoop.hbase.LocalHBaseCluster.shutdown(LocalHBaseCluster.java:427)
> at org.apache.hadoop.hbase.MiniHBaseCluster.shutdown(MiniHBaseCluster.java:492)
> at org.apache.hadoop.hbase.MiniHBaseCluster.init(MiniHBaseCluster.java:222)
> at org.apache.hadoop.hbase.MiniHBaseCluster.<init>(MiniHBaseCluster.java:86)
> at org.apache.hadoop.hbase.MiniHBaseCluster.<init>(MiniHBaseCluster.java:77)
> at org.apache.hadoop.hbase.MiniHBaseCluster.<init>(MiniHBaseCluster.java:64)
> at org.apache.flume.sink.hbase.TestAsyncHBaseSink.setUp(TestAsyncHBaseSink.java:123)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
> at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
> at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
> at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:27)
> at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
> at org.junit.runner.JUnitCore.run(JUnitCore.java:157)
> at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:63)
> java.lang.NullPointerException
> at org.apache.flume.sink.hbase.TestAsyncHBaseSink.tearDown(TestAsyncHBaseSink.java:162)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
> at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
> at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
> at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:36)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
> at org.junit.runner.JUnitCore.run(JUnitCore.java:157)
> at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:63)
> java.lang.NullPointerException
> at org.apache.flume.sink.hbase.TestAsyncHBaseSink.tearDown(TestAsyncHBaseSink.java:162)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
> at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
> at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
> at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:36)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
> at org.junit.runner.JUnitCore.run(JUnitCore.java:157)
> at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:63)
> java.lang.NullPointerException
> at org.apache.flume.sink.hbase.TestAsyncHBaseSink.tearDown(TestAsyncHBaseSink.java:162)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
> at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
> at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
> at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:36)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
> at org.junit.runner.JUnitCore.run(JUnitCore.java:157)
> at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:63)
> java.lang.NullPointerException
> at org.apache.flume.sink.hbase.TestAsyncHBaseSink.tearDown(TestAsyncHBaseSink.java:162)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
> at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
> at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
> at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:36)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
> at org.junit.runner.JUnitCore.run(JUnitCore.java:157)
> at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:63)
> Disconnected from the target VM, address: '127.0.0.1:55877', transport: 'socket'
> Process finished with exit code -1
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira