You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@apex.apache.org by Siyuan Hua <si...@datatorrent.com> on 2016/03/04 19:21:06 UTC

Exception in localmode

Has anyone seen this issue?

java.lang.RuntimeException: Error creating local cluster
    at
com.datatorrent.stram.LocalModeImpl.getController(LocalModeImpl.java:69)
    at
org.apache.apex.malhar.kafka.KafkaInputOperatorTest.testInputOperator(KafkaInputOperatorTest.java:218)
    at
org.apache.apex.malhar.kafka.KafkaInputOperatorTest.testInputOperator(KafkaInputOperatorTest.java:160)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
    at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
    at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
    at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
    at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
    at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
    at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
    at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
    at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
    at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
    at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
    at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
    at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
    at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
    at org.junit.runners.Suite.runChild(Suite.java:127)
    at org.junit.runners.Suite.runChild(Suite.java:26)
    at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
    at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
    at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
    at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
    at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
    at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
    at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
    at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
    at org.junit.runner.JUnitCore.run(JUnitCore.java:160)
    at
com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:78)
    at
com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:212)
    at
com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:68)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at com.intellij.rt.execution.application.AppMain.main(AppMain.java:140)
Caused by: java.lang.RuntimeException:
org.apache.hadoop.util.Shell$ExitCodeException: chmod: cannot access
‘/home/siyuan/gitworkspace/incubator-apex-malhar/kafka/target/com.datatorrent.stram.StramLocalCluster/checkpoints/2/_tmp’:
No such file or directory

    at
com.datatorrent.netlet.util.DTThrowable.wrapIfChecked(DTThrowable.java:50)
    at com.datatorrent.netlet.util.DTThrowable.rethrow(DTThrowable.java:31)
    at
com.datatorrent.common.util.AsyncFSStorageAgent.copyToHDFS(AsyncFSStorageAgent.java:124)
    at
com.datatorrent.stram.plan.physical.PhysicalPlan.initCheckpoint(PhysicalPlan.java:1069)
    at
com.datatorrent.stram.plan.physical.PhysicalPlan.<init>(PhysicalPlan.java:380)
    at
com.datatorrent.stram.StreamingContainerManager.<init>(StreamingContainerManager.java:344)
    at
com.datatorrent.stram.StreamingContainerManager.<init>(StreamingContainerManager.java:332)
    at
com.datatorrent.stram.StramLocalCluster.<init>(StramLocalCluster.java:310)
    at
com.datatorrent.stram.LocalModeImpl.getController(LocalModeImpl.java:67)
    ... 39 more
Caused by: org.apache.hadoop.util.Shell$ExitCodeException: chmod: cannot
access
‘/home/siyuan/gitworkspace/incubator-apex-malhar/kafka/target/com.datatorrent.stram.StramLocalCluster/checkpoints/2/_tmp’:
No such file or directory

    at org.apache.hadoop.util.Shell.runCommand(Shell.java:464)
    at org.apache.hadoop.util.Shell.run(Shell.java:379)
    at
org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:589)
    at org.apache.hadoop.util.Shell.execCommand(Shell.java:678)
    at org.apache.hadoop.util.Shell.execCommand(Shell.java:661)
    at
org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:639)
    at
org.apache.hadoop.fs.RawLocalFileSystem.create(RawLocalFileSystem.java:305)
    at org.apache.hadoop.fs.FileSystem.primitiveCreate(FileSystem.java:1011)
    at
org.apache.hadoop.fs.DelegateToFileSystem.createInternal(DelegateToFileSystem.java:85)
    at
org.apache.hadoop.fs.ChecksumFs$ChecksumFSOutputSummer.<init>(ChecksumFs.java:344)
    at org.apache.hadoop.fs.ChecksumFs.createInternal(ChecksumFs.java:390)
    at
org.apache.hadoop.fs.AbstractFileSystem.create(AbstractFileSystem.java:575)
    at org.apache.hadoop.fs.FileContext$3.next(FileContext.java:676)
    at org.apache.hadoop.fs.FileContext$3.next(FileContext.java:672)
    at org.apache.hadoop.fs.FSLinkResolver.resolve(FSLinkResolver.java:90)
    at org.apache.hadoop.fs.FileContext.create(FileContext.java:672)
    at
com.datatorrent.common.util.AsyncFSStorageAgent.copyToHDFS(AsyncFSStorageAgent.java:112)
    ... 45 more

Re: Exception in localmode

Posted by Siyuan Hua <si...@datatorrent.com>.

Looks like the _tmp file will always be there

On Fri, Mar 4, 2016 at 10:21 AM, Siyuan Hua <si...@datatorrent.com> wrote:

> Has anyone seen this issue?
>
> java.lang.RuntimeException: Error creating local cluster
>     at
> com.datatorrent.stram.LocalModeImpl.getController(LocalModeImpl.java:69)
>     at
> org.apache.apex.malhar.kafka.KafkaInputOperatorTest.testInputOperator(KafkaInputOperatorTest.java:218)
>     at
> org.apache.apex.malhar.kafka.KafkaInputOperatorTest.testInputOperator(KafkaInputOperatorTest.java:160)
>     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>     at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>     at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>     at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
>     at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
>     at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
>     at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
>     at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
>     at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
>     at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
>     at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
>     at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
>     at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
>     at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
>     at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
>     at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
>     at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
>     at org.junit.runners.Suite.runChild(Suite.java:127)
>     at org.junit.runners.Suite.runChild(Suite.java:26)
>     at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
>     at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
>     at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
>     at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
>     at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
>     at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
>     at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
>     at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
>     at org.junit.runner.JUnitCore.run(JUnitCore.java:160)
>     at
> com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:78)
>     at
> com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:212)
>     at
> com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:68)
>     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>     at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>     at com.intellij.rt.execution.application.AppMain.main(AppMain.java:140)
> Caused by: java.lang.RuntimeException:
> org.apache.hadoop.util.Shell$ExitCodeException: chmod: cannot access
> ‘/home/siyuan/gitworkspace/incubator-apex-malhar/kafka/target/com.datatorrent.stram.StramLocalCluster/checkpoints/2/_tmp’:
> No such file or directory
>
>     at
> com.datatorrent.netlet.util.DTThrowable.wrapIfChecked(DTThrowable.java:50)
>     at com.datatorrent.netlet.util.DTThrowable.rethrow(DTThrowable.java:31)
>     at
> com.datatorrent.common.util.AsyncFSStorageAgent.copyToHDFS(AsyncFSStorageAgent.java:124)
>     at
> com.datatorrent.stram.plan.physical.PhysicalPlan.initCheckpoint(PhysicalPlan.java:1069)
>     at
> com.datatorrent.stram.plan.physical.PhysicalPlan.<init>(PhysicalPlan.java:380)
>     at
> com.datatorrent.stram.StreamingContainerManager.<init>(StreamingContainerManager.java:344)
>     at
> com.datatorrent.stram.StreamingContainerManager.<init>(StreamingContainerManager.java:332)
>     at
> com.datatorrent.stram.StramLocalCluster.<init>(StramLocalCluster.java:310)
>     at
> com.datatorrent.stram.LocalModeImpl.getController(LocalModeImpl.java:67)
>     ... 39 more
> Caused by: org.apache.hadoop.util.Shell$ExitCodeException: chmod: cannot
> access
> ‘/home/siyuan/gitworkspace/incubator-apex-malhar/kafka/target/com.datatorrent.stram.StramLocalCluster/checkpoints/2/_tmp’:
> No such file or directory
>
>     at org.apache.hadoop.util.Shell.runCommand(Shell.java:464)
>     at org.apache.hadoop.util.Shell.run(Shell.java:379)
>     at
> org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:589)
>     at org.apache.hadoop.util.Shell.execCommand(Shell.java:678)
>     at org.apache.hadoop.util.Shell.execCommand(Shell.java:661)
>     at
> org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:639)
>     at
> org.apache.hadoop.fs.RawLocalFileSystem.create(RawLocalFileSystem.java:305)
>     at
> org.apache.hadoop.fs.FileSystem.primitiveCreate(FileSystem.java:1011)
>     at
> org.apache.hadoop.fs.DelegateToFileSystem.createInternal(DelegateToFileSystem.java:85)
>     at
> org.apache.hadoop.fs.ChecksumFs$ChecksumFSOutputSummer.<init>(ChecksumFs.java:344)
>     at org.apache.hadoop.fs.ChecksumFs.createInternal(ChecksumFs.java:390)
>     at
> org.apache.hadoop.fs.AbstractFileSystem.create(AbstractFileSystem.java:575)
>     at org.apache.hadoop.fs.FileContext$3.next(FileContext.java:676)
>     at org.apache.hadoop.fs.FileContext$3.next(FileContext.java:672)
>     at org.apache.hadoop.fs.FSLinkResolver.resolve(FSLinkResolver.java:90)
>     at org.apache.hadoop.fs.FileContext.create(FileContext.java:672)
>     at
> com.datatorrent.common.util.AsyncFSStorageAgent.copyToHDFS(AsyncFSStorageAgent.java:112)
>     ... 45 more
>
>