You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@beam.apache.org by "Kenneth Knowles (Jira)" <ji...@apache.org> on 2021/05/15 17:56:02 UTC
[jira] [Updated] (BEAM-2269) Apex Runner unable to launch local
cluster on Windows
[ https://issues.apache.org/jira/browse/BEAM-2269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kenneth Knowles updated BEAM-2269:
----------------------------------
Resolution: Fixed
Status: Resolved (was: Resolved)
Hello! Due to a bug in our Jira configuration, this issue had status:Resolved but resolution:Unresolved.
I am bulk editing these issues to have resolution:Fixed
If a different resolution is appropriate, please change it. To do this, click the "Resolve" button (you can do this even for closed issues) and set the Resolution field to the right value.
> Apex Runner unable to launch local cluster on Windows
> -----------------------------------------------------
>
> Key: BEAM-2269
> URL: https://issues.apache.org/jira/browse/BEAM-2269
> Project: Beam
> Issue Type: Bug
> Components: runner-apex
> Affects Versions: 2.0.0
> Environment: Windows 2016 against Apache Beam 2.0.0 RC2
> JDK 8u131
> Maven 3.5.0
> Reporter: Luke Cwik
> Priority: P3
>
> Command:
> {code}
> mvn compile exec:java -P apex-runner -D exec.mainClass=beamrc.WordCount -D exec.args="--runner=ApexRunner --output=output/apex-counts"
> {code}
> Fails with exception:
> {code}
> java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.codehaus.mojo.exec.ExecJavaMojo$1.run(ExecJavaMojo.java:293)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: java.lang.RuntimeException: Error creating local cluster
> at org.apache.apex.engine.EmbeddedAppLauncherImpl.getController(EmbeddedAppLauncherImpl.java:122)
> at org.apache.apex.engine.EmbeddedAppLauncherImpl.launchApp(EmbeddedAppLauncherImpl.java:71)
> at org.apache.apex.engine.EmbeddedAppLauncherImpl.launchApp(EmbeddedAppLauncherImpl.java:46)
> at org.apache.beam.runners.apex.ApexRunner.run(ApexRunner.java:175)
> at org.apache.beam.runners.apex.ApexRunner.run(ApexRunner.java:73)
> at org.apache.beam.sdk.Pipeline.run(Pipeline.java:295)
> at org.apache.beam.sdk.Pipeline.run(Pipeline.java:281)
> at beamrc.WordCount.main(WordCount.java:184)
> ... 6 more
> Caused by: java.lang.NullPointerException
> at java.lang.ProcessBuilder.start(ProcessBuilder.java:1012)
> at org.apache.hadoop.util.Shell.runCommand(Shell.java:482)
> at org.apache.hadoop.util.Shell.run(Shell.java:455)
> at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:715)
> at org.apache.hadoop.util.Shell.execCommand(Shell.java:808)
> at org.apache.hadoop.util.Shell.execCommand(Shell.java:791)
> at org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:656)
> at org.apache.hadoop.fs.RawLocalFileSystem.create(RawLocalFileSystem.java:298)
> at org.apache.hadoop.fs.FileSystem.primitiveCreate(FileSystem.java:1014)
> at org.apache.hadoop.fs.DelegateToFileSystem.createInternal(DelegateToFileSystem.java:85)
> at org.apache.hadoop.fs.ChecksumFs$ChecksumFSOutputSummer.<init>(ChecksumFs.java:347)
> at org.apache.hadoop.fs.ChecksumFs.createInternal(ChecksumFs.java:394)
> at org.apache.hadoop.fs.AbstractFileSystem.create(AbstractFileSystem.java:577)
> at org.apache.hadoop.fs.FileContext$3.next(FileContext.java:680)
> at org.apache.hadoop.fs.FileContext$3.next(FileContext.java:676)
> at org.apache.hadoop.fs.FSLinkResolver.resolve(FSLinkResolver.java:90)
> at org.apache.hadoop.fs.FileContext.create(FileContext.java:676)
> at com.datatorrent.common.util.AsyncFSStorageAgent.copyToHDFS(AsyncFSStorageAgent.java:119)
> at com.datatorrent.common.util.AsyncFSStorageAgent.flush(AsyncFSStorageAgent.java:156)
> at com.datatorrent.stram.plan.physical.PhysicalPlan.initCheckpoint(PhysicalPlan.java:1230)
> at com.datatorrent.stram.plan.physical.PhysicalPlan.<init>(PhysicalPlan.java:495)
> at com.datatorrent.stram.StreamingContainerManager.<init>(StreamingContainerManager.java:425)
> at com.datatorrent.stram.StreamingContainerManager.<init>(StreamingContainerManager.java:413)
> at com.datatorrent.stram.StramLocalCluster.<init>(StramLocalCluster.java:314)
> at org.apache.apex.engine.EmbeddedAppLauncherImpl.getController(EmbeddedAppLauncherImpl.java:120)
> ... 13 more
> {code}
> If Apex doesn't support Windows for local cluster execution then this should either be closed as an unsupported use case or changed to be a feature request.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)