You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@beam.apache.org by "Stanislav Kardashov (JIRA)" <ji...@apache.org> on 2018/05/23 16:45:00 UTC
[jira] [Updated] (BEAM-4380) Impossible to submit Google Dataflow
job from Windows machine
[ https://issues.apache.org/jira/browse/BEAM-4380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Stanislav Kardashov updated BEAM-4380:
--------------------------------------
Description:
Got exception when trying to submit Dataflow job from java application running on Windows machine.
{quote}java.lang.IllegalArgumentException: Location must be local or on Cloud Storage, got C:\Users\myuser\AppData\Local\Temp\dataflow-job341260414234234.json.
{quote}
Here:
[https://github.com/apache/beam/blob/67f6cf2c7786e339758854efb1cf1230909e4f1d/runners/google-cloud-dataflow-java/src/main/java/org/apache/beam/runners/dataflow/DataflowRunner.java#L746]
following check is performed: fileLocation.startsWith("/") || fileLocation.startsWith("gs://"),
which prohibits usage of any Windows-style file names.
Workaround is to provide explicit file location on GCS:
dataflowPipelineOptions.setDataflowJobFile("gs://mybucket/dataflowjob.pb");
was:
Got exception when trying to submit Dataflow job from java application running on Windows machine.
java.lang.IllegalArgumentException: Location must be local or on Cloud Storage, got C:\Users\myuser\AppData\Local\Temp\dataflow-job341260414234234.json.
Here:
[https://github.com/apache/beam/blob/67f6cf2c7786e339758854efb1cf1230909e4f1d/runners/google-cloud-dataflow-java/src/main/java/org/apache/beam/runners/dataflow/DataflowRunner.java#L746]
following check is performed: fileLocation.startsWith("/") || fileLocation.startsWith("gs://"),
which prohibits usage of any Windows-style file names.
Workaround is to provide explicit file location on GCS:
dataflowPipelineOptions.setDataflowJobFile("gs://mybucket/dataflowjob.pb");
> Impossible to submit Google Dataflow job from Windows machine
> -------------------------------------------------------------
>
> Key: BEAM-4380
> URL: https://issues.apache.org/jira/browse/BEAM-4380
> Project: Beam
> Issue Type: Bug
> Components: runner-dataflow
> Reporter: Stanislav Kardashov
> Assignee: Thomas Groh
> Priority: Trivial
>
> Got exception when trying to submit Dataflow job from java application running on Windows machine.
> {quote}java.lang.IllegalArgumentException: Location must be local or on Cloud Storage, got C:\Users\myuser\AppData\Local\Temp\dataflow-job341260414234234.json.
> {quote}
> Here:
> [https://github.com/apache/beam/blob/67f6cf2c7786e339758854efb1cf1230909e4f1d/runners/google-cloud-dataflow-java/src/main/java/org/apache/beam/runners/dataflow/DataflowRunner.java#L746]
> following check is performed: fileLocation.startsWith("/") || fileLocation.startsWith("gs://"),
> which prohibits usage of any Windows-style file names.
> Workaround is to provide explicit file location on GCS:
> dataflowPipelineOptions.setDataflowJobFile("gs://mybucket/dataflowjob.pb");
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)