[jira] [Work logged] (BEAM-2277) IllegalArgumentException when using Hadoop file system for WordCount example.

["ASF GitHub Bot (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/BEAM-2277?focusedWorklogId=136465&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-136465 ]

ASF GitHub Bot logged work on BEAM-2277:
----------------------------------------

                Author: ASF GitHub Bot
            Created on: 21/Aug/18 11:17
            Start Date: 21/Aug/18 11:17
    Worklog Time Spent: 10m 
      Work Description: JozoVilcek opened a new pull request #6259: [BEAM-2277] Do not drop empty authority
URL: https://github.com/apache/beam/pull/6259
 
 
   When empty authority is dropped, resource `"hdfs:///path/file"` becomes `"hdfs:/path/file"` which is then, from it's string form not parseable back via `FileSystem.matchNewResource()` to `hdfs` scheme `ResourceId`, but `file`. This was introduced by stricter scheme parsing as described in `BEAM-5180`

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Issue Time Tracking
-------------------

            Worklog Id:     (was: 136465)
            Time Spent: 10m
    Remaining Estimate: 0h

> IllegalArgumentException when using Hadoop file system for WordCount example.
> -----------------------------------------------------------------------------
>
>                 Key: BEAM-2277
>                 URL: https://issues.apache.org/jira/browse/BEAM-2277
>             Project: Beam
>          Issue Type: Bug
>          Components: z-do-not-use-sdk-java-extensions
>    Affects Versions: 2.6.0
>            Reporter: Aviem Zur
>            Assignee: Aviem Zur
>            Priority: Blocker
>             Fix For: 2.0.0
>
>          Time Spent: 10m
>  Remaining Estimate: 0h
>
> IllegalArgumentException when using Hadoop file system for WordCount example.
> Occurred when running WordCount example using Spark runner on a YARN cluster.
> Command-line arguments:
> {code:none}
> --runner=SparkRunner --inputFile=hdfs:///user/myuser/kinglear.txt --output=hdfs:///user/myuser/wc/wc
> {code}
> Stack trace:
> {code:none}
> java.lang.IllegalArgumentException: Expect srcResourceIds and destResourceIds have the same scheme, but received file, hdfs.
> 	at org.apache.beam.sdk.repackaged.com.google.common.base.Preconditions.checkArgument(Preconditions.java:122)
> 	at org.apache.beam.sdk.io.FileSystems.validateSrcDestLists(FileSystems.java:394)
> 	at org.apache.beam.sdk.io.FileSystems.copy(FileSystems.java:236)
> 	at org.apache.beam.sdk.io.FileBasedSink$WriteOperation.copyToOutputFiles(FileBasedSink.java:626)
> 	at org.apache.beam.sdk.io.FileBasedSink$WriteOperation.finalize(FileBasedSink.java:516)
> 	at org.apache.beam.sdk.io.WriteFiles$2.processElement(WriteFiles.java:592)
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)