[GitHub] [beam] kennknowles opened a new issue, #19025: Dynamic destination write to HDFS fails with IntervalWindow in shard template

[GitBox <gi...@apache.org>]

kennknowles opened a new issue, #19025:
URL: https://github.com/apache/beam/issues/19025

   I am doing a windowed write with dynamic destinations and it is failing with error message :
   ```
   
   java.lang.IllegalArgumentException: Expect srcResourceIds and destResourceIds have the same scheme,
   but received hdfs, window-1970-01-01T00
   ```
   
    
   
   I am writing from IntervalWindow, for which target file name created by DefaultFilenamePolicy is going to look something like this:
   ```
   
   window-1970-01-01T00:00:00.000Z-1970-01-01T01:00:00.000Z-pane-0-last-00000-of-00001.avro
   ```
   
   When resolving the path, HadoopResourceId is used and this one is backed by java URI. The filename like above is parsed incorrectly because of ':' character introduced by window-to-string. 
   
    
   
   Imported from Jira [BEAM-4672](https://issues.apache.org/jira/browse/BEAM-4672). Original Jira may contain additional context.
   Reported by: JozoVilcek.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org