You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@beam.apache.org by "ASF GitHub Bot (JIRA)" <ji...@apache.org> on 2016/11/21 19:44:58 UTC
[jira] [Commented] (BEAM-1020) FileBasedSink fails to remove
temporary directory on Windows operating system
[ https://issues.apache.org/jira/browse/BEAM-1020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15684531#comment-15684531 ]
ASF GitHub Bot commented on BEAM-1020:
--------------------------------------
GitHub user tenghuanhe opened a pull request:
https://github.com/apache/incubator-beam/pull/1397
[BEAM-1020] Fix removing temporary directory bug on Windows operating system
Be sure to do all of the following to help us incorporate your contribution
quickly and easily:
- [ ] Make sure the PR title is formatted like:
`[BEAM-<Jira issue #>] Description of pull request`
- [ ] Make sure tests pass via `mvn clean verify`. (Even better, enable
Travis-CI on your fork and ensure the whole test matrix passes).
- [ ] Replace `<Jira issue #>` in the title with the actual Jira issue
number, if there is one.
- [ ] If this contribution is large, please file an Apache
[Individual Contributor License Agreement](https://www.apache.org/licenses/icla.txt).
---
Failing to remove temporary directory is due to the use of wildcard * when matching all files in the temporary directory, while Windows does not support wildcard * in the file path.
`Collection<String> matches = factory.match(factory.resolve(tempDirectory, "*"));`
I think It'is not very appropriate to use wildcards when removing all files in the temporary directory because there is no matching pattern need, which may cause incompatibility problems on Windows operating system.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/tenghuanhe/incubator-beam master
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-beam/pull/1397.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1397
----
commit 26fcb3a962df9d55c8b98c510177b799d4b94cf4
Author: Tenghuan He <te...@gmail.com>
Date: 2016-11-21T19:17:19Z
Fix removing temporary directory on Windows operating system
----
> FileBasedSink fails to remove temporary directory on Windows operating system
> -----------------------------------------------------------------------------
>
> Key: BEAM-1020
> URL: https://issues.apache.org/jira/browse/BEAM-1020
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
> Affects Versions: Not applicable
> Reporter: Tenghuan He
> Assignee: Davor Bonaci
> Fix For: Not applicable
>
> Original Estimate: 12h
> Remaining Estimate: 12h
>
> Beam fails to remove temporary directory after completing pipelines and sinking output results on Windows operating system and throws the following exception
> {noformat}
> Exception in thread "main" org.apache.beam.sdk.Pipeline$PipelineExecutionException: java.nio.file.InvalidPathException: Illegal char <*> at index 0: *
> at org.apache.beam.runners.direct.DirectRunner.run(DirectRunner.java:348)
> at org.apache.beam.runners.direct.DirectRunner.run(DirectRunner.java:73)
> at org.apache.beam.sdk.Pipeline.run(Pipeline.java:179)
> at org.apache.beam.examples.MinimalWordCount.main(MinimalWordCount.java:116)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at com.intellij.rt.execution.application.AppMain.main(AppMain.java:144)
> Caused by: java.nio.file.InvalidPathException: Illegal char <*> at index 0: *
> at sun.nio.fs.WindowsPathParser.normalize(WindowsPathParser.java:182)
> at sun.nio.fs.WindowsPathParser.parse(WindowsPathParser.java:153)
> at sun.nio.fs.WindowsPathParser.parse(WindowsPathParser.java:77)
> at sun.nio.fs.WindowsPath.parse(WindowsPath.java:94)
> at sun.nio.fs.WindowsFileSystem.getPath(WindowsFileSystem.java:255)
> at sun.nio.fs.AbstractPath.resolve(AbstractPath.java:53)
> at org.apache.beam.sdk.util.FileIOChannelFactory.resolve(FileIOChannelFactory.java:159)
> at org.apache.beam.sdk.io.FileBasedSink$FileBasedWriteOperation.removeTemporaryFiles(FileBasedSink.java:469)
> at org.apache.beam.sdk.io.FileBasedSink$FileBasedWriteOperation.finalize(FileBasedSink.java:388)
> at org.apache.beam.sdk.io.Write$Bound$2.processElement(Write.java:417)
> {noformat}
> This is due to the use of wildcard * when matching all files in the temporary directory, while Windows does not support wildcard * in the file path.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)