[jira] [Commented] (FLINK-15248) FileUtils#compressDirectory behaves buggy when processing relative directory path

["Yang Wang (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/FLINK-15248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16995614#comment-16995614 ] 

Yang Wang commented on FLINK-15248:
-----------------------------------

How about add some java document description that the target should be an absolute path? The current caller in \{{JobGraphGenerator#addUserArtifactEntries}} has done like this.

> FileUtils#compressDirectory behaves buggy when processing relative directory path
> ---------------------------------------------------------------------------------
>
>                 Key: FLINK-15248
>                 URL: https://issues.apache.org/jira/browse/FLINK-15248
>             Project: Flink
>          Issue Type: Bug
>          Components: FileSystems
>    Affects Versions: 1.10.0
>            Reporter: Wei Zhong
>            Priority: Blocker
>              Labels: pull-request-available
>             Fix For: 1.10.0
>
>          Time Spent: 10m
>  Remaining Estimate: 0h
>
> _FileUtils#compressDirectory_ behaves buggy when processing relative directory path. If the path of target directory is a relative path, the relative path inside the target zip file can not be constructed correctly:
>  
> {code:java}
> public static Path compressDirectory(Path directory, Path target) throws IOException {
>    FileSystem sourceFs = directory.getFileSystem();
>    FileSystem targetFs = target.getFileSystem();
>    try (ZipOutputStream out = new ZipOutputStream(targetFs.create(target, FileSystem.WriteMode.NO_OVERWRITE))) {
>       addToZip(directory, sourceFs, directory.getParent(), out);
>    }
>    return target;
> }
> private static void addToZip(Path fileOrDirectory, FileSystem fs, Path rootDir, ZipOutputStream out) throws IOException {
>    String relativePath = fileOrDirectory.getPath().replace(rootDir.getPath() + '/', "");
>    if (fs.getFileStatus(fileOrDirectory).isDir()) {
>       out.putNextEntry(new ZipEntry(relativePath + '/'));
>       
>       // The containedFile.getPath() returns an absolute path but the rootDir
>       // could be a relative path or an empty string (if user only specify the 
>       // directory name as the relative path). In this case when calling this 
>       // method recursively the string replacement at the beginning of it will
>       // return a wrong result.
>       for (FileStatus containedFile : fs.listStatus(fileOrDirectory)) {
>          addToZip(containedFile.getPath(), fs, rootDir, out);
>       }
>    } else {
>       ZipEntry entry = new ZipEntry(relativePath);
>       out.putNextEntry(entry);
>       try (FSDataInputStream in = fs.open(fileOrDirectory)) {
>          IOUtils.copyBytes(in, out, false);
>       }
>       out.closeEntry();
>    }
> }{code}
>  
> Currently PyFlink allows users to upload python library directories and requirements cached directory, which will be compressed by _FileUtils#compressDirectory_ eventually. If users specify them via relative paths, this bug will be triggered and causes those features unavailable.
> we can fix this bug by converting the directory path to absolute path in _FileUtils#compressDirectory_ before calling _addToZip_ method.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)