[jira] [Assigned] (BEAM-5417) FileSystems.match behaviour diff between GCS and local file system

["Chamikara Jayalath (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/BEAM-5417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Chamikara Jayalath reassigned BEAM-5417:
----------------------------------------

    Assignee: Udi Meiri  (was: Chamikara Jayalath)

> FileSystems.match behaviour diff between GCS and local file system
> ------------------------------------------------------------------
>
>                 Key: BEAM-5417
>                 URL: https://issues.apache.org/jira/browse/BEAM-5417
>             Project: Beam
>          Issue Type: Bug
>          Components: sdk-py-core
>    Affects Versions: 2.5.0, 2.6.0
>            Reporter: Joar Wandborg
>            Assignee: Udi Meiri
>            Priority: Major
>          Time Spent: 7h
>  Remaining Estimate: 0h
>
> Given the directory structure:
>  
> {noformat}
> .
> ├── filesystem-match-test
> │   ├── a
> │   │   └── file.txt
> │   └── b
> │       └── file.txt
> └── filesystem-match-test.py
> {noformat}
>  
> Where {{filesystem-match-test.py}} contains:
> {code:python}
> from __future__ import print_function
> import os
> import posixpath
> from apache_beam.io.filesystem import MatchResult
> from apache_beam.io.filesystems import FileSystems
> BASES = [
>     os.path.join(os.path.dirname(__file__), "./"),
>     "gs://my-bucket/test/",
> ]
> pattern = "filesystem-match-test/*/file.txt"
> for base_path in BASES:
>     full_pattern = posixpath.join(base_path, pattern)
>     print("full_pattern: {}".format(full_pattern))
>     match_result = FileSystems.match([full_pattern])[0]  # type: MatchResult
>     print("metadata list: {}".format(match_result.metadata_list))
> {code}
> Running {{python filesystem-match-test.py}} does not match any files locally, but does match files on GCS:
> {noformat}
> full_pattern: ./filesystem-match-test/*/file.txt
> metadata list: []
> full_pattern: gs://my-bucket/test/filesystem-match-test/*/file.txt
> metadata list: [FileMetadata(gs://my-bucket/test/filesystem-match-test/a/file.txt, 6), FileMetadata(gs://my-bucket/test/filesystem-match-test/b/file.txt, 6)]
> {noformat}
> The expected result is that a/file.txt and b/file.txt should be matched for both patterns.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)