[jira] [Created] (BEAM-5417) FileSystems.match behaviour diff between GCS and local file system

["Joar Wandborg (JIRA)" <ji...@apache.org>]

Joar Wandborg created BEAM-5417:
-----------------------------------

             Summary: FileSystems.match behaviour diff between GCS and local file system
                 Key: BEAM-5417
                 URL: https://issues.apache.org/jira/browse/BEAM-5417
             Project: Beam
          Issue Type: Bug
          Components: sdk-py-core
    Affects Versions: 2.6.0
            Reporter: Joar Wandborg
            Assignee: Ahmet Altay


Given the directory structure:

 
{noformat}
.
├── filesystem-match-test
│   ├── a
│   │   └── file.txt
│   └── b
│       └── file.txt
└── filesystem-match-test.py
{noformat}
 

Where {{filesystem-match-test.py}} contains:
{code:python}
from __future__ import print_function
import os
import posixpath

from apache_beam.io.filesystem import MatchResult
from apache_beam.io.filesystems import FileSystems

BASES = [
    os.path.join(os.path.dirname(__file__), "./"),
    "gs://my-bucket/test/",
]
pattern = "filesystem-match-test/*/file.txt"
for base_path in BASES:
    full_pattern = posixpath.join(base_path, pattern)
    print("full_pattern: {}".format(full_pattern))
    match_result = FileSystems.match([full_pattern])[0]  # type: MatchResult
    print("metadata list: {}".format(match_result.metadata_list))
{code}
Running {{python filesystem-match-test.py}} does not match any files locally, but does match files on GCS:
{noformat}
full_pattern: ./filesystem-match-test/*/file.txt
metadata list: []
full_pattern: gs://my-bucket/test/filesystem-match-test/*/file.txt
metadata list: [FileMetadata(gs://my-bucket/test/filesystem-match-test/a/file.txt, 6), FileMetadata(gs://my-bucket/test/filesystem-match-test/b/file.txt, 6)]
{noformat}
The expected result is that a/file.txt and b/file.txt should be matched for both patterns.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)