[jira] [Created] (BEAM-2101) (Potential) Reading public GCS files requires authentication

["Ahmet Altay (JIRA)" <ji...@apache.org>]

Ahmet Altay created BEAM-2101:
---------------------------------

             Summary: (Potential) Reading public GCS files requires authentication
                 Key: BEAM-2101
                 URL: https://issues.apache.org/jira/browse/BEAM-2101
             Project: Beam
          Issue Type: Bug
          Components: sdk-py
            Reporter: Ahmet Altay


cc: [~sb2nov]

[~tibor.kiss@gmail.com] reported this on hackathon, while running wordcount. Even though plain gsutil works fine on the public file, wordcount fails. It is possible that some other authentication is required as part of the validation.

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "apache_beam/io/textio.py", line 390, in __init__
    skip_header_lines=skip_header_lines)
  File "apache_beam/io/textio.py", line 89, in __init__
    validate=validate)
  File "apache_beam/io/filebasedsource.py", line 103, in __init__
    self._validate()
  File "apache_beam/utils/value_provider.py", line 101, in _f
    return fnc(self, *args, **kwargs)
  File "apache_beam/io/filebasedsource.py", line 163, in _validate
    match_result = FileSystems.match([pattern], limits=[1])[0]
  File "apache_beam/io/filesystems.py", line 81, in match
    return filesystem.match(patterns, limits)
  File "apache_beam/io/gcp/gcsfilesystem.py", line 100, in match
    raise BeamIOError("Match operation failed", exceptions)
apache_beam.io.filesystem.BeamIOError: Match operation failed with exceptions {'gs://apache-beam-samples/shakespeare/hamlet.txt': ApplicationDefaultCredentialsError('The Application Default Credentials are not available. They are available if running in Google Compute Engine. Otherwise, the environment variable GOOGLE_APPLICATION_CREDENTIALS must be defined pointing to a file defining the credentials. See https://developers.google.com/accounts/docs/application-default-credentials for more information.',)}





--
This message was sent by Atlassian JIRA
(v6.3.15#6346)