[jira] [Created] (BEAM-9776) Beam SQL shell doesn't support creating an external table pointing to GCS, i.e., "gs://..."

["David Sabater (Jira)" <ji...@apache.org>]

David Sabater created BEAM-9776:
-----------------------------------

             Summary: Beam SQL shell doesn't support creating an external table pointing to GCS, i.e., "gs://..."
                 Key: BEAM-9776
                 URL: https://issues.apache.org/jira/browse/BEAM-9776
             Project: Beam
          Issue Type: Bug
          Components: dsl-sql
            Reporter: David Sabater
            Assignee: David Sabater


We are getting the below exception when creating an [external table|[https://beam.apache.org/documentation/dsls/sql/extensions/create-external-table/#text]] pointing to GCS from the [Beam SQL shell|[https://beam.apache.org/documentation/dsls/sql/shell/]]:

Caused by: java.lang.IllegalArgumentException: No filesystem found for scheme gs

It appears the interface [FileSystemRegistrar|[https://beam.apache.org/releases/javadoc/2.19.0/index.html?org/apache/beam/sdk/io/FileSystemRegistrar.html]] is not called before Calcite checks the contents of the file from the method:

org.apache.beam.sdk.extensions.sql.impl.BeamCalciteTable.getStatistic

Hence getting the exception as the GCS filesystem registrar is not initialised as part of the pipeline creation.

We should change the logic to create the pipeline and being able to register GCS before making these calls to validate the SQL statements.

 

Kind regards.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)