[jira] [Created] (BEAM-5151) Add EXTERNAL to CREATE TABLE statement

["Rui Wang (JIRA)" <ji...@apache.org>]

Rui Wang created BEAM-5151:
------------------------------

             Summary: Add EXTERNAL to CREATE TABLE statement
                 Key: BEAM-5151
                 URL: https://issues.apache.org/jira/browse/BEAM-5151
             Project: Beam
          Issue Type: Improvement
          Components: dsl-sql
            Reporter: Rui Wang
            Assignee: Rui Wang


BeamSQL allows [CREATE TABLE|https://beam.apache.org/documentation/dsls/sql/create-table/] statements to register virtual tables from external storage systems (e.g. BigQuery). 
 
BeamSQL is not a storage system, so any table registered by "CREATE TABLE" statement is essentially equivalent to be registered by "CREATE EXTERNAL TABLE", which requires the user to provide a LOCATION and BeamSQL will register the table outside of current execution environment based on LOCATION.
 
So I propose to add EXTERNAL keyword to "CREATE TABLE" in BeamSQL to help users understand they are registering tables, and BeamSQL does not create non existing tables by running CREATE TABLE (at least on some storage systems, if not all). 
 
We can make the EXTERNAL keyword either required or optional.
 
If we make the EXTERNAL keyword required:
 
Pros:
a. We can get rid of the registering table semantic on CREATE TABLE. 
b, We keep the room that we could add CREATE TABLE back in the future if we want CREATE TABLE to create, rather than not only register tables in BeamSQL. 
 
Cons:
1. CREATE TABLE syntax will not be supported so existing BeamSQL pipelines which has CREATE TABLE require changes.
2. It's required to type tedious EXTERNAL keyword every time, especially in SQL Shell.
 
If we make the EXTERNAL keyword optional, we will have reversed pros and cons above.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)