[GitHub] [arrow] emkornfield commented on a change in pull request #9368: [WIP] [POC] Flight SQL

[GitBox <gi...@apache.org>]

emkornfield commented on a change in pull request #9368:
URL: https://github.com/apache/arrow/pull/9368#discussion_r635405341



##########
File path: format/FlightSQL.proto
##########
@@ -0,0 +1,337 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ * <p>
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * <p>
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+syntax = "proto3";
+
+option java_package = "org.apache.arrow.flight.sql.impl";
+package arrow.flight.protocol.sql;
+
+/*
+ * Wrapper for the result of a "GetSQLInfo" action.
+ */
+message ActionGetSQLInfoResult {
+  /*
+   * Values are modelled after ODBC's SQLGetInfo() function. This information is intended to provide
+   * Flight SQL clients with basic, SQL syntax and SQL functions related information.
+   * More information types can be added in future releases.
+   * E.g. more SQL syntax support types, scalar functions support, type conversion support etc.
+   *
+   * Initially, Flight SQL will support the following information types:
+   * - Server Information
+   * -
+   *
+   * 1. Server Information: Provides basic information about the Flight SQL Server.
+   *
+   * The name of the Flight SQL Server.
+   * FLIGHT_SQL_SERVER_NAME
+   *
+   * The native version of the Flight SQL Server.
+   * FLIGHT_SQL_SERVER_VERSION
+   *
+   * The Arrow version of the Flight SQL Server.
+   * FLIGHT_SQL_SERVER_ARROW_VERSION
+   *
+   * Indicates whether the Flight SQL Server is read only.
+   * FLIGHT_SQL_SERVER_READ_ONLY
+   *
+   * Indicates whether the Flight SQL Server supports both read and write.
+   * FLIGHT_SQL_SERVER_READ_WRITE
+   *
+   * 2. SQL Syntax Information: provides information about SQL syntax supported by the Flight SQL Server.
+   *
+   * Indicates whether the Flight SQL Server supports CREATE and DROP of catalogs.
+   * In a SQL environment, a catalog is a collection of schemas.
+   * SQL_DDL_CATALOG
+   *
+   * Indicates whether the Flight SQL Server supports CREATE and DROP of schemas.
+   * In a SQL environment, a catalog is a collection of tables, views, indexes etc.
+   * SQL_DDL_SCHEMA
+   *
+   * Indicates whether the Flight SQL Server supports CREATE and DROP of tables.
+   * In a SQL environment, a table is a collection of rows of information. Each row of information
+   * may have one or more columns of data.
+   * SQL_DDL_TABLE
+   *
+   * Indicates the case sensitivity of catalog, table and schema names.
+   * SQL_IDENTIFIER_CASE
+   *
+   * Indicates the supported character(s) used to surround a delimited identifier.
+   * SQL_IDENTIFIER_QUOTE_CHAR
+   *
+   * Indicates case sensitivity of quoted identifiers.
+   * SQL_QUOTED_IDENTIFIER_CASE
+   *
+   */
+  map<string, GetSQLInfoValue> flight_sql_info = 1;
+
+  // Implementation specific custom SQLInfo values.
+  map<string, GetSQLInfoValue> custom_info = 2;
+}
+
+/*
+ * Wrapper for values returned in ActionGetSQLInfoResult.
+ */
+message GetSQLInfoValue {
+  oneof value {
+    string string_value = 1;
+    int32 integer_value = 2;
+    int32 integer_bitmask = 3;
+    int32 integer_flag = 4;
+    bytes binary_value = 5;
+    int64 long_value = 6;
+  }
+}
+
+/*
+ * Request message for the "GetCatalogs" action on a 
+ * Flight SQL enabled backend. 
+ * Requests a list of catalogs available in the server.
+ */
+message ActionGetCatalogsRequest {
+  /*
+   * Specifies the order of result values.
+   */ 
+  ResultsOrder order = 1;
+}
+
+/*
+ * Wrap the result of a "GetCatalogs" action.
+ */
+message ActionGetCatalogsResult {
+  repeated string catalog = 1;
+}
+
+/*
+ * Request message for the "GetSchemas" action on a 
+ * Flight SQL enabled backend. 
+ * Requests a list of schemas available in the server.
+ */
+message ActionGetSchemasRequest {
+  /*
+   * Specifies the order of result values with precedence:
+   *  - catalog
+   *  - schema
+   */
+  ResultsOrder order = 1;
+
+  /*
+   * Specifies the Catalog to search for schemas.
+   * If omitted, then schemas for all catalogs are searched.
+   */ 
+  string catalog = 2;
+
+  /*
+   * Specifies a filter pattern for schemas to search for.
+   * When no schema_filter_pattern is provided, the pattern will not be used to narrow the search.
+   * In the pattern string, two special characters can be used to denote matching rules:
+   *    - "%" means to match any substring with 0 or more characters.
+   *    - "_" means to match any one character.
+   */
+  string schema_filter_pattern = 3;
+}
+
+/*
+ * Wrap the result of a "GetSchemas" action.
+ */
+message ActionGetSchemasResult {
+  string catalog = 1;
+  string schema = 2;
+}
+
+/*
+ * Request message for the "GetTables" action on a 
+ * Flight SQL enabled backend. 
+ * Requests a list of tables available in the server.
+ */
+message ActionGetTablesRequest {
+  /*
+   * Specifies the order of result values with prescendence:
+   *  - catalog
+   *  - schema
+   *  - table_type
+   *  - table
+   */
+  ResultsOrder order = 1;
+
+  // Specifies the Catalog to search for schemas.
+  string catalog = 2;
+
+  /* Specifies a filter pattern for schemas to search for.
+   * When no schema_filter_pattern is provided, the pattern will not be used to narrow the search.
+   * In the pattern string, two special characters can be used to denote matching rules:
+   *    - "%" means to match any substring with 0 or more characters.
+   *    - "_" means to match any one character.
+   */
+  string schema_filter_pattern = 3;
+
+  /* Specifies a filter pattern for tables to search for.
+   * When no schema_filter_pattern is provided, the pattern will not be used to narrow the search.
+   * In the pattern string, two special characters can be used to denote matching rules:
+   *    - "%" means to match any substring with 0 or more characters.
+   *    - "_" means to match any one character.
+   */
+  string table_name_filter_pattern = 4;
+
+  // Specifies a filter of table types which must match.
+  repeated string table_types = 5;

Review comment:
       > I think just submitting metadata requests via SQL has other challenges from an implementation perspective. It necessitates either having the backend validate parsing of the pseudo table (essentially making it a real table/view) or having the FlightProducer parse the query, identify the pseudo table and its filters, then translating the request to a backend call.
   
   I agree it can be complicated, I think in many cases vendors already support this, and JDBC drivers end up doing this under the hood.  I haven't looked deeply but I was hoping this type of parsing would not be hard with something like Calcite. 




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org