You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2021/12/20 17:09:08 UTC

[GitHub] [arrow] jcralmeida opened a new pull request #11999: [JAVA] Add missing metadata on Arrow schemas returned by Flight SQL

jcralmeida opened a new pull request #11999:
URL: https://github.com/apache/arrow/pull/11999


   This add an auxiliary class FlightSqlColumnMetadata meant to read and write known metadata for Arrow schema fields, such as CATALOG_NAME, SCHEMA_NAME, TABLE_NAME, PRECISION, SCALE, IS_AUTO_INCREMENT, IS_CASE_SENSITIVE, IS_READ_ONLY and IS_SEARCHABLE.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [arrow] jcralmeida commented on pull request #11999: ARROW-15314: [C++][Java][FlightRPC] Add missing metadata on Arrow schemas returned by Flight SQL

Posted by GitBox <gi...@apache.org>.
jcralmeida commented on pull request #11999:
URL: https://github.com/apache/arrow/pull/11999#issuecomment-1011449010


   @lidavidm I've added the integration test for both PRs and fix the nits on both PRs 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [arrow] lidavidm closed pull request #11999: ARROW-15314: [C++][Java][FlightRPC] Add missing metadata on Arrow schemas returned by Flight SQL

Posted by GitBox <gi...@apache.org>.
lidavidm closed pull request #11999:
URL: https://github.com/apache/arrow/pull/11999


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [arrow] ursabot edited a comment on pull request #11999: ARROW-15314: [C++][Java][FlightRPC] Add missing metadata on Arrow schemas returned by Flight SQL

Posted by GitBox <gi...@apache.org>.
ursabot edited a comment on pull request #11999:
URL: https://github.com/apache/arrow/pull/11999#issuecomment-1079443371


   Benchmark runs are scheduled for baseline = acc6c2eb8ae580085296f608c1a0e8564269456d and contender = c2fac051f3b43835e33364c2aa6a38d1733f241f. c2fac051f3b43835e33364c2aa6a38d1733f241f is a master commit associated with this PR. Results will be available as each benchmark for each run completes.
   Conbench compare runs links:
   [Scheduled] [ec2-t3-xlarge-us-east-2](https://conbench.ursa.dev/compare/runs/d3a95aeb4c5248ed8dae1f73371f7e28...d588dcd7c8f8439bbc1049df4852c7a8/)
   [Scheduled] [test-mac-arm](https://conbench.ursa.dev/compare/runs/e5f98bce316647ec9ac0ef83f2de4458...5ca4e14a3a08432cbebfd4a4cbdf846d/)
   [Scheduled] [ursa-i9-9960x](https://conbench.ursa.dev/compare/runs/a77d0624011a463990d0eabe541be6df...3e39c657f5d94d949f9eb56be4ed345e/)
   [Finished :arrow_down:0.04% :arrow_up:0.0%] [ursa-thinkcentre-m75q](https://conbench.ursa.dev/compare/runs/5ca388d6438c409094189683c61c0a20...1294a96089f34e5ba22220660c0ea345/)
   Supported benchmarks:
   ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python, R. Runs only benchmarks with cloud = True
   test-mac-arm: Supported benchmark langs: C++, Python, R
   ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript
   ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [arrow] pitrou commented on a change in pull request #11999: ARROW-15314: [C++][Java][FlightRPC] Add missing metadata on Arrow schemas returned by Flight SQL

Posted by GitBox <gi...@apache.org>.
pitrou commented on a change in pull request #11999:
URL: https://github.com/apache/arrow/pull/11999#discussion_r800466177



##########
File path: format/FlightSql.proto
##########
@@ -441,7 +441,7 @@ enum SqlInfo {
    * - return 3   (\b11)    => [SQL_SUBQUERIES_IN_COMPARISONS, SQL_SUBQUERIES_IN_EXISTS];
    * - return 4   (\b100)   => [SQL_SUBQUERIES_IN_INS];
    * - return 5   (\b101)   => [SQL_SUBQUERIES_IN_COMPARISONS, SQL_SUBQUERIES_IN_INS];
-   * - return 6   (\b110)   => [SQL_SUBQUERIES_IN_INS, SQL_SUBQUERIES_IN_EXISTS];
+   * - return 6   (\b110)   => [SQL_SUBQUERIES_IN_COMPARISONS, SQL_SUBQUERIES_IN_EXISTS];

Review comment:
       Is this change valid? Judging by the above, SQL_SUBQUERIES_IN_COMPARISONS is 0b001 while SQL_SUBQUERIES_IN_INS is 0b100.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [arrow] jcralmeida commented on pull request #11999: ARROW-15314: [C++][Java][FlightRPC] Add missing metadata on Arrow schemas returned by Flight SQL

Posted by GitBox <gi...@apache.org>.
jcralmeida commented on pull request #11999:
URL: https://github.com/apache/arrow/pull/11999#issuecomment-1069057269


   Sorry @lidavidm. There were some errors related to the cases of the parameter. I've already fixed them


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [arrow] pitrou commented on a change in pull request #11999: ARROW-15314: [C++][Java][FlightRPC] Add missing metadata on Arrow schemas returned by Flight SQL

Posted by GitBox <gi...@apache.org>.
pitrou commented on a change in pull request #11999:
URL: https://github.com/apache/arrow/pull/11999#discussion_r831393745



##########
File path: format/FlightSql.proto
##########
@@ -933,6 +933,16 @@ message CommandGetDbSchemas {
  *  [optional] table_schema: bytes not null (schema of the table as described in Schema.fbs::Schema,
  *                                           it is serialized as an IPC message.)
  * >
+ * Fields on table_schema may contain the following metadata:
+ *  - ARROW:FLIGHT:SQL:CATALOG_NAME      - Table's catalog name
+ *  - ARROW:FLIGHT:SQL:DB_SCHEMA_NAME    - Database schema name
+ *  - ARROW:FLIGHT:SQL:TABLE_NAME        - Table name

Review comment:
       Is there a reason these three are duplicated from the fields above?




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [arrow] pitrou commented on a change in pull request #11999: ARROW-15314: [C++][Java][FlightRPC] Add missing metadata on Arrow schemas returned by Flight SQL

Posted by GitBox <gi...@apache.org>.
pitrou commented on a change in pull request #11999:
URL: https://github.com/apache/arrow/pull/11999#discussion_r831394204



##########
File path: format/FlightSql.proto
##########
@@ -933,6 +933,16 @@ message CommandGetDbSchemas {
  *  [optional] table_schema: bytes not null (schema of the table as described in Schema.fbs::Schema,
  *                                           it is serialized as an IPC message.)
  * >
+ * Fields on table_schema may contain the following metadata:
+ *  - ARROW:FLIGHT:SQL:CATALOG_NAME      - Table's catalog name
+ *  - ARROW:FLIGHT:SQL:DB_SCHEMA_NAME    - Database schema name
+ *  - ARROW:FLIGHT:SQL:TABLE_NAME        - Table name
+ *  - ARROW:FLIGHT:SQL:PRECISION         - Column precision/size
+ *  - ARROW:FLIGHT:SQL:SCALE             - Column scale/decimal digits if applicable
+ *  - ARROW:FLIGHT:SQL:IS_AUTO_INCREMENT - "1" indicates if the column is auto incremented, "0" otherwise.
+ *  - ARROW:FLIGHT:SQL:IS_CASE_SENSITIVE - "1" indicates if the column is case sensitive, "0" otherwise.
+ *  - ARROW:FLIGHT:SQL:IS_READ_ONLY      - "1" indicates if the column is read only, "0" otherwise.
+ *  - ARROW:FLIGHT:SQL:IS_SEARCHABLE     - "1" indicates if the column is searchable via WHERE clause, "0" otherwise.
  * The returned data should be ordered by catalog_name, db_schema_name, table_name, then table_type, followed by table_schema if requested.

Review comment:
       Ordering by table_schema is just a bytewise lexicographic ordering?




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [arrow] ursabot edited a comment on pull request #11999: ARROW-15314: [C++][Java][FlightRPC] Add missing metadata on Arrow schemas returned by Flight SQL

Posted by GitBox <gi...@apache.org>.
ursabot edited a comment on pull request #11999:
URL: https://github.com/apache/arrow/pull/11999#issuecomment-1079443371


   Benchmark runs are scheduled for baseline = acc6c2eb8ae580085296f608c1a0e8564269456d and contender = c2fac051f3b43835e33364c2aa6a38d1733f241f. c2fac051f3b43835e33364c2aa6a38d1733f241f is a master commit associated with this PR. Results will be available as each benchmark for each run completes.
   Conbench compare runs links:
   [Scheduled] [ec2-t3-xlarge-us-east-2](https://conbench.ursa.dev/compare/runs/d3a95aeb4c5248ed8dae1f73371f7e28...d588dcd7c8f8439bbc1049df4852c7a8/)
   [Finished :arrow_down:0.67% :arrow_up:0.34%] [test-mac-arm](https://conbench.ursa.dev/compare/runs/e5f98bce316647ec9ac0ef83f2de4458...5ca4e14a3a08432cbebfd4a4cbdf846d/)
   [Failed :arrow_down:0.0% :arrow_up:0.0%] [ursa-i9-9960x](https://conbench.ursa.dev/compare/runs/a77d0624011a463990d0eabe541be6df...3e39c657f5d94d949f9eb56be4ed345e/)
   [Finished :arrow_down:0.04% :arrow_up:0.0%] [ursa-thinkcentre-m75q](https://conbench.ursa.dev/compare/runs/5ca388d6438c409094189683c61c0a20...1294a96089f34e5ba22220660c0ea345/)
   Supported benchmarks:
   ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python, R. Runs only benchmarks with cloud = True
   test-mac-arm: Supported benchmark langs: C++, Python, R
   ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript
   ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [arrow] pitrou commented on a change in pull request #11999: ARROW-15314: [C++][Java][FlightRPC] Add missing metadata on Arrow schemas returned by Flight SQL

Posted by GitBox <gi...@apache.org>.
pitrou commented on a change in pull request #11999:
URL: https://github.com/apache/arrow/pull/11999#discussion_r800468926



##########
File path: format/FlightSql.proto
##########
@@ -933,6 +933,16 @@ message CommandGetDbSchemas {
  *  [optional] table_schema: bytes not null (schema of the table as described in Schema.fbs::Schema,
  *                                           it is serialized as an IPC message.)
  * >
+ * Fields on table_schema may contain the following metadata:
+ *  - ARROW:FLIGHT:SQL:CATALOG_NAME      - Table's catalog name
+ *  - ARROW:FLIGHT:SQL:DB_SCHEMA_NAME    - Database schema name
+ *  - ARROW:FLIGHT:SQL:TABLE_NAME        - Table name
+ *  - ARROW:FLIGHT:SQL:PRECISION         - Column precision/size
+ *  - ARROW:FLIGHT:SQL:SCALE             - Column scale/decimal digits

Review comment:
       Why are precision and scale useful? Can they annotation non-decimal columns?




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [arrow] ursabot commented on pull request #11999: ARROW-15314: [C++][Java][FlightRPC] Add missing metadata on Arrow schemas returned by Flight SQL

Posted by GitBox <gi...@apache.org>.
ursabot commented on pull request #11999:
URL: https://github.com/apache/arrow/pull/11999#issuecomment-1079443371


   Benchmark runs are scheduled for baseline = acc6c2eb8ae580085296f608c1a0e8564269456d and contender = c2fac051f3b43835e33364c2aa6a38d1733f241f. c2fac051f3b43835e33364c2aa6a38d1733f241f is a master commit associated with this PR. Results will be available as each benchmark for each run completes.
   Conbench compare runs links:
   [Scheduled] [ec2-t3-xlarge-us-east-2](https://conbench.ursa.dev/compare/runs/d3a95aeb4c5248ed8dae1f73371f7e28...d588dcd7c8f8439bbc1049df4852c7a8/)
   [Scheduled] [test-mac-arm](https://conbench.ursa.dev/compare/runs/e5f98bce316647ec9ac0ef83f2de4458...5ca4e14a3a08432cbebfd4a4cbdf846d/)
   [Scheduled] [ursa-i9-9960x](https://conbench.ursa.dev/compare/runs/a77d0624011a463990d0eabe541be6df...3e39c657f5d94d949f9eb56be4ed345e/)
   [Scheduled] [ursa-thinkcentre-m75q](https://conbench.ursa.dev/compare/runs/5ca388d6438c409094189683c61c0a20...1294a96089f34e5ba22220660c0ea345/)
   Supported benchmarks:
   ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python, R. Runs only benchmarks with cloud = True
   test-mac-arm: Supported benchmark langs: C++, Python, R
   ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript
   ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [arrow] lidavidm commented on a change in pull request #11999: [JAVA] Add missing metadata on Arrow schemas returned by Flight SQL

Posted by GitBox <gi...@apache.org>.
lidavidm commented on a change in pull request #11999:
URL: https://github.com/apache/arrow/pull/11999#discussion_r772618496



##########
File path: java/flight/flight-sql/src/main/java/org/apache/arrow/flight/sql/FlightSqlColumnMetadata.java
##########
@@ -0,0 +1,295 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.arrow.flight.sql;
+
+import java.util.HashMap;
+import java.util.Map;
+
+/**
+ * Metadata for a column in a Flight SQL query.
+ *
+ * This can be used with FlightSqlClient to access column's metadata contained in schemas returned
+ * by GetTables and query execution as follows:
+ * <pre>
+ *   FlightSqlColumnMetadata metadata = new FlightSqlColumnMetadata(field.getMetadata());
+ *   Integer precision = metadata.getPrecision();
+ * </pre>
+ *
+ * FlightSqlProducer can use this to set metadata on a column in a schema as follows:
+ * <pre>
+ *   FlightSqlColumnMetadata metadata = new FlightSqlColumnMetadata.Builder()
+ *         .precision(10)
+ *         .scale(5)
+ *         .build();
+ *   Field field = new Field("column", new FieldType(..., metadata.getMetadataMap()), null);
+ * </pre>
+ */
+public class FlightSqlColumnMetadata {
+
+  private static final String CATALOG_NAME = "CATALOG_NAME";
+  private static final String SCHEMA_NAME = "SCHEMA_NAME";
+  private static final String TABLE_NAME = "TABLE_NAME";
+  private static final String PRECISION = "PRECISION";
+  private static final String SCALE = "SCALE";
+  private static final String IS_AUTO_INCREMENT = "IS_AUTO_INCREMENT";
+  private static final String IS_CASE_SENSITIVE = "IS_CASE_SENSITIVE";
+  private static final String IS_READ_ONLY = "IS_READ_ONLY";
+  private static final String IS_SEARCHABLE = "IS_SEARCHABLE";
+
+  private static final String BOOLEAN_TRUE_STR = "1";
+  private static final String BOOLEAN_FALSE_STR = "0";
+
+  private final Map<String, String> metadataMap;
+
+  /**
+   * Creates a new instance of FlightSqlColumnMetadata.
+   */
+  public FlightSqlColumnMetadata(Map<String, String> metadataMap) {
+    this.metadataMap = metadataMap;

Review comment:
       Mostly to guard against something like 
   
   ```java
   var map = ...;
   var metadata = new FlightSqlColumnMetadata(map);
   map.put(..); // mutate map directly from under the metadata
   ```
   
   that said it's not a big deal, just being defensive here.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [arrow] lidavidm commented on a change in pull request #11999: Add missing metadata on Arrow schemas returned by Flight SQL

Posted by GitBox <gi...@apache.org>.
lidavidm commented on a change in pull request #11999:
URL: https://github.com/apache/arrow/pull/11999#discussion_r781538101



##########
File path: cpp/src/arrow/flight/sql/column_metadata.cc
##########
@@ -0,0 +1,187 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements.  See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership.  The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License.  You may obtain a copy of the License at
+//
+//   http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied.  See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+#include "arrow/flight/sql/column_metadata.h"
+
+#include <utility>
+
+namespace arrow {
+namespace flight {
+namespace sql {
+namespace {
+  /// \brief Constant variable used to convert boolean true value
+  ///        to a string.
+  const char* BOOLEAN_TRUE_STR = "YES";
+  /// \brief Constant variable used to convert boolean false value
+  ///        to a string.
+  const char* BOOLEAN_FALSE_STR = "NO";
+
+  std::string BooleanToString(bool boolean_value) {
+    return boolean_value ? BOOLEAN_TRUE_STR :
+           BOOLEAN_FALSE_STR;
+  }
+
+  bool StringToBoolean(const std::string& string_value) {
+    return string_value == BOOLEAN_TRUE_STR;
+  }
+}  // namespace
+
+const char* ColumnMetadata::kCatalogName = "ARROW:FLIGHT:SQL:CATALOG_NAME";
+const char* ColumnMetadata::kSchemaName = "ARROW:FLIGHT:SQL:SCHEMA_NAME";
+const char* ColumnMetadata::kTableName = "ARROW:FLIGHT:SQL:TABLE_NAME";
+const char* ColumnMetadata::kPrecision = "ARROW:FLIGHT:SQL:PRECISION";
+const char* ColumnMetadata::kScale = "ARROW:FLIGHT:SQL:SCALE";
+const char* ColumnMetadata::kIsAutoIncrement = "ARROW:FLIGHT:SQL:IS_AUTO_INCREMENT";
+const char* ColumnMetadata::kIsCaseSensitive = "ARROW:FLIGHT:SQL:IS_CASE_SENSITIVE";
+const char* ColumnMetadata::kIsReadOnly = "ARROW:FLIGHT:SQL:IS_READ_ONLY";
+const char* ColumnMetadata::kIsSearchable = "ARROW:FLIGHT:SQL:IS_SEARCHABLE";
+
+ColumnMetadata::ColumnMetadata(std::shared_ptr<arrow::KeyValueMetadata> metadata_map) :
+  metadata_map_(std::move(metadata_map)) {
+}
+
+arrow::Result<std::string> ColumnMetadata::GetCatalogName() const {
+  return metadata_map_->Get(kCatalogName);
+}
+
+arrow::Result<std::string> ColumnMetadata::GetSchemaName() const {
+  return metadata_map_->Get(kSchemaName);
+}
+
+arrow::Result<std::string> ColumnMetadata::GetTableName() const {
+  return metadata_map_->Get(kTableName);
+}
+
+arrow::Result<int32_t> ColumnMetadata::GetPrecision() const {
+  const Result <std::string> &result = metadata_map_->Get(kPrecision);
+  std::string precision_string;
+  ARROW_ASSIGN_OR_RAISE(precision_string, result);
+
+  return std::stoi(precision_string);
+}
+
+arrow::Result<int32_t> ColumnMetadata::GetScale() const {
+  std::string scale_string;
+  ARROW_ASSIGN_OR_RAISE(scale_string, metadata_map_->Get(kScale));
+
+  return std::stoi(scale_string);
+}
+
+arrow::Result<bool> ColumnMetadata::GetIsAutoIncrement() const {
+  std::string auto_increment_string;
+  ARROW_ASSIGN_OR_RAISE(auto_increment_string, metadata_map_->Get(kIsAutoIncrement));
+  return StringToBoolean(auto_increment_string);
+}
+
+arrow::Result<bool> ColumnMetadata::GetIsCaseSensitive() const {
+  std::string is_case_sensitive;
+  ARROW_ASSIGN_OR_RAISE(is_case_sensitive, metadata_map_->Get(kIsAutoIncrement));
+  return StringToBoolean(is_case_sensitive);
+}
+
+arrow::Result<bool> ColumnMetadata::GetIsReadOnly() const {
+  std::string is_read_only;
+  ARROW_ASSIGN_OR_RAISE(is_read_only, metadata_map_->Get(kIsAutoIncrement));
+  return StringToBoolean(is_read_only);
+}
+
+arrow::Result<bool> ColumnMetadata::GetIsSearchable() const {
+  std::string is_case_sensitive;
+  ARROW_ASSIGN_OR_RAISE(is_case_sensitive, metadata_map_->Get(kIsAutoIncrement));
+  return StringToBoolean(is_case_sensitive);
+}
+
+ColumnMetadata::ColumnMetadataBuilder ColumnMetadata::Builder() {
+  const ColumnMetadataBuilder &builder = ColumnMetadataBuilder{};
+  return builder;

Review comment:
       nit, but this could just be `return {};` I think (or `return ColumnMetadataBuilder()`).

##########
File path: format/FlightSql.proto
##########
@@ -933,6 +933,16 @@ message CommandGetDbSchemas {
  *  [optional] table_schema: bytes not null (schema of the table as described in Schema.fbs::Schema,
  *                                           it is serialized as an IPC message.)
  * >
+ * Fields on table_schema may contain the following metadata:
+ *  - ARROW:FLIGHT:SQL:CATALOG_NAME      - Table's catalog name
+ *  - ARROW:FLIGHT:SQL:DB_SCHEMA_NAME    - Database schema name
+ *  - ARROW:FLIGHT:SQL:TABLE_NAME        - Table name
+ *  - ARROW:FLIGHT:SQL:PRECISION         - Column precision/size
+ *  - ARROW:FLIGHT:SQL:SCALE             - Column scale/decimal digits
+ *  - ARROW:FLIGHT:SQL:IS_AUTO_INCREMENT - "1" indicates if the column is auto incremented, "0" otherwise.

Review comment:
       The source seems to use "YES" and "NO" instead of "1" and "0".

##########
File path: cpp/src/arrow/flight/sql/column_metadata.h
##########
@@ -0,0 +1,168 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements.  See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership.  The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License.  You may obtain a copy of the License at
+//
+//   http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied.  See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+#pragma once
+
+#include <string>
+
+#include "arrow/util/key_value_metadata.h"
+
+namespace arrow {
+namespace flight {
+namespace sql {
+
+/// \brief Helper class to set column metadata.
+class ColumnMetadata {
+ private:
+  std::shared_ptr<arrow::KeyValueMetadata> metadata_map_;
+  explicit ColumnMetadata(std::shared_ptr<arrow::KeyValueMetadata> metadata_map);
+
+ public:
+  class ColumnMetadataBuilder;
+
+  /// \brief Constant variable to hold the value of the key that
+  ///        will be used in the KeyValueMetadata class.
+  static const char* kCatalogName;
+  /// \brief Constant variable to hold the value of the key that
+  ///        will be used in the KeyValueMetadata class.
+  static const char* kSchemaName;
+  /// \brief Constant variable to hold the value of the key that
+  ///        will be used in the KeyValueMetadata class.
+  static const char* kTableName;
+  /// \brief Constant variable to hold the value of the key that
+  ///        will be used in the KeyValueMetadata class.
+  static const char* kPrecision;
+  /// \brief Constant variable to hold the value of the key that
+  ///        will be used in the KeyValueMetadata class.
+  static const char* kScale;
+  /// \brief Constant variable to hold the value of the key that
+  ///        will be used in the KeyValueMetadata class.
+  static const char* kIsAutoIncrement;
+  /// \brief Constant variable to hold the value of the key that
+  ///        will be used in the KeyValueMetadata class.
+  static const char* kIsCaseSensitive;
+  /// \brief Constant variable to hold the value of the key that
+  ///        will be used in the KeyValueMetadata class.
+  static const char* kIsReadOnly;
+  /// \brief Constant variable to hold the value of the key that
+  ///        will be used in the KeyValueMetadata class.
+  static const char* kIsSearchable;
+
+  /// \brief Static initializer.
+  static ColumnMetadataBuilder Builder();
+
+  /// \brief  Return the catalog name set in the KeyValueMetadata.
+  /// \return The catalog name.
+  arrow::Result<std::string> GetCatalogName() const;
+
+  /// \brief  Return the schema name set in the KeyValueMetadata.
+  /// \return The schema name.
+  arrow::Result<std::string> GetSchemaName() const;
+
+  /// \brief  Return the table name set in the KeyValueMetadata.
+  /// \return The table name.
+  arrow::Result<std::string> GetTableName() const;
+
+  /// \brief  Return the precision set in the KeyValueMetadata.
+  /// \return The precision.
+  arrow::Result<int32_t> GetPrecision() const;
+
+  /// \brief  Return the scale set in the KeyValueMetadata.
+  /// \return The scale.
+  arrow::Result<int32_t> GetScale() const;
+
+  /// \brief  Return the IsAutoIncrement set in the KeyValueMetadata.
+  /// \return The IsAutoIncrement.
+  arrow::Result<bool> GetIsAutoIncrement() const;
+
+  /// \brief  Return the IsCaseSensitive set in the KeyValueMetadata.
+  /// \return The IsCaseSensitive.
+  arrow::Result<bool> GetIsCaseSensitive() const;
+
+  /// \brief  Return the IsReadOnly set in the KeyValueMetadata.
+  /// \return The IsReadOnly.
+  arrow::Result<bool> GetIsReadOnly() const;
+
+  /// \brief  Return the IsSearchable set in the KeyValueMetadata.
+  /// \return The IsSearchable.
+  arrow::Result<bool> GetIsSearchable() const;
+
+  /// \brief  Return the KeyValueMetadata.
+  /// \return The KeyValueMetadata.
+  std::shared_ptr<arrow::KeyValueMetadata> GetMetadataMap() const;

Review comment:
       Hmm, this could be `const std::shared_ptr<>& metadata_map(); const` to save a shared_ptr copy & fit the naming convention better (pure getters are allowed to be snake_case)

##########
File path: cpp/src/arrow/flight/sql/column_metadata.h
##########
@@ -0,0 +1,168 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements.  See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership.  The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License.  You may obtain a copy of the License at
+//
+//   http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied.  See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+#pragma once
+
+#include <string>
+
+#include "arrow/util/key_value_metadata.h"
+
+namespace arrow {
+namespace flight {
+namespace sql {
+
+/// \brief Helper class to set column metadata.
+class ColumnMetadata {
+ private:
+  std::shared_ptr<arrow::KeyValueMetadata> metadata_map_;
+  explicit ColumnMetadata(std::shared_ptr<arrow::KeyValueMetadata> metadata_map);
+
+ public:
+  class ColumnMetadataBuilder;
+
+  /// \brief Constant variable to hold the value of the key that
+  ///        will be used in the KeyValueMetadata class.
+  static const char* kCatalogName;
+  /// \brief Constant variable to hold the value of the key that
+  ///        will be used in the KeyValueMetadata class.
+  static const char* kSchemaName;
+  /// \brief Constant variable to hold the value of the key that
+  ///        will be used in the KeyValueMetadata class.
+  static const char* kTableName;
+  /// \brief Constant variable to hold the value of the key that
+  ///        will be used in the KeyValueMetadata class.
+  static const char* kPrecision;
+  /// \brief Constant variable to hold the value of the key that
+  ///        will be used in the KeyValueMetadata class.
+  static const char* kScale;
+  /// \brief Constant variable to hold the value of the key that
+  ///        will be used in the KeyValueMetadata class.
+  static const char* kIsAutoIncrement;
+  /// \brief Constant variable to hold the value of the key that
+  ///        will be used in the KeyValueMetadata class.
+  static const char* kIsCaseSensitive;
+  /// \brief Constant variable to hold the value of the key that
+  ///        will be used in the KeyValueMetadata class.
+  static const char* kIsReadOnly;
+  /// \brief Constant variable to hold the value of the key that
+  ///        will be used in the KeyValueMetadata class.
+  static const char* kIsSearchable;
+
+  /// \brief Static initializer.
+  static ColumnMetadataBuilder Builder();
+
+  /// \brief  Return the catalog name set in the KeyValueMetadata.
+  /// \return The catalog name.
+  arrow::Result<std::string> GetCatalogName() const;
+
+  /// \brief  Return the schema name set in the KeyValueMetadata.
+  /// \return The schema name.
+  arrow::Result<std::string> GetSchemaName() const;
+
+  /// \brief  Return the table name set in the KeyValueMetadata.
+  /// \return The table name.
+  arrow::Result<std::string> GetTableName() const;
+
+  /// \brief  Return the precision set in the KeyValueMetadata.
+  /// \return The precision.
+  arrow::Result<int32_t> GetPrecision() const;
+
+  /// \brief  Return the scale set in the KeyValueMetadata.
+  /// \return The scale.
+  arrow::Result<int32_t> GetScale() const;
+
+  /// \brief  Return the IsAutoIncrement set in the KeyValueMetadata.
+  /// \return The IsAutoIncrement.
+  arrow::Result<bool> GetIsAutoIncrement() const;
+
+  /// \brief  Return the IsCaseSensitive set in the KeyValueMetadata.
+  /// \return The IsCaseSensitive.
+  arrow::Result<bool> GetIsCaseSensitive() const;
+
+  /// \brief  Return the IsReadOnly set in the KeyValueMetadata.
+  /// \return The IsReadOnly.
+  arrow::Result<bool> GetIsReadOnly() const;
+
+  /// \brief  Return the IsSearchable set in the KeyValueMetadata.
+  /// \return The IsSearchable.
+  arrow::Result<bool> GetIsSearchable() const;
+
+  /// \brief  Return the KeyValueMetadata.
+  /// \return The KeyValueMetadata.
+  std::shared_ptr<arrow::KeyValueMetadata> GetMetadataMap() const;
+
+  /// \brief A builder class to construct the ColumnMetadata object.
+  class ColumnMetadataBuilder {
+    std::shared_ptr<arrow::KeyValueMetadata> metadata_map_;

Review comment:
       nit: for consistency, can we explicitly declare this `private:` and put this below the `public` declarations?




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [arrow] jcralmeida commented on pull request #11999: Add missing metadata on Arrow schemas returned by Flight SQL

Posted by GitBox <gi...@apache.org>.
jcralmeida commented on pull request #11999:
URL: https://github.com/apache/arrow/pull/11999#issuecomment-1010395915


   Sure @lidavidm, let's combine both implementation and make one vote.
   
   By the way we don't have the Jira Ticket for this, where can I create the ticket?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [arrow] jcralmeida commented on a change in pull request #11999: [JAVA] Add missing metadata on Arrow schemas returned by Flight SQL

Posted by GitBox <gi...@apache.org>.
jcralmeida commented on a change in pull request #11999:
URL: https://github.com/apache/arrow/pull/11999#discussion_r772618934



##########
File path: format/FlightSql.proto
##########
@@ -55,7 +55,7 @@ message CommandGetSqlInfo {
    * Initially, Flight SQL will support the following information types:
    * - Server Information - Range [0-500)
    * - Syntax Information - Range [500-1000)
-   * Range [0-10,000) is reserved for defaults (see SqlInfo enum for default options). 
+   * Range [0-10,000] is reserved for defaults (see SqlInfo enum for default options).

Review comment:
       Yeah, this was an error. Reverting this change




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [arrow] jcralmeida commented on pull request #11999: [JAVA] Add missing metadata on Arrow schemas returned by Flight SQL

Posted by GitBox <gi...@apache.org>.
jcralmeida commented on pull request #11999:
URL: https://github.com/apache/arrow/pull/11999#issuecomment-1000203951


   No problem. I've already rebased both PRs.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [arrow] lidavidm commented on a change in pull request #11999: ARROW-15314: [C++][Java][FlightRPC] Add missing metadata on Arrow schemas returned by Flight SQL

Posted by GitBox <gi...@apache.org>.
lidavidm commented on a change in pull request #11999:
URL: https://github.com/apache/arrow/pull/11999#discussion_r820776537



##########
File path: format/FlightSql.proto
##########
@@ -933,6 +933,16 @@ message CommandGetDbSchemas {
  *  [optional] table_schema: bytes not null (schema of the table as described in Schema.fbs::Schema,
  *                                           it is serialized as an IPC message.)
  * >
+ * Fields on table_schema may contain the following metadata:
+ *  - ARROW:FLIGHT:SQL:CATALOG_NAME      - Table's catalog name
+ *  - ARROW:FLIGHT:SQL:DB_SCHEMA_NAME    - Database schema name
+ *  - ARROW:FLIGHT:SQL:TABLE_NAME        - Table name
+ *  - ARROW:FLIGHT:SQL:PRECISION         - Column precision/size
+ *  - ARROW:FLIGHT:SQL:SCALE             - Column scale/decimal digits

Review comment:
       Should we annotate this as `(if applicable for the column type)` or something?

##########
File path: format/FlightSql.proto
##########
@@ -441,7 +441,7 @@ enum SqlInfo {
    * - return 3   (\b11)    => [SQL_SUBQUERIES_IN_COMPARISONS, SQL_SUBQUERIES_IN_EXISTS];
    * - return 4   (\b100)   => [SQL_SUBQUERIES_IN_INS];
    * - return 5   (\b101)   => [SQL_SUBQUERIES_IN_COMPARISONS, SQL_SUBQUERIES_IN_INS];
-   * - return 6   (\b110)   => [SQL_SUBQUERIES_IN_INS, SQL_SUBQUERIES_IN_EXISTS];
+   * - return 6   (\b110)   => [SQL_SUBQUERIES_IN_COMPARISONS, SQL_SUBQUERIES_IN_EXISTS];

Review comment:
       Yes, this seems wrong. Additionally the enum values are sequential (0, 1, 2, 3) not bitmasks (unless those values are intended to be bit indices in which case we should document them as such).




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [arrow] jcralmeida commented on a change in pull request #11999: ARROW-15314: [C++][Java][FlightRPC] Add missing metadata on Arrow schemas returned by Flight SQL

Posted by GitBox <gi...@apache.org>.
jcralmeida commented on a change in pull request #11999:
URL: https://github.com/apache/arrow/pull/11999#discussion_r827188881



##########
File path: format/FlightSql.proto
##########
@@ -933,6 +933,16 @@ message CommandGetDbSchemas {
  *  [optional] table_schema: bytes not null (schema of the table as described in Schema.fbs::Schema,
  *                                           it is serialized as an IPC message.)
  * >
+ * Fields on table_schema may contain the following metadata:
+ *  - ARROW:FLIGHT:SQL:CATALOG_NAME      - Table's catalog name
+ *  - ARROW:FLIGHT:SQL:DB_SCHEMA_NAME    - Database schema name
+ *  - ARROW:FLIGHT:SQL:TABLE_NAME        - Table name
+ *  - ARROW:FLIGHT:SQL:PRECISION         - Column precision/size
+ *  - ARROW:FLIGHT:SQL:SCALE             - Column scale/decimal digits

Review comment:
       It makes senses. Added




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [arrow] jcralmeida commented on a change in pull request #11999: ARROW-15314: [C++][Java][FlightRPC] Add missing metadata on Arrow schemas returned by Flight SQL

Posted by GitBox <gi...@apache.org>.
jcralmeida commented on a change in pull request #11999:
URL: https://github.com/apache/arrow/pull/11999#discussion_r801982563



##########
File path: format/FlightSql.proto
##########
@@ -933,6 +933,16 @@ message CommandGetDbSchemas {
  *  [optional] table_schema: bytes not null (schema of the table as described in Schema.fbs::Schema,
  *                                           it is serialized as an IPC message.)
  * >
+ * Fields on table_schema may contain the following metadata:
+ *  - ARROW:FLIGHT:SQL:CATALOG_NAME      - Table's catalog name
+ *  - ARROW:FLIGHT:SQL:DB_SCHEMA_NAME    - Database schema name
+ *  - ARROW:FLIGHT:SQL:TABLE_NAME        - Table name
+ *  - ARROW:FLIGHT:SQL:PRECISION         - Column precision/size
+ *  - ARROW:FLIGHT:SQL:SCALE             - Column scale/decimal digits

Review comment:
       This is also requested by the JDBC and ODBC, to be extracted by them via ResultSetMetadata




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [arrow] lidavidm commented on a change in pull request #11999: Add missing metadata on Arrow schemas returned by Flight SQL

Posted by GitBox <gi...@apache.org>.
lidavidm commented on a change in pull request #11999:
URL: https://github.com/apache/arrow/pull/11999#discussion_r782306907



##########
File path: cpp/src/arrow/flight/sql/column_metadata.cc
##########
@@ -0,0 +1,186 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements.  See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership.  The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License.  You may obtain a copy of the License at
+//
+//   http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied.  See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+#include "arrow/flight/sql/column_metadata.h"
+
+#include <utility>
+
+namespace arrow {
+namespace flight {
+namespace sql {
+namespace {
+  /// \brief Constant variable used to convert boolean true value
+  ///        to a string.
+  const char* BOOLEAN_TRUE_STR = "1";
+  /// \brief Constant variable used to convert boolean false value
+  ///        to a string.
+  const char* BOOLEAN_FALSE_STR = "0";
+
+  std::string BooleanToString(bool boolean_value) {
+    return boolean_value ? BOOLEAN_TRUE_STR :
+           BOOLEAN_FALSE_STR;
+  }
+
+  bool StringToBoolean(const std::string& string_value) {
+    return string_value == BOOLEAN_TRUE_STR;
+  }
+}  // namespace
+
+const char* ColumnMetadata::kCatalogName = "ARROW:FLIGHT:SQL:CATALOG_NAME";
+const char* ColumnMetadata::kSchemaName = "ARROW:FLIGHT:SQL:SCHEMA_NAME";
+const char* ColumnMetadata::kTableName = "ARROW:FLIGHT:SQL:TABLE_NAME";
+const char* ColumnMetadata::kPrecision = "ARROW:FLIGHT:SQL:PRECISION";
+const char* ColumnMetadata::kScale = "ARROW:FLIGHT:SQL:SCALE";
+const char* ColumnMetadata::kIsAutoIncrement = "ARROW:FLIGHT:SQL:IS_AUTO_INCREMENT";
+const char* ColumnMetadata::kIsCaseSensitive = "ARROW:FLIGHT:SQL:IS_CASE_SENSITIVE";
+const char* ColumnMetadata::kIsReadOnly = "ARROW:FLIGHT:SQL:IS_READ_ONLY";
+const char* ColumnMetadata::kIsSearchable = "ARROW:FLIGHT:SQL:IS_SEARCHABLE";
+
+ColumnMetadata::ColumnMetadata(std::shared_ptr<arrow::KeyValueMetadata> metadata_map) :
+  metadata_map_(std::move(metadata_map)) {
+}
+
+arrow::Result<std::string> ColumnMetadata::GetCatalogName() const {
+  return metadata_map_->Get(kCatalogName);
+}
+
+arrow::Result<std::string> ColumnMetadata::GetSchemaName() const {
+  return metadata_map_->Get(kSchemaName);
+}
+
+arrow::Result<std::string> ColumnMetadata::GetTableName() const {
+  return metadata_map_->Get(kTableName);
+}
+
+arrow::Result<int32_t> ColumnMetadata::GetPrecision() const {
+  const Result <std::string> &result = metadata_map_->Get(kPrecision);

Review comment:
       This needs to be `arrow::Result` 
   
   Also, why the reference? It can just be:
   
   ```
   ARROW_ASSIGN_OR_RAISE(auto precision_string, metadata_map_->Get(kPrecision))
   ```

##########
File path: cpp/src/arrow/flight/sql/column_metadata.cc
##########
@@ -0,0 +1,186 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements.  See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership.  The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License.  You may obtain a copy of the License at
+//
+//   http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied.  See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+#include "arrow/flight/sql/column_metadata.h"
+
+#include <utility>
+
+namespace arrow {
+namespace flight {
+namespace sql {
+namespace {
+  /// \brief Constant variable used to convert boolean true value
+  ///        to a string.
+  const char* BOOLEAN_TRUE_STR = "1";
+  /// \brief Constant variable used to convert boolean false value
+  ///        to a string.
+  const char* BOOLEAN_FALSE_STR = "0";
+
+  std::string BooleanToString(bool boolean_value) {
+    return boolean_value ? BOOLEAN_TRUE_STR :
+           BOOLEAN_FALSE_STR;
+  }
+
+  bool StringToBoolean(const std::string& string_value) {
+    return string_value == BOOLEAN_TRUE_STR;
+  }
+}  // namespace
+
+const char* ColumnMetadata::kCatalogName = "ARROW:FLIGHT:SQL:CATALOG_NAME";
+const char* ColumnMetadata::kSchemaName = "ARROW:FLIGHT:SQL:SCHEMA_NAME";
+const char* ColumnMetadata::kTableName = "ARROW:FLIGHT:SQL:TABLE_NAME";
+const char* ColumnMetadata::kPrecision = "ARROW:FLIGHT:SQL:PRECISION";
+const char* ColumnMetadata::kScale = "ARROW:FLIGHT:SQL:SCALE";
+const char* ColumnMetadata::kIsAutoIncrement = "ARROW:FLIGHT:SQL:IS_AUTO_INCREMENT";
+const char* ColumnMetadata::kIsCaseSensitive = "ARROW:FLIGHT:SQL:IS_CASE_SENSITIVE";
+const char* ColumnMetadata::kIsReadOnly = "ARROW:FLIGHT:SQL:IS_READ_ONLY";
+const char* ColumnMetadata::kIsSearchable = "ARROW:FLIGHT:SQL:IS_SEARCHABLE";
+
+ColumnMetadata::ColumnMetadata(std::shared_ptr<arrow::KeyValueMetadata> metadata_map) :
+  metadata_map_(std::move(metadata_map)) {
+}
+
+arrow::Result<std::string> ColumnMetadata::GetCatalogName() const {
+  return metadata_map_->Get(kCatalogName);
+}
+
+arrow::Result<std::string> ColumnMetadata::GetSchemaName() const {
+  return metadata_map_->Get(kSchemaName);
+}
+
+arrow::Result<std::string> ColumnMetadata::GetTableName() const {
+  return metadata_map_->Get(kTableName);
+}
+
+arrow::Result<int32_t> ColumnMetadata::GetPrecision() const {
+  const Result <std::string> &result = metadata_map_->Get(kPrecision);

Review comment:
       BTW, these files need to be autoformatted.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [arrow] jcralmeida commented on a change in pull request #11999: [JAVA] Add missing metadata on Arrow schemas returned by Flight SQL

Posted by GitBox <gi...@apache.org>.
jcralmeida commented on a change in pull request #11999:
URL: https://github.com/apache/arrow/pull/11999#discussion_r772615930



##########
File path: java/flight/flight-sql/src/main/java/org/apache/arrow/flight/sql/FlightSqlColumnMetadata.java
##########
@@ -0,0 +1,295 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.arrow.flight.sql;
+
+import java.util.HashMap;
+import java.util.Map;
+
+/**
+ * Metadata for a column in a Flight SQL query.
+ *
+ * This can be used with FlightSqlClient to access column's metadata contained in schemas returned
+ * by GetTables and query execution as follows:
+ * <pre>
+ *   FlightSqlColumnMetadata metadata = new FlightSqlColumnMetadata(field.getMetadata());
+ *   Integer precision = metadata.getPrecision();
+ * </pre>
+ *
+ * FlightSqlProducer can use this to set metadata on a column in a schema as follows:
+ * <pre>
+ *   FlightSqlColumnMetadata metadata = new FlightSqlColumnMetadata.Builder()
+ *         .precision(10)
+ *         .scale(5)
+ *         .build();
+ *   Field field = new Field("column", new FieldType(..., metadata.getMetadataMap()), null);
+ * </pre>
+ */
+public class FlightSqlColumnMetadata {
+
+  private static final String CATALOG_NAME = "CATALOG_NAME";
+  private static final String SCHEMA_NAME = "SCHEMA_NAME";
+  private static final String TABLE_NAME = "TABLE_NAME";
+  private static final String PRECISION = "PRECISION";
+  private static final String SCALE = "SCALE";
+  private static final String IS_AUTO_INCREMENT = "IS_AUTO_INCREMENT";
+  private static final String IS_CASE_SENSITIVE = "IS_CASE_SENSITIVE";
+  private static final String IS_READ_ONLY = "IS_READ_ONLY";
+  private static final String IS_SEARCHABLE = "IS_SEARCHABLE";
+
+  private static final String BOOLEAN_TRUE_STR = "1";
+  private static final String BOOLEAN_FALSE_STR = "0";
+
+  private final Map<String, String> metadataMap;
+
+  /**
+   * Creates a new instance of FlightSqlColumnMetadata.
+   */
+  public FlightSqlColumnMetadata(Map<String, String> metadataMap) {
+    this.metadataMap = metadataMap;

Review comment:
       Is it necessary? The ColumnMetadata doesn't have setter, it is only constructor through the builder. 




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [arrow] jcralmeida commented on a change in pull request #11999: [JAVA] Add missing metadata on Arrow schemas returned by Flight SQL

Posted by GitBox <gi...@apache.org>.
jcralmeida commented on a change in pull request #11999:
URL: https://github.com/apache/arrow/pull/11999#discussion_r772641971



##########
File path: java/flight/flight-sql/src/main/java/org/apache/arrow/flight/sql/FlightSqlColumnMetadata.java
##########
@@ -0,0 +1,295 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.arrow.flight.sql;
+
+import java.util.HashMap;
+import java.util.Map;
+
+/**
+ * Metadata for a column in a Flight SQL query.
+ *
+ * This can be used with FlightSqlClient to access column's metadata contained in schemas returned
+ * by GetTables and query execution as follows:
+ * <pre>
+ *   FlightSqlColumnMetadata metadata = new FlightSqlColumnMetadata(field.getMetadata());
+ *   Integer precision = metadata.getPrecision();
+ * </pre>
+ *
+ * FlightSqlProducer can use this to set metadata on a column in a schema as follows:
+ * <pre>
+ *   FlightSqlColumnMetadata metadata = new FlightSqlColumnMetadata.Builder()
+ *         .precision(10)
+ *         .scale(5)
+ *         .build();
+ *   Field field = new Field("column", new FieldType(..., metadata.getMetadataMap()), null);
+ * </pre>
+ */
+public class FlightSqlColumnMetadata {
+
+  private static final String CATALOG_NAME = "CATALOG_NAME";
+  private static final String SCHEMA_NAME = "SCHEMA_NAME";
+  private static final String TABLE_NAME = "TABLE_NAME";
+  private static final String PRECISION = "PRECISION";
+  private static final String SCALE = "SCALE";
+  private static final String IS_AUTO_INCREMENT = "IS_AUTO_INCREMENT";
+  private static final String IS_CASE_SENSITIVE = "IS_CASE_SENSITIVE";
+  private static final String IS_READ_ONLY = "IS_READ_ONLY";
+  private static final String IS_SEARCHABLE = "IS_SEARCHABLE";
+
+  private static final String BOOLEAN_TRUE_STR = "1";
+  private static final String BOOLEAN_FALSE_STR = "0";
+
+  private final Map<String, String> metadataMap;
+
+  /**
+   * Creates a new instance of FlightSqlColumnMetadata.
+   */
+  public FlightSqlColumnMetadata(Map<String, String> metadataMap) {
+    this.metadataMap = metadataMap;

Review comment:
       It makes totally what you are saying. I haven't seen this possibility




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [arrow] github-actions[bot] commented on pull request #11999: [JAVA] Add missing metadata on Arrow schemas returned by Flight SQL

Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #11999:
URL: https://github.com/apache/arrow/pull/11999#issuecomment-998114860


   <!--
     Licensed to the Apache Software Foundation (ASF) under one
     or more contributor license agreements.  See the NOTICE file
     distributed with this work for additional information
     regarding copyright ownership.  The ASF licenses this file
     to you under the Apache License, Version 2.0 (the
     "License"); you may not use this file except in compliance
     with the License.  You may obtain a copy of the License at
   
       http://www.apache.org/licenses/LICENSE-2.0
   
     Unless required by applicable law or agreed to in writing,
     software distributed under the License is distributed on an
     "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
     KIND, either express or implied.  See the License for the
     specific language governing permissions and limitations
     under the License.
   -->
   
   Thanks for opening a pull request!
   
   If this is not a [minor PR](https://github.com/apache/arrow/blob/master/CONTRIBUTING.md#Minor-Fixes). Could you open an issue for this pull request on JIRA? https://issues.apache.org/jira/browse/ARROW
   
   Opening JIRAs ahead of time contributes to the [Openness](http://theapacheway.com/open/#:~:text=Openness%20allows%20new%20users%20the,must%20happen%20in%20the%20open.) of the Apache Arrow project.
   
   Then could you also rename pull request title in the following format?
   
       ARROW-${JIRA_ID}: [${COMPONENT}] ${SUMMARY}
   
   or
   
       MINOR: [${COMPONENT}] ${SUMMARY}
   
   See also:
   
     * [Other pull requests](https://github.com/apache/arrow/pulls/)
     * [Contribution Guidelines - How to contribute patches](https://arrow.apache.org/docs/developers/contributing.html#how-to-contribute-patches)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [arrow] jcralmeida commented on pull request #11999: [JAVA] Add missing metadata on Arrow schemas returned by Flight SQL

Posted by GitBox <gi...@apache.org>.
jcralmeida commented on pull request #11999:
URL: https://github.com/apache/arrow/pull/11999#issuecomment-998241020


   > Is the metadata here sufficient to cover at least JDBC and ODBC?
   Yes, we thought of this values thinking on both JDBC and ODBC.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [arrow] jcralmeida commented on a change in pull request #11999: Add missing metadata on Arrow schemas returned by Flight SQL

Posted by GitBox <gi...@apache.org>.
jcralmeida commented on a change in pull request #11999:
URL: https://github.com/apache/arrow/pull/11999#discussion_r782374568



##########
File path: cpp/src/arrow/flight/sql/column_metadata.cc
##########
@@ -0,0 +1,186 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements.  See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership.  The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License.  You may obtain a copy of the License at
+//
+//   http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied.  See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+#include "arrow/flight/sql/column_metadata.h"
+
+#include <utility>
+
+namespace arrow {
+namespace flight {
+namespace sql {
+namespace {
+  /// \brief Constant variable used to convert boolean true value
+  ///        to a string.
+  const char* BOOLEAN_TRUE_STR = "1";
+  /// \brief Constant variable used to convert boolean false value
+  ///        to a string.
+  const char* BOOLEAN_FALSE_STR = "0";
+
+  std::string BooleanToString(bool boolean_value) {
+    return boolean_value ? BOOLEAN_TRUE_STR :
+           BOOLEAN_FALSE_STR;
+  }
+
+  bool StringToBoolean(const std::string& string_value) {
+    return string_value == BOOLEAN_TRUE_STR;
+  }
+}  // namespace
+
+const char* ColumnMetadata::kCatalogName = "ARROW:FLIGHT:SQL:CATALOG_NAME";
+const char* ColumnMetadata::kSchemaName = "ARROW:FLIGHT:SQL:SCHEMA_NAME";
+const char* ColumnMetadata::kTableName = "ARROW:FLIGHT:SQL:TABLE_NAME";
+const char* ColumnMetadata::kPrecision = "ARROW:FLIGHT:SQL:PRECISION";
+const char* ColumnMetadata::kScale = "ARROW:FLIGHT:SQL:SCALE";
+const char* ColumnMetadata::kIsAutoIncrement = "ARROW:FLIGHT:SQL:IS_AUTO_INCREMENT";
+const char* ColumnMetadata::kIsCaseSensitive = "ARROW:FLIGHT:SQL:IS_CASE_SENSITIVE";
+const char* ColumnMetadata::kIsReadOnly = "ARROW:FLIGHT:SQL:IS_READ_ONLY";
+const char* ColumnMetadata::kIsSearchable = "ARROW:FLIGHT:SQL:IS_SEARCHABLE";
+
+ColumnMetadata::ColumnMetadata(std::shared_ptr<arrow::KeyValueMetadata> metadata_map) :
+  metadata_map_(std::move(metadata_map)) {
+}
+
+arrow::Result<std::string> ColumnMetadata::GetCatalogName() const {
+  return metadata_map_->Get(kCatalogName);
+}
+
+arrow::Result<std::string> ColumnMetadata::GetSchemaName() const {
+  return metadata_map_->Get(kSchemaName);
+}
+
+arrow::Result<std::string> ColumnMetadata::GetTableName() const {
+  return metadata_map_->Get(kTableName);
+}
+
+arrow::Result<int32_t> ColumnMetadata::GetPrecision() const {
+  const Result <std::string> &result = metadata_map_->Get(kPrecision);

Review comment:
       I've notice this error when running one of the actions tests.
   
   I fixed and reformated all the files.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [arrow] lidavidm commented on pull request #11999: ARROW-15314: [C++][Java][FlightRPC] Add missing metadata on Arrow schemas returned by Flight SQL

Posted by GitBox <gi...@apache.org>.
lidavidm commented on pull request #11999:
URL: https://github.com/apache/arrow/pull/11999#issuecomment-1011256249


   Thanks for linking this up. BTW, are integration tests planned here too?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [arrow] lidavidm commented on a change in pull request #11999: ARROW-15314: [C++][Java][FlightRPC] Add missing metadata on Arrow schemas returned by Flight SQL

Posted by GitBox <gi...@apache.org>.
lidavidm commented on a change in pull request #11999:
URL: https://github.com/apache/arrow/pull/11999#discussion_r783440800



##########
File path: cpp/src/arrow/flight/sql/server_test.cc
##########
@@ -360,14 +363,26 @@ TEST_F(TestFlightSqlServer, TestCommandGetTablesWithIncludedSchemas) {
   std::shared_ptr<Table> table;
   ASSERT_OK(stream->ReadAll(&table));
 
+  const char* db_table_name = "intTable";

Review comment:
       nit, but just use std::string here




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [arrow] jcralmeida commented on a change in pull request #11999: ARROW-15314: [C++][Java][FlightRPC] Add missing metadata on Arrow schemas returned by Flight SQL

Posted by GitBox <gi...@apache.org>.
jcralmeida commented on a change in pull request #11999:
URL: https://github.com/apache/arrow/pull/11999#discussion_r801981009



##########
File path: format/FlightSql.proto
##########
@@ -1262,6 +1272,16 @@ message ActionClosePreparedStatementRequest {
  * Represents a SQL query. Used in the command member of FlightDescriptor
  * for the following RPC calls:

Review comment:
       The message itself is different, what is similar here is the response since the column metadata can be retrieved via both command




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [arrow] lidavidm commented on pull request #11999: ARROW-15314: [C++][Java][FlightRPC] Add missing metadata on Arrow schemas returned by Flight SQL

Posted by GitBox <gi...@apache.org>.
lidavidm commented on pull request #11999:
URL: https://github.com/apache/arrow/pull/11999#issuecomment-1068398462


   There was one doc lint hidden in a pipeline:
   
   ```
   /arrow/cpp/src/arrow/flight/sql/column_metadata.h:139: error: argument 'IsAutoIncrement' of command @param is not found in the argument list of arrow::flight::sql::ColumnMetadata::ColumnMetadataBuilder::IsAutoIncrement(bool is_auto_increment) (warning treated as error, aborting now)
   ```
   
   I think other failures are flakes/unrelated.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [arrow] lidavidm commented on pull request #11999: ARROW-15314: [C++][Java][FlightRPC] Add missing metadata on Arrow schemas returned by Flight SQL

Posted by GitBox <gi...@apache.org>.
lidavidm commented on pull request #11999:
URL: https://github.com/apache/arrow/pull/11999#issuecomment-1069477677


   I believe they're unrelated, I'll kick them to try again (the Java one is definitely unrelated)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [arrow] lidavidm commented on pull request #11999: Add missing metadata on Arrow schemas returned by Flight SQL

Posted by GitBox <gi...@apache.org>.
lidavidm commented on pull request #11999:
URL: https://github.com/apache/arrow/pull/11999#issuecomment-1010404681


   It can be created here: https://issues.apache.org/jira/secure/Dashboard.jspa
   
   Then just update the PR title as described above: https://github.com/apache/arrow/pull/11999#issuecomment-998114860


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [arrow] github-actions[bot] commented on pull request #11999: ARROW-15314: [Flight SQL] Add missing metadata on Arrow schemas returned by Flight SQL

Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #11999:
URL: https://github.com/apache/arrow/pull/11999#issuecomment-1011251643






-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [arrow] pitrou commented on a change in pull request #11999: ARROW-15314: [C++][Java][FlightRPC] Add missing metadata on Arrow schemas returned by Flight SQL

Posted by GitBox <gi...@apache.org>.
pitrou commented on a change in pull request #11999:
URL: https://github.com/apache/arrow/pull/11999#discussion_r800468926



##########
File path: format/FlightSql.proto
##########
@@ -933,6 +933,16 @@ message CommandGetDbSchemas {
  *  [optional] table_schema: bytes not null (schema of the table as described in Schema.fbs::Schema,
  *                                           it is serialized as an IPC message.)
  * >
+ * Fields on table_schema may contain the following metadata:
+ *  - ARROW:FLIGHT:SQL:CATALOG_NAME      - Table's catalog name
+ *  - ARROW:FLIGHT:SQL:DB_SCHEMA_NAME    - Database schema name
+ *  - ARROW:FLIGHT:SQL:TABLE_NAME        - Table name
+ *  - ARROW:FLIGHT:SQL:PRECISION         - Column precision/size
+ *  - ARROW:FLIGHT:SQL:SCALE             - Column scale/decimal digits

Review comment:
       Why are precision and scale useful? Can they annotate non-decimal columns?




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [arrow] jcralmeida edited a comment on pull request #11999: [JAVA] Add missing metadata on Arrow schemas returned by Flight SQL

Posted by GitBox <gi...@apache.org>.
jcralmeida edited a comment on pull request #11999:
URL: https://github.com/apache/arrow/pull/11999#issuecomment-998241020


   > Is the metadata here sufficient to cover at least JDBC and ODBC?
   
   Yes, we thought of this values thinking on both JDBC and ODBC.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [arrow] jcralmeida commented on pull request #11999: [JAVA] Add missing metadata on Arrow schemas returned by Flight SQL

Posted by GitBox <gi...@apache.org>.
jcralmeida commented on pull request #11999:
URL: https://github.com/apache/arrow/pull/11999#issuecomment-998118064


   @lidavidm Here is the changes related to the column metadata that @jduo was talking about.
   
   Could you take a look and share your thoughts, please?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [arrow] lidavidm commented on pull request #11999: [JAVA] Add missing metadata on Arrow schemas returned by Flight SQL

Posted by GitBox <gi...@apache.org>.
lidavidm commented on pull request #11999:
URL: https://github.com/apache/arrow/pull/11999#issuecomment-998788688


   Just a heads up, this needs rebasing now (sorry for the churn).
   
   Given there are some format changes here, we can bundle this with the type info method vote.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [arrow] jcralmeida commented on a change in pull request #11999: Add missing metadata on Arrow schemas returned by Flight SQL

Posted by GitBox <gi...@apache.org>.
jcralmeida commented on a change in pull request #11999:
URL: https://github.com/apache/arrow/pull/11999#discussion_r782212203



##########
File path: cpp/src/arrow/flight/sql/column_metadata.cc
##########
@@ -0,0 +1,187 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements.  See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership.  The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License.  You may obtain a copy of the License at
+//
+//   http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied.  See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+#include "arrow/flight/sql/column_metadata.h"
+
+#include <utility>
+
+namespace arrow {
+namespace flight {
+namespace sql {
+namespace {
+  /// \brief Constant variable used to convert boolean true value
+  ///        to a string.
+  const char* BOOLEAN_TRUE_STR = "YES";
+  /// \brief Constant variable used to convert boolean false value
+  ///        to a string.
+  const char* BOOLEAN_FALSE_STR = "NO";
+
+  std::string BooleanToString(bool boolean_value) {
+    return boolean_value ? BOOLEAN_TRUE_STR :
+           BOOLEAN_FALSE_STR;
+  }
+
+  bool StringToBoolean(const std::string& string_value) {
+    return string_value == BOOLEAN_TRUE_STR;
+  }
+}  // namespace
+
+const char* ColumnMetadata::kCatalogName = "ARROW:FLIGHT:SQL:CATALOG_NAME";
+const char* ColumnMetadata::kSchemaName = "ARROW:FLIGHT:SQL:SCHEMA_NAME";
+const char* ColumnMetadata::kTableName = "ARROW:FLIGHT:SQL:TABLE_NAME";
+const char* ColumnMetadata::kPrecision = "ARROW:FLIGHT:SQL:PRECISION";
+const char* ColumnMetadata::kScale = "ARROW:FLIGHT:SQL:SCALE";
+const char* ColumnMetadata::kIsAutoIncrement = "ARROW:FLIGHT:SQL:IS_AUTO_INCREMENT";
+const char* ColumnMetadata::kIsCaseSensitive = "ARROW:FLIGHT:SQL:IS_CASE_SENSITIVE";
+const char* ColumnMetadata::kIsReadOnly = "ARROW:FLIGHT:SQL:IS_READ_ONLY";
+const char* ColumnMetadata::kIsSearchable = "ARROW:FLIGHT:SQL:IS_SEARCHABLE";
+
+ColumnMetadata::ColumnMetadata(std::shared_ptr<arrow::KeyValueMetadata> metadata_map) :
+  metadata_map_(std::move(metadata_map)) {
+}
+
+arrow::Result<std::string> ColumnMetadata::GetCatalogName() const {
+  return metadata_map_->Get(kCatalogName);
+}
+
+arrow::Result<std::string> ColumnMetadata::GetSchemaName() const {
+  return metadata_map_->Get(kSchemaName);
+}
+
+arrow::Result<std::string> ColumnMetadata::GetTableName() const {
+  return metadata_map_->Get(kTableName);
+}
+
+arrow::Result<int32_t> ColumnMetadata::GetPrecision() const {
+  const Result <std::string> &result = metadata_map_->Get(kPrecision);
+  std::string precision_string;
+  ARROW_ASSIGN_OR_RAISE(precision_string, result);
+
+  return std::stoi(precision_string);
+}
+
+arrow::Result<int32_t> ColumnMetadata::GetScale() const {
+  std::string scale_string;
+  ARROW_ASSIGN_OR_RAISE(scale_string, metadata_map_->Get(kScale));
+
+  return std::stoi(scale_string);
+}
+
+arrow::Result<bool> ColumnMetadata::GetIsAutoIncrement() const {
+  std::string auto_increment_string;
+  ARROW_ASSIGN_OR_RAISE(auto_increment_string, metadata_map_->Get(kIsAutoIncrement));
+  return StringToBoolean(auto_increment_string);
+}
+
+arrow::Result<bool> ColumnMetadata::GetIsCaseSensitive() const {
+  std::string is_case_sensitive;
+  ARROW_ASSIGN_OR_RAISE(is_case_sensitive, metadata_map_->Get(kIsAutoIncrement));
+  return StringToBoolean(is_case_sensitive);
+}
+
+arrow::Result<bool> ColumnMetadata::GetIsReadOnly() const {
+  std::string is_read_only;
+  ARROW_ASSIGN_OR_RAISE(is_read_only, metadata_map_->Get(kIsAutoIncrement));
+  return StringToBoolean(is_read_only);
+}
+
+arrow::Result<bool> ColumnMetadata::GetIsSearchable() const {
+  std::string is_case_sensitive;
+  ARROW_ASSIGN_OR_RAISE(is_case_sensitive, metadata_map_->Get(kIsAutoIncrement));
+  return StringToBoolean(is_case_sensitive);
+}
+
+ColumnMetadata::ColumnMetadataBuilder ColumnMetadata::Builder() {
+  const ColumnMetadataBuilder &builder = ColumnMetadataBuilder{};
+  return builder;

Review comment:
       Fixed

##########
File path: cpp/src/arrow/flight/sql/column_metadata.h
##########
@@ -0,0 +1,168 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements.  See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership.  The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License.  You may obtain a copy of the License at
+//
+//   http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied.  See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+#pragma once
+
+#include <string>
+
+#include "arrow/util/key_value_metadata.h"
+
+namespace arrow {
+namespace flight {
+namespace sql {
+
+/// \brief Helper class to set column metadata.
+class ColumnMetadata {
+ private:
+  std::shared_ptr<arrow::KeyValueMetadata> metadata_map_;
+  explicit ColumnMetadata(std::shared_ptr<arrow::KeyValueMetadata> metadata_map);
+
+ public:
+  class ColumnMetadataBuilder;
+
+  /// \brief Constant variable to hold the value of the key that
+  ///        will be used in the KeyValueMetadata class.
+  static const char* kCatalogName;
+  /// \brief Constant variable to hold the value of the key that
+  ///        will be used in the KeyValueMetadata class.
+  static const char* kSchemaName;
+  /// \brief Constant variable to hold the value of the key that
+  ///        will be used in the KeyValueMetadata class.
+  static const char* kTableName;
+  /// \brief Constant variable to hold the value of the key that
+  ///        will be used in the KeyValueMetadata class.
+  static const char* kPrecision;
+  /// \brief Constant variable to hold the value of the key that
+  ///        will be used in the KeyValueMetadata class.
+  static const char* kScale;
+  /// \brief Constant variable to hold the value of the key that
+  ///        will be used in the KeyValueMetadata class.
+  static const char* kIsAutoIncrement;
+  /// \brief Constant variable to hold the value of the key that
+  ///        will be used in the KeyValueMetadata class.
+  static const char* kIsCaseSensitive;
+  /// \brief Constant variable to hold the value of the key that
+  ///        will be used in the KeyValueMetadata class.
+  static const char* kIsReadOnly;
+  /// \brief Constant variable to hold the value of the key that
+  ///        will be used in the KeyValueMetadata class.
+  static const char* kIsSearchable;
+
+  /// \brief Static initializer.
+  static ColumnMetadataBuilder Builder();
+
+  /// \brief  Return the catalog name set in the KeyValueMetadata.
+  /// \return The catalog name.
+  arrow::Result<std::string> GetCatalogName() const;
+
+  /// \brief  Return the schema name set in the KeyValueMetadata.
+  /// \return The schema name.
+  arrow::Result<std::string> GetSchemaName() const;
+
+  /// \brief  Return the table name set in the KeyValueMetadata.
+  /// \return The table name.
+  arrow::Result<std::string> GetTableName() const;
+
+  /// \brief  Return the precision set in the KeyValueMetadata.
+  /// \return The precision.
+  arrow::Result<int32_t> GetPrecision() const;
+
+  /// \brief  Return the scale set in the KeyValueMetadata.
+  /// \return The scale.
+  arrow::Result<int32_t> GetScale() const;
+
+  /// \brief  Return the IsAutoIncrement set in the KeyValueMetadata.
+  /// \return The IsAutoIncrement.
+  arrow::Result<bool> GetIsAutoIncrement() const;
+
+  /// \brief  Return the IsCaseSensitive set in the KeyValueMetadata.
+  /// \return The IsCaseSensitive.
+  arrow::Result<bool> GetIsCaseSensitive() const;
+
+  /// \brief  Return the IsReadOnly set in the KeyValueMetadata.
+  /// \return The IsReadOnly.
+  arrow::Result<bool> GetIsReadOnly() const;
+
+  /// \brief  Return the IsSearchable set in the KeyValueMetadata.
+  /// \return The IsSearchable.
+  arrow::Result<bool> GetIsSearchable() const;
+
+  /// \brief  Return the KeyValueMetadata.
+  /// \return The KeyValueMetadata.
+  std::shared_ptr<arrow::KeyValueMetadata> GetMetadataMap() const;

Review comment:
       Fixed




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [arrow] lidavidm commented on a change in pull request #11999: ARROW-15314: [C++][Java][FlightRPC] Add missing metadata on Arrow schemas returned by Flight SQL

Posted by GitBox <gi...@apache.org>.
lidavidm commented on a change in pull request #11999:
URL: https://github.com/apache/arrow/pull/11999#discussion_r827084653



##########
File path: format/FlightSql.proto
##########
@@ -441,7 +441,7 @@ enum SqlInfo {
    * - return 3   (\b11)    => [SQL_SUBQUERIES_IN_COMPARISONS, SQL_SUBQUERIES_IN_EXISTS];
    * - return 4   (\b100)   => [SQL_SUBQUERIES_IN_INS];
    * - return 5   (\b101)   => [SQL_SUBQUERIES_IN_COMPARISONS, SQL_SUBQUERIES_IN_INS];
-   * - return 6   (\b110)   => [SQL_SUBQUERIES_IN_INS, SQL_SUBQUERIES_IN_EXISTS];
+   * - return 6   (\b110)   => [SQL_SUBQUERIES_IN_COMPARISONS, SQL_SUBQUERIES_IN_EXISTS];

Review comment:
       @jcralmeida any comment here? I forgot about this, let's fix these last few things and then kick off the vote




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [arrow] jcralmeida commented on a change in pull request #11999: ARROW-15314: [C++][Java][FlightRPC] Add missing metadata on Arrow schemas returned by Flight SQL

Posted by GitBox <gi...@apache.org>.
jcralmeida commented on a change in pull request #11999:
URL: https://github.com/apache/arrow/pull/11999#discussion_r827186080



##########
File path: format/FlightSql.proto
##########
@@ -441,7 +441,7 @@ enum SqlInfo {
    * - return 3   (\b11)    => [SQL_SUBQUERIES_IN_COMPARISONS, SQL_SUBQUERIES_IN_EXISTS];
    * - return 4   (\b100)   => [SQL_SUBQUERIES_IN_INS];
    * - return 5   (\b101)   => [SQL_SUBQUERIES_IN_COMPARISONS, SQL_SUBQUERIES_IN_INS];
-   * - return 6   (\b110)   => [SQL_SUBQUERIES_IN_INS, SQL_SUBQUERIES_IN_EXISTS];
+   * - return 6   (\b110)   => [SQL_SUBQUERIES_IN_COMPARISONS, SQL_SUBQUERIES_IN_EXISTS];

Review comment:
       Oh sorry @lidavidm , I've changed back to how it was which was the correct one. I don't know how/why I've changed it 




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [arrow] jcralmeida commented on pull request #11999: ARROW-15314: [C++][Java][FlightRPC] Add missing metadata on Arrow schemas returned by Flight SQL

Posted by GitBox <gi...@apache.org>.
jcralmeida commented on pull request #11999:
URL: https://github.com/apache/arrow/pull/11999#issuecomment-1069401870


   Hi @lidavidm I've got some failures on some CI jobs but according to I'm seeing they are unrelated to code. Is it right?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [arrow] jcralmeida commented on a change in pull request #11999: Add missing metadata on Arrow schemas returned by Flight SQL

Posted by GitBox <gi...@apache.org>.
jcralmeida commented on a change in pull request #11999:
URL: https://github.com/apache/arrow/pull/11999#discussion_r782212397



##########
File path: cpp/src/arrow/flight/sql/column_metadata.h
##########
@@ -0,0 +1,168 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements.  See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership.  The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License.  You may obtain a copy of the License at
+//
+//   http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied.  See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+#pragma once
+
+#include <string>
+
+#include "arrow/util/key_value_metadata.h"
+
+namespace arrow {
+namespace flight {
+namespace sql {
+
+/// \brief Helper class to set column metadata.
+class ColumnMetadata {
+ private:
+  std::shared_ptr<arrow::KeyValueMetadata> metadata_map_;
+  explicit ColumnMetadata(std::shared_ptr<arrow::KeyValueMetadata> metadata_map);
+
+ public:
+  class ColumnMetadataBuilder;
+
+  /// \brief Constant variable to hold the value of the key that
+  ///        will be used in the KeyValueMetadata class.
+  static const char* kCatalogName;
+  /// \brief Constant variable to hold the value of the key that
+  ///        will be used in the KeyValueMetadata class.
+  static const char* kSchemaName;
+  /// \brief Constant variable to hold the value of the key that
+  ///        will be used in the KeyValueMetadata class.
+  static const char* kTableName;
+  /// \brief Constant variable to hold the value of the key that
+  ///        will be used in the KeyValueMetadata class.
+  static const char* kPrecision;
+  /// \brief Constant variable to hold the value of the key that
+  ///        will be used in the KeyValueMetadata class.
+  static const char* kScale;
+  /// \brief Constant variable to hold the value of the key that
+  ///        will be used in the KeyValueMetadata class.
+  static const char* kIsAutoIncrement;
+  /// \brief Constant variable to hold the value of the key that
+  ///        will be used in the KeyValueMetadata class.
+  static const char* kIsCaseSensitive;
+  /// \brief Constant variable to hold the value of the key that
+  ///        will be used in the KeyValueMetadata class.
+  static const char* kIsReadOnly;
+  /// \brief Constant variable to hold the value of the key that
+  ///        will be used in the KeyValueMetadata class.
+  static const char* kIsSearchable;
+
+  /// \brief Static initializer.
+  static ColumnMetadataBuilder Builder();
+
+  /// \brief  Return the catalog name set in the KeyValueMetadata.
+  /// \return The catalog name.
+  arrow::Result<std::string> GetCatalogName() const;
+
+  /// \brief  Return the schema name set in the KeyValueMetadata.
+  /// \return The schema name.
+  arrow::Result<std::string> GetSchemaName() const;
+
+  /// \brief  Return the table name set in the KeyValueMetadata.
+  /// \return The table name.
+  arrow::Result<std::string> GetTableName() const;
+
+  /// \brief  Return the precision set in the KeyValueMetadata.
+  /// \return The precision.
+  arrow::Result<int32_t> GetPrecision() const;
+
+  /// \brief  Return the scale set in the KeyValueMetadata.
+  /// \return The scale.
+  arrow::Result<int32_t> GetScale() const;
+
+  /// \brief  Return the IsAutoIncrement set in the KeyValueMetadata.
+  /// \return The IsAutoIncrement.
+  arrow::Result<bool> GetIsAutoIncrement() const;
+
+  /// \brief  Return the IsCaseSensitive set in the KeyValueMetadata.
+  /// \return The IsCaseSensitive.
+  arrow::Result<bool> GetIsCaseSensitive() const;
+
+  /// \brief  Return the IsReadOnly set in the KeyValueMetadata.
+  /// \return The IsReadOnly.
+  arrow::Result<bool> GetIsReadOnly() const;
+
+  /// \brief  Return the IsSearchable set in the KeyValueMetadata.
+  /// \return The IsSearchable.
+  arrow::Result<bool> GetIsSearchable() const;
+
+  /// \brief  Return the KeyValueMetadata.
+  /// \return The KeyValueMetadata.
+  std::shared_ptr<arrow::KeyValueMetadata> GetMetadataMap() const;
+
+  /// \brief A builder class to construct the ColumnMetadata object.
+  class ColumnMetadataBuilder {
+    std::shared_ptr<arrow::KeyValueMetadata> metadata_map_;

Review comment:
       Fixed

##########
File path: format/FlightSql.proto
##########
@@ -933,6 +933,16 @@ message CommandGetDbSchemas {
  *  [optional] table_schema: bytes not null (schema of the table as described in Schema.fbs::Schema,
  *                                           it is serialized as an IPC message.)
  * >
+ * Fields on table_schema may contain the following metadata:
+ *  - ARROW:FLIGHT:SQL:CATALOG_NAME      - Table's catalog name
+ *  - ARROW:FLIGHT:SQL:DB_SCHEMA_NAME    - Database schema name
+ *  - ARROW:FLIGHT:SQL:TABLE_NAME        - Table name
+ *  - ARROW:FLIGHT:SQL:PRECISION         - Column precision/size
+ *  - ARROW:FLIGHT:SQL:SCALE             - Column scale/decimal digits
+ *  - ARROW:FLIGHT:SQL:IS_AUTO_INCREMENT - "1" indicates if the column is auto incremented, "0" otherwise.

Review comment:
       Fixed




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [arrow] jcralmeida commented on pull request #11999: ARROW-15314: [C++][Java][FlightRPC] Add missing metadata on Arrow schemas returned by Flight SQL

Posted by GitBox <gi...@apache.org>.
jcralmeida commented on pull request #11999:
URL: https://github.com/apache/arrow/pull/11999#issuecomment-1011276905


   @lidavidm Sorry, I've found an error here. I'm fixing it and I will let you know when I finish it


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [arrow] jcralmeida commented on a change in pull request #11999: ARROW-15314: [C++][Java][FlightRPC] Add missing metadata on Arrow schemas returned by Flight SQL

Posted by GitBox <gi...@apache.org>.
jcralmeida commented on a change in pull request #11999:
URL: https://github.com/apache/arrow/pull/11999#discussion_r783442065



##########
File path: cpp/src/arrow/flight/sql/server_test.cc
##########
@@ -360,14 +363,26 @@ TEST_F(TestFlightSqlServer, TestCommandGetTablesWithIncludedSchemas) {
   std::shared_ptr<Table> table;
   ASSERT_OK(stream->ReadAll(&table));
 
+  const char* db_table_name = "intTable";

Review comment:
       No problem, I will fix it 




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [arrow] pitrou commented on a change in pull request #11999: ARROW-15314: [C++][Java][FlightRPC] Add missing metadata on Arrow schemas returned by Flight SQL

Posted by GitBox <gi...@apache.org>.
pitrou commented on a change in pull request #11999:
URL: https://github.com/apache/arrow/pull/11999#discussion_r800471848



##########
File path: format/FlightSql.proto
##########
@@ -1262,6 +1272,16 @@ message ActionClosePreparedStatementRequest {
  * Represents a SQL query. Used in the command member of FlightDescriptor
  * for the following RPC calls:

Review comment:
       Why does this have the same description as CommandGetTables? Do we have two different messages for doing the same thing?




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [arrow] jcralmeida commented on a change in pull request #11999: ARROW-15314: [C++][Java][FlightRPC] Add missing metadata on Arrow schemas returned by Flight SQL

Posted by GitBox <gi...@apache.org>.
jcralmeida commented on a change in pull request #11999:
URL: https://github.com/apache/arrow/pull/11999#discussion_r832231006



##########
File path: format/FlightSql.proto
##########
@@ -933,6 +933,16 @@ message CommandGetDbSchemas {
  *  [optional] table_schema: bytes not null (schema of the table as described in Schema.fbs::Schema,
  *                                           it is serialized as an IPC message.)
  * >
+ * Fields on table_schema may contain the following metadata:
+ *  - ARROW:FLIGHT:SQL:CATALOG_NAME      - Table's catalog name
+ *  - ARROW:FLIGHT:SQL:DB_SCHEMA_NAME    - Database schema name
+ *  - ARROW:FLIGHT:SQL:TABLE_NAME        - Table name
+ *  - ARROW:FLIGHT:SQL:PRECISION         - Column precision/size
+ *  - ARROW:FLIGHT:SQL:SCALE             - Column scale/decimal digits if applicable
+ *  - ARROW:FLIGHT:SQL:IS_AUTO_INCREMENT - "1" indicates if the column is auto incremented, "0" otherwise.
+ *  - ARROW:FLIGHT:SQL:IS_CASE_SENSITIVE - "1" indicates if the column is case sensitive, "0" otherwise.
+ *  - ARROW:FLIGHT:SQL:IS_READ_ONLY      - "1" indicates if the column is read only, "0" otherwise.
+ *  - ARROW:FLIGHT:SQL:IS_SEARCHABLE     - "1" indicates if the column is searchable via WHERE clause, "0" otherwise.
  * The returned data should be ordered by catalog_name, db_schema_name, table_name, then table_type, followed by table_schema if requested.

Review comment:
       Yes




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [arrow] jcralmeida commented on a change in pull request #11999: ARROW-15314: [C++][Java][FlightRPC] Add missing metadata on Arrow schemas returned by Flight SQL

Posted by GitBox <gi...@apache.org>.
jcralmeida commented on a change in pull request #11999:
URL: https://github.com/apache/arrow/pull/11999#discussion_r832229376



##########
File path: format/FlightSql.proto
##########
@@ -933,6 +933,16 @@ message CommandGetDbSchemas {
  *  [optional] table_schema: bytes not null (schema of the table as described in Schema.fbs::Schema,
  *                                           it is serialized as an IPC message.)
  * >
+ * Fields on table_schema may contain the following metadata:
+ *  - ARROW:FLIGHT:SQL:CATALOG_NAME      - Table's catalog name
+ *  - ARROW:FLIGHT:SQL:DB_SCHEMA_NAME    - Database schema name
+ *  - ARROW:FLIGHT:SQL:TABLE_NAME        - Table name

Review comment:
       Just because they should unified in every place it appears




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [arrow] ursabot edited a comment on pull request #11999: ARROW-15314: [C++][Java][FlightRPC] Add missing metadata on Arrow schemas returned by Flight SQL

Posted by GitBox <gi...@apache.org>.
ursabot edited a comment on pull request #11999:
URL: https://github.com/apache/arrow/pull/11999#issuecomment-1079443371


   Benchmark runs are scheduled for baseline = acc6c2eb8ae580085296f608c1a0e8564269456d and contender = c2fac051f3b43835e33364c2aa6a38d1733f241f. c2fac051f3b43835e33364c2aa6a38d1733f241f is a master commit associated with this PR. Results will be available as each benchmark for each run completes.
   Conbench compare runs links:
   [Scheduled] [ec2-t3-xlarge-us-east-2](https://conbench.ursa.dev/compare/runs/d3a95aeb4c5248ed8dae1f73371f7e28...d588dcd7c8f8439bbc1049df4852c7a8/)
   [Finished :arrow_down:0.67% :arrow_up:0.34%] [test-mac-arm](https://conbench.ursa.dev/compare/runs/e5f98bce316647ec9ac0ef83f2de4458...5ca4e14a3a08432cbebfd4a4cbdf846d/)
   [Scheduled] [ursa-i9-9960x](https://conbench.ursa.dev/compare/runs/a77d0624011a463990d0eabe541be6df...3e39c657f5d94d949f9eb56be4ed345e/)
   [Finished :arrow_down:0.04% :arrow_up:0.0%] [ursa-thinkcentre-m75q](https://conbench.ursa.dev/compare/runs/5ca388d6438c409094189683c61c0a20...1294a96089f34e5ba22220660c0ea345/)
   Supported benchmarks:
   ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python, R. Runs only benchmarks with cloud = True
   test-mac-arm: Supported benchmark langs: C++, Python, R
   ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript
   ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [arrow] ursabot edited a comment on pull request #11999: ARROW-15314: [C++][Java][FlightRPC] Add missing metadata on Arrow schemas returned by Flight SQL

Posted by GitBox <gi...@apache.org>.
ursabot edited a comment on pull request #11999:
URL: https://github.com/apache/arrow/pull/11999#issuecomment-1079443371


   Benchmark runs are scheduled for baseline = acc6c2eb8ae580085296f608c1a0e8564269456d and contender = c2fac051f3b43835e33364c2aa6a38d1733f241f. c2fac051f3b43835e33364c2aa6a38d1733f241f is a master commit associated with this PR. Results will be available as each benchmark for each run completes.
   Conbench compare runs links:
   [Finished :arrow_down:0.0% :arrow_up:0.0%] [ec2-t3-xlarge-us-east-2](https://conbench.ursa.dev/compare/runs/d3a95aeb4c5248ed8dae1f73371f7e28...d588dcd7c8f8439bbc1049df4852c7a8/)
   [Finished :arrow_down:0.67% :arrow_up:0.34%] [test-mac-arm](https://conbench.ursa.dev/compare/runs/e5f98bce316647ec9ac0ef83f2de4458...5ca4e14a3a08432cbebfd4a4cbdf846d/)
   [Failed :arrow_down:0.0% :arrow_up:0.0%] [ursa-i9-9960x](https://conbench.ursa.dev/compare/runs/a77d0624011a463990d0eabe541be6df...3e39c657f5d94d949f9eb56be4ed345e/)
   [Finished :arrow_down:0.04% :arrow_up:0.0%] [ursa-thinkcentre-m75q](https://conbench.ursa.dev/compare/runs/5ca388d6438c409094189683c61c0a20...1294a96089f34e5ba22220660c0ea345/)
   Supported benchmarks:
   ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python, R. Runs only benchmarks with cloud = True
   test-mac-arm: Supported benchmark langs: C++, Python, R
   ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript
   ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [arrow] lidavidm commented on a change in pull request #11999: [JAVA] Add missing metadata on Arrow schemas returned by Flight SQL

Posted by GitBox <gi...@apache.org>.
lidavidm commented on a change in pull request #11999:
URL: https://github.com/apache/arrow/pull/11999#discussion_r772574683



##########
File path: format/FlightSql.proto
##########
@@ -55,7 +55,7 @@ message CommandGetSqlInfo {
    * Initially, Flight SQL will support the following information types:
    * - Server Information - Range [0-500)
    * - Syntax Information - Range [500-1000)
-   * Range [0-10,000) is reserved for defaults (see SqlInfo enum for default options). 
+   * Range [0-10,000] is reserved for defaults (see SqlInfo enum for default options).

Review comment:
       This seems wrong if custom options start at 10_000.

##########
File path: format/FlightSql.proto
##########
@@ -933,6 +933,16 @@ message CommandGetDbSchemas {
  *  [optional] table_schema: bytes not null (schema of the table as described in Schema.fbs::Schema,
  *                                           it is serialized as an IPC message.)
  * >
+ * Fields on table_schema may contain the following metadata:
+ *  - CATALOG_NAME      - Table's catalog name
+ *  - SCHEMA_NAME       - Table's schema name
+ *  - TABLE_NAME        - Table name
+ *  - PRECISION         - Column precision/size
+ *  - SCALE             - Column scale/decimal digits
+ *  - IS_AUTO_INCREMENT - "1" if column is auto incremented, "0" otherwise.
+ *  - IS_CASE_SENSITIVE - "1" if column is case sensitive, "0" otherwise.
+ *  - IS_READ_ONLY      - "1" if column is read only, "0" otherwise.
+ *  - IS_SEARCHABLE     - "1" if column is searchable, "0" otherwise.

Review comment:
       Can we namespace these values? e.g. `arrow.flight.sql.catalog_name`.
   
   Or actually, in analogy with [extension types](https://arrow.apache.org/docs/format/Columnar.html#extension-types) it should be something like `ARROW:FLIGHT:SQL:catalog_name`

##########
File path: java/flight/flight-sql/src/main/java/org/apache/arrow/flight/sql/FlightSqlColumnMetadata.java
##########
@@ -0,0 +1,295 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.arrow.flight.sql;
+
+import java.util.HashMap;
+import java.util.Map;
+
+/**
+ * Metadata for a column in a Flight SQL query.
+ *
+ * This can be used with FlightSqlClient to access column's metadata contained in schemas returned
+ * by GetTables and query execution as follows:
+ * <pre>
+ *   FlightSqlColumnMetadata metadata = new FlightSqlColumnMetadata(field.getMetadata());
+ *   Integer precision = metadata.getPrecision();
+ * </pre>
+ *
+ * FlightSqlProducer can use this to set metadata on a column in a schema as follows:
+ * <pre>
+ *   FlightSqlColumnMetadata metadata = new FlightSqlColumnMetadata.Builder()
+ *         .precision(10)
+ *         .scale(5)
+ *         .build();
+ *   Field field = new Field("column", new FieldType(..., metadata.getMetadataMap()), null);
+ * </pre>
+ */
+public class FlightSqlColumnMetadata {
+
+  private static final String CATALOG_NAME = "CATALOG_NAME";
+  private static final String SCHEMA_NAME = "SCHEMA_NAME";
+  private static final String TABLE_NAME = "TABLE_NAME";
+  private static final String PRECISION = "PRECISION";
+  private static final String SCALE = "SCALE";
+  private static final String IS_AUTO_INCREMENT = "IS_AUTO_INCREMENT";
+  private static final String IS_CASE_SENSITIVE = "IS_CASE_SENSITIVE";
+  private static final String IS_READ_ONLY = "IS_READ_ONLY";
+  private static final String IS_SEARCHABLE = "IS_SEARCHABLE";
+
+  private static final String BOOLEAN_TRUE_STR = "1";
+  private static final String BOOLEAN_FALSE_STR = "0";
+
+  private final Map<String, String> metadataMap;
+
+  /**
+   * Creates a new instance of FlightSqlColumnMetadata.
+   */
+  public FlightSqlColumnMetadata(Map<String, String> metadataMap) {
+    this.metadataMap = metadataMap;
+  }
+
+  /**
+   * Returns the metadata map.
+   * @return The metadata map.
+   */
+  public Map<String, String> getMetadataMap() {
+    return metadataMap;

Review comment:
       Similarly, this might want to wrap in `Collections.unmodifiableMap`.

##########
File path: java/flight/flight-sql/src/main/java/org/apache/arrow/flight/sql/FlightSqlColumnMetadata.java
##########
@@ -0,0 +1,295 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.arrow.flight.sql;
+
+import java.util.HashMap;
+import java.util.Map;
+
+/**
+ * Metadata for a column in a Flight SQL query.
+ *
+ * This can be used with FlightSqlClient to access column's metadata contained in schemas returned
+ * by GetTables and query execution as follows:
+ * <pre>
+ *   FlightSqlColumnMetadata metadata = new FlightSqlColumnMetadata(field.getMetadata());
+ *   Integer precision = metadata.getPrecision();
+ * </pre>
+ *
+ * FlightSqlProducer can use this to set metadata on a column in a schema as follows:
+ * <pre>
+ *   FlightSqlColumnMetadata metadata = new FlightSqlColumnMetadata.Builder()
+ *         .precision(10)
+ *         .scale(5)
+ *         .build();
+ *   Field field = new Field("column", new FieldType(..., metadata.getMetadataMap()), null);
+ * </pre>
+ */
+public class FlightSqlColumnMetadata {
+
+  private static final String CATALOG_NAME = "CATALOG_NAME";
+  private static final String SCHEMA_NAME = "SCHEMA_NAME";
+  private static final String TABLE_NAME = "TABLE_NAME";
+  private static final String PRECISION = "PRECISION";
+  private static final String SCALE = "SCALE";
+  private static final String IS_AUTO_INCREMENT = "IS_AUTO_INCREMENT";
+  private static final String IS_CASE_SENSITIVE = "IS_CASE_SENSITIVE";
+  private static final String IS_READ_ONLY = "IS_READ_ONLY";
+  private static final String IS_SEARCHABLE = "IS_SEARCHABLE";
+
+  private static final String BOOLEAN_TRUE_STR = "1";
+  private static final String BOOLEAN_FALSE_STR = "0";
+
+  private final Map<String, String> metadataMap;
+
+  /**
+   * Creates a new instance of FlightSqlColumnMetadata.
+   */
+  public FlightSqlColumnMetadata(Map<String, String> metadataMap) {
+    this.metadataMap = metadataMap;

Review comment:
       Should this make a copy of the map to guard against mutation?

##########
File path: format/FlightSql.proto
##########
@@ -933,6 +933,16 @@ message CommandGetDbSchemas {
  *  [optional] table_schema: bytes not null (schema of the table as described in Schema.fbs::Schema,
  *                                           it is serialized as an IPC message.)
  * >
+ * Fields on table_schema may contain the following metadata:
+ *  - CATALOG_NAME      - Table's catalog name
+ *  - SCHEMA_NAME       - Table's schema name
+ *  - TABLE_NAME        - Table name
+ *  - PRECISION         - Column precision/size
+ *  - SCALE             - Column scale/decimal digits
+ *  - IS_AUTO_INCREMENT - "1" if column is auto incremented, "0" otherwise.
+ *  - IS_CASE_SENSITIVE - "1" if column is case sensitive, "0" otherwise.
+ *  - IS_READ_ONLY      - "1" if column is read only, "0" otherwise.
+ *  - IS_SEARCHABLE     - "1" if column is searchable, "0" otherwise.

Review comment:
       Can we also define what "searchable" means in this context?

##########
File path: format/FlightSql.proto
##########
@@ -933,6 +933,16 @@ message CommandGetDbSchemas {
  *  [optional] table_schema: bytes not null (schema of the table as described in Schema.fbs::Schema,
  *                                           it is serialized as an IPC message.)
  * >
+ * Fields on table_schema may contain the following metadata:
+ *  - CATALOG_NAME      - Table's catalog name
+ *  - SCHEMA_NAME       - Table's schema name
+ *  - TABLE_NAME        - Table name
+ *  - PRECISION         - Column precision/size
+ *  - SCALE             - Column scale/decimal digits
+ *  - IS_AUTO_INCREMENT - "1" if column is auto incremented, "0" otherwise.
+ *  - IS_CASE_SENSITIVE - "1" if column is case sensitive, "0" otherwise.
+ *  - IS_READ_ONLY      - "1" if column is read only, "0" otherwise.
+ *  - IS_SEARCHABLE     - "1" if column is searchable, "0" otherwise.

Review comment:
       Assuming that if a particular metadata is not known or not supported, it should just be omitted?

##########
File path: format/FlightSql.proto
##########
@@ -933,6 +933,16 @@ message CommandGetDbSchemas {
  *  [optional] table_schema: bytes not null (schema of the table as described in Schema.fbs::Schema,
  *                                           it is serialized as an IPC message.)
  * >
+ * Fields on table_schema may contain the following metadata:
+ *  - CATALOG_NAME      - Table's catalog name
+ *  - SCHEMA_NAME       - Table's schema name

Review comment:
       "DB_SCHEMA_NAME - database schema name" for consistency?




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org