[GitHub] [druid] kfaraz commented on a change in pull request #11914: SQL: Add type headers to response formats.

[GitBox <gi...@apache.org>]

kfaraz commented on a change in pull request #11914:
URL: https://github.com/apache/druid/pull/11914#discussion_r747943168



##########
File path: docs/querying/sql.md
##########
@@ -879,6 +879,8 @@ Submit your query as the value of a "query" field in the JSON object within the
 |`query`|SQL query string.| none (required)|
 |`resultFormat`|Format of query results. See [Responses](#responses) for details.|`"object"`|
 |`header`|Whether or not to include a header row for the query result. See [Responses](#responses) for details.|`false`|
+|`typesHeader`|Whether or not to include type information in the header. Only applicable when `header` is `true`. See [Responses](#responses) for details.|`false`|
+|`sqlTypesHeader`|Whether or not to include type information in the header. Only applicable when `header` is `true`. See [Responses](#responses) for details.|`false`|

Review comment:
       ```suggestion
   |`sqlTypesHeader`|Whether or not to include SQL type information in the header. Only applicable when `header` is `true`. See [Responses](#responses) for details.|`false`|
   ```

##########
File path: docs/querying/sql.md
##########
@@ -920,44 +922,60 @@ Metadata is available over HTTP POST by querying [metadata tables](#metadata-tab
 
 #### Responses
 
+##### Result formats
+
 Druid SQL's HTTP POST API supports a variety of result formats. You can specify these by adding a "resultFormat"
 parameter, like:
 
 ```json
 {
   "query" : "SELECT COUNT(*) FROM data_source WHERE foo = 'bar' AND __time > TIMESTAMP '2000-01-01 00:00:00'",
-  "resultFormat" : "object"
+  "resultFormat" : "array"
 }
 ```
 
-The supported result formats are:
-
-|Format|Description|Content-Type|
-|------|-----------|------------|
-|`object`|The default, a JSON array of JSON objects. Each object's field names match the columns returned by the SQL query, and are provided in the same order as the SQL query.|application/json|
-|`array`|JSON array of JSON arrays. Each inner array has elements matching the columns returned by the SQL query, in order.|application/json|
-|`objectLines`|Like "object", but the JSON objects are separated by newlines instead of being wrapped in a JSON array. This can make it easier to parse the entire response set as a stream, if you do not have ready access to a streaming JSON parser. To make it possible to detect a truncated response, this format includes a trailer of one blank line.|text/plain|
-|`arrayLines`|Like "array", but the JSON arrays are separated by newlines instead of being wrapped in a JSON array. This can make it easier to parse the entire response set as a stream, if you do not have ready access to a streaming JSON parser. To make it possible to detect a truncated response, this format includes a trailer of one blank line.|text/plain|
-|`csv`|Comma-separated values, with one row per line. Individual field values may be escaped by being surrounded in double quotes. If double quotes appear in a field value, they will be escaped by replacing them with double-double-quotes like `""this""`. To make it possible to detect a truncated response, this format includes a trailer of one blank line.|text/csv|
-
-You can additionally request a header by setting "header" to true in your request, like:
+You can additionally request a header with information about column names by setting `header` to true in your request.
+When you set `header` to true, you can optionally include `typesHeader` and `sqlTypesHeader` as well, which gives
+you information about [Druid runtime and SQL types](#data-types) respectively. You can request all these headers
+with a request like:
 
 ```json
 {
   "query" : "SELECT COUNT(*) FROM data_source WHERE foo = 'bar' AND __time > TIMESTAMP '2000-01-01 00:00:00'",
-  "resultFormat" : "arrayLines",
-  "header" : true
+  "resultFormat" : "array",
+  "header" : true,
+  "typesHeader" : true,
+  "sqlTypesHeader" : true
 }
 ```
 
-In this case, the first result of the response body is the header row. For the `csv`, `array`, and `arrayLines` formats, the header
-will be a list of column names. For the `object` and `objectLines` formats, the header will be an object where the
-keys are column names, and the values are null.
+The supported result formats are:
+
+|Format|Description|Header description|Content-Type|
+|------|-----------|------------------|------------|
+|`object`|The default, a JSON array of JSON objects. Each object's field names match the columns returned by the SQL query, and are provided in the same order as the SQL query.|If `header` is true, the first row is an object where the fields are column names. Each field's value is either null (if `typesHeader` and `sqlTypesHeader` are false) or an object that contains the Druid type as `type` (if `typesHeader` is true) and the SQL type as `sqlType` (if `sqlTypesHeader` is true).|application/json|
+|`array`|JSON array of JSON arrays. Each inner array has elements matching the columns returned by the SQL query, in order.|If `header` is true, the first row is an array of column names. If `typesHeader` is true, the next row is an array of Druid types. If `sqlTypesHeader` is true, the next row is an array of SQL types.|application/json|
+|`objectLines`|Like "object", but the JSON objects are separated by newlines instead of being wrapped in a JSON array. This can make it easier to parse the entire response set as a stream, if you do not have ready access to a streaming JSON parser. To make it possible to detect a truncated response, this format includes a trailer of one blank line.|Same as "object".|text/plain|
+|`arrayLines`|Like "array", but the JSON arrays are separated by newlines instead of being wrapped in a JSON array. This can make it easier to parse the entire response set as a stream, if you do not have ready access to a streaming JSON parser. To make it possible to detect a truncated response, this format includes a trailer of one blank line.|Same as "array", except the rows are separated by newlines.|text/plain|
+|`csv`|Comma-separated values, with one row per line. Individual field values may be escaped by being surrounded in double quotes. If double quotes appear in a field value, they will be escaped by replacing them with double-double-quotes like `""this""`. To make it possible to detect a truncated response, this format includes a trailer of one blank line.|Same as "array", except the lists are in CSV format.|text/csv|
+
+If `typesHeader` is set to true, [Druid type](#data-types) information is included in the response. Complex types,
+like sketches, will be reported as `COMPLEX<typeName>` if a particular complex type name is known for that field,
+or as `COMPLEX` if the particular type name is unknown or mixed. If `sqlTypesHeader` is set to true,
+[SQL type](#data-types) information is included in the response. It is possible to set both `typesHeader` and
+`sqlTypesHeader` at once. Both parameters require that `header` is also set.
+
+To aid in building clients that are compatible with older Druid versions, Druid returns the HTTP header
+`X-Druid-SQL-Header-Included: yes` if `header` was set to true and if the version of Druid you are talking to
+understands the `typesHeader` and `sqlTypesHeader` parameters. This happens even if you do not set `typesHeader`
+or `sqlTypesHeader`.

Review comment:
       ```suggestion
   `X-Druid-SQL-Header-Included: yes` if `header` was set to true and if the version of Druid the client is connected to
   understands the `typesHeader` and `sqlTypesHeader` parameters. Note that this header is set irrespective of whether `typesHeader`
   or `sqlTypesHeader` are set or not.
   ```

##########
File path: sql/src/main/java/org/apache/druid/sql/http/ObjectWriter.java
##########
@@ -93,4 +96,44 @@ public void close() throws IOException
   {
     jsonGenerator.close();
   }
+
+  static void writeHeader(

Review comment:
       Do we need to keep these new methods static?
   If it's for testing, we could test these by calling the `writeHeader` method provided by the interface `ResultFormat.Writer`.
   
   Same comment in other writer implementations too.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org