[GitHub] [druid] gianm commented on a diff in pull request #12770: Type of __time column is determined by RowSignature in case of External Datasource

[GitBox <gi...@apache.org>]

gianm commented on code in PR #12770:
URL: https://github.com/apache/druid/pull/12770#discussion_r919181119


##########
sql/src/main/java/org/apache/druid/sql/calcite/external/ExternalTableMacro.java:
##########
@@ -59,6 +63,21 @@ public TranslatableTable apply(final List<Object> arguments)
       final InputFormat inputFormat = jsonMapper.readValue((String) arguments.get(1), InputFormat.class);
       final RowSignature signature = jsonMapper.readValue((String) arguments.get(2), RowSignature.class);
 
+      // Prevent a RowSignature that has a ColumnSignature with name "__time" and type that is not LONG because it will most
+      // likely  be used in a Calcite function, and cause an exception.
+      // Columns with name __time are automatically assumed to be of the type SqlTypeName.TIMESTAMP because of the logic
+      // present in RowSignatures#toRelDataType
+      // This is not the most optimal approach because of the following caveats:
+      // 1. __time in the external data source may map to a different column
+      // 2. __time in the external data source might be ignored while ingesting the data
+      // 3. Even if __time is of type LONG, it may be used with some function, which would cause the abovementioned Calcite failure
+      // The most optimal solution would be to prevent assumption of the __time column having the type SqlTypeName.TIMESTAMP

Review Comment:
   The idea was that we don't want to use the default conversion for regular Druid tables, since that would be BIGINT, and we actually want TIMESTAMP. This rationale only really applies to regular Druid tables.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org