[GitHub] [superset] kekwan commented on issue #13009: Incorrect SELECT * statement previewing tables with timestamp partitions

[GitBox <gi...@apache.org>]

kekwan commented on issue #13009:
URL: https://github.com/apache/superset/issues/13009#issuecomment-776886300


   Hey @tritonrc. I also fixed the issue but had a slightly different fix for my scenario.
   
   Issue 1 was our Presto `TIMESTAMP` columns were getting marked as `OTHER` (not `TIME` either) due to Trino/Presto's new timestamp precision type. We followed this https://github.com/apache/superset/issues/11138#issuecomment-702475356, to omit timestamp precision and it correctly identified our `TIMESTAMP` columns.
   
   Even this didn't fix our issue with previewing tables however. I believe Presto/Trino doesn't do automatic casting in `WHERE` statements so when the SELECT * is generated for preview tables, it needs to explicitly cast the partition value to 
   `TIMESTAMP`. This is a valid query in Presto (notice the explicit casting of the partition value)
   
   ```
   SELECT "event_type" AS "event_type",
          "trace_id" AS "trace_id",
          "tenant" AS "tenant",
          "core_tenant" AS "core_tenant",
          "event_timestamp" AS "event_timestamp",
          "table_id" AS "table_id",
          "dtr_workspace_id" AS "dtr_workspace_id",
          "connectors" AS "connectors",
          "table_type" AS "table_type",
          "row_count" AS "row_count",
          "field_count" AS "field_count",
          "partition_count" AS "partition_count",
          "data_size_bytes" AS "data_size_bytes",
          "datetimepartition" AS "datetimepartition",
          "etl_ts" AS "etl_ts",
          "insert_try_number" AS "insert_try_number"
   FROM "events"."data_tables"
   WHERE "datetimepartition" = TIMESTAMP '2021-02-08 16:00:00.000'
     AND "etl_ts" = TIMESTAMP '2021-02-08 16:30:00.000'
     AND "insert_try_number" = 1
   LIMIT 100
   ```
   
   In order to do this, I implemented a TypeDecorator class which overrode existing functionality of the SQLA timestamp type and instantiated that class here when the column type is timestamp https://github.com/apache/superset/blob/master/superset/db_engine_specs/presto.py#L917


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org