You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@superset.apache.org by "tuttle (via GitHub)" <gi...@apache.org> on 2023/06/20 12:13:07 UTC
[GitHub] [superset] tuttle opened a new issue, #24458: TO_DATE and TO_TIMESTAMP are slow for Citus
tuttle opened a new issue, #24458:
URL: https://github.com/apache/superset/issues/24458
We connect to Citus 10 columnar partitioned table. Superset generates this code for the DATE Temporal column:
```sql
WHERE period >= TO_DATE('2023-03-19', 'YYYY-MM-DD')
AND period < TO_DATE('2023-06-19', 'YYYY-MM-DD')
```
Unfortunately, that is VERY SLOW, because Citus apparently evaluates `TO_DATE()` for every row.
80 seconds in our case.
If Superset had generated
```sql
WHERE period >= '2023-03-19'::date
AND period < '2023-06-19'::date
```
then such query is super fast, only 3 seconds. Both scan 1353 partitions (partitioned by the different column than `period`, btw).
IMHO it is in the function [convert_dttm()](https://github.com/apache/superset/blob/3e76736874f5a7b1ca35a9f3b13cbb4bc3791f69/superset/db_engine_specs/postgres.py#L329). Is there any reason for function call?
I believe it is easily possible to do
```python
return f"'{dttm.date().isoformat()}'::date"
```
instead of generating `TO_DATE()`. Similarly
```python
return f"'{dttm_formatted}'::timestamptz"
```
instead of calling `TO_TIMESTAMP()` below that.
Thanks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [I] TO_DATE and TO_TIMESTAMP are slow for Citus [superset]
Posted by "rusackas (via GitHub)" <gi...@apache.org>.
rusackas closed issue #24458: TO_DATE and TO_TIMESTAMP are slow for Citus
URL: https://github.com/apache/superset/issues/24458
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] tuttle commented on issue #24458: TO_DATE and TO_TIMESTAMP are slow for Citus
Posted by "tuttle (via GitHub)" <gi...@apache.org>.
tuttle commented on issue #24458:
URL: https://github.com/apache/superset/issues/24458#issuecomment-1599062286
An update to show that it's rather not a type issue:
```
# SELECT pg_typeof(TO_DATE('2023-02-19 00:00:00.000000', 'YYYY-MM-DD HH24:MI:SS.US'));
pg_typeof
-----------
date
# SELECT pg_typeof('2023-02-19'::date);
pg_typeof
-----------
date
```
Output of `\d table` shows
```
period | date | | not null |
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [I] TO_DATE and TO_TIMESTAMP are slow for Citus [superset]
Posted by "rusackas (via GitHub)" <gi...@apache.org>.
rusackas commented on issue #24458:
URL: https://github.com/apache/superset/issues/24458#issuecomment-1977331985
I don't know of anyone using this DB to help replicate/validate/diagnose the issue. Closing this as stale since it's been silent for so long, and we're trying to steer toward a more actionable Issues backlog. If you are still encountering this in current versions (currently 3.x) please re-open this issue, open a new Issue with updated context, or raise a PR to address the problem. Thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org