You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@superset.apache.org by "bionexit (via GitHub)" <gi...@apache.org> on 2024/05/20 12:17:42 UTC

[I] Fatal error while saving query as dataset with Doris and {{ from_dttm }} & {{ to_dttm }} [superset]

bionexit opened a new issue, #28600:
URL: https://github.com/apache/superset/issues/28600

   ### Bug description
   
   Deploy superset with docker compose, and successfully connected Doris. 
   But When I save the query as dataset. An Fatal error showed as failed. 
   
   The following query could be run smoothly. 
   
   ```
   SELECT asset_id,asset_code,asset_type_name,created_date
   FROM doris_asset
   WHERE 1=1 
     {% if filter_values('asset_type_name')|length %}
       AND asset_type_name IN {{ filter_values('asset_type_name')|where_in}}
     {% endif %}
     
     {% if from_dttm is not none %}
        AND created_date > '{{ from_dttm }}' 
     {% endif %}
     
     {% if created_date is not none %}
        AND created_date < '{{ to_dttm }}' 
     {% endif %}
   ```
   also i fill the Template parameters as following
   ```
   {
     "from_dttm": "2020-01-01",
     "to_dttm": "2099-01-01"
   }
   ```
   ![image](https://github.com/apache/superset/assets/40860580/19805704-62ab-47e9-ba48-be12400589d9)
   
   
   
   
   
   ### How to reproduce the bug
   
   1. Run `pip install pydoris` inside the superset_app
   2. Connect a Doris FE.
   3. Go to Sql lab.
   4. create a query with {{ from_dttm }}or {{ to_dttm }}
   5. click save -> save dataset
   6. error occurred.
    
   
   ### Screenshots/recordings
   
   Error log in `sudo docker logs -f --tail=300 superset_app`
   
   
   ```
   2024-05-20 19:57:03,982:DEBUG:superset.sql_parse:Parsing with sqlparse statement: SELECT asset_id,asset_code,asset_type_name,created_date
   FROM doris_asset
   WHERE 1=1
   
   
   
        AND created_date > '{{ from_dttm }}'
   
   
   
        AND created_date < '{{ to_dttm }}'
   2024-05-20 19:57:03,989:DEBUG:superset.stats_logger:[stats_logger] (incr) DatasetRestApi.post.warning
   2024-05-20 19:57:03,989:ERROR:flask_appbuilder.api:(1105, "errCode = 2, detailMessage = Incorrect datetime value: CAST('{{ from_dttm }}' AS DATETIMEV2(0)) in expression: (`created_date` > CAST('{{ from_dttm }}' AS DATETIMEV2(0)))")
   Traceback (most recent call last):
     File "/app/superset/connectors/sqla/utils.py", line 147, in get_columns_description
       cursor.execute(query)
     File "/usr/local/lib/python3.10/site-packages/MySQLdb/cursors.py", line 206, in execute
       res = self._query(query)
     File "/usr/local/lib/python3.10/site-packages/MySQLdb/cursors.py", line 319, in _query
       db.query(q)
     File "/usr/local/lib/python3.10/site-packages/MySQLdb/connections.py", line 254, in query
       _mysql.connection.query(self, query)
   MySQLdb._exceptions.OperationalError: (1105, "errCode = 2, detailMessage = Incorrect datetime value: CAST('{{ from_dttm }}' AS DATETIMEV2(0)) in expression: (`created_date` > CAST('{{ from_dttm }}' AS DATETIMEV2(0)))")
   
   The above exception was the direct cause of the following exception:
   
   Traceback (most recent call last):
     File "/usr/local/lib/python3.10/site-packages/flask_appbuilder/api/__init__.py", line 110, in wraps
       return f(self, *args, **kwargs)
     File "/app/superset/views/base_api.py", line 127, in wraps
       raise ex
     File "/app/superset/views/base_api.py", line 121, in wraps
       duration, response = time_function(f, self, *args, **kwargs)
     File "/app/superset/utils/core.py", line 1463, in time_function
       response = func(*args, **kwargs)
     File "/app/superset/utils/log.py", line 255, in wrapper
       value = f(*args, **kwargs)
     File "/app/superset/views/base_api.py", line 93, in wraps
       return f(self, *args, **kwargs)
     File "/app/superset/datasets/api.py", line 329, in post
       new_model = CreateDatasetCommand(item).run()
     File "/app/superset/commands/dataset/create.py", line 52, in run
       dataset.fetch_metadata(commit=False)
     File "/app/superset/connectors/sqla/models.py", line 1830, in fetch_metadata
       new_columns = self.external_metadata()
     File "/app/superset/connectors/sqla/models.py", line 1322, in external_metadata
       return get_virtual_table_metadata(dataset=self)
     File "/app/superset/connectors/sqla/utils.py", line 132, in get_virtual_table_metadata
       return get_columns_description(dataset.database, dataset.schema, statements[0])
     File "/app/superset/connectors/sqla/utils.py", line 153, in get_columns_description
       raise SupersetGenericDBErrorException(message=str(ex)) from ex
   superset.exceptions.SupersetGenericDBErrorException: (1105, "errCode = 2, detailMessage = Incorrect datetime value: CAST('{{ from_dttm }}' AS DATETIMEV2(0)) in expression: (`created_date` > CAST('{{ from_dttm }}' AS DATETIMEV2(0)))")
   2024-05-20 19:57:03,991:INFO:sqlalchemy.engine.Engine:ROLLBACK
   2024-05-20 19:57:03,991 INFO sqlalchemy.engine.Engine ROLLBACK
   ```
   
   ### Superset version
   
   3.1.3
   
   ### Python version
   
   3.9
   
   ### Node version
   
   I don't know
   
   ### Browser
   
   Chrome
   
   ### Additional context
   
   _No response_
   
   ### Checklist
   
   - [X] I have searched Superset docs and Slack and didn't find a solution to my problem.
   - [X] I have searched the GitHub issue tracker and didn't find a similar bug report.
   - [X] I have checked Superset's logs for errors and if I found a relevant Python stacktrace, I included it here as text in the "additional context" section.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org


Re: [I] Fatal error while saving query as dataset with Doris and {{ from_dttm }} & {{ to_dttm }} [superset]

Posted by "dosubot[bot] (via GitHub)" <gi...@apache.org>.
dosubot[bot] commented on issue #28600:
URL: https://github.com/apache/superset/issues/28600#issuecomment-2120346763

   <!-- Greeting -->
   Hello @bionexit,
   
   I'm [Dosu](https://dosu.dev), a bot here to help you solve bugs, answer your questions, and guide you to becoming a contributor while you wait for a human maintainer. I'm here to assist you!
   
   Let's take a closer look at the issue you're facing with saving queries as datasets in Superset. I'll provide you with a full answer shortly.
   
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org


Re: [I] Fatal error while saving query as dataset with Doris and {{ from_dttm }} & {{ to_dttm }} [superset]

Posted by "rusackas (via GitHub)" <gi...@apache.org>.
rusackas commented on issue #28600:
URL: https://github.com/apache/superset/issues/28600#issuecomment-2305400320

   I don't think this is specifically Doris related, actually. I think this applies to saving queries with Jinja in general.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org


Re: [I] Fatal error while saving query as dataset with Doris and {{ from_dttm }} & {{ to_dttm }} [superset]

Posted by "rusackas (via GitHub)" <gi...@apache.org>.
rusackas closed issue #28600: Fatal error while saving query as dataset with Doris and {{ from_dttm }} & {{ to_dttm }}
URL: https://github.com/apache/superset/issues/28600


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org


Re: [I] Fatal error while saving query as dataset with Doris and {{ from_dttm }} & {{ to_dttm }} [superset]

Posted by "rusackas (via GitHub)" <gi...@apache.org>.
rusackas commented on issue #28600:
URL: https://github.com/apache/superset/issues/28600#issuecomment-2305404848

   Closing as a duplicate of https://github.com/apache/superset/issues/25786... let's take the conversation (and hopefully a solution!) there. 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org


Re: [I] Fatal error while saving query as dataset with Doris and {{ from_dttm }} & {{ to_dttm }} [superset]

Posted by "rusackas (via GitHub)" <gi...@apache.org>.
rusackas commented on issue #28600:
URL: https://github.com/apache/superset/issues/28600#issuecomment-2121015783

   FYI @liujiwen-up, in case you know anything about this :)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org