You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2022/01/26 12:43:32 UTC

[GitHub] [airflow] Skuratovskyi opened a new issue #21125: apache-airflow-providers-trino issue when I use "run" method, query is not performed

Skuratovskyi opened a new issue #21125:
URL: https://github.com/apache/airflow/issues/21125


   ### Apache Airflow version
   
   2.2.3 (latest released)
   
   ### What happened
   
   When I use Trino provider I use "run" method in TrinoHook, because I need run "create table" query.
   But these queries are not performed on Trino server. 
   
   ### What you expected to happen
   
   I expected, that queries that were started from "run" method should be performed)
   
   
   ### How to reproduce
   
   Create DAG file with Trino provider with this code and some CREATE Table hql request.
   
   ```
   ...
       th = TrinoHook()
   
       @task(task_id="create_table")
       def create_table(ds=None, **kwargs):
           th.run("CREATE TABLE IF NOT EXISTS ...", handler=next_page)
   
       cpt = create_table()
   
       cpt
   ```
   This code does not perform operation, but right POST request is sent to trino server.
   
   ### Operating System
   
   Kubuntu, 20.04.3 LTS
   
   ### Versions of Apache Airflow Providers
   
   apache-airflow-providers-trino=2.0.1
   
   ### Deployment
   
   Docker-Compose
   
   ### Deployment details
   
   _No response_
   
   ### Anything else
   
   I think it is needed to add support of handler for run method in provider hook in file airflow/providers/trino/hooks/trino.py
   In this case we get possibility to write handler, that can perform fetchAll() on returned Cursor object, for example
   
   Something like this:
   
   ```
   def run(
               self,
               hql,
               autocommit: bool = False,
               parameters: Optional[dict] = None,
               handler=None
       ) -> None:
           """Execute the statement against Trino. Can be used to create views."""
           return super().run(sql=self._strip_sql(hql), parameters=parameters, handler=handler)
   ```
           
   Because now code does not send handler to dbapi parent method
    
   ` return super().run(sql=self._strip_sql(hql), parameters=parameters)`
   
   ### Are you willing to submit PR?
   
   - [ ] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] kazanzhy commented on issue #21125: apache-airflow-providers-trino issue when I use "run" method, query is not performed

Posted by GitBox <gi...@apache.org>.
kazanzhy commented on issue #21125:
URL: https://github.com/apache/airflow/issues/21125#issuecomment-1032050659


   Seems the passing handler is accidentally missed.
   Talking about `fetchAll(), did you tried the `trino_hook.get_records`?
   It returns `cursor.fetchall()`


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] kazanzhy edited a comment on issue #21125: apache-airflow-providers-trino issue when I use "run" method, query is not performed

Posted by GitBox <gi...@apache.org>.
kazanzhy edited a comment on issue #21125:
URL: https://github.com/apache/airflow/issues/21125#issuecomment-1032050659


   Seems the passing handler is accidentally missed.
   Talking about `fetchAll()`, did you try the `trino_hook.get_records`?
   It returns `cursor.fetchall()`


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] eladkal closed issue #21125: apache-airflow-providers-trino issue when I use "run" method, query is not performed

Posted by GitBox <gi...@apache.org>.
eladkal closed issue #21125:
URL: https://github.com/apache/airflow/issues/21125


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] boring-cyborg[bot] commented on issue #21125: apache-airflow-providers-trino issue when I use "run" method, query is not performed

Posted by GitBox <gi...@apache.org>.
boring-cyborg[bot] commented on issue #21125:
URL: https://github.com/apache/airflow/issues/21125#issuecomment-1022163614


   Thanks for opening your first issue here! Be sure to follow the issue template!
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org