You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "jonathanleek (via GitHub)" <gi...@apache.org> on 2023/10/16 14:12:24 UTC

[I] Token expiring with deferred SnowflakeOperator [airflow]

jonathanleek opened a new issue, #34968:
URL: https://github.com/apache/airflow/issues/34968

   ### Apache Airflow version
   
   Other Airflow 2 version (please specify below)
   
   ### What happened
   
   In Airflow 2.7.1, it appears that the SnowflakeOperator is not renewing it's token when deferred, causing the task to fail to resume.
   
   ### What you think should happen instead
   
   Token should renew, similar to how the AWSBaseHook does (https://github.com/astronomer/astronomer-providers/issues/1292)
   
   ### How to reproduce
   
   Defer a Snowflake Sensor for an extended period of time.
   
   ### Operating System
   
   linux
   
   ### Versions of Apache Airflow Providers
   
   _No response_
   
   ### Deployment
   
   Other 3rd-party Helm chart
   
   ### Deployment details
   
   _No response_
   
   ### Anything else
   
   _No response_
   
   ### Are you willing to submit PR?
   
   - [X] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] Token expiring with deferred SnowflakeOperator [airflow]

Posted by "boring-cyborg[bot] (via GitHub)" <gi...@apache.org>.
boring-cyborg[bot] commented on issue #34968:
URL: https://github.com/apache/airflow/issues/34968#issuecomment-1764572093

   Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] Token expiring with deferred SnowflakeOperator [airflow]

Posted by "Taragolis (via GitHub)" <gi...@apache.org>.
Taragolis commented on issue #34968:
URL: https://github.com/apache/airflow/issues/34968#issuecomment-1764990241

   > /usr/local/lib/python3.11/site-packages/astronomer/providers/snowflake/operators/snowflake.py"
   
   This provider is not a part of [Community Providers](https://airflow.apache.org/docs/#providers-packagesdocsapache-airflow-providersindexhtml)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] Token expiring with deferred SnowflakeOperator [airflow]

Posted by "Taragolis (via GitHub)" <gi...@apache.org>.
Taragolis closed issue #34968: Token expiring with deferred SnowflakeOperator
URL: https://github.com/apache/airflow/issues/34968


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] Token expiring with deferred SnowflakeOperator [airflow]

Posted by "jonathanleek (via GitHub)" <gi...@apache.org>.
jonathanleek commented on issue #34968:
URL: https://github.com/apache/airflow/issues/34968#issuecomment-1764587048

   `[2023-09-26T13:10:00.446+0000] {snowflake.py:219} INFO - Task deferred
   [2023-09-26T13:10:00.453+0000] {taskinstance.py:1524} INFO - Pausing task as DEFERRED. dag_id=*****, task_id=*******, execution_date=20230925T080000, start_date=20230926T130958`
   
   When it resumed 5 minutes later it failed
   `[2023-09-26T13:15:29.054+0000] {taskinstance.py:1935} ERROR - Task failed with exception
   Traceback (most recent call last):
     File "/usr/local/lib/python3.11/site-packages/airflow/models/baseoperator.py", line 1608, in resume_execution
       return execute_callable(context)
              ^^^^^^^^^^^^^^^^^^^^^^^^^
     File "/usr/local/lib/python3.11/site-packages/astronomer/providers/snowflake/operators/snowflake.py", line 242, in execute_complete
       msg = f"sfquid: {event['query_id']}, {event['type']}: {event['message']}"
                        ~~~~~^^^^^^^^^^^^
   KeyError: 'query_id'`


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org