You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2022/03/16 21:51:34 UTC
[GitHub] [airflow] dstandish opened a new pull request #22326: Remove incorrect deprecation warning in secrets backend
dstandish opened a new pull request #22326:
URL: https://github.com/apache/airflow/pull/22326
When the no value is found with `get_conn_value`, the warning was being triggered, even though `get_conn_value` was implemented and just returned no value (cus there wasn't one).
Now we make the logic a little tighter and only raise the dep warning when `get_conn_value` not implemented, which is what we intended to do in the first place.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] eladkal commented on pull request #22326: Remove incorrect deprecation warning in secrets backend
Posted by GitBox <gi...@apache.org>.
eladkal commented on pull request #22326:
URL: https://github.com/apache/airflow/pull/22326#issuecomment-1071080086
> Seems maybe you are using an external secrets backend? I cannot repro. We still need to update other secrets backends. But this fixes the issue where it _yes_ get_conn_value is implemented but just the requested conn isn't there.
Using breeze `./breeze start-airflow --backend postgres --db-reset`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] eladkal commented on pull request #22326: Remove incorrect deprecation warning in secrets backend
Posted by GitBox <gi...@apache.org>.
eladkal commented on pull request #22326:
URL: https://github.com/apache/airflow/pull/22326#issuecomment-1070589354
I still get the warnings for the example in https://github.com/apache/airflow/pull/19857#issuecomment-1069533737
```
[2022-03-17, 08:25:32 UTC] {logging_mixin.py:115} WARNING - /opt/***/***/secrets/base_secrets.py:95 PendingDeprecationWarning: This method is deprecated. Please use `***.secrets.environment_variables.EnvironmentVariablesBackend.get_conn_value`.
[2022-03-17, 08:25:32 UTC] {logging_mixin.py:115} WARNING - /opt/***/***/models/connection.py:420 PendingDeprecationWarning: Method `get_conn_uri` is deprecated. Please use `get_conn_value`.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] dstandish merged pull request #22326: Remove incorrect deprecation warning in secrets backend
Posted by GitBox <gi...@apache.org>.
dstandish merged pull request #22326:
URL: https://github.com/apache/airflow/pull/22326
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on pull request #22326: Remove incorrect deprecation warning in secrets backend
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #22326:
URL: https://github.com/apache/airflow/pull/22326#issuecomment-1071164979
The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and take the risk. If they don't merge it quickly - please rebase it to the latest main at your convenience, or amend the last commit of the PR, and push it with --force-with-lease.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] eladkal commented on pull request #22326: Remove incorrect deprecation warning in secrets backend
Posted by GitBox <gi...@apache.org>.
eladkal commented on pull request #22326:
URL: https://github.com/apache/airflow/pull/22326#issuecomment-1071080086
> Seems maybe you are using an external secrets backend? I cannot repro. We still need to update other secrets backends. But this fixes the issue where it _yes_ get_conn_value is implemented but just the requested conn isn't there.
Using breeze `./breeze start-airflow --backend postgres --db-reset`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] dstandish merged pull request #22326: Remove incorrect deprecation warning in secrets backend
Posted by GitBox <gi...@apache.org>.
dstandish merged pull request #22326:
URL: https://github.com/apache/airflow/pull/22326
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] dstandish commented on pull request #22326: Remove incorrect deprecation warning in secrets backend
Posted by GitBox <gi...@apache.org>.
dstandish commented on pull request #22326:
URL: https://github.com/apache/airflow/pull/22326#issuecomment-1071103444
i ran in breeze... cannot repro. maybe you are on wrong branch or something?
```
root@ac116578f51e:/opt/airflow# python
Python 3.7.12 (default, Mar 2 2022, 05:37:59)
[GCC 10.2.1 20210110] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from airflow.providers.amazon.aws.hooks.s3 import S3Hook
/opt/airflow/airflow/configuration.py:377: FutureWarning: The 'dag_default_view' setting in [webserver] has the old default value of 'tree'. This value has been changed to 'grid' in the running config, but please update your config before Apache Airflow 3.0.
FutureWarning,
/opt/airflow/airflow/configuration.py:377: FutureWarning: The 'log_filename_template' setting in [logging] has the old default value of '{{ ti.dag_id }}/{{ ti.task_id }}/{{ ts }}/{{ try_number }}.log'. This value has been changed to 'dag_id={{ ti.dag_id }}/run_id={{ ti.run_id }}/task_id={{ ti.task_id }}/{%% if ti.map_index >= 0 %%}map_index={{ ti.map_index }}/{%% endif %%}attempt={{ try_number }}.log' in the running config, but please update your config before Apache Airflow 3.0.
FutureWarning,
/opt/airflow/airflow/configuration.py:435: DeprecationWarning: The auth_backend option in [api] has been renamed to auth_backends - the old setting has been used, but please update your config.
option = self._get_option_from_config_file(deprecated_key, deprecated_section, key, kwargs, section)
/opt/airflow/airflow/configuration.py:377: FutureWarning: The 'auth_backends' setting in [api] has the old default value of 'airflow.api.auth.backend.deny_all'. This value has been changed to 'airflow.api.auth.backend.session' in the running config, but please update your config before Apache Airflow 3.0.
FutureWarning,
>>> s3_hook = S3Hook('hello')
>>> s3_hook.load_file(filename='bla', key='blaa', bucket_name='fff')
/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/relationships.py:3463 SAWarning: relationship 'RenderedTaskInstanceFields.dag_run' will copy column rendered_task_instance_fields.dag_id to column dag_run.dag_id, which conflicts with relationship(s): 'BaseXCom.dag_run' (copies xcom.dag_id to dag_run.dag_id). If this is not the intention, consider if these relationships should be linked with back_populates, or if viewonly=True should be applied to one or more if they are read-only. For the less common case that foreign key constraints are partially overlapping, the orm.foreign() annotation can be used to isolate the columns that should be written towards. The 'overlaps' parameter may be used to remove this warning. (Background on this error at: http://sqlalche.me/e/14/qzyx)
/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/relationships.py:3463 SAWarning: relationship 'RenderedTaskInstanceFields.dag_run' will copy column rendered_task_instance_fields.run_id to column dag_run.run_id, which conflicts with relationship(s): 'BaseXCom.dag_run' (copies xcom.run_id to dag_run.run_id). If this is not the intention, consider if these relationships should be linked with back_populates, or if viewonly=True should be applied to one or more if they are read-only. For the less common case that foreign key constraints are partially overlapping, the orm.foreign() annotation can be used to isolate the columns that should be written towards. The 'overlaps' parameter may be used to remove this warning. (Background on this error at: http://sqlalche.me/e/14/qzyx)
[2022-03-17 17:07:57,825] {base_aws.py:447} WARNING - Unable to use Airflow Connection for credentials.
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/opt/airflow/airflow/providers/amazon/aws/hooks/s3.py", line 63, in wrapper
return func(*bound_args.args, **bound_args.kwargs)
File "/opt/airflow/airflow/providers/amazon/aws/hooks/s3.py", line 91, in wrapper
return func(*bound_args.args, **bound_args.kwargs)
File "/opt/airflow/airflow/providers/amazon/aws/hooks/s3.py", line 525, in load_file
if not replace and self.check_for_key(key, bucket_name):
File "/opt/airflow/airflow/providers/amazon/aws/hooks/s3.py", line 63, in wrapper
return func(*bound_args.args, **bound_args.kwargs)
File "/opt/airflow/airflow/providers/amazon/aws/hooks/s3.py", line 91, in wrapper
return func(*bound_args.args, **bound_args.kwargs)
File "/opt/airflow/airflow/providers/amazon/aws/hooks/s3.py", line 363, in check_for_key
self.get_conn().head_object(Bucket=bucket_name, Key=key)
File "/usr/local/lib/python3.7/site-packages/botocore/client.py", line 395, in _api_call
return self._make_api_call(operation_name, kwargs)
File "/usr/local/lib/python3.7/site-packages/botocore/client.py", line 712, in _make_api_call
operation_model, request_dict, request_context)
File "/usr/local/lib/python3.7/site-packages/botocore/client.py", line 731, in _make_request
return self._endpoint.make_request(operation_model, request_dict)
File "/usr/local/lib/python3.7/site-packages/botocore/endpoint.py", line 107, in make_request
return self._send_request(request_dict, operation_model)
File "/usr/local/lib/python3.7/site-packages/botocore/endpoint.py", line 180, in _send_request
request = self.create_request(request_dict, operation_model)
File "/usr/local/lib/python3.7/site-packages/botocore/endpoint.py", line 121, in create_request
operation_name=operation_model.name)
File "/usr/local/lib/python3.7/site-packages/botocore/hooks.py", line 357, in emit
return self._emitter.emit(aliased_event_name, **kwargs)
File "/usr/local/lib/python3.7/site-packages/botocore/hooks.py", line 228, in emit
return self._emit(event_name, kwargs)
File "/usr/local/lib/python3.7/site-packages/botocore/hooks.py", line 211, in _emit
response = handler(**kwargs)
File "/usr/local/lib/python3.7/site-packages/botocore/signers.py", line 93, in handler
return self.sign(operation_name, request)
File "/usr/local/lib/python3.7/site-packages/botocore/signers.py", line 165, in sign
auth.add_auth(request)
File "/usr/local/lib/python3.7/site-packages/botocore/auth.py", line 388, in add_auth
raise NoCredentialsError()
botocore.exceptions.NoCredentialsError: Unable to locate credentials
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] dstandish commented on pull request #22326: Remove incorrect deprecation warning in secrets backend
Posted by GitBox <gi...@apache.org>.
dstandish commented on pull request #22326:
URL: https://github.com/apache/airflow/pull/22326#issuecomment-1071078846
Seems maybe you are using an external secrets backend? I cannot repro. We still need to update other secrets backends. But this fixes the issue where it _yes_ get_conn_value is implemented but just the requested conn isn't there.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on pull request #22326: Remove incorrect deprecation warning in secrets backend
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #22326:
URL: https://github.com/apache/airflow/pull/22326#issuecomment-1071164979
The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and take the risk. If they don't merge it quickly - please rebase it to the latest main at your convenience, or amend the last commit of the PR, and push it with --force-with-lease.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] dstandish commented on pull request #22326: Remove incorrect deprecation warning in secrets backend
Posted by GitBox <gi...@apache.org>.
dstandish commented on pull request #22326:
URL: https://github.com/apache/airflow/pull/22326#issuecomment-1071084383
ok will take a look
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] dstandish commented on pull request #22326: Remove incorrect deprecation warning in secrets backend
Posted by GitBox <gi...@apache.org>.
dstandish commented on pull request #22326:
URL: https://github.com/apache/airflow/pull/22326#issuecomment-1071078846
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org