You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2020/04/06 13:05:11 UTC
[GitHub] [airflow] iamshwin opened a new issue #8163: Google `Credentials`
object has not attribute `with_subject`
iamshwin opened a new issue #8163: Google `Credentials` object has not attribute `with_subject`
URL: https://github.com/apache/airflow/issues/8163
**Apache Airflow version**:
master
**What happened**:
When`delegate_to` is supplied and no `key_path` or `keyfile_dict` is supplied (i.e. `google.auth.default()` is called) the `Credentials` object has no attribute `with_subject`.
This is called in `get_credentials_and_project_id` in https://github.com/apache/airflow/blob/master/airflow/providers/google/cloud/utils/credentials_provider.py
`AttributeError: 'Credentials' object has no attribute 'with_subject'`
**What you expected to happen**:
Correct authentication, or raise an Exception that you shouldn't be supplying a `delegate_to` when you haven't supplied a `key_path` or `keyfile_dict`.
**How to reproduce it**:
Supply a `delegate_to` argument in the Operator, but do not supply a `key_path` or `keyfile_dict`.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] mik-laj edited a comment on issue #8163: Google
`Credentials` object has not attribute `with_subject`
Posted by GitBox <gi...@apache.org>.
mik-laj edited a comment on issue #8163: Google `Credentials` object has not attribute `with_subject`
URL: https://github.com/apache/airflow/issues/8163#issuecomment-609843452
I'm afraid this problem may be a little more complex.
``google.auth.default()`` supports 4 methods of authentication methods in following order:
* the credentials from the `GOOGLE_APPLICATION_CREDENTIALS` environment variable.
* the credentials from the Cloud SDK
* Google App Engine App Identity credentials,
* the credentials and project ID from the GCE Metadata Service
Some methods support delegate_to correctly, and others do not support this. Can you tell me how you are authorized to limit the scope of the problem?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] mik-laj commented on issue #8163: Google `Credentials`
object has not attribute `with_subject`
Posted by GitBox <gi...@apache.org>.
mik-laj commented on issue #8163: Google `Credentials` object has not attribute `with_subject`
URL: https://github.com/apache/airflow/issues/8163#issuecomment-610372050
Yes. We should do it. Can you deal with this? I think that if someone passes the delegate_to parameter and the current credentials object does not have the with_subject attribute, an exception should be raised: "The current authentication method does not support account impersonate".
The corresponding code is here:
https://github.com/apache/airflow/blob/beef6c2/airflow/providers/google/cloud/utils/credentials_provider.py#L243-L244
Tests: https://github.com/apache/airflow/blob/8a02402/tests/providers/google/common/hooks/test_base_google.py#L337
Contributor guide: https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst
Development environment: https://github.com/apache/airflow/blob/master/BREEZE.rst
The community is waiting for your contribution 🐈
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] boring-cyborg[bot] commented on issue #8163: Google
`Credentials` object has not attribute `with_subject`
Posted by GitBox <gi...@apache.org>.
boring-cyborg[bot] commented on issue #8163: Google `Credentials` object has not attribute `with_subject`
URL: https://github.com/apache/airflow/issues/8163#issuecomment-609781402
Thanks for opening your first issue here! Be sure to follow the issue template!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] mik-laj edited a comment on issue #8163: Google
`Credentials` object has not attribute `with_subject`
Posted by GitBox <gi...@apache.org>.
mik-laj edited a comment on issue #8163: Google `Credentials` object has not attribute `with_subject`
URL: https://github.com/apache/airflow/issues/8163#issuecomment-610011525
Cloud Composer uses credentials from the metadata server in the client project and credentials from App Engine in the tenant project (webserver).
In the case of metadata server I am sure that delegate cannot be done because the JWT token is not generated. The access token is read from the URL.
http://metadata.google.internal/computeMetadata/v1/instance/service-accounts/default/token using GET. method
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] iamshwin commented on issue #8163: Google `Credentials`
object has not attribute `with_subject`
Posted by GitBox <gi...@apache.org>.
iamshwin commented on issue #8163: Google `Credentials` object has not attribute `with_subject`
URL: https://github.com/apache/airflow/issues/8163#issuecomment-610285308
Ok, so that does that mean that any time someone specifies a `delegate_to` in any Operator in Cloud Composer, the authentication will fail? Should we raise an explicit exception when people try and do that? It would make for a much better user experience.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] mik-laj edited a comment on issue #8163: Google
`Credentials` object has not attribute `with_subject`
Posted by GitBox <gi...@apache.org>.
mik-laj edited a comment on issue #8163: Google `Credentials` object has not attribute `with_subject`
URL: https://github.com/apache/airflow/issues/8163#issuecomment-610011525
Cloud Composer uses credentials from the metadata server in the client project and credentials from App Engine in the tenant project (webserver).
In the case of metadata server I am sure that delegate cannot be done because the JWT token is not generated. The access token is read from the URL.
http://metadata.google.internal/computeMetadata/v1/instance/service-accounts/default/token via GET.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] mik-laj commented on issue #8163: Google `Credentials`
object has not attribute `with_subject`
Posted by GitBox <gi...@apache.org>.
mik-laj commented on issue #8163: Google `Credentials` object has not attribute `with_subject`
URL: https://github.com/apache/airflow/issues/8163#issuecomment-610011525
Cloud Composer uses credentials from the metadata server in the client project and credentials from App Engine in the tenant project (webserver).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] iamshwin commented on issue #8163: Google `Credentials`
object has not attribute `with_subject`
Posted by GitBox <gi...@apache.org>.
iamshwin commented on issue #8163: Google `Credentials` object has not attribute `with_subject`
URL: https://github.com/apache/airflow/issues/8163#issuecomment-610448041
Yes, I'll open a PR.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] mik-laj closed issue #8163: Google `Credentials` object
has not attribute `with_subject`
Posted by GitBox <gi...@apache.org>.
mik-laj closed issue #8163: Google `Credentials` object has not attribute `with_subject`
URL: https://github.com/apache/airflow/issues/8163
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] mik-laj commented on issue #8163: Google `Credentials`
object has not attribute `with_subject`
Posted by GitBox <gi...@apache.org>.
mik-laj commented on issue #8163: Google `Credentials` object has not attribute `with_subject`
URL: https://github.com/apache/airflow/issues/8163#issuecomment-610488982
I'm glad. Thanks in advance. 😻 I am waiting.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] mik-laj edited a comment on issue #8163: Google
`Credentials` object has not attribute `with_subject`
Posted by GitBox <gi...@apache.org>.
mik-laj edited a comment on issue #8163: Google `Credentials` object has not attribute `with_subject`
URL: https://github.com/apache/airflow/issues/8163#issuecomment-610372050
Yes. We should do it. Can you deal with this? I think that if someone passes the delegate_to parameter and the current credentials object does not have the with_subject attribute, an exception should be raised: "The current authentication method does not support account impersonate. Please use service-account for authorization.".
The corresponding code is here:
https://github.com/apache/airflow/blob/beef6c2/airflow/providers/google/cloud/utils/credentials_provider.py#L243-L244
Tests: https://github.com/apache/airflow/blob/8a02402/tests/providers/google/common/hooks/test_base_google.py#L337
Contributor guide: https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst
Development environment: https://github.com/apache/airflow/blob/master/BREEZE.rst
The community is waiting for your contribution 🐈
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] iamshwin commented on issue #8163: Google `Credentials`
object has not attribute `with_subject`
Posted by GitBox <gi...@apache.org>.
iamshwin commented on issue #8163: Google `Credentials` object has not attribute `with_subject`
URL: https://github.com/apache/airflow/issues/8163#issuecomment-610004592
We are running Airflow on Cloud Composer (GCP's managed service). It looks like it is using the credentials from the Cloud SDK.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] mik-laj commented on issue #8163: Google `Credentials`
object has not attribute `with_subject`
Posted by GitBox <gi...@apache.org>.
mik-laj commented on issue #8163: Google `Credentials` object has not attribute `with_subject`
URL: https://github.com/apache/airflow/issues/8163#issuecomment-609843452
I'm afraid this problem may be a little more complex.
``google.auth.default()`` supports 4 methods of authentication methods in following order:
* the credentials from the GOOGLE_APPLICATION_CREDENTIALS environment variable.
* the credentials from the Cloud SDK
* Google App Engine App Identity credentials,
* the credentials and project ID from the GCE Metadata Service
Some methods support delegate_to correctly, and others do not support this. Can you tell me how you are authorized to limit the scope of the problem?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services