You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@airflow.apache.org by Pras Srinivasan <pr...@glassdoor.com> on 2017/09/01 01:03:34 UTC
User delegation does not work on current GoogleCloudBaseHook
I'm upgrading from airflow 1.7 to 1.8.2rc4. I noticed that the user
delegation feature does not work for service accounts when inheriting from
GoogleCloudBaseHook anymore .
Older versions of this hook used to support delegation when
SignedJwtAssertionCredentials was being used. Actually, the current code in
master still has some code left over from when
SignedJwtAssertionCredentials was being used. Specifically these lines
(#68-#70) in gcp_api_base_hook.py :
kwargs = {}
if self.delegate_to:
kwargs['sub'] = self.delegate_to
However, this information is not used anywhere and the _authorize method
simply returns a HTTP object without allowing for delegation.
I think the changes that need to be made are:
1) Remove lines 68-70
2) Add a couple of lines after line #83 that enable returning a delegated
credential object :
if self.delegate_to:
credentials = credentials.create_delegated(self.delegate_to)
Can another dev please review/confirm that my understanding is correct? I'm
happy to open a JIRA on Apache, as well as submit the fix.
Thanks much!
Pras
Re: User delegation does not work on current GoogleCloudBaseHook
Posted by Pras Srinivasan <pr...@glassdoor.com>.
Thank you @fenglu-g!
I agree, Google does not make it clear why they made a choice to move away
from implementing delegation while credentials are being inited in the
python. Especially when the java version still supports it. Though, I can
see how by making it a method, a dev can now reuse the same credentials
object to delegate between multiple users during the same session.
On Thu, Aug 31, 2017 at 11:07 PM, Feng Lu <fe...@google.com.invalid> wrote:
> That looks right to me.
>
> Unfortunately Python client lib, unlike the java client lib
> <https://developers.google.com/api-client-library/java/
> google-api-java-client/reference/1.19.1/com/google/
> api/client/googleapis/auth/oauth2/GoogleCredential>,
> doesn't support generating GoogleCredentials while impersonating another
> user/service account.
> Otherwise, the code can be much simplified and we only need to deal with
> GoogleCrentials.
>
> Happy to take a look at your PR too, just @fenglu-g.
>
> On Thu, Aug 31, 2017 at 6:03 PM, Pras Srinivasan <
> pras.srinivasan@glassdoor.com> wrote:
>
> > I'm upgrading from airflow 1.7 to 1.8.2rc4. I noticed that the user
> > delegation feature does not work for service accounts when inheriting
> from
> > GoogleCloudBaseHook anymore .
> >
> > Older versions of this hook used to support delegation when
> > SignedJwtAssertionCredentials was being used. Actually, the current code
> in
> > master still has some code left over from when
> > SignedJwtAssertionCredentials was being used. Specifically these lines
> > (#68-#70) in gcp_api_base_hook.py :
> >
> > kwargs = {}
> > if self.delegate_to:
> > kwargs['sub'] = self.delegate_to
> >
> > However, this information is not used anywhere and the _authorize method
> > simply returns a HTTP object without allowing for delegation.
> >
> > I think the changes that need to be made are:
> > 1) Remove lines 68-70
> > 2) Add a couple of lines after line #83 that enable returning a delegated
> > credential object :
> > if self.delegate_to:
> > credentials = credentials.create_delegated(self.delegate_to)
> >
> > Can another dev please review/confirm that my understanding is correct?
> I'm
> > happy to open a JIRA on Apache, as well as submit the fix.
> >
> > Thanks much!
> > Pras
> >
>
--
Pras
Re: User delegation does not work on current GoogleCloudBaseHook
Posted by Feng Lu <fe...@google.com.INVALID>.
That looks right to me.
Unfortunately Python client lib, unlike the java client lib
<https://developers.google.com/api-client-library/java/google-api-java-client/reference/1.19.1/com/google/api/client/googleapis/auth/oauth2/GoogleCredential>,
doesn't support generating GoogleCredentials while impersonating another
user/service account.
Otherwise, the code can be much simplified and we only need to deal with
GoogleCrentials.
Happy to take a look at your PR too, just @fenglu-g.
On Thu, Aug 31, 2017 at 6:03 PM, Pras Srinivasan <
pras.srinivasan@glassdoor.com> wrote:
> I'm upgrading from airflow 1.7 to 1.8.2rc4. I noticed that the user
> delegation feature does not work for service accounts when inheriting from
> GoogleCloudBaseHook anymore .
>
> Older versions of this hook used to support delegation when
> SignedJwtAssertionCredentials was being used. Actually, the current code in
> master still has some code left over from when
> SignedJwtAssertionCredentials was being used. Specifically these lines
> (#68-#70) in gcp_api_base_hook.py :
>
> kwargs = {}
> if self.delegate_to:
> kwargs['sub'] = self.delegate_to
>
> However, this information is not used anywhere and the _authorize method
> simply returns a HTTP object without allowing for delegation.
>
> I think the changes that need to be made are:
> 1) Remove lines 68-70
> 2) Add a couple of lines after line #83 that enable returning a delegated
> credential object :
> if self.delegate_to:
> credentials = credentials.create_delegated(self.delegate_to)
>
> Can another dev please review/confirm that my understanding is correct? I'm
> happy to open a JIRA on Apache, as well as submit the fix.
>
> Thanks much!
> Pras
>