You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by GitBox <gi...@apache.org> on 2022/04/01 16:16:26 UTC
[GitHub] [beam] ryanthompson591 opened a new pull request #17244: [BEAM-14014] Support impersonation credentials in dataflow runner
ryanthompson591 opened a new pull request #17244:
URL: https://github.com/apache/beam/pull/17244
Adds the ability to set impersonation credentials for jobs running in dataflow.
Adds the ability to set the target_principal and delegate accounts to the authentication singleton.
See the [design doc](https://docs.google.com/document/d/13KRYiq5JAcs-leznzXI_knvqp7ud0u3YASVqK-yMeQw/edit)
------------------------
Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:
- [ ] [**Choose reviewer(s)**](https://beam.apache.org/contribute/#make-your-change) and mention them in a comment (`R: @username`).
- [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
- [ ] Update `CHANGES.md` with noteworthy changes.
- [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
See the [Contributor Guide](https://beam.apache.org/contribute) for more tips on [how to make review process smoother](https://beam.apache.org/contribute/#make-reviewers-job-easier).
To check the build health, please visit [https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md](https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md)
GitHub Actions Tests Status (on master branch)
------------------------------------------------------------------------------------------------
[](https://github.com/apache/beam/actions?query=workflow%3A%22Build+python+source+distribution+and+wheels%22+branch%3Amaster+event%3Aschedule)
[](https://github.com/apache/beam/actions?query=workflow%3A%22Python+Tests%22+branch%3Amaster+event%3Aschedule)
[](https://github.com/apache/beam/actions?query=workflow%3A%22Java+Tests%22+branch%3Amaster+event%3Aschedule)
See [CI.md](https://github.com/apache/beam/blob/master/CI.md) for more information about GitHub Actions CI.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] lukecwik commented on a change in pull request #17244: [BEAM-14014] Support impersonation credentials in dataflow runner
Posted by GitBox <gi...@apache.org>.
lukecwik commented on a change in pull request #17244:
URL: https://github.com/apache/beam/pull/17244#discussion_r840768114
##########
File path: sdks/python/apache_beam/internal/gcp/auth.py
##########
@@ -125,30 +139,22 @@ def get_service_credentials(cls):
"socket default timeout is %s seconds.", socket.getdefaulttimeout())
cls._credentials = cls._get_service_credentials()
+ cls._add_impersonation_credentials()
cls._credentials_init = True
return cls._credentials
@staticmethod
- def _get_service_credentials():
+ def _get_service_credentials(cls):
if not _GOOGLE_AUTH_AVAILABLE:
_LOGGER.warning(
'Unable to find default credentials because the google-auth library '
'is not available. Install the gcp extra (apache_beam[gcp]) to use '
'Google default credentials. Connecting anonymously.')
return None
- client_scopes = [
- 'https://www.googleapis.com/auth/bigquery',
Review comment:
CC: @chamikaramj
Would it make sense to combine the scopes in Java (GcpCredentialFactory.java) and in Python in a properties file or proto enum so that XLang will have the same scopes irregardless of whether the Pipeline is constructed using Python or Java.
We can combine the two lists together. Feel free top create a different JIRA about this and defer to later.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] asf-ci commented on pull request #17244: [BEAM-14014] Support impersonation credentials in dataflow runner
Posted by GitBox <gi...@apache.org>.
asf-ci commented on pull request #17244:
URL: https://github.com/apache/beam/pull/17244#issuecomment-1086104151
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] chamikaramj commented on a change in pull request #17244: [BEAM-14014] Support impersonation credentials in dataflow runner
Posted by GitBox <gi...@apache.org>.
chamikaramj commented on a change in pull request #17244:
URL: https://github.com/apache/beam/pull/17244#discussion_r840775779
##########
File path: sdks/python/apache_beam/internal/gcp/auth.py
##########
@@ -125,30 +139,22 @@ def get_service_credentials(cls):
"socket default timeout is %s seconds.", socket.getdefaulttimeout())
cls._credentials = cls._get_service_credentials()
+ cls._add_impersonation_credentials()
cls._credentials_init = True
return cls._credentials
@staticmethod
- def _get_service_credentials():
+ def _get_service_credentials(cls):
if not _GOOGLE_AUTH_AVAILABLE:
_LOGGER.warning(
'Unable to find default credentials because the google-auth library '
'is not available. Install the gcp extra (apache_beam[gcp]) to use '
'Google default credentials. Connecting anonymously.')
return None
- client_scopes = [
- 'https://www.googleapis.com/auth/bigquery',
Review comment:
+1. That will make things much more consistent for x-lang (and users of different SDKs in general).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org