You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2022/09/22 15:24:32 UTC
[GitHub] [airflow] dinigo opened a new issue, #26600: Google Cloud Batch Jobs Hooks, Operators & Sensors
dinigo opened a new issue, #26600:
URL: https://github.com/apache/airflow/issues/26600
### Description
[Batch Jobs](https://cloud.google.com/batch) were just introduced. They're still in "preview" and only avail in certain regions. But it will be great to support this from GA day 1.
### Use case/motivation
Batch jobs are a very very good alternative to `KubernetesPodOperator` functionality. Fewer configuration, fewer overhead both in the Data Engineers and for the DevOps maintaining the solution.
Also a good solution to decouple orchestration from execution, rather than using —for example— Cloud Build (GCP).
### Related issues
_No response_
### Are you willing to submit a PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] bhirsz commented on issue #26600: Google Cloud Batch Jobs Hooks, Operators & Sensors
Posted by GitBox <gi...@apache.org>.
bhirsz commented on issue #26600:
URL: https://github.com/apache/airflow/issues/26600#issuecomment-1257553131
Thank you, I will pass it to the team that is working on increasing the coverage of the Google Cloud services in the Airflow. Just a note that the service needs a python SDK (ideally) or API, and lack of those can postpone support in the Airflow. And of course since it's open source it can be added by anyone, any contribution is welcome :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] Yaro1 commented on issue #26600: Google Cloud Batch Jobs Hooks, Operators & Sensors
Posted by "Yaro1 (via GitHub)" <gi...@apache.org>.
Yaro1 commented on issue #26600:
URL: https://github.com/apache/airflow/issues/26600#issuecomment-1474873522
please assign to me
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] eladkal commented on issue #26600: Google Cloud Batch Jobs Hooks, Operators & Sensors
Posted by "eladkal (via GitHub)" <gi...@apache.org>.
eladkal commented on issue #26600:
URL: https://github.com/apache/airflow/issues/26600#issuecomment-1684911826
Solved in https://github.com/apache/airflow/pull/32606
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] dinigo commented on issue #26600: Google Cloud Batch Jobs Hooks, Operators & Sensors
Posted by GitBox <gi...@apache.org>.
dinigo commented on issue #26600:
URL: https://github.com/apache/airflow/issues/26600#issuecomment-1258369732
There's both, a REST and gRPC API, and also a [python library](https://github.com/googleapis/python-batch)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] msaad7777 commented on issue #26600: Google Cloud Batch Jobs Hooks, Operators & Sensors
Posted by "msaad7777 (via GitHub)" <gi...@apache.org>.
msaad7777 commented on issue #26600:
URL: https://github.com/apache/airflow/issues/26600#issuecomment-1519286074
Hello! I was wondering if this issue is currently assigned to anyone? If not, I would be interested in taking it on and submitting a PR to address it. Please let me know if it is available to be assigned or if there are any other steps I need to take to get started on this. Thank you!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #26600: Google Cloud Batch Jobs Hooks, Operators & Sensors
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on issue #26600:
URL: https://github.com/apache/airflow/issues/26600#issuecomment-1519527131
> Hello! I was wondering if this issue is currently assigned to anyone? If not, I would be interested in taking it on and submitting a PR to address it. Please let me know if it is available to be assigned or if there are any other steps I need to take to get started on this. Thank you!
You need to talk to @Yaro1 and find out if the work started there is going to be continued (as you can clearly see in the issue - there is a PR in progress) .
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] eladkal closed issue #26600: Google Cloud Batch Jobs Hooks, Operators & Sensors
Posted by "eladkal (via GitHub)" <gi...@apache.org>.
eladkal closed issue #26600: Google Cloud Batch Jobs Hooks, Operators & Sensors
URL: https://github.com/apache/airflow/issues/26600
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org