You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2022/03/06 02:03:50 UTC
[GitHub] [airflow] ajayganti3 opened a new issue #22019: Google cloud dataproc: Add an operator to run a serverless spark in dataproc
ajayganti3 opened a new issue #22019:
URL: https://github.com/apache/airflow/issues/22019
### Description
Recently GCP has launched serverless spark on dataproc. It will be really great if Airflow has an operator to submit a serverless spark job. Please refer to the [dataproc serverless documentation](https://cloud.google.com/dataproc-serverless/docs/overview) and its [gloud sdk documentation](https://cloud.google.com/sdk/gcloud/reference/dataproc/batches)
### Use case/motivation
Using this serverless dataproc we can run spark jobs easily without managing any infrastructure, autoscaling and work loads.
### Related issues
_No response_
### Are you willing to submit a PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] aoelvp94 commented on issue #22019: Google cloud dataproc: Add an operator to run a serverless spark in dataproc
Posted by GitBox <gi...@apache.org>.
aoelvp94 commented on issue #22019:
URL: https://github.com/apache/airflow/issues/22019#issuecomment-1066741757
DId you see this PR https://github.com/apache/airflow/pull/19248?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk closed issue #22019: Google cloud dataproc: Add an operator to run a serverless spark in dataproc
Posted by GitBox <gi...@apache.org>.
potiuk closed issue #22019:
URL: https://github.com/apache/airflow/issues/22019
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #22019: Google cloud dataproc: Add an operator to run a serverless spark in dataproc
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #22019:
URL: https://github.com/apache/airflow/issues/22019#issuecomment-1068201478
Closing as implemented in #19248
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] boring-cyborg[bot] commented on issue #22019: Google cloud dataproc: Add an operator to run a serverless spark in dataproc
Posted by GitBox <gi...@apache.org>.
boring-cyborg[bot] commented on issue #22019:
URL: https://github.com/apache/airflow/issues/22019#issuecomment-1059877148
Thanks for opening your first issue here! Be sure to follow the issue template!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org