You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "ASF GitHub Bot (Jira)" <ji...@apache.org> on 2020/03/10 12:57:00 UTC
[jira] [Commented] (AIRFLOW-6542) sparkKubernetes operator for
https://github.com/GoogleCloudPlatform/spark-on-k8s-operator
[ https://issues.apache.org/jira/browse/AIRFLOW-6542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17055913#comment-17055913 ]
ASF GitHub Bot commented on AIRFLOW-6542:
-----------------------------------------
kaxil commented on pull request #7163: [AIRFLOW-6542] add spark-on-k8s operator/hook/sensor
URL: https://github.com/apache/airflow/pull/7163
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
> sparkKubernetes operator for https://github.com/GoogleCloudPlatform/spark-on-k8s-operator
> -----------------------------------------------------------------------------------------
>
> Key: AIRFLOW-6542
> URL: https://issues.apache.org/jira/browse/AIRFLOW-6542
> Project: Apache Airflow
> Issue Type: Improvement
> Components: contrib
> Affects Versions: 1.10.7
> Reporter: Roi Teveth
> Priority: Minor
>
> hi,
> we working on spark on Kubernetes POC using the google cloud platform spark-k8s-operator [https://github.com/GoogleCloudPlatform/spark-on-k8s-operator] and haven't found native airflow integration for it so we wrote one:
> kubernetes_hook which create and get kuberenetes crd object
> spark_kubernetes_operator which sends sparkapplication crd to kubernetes cluster
> spark_kubernetes_sensor which poke sparkapplication state
> operator example with spark-pi application:[https://github.com/roitvt/airflow-spark-on-k8s-operator]
> I'll be glad to contribute our operator to airflow contrib
> Thanks
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)