You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "ASF GitHub Bot (Jira)" <ji...@apache.org> on 2020/08/28 14:58:00 UTC
[jira] [Commented] (AIRFLOW-6542) sparkKubernetes operator for
https://github.com/GoogleCloudPlatform/spark-on-k8s-operator
[ https://issues.apache.org/jira/browse/AIRFLOW-6542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17186591#comment-17186591 ]
ASF GitHub Bot commented on AIRFLOW-6542:
-----------------------------------------
janarthanandev commented on pull request #7163:
URL: https://github.com/apache/airflow/pull/7163#issuecomment-682663370
@roitvt any update on backport package for cncf.kubernetes
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
> sparkKubernetes operator for https://github.com/GoogleCloudPlatform/spark-on-k8s-operator
> -----------------------------------------------------------------------------------------
>
> Key: AIRFLOW-6542
> URL: https://issues.apache.org/jira/browse/AIRFLOW-6542
> Project: Apache Airflow
> Issue Type: Improvement
> Components: contrib
> Affects Versions: 1.10.7
> Reporter: Roi Teveth
> Priority: Minor
> Fix For: 2.0.0
>
>
> hi,
> we working on spark on Kubernetes POC using the google cloud platform spark-k8s-operator [https://github.com/GoogleCloudPlatform/spark-on-k8s-operator] and haven't found native airflow integration for it so we wrote one:
> kubernetes_hook which create and get kuberenetes crd object
> spark_kubernetes_operator which sends sparkapplication crd to kubernetes cluster
> spark_kubernetes_sensor which poke sparkapplication state
> operator example with spark-pi application:[https://github.com/roitvt/airflow-spark-on-k8s-operator]
> I'll be glad to contribute our operator to airflow contrib
> Thanks
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)