You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@airflow.apache.org by Ruslan Dautkhanov <da...@gmail.com> on 2018/04/23 02:28:23 UTC
k8s example DAGs
Is it possible to make kubernetes examples installed optionally?
We don't use Kubernetes and a bare Airflow install fills logs with
following :
2018-04-22 19:49:04,718 ERROR - Failed to import:
> /opt/airflow/airflow-20180420/src/apache-airflow/airflow/
> example_dags/example_kubernetes_operator.py
> Traceback (most recent call last):
> File "/opt/airflow/airflow-20180420/src/apache-airflow/airflow/models.py",
> line 300, in process_file
> m = imp.load_source(mod_name, filepath)
> File "/opt/airflow/airflow-20180420/src/apache-airflow/
> airflow/example_dags/example_kubernetes_operator.py", line 19, in <module>
> from airflow.contrib.operators.kubernetes_pod_operator import
> KubernetesPodOperator
> File "/opt/airflow/airflow-20180420/src/apache-airflow/
> airflow/contrib/operators/kubernetes_pod_operator.py", line 21, in
> <module>
> from airflow.contrib.kubernetes import kube_client, pod_generator,
> pod_launcher
> File "/opt/airflow/airflow-20180420/src/apache-airflow/
> airflow/contrib/kubernetes/pod_launcher.py", line 25, in <module>
> from kubernetes import watch
> ImportError: No module named kubernetes
Would be great to make examples driven by what modules installed if they
have external dependencies,
Thanks!
Ruslan Dautkhanov
Re: k8s example DAGs
Posted by Ruslan Dautkhanov <da...@gmail.com>.
Hi Fokko,
Sure, https://issues.apache.org/jira/browse/AIRFLOW-2358
Thanks!
--
Ruslan Dautkhanov
On Mon, Apr 23, 2018 at 12:41 AM, Driesprong, Fokko <fo...@driesprong.frl>
wrote:
> Hi Ruslan,
>
> This is a good point. I also get No module named kubernetes exceptions when
> running the initdb. This should be fixed. Could you create a Jira ticket
> for this?
>
> Cheers, Fokko
>
> 2018-04-23 4:28 GMT+02:00 Ruslan Dautkhanov <da...@gmail.com>:
>
> > Is it possible to make kubernetes examples installed optionally?
> >
> > We don't use Kubernetes and a bare Airflow install fills logs with
> > following :
> >
> > 2018-04-22 19:49:04,718 ERROR - Failed to import:
> > > /opt/airflow/airflow-20180420/src/apache-airflow/airflow/
> > > example_dags/example_kubernetes_operator.py
> > > Traceback (most recent call last):
> > > File "/opt/airflow/airflow-20180420/src/apache-airflow/
> > airflow/models.py",
> > > line 300, in process_file
> > > m = imp.load_source(mod_name, filepath)
> > > File "/opt/airflow/airflow-20180420/src/apache-airflow/
> > > airflow/example_dags/example_kubernetes_operator.py", line 19, in
> > <module>
> > > from airflow.contrib.operators.kubernetes_pod_operator import
> > > KubernetesPodOperator
> > > File "/opt/airflow/airflow-20180420/src/apache-airflow/
> > > airflow/contrib/operators/kubernetes_pod_operator.py", line 21, in
> > > <module>
> > > from airflow.contrib.kubernetes import kube_client, pod_generator,
> > > pod_launcher
> > > File "/opt/airflow/airflow-20180420/src/apache-airflow/
> > > airflow/contrib/kubernetes/pod_launcher.py", line 25, in <module>
> > > from kubernetes import watch
> > > ImportError: No module named kubernetes
> >
> >
> > Would be great to make examples driven by what modules installed if they
> > have external dependencies,
> >
> >
> > Thanks!
> >
> > Ruslan Dautkhanov
> >
>
Re: k8s example DAGs
Posted by "Driesprong, Fokko" <fo...@driesprong.frl>.
Hi Ruslan,
This is a good point. I also get No module named kubernetes exceptions when
running the initdb. This should be fixed. Could you create a Jira ticket
for this?
Cheers, Fokko
2018-04-23 4:28 GMT+02:00 Ruslan Dautkhanov <da...@gmail.com>:
> Is it possible to make kubernetes examples installed optionally?
>
> We don't use Kubernetes and a bare Airflow install fills logs with
> following :
>
> 2018-04-22 19:49:04,718 ERROR - Failed to import:
> > /opt/airflow/airflow-20180420/src/apache-airflow/airflow/
> > example_dags/example_kubernetes_operator.py
> > Traceback (most recent call last):
> > File "/opt/airflow/airflow-20180420/src/apache-airflow/
> airflow/models.py",
> > line 300, in process_file
> > m = imp.load_source(mod_name, filepath)
> > File "/opt/airflow/airflow-20180420/src/apache-airflow/
> > airflow/example_dags/example_kubernetes_operator.py", line 19, in
> <module>
> > from airflow.contrib.operators.kubernetes_pod_operator import
> > KubernetesPodOperator
> > File "/opt/airflow/airflow-20180420/src/apache-airflow/
> > airflow/contrib/operators/kubernetes_pod_operator.py", line 21, in
> > <module>
> > from airflow.contrib.kubernetes import kube_client, pod_generator,
> > pod_launcher
> > File "/opt/airflow/airflow-20180420/src/apache-airflow/
> > airflow/contrib/kubernetes/pod_launcher.py", line 25, in <module>
> > from kubernetes import watch
> > ImportError: No module named kubernetes
>
>
> Would be great to make examples driven by what modules installed if they
> have external dependencies,
>
>
> Thanks!
>
> Ruslan Dautkhanov
>