You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@airflow.apache.org by Eamon Keane <ea...@gmail.com> on 2019/09/01 16:39:34 UTC

Re: Specify volume for Kubernetes Executor created workers

Arbitrary mounts not implemented currently I don't think. However look at
the pod mutation hook - it should allow you to inject a well-known PV for a
given dag run.

https://github.com/apache/airflow/pull/5359
https://github.com/pgagnon/airflow/blob/0da3f6e3c4762c1d7376c716a6f1ba375e6513e5/airflow/settings.py#L102

On Sat, Aug 31, 2019 at 7:15 PM Sourabh Bajaj <so...@neeva.co> wrote:

> Hi,
>
> I'm trying to run Airflow with the new Kubernetes Executor and was trying
> to have each worker that is started attach a persistant volume they can use
> for saving intermediate results.
>
> I could not find how to pass this in the config when this is neither dags
> or logs. One workaround I had was to use the logs volume config since I've
> remote logging enabled but not sure if that'll have other side effects.
>
> -Sourabh
>