You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "ASF GitHub Bot (JIRA)" <ji...@apache.org> on 2018/08/01 19:00:00 UTC
[jira] [Work logged] (AIRFLOW-2661) Kubernetes Executor -
dags_volume_subpath and logs_volume_subpath don't do anything
[ https://issues.apache.org/jira/browse/AIRFLOW-2661?focusedWorklogId=129911&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-129911 ]
ASF GitHub Bot logged work on AIRFLOW-2661:
-------------------------------------------
Author: ASF GitHub Bot
Created on: 01/Aug/18 18:59
Start Date: 01/Aug/18 18:59
Worklog Time Spent: 10m
Work Description: ashb commented on issue #3537: [AIRFLOW-2661] fix k8s config *_volume_subpath
URL: https://github.com/apache/incubator-airflow/pull/3537#issuecomment-409685506
@bolkedebruin Any objections to me cherry-picking this into v1-10-stable and -test? I've tried it on my fork and Travis tests passed:
- https://travis-ci.org/ashb/incubator-airflow/builds/410828962
- https://travis-ci.org/ashb/incubator-airflow/builds/410829126
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Issue Time Tracking
-------------------
Worklog Id: (was: 129911)
Time Spent: 10m
Remaining Estimate: 23h 50m (was: 24h)
> Kubernetes Executor - dags_volume_subpath and logs_volume_subpath don't do anything
> -----------------------------------------------------------------------------------
>
> Key: AIRFLOW-2661
> URL: https://issues.apache.org/jira/browse/AIRFLOW-2661
> Project: Apache Airflow
> Issue Type: Bug
> Components: executor
> Affects Versions: Airflow 2.0
> Environment: Kubernetes 1.10 (kops)
> Reporter: Ravi Kotecha
> Assignee: Ravi Kotecha
> Priority: Major
> Labels: Kubernetes, pull-request-available
> Fix For: 2.0.0
>
> Original Estimate: 24h
> Time Spent: 10m
> Remaining Estimate: 23h 50m
>
> It looks like the configuration options:
> {code:java}
> dags_volume_subpath =
> logs_volume_subpath =
> {code}
> don't do anything when set. This is because the
> {{_constructVolume()}}
> function is called in [https://github.com/apache/incubator-airflow/blob/ef3ead167be9d7dd72f75da9d1f4e00438b8ea6e/airflow/contrib/kubernetes/worker_configuration.py#L124]. It sets subPath on the {{volume}} rather than the mount.
> This function should be changed to not take the optional parameter {{subPath}} and we should use {{dags_volume_subpath}} if provided in the construction of the {{volumeMount}}
> If you look at [https://kubernetes.io/docs/concepts/storage/volumes/#using-subpath] we can see that it's a property of {{volumeMount}} not a {{Volume}}. We are correctly handling this for the the {{configMap}} on line 167.
> My colleague has also commented on the original PR that added this functionality: [https://github.com/apache/incubator-airflow/commit/e30a1f451aa5ec5aca4c886067ba8946a3d33395#r29462135]
> Pull Request:
> https://github.com/apache/incubator-airflow/pull/3537
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)