You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2020/08/26 06:36:28 UTC
[GitHub] [airflow] aimran-adroll opened a new issue #10574: Dag in existing persistent volume claim not registering
aimran-adroll opened a new issue #10574:
URL: https://github.com/apache/airflow/issues/10574
<!--
Welcome to Apache Airflow! For a smooth issue process, try to answer the following questions.
Don't worry if they're not all applicable; just try to include what you can :-)
If you need to include code snippets or logs, please put them in fenced code
blocks. If they're super-long, please use the details tag like
<details><summary>super-long log</summary> lots of stuff </details>
Please delete these comment blocks before submitting the issue.
-->
<!--
IMPORTANT!!!
PLEASE CHECK "SIMILAR TO X EXISTING ISSUES" OPTION IF VISIBLE
NEXT TO "SUBMIT NEW ISSUE" BUTTON!!!
PLEASE CHECK IF THIS ISSUE HAS BEEN REPORTED PREVIOUSLY USING SEARCH!!!
Please complete the next sections or the issue will be closed.
This questions are the first thing we need to know to understand the context.
-->
**Apache Airflow version**: v1.10.10 (server) / master for chart
**Kubernetes version (if you are using kubernetes)**: 1.18.6 (client) / v1.17.9-eks-4c6976
**Environment**:
- **Cloud provider or hardware configuration**: AWS EKS
- **OS** (e.g. from /etc/os-release):
- **Kernel** (e.g. `uname -a`):
- **Install tools**:
- **Others**:
**What happened**: Dag in existing `persistent` volume claim is not showing up
I am a complete beginner with airflow. I am using the chart from the repo and an official image (see reproduce below)
Trying to understand why the asymmetry between the two files:
1. chart/templates/scheduler/scheduler-deployment.yaml
2. chart/templates/webserver/webserver-deployment.yaml
File (2) has a logic around `persistence.enabled` for the volumeMount while (1) does not (as of master).
If I use the following patch, it seems to register my dags ✅
```
diff --git a/chart/templates/scheduler/scheduler-deployment.yaml b/chart/templates/scheduler/scheduler-deployment.yaml
index 933155687..1c48ef722 100644
--- a/chart/templates/scheduler/scheduler-deployment.yaml
+++ b/chart/templates/scheduler/scheduler-deployment.yaml
@@ -145,7 +145,7 @@ spec:
subPath: airflow_local_settings.py
readOnly: true
{{- end }}
-{{- if .Values.dags.gitSync.enabled }}
+{{- if or .Values.dags.gitSync.enabled .Values.dags.persistence.enabled }}
- name: dags
mountPath: {{ template "airflow_dags_mount_path" . }}
{{- include "git_sync_container" . | indent 8 }}
```
<!-- (please include exact error messages if you can) -->
**What you expected to happen**:
I would expect the pvc to be mounted along with my dags. At first I assumed there is some fancy back-n-forth happening between webserver and scheduler. But I waited and no Dags showed up in the UI.
<!-- What do you think went wrong? -->
**How to reproduce it**:
## install helm chart from apache/airflow
```
helm repo add stable https://kubernetes-charts.storage.googleapis.com
helm dep update
```
## install release
```
helm install airflow . --namespace airflow-dev \
--set defaultAirflowTag=v1-10-test-python3.7 \
--set dags.persistence.enabled=true \
--set dags.persistence.existingClaim=airflow-dag-pvc \
--set dags.gitSync.enabled=false
```
**Anything else we need to know**:
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on issue #10574: Dag in existing persistent volume claim not registering
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on issue #10574:
URL: https://github.com/apache/airflow/issues/10574#issuecomment-904217119
This issue has been closed because it has not received response from the issue author.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] aimran-adroll commented on issue #10574: Dag in existing persistent volume claim not registering
Posted by GitBox <gi...@apache.org>.
aimran-adroll commented on issue #10574:
URL: https://github.com/apache/airflow/issues/10574#issuecomment-683337376
#10630 may be the fix. Have to test it though
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] boring-cyborg[bot] commented on issue #10574: Dag in existing persistent volume claim not registering
Posted by GitBox <gi...@apache.org>.
boring-cyborg[bot] commented on issue #10574:
URL: https://github.com/apache/airflow/issues/10574#issuecomment-680688882
Thanks for opening your first issue here! Be sure to follow the issue template!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] eladkal commented on issue #10574: Dag in existing persistent volume claim not registering
Posted by GitBox <gi...@apache.org>.
eladkal commented on issue #10574:
URL: https://github.com/apache/airflow/issues/10574#issuecomment-859506052
@aimran-adroll is this still an issue?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] closed issue #10574: Dag in existing persistent volume claim not registering
Posted by GitBox <gi...@apache.org>.
github-actions[bot] closed issue #10574:
URL: https://github.com/apache/airflow/issues/10574
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on issue #10574: Dag in existing persistent volume claim not registering
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on issue #10574:
URL: https://github.com/apache/airflow/issues/10574#issuecomment-878677118
This issue has been automatically marked as stale because it has been open for 30 days with no response from the author. It will be closed in next 7 days if no further activity occurs from the issue author.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org