You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "sidshas03 (via GitHub)" <gi...@apache.org> on 2026/03/10 14:50:16 UTC

[PR] Fix git-sync rotated credentials support in Helm chart [airflow]

sidshas03 opened a new pull request, #63276:
URL: https://github.com/apache/airflow/pull/63276

   Fixes #63253
   
   Hi team, this PR is to handle token rotation properly when `dags.gitSync.credentialsSecret` is used.
   
   ### Problem
   Right now password/token is passed as env var (`GIT_SYNC_PASSWORD` / `GITSYNC_PASSWORD`).  
   When secret is rotated (for example ESO GitHub App token), running git-sync container does not pick the new value until restart.
   
   ### What I changed
   - Added secret volume mount for git-sync credentials (`git-sync-credentials`) in:
     - scheduler
     - worker
     - triggerer
     - dag-processor
   - Switched password config to file-based envs:
     - `GIT_SYNC_PASSWORD_FILE=/etc/git-secret/credentials/GIT_SYNC_PASSWORD`
     - `GITSYNC_PASSWORD_FILE=/etc/git-secret/credentials/GITSYNC_PASSWORD`
   - Kept username keys from secret as before (`GIT_SYNC_USERNAME` / `GITSYNC_USERNAME`).
   - Updated chart values comments + schema description.
   - Added/updated helm tests for scheduler/worker/triggerer/dag-processor.
   
   ### Why this fix
   git-sync can re-read password from file path on sync loop, so rotated token is picked without forcing pod restart.
   
   ### Testing
   Ran helm unit tests for the updated git-sync paths and related suites (`HELM_TEST_KUBERNETES_VERSION=1.32.8`), and selected tests are passing.
   
   1. Ran focused Helm unit tests for the new credential-file behavior:
      - `test_git_sync_scheduler.py::test_should_set_username_and_password_file_env_variables`
      - `test_git_sync_worker.py::test_should_set_password_file_env_variables_when_credentials_secret_is_configured`
      - `test_git_sync_triggerer.py::test_should_set_password_file_env_variables_when_credentials_secret_is_configured`
      - `test_dag_processor.py::test_should_set_password_file_env_variables_when_credentials_secret_is_configured`
      Result: **4 passed**
   
   2. Ran broader git-sync regression subset:
      - Full files:
        - `helm-tests/tests/helm_tests/other/test_git_sync_scheduler.py`
        - `helm-tests/tests/helm_tests/other/test_git_sync_worker.py`
        - `helm-tests/tests/helm_tests/other/test_git_sync_triggerer.py`
      - Plus dag-processor git-sync checks:
        - `test_validate_if_ssh_params_are_added_with_git_ssh_key`
        - `test_should_set_password_file_env_variables_when_credentials_secret_is_configured`
      Result: **42 passed**
   
   3. Sanity checks:
      - `jq empty chart/values.schema.json` (schema JSON valid)
      - Python compile check for modified tests (`compileall`) passed
   
   Environment note: tests were run with `HELM_TEST_KUBERNETES_VERSION=1.32.8`.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Fix git-sync rotated credentials support in Helm chart [airflow]

Posted by "Miretpl (via GitHub)" <gi...@apache.org>.
Miretpl commented on code in PR #63276:
URL: https://github.com/apache/airflow/pull/63276#discussion_r2914240259


##########
chart/templates/scheduler/scheduler-deployment.yaml:
##########
@@ -344,6 +344,9 @@ spec:
         {{- if or .Values.dags.gitSync.sshKeySecret .Values.dags.gitSync.sshKey}}
           {{- include "git_sync_ssh_key_volume" . | indent 8 }}
         {{- end }}
+        {{- if .Values.dags.gitSync.credentialsSecret }}

Review Comment:
   ```suggestion
           {{- if and $localOrDagProcessorDisabled .Values.dags.gitSync.enabled .Values.dags.gitSync.credentialsSecret }}
   ```
   following git sync container falgs for scheduler



##########
chart/values.schema.json:
##########
@@ -1076,7 +1076,7 @@
                         "tag": {
                             "description": "The gitSync image tag.",
                             "type": "string",
-                            "default": "v4.4.2"
+                            "default": "v4.6.0"

Review Comment:
   I believe this should be in a separate PR as it is unrelated change for this one.



##########
chart/values.schema.json:
##########
@@ -10801,7 +10801,7 @@
                             }
                         },
                         "credentialsSecret": {
-                            "description": "Name of a Secret containing the repo `GIT_SYNC_USERNAME` and `GIT_SYNC_PASSWORD`.",
+                            "description": "Name of a Secret containing `GIT_SYNC_USERNAME`, `GITSYNC_USERNAME`, `GIT_SYNC_PASSWORD`, and `GITSYNC_PASSWORD` keys. The password keys are mounted as files and used via `*_PASSWORD_FILE` env vars so rotated credentials can be re-read.",

Review Comment:
   You probably need to create a new field dedicated for the file as this breaks backward-compatibility.



##########
chart/templates/workers/worker-deployment.yaml:
##########
@@ -482,6 +482,9 @@ spec:
         {{- if or .Values.dags.gitSync.sshKeySecret .Values.dags.gitSync.sshKey}}
           {{- include "git_sync_ssh_key_volume" . | indent 8 }}
         {{- end }}
+        {{- if .Values.dags.gitSync.credentialsSecret }}

Review Comment:
   ```suggestion
           {{- if and .Values.dags.gitSync.enabled (not .Values.dags.persistence.enabled) }}
   ```
   following git sync container falgs for workers



##########
chart/templates/_helpers.yaml:
##########
@@ -247,16 +255,10 @@ If release name contains chart name it will be used as a full name.
         secretKeyRef:
           name: {{ .Values.dags.gitSync.credentialsSecret | quote }}
           key: GITSYNC_USERNAME
-    - name: GIT_SYNC_PASSWORD
-      valueFrom:
-        secretKeyRef:
-          name: {{ .Values.dags.gitSync.credentialsSecret | quote }}
-          key: GIT_SYNC_PASSWORD
-    - name: GITSYNC_PASSWORD
-      valueFrom:
-        secretKeyRef:
-          name: {{ .Values.dags.gitSync.credentialsSecret | quote }}
-          key: GITSYNC_PASSWORD
+    - name: GIT_SYNC_PASSWORD_FILE
+      value: "/etc/git-secret/credentials/GIT_SYNC_PASSWORD"
+    - name: GITSYNC_PASSWORD_FILE
+      value: "/etc/git-secret/credentials/GITSYNC_PASSWORD"

Review Comment:
   This change breaks backward-compatibility. You could prevent that by making if condition on these envs where, by default, old behaviour will be in place.



##########
chart/templates/triggerer/triggerer-deployment.yaml:
##########
@@ -300,6 +300,9 @@ spec:
         {{- if or .Values.dags.gitSync.sshKeySecret .Values.dags.gitSync.sshKey}}
           {{- include "git_sync_ssh_key_volume" . | nindent 8 }}
         {{- end }}
+        {{- if .Values.dags.gitSync.credentialsSecret }}

Review Comment:
   ```suggestion
           {{- if and .Values.dags.gitSync.enabled (not .Values.dags.persistence.enabled) }}
   ```
   following git sync container falgs for triggerer



##########
chart/templates/_helpers.yaml:
##########
@@ -204,6 +204,14 @@ If release name contains chart name it will be used as a full name.
     defaultMode: 288
 {{- end }}
 
+{{/*  Git credentials volume */}}

Review Comment:
   ```suggestion
   {{/* Git credentials volume */}}
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Fix git-sync rotated credentials support in Helm chart [airflow]

Posted by "sidshas03 (via GitHub)" <gi...@apache.org>.
sidshas03 commented on PR #63276:
URL: https://github.com/apache/airflow/pull/63276#issuecomment-4034356548

   Thanks for the detailed review. I got your point on the backward compatibility part, and I will rework this accordingly. I will keep the existing `credentialsSecret` behaviour as it is, and instead add a separate field for the file based password support, so the current contract does not change. I will also remove the git-sync image tag bump from this PR and take it separately. And yes, I will add the corresponding change in the pod-template-file path used by KubernetesExecutor as well. 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Fix git-sync rotated credentials support in Helm chart [airflow]

Posted by "sidshas03 (via GitHub)" <gi...@apache.org>.
sidshas03 closed pull request #63276: Fix git-sync rotated credentials support in Helm chart
URL: https://github.com/apache/airflow/pull/63276


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Fix git-sync rotated credentials support in Helm chart [airflow]

Posted by "electrical (via GitHub)" <gi...@apache.org>.
electrical commented on PR #63276:
URL: https://github.com/apache/airflow/pull/63276#issuecomment-4034859878

   @jscheffl Do you have an estimation on the timeframe for the new Helm chart?
   The current issue that this PR solves is not a major problem but would be nice to have it resolved to avoid crashing containers all the time :)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Fix git-sync rotated credentials support in Helm chart [airflow]

Posted by "electrical (via GitHub)" <gi...@apache.org>.
electrical commented on PR #63276:
URL: https://github.com/apache/airflow/pull/63276#issuecomment-4033076627

   @sidshas03 Airflow currently uses version 4.4.2 of gitsync, according to https://github.com/kubernetes/git-sync/pull/976 the feature required is in 4.6.0?
   I could be missing something :)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Fix git-sync rotated credentials support in Helm chart [airflow]

Posted by "sidshas03 (via GitHub)" <gi...@apache.org>.
sidshas03 commented on PR #63276:
URL: https://github.com/apache/airflow/pull/63276#issuecomment-4034741610

   Thanks for the guidance @jscheffl. I’m parking this PR as discussed. I’m interested to work on the chart v2 follow-up (git-sync >= 4.6.0) as well, please tag me when that issue/epic is created.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Fix git-sync rotated credentials support in Helm chart [airflow]

Posted by "jscheffl (via GitHub)" <gi...@apache.org>.
jscheffl commented on PR #63276:
URL: https://github.com/apache/airflow/pull/63276#issuecomment-4034926896

   @electrical We had initial discussions, are currently collecting topics and will drop a DISCUSS to devlist once we have a (draft) plan.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Fix git-sync rotated credentials support in Helm chart [airflow]

Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on PR #63276:
URL: https://github.com/apache/airflow/pull/63276#issuecomment-4296500870

   @sidshas03 This draft PR has been inactive for 20 days since the last triage comment and no response from the author. Closing to keep the queue clean.
   
   You are welcome to reopen this PR when you resume work, or to open a new one addressing the issues previously raised. There is no rush — take your time.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Fix git-sync rotated credentials support in Helm chart [airflow]

Posted by "jscheffl (via GitHub)" <gi...@apache.org>.
jscheffl commented on PR #63276:
URL: https://github.com/apache/airflow/pull/63276#issuecomment-4034608607

   Note: We are currently planning and discussing a revamt to a chart v2 where we want to re-model some things and also drop some legacy. Would it be OK to "park" this PR, drop all git sync legacy support and use minimum 4.6.0 from then on and have this as a new feature? w/o all the backwards compatatibility complexity?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Fix git-sync rotated credentials support in Helm chart [airflow]

Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk closed pull request #63276: Fix git-sync rotated credentials support in Helm chart
URL: https://github.com/apache/airflow/pull/63276


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Fix git-sync rotated credentials support in Helm chart [airflow]

Posted by "sidshas03 (via GitHub)" <gi...@apache.org>.
sidshas03 commented on PR #63276:
URL: https://github.com/apache/airflow/pull/63276#issuecomment-4033241798

   > @sidshas03 Airflow currently uses version 4.4.2 of gitsync, according to [kubernetes/git-sync#976](https://github.com/kubernetes/git-sync/pull/976) the feature required is in 4.6.0? I could be missing something :)
   
   Good catch, thanks for pointing this out. You’re right, in git-sync v4.4.2, --password-file exists but it is not re-read on every sync. The auto re-read for rotated credentials came in v4.6.0 via kubernetes/git-sync#976. So to make this fix effective by default, we should run git-sync >= v4.6.0.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Fix git-sync rotated credentials support in Helm chart [airflow]

Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on PR #63276:
URL: https://github.com/apache/airflow/pull/63276#issuecomment-4178835681

   @sidshas03 This PR has been converted to **draft** because it does not yet meet our [Pull Request quality criteria](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#pull-request-quality-criteria).
   
   **Issues found:**
   - :x: **Merge conflicts**: This PR has merge conflicts with the `main` branch. Your branch is 715 commits behind `main`. Please rebase your branch (`git fetch origin && git rebase origin/main`), resolve the conflicts, and push again. See [contributing quick start](https://github.com/apache/airflow/blob/main/contributing-docs/03a_contributors_quick_start_beginners.rst).
   
   > **Note:** Your branch is **715 commits behind `main`**. Some check failures may be caused by changes in the base branch rather than by your PR. Please rebase your branch and push again to get up-to-date CI results.
   
   **What to do next:**
   - The comment informs you what you need to do.
   - Fix each issue, then mark the PR as "Ready for review" in the GitHub UI - but only after making sure that all the issues are fixed.
   - There is no rush — take your time and work at your own pace. We appreciate your contribution and are happy to wait for updates.
   - Maintainers will then proceed with a normal review.
   
   Converting a PR to draft is **not** a rejection — it is an invitation to bring the PR up to the project's standards so that maintainer review time is spent productively. There is no rush — take your time and work at your own pace. We appreciate your contribution and are happy to wait for updates. If you have questions, feel free to ask on the [Airflow Slack](https://s.apache.org/airflow-slack).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org