You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "gooosetavo (via GitHub)" <gi...@apache.org> on 2026/01/05 16:53:20 UTC

[I] Airflow Helm Chart: Scheduler Should Support Full Probe Configuration (livenessProbe/readinessProbe/readinessProbe) [airflow]

gooosetavo opened a new issue, #60134:
URL: https://github.com/apache/airflow/issues/60134

   ### Official Helm Chart version
   
   1.18.0 (latest released)
   
   ### Apache Airflow version
   
   3.0.2 (affects any)
   
   ### Kubernetes Version
   
   1.34 (affects any)
   
   ### Helm Chart configuration
   
   ```yaml
   scheduler:
     livenessProbe:
       initialDelaySeconds: 20
       periodSeconds: 10
       timeoutSeconds: 5
       failureThreshold: 3
       # Attempt to set a command, or configure as TCP/HTTP (see below attempts)
       # command: ["airflow", "jobs", "check", "--job-type", "SchedulerJob", "--local"]
   # Attempted to add:
   #  readinessProbe:
   #    tcpSocket:
   #      port: 8793
   ```
   
   ### Docker Image customizations
   
   None. Using the official Apache Airflow image.
   
   ### What happened
   
   I attempted to enable and customize both liveness and readiness probes for the scheduler pod. However, the `scheduler` section of the Helm values only permits a very limited set of probe fields. The only keys allowed are the timing parameters and `command` for the liveness probe. Attempts to add typical Kubernetes probe types (like `tcpSocket`, `httpGet`, or even `readinessProbe:` at all) result in schema validation errors during `helm template` or `install`, e.g.:
   
   ```
   Error: values don't meet the specifications of the schema(s) in the following chart(s):
   airflow:
   - scheduler: Additional property readinessProbe is not allowed
   - scheduler.livenessProbe: Additional property tcpSocket is not allowed
   ```
   
   ### What you think should happen instead
   
   Both `livenessProbe` and `readinessProbe` should support the full set of [Kubernetes probe options](https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes/) (such as `exec`, `tcpSocket`, `httpGet`, etc.) for the scheduler, as is already the case with other components in the chart (like workers and webserver). Ideally, there should also be a `readinessProbe` supported for `scheduler` the same way as with the other pods.
   
   
   ### How to reproduce
   
   1. Try to set a readiness probe for the scheduler in `values.yaml`:
       ```yaml
       scheduler:
         readinessProbe:
           tcpSocket:
             port: 8793
       ```
   2. Or try to set a non-default liveness probe for the scheduler:
       ```yaml
       scheduler:
         livenessProbe:
           tcpSocket:
             port: 8793
           initialDelaySeconds: 10
       ```
   3. Run:
       ```bash
       helm template my-airflow apache-airflow/airflow -f values.yaml
       ```
   4. You get schema validation errors:
       ```
       scheduler: Additional property readinessProbe is not allowed
       scheduler.livenessProbe: Additional property tcpSocket is not allowed
       ```
   
   ### Anything else
   
   Every time I try to set scheduler probe fields beyond the very limited default, I hit this error. The issue occurs consistently on new chart pulls as well.
   
   
   ### Are you willing to submit PR?
   
   - [x] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] Airflow Helm Chart: Scheduler Should Support Full Probe Configuration (livenessProbe/readinessProbe/readinessProbe) [airflow]

Posted by "gooosetavo (via GitHub)" <gi...@apache.org>.
gooosetavo commented on issue #60134:
URL: https://github.com/apache/airflow/issues/60134#issuecomment-3774119770

   @GaneshPatil7517 I can confirm your PR fixes the issue!
   
   Here's a test script you can run in the project root to test this: 
   ```bash
   #!/bin/bash
   
   
   CHART_DIR="$(dirname "$0")/chart"
   
   run_probe_check() {
     local description="$1"
     local helm_args="$2"
     local probe_name="$3"
     echo "=== $description ==="
     # Capture both output and error
     output_and_err=$(helm template airflow "$CHART_DIR" -s templates/scheduler/scheduler-deployment.yaml $helm_args 2>&1)
     probe_output=$(echo "$output_and_err" | grep -A10 "$probe_name:")
     if echo "$output_and_err" | grep -q "Error:"; then
       echo "$output_and_err"
     elif [ -n "$probe_output" ]; then
       echo "$probe_output"
     else
       echo "<no output>"
     fi
   }
   
   run_probe_check "livenessProbe enabled" "" "livenessProbe"
   run_probe_check "livenessProbe disabled" "--set scheduler.livenessProbe.enabled=false" "livenessProbe"
   run_probe_check "readinessProbe enabled" "--set scheduler.readinessProbe.enabled=true" "readinessProbe"
   run_probe_check "readinessProbe disabled" "--set scheduler.readinessProbe.enabled=false" "readinessProbe"
   run_probe_check "startupProbe enabled" "" "startupProbe"
   run_probe_check "startupProbe disabled" "--set scheduler.startupProbe.enabled=false" "startupProbe"
   ```
   output from your branch:
   ```
   === livenessProbe enabled ===
             livenessProbe:
               initialDelaySeconds: 10
               timeoutSeconds: 20
               failureThreshold: 5
               periodSeconds: 60
               exec:
                 command:              
                   - sh
                   - -c
                   - |
                     CONNECTION_CHECK_MAX_COUNT=0 AIRFLOW__LOGGING__LOGGING_LEVEL=ERROR exec /entrypoint \
   === livenessProbe disabled ===
   <no output>
   === readinessProbe enabled ===
             readinessProbe:
               initialDelaySeconds: 10
               timeoutSeconds: 20
               failureThreshold: 5
               periodSeconds: 60
               exec:
                 command:              
                   - sh
                   - -c
                   - |
                     CONNECTION_CHECK_MAX_COUNT=0 AIRFLOW__LOGGING__LOGGING_LEVEL=ERROR exec /entrypoint \
   === readinessProbe disabled ===
   <no output>
   === startupProbe enabled ===
             startupProbe:
               initialDelaySeconds: 0
               timeoutSeconds: 20
               failureThreshold: 6
               periodSeconds: 10
               exec:
                 command:              
                   - sh
                   - -c
                   - |
                     CONNECTION_CHECK_MAX_COUNT=0 AIRFLOW__LOGGING__LOGGING_LEVEL=ERROR exec /entrypoint \
   === startupProbe disabled ===
   <no output>
   ```
   
   output from main (schema doesn't permit setting some of these values):
   ```
   === livenessProbe enabled ===
             livenessProbe:
               initialDelaySeconds: 10
               timeoutSeconds: 20
               failureThreshold: 5
               periodSeconds: 60
               exec:
                 command:              
                   - sh
                   - -c
                   - |
                     CONNECTION_CHECK_MAX_COUNT=0 AIRFLOW__LOGGING__LOGGING_LEVEL=ERROR exec /entrypoint \
   === livenessProbe disabled ===
   Error: values don't meet the specifications of the schema(s) in the following chart(s):
   airflow:
   - at '/scheduler/livenessProbe': additional properties 'enabled' not allowed
   === readinessProbe enabled ===
   Error: values don't meet the specifications of the schema(s) in the following chart(s):
   airflow:
   - at '/scheduler': additional properties 'readinessProbe' not allowed
   === readinessProbe disabled ===
   Error: values don't meet the specifications of the schema(s) in the following chart(s):
   airflow:
   - at '/scheduler': additional properties 'readinessProbe' not allowed
   === startupProbe enabled ===
             startupProbe:
               initialDelaySeconds: 0
               timeoutSeconds: 20
               failureThreshold: 6
               periodSeconds: 10
               exec:
                 command:              
                   - sh
                   - -c
                   - |
                     CONNECTION_CHECK_MAX_COUNT=0 AIRFLOW__LOGGING__LOGGING_LEVEL=ERROR exec /entrypoint \
   === startupProbe disabled ===
   Error: values don't meet the specifications of the schema(s) in the following chart(s):
   airflow:
   - at '/scheduler/startupProbe': additional properties 'enabled' not allowed
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] Airflow Helm Chart: Scheduler Should Support Full Probe Configuration (livenessProbe/readinessProbe/readinessProbe) [airflow]

Posted by "GaneshPatil7517 (via GitHub)" <gi...@apache.org>.
GaneshPatil7517 commented on issue #60134:
URL: https://github.com/apache/airflow/issues/60134#issuecomment-3720556077

   hey @gooosetavo can you assign me this issue i want to work on this.....


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] Airflow Helm Chart: Scheduler Should Support Full Probe Configuration (livenessProbe/readinessProbe/readinessProbe) [airflow]

Posted by "GaneshPatil7517 (via GitHub)" <gi...@apache.org>.
GaneshPatil7517 commented on issue #60134:
URL: https://github.com/apache/airflow/issues/60134#issuecomment-3776158471

   Hi @gooosetavo, thank you for taking the time to test and verify the fix! I appreciate the detailed test script - it's really helpful for validation.
   
   I've resolved the merge conflict with upstream main. The PR should now be ready for review and merge. Let me know if there's anything else needed...


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] Airflow Helm Chart: Scheduler Should Support Full Probe Configuration (livenessProbe/readinessProbe/readinessProbe) [airflow]

Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on issue #60134:
URL: https://github.com/apache/airflow/issues/60134#issuecomment-3986417577

   @GaneshPatil7517 We are unassigning you from this issue as part of our updated [assignment policy](https://github.com/apache/airflow/blob/main/contributing-docs/04_how_to_contribute.rst#contribute-code-changes).
   
   This is not meant to discourage your contribution — quite the opposite! You are still very welcome to work on this issue and submit a PR for it. Simply comment that you are working on it and open a PR when ready.
   
   We found that formal assignments were not working well, as they often prevented others from contributing when the assignee was not actively working on the issue.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] Airflow Helm Chart: Scheduler Should Support Full Probe Configuration (livenessProbe/readinessProbe/readinessProbe) [airflow]

Posted by "Yassen717 (via GitHub)" <gi...@apache.org>.
Yassen717 commented on issue #60134:
URL: https://github.com/apache/airflow/issues/60134#issuecomment-6242203013

   Working on this — opened #74934 with a focused implementation: adds \`readinessProbe\` support, \`enabled\` flags, and \`httpGet\`/\`tcpSocket\`/\`grpc\` handlers for all scheduler probes, plus helm unit tests. Default rendering is unchanged.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] Airflow Helm Chart: Scheduler Should Support Full Probe Configuration (livenessProbe/readinessProbe/readinessProbe) [airflow]

Posted by "boring-cyborg[bot] (via GitHub)" <gi...@apache.org>.
boring-cyborg[bot] commented on issue #60134:
URL: https://github.com/apache/airflow/issues/60134#issuecomment-3711271238

   Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] Airflow Helm Chart: Scheduler Should Support Full Probe Configuration (livenessProbe/readinessProbe/readinessProbe) [airflow]

Posted by "GaneshPatil7517 (via GitHub)" <gi...@apache.org>.
GaneshPatil7517 commented on issue #60134:
URL: https://github.com/apache/airflow/issues/60134#issuecomment-3728628181

   hey @gooosetavo & @potiuk Can you review this: https://github.com/apache/airflow/pull/60321


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org