You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2023/01/04 16:49:09 UTC

[GitHub] [airflow] minnieshi opened a new issue, #28734: helmChart values.yaml with log persistence enabled follow the documentation cause errors

minnieshi opened a new issue, #28734:
URL: https://github.com/apache/airflow/issues/28734

   ### Official Helm Chart version
   
   1.7.0 (latest released)
   
   ### Apache Airflow version
   
   2.4.1
   
   ### Kubernetes Version
   
   v1.22.6
   
   ### Helm Chart configuration
   
   see below
   
   ### Docker Image customisations
   
   _No response_
   
   ### What happened
   
   Airflow  official chart 1.7 (default airflow app version 2.4.1) onto Kubernetes cluster (created by KinD, also tried in AKS Azure Kubernetes, same results), attempt to enable log persistence like the documentation here: https://airflow.apache.org/docs/helm-chart/stable/manage-logs.html#celery-worker-log-persistence
   says one can enable the log persistence.
   
   details are below:
   
   I had installed airflow see above info, successfully and trying now to upgrade it to use PVC (the SC, PV, PVC have been created successfully, as the state shows Bound) for DAGs and LOGs.
   but the helm upgrade fails with failure message (
   Error: UPGRADE FAILED: cannot patch "airflow-worker" with kind StatefulSet: StatefulSet.apps "airflow-worker" is invalid: spec: Forbidden: updates to statefulset spec for fields other than 'replicas', 'template', 'updateStrategy', 'persistentVolumeClaimRetentionPolicy' and 'minReadySeconds' are forbidden
   )
   the section which are ‘new’ in the values2.yaml is below
   which essentially came from page: https://airflow.apache.org/docs/helm-chart/stable/manage-dags-files.html the mountain DAG option (4th option)
   dags:
     persistence:
       enabled: true
       existingClaim: airflow-dags-pvc
       subPath: ~
   
   logs:
     persistence:
       enabled: true
       existingClaim: airflow-logs-pvc
   I do not really understand why the airflow-worker was a StatefulSet before, and i do not understand why asking for persistence dags, logs would cause this error;
   
   ### What you think should happen instead
   
   _No response_
   
   ### How to reproduce
   
   _No response_
   
   ### Anything else
   
   _No response_
   
   ### Are you willing to submit PR?
   
   - [ ] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] hussein-awala commented on issue #28734: helmChart values.yaml with log persistence enabled follow the documentation cause errors

Posted by GitBox <gi...@apache.org>.
hussein-awala commented on issue #28734:
URL: https://github.com/apache/airflow/issues/28734#issuecomment-1371568027

   This is not a problem in the chart but a restriction in kubernetes, as there are some immutable fields in the statefulset (the kubernetes object used to run ariflow worker when the persistence is enabled).
   To solve the problem, you can delete `airflow-worker` statefulset and reinstall the helm release in order to recreate the statefulset and attach your existing PVC.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] minnieshi commented on issue #28734: helmChart values.yaml with log persistence enabled follow the documentation cause errors

Posted by GitBox <gi...@apache.org>.
minnieshi commented on issue #28734:
URL: https://github.com/apache/airflow/issues/28734#issuecomment-1372381523

   yes, i have solved the issue with my experienced colleague; and i will update the airflow doc to help others in the future.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] minnieshi closed issue #28734: helmChart values.yaml with log persistence enabled follow the documentation cause errors

Posted by GitBox <gi...@apache.org>.
minnieshi closed issue #28734: helmChart values.yaml with log persistence enabled follow the documentation cause errors
URL: https://github.com/apache/airflow/issues/28734


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] minnieshi commented on issue #28734: helmChart values.yaml with log persistence enabled follow the documentation cause errors

Posted by GitBox <gi...@apache.org>.
minnieshi commented on issue #28734:
URL: https://github.com/apache/airflow/issues/28734#issuecomment-1372382040

   close this issue now.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org