You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2021/10/17 00:36:03 UTC

[GitHub] [airflow] kapopken opened a new issue #19007: argument execution_date: invalid parse value

kapopken opened a new issue #19007:
URL: https://github.com/apache/airflow/issues/19007


   ### Apache Airflow version
   
   2.2.0 (latest released)
   
   ### Operating System
   
   apache/airflow:2.2.0-python3.8
   
   ### Versions of Apache Airflow Providers
   
   ```
   RUN pip install --user Flask-OpenID && \
       pip install --user authlib && \
       pip install --user cryptography && \
       pip install --user apache-airflow[crypto] && \
       pip install --user apache-airflow['wasb'] && \
       pip install --user azure-storage-blob===12.7.1 && \
       pip install --user lib && \
       pip install --user apache-airflow['kubernetes'] && \
       pip install --user pyodbc===4.0.30 && \
       pip install --user pyrecord==1.0.1 && \
       pip install --user requests===2.25.1 && \
       pip install --user plyvel
   ```
   
   ### Deployment
   
   Other 3rd-party Helm chart
   
   ### Deployment details
   
   Deploy to kubernetes using a custom helm chart. 
   Using a pod template for the jobs themselves
   ```
   apiVersion: v1
   kind: Pod
   metadata:
     name: dummy-name # required by airflow. Will be replace at run time
     labels:
       aadpodidbinding: airflow
   spec:
     serviceAccountName: bm-airflow-service
     restartPolicy: Never
     containers:
       - name: base # required by airflow. Will be replace at run time
         volumeMounts:
           - name: bm
             mountPath: /opt/airflow/bm
           - name: kv-secrets
             mountPath: "/mnt/secrets-store"
             readOnly: true
         envFrom:
           - configMapRef:
               name: bm-airflow
           - secretRef:
               name: bm-airflow-secrets
     volumes:
       - name: bm
         persistentVolumeClaim:
           claimName: bm-airflow-pvc
       - name: kv-secrets
         csi:
           driver: secrets-store.csi.k8s.io
           readOnly: true
           volumeAttributes:
             secretProviderClass: bmcl-service-airflow-bmcl-airflow-kv 
   ```
   
   ### What happened
   
   Upgraded to 2.2.0 and suddenly received this error
   airflow tasks run command error: argument execution_date: invalid parse value: 'scheduled__2021-10-15T10:53:49.851441+00:00', see help above.
   
   rendered template is
   ```
     containers:
       - name: base
         image: 'opsbmclacr.azurecr.io/bm-airflow:latest'
         args:
           - airflow
           - tasks
           - run
           - tutorial
           - print_date
           - 'scheduled__2021-10-14T11:15:54.266329+00:00'
           - '--local'
           - '--subdir'
           - DAGS_FOLDER/sample.py
   ```
   
   ### What you expected to happen
   
   The dag should run
   
   ### How to reproduce
   
   I am using this dag: https://airflow.apache.org/docs/apache-airflow/stable/_modules/airflow/example_dags/tutorial.html
   
   run it on version 2.2.0 that was upgraded from an older version of airflow. Trigger task
   
   ### Anything else
   
   Happens every time. No dags are runnable
   
   ### Are you willing to submit PR?
   
   - [X] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [airflow] ephraimbuddy commented on issue #19007: argument execution_date: invalid parse value

Posted by GitBox <gi...@apache.org>.
ephraimbuddy commented on issue #19007:
URL: https://github.com/apache/airflow/issues/19007#issuecomment-983574331


   It seems your worker is still using an old image instead of the one you upgraded to. Make sure your workers are using the new image


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [airflow] ptrhck commented on issue #19007: argument execution_date: invalid parse value

Posted by GitBox <gi...@apache.org>.
ptrhck commented on issue #19007:
URL: https://github.com/apache/airflow/issues/19007#issuecomment-983544526


   How can I check if the dagrun exist? I can see it in the UI as "up_for_retry".
   
   In addition, I have seen in the worker logs this on startup:
   
   `alembic.script.revision.ResolutionError: No such revision or branch '7b2661a43ba3'
   ...
   alembic.util.exc.CommandError: Can't locate revision identified by '7b2661a43ba3'` 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [airflow] ptrhck edited a comment on issue #19007: argument execution_date: invalid parse value

Posted by GitBox <gi...@apache.org>.
ptrhck edited a comment on issue #19007:
URL: https://github.com/apache/airflow/issues/19007#issuecomment-983494569


   I am facing the very same issue on my worker:
   
   ```
   [2021-12-01 10:07:40,770: WARNING/ForkPoolWorker-15] airflow tasks run command error: argument execution_date: invalid parse value: 'manual__2021-12-01T11:07:14+01:00', see help above.
   [2021-12-01 10:07:40,782: ERROR/ForkPoolWorker-15] Task airflow.executors.celery_executor.execute_command[fd1fc02c-a25d-4157-9d22-f763aa8fd647] raised unexpected: AirflowException('Celery command failed on host: xxxxxx.eu-central-1.compute.internal')
   ```
   
   What fixxed your issue?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [airflow] ptrhck commented on issue #19007: argument execution_date: invalid parse value

Posted by GitBox <gi...@apache.org>.
ptrhck commented on issue #19007:
URL: https://github.com/apache/airflow/issues/19007#issuecomment-983494569


   I am facing the very same issue on my worker:
   
   `
   2021-12-01T10:07:40.771Z | [2021-12-01 10:07:40,770: WARNING/ForkPoolWorker-15] airflow tasks run command error: argument execution_date: invalid parse value: 'manual__2021-12-01T11:07:14+01:00', see help above.
   -- | --
     | 2021-12-01T10:07:40.783Z | [2021-12-01 10:07:40,782: ERROR/ForkPoolWorker-15] Task airflow.executors.celery_executor.execute_command[fd1fc02c-a25d-4157-9d22-f763aa8fd647] raised unexpected: AirflowException('Celery command failed on host: xxxxx.eu-central-1.compute.internal')
     | 2021-12-01T10:07:40.783Z | Traceback (most recent call last):
     | 2021-12-01T10:07:40.783Z | File "/home/airflow/.local/lib/python3.8/site-packages/celery/app/trace.py", line 412, in trace_task
     | 2021-12-01T10:07:40.783Z | R = retval = fun(*args, **kwargs)
     | 2021-12-01T10:07:40.783Z | File "/home/airflow/.local/lib/python3.8/site-packages/celery/app/trace.py", line 704, in __protected_call__
     | 2021-12-01T10:07:40.783Z | return self.run(*args, **kwargs)
     | 2021-12-01T10:07:40.783Z | File "/home/airflow/.local/lib/python3.8/site-packages/airflow/executors/celery_executor.py", line 88, in execute_command
     | 2021-12-01T10:07:40.783Z | _execute_in_fork(command_to_exec)
     | 2021-12-01T10:07:40.783Z | File "/home/airflow/.local/lib/python3.8/site-packages/airflow/executors/celery_executor.py", line 99, in _execute_in_fork
     | 2021-12-01T10:07:40.783Z | raise AirflowException('Celery command failed on host: ' + get_hostname())
     | 2021-12-01T10:07:40.783Z | airflow.exceptions.AirflowException: Celery command failed on host: xxxx.eu-central-1.compute.internal
   `
   
   What fixxed your issue?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [airflow] ephraimbuddy commented on issue #19007: argument execution_date: invalid parse value

Posted by GitBox <gi...@apache.org>.
ephraimbuddy commented on issue #19007:
URL: https://github.com/apache/airflow/issues/19007#issuecomment-983517082


   > I am facing the very same issue on my worker:
   > 
   > ```
   > [2021-12-01 10:07:40,770: WARNING/ForkPoolWorker-15] airflow tasks run command error: argument execution_date: invalid parse value: 'manual__2021-12-01T11:07:14+01:00', see help above.
   > [2021-12-01 10:07:40,782: ERROR/ForkPoolWorker-15] Task airflow.executors.celery_executor.execute_command[fd1fc02c-a25d-4157-9d22-f763aa8fd647] raised unexpected: AirflowException('Celery command failed on host: xxxxxx.eu-central-1.compute.internal')
   > ```
   > 
   > What fixxed your issue?
   
   The dagrun couldn't be found. The log was suppressed here https://github.com/apache/airflow/blob/43de625d4246af7014f64941f8effb09997731cb/airflow/cli/commands/task_command.py#L61-L62
   
   Does the dagrun exist?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [airflow] kapopken closed issue #19007: argument execution_date: invalid parse value

Posted by GitBox <gi...@apache.org>.
kapopken closed issue #19007:
URL: https://github.com/apache/airflow/issues/19007


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [airflow] mik-laj commented on issue #19007: argument execution_date: invalid parse value

Posted by GitBox <gi...@apache.org>.
mik-laj commented on issue #19007:
URL: https://github.com/apache/airflow/issues/19007#issuecomment-944261431


   CC: @ephraimbuddy 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [airflow] ephraimbuddy commented on issue #19007: argument execution_date: invalid parse value

Posted by GitBox <gi...@apache.org>.
ephraimbuddy commented on issue #19007:
URL: https://github.com/apache/airflow/issues/19007#issuecomment-983547604


   > How can I check if the dagrun exist? I can see it in the UI as "up_for_retry".
   > 
   > In addition, I have seen in the worker logs this on startup:
   > 
   > `alembic.script.revision.ResolutionError: No such revision or branch '7b2661a43ba3' ... alembic.util.exc.CommandError: Can't locate revision identified by '7b2661a43ba3'`
   
   Can you explain what you did before getting these errors? Are you upgrading? You should run `airflow db upgrade` when upgrading to a version. You may want to explain more here or ask questions using discussion


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [airflow] ptrhck commented on issue #19007: argument execution_date: invalid parse value

Posted by GitBox <gi...@apache.org>.
ptrhck commented on issue #19007:
URL: https://github.com/apache/airflow/issues/19007#issuecomment-983556111


   I did an update in the past and most likely airflow db upgrade hast not been executed. However, the error of this issue occured out of the sudden after restarting airflow as cpu and memory have been increased


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [airflow] kapopken commented on issue #19007: argument execution_date: invalid parse value

Posted by GitBox <gi...@apache.org>.
kapopken commented on issue #19007:
URL: https://github.com/apache/airflow/issues/19007#issuecomment-944401814


   Closing issue. The issue ended up being a typo which went to our old docker image repo 2 letter difference. This issue is solved.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [airflow] boring-cyborg[bot] commented on issue #19007: argument execution_date: invalid parse value

Posted by GitBox <gi...@apache.org>.
boring-cyborg[bot] commented on issue #19007:
URL: https://github.com/apache/airflow/issues/19007#issuecomment-944230417


   Thanks for opening your first issue here! Be sure to follow the issue template!
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org