[GitHub] [airflow] Michal-Biernacki opened a new issue #20875: Serialization error after successful DAG run

[GitBox <gi...@apache.org>]

Michal-Biernacki opened a new issue #20875:
URL: https://github.com/apache/airflow/issues/20875


   ### Apache Airflow version
   
   2.2.3 (latest released)
   
   ### What happened
   
   In case within the DAG definition we use both `params` and `on_success_callback` then after triggering the DAG from the UI (option "Trigger DAG") and the finish of the successful DAG run I got the _DAG Import Errors_ error displayed in the UI.
   
   Issue is related with the `_serialize_params_dict` method of `BaseSerialization` class.
   Part of the error message:
   ```
   if f'{v.__module__}.{v.__class__.__name__}' == 'airflow.models.param.Param':
   AttributeError: 'str' object has no attribute '__module__'
   ```
   In consequence there is also no update in the `serialized_dag` table of backend db.
   
   Additionally:
   - using the script provided in this ticket #19471 results in the successful serialization
   - making any change to the code results in the successful serialization
   So error occurs only after the successful DAG run from UI (I did not check the output of the run from CLI or just scheduled run)
   
   In case we remove the `params` or `on_success_callback` attribute then there is no serialization error.
   
   To confirm this issue I set up completely new Airflow instance in separate virtualenv and still had the same issue.
   
   ### What you expected to happen
   
   Successful serialization after the successful DAG run
   
   ### How to reproduce
   
   To reproduce this error I prepared very simple DAG presented below.
   
   Additional information about the setup:
   - executor = LocalExecutor
   - sql_alchemy_conn = Postgres db
   
   ```
   from airflow import DAG
   from airflow.utils import timezone
   from airflow.operators.dummy import DummyOperator
   from airflow.models.param import Param
   
   def func_on_success(context):
       pass
   
   with DAG(
       dag_id="test",
       start_date=timezone.datetime(2022, 1, 1),
       schedule_interval='@once',
       params={"param_test": Param("param_value_test")},
       on_success_callback=func_on_success
   ) as dag:
       DummyOperator(task_id='task_1')
   ```
   
   After successful execution of this DAG through "Trigger DAG" option in the UI the error should be displayed (of course after refresh of the webpage).
   
   ### Operating System
   
   MacOS 12.1
   
   ### Versions of Apache Airflow Providers
   
   n/a
   
   ### Deployment
   
   Virtualenv installation
   
   ### Deployment details
   
   - executor = LocalExecutor
   - sql_alchemy_conn = Postgres db
   
   ### Anything else
   
   There is one scenario in which there will be no error:
   - make some changes to the DAG code
   - in consequence DAG is serialized automatically
   - trigger DAG and finish it successfully within the `min_serialized_dag_update_interval` seconds
   then there will be no error because serialization process will not start after the DAG run (at least this is my explanation).
   
   ### Are you willing to submit PR?
   
   - [ ] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] boring-cyborg[bot] commented on issue #20875: Serialization error after successful DAG run

[GitBox <gi...@apache.org>]

boring-cyborg[bot] commented on issue #20875:
URL: https://github.com/apache/airflow/issues/20875#issuecomment-1013090533


   Thanks for opening your first issue here! Be sure to follow the issue template!
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] uranusjr closed issue #20875: Serialization error after successful DAG run

[GitBox <gi...@apache.org>]

uranusjr closed issue #20875:
URL: https://github.com/apache/airflow/issues/20875


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] uranusjr commented on issue #20875: Serialization error after successful DAG run

[GitBox <gi...@apache.org>]

uranusjr commented on issue #20875:
URL: https://github.com/apache/airflow/issues/20875#issuecomment-1014273505


   Already fixed in main.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org