[GitHub] [airflow] turbaszek commented on pull request #12327: Use different deserialization method in XCom init_on_load

[GitBox <gi...@apache.org>]

turbaszek commented on pull request #12327:
URL: https://github.com/apache/airflow/pull/12327#issuecomment-726684675


   > What was the error you got (in text please) when using this backend? I don't quite see how this would be used in your case.
   
   @ashb  as mentioned in #12315 the first problem I saw was loading of XCom model view in web ui:
   ```
     File "/usr/local/lib/python3.8/site-packages/flask_appbuilder/templates/appbuilder/general/widgets/base_list.html", line 23, in top-level template code
       {% block begin_loop_values %}
     File "/opt/airflow/airflow/www/templates/airflow/model_list.html", line 80, in block "begin_loop_values"
       {% elif item[value] != None %}
     File "/usr/local/lib/python3.8/site-packages/pandas/core/generic.py", line 1326, in __nonzero__
       raise ValueError(
   ValueError: The truth value of a DataFrame is ambiguous. Use a.empty, a.bool(), a.item(), a.any() or a.all().
   ```
   
   However, as described there it can be fixed by changing `!= None` to `is not none` in model template. Here's the result:
   
   <img width="1673" alt="Screenshot 2020-11-13 at 11 21 10" src="https://user-images.githubusercontent.com/9528307/99061788-58391e00-25a2-11eb-88b2-fe80bcdd226d.png">
   
   But the presence of dataframes in this table suggests that they are retrieved from GCS bucket (they are not stored in XCom table, where I store only reference to the bucket object). To show this view we do:
   ```
           xcomlist = (
               session.query(XCom)
               .filter(XCom.dag_id == dag_id, XCom.task_id == task_id, XCom.execution_date == dttm)
               .all()
           )
   ```
   Which queries XComs and calls `deserialize_value` when recreating XCom object from database in `init_on_load` method.
   
   What this PR propose is to have `orm_deserialize_value` for "get value from database to create XCom object" and  `deserialize_value` to "deserialize XCom object into other object". I'm happy to hear other possible options, however this one is simplest one that came to my mind.
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org