You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2023/01/05 10:27:43 UTC

[GitHub] [airflow] uranusjr opened a new issue, #28741: RecursionError with new refactored serialization

uranusjr opened a new issue, #28741:
URL: https://github.com/apache/airflow/issues/28741

   ### Apache Airflow version
   
   main (development)
   
   ### What happened
   
   ```pycon
   >>> from airflow.utils.json import XComEncoder
   >>> XComEncoder().encode(set())
   Traceback (most recent call last):
     File "<stdin>", line 1, in <module>
     File "/opt/airflow/airflow/utils/json.py", line 91, in encode
       return super().encode(o)
     File "/usr/local/lib/python3.7/json/encoder.py", line 199, in encode
       chunks = self.iterencode(o, _one_shot=True)
     File "/usr/local/lib/python3.7/json/encoder.py", line 257, in iterencode
       return _iterencode(o, 0)
     File "/opt/airflow/airflow/utils/json.py", line 82, in default
       return serialize(o)
     File "/opt/airflow/airflow/serialization/serde.py", line 97, in serialize
       if isinstance(o, _primitives):
   RecursionError: maximum recursion depth exceeded in __instancecheck__
   ```
   
   ### What you think should happen instead
   
   The encoder should correctly encode a value, or raise TypeError if it is not encodable.
   
   ### How to reproduce
   
   Simply use `XComEncoder` to encode anything that’s not encodable by the built-in JSONEncoder (i.e. something that’d go through the custom `default` hook).
   
   ### Operating System
   
   Any
   
   ### Versions of Apache Airflow Providers
   
   Irrelevant
   
   ### Deployment
   
   Official Apache Airflow Helm Chart
   
   ### Deployment details
   
   _No response_
   
   ### Anything else
   
   The problem here is because `serialize` does not guarantee to return a JSON-serializable object. So when `default` receives an set, it returns a set, and the encoder would try to serialize that set again, calling default, and eventually the stack explodes.
   
   The simplest solution would be to add an additional flag to `serialize` so it can return a JSON-compatible type if requested.
   
   ### Are you willing to submit PR?
   
   - [X] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] bolkedebruin commented on issue #28741: RecursionError with new refactored serialization

Posted by "bolkedebruin (via GitHub)" <gi...@apache.org>.
bolkedebruin commented on issue #28741:
URL: https://github.com/apache/airflow/issues/28741#issuecomment-1519633084

   This is not an issue anymore
   ```
   from airflow.utils.json import XComEncoder
   XComEncoder().encode(set())
   
   '{"__classname__": "builtins.set", "__version__": 1, "__data__": []}'
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] bolkedebruin closed issue #28741: RecursionError with new refactored serialization

Posted by "bolkedebruin (via GitHub)" <gi...@apache.org>.
bolkedebruin closed issue #28741: RecursionError with new refactored serialization
URL: https://github.com/apache/airflow/issues/28741


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] potiuk commented on issue #28741: RecursionError with new refactored serialization

Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #28741:
URL: https://github.com/apache/airflow/issues/28741#issuecomment-1385170952

   cc: @bolkedebruin 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] uranusjr commented on issue #28741: RecursionError with new refactored serialization

Posted by GitBox <gi...@apache.org>.
uranusjr commented on issue #28741:
URL: https://github.com/apache/airflow/issues/28741#issuecomment-1372039308

   Problematic code introduced in #28067.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org