You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@fory.apache.org by "esafak (via GitHub)" <gi...@apache.org> on 2025/07/09 16:13:26 UTC
[PR] feat(python): Add StatefulSerializer using __getstate__, __setstate__ [fory]
esafak opened a new pull request, #2400:
URL: https://github.com/apache/fory/pull/2400
## What does this PR do?
* add support for objects with __getstate__ and __setstate__ in the registry
* implement StatefulSerializer for cross-language, cross-version
* use StatefulSerializer for objects with __getstate__/__setstate__ (excluding pandas)
* update tests to include various stateful and argument-based objects
* test serialization/deserialization and serializer selection correctness
* ensure nested and reference tracking scenarios are covered
* validate that objects with __getnewargs_ex__ and __getnewargs__ work properly
* verify that the new serializer is used and state is restored accurately
## Related issues
Closes #2293
## Does this PR introduce any user-facing change?
- [ ] Does this PR introduce any public API change?
- [ ] Does this PR introduce any binary protocol compatibility change?
## Benchmark
Not done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@fory.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@fory.apache.org
For additional commands, e-mail: commits-help@fory.apache.org
Re: [PR] feat(python): Add StatefulSerializer using __getstate__, __setstate__ [fory]
Posted by "chaokunyang (via GitHub)" <gi...@apache.org>.
chaokunyang merged PR #2400:
URL: https://github.com/apache/fory/pull/2400
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@fory.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@fory.apache.org
For additional commands, e-mail: commits-help@fory.apache.org
Re: [PR] feat(python): Add StatefulSerializer using __getstate__, __setstate__ [fory]
Posted by "chaokunyang (via GitHub)" <gi...@apache.org>.
chaokunyang commented on code in PR #2400:
URL: https://github.com/apache/fory/pull/2400#discussion_r2195583662
##########
python/pyfory/serializer.py:
##########
@@ -703,6 +703,59 @@ def to_buffer(self) -> "Buffer":
return Buffer(self.binary)
+class StatefulSerializer(CrossLanguageCompatibleSerializer):
+ """
+ Serializer for objects that support __getstate__ and __setstate__.
+ Uses Fory's native serialization for better cross-language support.
+ """
+
+ def __init__(self, fory, cls):
+ super().__init__(fory, cls)
+ self.cls = cls
+
+ def write(self, buffer, value):
+ # Get the state using __getstate__
+ state = value.__getstate__()
+
+ # Check for constructor arguments
+ args = ()
+ kwargs = {}
+
+ if hasattr(value, "__getnewargs_ex__"):
Review Comment:
Do we need to cache this method ref as a field in this serializer? It's rare that users override this mehod on a instance variable.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@fory.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@fory.apache.org
For additional commands, e-mail: commits-help@fory.apache.org
Re: [PR] feat(python): Add StatefulSerializer using __getstate__, __setstate__ [fory]
Posted by "esafak (via GitHub)" <gi...@apache.org>.
esafak commented on code in PR #2400:
URL: https://github.com/apache/fory/pull/2400#discussion_r2195953890
##########
python/pyfory/serializer.py:
##########
@@ -703,6 +703,59 @@ def to_buffer(self) -> "Buffer":
return Buffer(self.binary)
+class StatefulSerializer(CrossLanguageCompatibleSerializer):
+ """
+ Serializer for objects that support __getstate__ and __setstate__.
+ Uses Fory's native serialization for better cross-language support.
+ """
+
+ def __init__(self, fory, cls):
+ super().__init__(fory, cls)
+ self.cls = cls
+
+ def write(self, buffer, value):
+ # Get the state using __getstate__
+ state = value.__getstate__()
+
+ # Check for constructor arguments
+ args = ()
+ kwargs = {}
+
+ if hasattr(value, "__getnewargs_ex__"):
Review Comment:
Is this better?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@fory.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@fory.apache.org
For additional commands, e-mail: commits-help@fory.apache.org
Re: [PR] feat(python): Add StatefulSerializer using __getstate__, __setstate__ [fory]
Posted by "chaokunyang (via GitHub)" <gi...@apache.org>.
chaokunyang commented on code in PR #2400:
URL: https://github.com/apache/fory/pull/2400#discussion_r2196479431
##########
python/pyfory/serializer.py:
##########
@@ -703,6 +703,52 @@ def to_buffer(self) -> "Buffer":
return Buffer(self.binary)
+class StatefulSerializer(CrossLanguageCompatibleSerializer):
+ """
+ Serializer for objects that support __getstate__ and __setstate__.
+ Uses Fory's native serialization for better cross-language support.
+ """
+
+ def __init__(self, fory, cls):
+ super().__init__(fory, cls)
+ self.cls = cls
+ # Cache the method references as fields in the serializer.
+ self._getnewargs_ex = getattr(cls, "__getnewargs_ex__", None)
Review Comment:
```suggestion
self._getnewargs_func = getattr(cls, "__getnewargs_ex__", None) or getattr(cls, "__getnewargs__", None)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@fory.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@fory.apache.org
For additional commands, e-mail: commits-help@fory.apache.org
Re: [PR] feat(python): Add StatefulSerializer using __getstate__, __setstate__ [fory]
Posted by "chaokunyang (via GitHub)" <gi...@apache.org>.
chaokunyang commented on code in PR #2400:
URL: https://github.com/apache/fory/pull/2400#discussion_r2196480504
##########
python/pyfory/serializer.py:
##########
@@ -703,6 +703,59 @@ def to_buffer(self) -> "Buffer":
return Buffer(self.binary)
+class StatefulSerializer(CrossLanguageCompatibleSerializer):
+ """
+ Serializer for objects that support __getstate__ and __setstate__.
+ Uses Fory's native serialization for better cross-language support.
+ """
+
+ def __init__(self, fory, cls):
+ super().__init__(fory, cls)
+ self.cls = cls
+
+ def write(self, buffer, value):
+ # Get the state using __getstate__
+ state = value.__getstate__()
+
+ # Check for constructor arguments
+ args = ()
+ kwargs = {}
+
+ if hasattr(value, "__getnewargs_ex__"):
Review Comment:
Looks good
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@fory.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@fory.apache.org
For additional commands, e-mail: commits-help@fory.apache.org
Re: [PR] feat(python): Add StatefulSerializer using __getstate__, __setstate__ [fory]
Posted by "chaokunyang (via GitHub)" <gi...@apache.org>.
chaokunyang commented on code in PR #2400:
URL: https://github.com/apache/fory/pull/2400#discussion_r2196479431
##########
python/pyfory/serializer.py:
##########
@@ -703,6 +703,52 @@ def to_buffer(self) -> "Buffer":
return Buffer(self.binary)
+class StatefulSerializer(CrossLanguageCompatibleSerializer):
+ """
+ Serializer for objects that support __getstate__ and __setstate__.
+ Uses Fory's native serialization for better cross-language support.
+ """
+
+ def __init__(self, fory, cls):
+ super().__init__(fory, cls)
+ self.cls = cls
+ # Cache the method references as fields in the serializer.
+ self._getnewargs_ex = getattr(cls, "__getnewargs_ex__", None)
Review Comment:
```suggestion
self._getnewargs_func = getattr(cls, "__getnewargs_ex__", None) or getattr(cls, "__getnewargs__", None)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@fory.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@fory.apache.org
For additional commands, e-mail: commits-help@fory.apache.org
Re: [PR] feat(python): Add StatefulSerializer using __getstate__, __setstate__ [fory]
Posted by "esafak (via GitHub)" <gi...@apache.org>.
esafak commented on code in PR #2400:
URL: https://github.com/apache/fory/pull/2400#discussion_r2196502424
##########
python/pyfory/_registry.py:
##########
@@ -485,6 +488,15 @@ def _create_serializer(self, cls):
serializer = DataClassSerializer(self.fory, cls)
elif issubclass(cls, enum.Enum):
serializer = EnumSerializer(self.fory, cls)
+ elif hasattr(cls, "__getstate__") and hasattr(cls, "__setstate__"):
+ # Use StatefulSerializer for objects that support __getstate__ and __setstate__
+ # But exclude certain types that have incompatible state methods
+ module_name = getattr(cls, "__module__", "")
+ if module_name.startswith("pandas."):
Review Comment:
What do you want to do differently?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@fory.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@fory.apache.org
For additional commands, e-mail: commits-help@fory.apache.org
Re: [PR] feat(python): Add StatefulSerializer using __getstate__, __setstate__ [fory]
Posted by "chaokunyang (via GitHub)" <gi...@apache.org>.
chaokunyang commented on code in PR #2400:
URL: https://github.com/apache/fory/pull/2400#discussion_r2196481395
##########
python/pyfory/_registry.py:
##########
@@ -485,6 +488,15 @@ def _create_serializer(self, cls):
serializer = DataClassSerializer(self.fory, cls)
elif issubclass(cls, enum.Enum):
serializer = EnumSerializer(self.fory, cls)
+ elif hasattr(cls, "__getstate__") and hasattr(cls, "__setstate__"):
+ # Use StatefulSerializer for objects that support __getstate__ and __setstate__
+ # But exclude certain types that have incompatible state methods
+ module_name = getattr(cls, "__module__", "")
+ if module_name.startswith("pandas."):
Review Comment:
Could we create an issue to address this in the future?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@fory.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@fory.apache.org
For additional commands, e-mail: commits-help@fory.apache.org
Re: [PR] feat(python): Add StatefulSerializer using __getstate__, __setstate__ [fory]
Posted by "chaokunyang (via GitHub)" <gi...@apache.org>.
chaokunyang commented on code in PR #2400:
URL: https://github.com/apache/fory/pull/2400#discussion_r2195587725
##########
python/pyfory/serializer.py:
##########
@@ -703,6 +703,59 @@ def to_buffer(self) -> "Buffer":
return Buffer(self.binary)
+class StatefulSerializer(CrossLanguageCompatibleSerializer):
+ """
+ Serializer for objects that support __getstate__ and __setstate__.
+ Uses Fory's native serialization for better cross-language support.
+ """
+
+ def __init__(self, fory, cls):
+ super().__init__(fory, cls)
+ self.cls = cls
+
+ def write(self, buffer, value):
+ # Get the state using __getstate__
+ state = value.__getstate__()
+
+ # Check for constructor arguments
+ args = ()
+ kwargs = {}
+
+ if hasattr(value, "__getnewargs_ex__"):
Review Comment:
Or we can left this in future PR, current implementation has better compatibility
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@fory.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@fory.apache.org
For additional commands, e-mail: commits-help@fory.apache.org