You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@fury.apache.org by "yoohaemin (via GitHub)" <gi...@apache.org> on 2024/09/21 15:21:24 UTC
[D] (Java/Scala) How do I specify deserialize behavior on schema change? [fury]
GitHub user yoohaemin created a discussion: (Java/Scala) How do I specify deserialize behavior on schema change?
Hi, we are looking into using fury for our new Scala project.
In our project, we need to ensure that the two versions of the application (versions right after and right before each other) can always talk to each other.
Obviously, when adding a new class, it can't be deserialized in an old version, no matter which serialization library we are using, so the deploys will need to be in two steps.
However, the modifications of existing classes are trickier. There can be a few cases:
- Field added (how to specify default value when serialized payload from old version does not contain the field)
- Field made optional (Int -> Option[Int])
- Field removed
- Field type changed, with a mapping function (generalized version of 2nd case)
How do I express such changes to Fury, so it can handle these cases?
I will be using compatible mode.
Thanks for reading.
GitHub link: https://github.com/apache/fury/discussions/1848
----
This is an automatically sent email for commits@fury.apache.org.
To unsubscribe, please send an email to: commits-unsubscribe@fury.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@fury.apache.org
For additional commands, e-mail: commits-help@fury.apache.org
Re: [D] (Java/Scala) How do I specify deserialize behavior on schema change? [fury]
Posted by "chaokunyang (via GitHub)" <gi...@apache.org>.
GitHub user chaokunyang edited a comment on the discussion: (Java/Scala) How do I specify deserialize behavior on schema change?
Fields values of changed types/name will be ignored when deserialization
GitHub link: https://github.com/apache/fury/discussions/1848#discussioncomment-10714102
----
This is an automatically sent email for commits@fury.apache.org.
To unsubscribe, please send an email to: commits-unsubscribe@fury.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@fury.apache.org
For additional commands, e-mail: commits-help@fury.apache.org
Re: [D] (Java/Scala) How do I specify deserialize behavior on schema change? [fury]
Posted by "yoohaemin (via GitHub)" <gi...@apache.org>.
GitHub user yoohaemin edited a discussion: (Java/Scala) How do I specify deserialize behavior on schema change?
Hi, we are looking into using fury for our new Scala project.
In our project, we need to ensure that the two versions of the application (versions right after and right before each other) can always talk to each other.
Obviously, when adding a new class, it can't be deserialized in an old version, no matter which serialization library we are using, so the deploys will need to be in two steps.
However, the modifications of existing classes are trickier. There can be a few cases:
- Field added
- Field made optional (Int -> Option[Int])
- Field removed
- Field type changed, with a mapping function
- Field renamed
These problems can be solved by specifying behaviors to fury on a field:
- specify a default value on deserialization
- specify a mapping function from a value of different type to the current type
- specify a renaming function
How do I express such changes to Fury, so it can handle these cases?
I will be using compatible mode.
Thanks for reading.
GitHub link: https://github.com/apache/fury/discussions/1848
----
This is an automatically sent email for commits@fury.apache.org.
To unsubscribe, please send an email to: commits-unsubscribe@fury.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@fury.apache.org
For additional commands, e-mail: commits-help@fury.apache.org
Re: [D] (Java/Scala) How do I specify deserialize behavior on schema change? [fury]
Posted by "yoohaemin (via GitHub)" <gi...@apache.org>.
GitHub user yoohaemin edited a discussion: (Java/Scala) How do I specify deserialize behavior on schema change?
Hi, we are looking into using fury for our new Scala project.
In our project, we need to ensure that the two versions of the application (versions right after and right before each other) can always talk to each other.
Obviously, when adding a new class, it can't be deserialized in an old version, no matter which serialization library we are using, so the deploys will need to be in two steps.
However, the modifications of existing classes are trickier. There can be a few cases:
- Field added (how to specify default value when serialized payload from old version does not contain the field)
- Field made optional (Int -> Option[Int])
- Field removed
- Field type changed, with a mapping function (generalized version of 2nd case)
- Field renamed
These problems can be solved by specifying behaviors to fury:
- specify a default value on deserialization
- specify a mapping function from a value of different type to the current type
- specify a renaming function
How do I express such changes to Fury, so it can handle these cases?
I will be using compatible mode.
Thanks for reading.
GitHub link: https://github.com/apache/fury/discussions/1848
----
This is an automatically sent email for commits@fury.apache.org.
To unsubscribe, please send an email to: commits-unsubscribe@fury.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@fury.apache.org
For additional commands, e-mail: commits-help@fury.apache.org
Re: [D] (Java/Scala) How do I specify deserialize behavior on schema change? [fury]
Posted by "chaokunyang (via GitHub)" <gi...@apache.org>.
GitHub user chaokunyang added a comment to the discussion: (Java/Scala) How do I specify deserialize behavior on schema change?
We do not support field annotation, we have plan for it, But we don't have time for this feature currently. It not complex,extending Fury ClassDef is enough
GitHub link: https://github.com/apache/fury/discussions/1848#discussioncomment-10743778
----
This is an automatically sent email for commits@fury.apache.org.
To unsubscribe, please send an email to: commits-unsubscribe@fury.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@fury.apache.org
For additional commands, e-mail: commits-help@fury.apache.org
Re: [D] (Java/Scala) How do I specify deserialize behavior on schema change? [fury]
Posted by "istinnstudio (via GitHub)" <gi...@apache.org>.
GitHub user istinnstudio edited a comment on the discussion: (Java/Scala) How do I specify deserialize behavior on schema change?
I have just tested fury as a potential future alternative for the FST library. FST has some kind of support for versioning, supporting only new fields adding. Deleting is not supported. Field rename is possible but changing the field type is like adding a new one. So I end up will "dummy" fields in code, stayed there by mistake or feature update, but there is at least backwards compatibility and old files can be deserialized ok. Old version fields thought should stay for ever in their classes even if they are useless null or empty. FST uses an incremental annotation for this @ Version (1) ... (n) before any new field declaration. I am hoping at least for a feature like this one or something better or at least example code on how to achieve backwards compatibility in basic class update scenarios. I do not know if there are other better solutions that can handle deleted fields without breaking the deserialization. I also do not know if there is any documentation or code examples of fur
y that can help to manage "versioning" scenarios.
GitHub link: https://github.com/apache/fury/discussions/1848#discussioncomment-10742759
----
This is an automatically sent email for commits@fury.apache.org.
To unsubscribe, please send an email to: commits-unsubscribe@fury.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@fury.apache.org
For additional commands, e-mail: commits-help@fury.apache.org
Re: [D] (Java/Scala) How do I specify deserialize behavior on schema change? [fury]
Posted by "yoohaemin (via GitHub)" <gi...@apache.org>.
GitHub user yoohaemin edited a comment on the discussion: (Java/Scala) How do I specify deserialize behavior on schema change?
Interesting idea, but will this detection add overhead to Java users? Maybe performance wise, it's ok because a field missing won't be too frequent of a case.
I also think with the planned unroll annotation (Scala 3), users will have a workaround.
https://docs.scala-lang.org/sips/unroll-default-arguments.html
This sip is decided to be implemented.
GitHub link: https://github.com/apache/fury/discussions/1848#discussioncomment-10716107
----
This is an automatically sent email for commits@fury.apache.org.
To unsubscribe, please send an email to: commits-unsubscribe@fury.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@fury.apache.org
For additional commands, e-mail: commits-help@fury.apache.org
Re: [D] (Java/Scala) How do I specify deserialize behavior on schema change? [fury]
Posted by "yoohaemin (via GitHub)" <gi...@apache.org>.
GitHub user yoohaemin added a comment to the discussion: (Java/Scala) How do I specify deserialize behavior on schema change?
Thanks for the pointers. Let me see if I can come up with something 🙂
GitHub link: https://github.com/apache/fury/discussions/1848#discussioncomment-10716069
----
This is an automatically sent email for commits@fury.apache.org.
To unsubscribe, please send an email to: commits-unsubscribe@fury.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@fury.apache.org
For additional commands, e-mail: commits-help@fury.apache.org
Re: [D] (Java/Scala) How do I specify deserialize behavior on schema change? [fury]
Posted by "chaokunyang (via GitHub)" <gi...@apache.org>.
GitHub user chaokunyang added a comment to the discussion: (Java/Scala) How do I specify deserialize behavior on schema change?
This is great, I will take a look at it later
GitHub link: https://github.com/apache/fury/discussions/1848#discussioncomment-10860824
----
This is an automatically sent email for commits@fury.apache.org.
To unsubscribe, please send an email to: commits-unsubscribe@fury.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@fury.apache.org
For additional commands, e-mail: commits-help@fury.apache.org
Re: [D] (Java/Scala) How do I specify deserialize behavior on schema change? [fury]
Posted by "istinnstudio (via GitHub)" <gi...@apache.org>.
GitHub user istinnstudio edited a comment on the discussion: (Java/Scala) How do I specify deserialize behavior on schema change?
I have just tested fury as a potential future alternative for the FST library. FST has some kind of support for versioning, supporting only new fields adding. Deleting is not supported. Field rename is possible but changing the field type is like adding a new one. So I end up will "dummy" fields in code, stayed there by mistake or feature update, but there is at least backwards compatibility and old files can be deserialized ok. Old version fields thought should stay for ever in their classes even if they are useless null or empty. FST uses an incremental annotation for this @ Version 1 ... n before any new field declaration. I am hoping at least for a feature like this one or something better or at least example code on how to achieve backwards compatibility in basic class update scenarios. I do not know if there are other better solutions that can handle deleted fields without breaking the deserialization. I also do not know if there is any documentation or code examples of fury th
at can help to manage "versioning" scenarios.
GitHub link: https://github.com/apache/fury/discussions/1848#discussioncomment-10742759
----
This is an automatically sent email for commits@fury.apache.org.
To unsubscribe, please send an email to: commits-unsubscribe@fury.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@fury.apache.org
For additional commands, e-mail: commits-help@fury.apache.org
Re: [D] (Java/Scala) How do I specify deserialize behavior on schema change? [fury]
Posted by "chaokunyang (via GitHub)" <gi...@apache.org>.
GitHub user chaokunyang added a comment to the discussion: (Java/Scala) How do I specify deserialize behavior on schema change?
Is it added as null? We don't support scala default value yet. It's reported in https://github.com/apache/fury/issues/1683 too, but seems @LoranceChen has stopped working at it
GitHub link: https://github.com/apache/fury/discussions/1848#discussioncomment-10715739
----
This is an automatically sent email for commits@fury.apache.org.
To unsubscribe, please send an email to: commits-unsubscribe@fury.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@fury.apache.org
For additional commands, e-mail: commits-help@fury.apache.org
Re: [D] (Java/Scala) How do I specify deserialize behavior on schema change? [fury]
Posted by "chaokunyang (via GitHub)" <gi...@apache.org>.
GitHub user chaokunyang added a comment to the discussion: (Java/Scala) How do I specify deserialize behavior on schema change?
Changed types/name will be ignored when deserialization
GitHub link: https://github.com/apache/fury/discussions/1848#discussioncomment-10714102
----
This is an automatically sent email for commits@fury.apache.org.
To unsubscribe, please send an email to: commits-unsubscribe@fury.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@fury.apache.org
For additional commands, e-mail: commits-help@fury.apache.org
Re: [D] (Java/Scala) How do I specify deserialize behavior on schema change? [fury]
Posted by "chaokunyang (via GitHub)" <gi...@apache.org>.
GitHub user chaokunyang added a comment to the discussion: (Java/Scala) How do I specify deserialize behavior on schema change?
Fury supports compatibility. If you set compatibleMode to Compatiblle, you can add /remove fields without brraking deserialization. Field type change is OK, it won't break deserialization. But We will take it as a new field, so the original field data are lost
GitHub link: https://github.com/apache/fury/discussions/1848#discussioncomment-10743766
----
This is an automatically sent email for commits@fury.apache.org.
To unsubscribe, please send an email to: commits-unsubscribe@fury.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@fury.apache.org
For additional commands, e-mail: commits-help@fury.apache.org
Re: [D] (Java/Scala) How do I specify deserialize behavior on schema change? [fury]
Posted by "istinnstudio (via GitHub)" <gi...@apache.org>.
GitHub user istinnstudio added a comment to the discussion: (Java/Scala) How do I specify deserialize behavior on schema change?
I have just tested fury as a potential future alternative for the FST library. FST has some kind of support for versioning, supporting only new fields adding. Deleting is not supported. Field rename is possible but changing the field type is like adding a new one. So I end up will "dummy" fields in code, stayed there by mistake or feature update, but there is at least backwards compatibility and old files can be deserialized ok. Old version fields thought should stay for ever in their classes. FST uses an incremental annotation for this @ Version 1 ... n before any new field declaration. I am hoping at least for a feature like this one or something better or at least example code on how to achieve backwards compatibility in basic class update scenarios. I do not know if there are other better solutions that can handle deleted fields without breaking the deserialization. I also do not know if there is any documentation or code examples of fury that can help to manage "versioning" scen
arios.
GitHub link: https://github.com/apache/fury/discussions/1848#discussioncomment-10742759
----
This is an automatically sent email for commits@fury.apache.org.
To unsubscribe, please send an email to: commits-unsubscribe@fury.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@fury.apache.org
For additional commands, e-mail: commits-help@fury.apache.org
Re: [D] (Java/Scala) How do I specify deserialize behavior on schema change? [fury]
Posted by "yoohaemin (via GitHub)" <gi...@apache.org>.
GitHub user yoohaemin added a comment to the discussion: (Java/Scala) How do I specify deserialize behavior on schema change?
Interesting idea, but will this detection add overhead to Java users? Maybe performance wise, it's ok because a field missing won't be too frequent of a case.
I also think with the planned unroll annotation (Scala 3), the users of the code will have a workaround.
https://docs.scala-lang.org/sips/unroll-default-arguments.html
This sip is decided to be implemented.
GitHub link: https://github.com/apache/fury/discussions/1848#discussioncomment-10716107
----
This is an automatically sent email for commits@fury.apache.org.
To unsubscribe, please send an email to: commits-unsubscribe@fury.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@fury.apache.org
For additional commands, e-mail: commits-help@fury.apache.org
Re: [D] (Java/Scala) How do I specify deserialize behavior on schema change? [fury]
Posted by "yoohaemin (via GitHub)" <gi...@apache.org>.
GitHub user yoohaemin added a comment to the discussion: (Java/Scala) How do I specify deserialize behavior on schema change?
@chaokunyang Here's the WIP PR. It lacks test or docs, and I haven't run benchmarks yet.
https://github.com/apache/fury/pull/1870
Sorry it took a long time, I only have weekend to work on this.
GitHub link: https://github.com/apache/fury/discussions/1848#discussioncomment-10857964
----
This is an automatically sent email for commits@fury.apache.org.
To unsubscribe, please send an email to: commits-unsubscribe@fury.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@fury.apache.org
For additional commands, e-mail: commits-help@fury.apache.org
Re: [D] (Java/Scala) How do I specify deserialize behavior on schema change? [fury]
Posted by "yoohaemin (via GitHub)" <gi...@apache.org>.
GitHub user yoohaemin edited a discussion: (Java/Scala) How do I specify deserialize behavior on schema change?
Hi, we are looking into using fury for our new Scala project.
In our project, we need to ensure that the two versions of the application (versions right after and right before each other) can always talk to each other.
Obviously, when adding a new class, it can't be deserialized in an old version, no matter which serialization library we are using, so the deploys will need to be in two steps.
However, the modifications of existing classes are trickier. There can be a few cases:
- Field added (how to specify default value when serialized payload from old version does not contain the field)
- Field made optional (Int -> Option[Int])
- Field removed
- Field type changed, with a mapping function (generalized version of 2nd case)
- Field renamed
These problems can be solved by specifying behaviors to fury on a field:
- specify a default value on deserialization
- specify a mapping function from a value of different type to the current type
- specify a renaming function
How do I express such changes to Fury, so it can handle these cases?
I will be using compatible mode.
Thanks for reading.
GitHub link: https://github.com/apache/fury/discussions/1848
----
This is an automatically sent email for commits@fury.apache.org.
To unsubscribe, please send an email to: commits-unsubscribe@fury.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@fury.apache.org
For additional commands, e-mail: commits-help@fury.apache.org
Re: [D] (Java/Scala) How do I specify deserialize behavior on schema change? [fury]
Posted by "yoohaemin (via GitHub)" <gi...@apache.org>.
GitHub user yoohaemin added a comment to the discussion: (Java/Scala) How do I specify deserialize behavior on schema change?
What is the behavior when a field is added but payload doesn't contain the field? Is it added as null?
I am guessing if a field exists in payload but not in schema, it will be ignored?
GitHub link: https://github.com/apache/fury/discussions/1848#discussioncomment-10715120
----
This is an automatically sent email for commits@fury.apache.org.
To unsubscribe, please send an email to: commits-unsubscribe@fury.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@fury.apache.org
For additional commands, e-mail: commits-help@fury.apache.org
Re: [D] (Java/Scala) How do I specify deserialize behavior on schema change? [fury]
Posted by "yoohaemin (via GitHub)" <gi...@apache.org>.
GitHub user yoohaemin edited a comment on the discussion: (Java/Scala) How do I specify deserialize behavior on schema change?
Interesting idea, but will this detection add overhead to Java users? Maybe performance wise, it's ok because a field missing won't be too frequent of a case.
I also think with the planned unroll annotation (Scala 3), the Scala users will have a workaround.
https://docs.scala-lang.org/sips/unroll-default-arguments.html
This sip is decided to be implemented.
GitHub link: https://github.com/apache/fury/discussions/1848#discussioncomment-10716107
----
This is an automatically sent email for commits@fury.apache.org.
To unsubscribe, please send an email to: commits-unsubscribe@fury.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@fury.apache.org
For additional commands, e-mail: commits-help@fury.apache.org
Re: [D] (Java/Scala) How do I specify deserialize behavior on schema change? [fury]
Posted by "chaokunyang (via GitHub)" <gi...@apache.org>.
GitHub user chaokunyang added a comment to the discussion: (Java/Scala) How do I specify deserialize behavior on schema change?
However, I think it's possible to providea callback to let users handle such situations. It's possible to extend Fury to pass objects and incompatible fields to users to let them handle such cases
GitHub link: https://github.com/apache/fury/discussions/1848#discussioncomment-10714510
----
This is an automatically sent email for commits@fury.apache.org.
To unsubscribe, please send an email to: commits-unsubscribe@fury.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@fury.apache.org
For additional commands, e-mail: commits-help@fury.apache.org
Re: [D] (Java/Scala) How do I specify deserialize behavior on schema change? [fury]
Posted by "chaokunyang (via GitHub)" <gi...@apache.org>.
GitHub user chaokunyang added a comment to the discussion: (Java/Scala) How do I specify deserialize behavior on schema change?
This seems to be a general feature request, would you like to contribute to it.
The main changes would be `org.apache.fury.builder.MetaSharedCodecBuilder#setFieldValue`:
```java
@Override
protected Expression setFieldValue(Expression bean, Descriptor descriptor, Expression value) {
if (descriptor.getField() == null) {
// Field doesn't exist in current class, skip set this field value.
// Note that the field value shouldn't be an inlined value, otherwise field value read may
// be ignored.
// Add an ignored call here to make expression type to void.
return new Expression.StaticInvoke(ExceptionUtils.class, "ignore", value);
}
return super.setFieldValue(bean, descriptor, value);
}
```
And `org.apache.fury.serializer.MetaSharedSerializer#read`:
```java
if (fieldAccessor != null) {
short classId = fieldInfo.classId;
if (ObjectSerializer.readPrimitiveFieldValueFailed(
fury, buffer, obj, fieldAccessor, classId)
&& ObjectSerializer.readBasicObjectFieldValueFailed(
fury, buffer, obj, fieldAccessor, classId)) {
assert fieldInfo.classInfo != null;
Object fieldValue =
ObjectSerializer.readFinalObjectFieldValue(
fury, refResolver, classResolver, fieldInfo, isFinal, buffer);
fieldAccessor.putObject(obj, fieldValue);
}
} else {
if (skipPrimitiveFieldValueFailed(fury, fieldInfo.classId, buffer)) {
if (fieldInfo.classInfo == null) {
// TODO(chaokunyang) support registered serializer in peer with ref tracking disabled.
fury.readRef(buffer, classInfoHolder);
} else {
ObjectSerializer.readFinalObjectFieldValue(
fury, refResolver, classResolver, fieldInfo, isFinal, buffer);
}
}
}
```
GitHub link: https://github.com/apache/fury/discussions/1848#discussioncomment-10715752
----
This is an automatically sent email for commits@fury.apache.org.
To unsubscribe, please send an email to: commits-unsubscribe@fury.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@fury.apache.org
For additional commands, e-mail: commits-help@fury.apache.org
Re: [D] (Java/Scala) How do I specify deserialize behavior on schema change? [fury]
Posted by "chaokunyang (via GitHub)" <gi...@apache.org>.
GitHub user chaokunyang edited a comment on the discussion: (Java/Scala) How do I specify deserialize behavior on schema change?
It will be added as a null. We don't support scala default value yet. It's reported in https://github.com/apache/fury/issues/1683 too, but seems @LoranceChen has stopped working at it
GitHub link: https://github.com/apache/fury/discussions/1848#discussioncomment-10715739
----
This is an automatically sent email for commits@fury.apache.org.
To unsubscribe, please send an email to: commits-unsubscribe@fury.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@fury.apache.org
For additional commands, e-mail: commits-help@fury.apache.org
Re: [D] (Java/Scala) How do I specify deserialize behavior on schema change? [fury]
Posted by "yoohaemin (via GitHub)" <gi...@apache.org>.
GitHub user yoohaemin edited a comment on the discussion: (Java/Scala) How do I specify deserialize behavior on schema change?
What is the behavior when a field is added but payload doesn't contain the field? Is it added as null?
I am guessing if a field exists in payload but not in schema, it will be ignored.
GitHub link: https://github.com/apache/fury/discussions/1848#discussioncomment-10715120
----
This is an automatically sent email for commits@fury.apache.org.
To unsubscribe, please send an email to: commits-unsubscribe@fury.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@fury.apache.org
For additional commands, e-mail: commits-help@fury.apache.org
Re: [D] (Java/Scala) How do I specify deserialize behavior on schema change? [fury]
Posted by "chaokunyang (via GitHub)" <gi...@apache.org>.
GitHub user chaokunyang edited a comment on the discussion: (Java/Scala) How do I specify deserialize behavior on schema change?
However, I think it's possible to provide a callback to let users handle such situations. It's possible to extend Fury to pass objects and incompatible fields to users to let them handle such cases
GitHub link: https://github.com/apache/fury/discussions/1848#discussioncomment-10714510
----
This is an automatically sent email for commits@fury.apache.org.
To unsubscribe, please send an email to: commits-unsubscribe@fury.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@fury.apache.org
For additional commands, e-mail: commits-help@fury.apache.org
Re: [D] (Java/Scala) How do I specify deserialize behavior on schema change? [fury]
Posted by "chaokunyang (via GitHub)" <gi...@apache.org>.
GitHub user chaokunyang added a comment to the discussion: (Java/Scala) How do I specify deserialize behavior on schema change?
Compatible mode supports adding/removing fields. Fields type change and fields rename are not supportted
GitHub link: https://github.com/apache/fury/discussions/1848#discussioncomment-10714061
----
This is an automatically sent email for commits@fury.apache.org.
To unsubscribe, please send an email to: commits-unsubscribe@fury.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@fury.apache.org
For additional commands, e-mail: commits-help@fury.apache.org
Re: [D] (Java/Scala) How do I specify deserialize behavior on schema change? [fury]
Posted by "chaokunyang (via GitHub)" <gi...@apache.org>.
GitHub user chaokunyang added a comment to the discussion: (Java/Scala) How do I specify deserialize behavior on schema change?
Yeah, a WIP PR would be awesome, I can provide more inputs based on your PR
GitHub link: https://github.com/apache/fury/discussions/1848#discussioncomment-10746501
----
This is an automatically sent email for commits@fury.apache.org.
To unsubscribe, please send an email to: commits-unsubscribe@fury.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@fury.apache.org
For additional commands, e-mail: commits-help@fury.apache.org
Re: [D] (Java/Scala) How do I specify deserialize behavior on schema change? [fury]
Posted by "istinnstudio (via GitHub)" <gi...@apache.org>.
GitHub user istinnstudio added a comment to the discussion: (Java/Scala) How do I specify deserialize behavior on schema change?
Thanks for the suggestion! It seems though that setting compatibleMode to Compatible gives me some inconsistencies/errors on my fst to fury file convertor, I need further tests with different sets of parameters. But I have already a successful use of fury for large object deep copy operations to ram, before writing to disk with FST, that does not need compatibility mode... So for now this hybrid works well in my case. I will have an eye on fury updates and the versioning context. Great work.
GitHub link: https://github.com/apache/fury/discussions/1848#discussioncomment-10744662
----
This is an automatically sent email for commits@fury.apache.org.
To unsubscribe, please send an email to: commits-unsubscribe@fury.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@fury.apache.org
For additional commands, e-mail: commits-help@fury.apache.org
Re: [D] (Java/Scala) How do I specify deserialize behavior on schema change? [fury]
Posted by "chaokunyang (via GitHub)" <gi...@apache.org>.
GitHub user chaokunyang added a comment to the discussion: (Java/Scala) How do I specify deserialize behavior on schema change?
We have a big optimization for scala serializaiton in https://github.com/apache/fury/pull/1853, I believe it will improve scala collection serializaiton performance greatly
GitHub link: https://github.com/apache/fury/discussions/1848#discussioncomment-10747777
----
This is an automatically sent email for commits@fury.apache.org.
To unsubscribe, please send an email to: commits-unsubscribe@fury.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@fury.apache.org
For additional commands, e-mail: commits-help@fury.apache.org
Re: [D] (Java/Scala) How do I specify deserialize behavior on schema change? [fury]
Posted by "yoohaemin (via GitHub)" <gi...@apache.org>.
GitHub user yoohaemin added a comment to the discussion: (Java/Scala) How do I specify deserialize behavior on schema change?
I already made the codecbuilder to work, but the serializer is a bit tricky. Maybe I will already open a WIP PR so I can get input on what is a preferred approach.
GitHub link: https://github.com/apache/fury/discussions/1848#discussioncomment-10745187
----
This is an automatically sent email for commits@fury.apache.org.
To unsubscribe, please send an email to: commits-unsubscribe@fury.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@fury.apache.org
For additional commands, e-mail: commits-help@fury.apache.org