You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@fury.apache.org by "w-yfan (via GitHub)" <gi...@apache.org> on 2025/02/18 13:11:59 UTC
[I] [Java] The initialization size of the classIdGenerator is different between JDK 8 and JDK 8+ [fury]
w-yfan opened a new issue, #2070:
URL: https://github.com/apache/fury/issues/2070
### Search before asking
- [x] I had searched in the [issues](https://github.com/apache/fury/issues) and found no similar issues.
### Version
fury：0.9.0
jdk: 8 & 11
### Component(s)
Java
### Minimal reproduce step
@Data
@AllArgsConstructor
static class Pojo {
private String str;
}
@Test
public void testSerializationUsingJdk11() {
// run with jdk11
Fury fury = Fury.builder()
.withLanguage(Language.JAVA)
.withRefTracking(false)
.withCompatibleMode(CompatibleMode.SCHEMA_CONSISTENT)
.withAsyncCompilation(true)
.build();
fury.register(Pojo.class);
byte[] bytes = fury.serialize(new Pojo("test"));
writeBytesToFile("example.bytes", bytes);
}
@SneakyThrows
@Test
public void testDeserializeUsingJdk8() {
// run with jdk8
Fury fury = Fury.builder()
.withLanguage(Language.JAVA)
.withRefTracking(false)
.withCompatibleMode(CompatibleMode.SCHEMA_CONSISTENT)
.withAsyncCompilation(true)
.build();
fury.register(Pojo.class);
byte[] bytes = readBytesFromFile("example.bytes");
System.out.println(fury.deserialize(bytes));
}
public static byte[] readBytesFromFile(String filePath) throws IOException {
try (FileInputStream fis = new FileInputStream(filePath)) {
byte[] data = new byte[fis.available()];
int read = fis.read(data);
return data;
} catch (IOException e) {
throw new IOException("Error reading file: " + e.getMessage());
}
}
public static void writeBytesToFile(String filePath, byte[] data) {
try (FileOutputStream fos = new FileOutputStream(filePath)) {
fos.write(data);
} catch (IOException e) {
System.out.println("Error writing to file: " + e.getMessage());
}
}
### What did you expect to see?
Complete the deserialization process correctly.
### What did you see instead?
The method org.apache.fury.serializer.ObjectSerializer.read() is not used.
The stack trace is as follows:
java.lang.IndexOutOfBoundsException: readerIndex(8) + length(6) exceeds size(10):
org.apache.fury.memory.MemoryBuffer$BoundChecker@12405818
at org.apache.fury.memory.MemoryBuffer$BoundChecker.fillBuffer(MemoryBuffer.java:186)
at org.apache.fury.memory.MemoryBuffer.readInt64(MemoryBuffer.java:1431)
at org.apache.fury.resolver.MetaStringResolver.readBigMetaStringBytes(MetaStringResolver.java:199)
at org.apache.fury.resolver.MetaStringResolver.readMetaStringBytesWithFlag(MetaStringResolver.java:159)
at org.apache.fury.resolver.ClassResolver.readClassInfoFromBytes(ClassResolver.java:1732)
at org.apache.fury.resolver.ClassResolver.readClassInfo(ClassResolver.java:1666)
at org.apache.fury.Fury.readRef(Fury.java:861)
at org.apache.fury.serializer.collection.GuavaCollectionSerializers$ImmutableSortedMapSerializer.newMap(GuavaCollectionSerializers.java:377)
at org.apache.fury.serializer.collection.MapSerializer.read(MapSerializer.java:67)
at org.apache.fury.serializer.collection.MapSerializer.read(MapSerializer.java:27)
at org.apache.fury.Fury.readDataInternal(Fury.java:959)
at org.apache.fury.Fury.readRef(Fury.java:861)
at org.apache.fury.Fury.deserialize(Fury.java:793)
at org.apache.fury.Fury.deserialize(Fury.java:714)
at com.micar.tsp.realdata.JdkTest.testDeserializeUsingJdk8(JdkTest.java:54)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63)
at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:69)
at com.intellij.rt.junit.IdeaTestRunner$Repeater$1.execute(IdeaTestRunner.java:38)
at com.intellij.rt.execution.junit.TestsRepeater.repeat(TestsRepeater.java:11)
at com.intellij.rt.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:35)
at com.intellij.rt.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:232)
at com.intellij.rt.junit.JUnitStarter.main(JUnitStarter.java:55)
### Anything Else?
Both ImmutableCollectionSerializers and SubListSerializers register java.util.ImmutableCollections$SubList, but JDK 8 does not have java.util.ImmutableCollections$SubList, resulting in JDK 8 registering two placeholders.
In JDK 11, although java.util.ImmutableCollections$SubList is registered twice, "If a serializer exists before, it will be replaced by the new serializer." This causes the ClassResolver.ExtRegistry#classIdGenerator in JDK 11 to have one less entry compared to JDK 8.
The final result: when JDK 8 and JDK 11 are used together, the correct serializers cannot be retrieved through classIdGenerator.
### Are you willing to submit a PR?
- [x] I'm willing to submit a PR!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@fury.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@fury.apache.org
For additional commands, e-mail: commits-help@fury.apache.org
Re: [I] [Java] The initialization size of the classIdGenerator is different between JDK 8 and JDK 8+ [fury]
Posted by "w-yfan (via GitHub)" <gi...@apache.org>.
w-yfan commented on issue #2070:
URL: https://github.com/apache/fury/issues/2070#issuecomment-2665685898
Should the registration of java.util.ImmutableCollections$SubList in org.apache.fury.serializer.collection.SubListSerializers be removed?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@fury.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@fury.apache.org
For additional commands, e-mail: commits-help@fury.apache.org
Re: [I] [Java] The initialization size of the classIdGenerator is different between JDK 8 and JDK 8+ [fury]
Posted by "chaokunyang (via GitHub)" <gi...@apache.org>.
chaokunyang commented on issue #2070:
URL: https://github.com/apache/fury/issues/2070#issuecomment-2665705334
Yes, it should be removed, we already registered it in ImmutableCollectionSerializers. Could you add a PR for this and add an ut in `integration_tests/jdk_compatibility_tests`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@fury.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@fury.apache.org
For additional commands, e-mail: commits-help@fury.apache.org
Re: [I] [Java] The initialization size of the classIdGenerator is different between JDK 8 and JDK 8+ [fury]
Posted by "chaokunyang (via GitHub)" <gi...@apache.org>.
chaokunyang closed issue #2070: [Java] The initialization size of the classIdGenerator is different between JDK 8 and JDK 8+
URL: https://github.com/apache/fury/issues/2070
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@fury.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@fury.apache.org
For additional commands, e-mail: commits-help@fury.apache.org
Re: [I] [Java] The initialization size of the classIdGenerator is different between JDK 8 and JDK 8+ [fury]
Posted by "w-yfan (via GitHub)" <gi...@apache.org>.
w-yfan commented on issue #2070:
URL: https://github.com/apache/fury/issues/2070#issuecomment-2665757218
Thank you to take a look at this issue @chaokunyang
sure, I will try to fix this bug
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@fury.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@fury.apache.org
For additional commands, e-mail: commits-help@fury.apache.org
Re: [I] [Java] The initialization size of the classIdGenerator is different between JDK 8 and JDK 8+ [fury]
Posted by "chaokunyang (via GitHub)" <gi...@apache.org>.
chaokunyang closed issue #2070: [Java] The initialization size of the classIdGenerator is different between JDK 8 and JDK 8+
URL: https://github.com/apache/fury/issues/2070
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@fury.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@fury.apache.org
For additional commands, e-mail: commits-help@fury.apache.org