You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@thrift.apache.org by "ojciec pijo (JIRA)" <ji...@apache.org> on 2011/06/29 20:04:29 UTC
[jira] [Created] (THRIFT-1226) command line option to disable
overriding java serialization
command line option to disable overriding java serialization
------------------------------------------------------------
Key: THRIFT-1226
URL: https://issues.apache.org/jira/browse/THRIFT-1226
Project: Thrift
Issue Type: Improvement
Components: Java - Compiler
Affects Versions: 0.6.1, 0.6
Reporter: ojciec pijo
bugfix THRIFT-1038 is a breaking change since it overrides default java serialization mechanism. as a result, objects generated by Thrift v6 are not compatible with java serialization rules (references between objects are lost after deserialization). Please provide a command line parameter to disable generating readObject/writeObject as a workaround.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (THRIFT-1226) command line option to disable
overriding java serialization
Posted by "ojciec pijo (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-1226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
ojciec pijo updated THRIFT-1226:
--------------------------------
Attachment: testcase.zip
unit test that shows difference between v5 and v6
> command line option to disable overriding java serialization
> ------------------------------------------------------------
>
> Key: THRIFT-1226
> URL: https://issues.apache.org/jira/browse/THRIFT-1226
> Project: Thrift
> Issue Type: Improvement
> Components: Java - Compiler
> Affects Versions: 0.6, 0.6.1
> Reporter: ojciec pijo
> Attachments: testcase.zip
>
>
> bugfix THRIFT-1038 is a breaking change since it overrides default java serialization mechanism. as a result, objects generated by Thrift v6 are not compatible with java serialization rules (references between objects are lost after deserialization). Please provide a command line parameter to disable generating readObject/writeObject as a workaround.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (THRIFT-1226) command line option to disable
overriding java serialization
Posted by "Mathias Herberts (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-1226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13057952#comment-13057952 ]
Mathias Herberts commented on THRIFT-1226:
------------------------------------------
Since binary fields in 0.6.0+ are backed by ByteBuffer which are not serializable but Thrift structs are declared Serializable, your suggestion would be that we provide a switch to generate buggy structs, this does not sound right, especially since the moment you will add a binary field to one of your struct your Java Serialization will break.
> command line option to disable overriding java serialization
> ------------------------------------------------------------
>
> Key: THRIFT-1226
> URL: https://issues.apache.org/jira/browse/THRIFT-1226
> Project: Thrift
> Issue Type: Improvement
> Components: Java - Compiler
> Affects Versions: 0.6, 0.6.1
> Reporter: ojciec pijo
> Attachments: testcase.zip
>
>
> bugfix THRIFT-1038 is a breaking change since it overrides default java serialization mechanism. as a result, objects generated by Thrift v6 are not compatible with java serialization rules (references between objects are lost after deserialization). Please provide a command line parameter to disable generating readObject/writeObject as a workaround.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Issue Comment Edited] (THRIFT-1226) command line option to
disable overriding java serialization
Posted by "ojciec pijo (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-1226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13057508#comment-13057508 ]
ojciec pijo edited comment on THRIFT-1226 at 6/29/11 10:14 PM:
---------------------------------------------------------------
I think there's no java serialization spec per se, but default mechanism always keeps references if particular instance is saved to the same stream more than once. it is mentioned here: http://java.sun.com/developer/technicalArticles/Programming/serialization/ ("Caching Objects in the Stream").
For a testcase, I have generated the same pair of objects with Thrift 5 and 6. Test passes for Thrift5, fails for Thrift6:
{code}
[...]
@Test
public void testVersion5() throws Exception {
A5 a = new A5();
B5 b = new B5();
//add the same instance twice
b.addToItems(a);
b.addToItems(a);
B5 deserialized = (B5)serializeAndDeserialize(b);
//with java serialization
//we expect that references are preserved and [0] == [1]
assertSame(deserialized.getItems().get(0), deserialized.getItems().get(1));
}
@Test
public void testVersion6() throws Exception {
A6 a = new A6();
B6 b = new B6();
//add the same instance twice
b.addToItems(a);
b.addToItems(a);
B6 deserialized = (B6)serializeAndDeserialize(b);
//since Thrift6 overrides default java serialization by providing readObject/writeObject,
//now it behaves in 'thrift' way - references are replaced by copies
//and this assertion FAILS!
assertSame(deserialized.getItems().get(0), deserialized.getItems().get(1));
}
{code}
was (Author: ojciecpijo):
I think there's no java serialization spec per se, but default mechanism always keeps references if particular instance is saved to the same stream more than once. it is mentioned here: http://java.sun.com/developer/technicalArticles/Programming/serialization/ ("Caching Objects in the Stream").
For a testcase, I have generated the same pair of objects with Thrift 5 and 6. Test passes for Thrift5, fails for Thrift6:
[...]
@Test
public void testVersion5() throws Exception {
A5 a = new A5();
B5 b = new B5();
//add the same instance twice
b.addToItems(a);
b.addToItems(a);
B5 deserialized = (B5)serializeAndDeserialize(b);
//with java serialization
//we expect that references are preserved and [0] == [1]
assertSame(deserialized.getItems().get(0), deserialized.getItems().get(1));
}
@Test
public void testVersion6() throws Exception {
A6 a = new A6();
B6 b = new B6();
//add the same instance twice
b.addToItems(a);
b.addToItems(a);
B6 deserialized = (B6)serializeAndDeserialize(b);
//since Thrift6 overrides default java serialization by providing readObject/writeObject,
//now it behaves in 'thrift' way - references are replaced by copies
//and this assertion FAILS!
assertSame(deserialized.getItems().get(0), deserialized.getItems().get(1));
}
> command line option to disable overriding java serialization
> ------------------------------------------------------------
>
> Key: THRIFT-1226
> URL: https://issues.apache.org/jira/browse/THRIFT-1226
> Project: Thrift
> Issue Type: Improvement
> Components: Java - Compiler
> Affects Versions: 0.6, 0.6.1
> Reporter: ojciec pijo
> Attachments: testcase.zip
>
>
> bugfix THRIFT-1038 is a breaking change since it overrides default java serialization mechanism. as a result, objects generated by Thrift v6 are not compatible with java serialization rules (references between objects are lost after deserialization). Please provide a command line parameter to disable generating readObject/writeObject as a workaround.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (THRIFT-1226) command line option to disable
overriding java serialization
Posted by "Mathias Herberts (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-1226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13057545#comment-13057545 ]
Mathias Herberts commented on THRIFT-1226:
------------------------------------------
Prior to THRIFT-1038, Thrift structures which contained binary fields were not serializable despite implementing Serializable (because ByteBuffer is not Serializable), THRIFT-1038 corrected this by using a TProtocol to write the object on the ObjectOutputStream, thus using Thrift's serialization mechanism.
The 'default' Java serialization mechanism calls writeObject on the object to serialize, the default implementation of this method serializes primitive types and calls ObjectOutputStream.writeObject on referenced objects.
The patch introduced by THRIFT-1038 makes the generated Thrift structures Serializable but using a custom serialization mechanism, not the 'default' one.
To the best of my knowledge, the Java serialization spec (http://download.oracle.com/javase/1.5.0/docs/guide/serialization/spec/serialTOC.html) does not impose that object graph conservation property to custom serialization mechanisms.
One cannot therefore expect the object graph conservation to apply to Thrift structures serialized using the custom writeObject methods.
I'd be glad to hear anyone else's view of this issue.
> command line option to disable overriding java serialization
> ------------------------------------------------------------
>
> Key: THRIFT-1226
> URL: https://issues.apache.org/jira/browse/THRIFT-1226
> Project: Thrift
> Issue Type: Improvement
> Components: Java - Compiler
> Affects Versions: 0.6, 0.6.1
> Reporter: ojciec pijo
> Attachments: testcase.zip
>
>
> bugfix THRIFT-1038 is a breaking change since it overrides default java serialization mechanism. as a result, objects generated by Thrift v6 are not compatible with java serialization rules (references between objects are lost after deserialization). Please provide a command line parameter to disable generating readObject/writeObject as a workaround.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (THRIFT-1226) command line option to disable
overriding java serialization
Posted by "ojciec pijo (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-1226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13057669#comment-13057669 ]
ojciec pijo commented on THRIFT-1226:
-------------------------------------
I perfectly understand that thrift serialization is different than 'default' one, which is fine, and always was. the problem is that prior to ver6, default java serialization wasn't affected. Since this is breaking change for systems that relayed on this feature, it would be good to see some 'backward compatibility' option - e.g. possibility to disable this improvement via command line parameter.
> command line option to disable overriding java serialization
> ------------------------------------------------------------
>
> Key: THRIFT-1226
> URL: https://issues.apache.org/jira/browse/THRIFT-1226
> Project: Thrift
> Issue Type: Improvement
> Components: Java - Compiler
> Affects Versions: 0.6, 0.6.1
> Reporter: ojciec pijo
> Attachments: testcase.zip
>
>
> bugfix THRIFT-1038 is a breaking change since it overrides default java serialization mechanism. as a result, objects generated by Thrift v6 are not compatible with java serialization rules (references between objects are lost after deserialization). Please provide a command line parameter to disable generating readObject/writeObject as a workaround.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (THRIFT-1226) command line option to disable
overriding java serialization
Posted by "Mathias Herberts (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-1226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13057449#comment-13057449 ]
Mathias Herberts commented on THRIFT-1226:
------------------------------------------
What part of the java serialization spec does the code introduced in THRIFT-1038 violate?
> command line option to disable overriding java serialization
> ------------------------------------------------------------
>
> Key: THRIFT-1226
> URL: https://issues.apache.org/jira/browse/THRIFT-1226
> Project: Thrift
> Issue Type: Improvement
> Components: Java - Compiler
> Affects Versions: 0.6, 0.6.1
> Reporter: ojciec pijo
>
> bugfix THRIFT-1038 is a breaking change since it overrides default java serialization mechanism. as a result, objects generated by Thrift v6 are not compatible with java serialization rules (references between objects are lost after deserialization). Please provide a command line parameter to disable generating readObject/writeObject as a workaround.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (THRIFT-1226) command line option to disable
overriding java serialization
Posted by "ojciec pijo (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-1226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13057508#comment-13057508 ]
ojciec pijo commented on THRIFT-1226:
-------------------------------------
I think there's no java serialization spec per se, but default mechanism always keeps references if particular instance is saved to the same stream more than once. it is mentioned here: http://java.sun.com/developer/technicalArticles/Programming/serialization/ ("Caching Objects in the Stream").
For a testcase, I have generated the same pair of objects with Thrift 5 and 6. Test passes for Thrift5, fails for Thrift6:
[...]
@Test
public void testVersion5() throws Exception {
A5 a = new A5();
B5 b = new B5();
//add the same instance twice
b.addToItems(a);
b.addToItems(a);
B5 deserialized = (B5)serializeAndDeserialize(b);
//with java serialization
//we expect that references are preserved and [0] == [1]
assertSame(deserialized.getItems().get(0), deserialized.getItems().get(1));
}
@Test
public void testVersion6() throws Exception {
A6 a = new A6();
B6 b = new B6();
//add the same instance twice
b.addToItems(a);
b.addToItems(a);
B6 deserialized = (B6)serializeAndDeserialize(b);
//since Thrift6 overrides default java serialization by providing readObject/writeObject,
//now it behaves in 'thrift' way - references are replaced by copies
//and this assertion FAILS!
assertSame(deserialized.getItems().get(0), deserialized.getItems().get(1));
}
> command line option to disable overriding java serialization
> ------------------------------------------------------------
>
> Key: THRIFT-1226
> URL: https://issues.apache.org/jira/browse/THRIFT-1226
> Project: Thrift
> Issue Type: Improvement
> Components: Java - Compiler
> Affects Versions: 0.6, 0.6.1
> Reporter: ojciec pijo
> Attachments: testcase.zip
>
>
> bugfix THRIFT-1038 is a breaking change since it overrides default java serialization mechanism. as a result, objects generated by Thrift v6 are not compatible with java serialization rules (references between objects are lost after deserialization). Please provide a command line parameter to disable generating readObject/writeObject as a workaround.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (THRIFT-1226) command line option to disable
overriding java serialization
Posted by "Bryan Duxbury (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-1226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13057996#comment-13057996 ]
Bryan Duxbury commented on THRIFT-1226:
---------------------------------------
I agree with Mathias that I'd prefer not to commit a patch that allows the generation of buggy serialization. I would rather see a patch that makes it easy to collapse the object graph down again after you complete deserialization.
> command line option to disable overriding java serialization
> ------------------------------------------------------------
>
> Key: THRIFT-1226
> URL: https://issues.apache.org/jira/browse/THRIFT-1226
> Project: Thrift
> Issue Type: Improvement
> Components: Java - Compiler
> Affects Versions: 0.6, 0.6.1
> Reporter: ojciec pijo
> Attachments: testcase.zip
>
>
> bugfix THRIFT-1038 is a breaking change since it overrides default java serialization mechanism. as a result, objects generated by Thrift v6 are not compatible with java serialization rules (references between objects are lost after deserialization). Please provide a command line parameter to disable generating readObject/writeObject as a workaround.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Issue Comment Edited] (THRIFT-1226) command line option to
disable overriding java serialization
Posted by "ojciec pijo (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-1226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13057508#comment-13057508 ]
ojciec pijo edited comment on THRIFT-1226 at 6/29/11 10:13 PM:
---------------------------------------------------------------
I think there's no java serialization spec per se, but default mechanism always keeps references if particular instance is saved to the same stream more than once. it is mentioned here: http://java.sun.com/developer/technicalArticles/Programming/serialization/ ("Caching Objects in the Stream").
For a testcase, I have generated the same pair of objects with Thrift 5 and 6. Test passes for Thrift5, fails for Thrift6:
[...]
@Test
public void testVersion5() throws Exception {
A5 a = new A5();
B5 b = new B5();
//add the same instance twice
b.addToItems(a);
b.addToItems(a);
B5 deserialized = (B5)serializeAndDeserialize(b);
//with java serialization
//we expect that references are preserved and [0] == [1]
assertSame(deserialized.getItems().get(0), deserialized.getItems().get(1));
}
@Test
public void testVersion6() throws Exception {
A6 a = new A6();
B6 b = new B6();
//add the same instance twice
b.addToItems(a);
b.addToItems(a);
B6 deserialized = (B6)serializeAndDeserialize(b);
//since Thrift6 overrides default java serialization by providing readObject/writeObject,
//now it behaves in 'thrift' way - references are replaced by copies
//and this assertion FAILS!
assertSame(deserialized.getItems().get(0), deserialized.getItems().get(1));
}
was (Author: ojciecpijo):
I think there's no java serialization spec per se, but default mechanism always keeps references if particular instance is saved to the same stream more than once. it is mentioned here: http://java.sun.com/developer/technicalArticles/Programming/serialization/ ("Caching Objects in the Stream").
For a testcase, I have generated the same pair of objects with Thrift 5 and 6. Test passes for Thrift5, fails for Thrift6:
[...]
@Test
public void testVersion5() throws Exception {
A5 a = new A5();
B5 b = new B5();
//add the same instance twice
b.addToItems(a);
b.addToItems(a);
B5 deserialized = (B5)serializeAndDeserialize(b);
//with java serialization
//we expect that references are preserved and [0] == [1]
assertSame(deserialized.getItems().get(0), deserialized.getItems().get(1));
}
@Test
public void testVersion6() throws Exception {
A6 a = new A6();
B6 b = new B6();
//add the same instance twice
b.addToItems(a);
b.addToItems(a);
B6 deserialized = (B6)serializeAndDeserialize(b);
//since Thrift6 overrides default java serialization by providing readObject/writeObject,
//now it behaves in 'thrift' way - references are replaced by copies
//and this assertion FAILS!
assertSame(deserialized.getItems().get(0), deserialized.getItems().get(1));
}
> command line option to disable overriding java serialization
> ------------------------------------------------------------
>
> Key: THRIFT-1226
> URL: https://issues.apache.org/jira/browse/THRIFT-1226
> Project: Thrift
> Issue Type: Improvement
> Components: Java - Compiler
> Affects Versions: 0.6, 0.6.1
> Reporter: ojciec pijo
> Attachments: testcase.zip
>
>
> bugfix THRIFT-1038 is a breaking change since it overrides default java serialization mechanism. as a result, objects generated by Thrift v6 are not compatible with java serialization rules (references between objects are lost after deserialization). Please provide a command line parameter to disable generating readObject/writeObject as a workaround.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira