You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@thrift.apache.org by "Stefan Gmeiner (Created) (JIRA)" <ji...@apache.org> on 2012/03/01 13:51:59 UTC
[jira] [Created] (THRIFT-1528) Iconsistency in optional fields
between Java/C# and python
Iconsistency in optional fields between Java/C# and python
----------------------------------------------------------
Key: THRIFT-1528
URL: https://issues.apache.org/jira/browse/THRIFT-1528
Project: Thrift
Issue Type: Bug
Components: Python - Compiler
Affects Versions: 0.8
Reporter: Stefan Gmeiner
Priority: Minor
If a struct contains optional fields with default values the generated python code serialize differently than Java or C# code.
In Java or C# optional fields are only serialized if a field was set by the client. If not the field is omited during serialization. This is possible because C#/Java maintains for each field a 'isset'-boolean which records if a field was set or not.
However the generated python code does not have such a 'isset'-structure. It writes every field which is not equal None. As the constructor initialize the optional fields with their default value, these fields are written whether they are set or not.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (THRIFT-1528) Iconsistency in optional fields
between Java/C# and python
Posted by "Diwaker Gupta (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-1528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13220425#comment-13220425 ]
Diwaker Gupta commented on THRIFT-1528:
---------------------------------------
As with most things, there's a tradeoff here. For protocols where most fields are marked as optional and have default values, *and* you have tight control (and versioning) of the end points, it will be a lot more efficient to omit serializing such fields and have the end points populate them with defaults during deserialization.
Providing *binary* compatibility however, is simpler if we serialize default values for optional fields. For instance, what happens if the client does set a field but the value is the same as the default?
At the end of the day it's more important for Thrift language bindings to be mutually consistent. Will, do you think this is worth discussing on thrift-dev?
> Iconsistency in optional fields between Java/C# and python
> ----------------------------------------------------------
>
> Key: THRIFT-1528
> URL: https://issues.apache.org/jira/browse/THRIFT-1528
> Project: Thrift
> Issue Type: Bug
> Components: Python - Compiler
> Affects Versions: 0.8
> Reporter: Stefan Gmeiner
> Priority: Minor
>
> If a struct contains optional fields with default values the generated python code serialize differently than Java or C# code.
> In Java or C# optional fields are only serialized if a field was set by the client. If not the field is omited during serialization. This is possible because C#/Java maintains for each field a 'isset'-boolean which records if a field was set or not.
> However the generated python code does not have such a 'isset'-structure. It writes every field which is not equal None. As the constructor initialize the optional fields with their default value, these fields are written whether they are set or not.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (THRIFT-1528) Iconsistency in optional fields
between Java/C# and python
Posted by "Diwaker Gupta (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-1528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13227326#comment-13227326 ]
Diwaker Gupta commented on THRIFT-1528:
---------------------------------------
bq. From my perspective, its insane to have a field have a value and not to serialize it. In fact, the whole "isset" thing in Java is just a hack around the fact that you can't have null primitives. I would vote for standardizing on default values being considered set at object creation.
+1 for Bryan's suggestion. In fact, my first patch for THRIFT-1394 did precisely this for C++. I'm happy to provide patches for at least C++ and Java so we _always_ serialize optional fields with defaults specified, even if they were never explicitly set.
bq. I would prefer to go with nullable types across the board as well, but since this currently is not an option for primitives I think we need to keep isset for those types for now.
Jake: there are nullable boxed types for most Java primitives (Int, Long, Boolean etc). If the Thrift compiler uses those, then we could have nullable types across the board right?
> Iconsistency in optional fields between Java/C# and python
> ----------------------------------------------------------
>
> Key: THRIFT-1528
> URL: https://issues.apache.org/jira/browse/THRIFT-1528
> Project: Thrift
> Issue Type: Bug
> Components: Python - Compiler
> Affects Versions: 0.8
> Reporter: Stefan Gmeiner
> Priority: Minor
>
> If a struct contains optional fields with default values the generated python code serialize differently than Java or C# code.
> In Java or C# optional fields are only serialized if a field was set by the client. If not the field is omited during serialization. This is possible because C#/Java maintains for each field a 'isset'-boolean which records if a field was set or not.
> However the generated python code does not have such a 'isset'-structure. It writes every field which is not equal None. As the constructor initialize the optional fields with their default value, these fields are written whether they are set or not.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (THRIFT-1528) Iconsistency in optional fields
between Java/C# and python
Posted by "Bryan Duxbury (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-1528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13221296#comment-13221296 ]
Bryan Duxbury commented on THRIFT-1528:
---------------------------------------
>From my perspective, its insane to have a field have a value and not to serialize it. In fact, the whole "isset" thing in Java is just a hack around the fact that you can't have null primitives. I would vote for standardizing on default values being considered set at object creation.
(An interesting detail is that in Java, the isset field only covers fields that's can't be natively set to null, so a default value for a struct, string, binary, or collection type actually would be serialized, while a bool, byte, short, int, long, or double would not be.)
> Iconsistency in optional fields between Java/C# and python
> ----------------------------------------------------------
>
> Key: THRIFT-1528
> URL: https://issues.apache.org/jira/browse/THRIFT-1528
> Project: Thrift
> Issue Type: Bug
> Components: Python - Compiler
> Affects Versions: 0.8
> Reporter: Stefan Gmeiner
> Priority: Minor
>
> If a struct contains optional fields with default values the generated python code serialize differently than Java or C# code.
> In Java or C# optional fields are only serialized if a field was set by the client. If not the field is omited during serialization. This is possible because C#/Java maintains for each field a 'isset'-boolean which records if a field was set or not.
> However the generated python code does not have such a 'isset'-structure. It writes every field which is not equal None. As the constructor initialize the optional fields with their default value, these fields are written whether they are set or not.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (THRIFT-1528) Iconsistency in optional fields
between Java/C# and python
Posted by "Jens Geyer (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-1528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13228762#comment-13228762 ]
Jens Geyer commented on THRIFT-1528:
------------------------------------
{quote}
service User(id, optional string name, optional bool isAwesome=false)
first call User(id=1, isAwesome=true)
second call User(id=1, name="foo")
If the second call had to serialize the default optional value no matter what it would cause "foo" to no longer be awesome even thought it was originally set in the first call and was an optional parameter for the second call.
{quote}
Mmmh.
In essence this would mean, that there there will always be an (explicit or implicit) input value.
If I don't overlook anything, we have two possible cases here:
# Either isAwesome is not really optional, since it always has an (explicit or implicit) value.
# Or, if we value the optional higher, omitted values would never be set, the default is obsolete.
This raises the big question, what the intention of the entire construct "optionally set with default" would be? What is the expected behaviour, exactly? Are there any use cases for this combination?
> Iconsistency in optional fields between Java/C# and python
> ----------------------------------------------------------
>
> Key: THRIFT-1528
> URL: https://issues.apache.org/jira/browse/THRIFT-1528
> Project: Thrift
> Issue Type: Bug
> Components: Python - Compiler
> Affects Versions: 0.8
> Reporter: Stefan Gmeiner
> Priority: Minor
>
> If a struct contains optional fields with default values the generated python code serialize differently than Java or C# code.
> In Java or C# optional fields are only serialized if a field was set by the client. If not the field is omited during serialization. This is possible because C#/Java maintains for each field a 'isset'-boolean which records if a field was set or not.
> However the generated python code does not have such a 'isset'-structure. It writes every field which is not equal None. As the constructor initialize the optional fields with their default value, these fields are written whether they are set or not.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (THRIFT-1528) Iconsistency in optional fields
between Java/C# and python
Posted by "Nate Rosenblum (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-1528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13220160#comment-13220160 ]
Nate Rosenblum commented on THRIFT-1528:
----------------------------------------
A similar inconsistency applies to C++ structures, where all optional fields that have default values are serialized (the isset predicate is initialized to true for such fields). These inconsistencies break applications that depend on consistency of serialized messages regardless of platform (e.g., computing HMAC on messages).
> Iconsistency in optional fields between Java/C# and python
> ----------------------------------------------------------
>
> Key: THRIFT-1528
> URL: https://issues.apache.org/jira/browse/THRIFT-1528
> Project: Thrift
> Issue Type: Bug
> Components: Python - Compiler
> Affects Versions: 0.8
> Reporter: Stefan Gmeiner
> Priority: Minor
>
> If a struct contains optional fields with default values the generated python code serialize differently than Java or C# code.
> In Java or C# optional fields are only serialized if a field was set by the client. If not the field is omited during serialization. This is possible because C#/Java maintains for each field a 'isset'-boolean which records if a field was set or not.
> However the generated python code does not have such a 'isset'-structure. It writes every field which is not equal None. As the constructor initialize the optional fields with their default value, these fields are written whether they are set or not.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (THRIFT-1528) Iconsistency in optional fields
between Java/C# and python
Posted by "Stefan Gmeiner (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-1528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13220731#comment-13220731 ]
Stefan Gmeiner commented on THRIFT-1528:
----------------------------------------
I found the following statement in [Thrift Missing Guide|http://diwakergupta.github.com/thrift-missing-guide/]:
{quote}
Changing a default value is generally OK, as long as you remember that default values are never sent over the wire. Thus, if a program receives a message in which a particular field isn’t set, the program will see the default value as it was defined in that program’s version of the protocol. It will NOT see the default value that was defined in the sender’s code.
{quote}
Example:
If you update a server with a changed default value you will get different behavior depending if you use Java or Python as old client. If the old Java client talks to the new server (without using the default field) the server will see the new default value whereas the the python client will sent the old default value and thus the server will get the old default value.
I'm not sure which one should be preferred but I think it should not depending on the client language used. I don't see an reason why some language should behave different than the static ones (to ensure binary compatibility the thrift compiler could provide a language option to control the generation of the code).
> Iconsistency in optional fields between Java/C# and python
> ----------------------------------------------------------
>
> Key: THRIFT-1528
> URL: https://issues.apache.org/jira/browse/THRIFT-1528
> Project: Thrift
> Issue Type: Bug
> Components: Python - Compiler
> Affects Versions: 0.8
> Reporter: Stefan Gmeiner
> Priority: Minor
>
> If a struct contains optional fields with default values the generated python code serialize differently than Java or C# code.
> In Java or C# optional fields are only serialized if a field was set by the client. If not the field is omited during serialization. This is possible because C#/Java maintains for each field a 'isset'-boolean which records if a field was set or not.
> However the generated python code does not have such a 'isset'-structure. It writes every field which is not equal None. As the constructor initialize the optional fields with their default value, these fields are written whether they are set or not.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (THRIFT-1528) Iconsistency in optional fields
between Java/C# and python
Posted by "Jake Farrell (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-1528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13227560#comment-13227560 ]
Jake Farrell commented on THRIFT-1528:
--------------------------------------
Diwaker: Autoboxing adds unnecessary overhead and should not be used in performance intensive code
> Iconsistency in optional fields between Java/C# and python
> ----------------------------------------------------------
>
> Key: THRIFT-1528
> URL: https://issues.apache.org/jira/browse/THRIFT-1528
> Project: Thrift
> Issue Type: Bug
> Components: Python - Compiler
> Affects Versions: 0.8
> Reporter: Stefan Gmeiner
> Priority: Minor
>
> If a struct contains optional fields with default values the generated python code serialize differently than Java or C# code.
> In Java or C# optional fields are only serialized if a field was set by the client. If not the field is omited during serialization. This is possible because C#/Java maintains for each field a 'isset'-boolean which records if a field was set or not.
> However the generated python code does not have such a 'isset'-structure. It writes every field which is not equal None. As the constructor initialize the optional fields with their default value, these fields are written whether they are set or not.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (THRIFT-1528) Iconsistency in optional fields
between Java/C# and python
Posted by "Jake Farrell (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-1528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13226951#comment-13226951 ]
Jake Farrell commented on THRIFT-1528:
--------------------------------------
using the following example of a update service you can not tell if an optional primitive parameter was set or its using the default value without isset.
service User(id, optional string name, optional bool isAwesome=false)
first call User(id=1, isAwesome=true)
second call User(id=1, name="foo")
If the second call had to serialize the default optional value no matter what it would cause "foo" to no longer be awesome even thought it was originally set in the first call and was an optional parameter for the second call. I would prefer to go with nullable types across the board as well, but since this currently is not an option for primitives I think we need to keep isset for those types for now.
> Iconsistency in optional fields between Java/C# and python
> ----------------------------------------------------------
>
> Key: THRIFT-1528
> URL: https://issues.apache.org/jira/browse/THRIFT-1528
> Project: Thrift
> Issue Type: Bug
> Components: Python - Compiler
> Affects Versions: 0.8
> Reporter: Stefan Gmeiner
> Priority: Minor
>
> If a struct contains optional fields with default values the generated python code serialize differently than Java or C# code.
> In Java or C# optional fields are only serialized if a field was set by the client. If not the field is omited during serialization. This is possible because C#/Java maintains for each field a 'isset'-boolean which records if a field was set or not.
> However the generated python code does not have such a 'isset'-structure. It writes every field which is not equal None. As the constructor initialize the optional fields with their default value, these fields are written whether they are set or not.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (THRIFT-1528) Iconsistency in optional fields
between Java/C# and python
Posted by "Jens Geyer (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-1528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13226901#comment-13226901 ]
Jens Geyer commented on THRIFT-1528:
------------------------------------
{quote}From my perspective, its insane to have a field have a value and not to serialize it. In fact, the whole "isset" thing in Java is just a hack around the fact that you can't have null primitives. I would vote for standardizing on default values being considered set at object creation.{quote}
+1
> Iconsistency in optional fields between Java/C# and python
> ----------------------------------------------------------
>
> Key: THRIFT-1528
> URL: https://issues.apache.org/jira/browse/THRIFT-1528
> Project: Thrift
> Issue Type: Bug
> Components: Python - Compiler
> Affects Versions: 0.8
> Reporter: Stefan Gmeiner
> Priority: Minor
>
> If a struct contains optional fields with default values the generated python code serialize differently than Java or C# code.
> In Java or C# optional fields are only serialized if a field was set by the client. If not the field is omited during serialization. This is possible because C#/Java maintains for each field a 'isset'-boolean which records if a field was set or not.
> However the generated python code does not have such a 'isset'-structure. It writes every field which is not equal None. As the constructor initialize the optional fields with their default value, these fields are written whether they are set or not.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (THRIFT-1528) Iconsistency in optional fields
between Java/C# and python
Posted by "Stefan Gmeiner (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-1528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13227343#comment-13227343 ]
Stefan Gmeiner commented on THRIFT-1528:
----------------------------------------
+1 for Bryan's
This would also fix the problem, that in Java fields could be set using public field access which will not set its corresponding isset-field.
> Iconsistency in optional fields between Java/C# and python
> ----------------------------------------------------------
>
> Key: THRIFT-1528
> URL: https://issues.apache.org/jira/browse/THRIFT-1528
> Project: Thrift
> Issue Type: Bug
> Components: Python - Compiler
> Affects Versions: 0.8
> Reporter: Stefan Gmeiner
> Priority: Minor
>
> If a struct contains optional fields with default values the generated python code serialize differently than Java or C# code.
> In Java or C# optional fields are only serialized if a field was set by the client. If not the field is omited during serialization. This is possible because C#/Java maintains for each field a 'isset'-boolean which records if a field was set or not.
> However the generated python code does not have such a 'isset'-structure. It writes every field which is not equal None. As the constructor initialize the optional fields with their default value, these fields are written whether they are set or not.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (THRIFT-1528) Iconsistency in optional fields
between Java/C# and python
Posted by "Will Pierce (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-1528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13220311#comment-13220311 ]
Will Pierce commented on THRIFT-1528:
-------------------------------------
i think the java and c# implementations should follow the c++ and python implementations, otherwise it doesn't make sense for a field to have a default value at all, if it isn't included when serializing, does it?
> Iconsistency in optional fields between Java/C# and python
> ----------------------------------------------------------
>
> Key: THRIFT-1528
> URL: https://issues.apache.org/jira/browse/THRIFT-1528
> Project: Thrift
> Issue Type: Bug
> Components: Python - Compiler
> Affects Versions: 0.8
> Reporter: Stefan Gmeiner
> Priority: Minor
>
> If a struct contains optional fields with default values the generated python code serialize differently than Java or C# code.
> In Java or C# optional fields are only serialized if a field was set by the client. If not the field is omited during serialization. This is possible because C#/Java maintains for each field a 'isset'-boolean which records if a field was set or not.
> However the generated python code does not have such a 'isset'-structure. It writes every field which is not equal None. As the constructor initialize the optional fields with their default value, these fields are written whether they are set or not.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Issue Comment Edited] (THRIFT-1528) Iconsistency in
optional fields between Java/C# and python
Posted by "Jens Geyer (Issue Comment Edited) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-1528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13226901#comment-13226901 ]
Jens Geyer edited comment on THRIFT-1528 at 3/10/12 5:21 PM:
-------------------------------------------------------------
{quote}From my perspective, its insane to have a field have a value and not to serialize it. [...] I would vote for standardizing on default values being considered set at object creation.{quote}
+1
was (Author: jensg):
{quote}From my perspective, its insane to have a field have a value and not to serialize it. In fact, the whole "isset" thing in Java is just a hack around the fact that you can't have null primitives. I would vote for standardizing on default values being considered set at object creation.{quote}
+1
> Iconsistency in optional fields between Java/C# and python
> ----------------------------------------------------------
>
> Key: THRIFT-1528
> URL: https://issues.apache.org/jira/browse/THRIFT-1528
> Project: Thrift
> Issue Type: Bug
> Components: Python - Compiler
> Affects Versions: 0.8
> Reporter: Stefan Gmeiner
> Priority: Minor
>
> If a struct contains optional fields with default values the generated python code serialize differently than Java or C# code.
> In Java or C# optional fields are only serialized if a field was set by the client. If not the field is omited during serialization. This is possible because C#/Java maintains for each field a 'isset'-boolean which records if a field was set or not.
> However the generated python code does not have such a 'isset'-structure. It writes every field which is not equal None. As the constructor initialize the optional fields with their default value, these fields are written whether they are set or not.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira