You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@thrift.apache.org by "roshanjrajan-zip (via GitHub)" <gi...@apache.org> on 2023/06/28 22:15:57 UTC

[GitHub] [thrift] roshanjrajan-zip opened a new pull request, #2824: Use Enum not string in Struct serialization/deserialization

roshanjrajan-zip opened a new pull request, #2824:
URL: https://github.com/apache/thrift/pull/2824

   <!-- Explain the changes in the pull request below: -->
   When serializing and deserializing Enums for Structs, we convert the Enum to a string instead of keeping it as an Enum. What this means is that the thrift file shows that an Enum is passed in the struct, when actually a string is passed in. This is very confusing, breaks the thrift contract and can arbitrarily lead to KeyError and such in the serialization/deserialization logic.
   
   Given a thrift file like the following:
   ```
   enum TestEnum {
     TestEnum0 = 0,
     TestEnum1 = 1,
   }
   
   struct TestStruct {
       1: optional string param1
       2: optional TestEnum param2
   }
   ```
   
   this is how you would create a struct before and after the change.
   ```python
   # Before
   TestStruct(param1="test_string", param2=TestEnum.TestEnum1.name) # or pass in "TestEnum1"
   
   # After
   TestStruct(param1="test_string", param2=TestEnum.TestEnum1)
   ```
   
   Serialization/Deserialization Code Change Example- 
   ```python
   class TestStruct(object):
       """
       Attributes:
        - param1
        - param2
   
       """
   
   
       def __init__(self, param1=None, param2=None,):
           self.param1 = param1
           self.param2 = param2
   
       def read(self, iprot):
           if iprot._fast_decode is not None and isinstance(iprot.trans, TTransport.CReadableTransport) and self.thrift_spec is not None:
               iprot._fast_decode(self, iprot, [self.__class__, self.thrift_spec])
               return
           iprot.readStructBegin()
           while True:
               (fname, ftype, fid) = iprot.readFieldBegin()
               if ftype == TType.STOP:
                   break
               if fid == 1:
                   if ftype == TType.STRING:
                       self.param1 = iprot.readString().decode('utf-8', errors='replace') if sys.version_info[0] == 2 else iprot.readString()
                   else:
                       iprot.skip(ftype)
               elif fid == 2:
                   if ftype == TType.I32:
   --                    self.param2 = TestEnum(iprot.readI32()).name
   ++                    self.param2 = TestEnum(iprot.readI32())
                   else:
                       iprot.skip(ftype)
               else:
                   iprot.skip(ftype)
               iprot.readFieldEnd()
           iprot.readStructEnd()
   
       def write(self, oprot):
           if oprot._fast_encode is not None and self.thrift_spec is not None:
               oprot.trans.write(oprot._fast_encode(self, [self.__class__, self.thrift_spec]))
               return
           oprot.writeStructBegin('TestStruct')
           if self.param1 is not None:
               oprot.writeFieldBegin('param1', TType.STRING, 1)
               oprot.writeString(self.param1.encode('utf-8') if sys.version_info[0] == 2 else self.param1)
               oprot.writeFieldEnd()
           if self.param2 is not None:
               oprot.writeFieldBegin('param2', TType.I32, 2)
   --            oprot.writeI32(TestEnum[self.param2].value)
   ++           oprot.writeI32(self.param2.value)
               oprot.writeFieldEnd()
           oprot.writeFieldStop()
           oprot.writeStructEnd()
   
       def validate(self):
           return
   
       def __repr__(self):
           L = ['%s=%r' % (key, value)
                for key, value in self.__dict__.items()]
           return '%s(%s)' % (self.__class__.__name__, ', '.join(L))
   
       def __eq__(self, other):
           return isinstance(other, self.__class__) and self.__dict__ == other.__dict__
   
       def __ne__(self, other):
           return not (self == other)
   all_structs.append(TestStruct)
   TestStruct.thrift_spec = (
       None,  # 0
       (1, TType.STRING, 'param1', 'UTF8', None, ),  # 1
       (2, TType.I32, 'param2', None, None, ),  # 2
   )
   fix_spec(all_structs)
   del all_structs
   ```
   
     
   
   <!-- We recommend you review the checklist/tips before submitting a pull request. -->
   
   - [ ] Did you create an [Apache Jira](https://issues.apache.org/jira/projects/THRIFT/issues/) ticket?  ([Request account here](https://selfserve.apache.org/jira-account.html), not required for trivial changes)
   - [ ] If a ticket exists: Does your pull request title follow the pattern "THRIFT-NNNN: describe my issue"?
   - [x] Did you squash your changes to a single commit?  (not required, but preferred)
   - [x] Did you do your best to avoid breaking changes?  If one was needed, did you label the Jira ticket with "Breaking-Change"?
   - [ ] If your change does not involve any code, include `[skip ci]` anywhere in the commit message to free up build resources.
   
   <!--
     The Contributing Guide at:
     https://github.com/apache/thrift/blob/master/CONTRIBUTING.md
     has more details and tips for committing properly.
   -->
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@thrift.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [thrift] fishy commented on pull request #2824: Use Enum or string in Struct serialization/deserialization for py:enum gen code

Posted by "fishy (via GitHub)" <gi...@apache.org>.

fishy commented on PR #2824:
URL: https://github.com/apache/thrift/pull/2824#issuecomment-1613761528

   right but this is python so it's very hard to force anything, I guess? we don't even have type annotation (I believe there's an open ticket for that just no one picking it up yet).
   
   so even if we move the logic to constructor the risk of someone doing other thing to override it is. I guess the current approach makes sense in the given circumstances.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@thrift.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org