You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@thrift.apache.org by "Manish Shah (JIRA)" <ji...@apache.org> on 2008/05/23 19:21:56 UTC
[jira] Created: (THRIFT-20) encoding/decoding a ThriftStruct
encoding/decoding a ThriftStruct
--------------------------------
Key: THRIFT-20
URL: https://issues.apache.org/jira/browse/THRIFT-20
Project: Thrift
Issue Type: New Feature
Reporter: Manish Shah
Priority: Minor
I'd like to define Thrift types without a service definition. The idea being that we can define a collection of Thrift types/objects/structs that can be used by many different services and can be encoded and decoded just the same.
Currently, a ThriftStruct gets de/encoded when being passed to (or returned from) a method in a defined service. I could do the de/encoding with a protocol and transport myself, but it would be more convenient if there were methods to abstract this work away.
What i'd like to do (and here is where i would like input) is to define an encode/decode method on ThriftStruct to take a type and a protocol and return back an encoded or decoded version of the type given. The procedure would make use of the TMemoryBuffer as the transport to use by the given protocol.
Anything wrong with this approach?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (THRIFT-20) encoding/decoding a ThriftStruct
Posted by "Kevin Clark (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-20?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kevin Clark resolved THRIFT-20.
-------------------------------
Resolution: Fixed
Resolved by push of patches in THRIFT-38. Thanks for the contribution!
> encoding/decoding a ThriftStruct
> --------------------------------
>
> Key: THRIFT-20
> URL: https://issues.apache.org/jira/browse/THRIFT-20
> Project: Thrift
> Issue Type: New Feature
> Components: Library (Ruby)
> Reporter: Manish Shah
> Assignee: Kevin Clark
> Priority: Minor
> Attachments: ruby_serializer_deserializer.patch
>
>
> I'd like to define Thrift types without a service definition. The idea being that we can define a collection of Thrift types/objects/structs that can be used by many different services and can be encoded and decoded just the same.
> Currently, a ThriftStruct gets de/encoded when being passed to (or returned from) a method in a defined service. I could do the de/encoding with a protocol and transport myself, but it would be more convenient if there were methods to abstract this work away.
> What i'd like to do (and here is where i would like input) is to define an encode/decode method on ThriftStruct to take a type and a protocol and return back an encoded or decoded version of the type given. The procedure would make use of the TMemoryBuffer as the transport to use by the given protocol.
> Anything wrong with this approach?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (THRIFT-20) encoding/decoding a ThriftStruct
Posted by "Manish Shah (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-20?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Manish Shah updated THRIFT-20:
------------------------------
Attachment: ruby_serializer_deserializer.patch
there were some bugs in issues with the old patch. this new one should resolve them.
> encoding/decoding a ThriftStruct
> --------------------------------
>
> Key: THRIFT-20
> URL: https://issues.apache.org/jira/browse/THRIFT-20
> Project: Thrift
> Issue Type: New Feature
> Reporter: Manish Shah
> Priority: Minor
> Attachments: ruby_serializer_deserializer.patch
>
>
> I'd like to define Thrift types without a service definition. The idea being that we can define a collection of Thrift types/objects/structs that can be used by many different services and can be encoded and decoded just the same.
> Currently, a ThriftStruct gets de/encoded when being passed to (or returned from) a method in a defined service. I could do the de/encoding with a protocol and transport myself, but it would be more convenient if there were methods to abstract this work away.
> What i'd like to do (and here is where i would like input) is to define an encode/decode method on ThriftStruct to take a type and a protocol and return back an encoded or decoded version of the type given. The procedure would make use of the TMemoryBuffer as the transport to use by the given protocol.
> Anything wrong with this approach?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (THRIFT-20) encoding/decoding a ThriftStruct
Posted by "Manish Shah (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-20?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Manish Shah updated THRIFT-20:
------------------------------
I created a patch to add ruby versions of TSerializer and TDeserializer, however, jira seemed to hang every time i tried to attach the file to the issue. I copied the patch below.
One quirk here i wanted to point out specific to the ruby libs is that TMemoryBuffer did not have a way to be intialized around an existing byte[]/buffer. I added a setBuffer method. This could be done similiarly by adding an optional buffer to the initialize block. I'm indifferent to either option.
Index: lib/rb/lib/thrift/TSerializer.rb
===================================================================
--- lib/rb/lib/thrift/TSerializer.rb (revision 0)
+++ lib/rb/lib/thrift/TSerializer.rb (revision 0)
@@ -0,0 +1,28 @@
+require File.dirname(__FILE__) + '/transport/ttransport'
+require File.dirname(__FILE__) + '/protocol/tbinaryprotocol'
+
+
+class TSerializer
+
+ attr_accessor :transport, :protocol
+
+ # Create a new TSerializer with the given Protocol Factory.
+ # Defaults to using TBinaryProtocolFactory
+ def initialize(protocolFactory = TBinaryProtocolFactory)
+ @transport = TMemoryBuffer.new
+ @protocol = protocolFactory.getProtocol(@transport)
+ end
+
+
+ # Serialize base (Thrift object) into a byte[] by:
+ # 1. clearing the byte[] in TMemoryBuffer
+ # 2. write the object into it via @protocol
+ # 3. grab the object out of TMemoryBuffer
+ def serialize(base)
+ transport.resetBuffer
+ base.write(protocol)
+ transport.getBuffer
+ end
+
+
+end
\ No newline at end of file
Index: lib/rb/lib/thrift/TDeserializer.rb
===================================================================
--- lib/rb/lib/thrift/TDeserializer.rb (revision 0)
+++ lib/rb/lib/thrift/TDeserializer.rb (revision 0)
@@ -0,0 +1,22 @@
+require File.dirname(__FILE__) + '/transport/ttransport'
+require File.dirname(__FILE__) + '/protocol/tbinaryprotocol'
+
+class TDeserializer
+
+ attr_accessor :transport, :protocol
+
+ def initialize(protocolFactory = TBinaryProtocolFactory)
+ @transport = TMemoryBuffer.new
+ @protocol = protocolFactory.getProtocol(@transport)
+ end
+
+ # Deserialize base (Thrift object) by:
+ # 1. set the transport to hold the bytes to interpret
+ # 2. use the protocol to read the data into base
+ def deserialize(base, bytes)
+ @transport.setBuffer(bytes)
+ base.read(@protocol)
+ base
+ end
+
+end
Index: lib/rb/lib/thrift/transport/ttransport.rb
===================================================================
--- lib/rb/lib/thrift/transport/ttransport.rb (revision 1041)
+++ lib/rb/lib/thrift/transport/ttransport.rb (working copy)
@@ -232,6 +232,10 @@
return @buf
end
+ def setBuffer(buf)
+ @buf = buf
+ end
+
def resetBuffer(new_buf = '')
@buf = new_buf
@sz = new_buf.length
> encoding/decoding a ThriftStruct
> --------------------------------
>
> Key: THRIFT-20
> URL: https://issues.apache.org/jira/browse/THRIFT-20
> Project: Thrift
> Issue Type: New Feature
> Reporter: Manish Shah
> Priority: Minor
>
> I'd like to define Thrift types without a service definition. The idea being that we can define a collection of Thrift types/objects/structs that can be used by many different services and can be encoded and decoded just the same.
> Currently, a ThriftStruct gets de/encoded when being passed to (or returned from) a method in a defined service. I could do the de/encoding with a protocol and transport myself, but it would be more convenient if there were methods to abstract this work away.
> What i'd like to do (and here is where i would like input) is to define an encode/decode method on ThriftStruct to take a type and a protocol and return back an encoded or decoded version of the type given. The procedure would make use of the TMemoryBuffer as the transport to use by the given protocol.
> Anything wrong with this approach?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (THRIFT-20) encoding/decoding a ThriftStruct
Posted by "David Reiss (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-20?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12599504#action_12599504 ]
David Reiss commented on THRIFT-20:
-----------------------------------
I'm not sure if we can accept a patch pasted in like this, because the normal upload procedure has a checkbox that allows you to state that the ASF can distribute this patch under the terms of the ASL. Could you try uploading the patch from a different browser, perhaps?
> encoding/decoding a ThriftStruct
> --------------------------------
>
> Key: THRIFT-20
> URL: https://issues.apache.org/jira/browse/THRIFT-20
> Project: Thrift
> Issue Type: New Feature
> Reporter: Manish Shah
> Priority: Minor
>
> I'd like to define Thrift types without a service definition. The idea being that we can define a collection of Thrift types/objects/structs that can be used by many different services and can be encoded and decoded just the same.
> Currently, a ThriftStruct gets de/encoded when being passed to (or returned from) a method in a defined service. I could do the de/encoding with a protocol and transport myself, but it would be more convenient if there were methods to abstract this work away.
> What i'd like to do (and here is where i would like input) is to define an encode/decode method on ThriftStruct to take a type and a protocol and return back an encoded or decoded version of the type given. The procedure would make use of the TMemoryBuffer as the transport to use by the given protocol.
> Anything wrong with this approach?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (THRIFT-20) encoding/decoding a ThriftStruct
Posted by "Kevin Ballard (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-20?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12603719#action_12603719 ]
Kevin Ballard commented on THRIFT-20:
-------------------------------------
I just committed a version of this to my own ruby stuff (pri/kballard/ruby_lib_rewrite), including specs, so when that gets accepted into svn this issue should be closed.
> encoding/decoding a ThriftStruct
> --------------------------------
>
> Key: THRIFT-20
> URL: https://issues.apache.org/jira/browse/THRIFT-20
> Project: Thrift
> Issue Type: New Feature
> Components: Library (Ruby)
> Reporter: Manish Shah
> Priority: Minor
> Attachments: ruby_serializer_deserializer.patch
>
>
> I'd like to define Thrift types without a service definition. The idea being that we can define a collection of Thrift types/objects/structs that can be used by many different services and can be encoded and decoded just the same.
> Currently, a ThriftStruct gets de/encoded when being passed to (or returned from) a method in a defined service. I could do the de/encoding with a protocol and transport myself, but it would be more convenient if there were methods to abstract this work away.
> What i'd like to do (and here is where i would like input) is to define an encode/decode method on ThriftStruct to take a type and a protocol and return back an encoded or decoded version of the type given. The procedure would make use of the TMemoryBuffer as the transport to use by the given protocol.
> Anything wrong with this approach?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (THRIFT-20) encoding/decoding a ThriftStruct
Posted by "Manish Shah (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-20?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Manish Shah updated THRIFT-20:
------------------------------
Attachment: ruby_serializer_deserializer.patch
> encoding/decoding a ThriftStruct
> --------------------------------
>
> Key: THRIFT-20
> URL: https://issues.apache.org/jira/browse/THRIFT-20
> Project: Thrift
> Issue Type: New Feature
> Reporter: Manish Shah
> Priority: Minor
> Attachments: ruby_serializer_deserializer.patch
>
>
> I'd like to define Thrift types without a service definition. The idea being that we can define a collection of Thrift types/objects/structs that can be used by many different services and can be encoded and decoded just the same.
> Currently, a ThriftStruct gets de/encoded when being passed to (or returned from) a method in a defined service. I could do the de/encoding with a protocol and transport myself, but it would be more convenient if there were methods to abstract this work away.
> What i'd like to do (and here is where i would like input) is to define an encode/decode method on ThriftStruct to take a type and a protocol and return back an encoded or decoded version of the type given. The procedure would make use of the TMemoryBuffer as the transport to use by the given protocol.
> Anything wrong with this approach?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (THRIFT-20) encoding/decoding a ThriftStruct
Posted by "Manish Shah (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-20?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Manish Shah updated THRIFT-20:
------------------------------
Attachment: (was: ruby_serializer_deserializer.patch)
> encoding/decoding a ThriftStruct
> --------------------------------
>
> Key: THRIFT-20
> URL: https://issues.apache.org/jira/browse/THRIFT-20
> Project: Thrift
> Issue Type: New Feature
> Reporter: Manish Shah
> Priority: Minor
>
> I'd like to define Thrift types without a service definition. The idea being that we can define a collection of Thrift types/objects/structs that can be used by many different services and can be encoded and decoded just the same.
> Currently, a ThriftStruct gets de/encoded when being passed to (or returned from) a method in a defined service. I could do the de/encoding with a protocol and transport myself, but it would be more convenient if there were methods to abstract this work away.
> What i'd like to do (and here is where i would like input) is to define an encode/decode method on ThriftStruct to take a type and a protocol and return back an encoded or decoded version of the type given. The procedure would make use of the TMemoryBuffer as the transport to use by the given protocol.
> Anything wrong with this approach?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (THRIFT-20) encoding/decoding a ThriftStruct
Posted by "Manish Shah (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-20?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Manish Shah updated THRIFT-20:
------------------------------
Attachment: ruby_serializer_deserializer.patch
> encoding/decoding a ThriftStruct
> --------------------------------
>
> Key: THRIFT-20
> URL: https://issues.apache.org/jira/browse/THRIFT-20
> Project: Thrift
> Issue Type: New Feature
> Reporter: Manish Shah
> Priority: Minor
> Attachments: ruby_serializer_deserializer.patch
>
>
> I'd like to define Thrift types without a service definition. The idea being that we can define a collection of Thrift types/objects/structs that can be used by many different services and can be encoded and decoded just the same.
> Currently, a ThriftStruct gets de/encoded when being passed to (or returned from) a method in a defined service. I could do the de/encoding with a protocol and transport myself, but it would be more convenient if there were methods to abstract this work away.
> What i'd like to do (and here is where i would like input) is to define an encode/decode method on ThriftStruct to take a type and a protocol and return back an encoded or decoded version of the type given. The procedure would make use of the TMemoryBuffer as the transport to use by the given protocol.
> Anything wrong with this approach?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (THRIFT-20) encoding/decoding a ThriftStruct
Posted by "Bryan Duxbury (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-20?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12599457#action_12599457 ]
Bryan Duxbury commented on THRIFT-20:
-------------------------------------
+1 to making TSerializer and TDeserializer in Ruby (and other languages).
> encoding/decoding a ThriftStruct
> --------------------------------
>
> Key: THRIFT-20
> URL: https://issues.apache.org/jira/browse/THRIFT-20
> Project: Thrift
> Issue Type: New Feature
> Reporter: Manish Shah
> Priority: Minor
>
> I'd like to define Thrift types without a service definition. The idea being that we can define a collection of Thrift types/objects/structs that can be used by many different services and can be encoded and decoded just the same.
> Currently, a ThriftStruct gets de/encoded when being passed to (or returned from) a method in a defined service. I could do the de/encoding with a protocol and transport myself, but it would be more convenient if there were methods to abstract this work away.
> What i'd like to do (and here is where i would like input) is to define an encode/decode method on ThriftStruct to take a type and a protocol and return back an encoded or decoded version of the type given. The procedure would make use of the TMemoryBuffer as the transport to use by the given protocol.
> Anything wrong with this approach?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (THRIFT-20) encoding/decoding a ThriftStruct
Posted by "David Reiss (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-20?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12599446#action_12599446 ]
David Reiss commented on THRIFT-20:
-----------------------------------
We already have this in Java as TSerializer and TDeserializer. The code in other languages should be pretty similar. I'd prefer to see it as an external function rather than a method of the structure so that the structure code doesn't become dependent on any one transport implementation.
> encoding/decoding a ThriftStruct
> --------------------------------
>
> Key: THRIFT-20
> URL: https://issues.apache.org/jira/browse/THRIFT-20
> Project: Thrift
> Issue Type: New Feature
> Reporter: Manish Shah
> Priority: Minor
>
> I'd like to define Thrift types without a service definition. The idea being that we can define a collection of Thrift types/objects/structs that can be used by many different services and can be encoded and decoded just the same.
> Currently, a ThriftStruct gets de/encoded when being passed to (or returned from) a method in a defined service. I could do the de/encoding with a protocol and transport myself, but it would be more convenient if there were methods to abstract this work away.
> What i'd like to do (and here is where i would like input) is to define an encode/decode method on ThriftStruct to take a type and a protocol and return back an encoded or decoded version of the type given. The procedure would make use of the TMemoryBuffer as the transport to use by the given protocol.
> Anything wrong with this approach?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (THRIFT-20) encoding/decoding a ThriftStruct
Posted by "Kevin Clark (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-20?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kevin Clark reassigned THRIFT-20:
---------------------------------
Assignee: Kevin Clark
> encoding/decoding a ThriftStruct
> --------------------------------
>
> Key: THRIFT-20
> URL: https://issues.apache.org/jira/browse/THRIFT-20
> Project: Thrift
> Issue Type: New Feature
> Components: Library (Ruby)
> Reporter: Manish Shah
> Assignee: Kevin Clark
> Priority: Minor
> Attachments: ruby_serializer_deserializer.patch
>
>
> I'd like to define Thrift types without a service definition. The idea being that we can define a collection of Thrift types/objects/structs that can be used by many different services and can be encoded and decoded just the same.
> Currently, a ThriftStruct gets de/encoded when being passed to (or returned from) a method in a defined service. I could do the de/encoding with a protocol and transport myself, but it would be more convenient if there were methods to abstract this work away.
> What i'd like to do (and here is where i would like input) is to define an encode/decode method on ThriftStruct to take a type and a protocol and return back an encoded or decoded version of the type given. The procedure would make use of the TMemoryBuffer as the transport to use by the given protocol.
> Anything wrong with this approach?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (THRIFT-20) encoding/decoding a ThriftStruct
Posted by "Manish Shah (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-20?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Manish Shah updated THRIFT-20:
------------------------------
Attachment: (was: ruby_serializer_deserializer.patch)
> encoding/decoding a ThriftStruct
> --------------------------------
>
> Key: THRIFT-20
> URL: https://issues.apache.org/jira/browse/THRIFT-20
> Project: Thrift
> Issue Type: New Feature
> Reporter: Manish Shah
> Priority: Minor
>
> I'd like to define Thrift types without a service definition. The idea being that we can define a collection of Thrift types/objects/structs that can be used by many different services and can be encoded and decoded just the same.
> Currently, a ThriftStruct gets de/encoded when being passed to (or returned from) a method in a defined service. I could do the de/encoding with a protocol and transport myself, but it would be more convenient if there were methods to abstract this work away.
> What i'd like to do (and here is where i would like input) is to define an encode/decode method on ThriftStruct to take a type and a protocol and return back an encoded or decoded version of the type given. The procedure would make use of the TMemoryBuffer as the transport to use by the given protocol.
> Anything wrong with this approach?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (THRIFT-20) encoding/decoding a ThriftStruct
Posted by "David Reiss (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-20?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
David Reiss updated THRIFT-20:
------------------------------
Component/s: Library (Ruby)
> encoding/decoding a ThriftStruct
> --------------------------------
>
> Key: THRIFT-20
> URL: https://issues.apache.org/jira/browse/THRIFT-20
> Project: Thrift
> Issue Type: New Feature
> Components: Library (Ruby)
> Reporter: Manish Shah
> Priority: Minor
> Attachments: ruby_serializer_deserializer.patch
>
>
> I'd like to define Thrift types without a service definition. The idea being that we can define a collection of Thrift types/objects/structs that can be used by many different services and can be encoded and decoded just the same.
> Currently, a ThriftStruct gets de/encoded when being passed to (or returned from) a method in a defined service. I could do the de/encoding with a protocol and transport myself, but it would be more convenient if there were methods to abstract this work away.
> What i'd like to do (and here is where i would like input) is to define an encode/decode method on ThriftStruct to take a type and a protocol and return back an encoded or decoded version of the type given. The procedure would make use of the TMemoryBuffer as the transport to use by the given protocol.
> Anything wrong with this approach?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.