You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@thrift.apache.org by Bert Douglas <be...@tplogic.com> on 2012/07/12 06:48:14 UTC
erlang to c++ problems
I am using the same message schema, or thrift file in c++ and in erlang.
Also setting same three fields in a large struct/record.
Looking at network with wireshark.
Erlang client sends this:
0040 80 01 00 01 00 00 00 06 64 65 74 65 63 74 ........ ..detect
0050 00 00 00 00 0c 00 01 08 00 01 00 00 00 7b 04 00 ........ .....{..
0060 02 40 5e dd 3a 92 a3 05 53 00 0c 00 02 08 00 01 .@^.:... S.......
0070 00 00 00 2a 0c 00 03 00 0c 00 04 00 00 00 ...*.... ......
C++ client sends this:
80 01 00 01 00 00 00 06 64 65 74 65 63 74 ........ ..detect
0050 00 00 00 00 0c 00 01 08 00 01 00 00 00 7b 04 00 ........ .....{..
0060 02 40 fe 24 0c 9f be 76 c9 00 0c 00 02 08 00 01 .@.$...v ........
0070 00 00 03 db 08 00 02 00 00 00 00 0c 00 03 08 00 ........ ........
0080 01 00 00 00 00 04 00 02 00 00 00 00 00 00 00 00 ........ ........
0090 04 00 03 00 00 00 00 00 00 00 00 04 00 04 00 00 ........ ........
00a0 00 00 00 00 00 00 04 00 05 00 00 00 00 00 00 00 ........ ........
00b0 00 00 0c 00 04 08 00 01 00 00 00 00 04 00 02 00 ........ ........
00c0 00 00 00 00 00 00 00 04 00 03 00 00 00 00 00 00 ........ ........
00d0 00 00 04 00 04 00 00 00 00 00 00 00 00 04 00 05 ........ ........
00e0 00 00 00 00 00 00 00 00 00 04 00 05 00 00 00 00 ........ ........
00f0 00 00 00 00 04 00 06 00 00 00 00 00 00 00 00 08 ........ ........
0100 00 07 00 00 00 00 08 00 08 00 00 00 00 08 00 09 ........ ........
0110 00 00 00 00 04 00 0a 00 00 00 00 00 00 00 00 04 ........ ........
0120 00 0b 00 00 00 00 00 00 00 00 04 00 0c 00 00 00 ........ ........
0130 00 00 00 00 00 00 00 .......
I would like to dis-assemble this by hand from docs to understand the
meaning of what is being sent.
However, so far I have been unable to find a description of the wire format.
By way of example, the google protocol buffer encoding is described here:
https://developers.google.com/protocol-buffers/docs/encoding
Is there a similar document for thrift ?
Thanks,
Bert Douglas
Re: erlang to c++ problems
Posted by Bert Douglas <be...@tplogic.com>.
Hi Guys,
Thanks for the replies.
As for questions -
-- I have not tried json.
-- Did not use any exceptions.
Sadly I am going to give up on thrift for my current project. I will give
it another shot in future.
This morning I started over with protocol buffers after spending two weeks
with thrift.
The c++ part is working with no problem.
Tomorrow will take a look at the erlang side. But I have seen the docs for
wire format for protobufs,
and I think that even if I have to code it entirely from scratch in erlang,
I will be ahead.
At least it is something I know I can do with confidence in a bounded
period of time.
The same can't be said for thrift.
--Bert
On Thu, Jul 12, 2012 at 1:59 PM, Roger Meier <ro...@bufferoverflow.ch>wrote:
> Do you have the same issue with json?
> Do you use any special Exceptions?
>
> Same behavior with other languages?
> Cross language test suite is still open...
> https://issues.apache.org/jira/browse/THRIFT-847
> The first step is test/test.sh which implements cross language tests, see
>
> https://builds.apache.org/view/S-Z/view/Thrift/job/Thrift/lastSuccessfulBuil
> d/artifact/thrift/test/test.log
>
> lib/cpp/src/thrift/protocol/TBinaryProtocol.tcc might help a bit.
>
> All the best!
>
> roger
> ;-r
>
> PS: build servers are unable to build erlang!
> See https://issues.apache.org/jira/browse/THRIFT-1357
>
>
> > -----Ursprüngliche Nachricht-----
> > Von: Rush Manbert [mailto:rush@manbert.com]
> > Gesendet: Donnerstag, 12. Juli 2012 19:16
> > An: dev@thrift.apache.org
> > Betreff: Re: erlang to c++ problems
> >
> > On Jul 11, 2012, at 9:48 PM, Bert Douglas wrote:
> >
> > > I am using the same message schema, or thrift file in c++ and in
> erlang.
> > > Also setting same three fields in a large struct/record.
> > > Looking at network with wireshark.
> > >
> > > Erlang client sends this:
> > >
> > > 0040 80 01 00 01 00 00 00 06 64 65 74 65 63 74 ........
> ..detect
> > > 0050 00 00 00 00 0c 00 01 08 00 01 00 00 00 7b 04 00 ........
> .....{..
> > > 0060 02 40 5e dd 3a 92 a3 05 53 00 0c 00 02 08 00 01 .@^.:...
> S.......
> > > 0070 00 00 00 2a 0c 00 03 00 0c 00 04 00 00 00 ...*....
> ......
> > >
> > > C++ client sends this:
> > >
> > > 80 01 00 01 00 00 00 06 64 65 74 65 63 74 ........
> ..detect
> > > 0050 00 00 00 00 0c 00 01 08 00 01 00 00 00 7b 04 00 ........
> .....{..
> > > 0060 02 40 fe 24 0c 9f be 76 c9 00 0c 00 02 08 00 01 .@.$...v
> ........
> > > 0070 00 00 03 db 08 00 02 00 00 00 00 0c 00 03 08 00 ........
> ........
> > > 0080 01 00 00 00 00 04 00 02 00 00 00 00 00 00 00 00 ........
> ........
> > > 0090 04 00 03 00 00 00 00 00 00 00 00 04 00 04 00 00 ........
> ........
> > > 00a0 00 00 00 00 00 00 04 00 05 00 00 00 00 00 00 00 ........
> ........
> > > 00b0 00 00 0c 00 04 08 00 01 00 00 00 00 04 00 02 00 ........
> ........
> > > 00c0 00 00 00 00 00 00 00 04 00 03 00 00 00 00 00 00 ........
> ........
> > > 00d0 00 00 04 00 04 00 00 00 00 00 00 00 00 04 00 05 ........
> ........
> > > 00e0 00 00 00 00 00 00 00 00 00 04 00 05 00 00 00 00 ........
> ........
> > > 00f0 00 00 00 00 04 00 06 00 00 00 00 00 00 00 00 08 ........
> ........
> > > 0100 00 07 00 00 00 00 08 00 08 00 00 00 00 08 00 09 ........
> ........
> > > 0110 00 00 00 00 04 00 0a 00 00 00 00 00 00 00 00 04 ........
> ........
> > > 0120 00 0b 00 00 00 00 00 00 00 00 04 00 0c 00 00 00 ........
> ........
> > > 0130 00 00 00 00 00 00 00 .......
> > >
> > > I would like to dis-assemble this by hand from docs to understand the
> > > meaning of what is being sent.
> > >
> > > However, so far I have been unable to find a description of the wire
> > format.
> > >
> > > By way of example, the google protocol buffer encoding is described
> here:
> > > https://developers.google.com/protocol-buffers/docs/encoding
> > >
> > > Is there a similar document for thrift ?
> >
> > Hi Bert,
> >
> > AFAIK, there is no such document for Thrift. However, for someone who is
> > looking at the packets with wireshark I feel it's reasonable to suggest
> that
> > you just trace through the client side message send. Just set a
> breakpoint
> in
> > the write() method in your *_types.cpp file and watch how it fills the
> > transmit buffer.
> >
> > This is good for the C++ client. I'm afraid that I have no idea how to do
> the
> > equivalent with the erlang side.
> >
> > Best regards,
> > Rush
>
>
>
AW: erlang to c++ problems
Posted by Roger Meier <ro...@bufferoverflow.ch>.
Do you have the same issue with json?
Do you use any special Exceptions?
Same behavior with other languages?
Cross language test suite is still open...
https://issues.apache.org/jira/browse/THRIFT-847
The first step is test/test.sh which implements cross language tests, see
https://builds.apache.org/view/S-Z/view/Thrift/job/Thrift/lastSuccessfulBuil
d/artifact/thrift/test/test.log
lib/cpp/src/thrift/protocol/TBinaryProtocol.tcc might help a bit.
All the best!
roger
;-r
PS: build servers are unable to build erlang!
See https://issues.apache.org/jira/browse/THRIFT-1357
> -----Ursprüngliche Nachricht-----
> Von: Rush Manbert [mailto:rush@manbert.com]
> Gesendet: Donnerstag, 12. Juli 2012 19:16
> An: dev@thrift.apache.org
> Betreff: Re: erlang to c++ problems
>
> On Jul 11, 2012, at 9:48 PM, Bert Douglas wrote:
>
> > I am using the same message schema, or thrift file in c++ and in erlang.
> > Also setting same three fields in a large struct/record.
> > Looking at network with wireshark.
> >
> > Erlang client sends this:
> >
> > 0040 80 01 00 01 00 00 00 06 64 65 74 65 63 74 ........
..detect
> > 0050 00 00 00 00 0c 00 01 08 00 01 00 00 00 7b 04 00 ........
.....{..
> > 0060 02 40 5e dd 3a 92 a3 05 53 00 0c 00 02 08 00 01 .@^.:...
S.......
> > 0070 00 00 00 2a 0c 00 03 00 0c 00 04 00 00 00 ...*.... ......
> >
> > C++ client sends this:
> >
> > 80 01 00 01 00 00 00 06 64 65 74 65 63 74 ........
..detect
> > 0050 00 00 00 00 0c 00 01 08 00 01 00 00 00 7b 04 00 ........
.....{..
> > 0060 02 40 fe 24 0c 9f be 76 c9 00 0c 00 02 08 00 01 .@.$...v
........
> > 0070 00 00 03 db 08 00 02 00 00 00 00 0c 00 03 08 00 ........
........
> > 0080 01 00 00 00 00 04 00 02 00 00 00 00 00 00 00 00 ........
........
> > 0090 04 00 03 00 00 00 00 00 00 00 00 04 00 04 00 00 ........
........
> > 00a0 00 00 00 00 00 00 04 00 05 00 00 00 00 00 00 00 ........
........
> > 00b0 00 00 0c 00 04 08 00 01 00 00 00 00 04 00 02 00 ........
........
> > 00c0 00 00 00 00 00 00 00 04 00 03 00 00 00 00 00 00 ........
........
> > 00d0 00 00 04 00 04 00 00 00 00 00 00 00 00 04 00 05 ........
........
> > 00e0 00 00 00 00 00 00 00 00 00 04 00 05 00 00 00 00 ........
........
> > 00f0 00 00 00 00 04 00 06 00 00 00 00 00 00 00 00 08 ........
........
> > 0100 00 07 00 00 00 00 08 00 08 00 00 00 00 08 00 09 ........
........
> > 0110 00 00 00 00 04 00 0a 00 00 00 00 00 00 00 00 04 ........
........
> > 0120 00 0b 00 00 00 00 00 00 00 00 04 00 0c 00 00 00 ........
........
> > 0130 00 00 00 00 00 00 00 .......
> >
> > I would like to dis-assemble this by hand from docs to understand the
> > meaning of what is being sent.
> >
> > However, so far I have been unable to find a description of the wire
> format.
> >
> > By way of example, the google protocol buffer encoding is described
here:
> > https://developers.google.com/protocol-buffers/docs/encoding
> >
> > Is there a similar document for thrift ?
>
> Hi Bert,
>
> AFAIK, there is no such document for Thrift. However, for someone who is
> looking at the packets with wireshark I feel it's reasonable to suggest
that
> you just trace through the client side message send. Just set a breakpoint
in
> the write() method in your *_types.cpp file and watch how it fills the
> transmit buffer.
>
> This is good for the C++ client. I'm afraid that I have no idea how to do
the
> equivalent with the erlang side.
>
> Best regards,
> Rush
Re: erlang to c++ problems
Posted by Rush Manbert <ru...@manbert.com>.
On Jul 11, 2012, at 9:48 PM, Bert Douglas wrote:
> I am using the same message schema, or thrift file in c++ and in erlang.
> Also setting same three fields in a large struct/record.
> Looking at network with wireshark.
>
> Erlang client sends this:
>
> 0040 80 01 00 01 00 00 00 06 64 65 74 65 63 74 ........ ..detect
> 0050 00 00 00 00 0c 00 01 08 00 01 00 00 00 7b 04 00 ........ .....{..
> 0060 02 40 5e dd 3a 92 a3 05 53 00 0c 00 02 08 00 01 .@^.:... S.......
> 0070 00 00 00 2a 0c 00 03 00 0c 00 04 00 00 00 ...*.... ......
>
> C++ client sends this:
>
> 80 01 00 01 00 00 00 06 64 65 74 65 63 74 ........ ..detect
> 0050 00 00 00 00 0c 00 01 08 00 01 00 00 00 7b 04 00 ........ .....{..
> 0060 02 40 fe 24 0c 9f be 76 c9 00 0c 00 02 08 00 01 .@.$...v ........
> 0070 00 00 03 db 08 00 02 00 00 00 00 0c 00 03 08 00 ........ ........
> 0080 01 00 00 00 00 04 00 02 00 00 00 00 00 00 00 00 ........ ........
> 0090 04 00 03 00 00 00 00 00 00 00 00 04 00 04 00 00 ........ ........
> 00a0 00 00 00 00 00 00 04 00 05 00 00 00 00 00 00 00 ........ ........
> 00b0 00 00 0c 00 04 08 00 01 00 00 00 00 04 00 02 00 ........ ........
> 00c0 00 00 00 00 00 00 00 04 00 03 00 00 00 00 00 00 ........ ........
> 00d0 00 00 04 00 04 00 00 00 00 00 00 00 00 04 00 05 ........ ........
> 00e0 00 00 00 00 00 00 00 00 00 04 00 05 00 00 00 00 ........ ........
> 00f0 00 00 00 00 04 00 06 00 00 00 00 00 00 00 00 08 ........ ........
> 0100 00 07 00 00 00 00 08 00 08 00 00 00 00 08 00 09 ........ ........
> 0110 00 00 00 00 04 00 0a 00 00 00 00 00 00 00 00 04 ........ ........
> 0120 00 0b 00 00 00 00 00 00 00 00 04 00 0c 00 00 00 ........ ........
> 0130 00 00 00 00 00 00 00 .......
>
> I would like to dis-assemble this by hand from docs to understand the
> meaning of what is being sent.
>
> However, so far I have been unable to find a description of the wire format.
>
> By way of example, the google protocol buffer encoding is described here:
> https://developers.google.com/protocol-buffers/docs/encoding
>
> Is there a similar document for thrift ?
Hi Bert,
AFAIK, there is no such document for Thrift. However, for someone who is looking at the packets with wireshark I feel it's reasonable to suggest that you just trace through the client side message send. Just set a breakpoint in the write() method in your *_types.cpp file and watch how it fills the transmit buffer.
This is good for the C++ client. I'm afraid that I have no idea how to do the equivalent with the erlang side.
Best regards,
Rush