int to double issues in TJSONProtocol

[Hussein Elgridly <hu...@broadinstitute.org>]

I spent a few hours this afternoon tracking down a frustrating bug that
stemmed from me accidentally assigning an int to a struct field that was
defined as a double.

The Python bindings didn't warn me that I was assigning a value of the
wrong type, nor did it convert between the two for me; instead, it went
ahead and serialized { "dbl" : 1 }, causing the receiving server to throw
"org.apache.thrift.protocol.TProtocolException: Unexpected character:}" and
return an HTTP 500.

The lack of specific information had me barking up many wrong trees before
I found the issue. There are a lot of }s in TJSONProtocol! I'm not sure
what I would have done had I not been able to generate the same API call
through a different application and compare with tcpflow.

May I suggest:

1. Throwing an exception or warning when attempting to write data of the
wrong type according to the Thrift spec; and
2. Adding more information (like position) to the "Unexpected character"
exception when receiving malformed data

This would aid greatly in writing Thrift code in dynamically typed
languages.

Thanks,

Hussein Elgridly
Senior Software Engineer, DSDE
The Broad Institute of MIT and Harvard

Re: int to double issues in TJSONProtocol

[Roger Meier <ro...@bufferoverflow.ch>]

Hi Hussein

Quoting Hussein Elgridly <hu...@broadinstitute.org>:

> I spent a few hours this afternoon tracking down a frustrating bug that
> stemmed from me accidentally assigning an int to a struct field that was
> defined as a double.
>
> The Python bindings didn't warn me that I was assigning a value of the
> wrong type, nor did it convert between the two for me; instead, it went
> ahead and serialized { "dbl" : 1 }, causing the receiving server to throw
> "org.apache.thrift.protocol.TProtocolException: Unexpected character:}" and
> return an HTTP 500.
>
> The lack of specific information had me barking up many wrong trees before
> I found the issue. There are a lot of }s in TJSONProtocol! I'm not sure
> what I would have done had I not been able to generate the same API call
> through a different application and compare with tcpflow.
>
> May I suggest:
>
> 1. Throwing an exception or warning when attempting to write data of the
> wrong type according to the Thrift spec; and
> 2. Adding more information (like position) to the "Unexpected character"
> exception when receiving malformed data
>
> This would aid greatly in writing Thrift code in dynamically typed
> languages.
Yes, this would be a nice improvement, feel free to contribute it:
https://thrift.apache.org/docs/HowToContribute

all the best!
roger


>
> Thanks,
>
> Hussein Elgridly
> Senior Software Engineer, DSDE
> The Broad Institute of MIT and Harvard