Re: thrift compile to CPython objects?

["John R. Frank" <jr...@mit.edu>]

>> Have you guys tried the fastbinary protocol or twisted?

after reading more source (and some great help from Randy Abernethy), I 
*think* I understand how to use fastbinary in python.

For an application to cause thrift-generated python code to use 
fastbinary, it seems necessary to do something like this try/except import 
statement to select between TBinaryProtocol and TBinaryProtocolAccelerated

Is this the best practice?  Is there a reason that TBinaryProtocol does 
not do this for us under the hood?

     from thrift.protocol.TBinaryProtocol import TBinaryProtocol, TBinaryProtocolAccelerated
     fastbinary_import_failure = None
     try:
         from thrift.protocol import fastbinary
         ## use faster C program to read/write
         protocol = TBinaryProtocolAccelerated

     except Exception, exc:
         fastbinary_import_failure = exc
         ## fall back to pure python
         protocol = TBinaryProtocol



Also, in order to make TBinaryProtocolAccelerated work with 
TBufferedTransport wrapping a flat file, I had to add this "readAll" 
method to the file object before passing into TBufferedTransport.

How does this differ from simply using read(sz)?

     def readAll(self, sz):
         '''
         This method allows TBinaryProtocolAccelerated to actually function.
         Copied from here
         http://svn.apache.org/repos/asf/hive/trunk/service/lib/py/thrift/transport/TTransport.py
         '''
         buff = ''
         have = 0
         while (have < sz):
             chunk = self.read(sz-have)
             have += len(chunk)
             buff += chunk

             if len(chunk) == 0:
                 raise EOFError()

         return buff


The code is all here:

https://github.com/trec-kba/streamcorpus/commit/68d752cf7f09c726bf5701324a65d78787349570


jrf