You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@thrift.apache.org by "j.barrett Strausser" <j....@gmail.com> on 2016/01/31 19:56:16 UTC
Thrift Transport LifeCycle
What is the best practice for lifecycle of a transport?
I'm finding that opening a transport per call is not effective. In
particular the below code seems to spend ~5 seconds opening up the
transport then ~300s per call for my actual service invocation. The 300ms
is more than acceptable.
Should I simply open the transport once per client and close at the end of
the application? If I do that an I experience transient network
disconnects, what will be client side exceptions thrown?
My client batches request so that the client is not overly chatty. Meaning
every 30s or so a message of a 1MB or so will be sent. Then approx ~2 mins
I will send a large message of size 15MB to 50MB
def make_transport_context(transport_context, rpc_type):
# Make socket
transport = TSSLSocket.TSSLSocket(transport_context.host,
transport_context.port,
ca_certs=transport_context.ca_certs,
validate=False)
transport.setTimeout(ms=transport_context.timeout_in_millis)
# Buffering is critical. Raw sockets are very slow
transport = TTransport.TBufferedTransport(transport)
# Wrap in a protocol
protocol = TBinaryProtocol.TBinaryProtocol(transport)
# Create a client to use the protocol encoder
client = rpc_type.Client(protocol)
return client, transport
@contextlib.contextmanager
def rpc_thrift_context(transport_context):
client, transport = make_transport_context(transport_context, DataRPC)
try:
transport.open()
yield client
finally:
transport.close()
Also asked at :
http://stackoverflow.com/questions/35117943/proper-lifecycle-of-python-thrift-client-transport
If you want those sweet SO points.
--
https://github.com/bearrito
@deepbearrito
Re: Thrift Transport LifeCycle
Posted by Randy Abernethy <ra...@apache.org>.
Connections are expensive and every transport open makes a connection. If
you can manage it, making a single connection and then issuing multiple
calls over that connection will save a considerable amount of overhead.
On Sun, Jan 31, 2016 at 10:56 AM, j.barrett Strausser <
j.barrett.strausser@gmail.com> wrote:
> What is the best practice for lifecycle of a transport?
>
> I'm finding that opening a transport per call is not effective. In
> particular the below code seems to spend ~5 seconds opening up the
> transport then ~300s per call for my actual service invocation. The 300ms
> is more than acceptable.
>
> Should I simply open the transport once per client and close at the end of
> the application? If I do that an I experience transient network
> disconnects, what will be client side exceptions thrown?
>
> My client batches request so that the client is not overly chatty. Meaning
> every 30s or so a message of a 1MB or so will be sent. Then approx ~2 mins
> I will send a large message of size 15MB to 50MB
>
> def make_transport_context(transport_context, rpc_type):
> # Make socket
>
>
> transport = TSSLSocket.TSSLSocket(transport_context.host,
> transport_context.port,
>
> ca_certs=transport_context.ca_certs,
> validate=False)
>
> transport.setTimeout(ms=transport_context.timeout_in_millis)
> # Buffering is critical. Raw sockets are very slow
> transport = TTransport.TBufferedTransport(transport)
>
> # Wrap in a protocol
> protocol = TBinaryProtocol.TBinaryProtocol(transport)
>
> # Create a client to use the protocol encoder
> client = rpc_type.Client(protocol)
> return client, transport
>
> @contextlib.contextmanager
> def rpc_thrift_context(transport_context):
> client, transport = make_transport_context(transport_context, DataRPC)
> try:
> transport.open()
> yield client
> finally:
> transport.close()
>
>
> Also asked at :
>
> http://stackoverflow.com/questions/35117943/proper-lifecycle-of-python-thrift-client-transport
> If you want those sweet SO points.
>
> --
>
>
> https://github.com/bearrito
> @deepbearrito
>