Thrift Server Implementations

[Christopher Wirt <ch...@struq.com>]

TL;DR:

Has anyone ever tried using the new thrift 0.9 TThreadSelectorServer for
their thrift server?

I did today and have found it performs pretty well.

Is this something people would like to see in the C* trunk?

 

 

 

Background:

Yesterday we upgraded from Cass 1.2.14 to Cass 2.0.5.

 

We have our apps all coded up so we can easily switch between thrift and
native very quickly. Historically we've always used the HsHa server as it
was the only server which provided us with the required level of performance
given the number of clients we have connecting and the strict 10ms 99th
%tile SLAs we have.

 

Prior to the upgrade we ran with native protocol for a day just to check
everything was still working. It did, but performed slightly worse than
thrift.

 

After the upgrade we found the new THsHaDisruptorServer wasn't working for
us under full load. "It worked fine in testing" I cried. Our CTO was not
amused. So I guess our load testing strategy isn't the best.

So we switched to the thrift sync server and found this performed terribly
above the 75%tile. 

Then we switched to the native and found the performance to be significantly
worse than in 1.2.14. (I'll leave questions and comments about those to a
different post. I checked all the code and very little has changed so not
sure what that's about)

 

Anyway, today I moved the old HsHa implementation and the new
TThreadSelectorServer into a 2.0.5 checkout, hooked them in, built, did a
bit of testing and I'm now running live.

 

We found the TThreadSelectorServer performed the best getting us back under
our SLA.

 

Thoughts and comments welcome

 

 

Chris

RE: Thrift Server Implementations

[Christopher Wirt <ch...@struq.com>]

Yes I am still running thrift 0.9 TThreadedSelectorServer without any serious problems.

I didn't realise before, but the old Cassandra CustomTHsHaServer implementation is very similar to TTSS. So this could easily be seen as a step backwards.

But given my current situation I'd love to see TTSS as an option.

The idea was mentioned in the comments here https://issues.apache.org/jira/browse/CASSANDRA-6285


-----Original Message-----
From: Chris Burroughs [mailto:chris.burroughs@gmail.com] 
Sent: 05 March 2014 17:02
To: user@cassandra.apache.org
Subject: Re: Thrift Server Implementations

On 02/13/2014 01:37 PM, Christopher Wirt wrote:
> Anyway, today I moved the old HsHa implementation and the new 
> TThreadSelectorServer into a 2.0.5 checkout, hooked them in, built, 
> did a bit of testing and I'm now running live.
>
>
>
> We found the TThreadSelectorServer performed the best getting us back 
> under our SLA.

Are you still running with the upstream TThreadSelectorServer?  Based on your experience is there any reason Cassandra should not adapt it.

Re: Thrift Server Implementations

[Chris Burroughs <ch...@gmail.com>]

On 02/13/2014 01:37 PM, Christopher Wirt wrote:
> Anyway, today I moved the old HsHa implementation and the new
> TThreadSelectorServer into a 2.0.5 checkout, hooked them in, built, did a
> bit of testing and I'm now running live.
>
>
>
> We found the TThreadSelectorServer performed the best getting us back under
> our SLA.

Are you still running with the upstream TThreadSelectorServer?  Based on 
your experience is there any reason Cassandra should not adapt it.

Re: Thrift Server Implementations

[Robert Coli <rc...@eventbrite.com>]

On Thu, Feb 13, 2014 at 10:37 AM, Christopher Wirt <ch...@struq.com>wrote:

> TL;DR:
>
> Has anyone ever tried using the new thrift 0.9 TThreadSelectorServer for
> their thrift server?
>
> I did today and have found it performs pretty well.
>
> Is this something people would like to see in the C* trunk?
>

I'd be interested in exploring improved Thrift server performance in trunk.

I suggest filing a JIRA, and letting us here in the thread know its URL!

=Rob