Which Java servers are safe to process high-latency requests on?

[Morgon Kanter <mo...@gmail.com>]

I'm writing an RPC service that could potentially have very high latency
(blocking, essentially). So I'm trying to figure out which if any of the
Java server choices are good for handling this sort of thing while still
scaling to a high QPS.

The best (and only) resource I found that enumerates all of the server
options for Java is
https://github.com/m1ch1/mapkeeper/wiki/Thrift-Java-Servers-Compared. From
this, it would seem like my options are THsHaServer,
TThreadedSelectorServer, and TThreadPoolServer. All of these share the same
idea of accept a connection and farm it out to a worker in a thread pool.
The downside seems to be that if there's a high QPS of big-latency
requests, the worker threads can get tied up.

Thus, I was wondering if there would be any interest in the development of
a new server type, TQuasarServer, which would utilize Quasar fibers as the
concurrency primitive for workers and therefore (at least theoretically[1])
not be limited via a worker thread pool. If I was to develop it, would such
a patch be likely to be accepted into Thrift, given that it introduces that
kind of optional external dependency?

Thanks,
-- Morgon

[1] It's not clear at what latency breakpoint you would get gains in
throughput; I'd obviously need to measure this.

Re: Which Java servers are safe to process high-latency requests on?

[Morgon Kanter <mo...@gmail.com>]

Hey Randy,

Thanks for the reply. Looks like someone else actually beat me to it
though: https://github.com/pinterest/quasar-thrift

Would it be possible for that to make its way into the official
distribution?

Cheers,
-- Morgon

On Tue, Nov 4, 2014 at 8:40 AM, Randy Abernethy <ra...@apache.org> wrote:

> Hi Morgan,
>
> I think a Quasar server would be a great addition for the community.
> Might be best committed as a contrib initially and then if interest is
> high it could be moved over to the Java lib.
>
> -Randy
>
> On Mon, Nov 3, 2014 at 10:17 PM, Morgon Kanter <mo...@gmail.com>
> wrote:
> > I'm writing an RPC service that could potentially have very high latency
> > (blocking, essentially). So I'm trying to figure out which if any of the
> > Java server choices are good for handling this sort of thing while still
> > scaling to a high QPS.
> >
> > The best (and only) resource I found that enumerates all of the server
> > options for Java is
> > https://github.com/m1ch1/mapkeeper/wiki/Thrift-Java-Servers-Compared.
> From
> > this, it would seem like my options are THsHaServer,
> > TThreadedSelectorServer, and TThreadPoolServer. All of these share the
> same
> > idea of accept a connection and farm it out to a worker in a thread pool.
> > The downside seems to be that if there's a high QPS of big-latency
> > requests, the worker threads can get tied up.
> >
> > Thus, I was wondering if there would be any interest in the development
> of
> > a new server type, TQuasarServer, which would utilize Quasar fibers as
> the
> > concurrency primitive for workers and therefore (at least
> theoretically[1])
> > not be limited via a worker thread pool. If I was to develop it, would
> such
> > a patch be likely to be accepted into Thrift, given that it introduces
> that
> > kind of optional external dependency?
> >
> > Thanks,
> > -- Morgon
> >
> > [1] It's not clear at what latency breakpoint you would get gains in
> > throughput; I'd obviously need to measure this.
>

Re: Which Java servers are safe to process high-latency requests on?

[Randy Abernethy <ra...@apache.org>]

Hi Morgan,

I think a Quasar server would be a great addition for the community.
Might be best committed as a contrib initially and then if interest is
high it could be moved over to the Java lib.

-Randy

On Mon, Nov 3, 2014 at 10:17 PM, Morgon Kanter <mo...@gmail.com> wrote:
> I'm writing an RPC service that could potentially have very high latency
> (blocking, essentially). So I'm trying to figure out which if any of the
> Java server choices are good for handling this sort of thing while still
> scaling to a high QPS.
>
> The best (and only) resource I found that enumerates all of the server
> options for Java is
> https://github.com/m1ch1/mapkeeper/wiki/Thrift-Java-Servers-Compared. From
> this, it would seem like my options are THsHaServer,
> TThreadedSelectorServer, and TThreadPoolServer. All of these share the same
> idea of accept a connection and farm it out to a worker in a thread pool.
> The downside seems to be that if there's a high QPS of big-latency
> requests, the worker threads can get tied up.
>
> Thus, I was wondering if there would be any interest in the development of
> a new server type, TQuasarServer, which would utilize Quasar fibers as the
> concurrency primitive for workers and therefore (at least theoretically[1])
> not be limited via a worker thread pool. If I was to develop it, would such
> a patch be likely to be accepted into Thrift, given that it introduces that
> kind of optional external dependency?
>
> Thanks,
> -- Morgon
>
> [1] It's not clear at what latency breakpoint you would get gains in
> throughput; I'd obviously need to measure this.