You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@thrift.apache.org by "Napolitano, Diane" <dn...@ets.org> on 2013/04/05 21:43:57 UTC
Question about Multithreaded Thrift Servers
Hello, I have a Thrift server (just "NameOfServer" here) written in Java which is initialized with the following inside of an inner class called ServerThread:
TThreadPoolServer.Args args = new TThreadPoolServer.Args(serverTransport);
args.maxWorkerThreads(10);
args.processor(processor);
TServer server = new TThreadPoolServer(args);
server.serve();
Then NameOfServer has two objects:
public static NameOfServerHandler handler;
public static NameOfServer.Processor processor;
And then a ServerThread object is created in my server's main:
handler = new NameOfServerHandler();
processor = new NameOfServer.Processor(handler);
Runnable r = new ServerThread(processor, portNum);
new Thread(r).start();
My question is: am I doing this right? Because when I send more than ten requests to the server, it throws an exception and dies, and all requests beyond the first 10 are stalled. Unless I'm misunderstanding threads, Java, and everything else, aren't these additional requests supposed to be queued and then served when one of the 10 threads becomes available?
If you need more code/context than this, definitely let me know.
Thanks,
Diane
RE: Question about Multithreaded Thrift Servers
Posted by "Napolitano, Diane" <dn...@ets.org>.
Oh, and I just found this, too: https://github.com/m1ch1/mapkeeper/wiki/Thrift-Java-Servers-Compared
Best,
Diane
________________________________________
From: Napolitano, Diane [dnapolitano@ets.org]
Sent: Monday, April 08, 2013 4:06 PM
To: user@thrift.apache.org
Subject: RE: Question about Multithreaded Thrift Servers
Wow, that's extremely useful. Thanks for sharing!
- Diane
________________________________________
From: Mike Stanley [mike@mikestanley.org]
Sent: Monday, April 08, 2013 2:42 PM
To: user@thrift.apache.org
Subject: Re: Question about Multithreaded Thrift Servers
HI Diane,
I found this article to be very helpful when trying to understand the
implementation and trade-offs of the various java server implementations --
http://mmalithh.blogspot.com/2012/10/comparison-of-apache-thrift-java-servers.html
hope it helps.
... Mike
On Mon, Apr 8, 2013 at 1:52 PM, Napolitano, Diane <dn...@ets.org>wrote:
> Your comment about the Java implementation of TThreadPoolServer got me
> thinking. Turns out this class will use pretty much any Java thread
> pool/executor configuration you throw at it, so TThreadPoolServer itself
> isn't a blocking queue, it just uses a blocking queue by default. Because
> of that, if I queue up 1000 jobs, 10 of them run and the other 990 fail and
> typically cause the server to crash.
>
> So, at this point in the code for my server:
> TThreadPoolServer.Args args = new TThreadPoolServer.Args(serverTransport);
> args.maxWorkerThreads(10);
> args.processor(processor);
>
> I added:
> args.executorService(new ScheduledThreadPoolExecutor(10));
>
> Then created the server object:
> TServer server = new TThreadPoolServer(args);
>
> Now Java is smart enough to queue (schedule) the other 990 jobs, so I no
> longer have this issue. :) (Also setting -Xmx2048m and
> -XX:-UseGCOverheadLimit helped too.)
>
> Thanks for your help!
>
> Best,
> Diane
> ________________________________________
> From: Ben Craig [ben.craig@ni.com]
> Sent: Friday, April 05, 2013 4:34 PM
> To: user@thrift.apache.org
> Subject: RE: Question about Multithreaded Thrift Servers
>
> Are you running out of ports / file descriptors? I have attempted to
> write performance tests in the past that stressed connect / disconnect,
> and I have routinely run into the problem of exhausting ports. This isn't
> because I failed to call close, but because "closing" a socket doesn't
> immediately free up the resources in the TCP/IP stack. It typically takes
> a few minutes for those resources to be freed.
>
>
>
> From: "Napolitano, Diane" <dn...@ets.org>
> To: "user@thrift.apache.org" <us...@thrift.apache.org>,
> Date: 04/05/2013 03:32 PM
> Subject: RE: Question about Multithreaded Thrift Servers
>
>
>
> Hmm, interesting. These requests are actually clients, several thousands
> of them actually :) , calling methods on the server. We have a cluster
> setup here, and I have one cluster job that is the server, and several
> thousand cluster jobs that are each one client, or one request to the
> server.
>
> So, hmm.
>
> Thanks,
> Diane
> ________________________________________
> From: Ben Craig [ben.craig@ni.com]
> Sent: Friday, April 05, 2013 4:20 PM
> To: user@thrift.apache.org
> Subject: RE: Question about Multithreaded Thrift Servers
>
> You only mention requests, so I'm not sure if these are all going over the
> same connection or not. One other thing that gets strange is that naming
> across the different languages isn't consistent. I am most comfortable
> with the C++ implementation, and so that's the behavior that I described.
> I vaguely recall that the Java implementation of TThreadPoolServer is more
> similar to C++'s NonBlockingServer, and I'm not sure what that does.
>
>
>
> From: "Napolitano, Diane" <dn...@ets.org>
> To: "user@thrift.apache.org" <us...@thrift.apache.org>,
> Date: 04/05/2013 03:02 PM
> Subject: RE: Question about Multithreaded Thrift Servers
>
>
>
> Really? So if threads 0 thru 9 are running, and 0-8 finish, the remaining
> requests in the queue (is there even a queue?) won't make use of 0-8?
>
> It's fine if they sit there waiting for 9 to finish, but in this case,
> even after 9 finishes, additional requests aren't being executed. If > 10
> requests are sent to it, the server throws an exception, and only executes
> the first 10. Also, sometimes it isn't the 10th request that is the
> slowest, it's some other one (the 2nd or whatever).
>
> Sorry, I'm confused because your answer makes it sound like what I'm
> seeing shouldn't be happening. :/
>
> Thanks,
> Diane
> ________________________________________
> From: Ben Craig [ben.craig@ni.com]
> Sent: Friday, April 05, 2013 3:55 PM
> To: user@thrift.apache.org
> Subject: Re: Question about Multithreaded Thrift Servers
>
> The threaded server uses one thread per connection. It does not get a
> request, and have an available thread process the request. This means
> that if you send 20 requests, and the 10th request takes a long time to
> process, the last ten requests will be stalled.
>
>
>
> From: "Napolitano, Diane" <dn...@ets.org>
> To: "user@thrift.apache.org" <us...@thrift.apache.org>,
> Date: 04/05/2013 02:44 PM
> Subject: Question about Multithreaded Thrift Servers
>
>
>
> Hello, I have a Thrift server (just "NameOfServer" here) written in Java
> which is initialized with the following inside of an inner class called
> ServerThread:
>
> TThreadPoolServer.Args args = new TThreadPoolServer.Args(serverTransport);
> args.maxWorkerThreads(10);
> args.processor(processor);
> TServer server = new TThreadPoolServer(args);
> server.serve();
>
> Then NameOfServer has two objects:
> public static NameOfServerHandler handler;
> public static NameOfServer.Processor processor;
>
> And then a ServerThread object is created in my server's main:
> handler = new NameOfServerHandler();
> processor = new NameOfServer.Processor(handler);
> Runnable r = new ServerThread(processor, portNum);
> new Thread(r).start();
>
> My question is: am I doing this right? Because when I send more than ten
> requests to the server, it throws an exception and dies, and all requests
> beyond the first 10 are stalled. Unless I'm misunderstanding threads,
> Java, and everything else, aren't these additional requests supposed to be
> queued and then served when one of the 10 threads becomes available?
>
> If you need more code/context than this, definitely let me know.
>
> Thanks,
> Diane
>
>
>
>
>
>
>
RE: Question about Multithreaded Thrift Servers
Posted by "Napolitano, Diane" <dn...@ets.org>.
Wow, that's extremely useful. Thanks for sharing!
- Diane
________________________________________
From: Mike Stanley [mike@mikestanley.org]
Sent: Monday, April 08, 2013 2:42 PM
To: user@thrift.apache.org
Subject: Re: Question about Multithreaded Thrift Servers
HI Diane,
I found this article to be very helpful when trying to understand the
implementation and trade-offs of the various java server implementations --
http://mmalithh.blogspot.com/2012/10/comparison-of-apache-thrift-java-servers.html
hope it helps.
... Mike
On Mon, Apr 8, 2013 at 1:52 PM, Napolitano, Diane <dn...@ets.org>wrote:
> Your comment about the Java implementation of TThreadPoolServer got me
> thinking. Turns out this class will use pretty much any Java thread
> pool/executor configuration you throw at it, so TThreadPoolServer itself
> isn't a blocking queue, it just uses a blocking queue by default. Because
> of that, if I queue up 1000 jobs, 10 of them run and the other 990 fail and
> typically cause the server to crash.
>
> So, at this point in the code for my server:
> TThreadPoolServer.Args args = new TThreadPoolServer.Args(serverTransport);
> args.maxWorkerThreads(10);
> args.processor(processor);
>
> I added:
> args.executorService(new ScheduledThreadPoolExecutor(10));
>
> Then created the server object:
> TServer server = new TThreadPoolServer(args);
>
> Now Java is smart enough to queue (schedule) the other 990 jobs, so I no
> longer have this issue. :) (Also setting -Xmx2048m and
> -XX:-UseGCOverheadLimit helped too.)
>
> Thanks for your help!
>
> Best,
> Diane
> ________________________________________
> From: Ben Craig [ben.craig@ni.com]
> Sent: Friday, April 05, 2013 4:34 PM
> To: user@thrift.apache.org
> Subject: RE: Question about Multithreaded Thrift Servers
>
> Are you running out of ports / file descriptors? I have attempted to
> write performance tests in the past that stressed connect / disconnect,
> and I have routinely run into the problem of exhausting ports. This isn't
> because I failed to call close, but because "closing" a socket doesn't
> immediately free up the resources in the TCP/IP stack. It typically takes
> a few minutes for those resources to be freed.
>
>
>
> From: "Napolitano, Diane" <dn...@ets.org>
> To: "user@thrift.apache.org" <us...@thrift.apache.org>,
> Date: 04/05/2013 03:32 PM
> Subject: RE: Question about Multithreaded Thrift Servers
>
>
>
> Hmm, interesting. These requests are actually clients, several thousands
> of them actually :) , calling methods on the server. We have a cluster
> setup here, and I have one cluster job that is the server, and several
> thousand cluster jobs that are each one client, or one request to the
> server.
>
> So, hmm.
>
> Thanks,
> Diane
> ________________________________________
> From: Ben Craig [ben.craig@ni.com]
> Sent: Friday, April 05, 2013 4:20 PM
> To: user@thrift.apache.org
> Subject: RE: Question about Multithreaded Thrift Servers
>
> You only mention requests, so I'm not sure if these are all going over the
> same connection or not. One other thing that gets strange is that naming
> across the different languages isn't consistent. I am most comfortable
> with the C++ implementation, and so that's the behavior that I described.
> I vaguely recall that the Java implementation of TThreadPoolServer is more
> similar to C++'s NonBlockingServer, and I'm not sure what that does.
>
>
>
> From: "Napolitano, Diane" <dn...@ets.org>
> To: "user@thrift.apache.org" <us...@thrift.apache.org>,
> Date: 04/05/2013 03:02 PM
> Subject: RE: Question about Multithreaded Thrift Servers
>
>
>
> Really? So if threads 0 thru 9 are running, and 0-8 finish, the remaining
> requests in the queue (is there even a queue?) won't make use of 0-8?
>
> It's fine if they sit there waiting for 9 to finish, but in this case,
> even after 9 finishes, additional requests aren't being executed. If > 10
> requests are sent to it, the server throws an exception, and only executes
> the first 10. Also, sometimes it isn't the 10th request that is the
> slowest, it's some other one (the 2nd or whatever).
>
> Sorry, I'm confused because your answer makes it sound like what I'm
> seeing shouldn't be happening. :/
>
> Thanks,
> Diane
> ________________________________________
> From: Ben Craig [ben.craig@ni.com]
> Sent: Friday, April 05, 2013 3:55 PM
> To: user@thrift.apache.org
> Subject: Re: Question about Multithreaded Thrift Servers
>
> The threaded server uses one thread per connection. It does not get a
> request, and have an available thread process the request. This means
> that if you send 20 requests, and the 10th request takes a long time to
> process, the last ten requests will be stalled.
>
>
>
> From: "Napolitano, Diane" <dn...@ets.org>
> To: "user@thrift.apache.org" <us...@thrift.apache.org>,
> Date: 04/05/2013 02:44 PM
> Subject: Question about Multithreaded Thrift Servers
>
>
>
> Hello, I have a Thrift server (just "NameOfServer" here) written in Java
> which is initialized with the following inside of an inner class called
> ServerThread:
>
> TThreadPoolServer.Args args = new TThreadPoolServer.Args(serverTransport);
> args.maxWorkerThreads(10);
> args.processor(processor);
> TServer server = new TThreadPoolServer(args);
> server.serve();
>
> Then NameOfServer has two objects:
> public static NameOfServerHandler handler;
> public static NameOfServer.Processor processor;
>
> And then a ServerThread object is created in my server's main:
> handler = new NameOfServerHandler();
> processor = new NameOfServer.Processor(handler);
> Runnable r = new ServerThread(processor, portNum);
> new Thread(r).start();
>
> My question is: am I doing this right? Because when I send more than ten
> requests to the server, it throws an exception and dies, and all requests
> beyond the first 10 are stalled. Unless I'm misunderstanding threads,
> Java, and everything else, aren't these additional requests supposed to be
> queued and then served when one of the 10 threads becomes available?
>
> If you need more code/context than this, definitely let me know.
>
> Thanks,
> Diane
>
>
>
>
>
>
>
Re: Question about Multithreaded Thrift Servers
Posted by Mike Stanley <mi...@mikestanley.org>.
HI Diane,
I found this article to be very helpful when trying to understand the
implementation and trade-offs of the various java server implementations --
http://mmalithh.blogspot.com/2012/10/comparison-of-apache-thrift-java-servers.html
hope it helps.
... Mike
On Mon, Apr 8, 2013 at 1:52 PM, Napolitano, Diane <dn...@ets.org>wrote:
> Your comment about the Java implementation of TThreadPoolServer got me
> thinking. Turns out this class will use pretty much any Java thread
> pool/executor configuration you throw at it, so TThreadPoolServer itself
> isn't a blocking queue, it just uses a blocking queue by default. Because
> of that, if I queue up 1000 jobs, 10 of them run and the other 990 fail and
> typically cause the server to crash.
>
> So, at this point in the code for my server:
> TThreadPoolServer.Args args = new TThreadPoolServer.Args(serverTransport);
> args.maxWorkerThreads(10);
> args.processor(processor);
>
> I added:
> args.executorService(new ScheduledThreadPoolExecutor(10));
>
> Then created the server object:
> TServer server = new TThreadPoolServer(args);
>
> Now Java is smart enough to queue (schedule) the other 990 jobs, so I no
> longer have this issue. :) (Also setting -Xmx2048m and
> -XX:-UseGCOverheadLimit helped too.)
>
> Thanks for your help!
>
> Best,
> Diane
> ________________________________________
> From: Ben Craig [ben.craig@ni.com]
> Sent: Friday, April 05, 2013 4:34 PM
> To: user@thrift.apache.org
> Subject: RE: Question about Multithreaded Thrift Servers
>
> Are you running out of ports / file descriptors? I have attempted to
> write performance tests in the past that stressed connect / disconnect,
> and I have routinely run into the problem of exhausting ports. This isn't
> because I failed to call close, but because "closing" a socket doesn't
> immediately free up the resources in the TCP/IP stack. It typically takes
> a few minutes for those resources to be freed.
>
>
>
> From: "Napolitano, Diane" <dn...@ets.org>
> To: "user@thrift.apache.org" <us...@thrift.apache.org>,
> Date: 04/05/2013 03:32 PM
> Subject: RE: Question about Multithreaded Thrift Servers
>
>
>
> Hmm, interesting. These requests are actually clients, several thousands
> of them actually :) , calling methods on the server. We have a cluster
> setup here, and I have one cluster job that is the server, and several
> thousand cluster jobs that are each one client, or one request to the
> server.
>
> So, hmm.
>
> Thanks,
> Diane
> ________________________________________
> From: Ben Craig [ben.craig@ni.com]
> Sent: Friday, April 05, 2013 4:20 PM
> To: user@thrift.apache.org
> Subject: RE: Question about Multithreaded Thrift Servers
>
> You only mention requests, so I'm not sure if these are all going over the
> same connection or not. One other thing that gets strange is that naming
> across the different languages isn't consistent. I am most comfortable
> with the C++ implementation, and so that's the behavior that I described.
> I vaguely recall that the Java implementation of TThreadPoolServer is more
> similar to C++'s NonBlockingServer, and I'm not sure what that does.
>
>
>
> From: "Napolitano, Diane" <dn...@ets.org>
> To: "user@thrift.apache.org" <us...@thrift.apache.org>,
> Date: 04/05/2013 03:02 PM
> Subject: RE: Question about Multithreaded Thrift Servers
>
>
>
> Really? So if threads 0 thru 9 are running, and 0-8 finish, the remaining
> requests in the queue (is there even a queue?) won't make use of 0-8?
>
> It's fine if they sit there waiting for 9 to finish, but in this case,
> even after 9 finishes, additional requests aren't being executed. If > 10
> requests are sent to it, the server throws an exception, and only executes
> the first 10. Also, sometimes it isn't the 10th request that is the
> slowest, it's some other one (the 2nd or whatever).
>
> Sorry, I'm confused because your answer makes it sound like what I'm
> seeing shouldn't be happening. :/
>
> Thanks,
> Diane
> ________________________________________
> From: Ben Craig [ben.craig@ni.com]
> Sent: Friday, April 05, 2013 3:55 PM
> To: user@thrift.apache.org
> Subject: Re: Question about Multithreaded Thrift Servers
>
> The threaded server uses one thread per connection. It does not get a
> request, and have an available thread process the request. This means
> that if you send 20 requests, and the 10th request takes a long time to
> process, the last ten requests will be stalled.
>
>
>
> From: "Napolitano, Diane" <dn...@ets.org>
> To: "user@thrift.apache.org" <us...@thrift.apache.org>,
> Date: 04/05/2013 02:44 PM
> Subject: Question about Multithreaded Thrift Servers
>
>
>
> Hello, I have a Thrift server (just "NameOfServer" here) written in Java
> which is initialized with the following inside of an inner class called
> ServerThread:
>
> TThreadPoolServer.Args args = new TThreadPoolServer.Args(serverTransport);
> args.maxWorkerThreads(10);
> args.processor(processor);
> TServer server = new TThreadPoolServer(args);
> server.serve();
>
> Then NameOfServer has two objects:
> public static NameOfServerHandler handler;
> public static NameOfServer.Processor processor;
>
> And then a ServerThread object is created in my server's main:
> handler = new NameOfServerHandler();
> processor = new NameOfServer.Processor(handler);
> Runnable r = new ServerThread(processor, portNum);
> new Thread(r).start();
>
> My question is: am I doing this right? Because when I send more than ten
> requests to the server, it throws an exception and dies, and all requests
> beyond the first 10 are stalled. Unless I'm misunderstanding threads,
> Java, and everything else, aren't these additional requests supposed to be
> queued and then served when one of the 10 threads becomes available?
>
> If you need more code/context than this, definitely let me know.
>
> Thanks,
> Diane
>
>
>
>
>
>
>
RE: Question about Multithreaded Thrift Servers
Posted by "Napolitano, Diane" <dn...@ets.org>.
Your comment about the Java implementation of TThreadPoolServer got me thinking. Turns out this class will use pretty much any Java thread pool/executor configuration you throw at it, so TThreadPoolServer itself isn't a blocking queue, it just uses a blocking queue by default. Because of that, if I queue up 1000 jobs, 10 of them run and the other 990 fail and typically cause the server to crash.
So, at this point in the code for my server:
TThreadPoolServer.Args args = new TThreadPoolServer.Args(serverTransport);
args.maxWorkerThreads(10);
args.processor(processor);
I added:
args.executorService(new ScheduledThreadPoolExecutor(10));
Then created the server object:
TServer server = new TThreadPoolServer(args);
Now Java is smart enough to queue (schedule) the other 990 jobs, so I no longer have this issue. :) (Also setting -Xmx2048m and -XX:-UseGCOverheadLimit helped too.)
Thanks for your help!
Best,
Diane
________________________________________
From: Ben Craig [ben.craig@ni.com]
Sent: Friday, April 05, 2013 4:34 PM
To: user@thrift.apache.org
Subject: RE: Question about Multithreaded Thrift Servers
Are you running out of ports / file descriptors? I have attempted to
write performance tests in the past that stressed connect / disconnect,
and I have routinely run into the problem of exhausting ports. This isn't
because I failed to call close, but because "closing" a socket doesn't
immediately free up the resources in the TCP/IP stack. It typically takes
a few minutes for those resources to be freed.
From: "Napolitano, Diane" <dn...@ets.org>
To: "user@thrift.apache.org" <us...@thrift.apache.org>,
Date: 04/05/2013 03:32 PM
Subject: RE: Question about Multithreaded Thrift Servers
Hmm, interesting. These requests are actually clients, several thousands
of them actually :) , calling methods on the server. We have a cluster
setup here, and I have one cluster job that is the server, and several
thousand cluster jobs that are each one client, or one request to the
server.
So, hmm.
Thanks,
Diane
________________________________________
From: Ben Craig [ben.craig@ni.com]
Sent: Friday, April 05, 2013 4:20 PM
To: user@thrift.apache.org
Subject: RE: Question about Multithreaded Thrift Servers
You only mention requests, so I'm not sure if these are all going over the
same connection or not. One other thing that gets strange is that naming
across the different languages isn't consistent. I am most comfortable
with the C++ implementation, and so that's the behavior that I described.
I vaguely recall that the Java implementation of TThreadPoolServer is more
similar to C++'s NonBlockingServer, and I'm not sure what that does.
From: "Napolitano, Diane" <dn...@ets.org>
To: "user@thrift.apache.org" <us...@thrift.apache.org>,
Date: 04/05/2013 03:02 PM
Subject: RE: Question about Multithreaded Thrift Servers
Really? So if threads 0 thru 9 are running, and 0-8 finish, the remaining
requests in the queue (is there even a queue?) won't make use of 0-8?
It's fine if they sit there waiting for 9 to finish, but in this case,
even after 9 finishes, additional requests aren't being executed. If > 10
requests are sent to it, the server throws an exception, and only executes
the first 10. Also, sometimes it isn't the 10th request that is the
slowest, it's some other one (the 2nd or whatever).
Sorry, I'm confused because your answer makes it sound like what I'm
seeing shouldn't be happening. :/
Thanks,
Diane
________________________________________
From: Ben Craig [ben.craig@ni.com]
Sent: Friday, April 05, 2013 3:55 PM
To: user@thrift.apache.org
Subject: Re: Question about Multithreaded Thrift Servers
The threaded server uses one thread per connection. It does not get a
request, and have an available thread process the request. This means
that if you send 20 requests, and the 10th request takes a long time to
process, the last ten requests will be stalled.
From: "Napolitano, Diane" <dn...@ets.org>
To: "user@thrift.apache.org" <us...@thrift.apache.org>,
Date: 04/05/2013 02:44 PM
Subject: Question about Multithreaded Thrift Servers
Hello, I have a Thrift server (just "NameOfServer" here) written in Java
which is initialized with the following inside of an inner class called
ServerThread:
TThreadPoolServer.Args args = new TThreadPoolServer.Args(serverTransport);
args.maxWorkerThreads(10);
args.processor(processor);
TServer server = new TThreadPoolServer(args);
server.serve();
Then NameOfServer has two objects:
public static NameOfServerHandler handler;
public static NameOfServer.Processor processor;
And then a ServerThread object is created in my server's main:
handler = new NameOfServerHandler();
processor = new NameOfServer.Processor(handler);
Runnable r = new ServerThread(processor, portNum);
new Thread(r).start();
My question is: am I doing this right? Because when I send more than ten
requests to the server, it throws an exception and dies, and all requests
beyond the first 10 are stalled. Unless I'm misunderstanding threads,
Java, and everything else, aren't these additional requests supposed to be
queued and then served when one of the 10 threads becomes available?
If you need more code/context than this, definitely let me know.
Thanks,
Diane
RE: Question about Multithreaded Thrift Servers
Posted by Ben Craig <be...@ni.com>.
Are you running out of ports / file descriptors? I have attempted to
write performance tests in the past that stressed connect / disconnect,
and I have routinely run into the problem of exhausting ports. This isn't
because I failed to call close, but because "closing" a socket doesn't
immediately free up the resources in the TCP/IP stack. It typically takes
a few minutes for those resources to be freed.
From: "Napolitano, Diane" <dn...@ets.org>
To: "user@thrift.apache.org" <us...@thrift.apache.org>,
Date: 04/05/2013 03:32 PM
Subject: RE: Question about Multithreaded Thrift Servers
Hmm, interesting. These requests are actually clients, several thousands
of them actually :) , calling methods on the server. We have a cluster
setup here, and I have one cluster job that is the server, and several
thousand cluster jobs that are each one client, or one request to the
server.
So, hmm.
Thanks,
Diane
________________________________________
From: Ben Craig [ben.craig@ni.com]
Sent: Friday, April 05, 2013 4:20 PM
To: user@thrift.apache.org
Subject: RE: Question about Multithreaded Thrift Servers
You only mention requests, so I'm not sure if these are all going over the
same connection or not. One other thing that gets strange is that naming
across the different languages isn't consistent. I am most comfortable
with the C++ implementation, and so that's the behavior that I described.
I vaguely recall that the Java implementation of TThreadPoolServer is more
similar to C++'s NonBlockingServer, and I'm not sure what that does.
From: "Napolitano, Diane" <dn...@ets.org>
To: "user@thrift.apache.org" <us...@thrift.apache.org>,
Date: 04/05/2013 03:02 PM
Subject: RE: Question about Multithreaded Thrift Servers
Really? So if threads 0 thru 9 are running, and 0-8 finish, the remaining
requests in the queue (is there even a queue?) won't make use of 0-8?
It's fine if they sit there waiting for 9 to finish, but in this case,
even after 9 finishes, additional requests aren't being executed. If > 10
requests are sent to it, the server throws an exception, and only executes
the first 10. Also, sometimes it isn't the 10th request that is the
slowest, it's some other one (the 2nd or whatever).
Sorry, I'm confused because your answer makes it sound like what I'm
seeing shouldn't be happening. :/
Thanks,
Diane
________________________________________
From: Ben Craig [ben.craig@ni.com]
Sent: Friday, April 05, 2013 3:55 PM
To: user@thrift.apache.org
Subject: Re: Question about Multithreaded Thrift Servers
The threaded server uses one thread per connection. It does not get a
request, and have an available thread process the request. This means
that if you send 20 requests, and the 10th request takes a long time to
process, the last ten requests will be stalled.
From: "Napolitano, Diane" <dn...@ets.org>
To: "user@thrift.apache.org" <us...@thrift.apache.org>,
Date: 04/05/2013 02:44 PM
Subject: Question about Multithreaded Thrift Servers
Hello, I have a Thrift server (just "NameOfServer" here) written in Java
which is initialized with the following inside of an inner class called
ServerThread:
TThreadPoolServer.Args args = new TThreadPoolServer.Args(serverTransport);
args.maxWorkerThreads(10);
args.processor(processor);
TServer server = new TThreadPoolServer(args);
server.serve();
Then NameOfServer has two objects:
public static NameOfServerHandler handler;
public static NameOfServer.Processor processor;
And then a ServerThread object is created in my server's main:
handler = new NameOfServerHandler();
processor = new NameOfServer.Processor(handler);
Runnable r = new ServerThread(processor, portNum);
new Thread(r).start();
My question is: am I doing this right? Because when I send more than ten
requests to the server, it throws an exception and dies, and all requests
beyond the first 10 are stalled. Unless I'm misunderstanding threads,
Java, and everything else, aren't these additional requests supposed to be
queued and then served when one of the 10 threads becomes available?
If you need more code/context than this, definitely let me know.
Thanks,
Diane
RE: Question about Multithreaded Thrift Servers
Posted by "Napolitano, Diane" <dn...@ets.org>.
Hmm, interesting. These requests are actually clients, several thousands of them actually :) , calling methods on the server. We have a cluster setup here, and I have one cluster job that is the server, and several thousand cluster jobs that are each one client, or one request to the server.
So, hmm.
Thanks,
Diane
________________________________________
From: Ben Craig [ben.craig@ni.com]
Sent: Friday, April 05, 2013 4:20 PM
To: user@thrift.apache.org
Subject: RE: Question about Multithreaded Thrift Servers
You only mention requests, so I'm not sure if these are all going over the
same connection or not. One other thing that gets strange is that naming
across the different languages isn't consistent. I am most comfortable
with the C++ implementation, and so that's the behavior that I described.
I vaguely recall that the Java implementation of TThreadPoolServer is more
similar to C++'s NonBlockingServer, and I'm not sure what that does.
From: "Napolitano, Diane" <dn...@ets.org>
To: "user@thrift.apache.org" <us...@thrift.apache.org>,
Date: 04/05/2013 03:02 PM
Subject: RE: Question about Multithreaded Thrift Servers
Really? So if threads 0 thru 9 are running, and 0-8 finish, the remaining
requests in the queue (is there even a queue?) won't make use of 0-8?
It's fine if they sit there waiting for 9 to finish, but in this case,
even after 9 finishes, additional requests aren't being executed. If > 10
requests are sent to it, the server throws an exception, and only executes
the first 10. Also, sometimes it isn't the 10th request that is the
slowest, it's some other one (the 2nd or whatever).
Sorry, I'm confused because your answer makes it sound like what I'm
seeing shouldn't be happening. :/
Thanks,
Diane
________________________________________
From: Ben Craig [ben.craig@ni.com]
Sent: Friday, April 05, 2013 3:55 PM
To: user@thrift.apache.org
Subject: Re: Question about Multithreaded Thrift Servers
The threaded server uses one thread per connection. It does not get a
request, and have an available thread process the request. This means
that if you send 20 requests, and the 10th request takes a long time to
process, the last ten requests will be stalled.
From: "Napolitano, Diane" <dn...@ets.org>
To: "user@thrift.apache.org" <us...@thrift.apache.org>,
Date: 04/05/2013 02:44 PM
Subject: Question about Multithreaded Thrift Servers
Hello, I have a Thrift server (just "NameOfServer" here) written in Java
which is initialized with the following inside of an inner class called
ServerThread:
TThreadPoolServer.Args args = new TThreadPoolServer.Args(serverTransport);
args.maxWorkerThreads(10);
args.processor(processor);
TServer server = new TThreadPoolServer(args);
server.serve();
Then NameOfServer has two objects:
public static NameOfServerHandler handler;
public static NameOfServer.Processor processor;
And then a ServerThread object is created in my server's main:
handler = new NameOfServerHandler();
processor = new NameOfServer.Processor(handler);
Runnable r = new ServerThread(processor, portNum);
new Thread(r).start();
My question is: am I doing this right? Because when I send more than ten
requests to the server, it throws an exception and dies, and all requests
beyond the first 10 are stalled. Unless I'm misunderstanding threads,
Java, and everything else, aren't these additional requests supposed to be
queued and then served when one of the 10 threads becomes available?
If you need more code/context than this, definitely let me know.
Thanks,
Diane
RE: Question about Multithreaded Thrift Servers
Posted by Ben Craig <be...@ni.com>.
You only mention requests, so I'm not sure if these are all going over the
same connection or not. One other thing that gets strange is that naming
across the different languages isn't consistent. I am most comfortable
with the C++ implementation, and so that's the behavior that I described.
I vaguely recall that the Java implementation of TThreadPoolServer is more
similar to C++'s NonBlockingServer, and I'm not sure what that does.
From: "Napolitano, Diane" <dn...@ets.org>
To: "user@thrift.apache.org" <us...@thrift.apache.org>,
Date: 04/05/2013 03:02 PM
Subject: RE: Question about Multithreaded Thrift Servers
Really? So if threads 0 thru 9 are running, and 0-8 finish, the remaining
requests in the queue (is there even a queue?) won't make use of 0-8?
It's fine if they sit there waiting for 9 to finish, but in this case,
even after 9 finishes, additional requests aren't being executed. If > 10
requests are sent to it, the server throws an exception, and only executes
the first 10. Also, sometimes it isn't the 10th request that is the
slowest, it's some other one (the 2nd or whatever).
Sorry, I'm confused because your answer makes it sound like what I'm
seeing shouldn't be happening. :/
Thanks,
Diane
________________________________________
From: Ben Craig [ben.craig@ni.com]
Sent: Friday, April 05, 2013 3:55 PM
To: user@thrift.apache.org
Subject: Re: Question about Multithreaded Thrift Servers
The threaded server uses one thread per connection. It does not get a
request, and have an available thread process the request. This means
that if you send 20 requests, and the 10th request takes a long time to
process, the last ten requests will be stalled.
From: "Napolitano, Diane" <dn...@ets.org>
To: "user@thrift.apache.org" <us...@thrift.apache.org>,
Date: 04/05/2013 02:44 PM
Subject: Question about Multithreaded Thrift Servers
Hello, I have a Thrift server (just "NameOfServer" here) written in Java
which is initialized with the following inside of an inner class called
ServerThread:
TThreadPoolServer.Args args = new TThreadPoolServer.Args(serverTransport);
args.maxWorkerThreads(10);
args.processor(processor);
TServer server = new TThreadPoolServer(args);
server.serve();
Then NameOfServer has two objects:
public static NameOfServerHandler handler;
public static NameOfServer.Processor processor;
And then a ServerThread object is created in my server's main:
handler = new NameOfServerHandler();
processor = new NameOfServer.Processor(handler);
Runnable r = new ServerThread(processor, portNum);
new Thread(r).start();
My question is: am I doing this right? Because when I send more than ten
requests to the server, it throws an exception and dies, and all requests
beyond the first 10 are stalled. Unless I'm misunderstanding threads,
Java, and everything else, aren't these additional requests supposed to be
queued and then served when one of the 10 threads becomes available?
If you need more code/context than this, definitely let me know.
Thanks,
Diane
RE: Question about Multithreaded Thrift Servers
Posted by "Napolitano, Diane" <dn...@ets.org>.
Really? So if threads 0 thru 9 are running, and 0-8 finish, the remaining requests in the queue (is there even a queue?) won't make use of 0-8?
It's fine if they sit there waiting for 9 to finish, but in this case, even after 9 finishes, additional requests aren't being executed. If > 10 requests are sent to it, the server throws an exception, and only executes the first 10. Also, sometimes it isn't the 10th request that is the slowest, it's some other one (the 2nd or whatever).
Sorry, I'm confused because your answer makes it sound like what I'm seeing shouldn't be happening. :/
Thanks,
Diane
________________________________________
From: Ben Craig [ben.craig@ni.com]
Sent: Friday, April 05, 2013 3:55 PM
To: user@thrift.apache.org
Subject: Re: Question about Multithreaded Thrift Servers
The threaded server uses one thread per connection. It does not get a
request, and have an available thread process the request. This means
that if you send 20 requests, and the 10th request takes a long time to
process, the last ten requests will be stalled.
From: "Napolitano, Diane" <dn...@ets.org>
To: "user@thrift.apache.org" <us...@thrift.apache.org>,
Date: 04/05/2013 02:44 PM
Subject: Question about Multithreaded Thrift Servers
Hello, I have a Thrift server (just "NameOfServer" here) written in Java
which is initialized with the following inside of an inner class called
ServerThread:
TThreadPoolServer.Args args = new TThreadPoolServer.Args(serverTransport);
args.maxWorkerThreads(10);
args.processor(processor);
TServer server = new TThreadPoolServer(args);
server.serve();
Then NameOfServer has two objects:
public static NameOfServerHandler handler;
public static NameOfServer.Processor processor;
And then a ServerThread object is created in my server's main:
handler = new NameOfServerHandler();
processor = new NameOfServer.Processor(handler);
Runnable r = new ServerThread(processor, portNum);
new Thread(r).start();
My question is: am I doing this right? Because when I send more than ten
requests to the server, it throws an exception and dies, and all requests
beyond the first 10 are stalled. Unless I'm misunderstanding threads,
Java, and everything else, aren't these additional requests supposed to be
queued and then served when one of the 10 threads becomes available?
If you need more code/context than this, definitely let me know.
Thanks,
Diane
Re: Question about Multithreaded Thrift Servers
Posted by Ben Craig <be...@ni.com>.
The threaded server uses one thread per connection. It does not get a
request, and have an available thread process the request. This means
that if you send 20 requests, and the 10th request takes a long time to
process, the last ten requests will be stalled.
From: "Napolitano, Diane" <dn...@ets.org>
To: "user@thrift.apache.org" <us...@thrift.apache.org>,
Date: 04/05/2013 02:44 PM
Subject: Question about Multithreaded Thrift Servers
Hello, I have a Thrift server (just "NameOfServer" here) written in Java
which is initialized with the following inside of an inner class called
ServerThread:
TThreadPoolServer.Args args = new TThreadPoolServer.Args(serverTransport);
args.maxWorkerThreads(10);
args.processor(processor);
TServer server = new TThreadPoolServer(args);
server.serve();
Then NameOfServer has two objects:
public static NameOfServerHandler handler;
public static NameOfServer.Processor processor;
And then a ServerThread object is created in my server's main:
handler = new NameOfServerHandler();
processor = new NameOfServer.Processor(handler);
Runnable r = new ServerThread(processor, portNum);
new Thread(r).start();
My question is: am I doing this right? Because when I send more than ten
requests to the server, it throws an exception and dies, and all requests
beyond the first 10 are stalled. Unless I'm misunderstanding threads,
Java, and everything else, aren't these additional requests supposed to be
queued and then served when one of the 10 threads becomes available?
If you need more code/context than this, definitely let me know.
Thanks,
Diane