You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@ws.apache.org by Filippo Capocasale <ke...@gsmbox.it> on 2004/03/30 15:37:08 UTC
XML-RPC server suddently stops responding
Hello everybody!
I am facing a severe problem and I really hope someone may help me.
I have a java application running on Solaris.
This application starts an xml-rpc servers, and some handler is registered.
I am using the version 1.2-b1 of Apache xml-rpc.
I have a huge number of clients connecting to this xml-rpc server.
Everything works correctly for several days (i.e. thowsands of connections); then, suddently, the server stops responding.
I don't see any error in the logs of my application, the process is still up and running (the same application has got another xml-rpc server listening on another port and it works perfectly), the server seems to be in listening on the port (I see it by the command netstat -a).
On the client side I see the error: "Unexpected end of file from server".
It looks like the server truncates every incoming connection.
There is no problem with firewalls: I've also tried to run a client on the same machine.
If I restart my application everything restarts working fine.
Well, this is the situation, and I really don't know what to do...
I just hope it is not an overload problem...
I would really apreciate every help or suggestion!!!
Thankyou very much,
Filippo
-----------------------------------------------------------------
Questo messaggio � stato inviato utilizzando http://it.my.gsmbox.com
Re: XML-RPC server suddently stops responding
Posted by John Wilson <tu...@wilson.co.uk>.
On 31 Mar 2004, at 09:43, Filippo Capocasale wrote:
> Hi Daniel!
> Thankyou very much for your help.
> I agree with you that it could be a resource issue, and I'm trying to
> investigate about this.
> I've tryed to replicate the problem in our developement environment,
> but I succeded only partially. Let me explain.
> I've prepared a java test program (even if in the real world the
> client for my server are written in C++...) that starts 100 threads
> and each thread sends 50 requests (of a single service, while the real
> clients invoke several services).
> I've launched this program on two PC in the same time.
> I succeeded to overload the xmlrpc server and, surprise surprise, the
> error on the client side is "Unexpected end of file from server", that
> is just the error I've in the production environment.
> But in my simulation the server doesn't stop completely from working:
> it send the error from some call, then is manage to fullfill some
> requests, then again errors and so on...
> Instead, in the case of the real problem in the production enviroment,
> the server seems to be definitively unable to respond to any request.
>
It could be that you are seeing an exhaustion of the number of sockets
available. If the clients connect, do a transaction and then disconnect
then you will be recycling a lot of sockets. A socket is not
immediately reusable, it has to sit in the TIME_WAIT state for a period
before it becomes free for reuse.
Check the setting for the duration of TIME_WAIT, it should be about 60
seconds.
Increase the number of sockets available.
Check that your clients are closing the sockets properly after use.
That's about all I can suggest at the moment.
John Wilson
The Wilson Partnership
http://www.wilson.co.uk
Re: XML-RPC server suddently stops responding
Posted by Filippo Capocasale <ke...@gsmbox.it>.
Hi Daniel!
Thankyou very much for your help.
I agree with you that it could be a resource issue, and I'm trying to investigate about this.
I've tryed to replicate the problem in our developement environment, but I succeded only partially. Let me explain.
I've prepared a java test program (even if in the real world the client for my server are written in C++...) that starts 100 threads and each thread sends 50 requests (of a single service, while the real clients invoke several services).
I've launched this program on two PC in the same time.
I succeeded to overload the xmlrpc server and, surprise surprise, the error on the client side is "Unexpected end of file from server", that is just the error I've in the production environment.
But in my simulation the server doesn't stop completely from working: it send the error from some call, then is manage to fullfill some requests, then again errors and so on...
Instead, in the case of the real problem in the production enviroment, the server seems to be definitively unable to respond to any request.
Of course for me it's very important to replicate the problem in the developement environment, since I can't perform my debugging tests in the production environment...
Thankyou for your attention once again.
Bye bye,
Filippo
Scrive Daniel <da...@yorku.ca>:
>
> Hi.. Could this be a resource issue (i.e. not enough sockets,
> file
> descriptors, etc.)?
>
> Seeing how this is really important to you, I'd put in logging
> statements
> to see where things die... You might even want to add extra
> logging to
> the WebServer.java file... Compiling the apache xmlrpc lib is
> quite
> straight forward.
>
> Regards,
> Daniel
>
> On Tue, 30 Mar 2004, Filippo Capocasale wrote:
>
> > Hi Dave,
> > unluckily I cannot restart the server, because we have to
> guarrantee 24H availability.
> > Anyway I thank you for your suggest!
> > Bye!
> > Filippo
> >
> >
> > Scrive Dave Stewart <ds...@aquaflo.com>:
> >
> > > Hi Filippo!
> > >
> > > I don't have any answers for you as to why this is
> happening,
> > > but I've
> > > been (unnecessarily?) fearful of something similar. In my
> > > case, I'm
> > > able to have cron restart the rpc server daily (at 1:00
> AM).
> > >
> > > If that is a possibility for you as well, we could get you
> > > going again.
> > > Especially if no one else has any ideas.
> > > ;-)
> > >
> > >
> > > On Mar 30, 2004, at 5:37 AM, Filippo Capocasale wrote:
> > >
> > > > Hello everybody!
> > > > I am facing a severe problem and I really hope someone
> may
> > > help me.
> > > > I have a java application running on Solaris.
> > > > This application starts an xml-rpc servers, and some
> handler
> > > is
> > > > registered.
> > > > I am using the version 1.2-b1 of Apache xml-rpc.
> > > > I have a huge number of clients connecting to this
> xml-rpc
> > > server.
> > > > Everything works correctly for several days (i.e.
> thowsands
> > > of
> > > > connections); then, suddently, the server stops
> responding.
> > > > I don't see any error in the logs of my application, the
> > > process is
> > > > still up and running (the same application has got
> another
> > > xml-rpc
> > > > server listening on another port and it works
> perfectly),
> > > the server
> > > > seems to be in listening on the port (I see it by the
> > > command netstat
> > > > -a).
> > > > On the client side I see the error: "Unexpected end of
> file
> > > from
> > > > server".
> > > > It looks like the server truncates every incoming
> > > connection.
> > > > There is no problem with firewalls: I've also tried to
> run a
> > > client on
> > > > the same machine.
> > > > If I restart my application everything restarts working
> > > fine.
> > > > Well, this is the situation, and I really don't know
> what to
> > > do...
> > > > I just hope it is not an overload problem...
> > > >
> > > > I would really apreciate every help or suggestion!!!
> > > > Thankyou very much,
> > > > Filippo
> > > >
> > > >
> > >
> -----------------------------------------------------------------
> > > > Questo messaggio � stato inviato utilizzando
> > > http://it.my.gsmbox.com
> > > >
> > >
> > >
> > > Dave Stewart
> > > Aqua~Flo Supply (Goleta CA)
> > > dstewart at aquaflo dot com
> > >
> > > The trouble with doing something right the first time
> > > is that nobody appreciates how difficult it was.
> > >
> > >
> >
> >
> >
> -----------------------------------------------------------------
> > Questo messaggio � stato inviato utilizzando
> http://it.my.gsmbox.com
> >
>
-----------------------------------------------------------------
Questo messaggio � stato inviato utilizzando http://it.my.gsmbox.com
Re: XML-RPC server suddently stops responding
Posted by Daniel <da...@yorku.ca>.
Hi.. Could this be a resource issue (i.e. not enough sockets, file
descriptors, etc.)?
Seeing how this is really important to you, I'd put in logging statements
to see where things die... You might even want to add extra logging to
the WebServer.java file... Compiling the apache xmlrpc lib is quite
straight forward.
Regards,
Daniel
On Tue, 30 Mar 2004, Filippo Capocasale wrote:
> Hi Dave,
> unluckily I cannot restart the server, because we have to guarrantee 24H availability.
> Anyway I thank you for your suggest!
> Bye!
> Filippo
>
>
> Scrive Dave Stewart <ds...@aquaflo.com>:
>
> > Hi Filippo!
> >
> > I don't have any answers for you as to why this is happening,
> > but I've
> > been (unnecessarily?) fearful of something similar. In my
> > case, I'm
> > able to have cron restart the rpc server daily (at 1:00 AM).
> >
> > If that is a possibility for you as well, we could get you
> > going again.
> > Especially if no one else has any ideas.
> > ;-)
> >
> >
> > On Mar 30, 2004, at 5:37 AM, Filippo Capocasale wrote:
> >
> > > Hello everybody!
> > > I am facing a severe problem and I really hope someone may
> > help me.
> > > I have a java application running on Solaris.
> > > This application starts an xml-rpc servers, and some handler
> > is
> > > registered.
> > > I am using the version 1.2-b1 of Apache xml-rpc.
> > > I have a huge number of clients connecting to this xml-rpc
> > server.
> > > Everything works correctly for several days (i.e. thowsands
> > of
> > > connections); then, suddently, the server stops responding.
> > > I don't see any error in the logs of my application, the
> > process is
> > > still up and running (the same application has got another
> > xml-rpc
> > > server listening on another port and it works perfectly),
> > the server
> > > seems to be in listening on the port (I see it by the
> > command netstat
> > > -a).
> > > On the client side I see the error: "Unexpected end of file
> > from
> > > server".
> > > It looks like the server truncates every incoming
> > connection.
> > > There is no problem with firewalls: I've also tried to run a
> > client on
> > > the same machine.
> > > If I restart my application everything restarts working
> > fine.
> > > Well, this is the situation, and I really don't know what to
> > do...
> > > I just hope it is not an overload problem...
> > >
> > > I would really apreciate every help or suggestion!!!
> > > Thankyou very much,
> > > Filippo
> > >
> > >
> > -----------------------------------------------------------------
> > > Questo messaggio � stato inviato utilizzando
> > http://it.my.gsmbox.com
> > >
> >
> >
> > Dave Stewart
> > Aqua~Flo Supply (Goleta CA)
> > dstewart at aquaflo dot com
> >
> > The trouble with doing something right the first time
> > is that nobody appreciates how difficult it was.
> >
> >
>
>
> -----------------------------------------------------------------
> Questo messaggio � stato inviato utilizzando http://it.my.gsmbox.com
>
XML-RPC md5 authentication
Posted by Sang Park <ir...@bittersoft.com>.
Hi all, I have a question that I hope is not too difficult.
I've written an XML-RPC server and would like to use md5 authentication
(or some other http based authentication scheme). Does anyone know
where I can find a good example or some documentation on how to go about
doing this?
Thanks
Sang
Re: XML-RPC server suddently stops responding
Posted by Filippo Capocasale <ke...@gsmbox.it>.
Hi Dave,
unluckily I cannot restart the server, because we have to guarrantee 24H availability.
Anyway I thank you for your suggest!
Bye!
Filippo
Scrive Dave Stewart <ds...@aquaflo.com>:
> Hi Filippo!
>
> I don't have any answers for you as to why this is happening,
> but I've
> been (unnecessarily?) fearful of something similar. In my
> case, I'm
> able to have cron restart the rpc server daily (at 1:00 AM).
>
> If that is a possibility for you as well, we could get you
> going again.
> Especially if no one else has any ideas.
> ;-)
>
>
> On Mar 30, 2004, at 5:37 AM, Filippo Capocasale wrote:
>
> > Hello everybody!
> > I am facing a severe problem and I really hope someone may
> help me.
> > I have a java application running on Solaris.
> > This application starts an xml-rpc servers, and some handler
> is
> > registered.
> > I am using the version 1.2-b1 of Apache xml-rpc.
> > I have a huge number of clients connecting to this xml-rpc
> server.
> > Everything works correctly for several days (i.e. thowsands
> of
> > connections); then, suddently, the server stops responding.
> > I don't see any error in the logs of my application, the
> process is
> > still up and running (the same application has got another
> xml-rpc
> > server listening on another port and it works perfectly),
> the server
> > seems to be in listening on the port (I see it by the
> command netstat
> > -a).
> > On the client side I see the error: "Unexpected end of file
> from
> > server".
> > It looks like the server truncates every incoming
> connection.
> > There is no problem with firewalls: I've also tried to run a
> client on
> > the same machine.
> > If I restart my application everything restarts working
> fine.
> > Well, this is the situation, and I really don't know what to
> do...
> > I just hope it is not an overload problem...
> >
> > I would really apreciate every help or suggestion!!!
> > Thankyou very much,
> > Filippo
> >
> >
> -----------------------------------------------------------------
> > Questo messaggio � stato inviato utilizzando
> http://it.my.gsmbox.com
> >
>
>
> Dave Stewart
> Aqua~Flo Supply (Goleta CA)
> dstewart at aquaflo dot com
>
> The trouble with doing something right the first time
> is that nobody appreciates how difficult it was.
>
>
-----------------------------------------------------------------
Questo messaggio � stato inviato utilizzando http://it.my.gsmbox.com
Re: XML-RPC server suddently stops responding
Posted by Dave Stewart <ds...@aquaflo.com>.
Hi Filippo!
I don't have any answers for you as to why this is happening, but I've
been (unnecessarily?) fearful of something similar. In my case, I'm
able to have cron restart the rpc server daily (at 1:00 AM).
If that is a possibility for you as well, we could get you going again.
Especially if no one else has any ideas.
;-)
On Mar 30, 2004, at 5:37 AM, Filippo Capocasale wrote:
> Hello everybody!
> I am facing a severe problem and I really hope someone may help me.
> I have a java application running on Solaris.
> This application starts an xml-rpc servers, and some handler is
> registered.
> I am using the version 1.2-b1 of Apache xml-rpc.
> I have a huge number of clients connecting to this xml-rpc server.
> Everything works correctly for several days (i.e. thowsands of
> connections); then, suddently, the server stops responding.
> I don't see any error in the logs of my application, the process is
> still up and running (the same application has got another xml-rpc
> server listening on another port and it works perfectly), the server
> seems to be in listening on the port (I see it by the command netstat
> -a).
> On the client side I see the error: "Unexpected end of file from
> server".
> It looks like the server truncates every incoming connection.
> There is no problem with firewalls: I've also tried to run a client on
> the same machine.
> If I restart my application everything restarts working fine.
> Well, this is the situation, and I really don't know what to do...
> I just hope it is not an overload problem...
>
> I would really apreciate every help or suggestion!!!
> Thankyou very much,
> Filippo
>
> -----------------------------------------------------------------
> Questo messaggio è stato inviato utilizzando http://it.my.gsmbox.com
>
Dave Stewart
Aqua~Flo Supply (Goleta CA)
dstewart at aquaflo dot com
The trouble with doing something right the first time
is that nobody appreciates how difficult it was.