You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@tomcat.apache.org by Amol Puglia <am...@yahoo.com> on 2012/02/06 09:58:28 UTC
Frequently Getting Service Temporaily availbale message from backend tomcat servers
Hello Team,
I have configured apache to load balanced six tomcat instances in the backend using mod_proxy_balance and mod_proxy_ajp modules.
I am frequently getting service temporary unavailable message and following error in the apache error logs.
[Mon Feb 06 09:07:28 2012] [error] proxy: BALANCER: (balancer://cluster). All workers are in error state for route (marsprod_rmiserver_3)
[Mon Feb 06 09:07:32 2012] [error] proxy: BALANCER: (balancer://cluster). All workers are in error state for route (marsprod_rmiserver_3)
[Mon Feb 06 09:07:36 2012] [error] proxy: BALANCER: (balancer://cluster). All workers are in error state for route (marsprod_rmiserver_3)
[Mon Feb 06 09:07:37 2012] [error] proxy: BALANCER: (balancer://cluster). All workers are in error state for route (marsprod_rmiserver_3)
[Mon Feb 06 09:07:42 2012] [error] proxy: BALANCER: (balancer://cluster). All workers are in error state for route (marsprod_rmiserver_3)
[Mon Feb 06 09:07:45 2012] [error] proxy: BALANCER: (balancer://cluster). All workers are in error state for route (marsprod_rmiserver_3)
[Mon Feb 06 09:08:02 2012] [error] proxy: BALANCER: (balancer://cluster). All workers are in error state for route (marsprod_rmiserver_3)
I am using following version of apache and tomcat.
Apache version :- Apache/2.2.16
Tomcat :- 6.0.26
Jdk :-1.6.0_24
Below is the configuration of my apache and tomcat.
# Port 80
Listen server_name:80
<VirtualHost _default_:80>
ServerName server_name
ServerAlias server_name
ServerAlias server_name
<Directory /eMatrix>
Order deny,allow
Deny from all
Allow from 153.88.251.174 153.88.251.160 153.88.251.165 153.88.251.170 153.88.251.212 153.88.251.199
</Directory>
Timeout 1800
ProxyTimeout 1800
ProxyRequests Off
ProxyPass /eMatrix balancer://cluster stickysession=JSESSIONID|jsessionid nofailover=On
#ProxyPass /eMatrix balancer://cluster stickysession=JSESSIONID nofailover=On
ProxyPreserveHost On
ProxyPass /eMatrix ajp://server_name:8009/eMatrix
ProxyPass /eMatrix ajp://server_name:8010/eMatrix
ProxyPass /eMatrix ajp://server_name:8011/eMatrix
ProxyPass /eMatrix ajp://server_name:9009/eMatrix
ProxyPass /eMatrix ajp://server_name:9010/eMatrix
ProxyPass /eMatrix ajp://server_name:9011/eMatrix
ProxyPassReverse /eMatrix ajp://server_name:8009/eMatrix
ProxyPassReverse /eMatrix ajp://server_name:8010/eMatrix
ProxyPassReverse /eMatrix ajp://server_name:8011/eMatrix
ProxyPassReverse /eMatrix ajp://server_name:9009/eMatrix
ProxyPassReverse /eMatrix ajp://server_name:9010/eMatrix
ProxyPassReverse /eMatrix ajp://server_name:9011/eMatrix
<Proxy balancer://cluster>
BalancerMember ajp://server_name:8009/eMatrix route=marsprod_rmiserver_1 loadfactor=33 retry=60
BalancerMember ajp://server_name:8010/eMatrix route=marsprod_rmiserver_2 loadfactor=33 retry=60
BalancerMember ajp://server_name:8011/eMatrix route=marsprod_rmiserver_3 loadfactor=33 retry=60
BalancerMember ajp://server_name:9009/eMatrix route=marsprod_rmiserver_4 loadfactor=33 retry=60
BalancerMember ajp://server_name:9010/eMatrix route=marsprod_rmiserver_5 loadfactor=33 retry=60
#load balancing performed based on number of user requests
#ProxySet lbmethod=byrequests
# Report server is for Report purpose only so balancing is required now.
BalancerMember ajp://server_name:9011/eMatrix route=marsprod_rmiserver_6 loadfactor=33 retry=60
ProxySet lbmethod=byrequests
</Proxy>
#Status page for balancer
<Location /balancer>
SetHandler balancer-manager
Order Deny,Allow
Deny from all
Allow from all
</Location>
ProxyStatus On
<Location /status>
SetHandler server-status
Order Deny,Allow
Deny from all
Allow from all
</Location>
ErrorLog /opt/web/apache/app/mxora/logs/error.log
</VirtualHost>
Tomcat Configuration Server.xml
<!-- for Native tomcat setup-->
<Connector port="8082" protocol="org.apache.coyote.http11.Http11AprProtocol"
maxHttpHeaderSize="8192"
maxThreads="60"
maxPostSize="0"
enableLookups="false"
redirectPort="8443"
acceptCount="150"
compression="on"
connectionTimeout="60000"
disableUploadTimeout="true"
URIEncoding="UTF-8"/>
<!-- Define an AJP 1.3 Connector on port 9084 -->
<Connector port="8010"
protocol="AJP/1.3"
redirectPort="8443"
maxThreads="60"
enableLookups="false"
maxPostSize="0"
connectionTimeout="60000"
URIEncoding="UTF-8"/>
Kindly help me to trobleshoot the issue.
Re: Frequently Getting Service Temporaily availbale message from
backend tomcat servers
Posted by André Warnier <aw...@ice-sa.com>.
Amol Puglia wrote:
> Hi,
>
> Thanks for the information.
>
>
> Is it advisable to use worker mpm or prefork mpm?
The Apache httpd on-line documentation or support mailing list would be a better place for
that question.
Without guarantees, what I remember : the prefork mpm is very stable, and the worker mpm
was said (a while ago) to be rather experimental and not recommended for production sites.
(But that's old info and it may have changed since).
Check it with the on-line documentation for your version.
The worker mpm is supposed to be more scalable for very large loads, because it uses
threads versus children processes, so its memory and resource usage is probably less.
>
> If i use worker mpm will it resolve this issue?
It is not possible to answer that without knowing what your issue really is.
Pid sent some comments on your configuration. Read them and try them out first.
Do not start changing the basics of your setup, because then you'll never know what your
problem was and what fixed it.
>
>
> Please let me know what shall i do if i have to use worker mpm.
I would not do that. Resolve the current issue first.
You can later try with the worker mpm, to see if in your case it works better.
You have not told us what kind of load (number of requests per second, average time to
process one request) you expect your server to be submitted to, so it's hard to say more
right now.
In simple cases, the basic rule is : for each request that is being processed /at the same
time/ by your front-end Apache httpd, there will be one active AJP connection to a Tomcat
back-end, and a Tomcat thread active to process that request.
If your front-end is a prefork type, then the number of requests being processed at the
same time is limited by the number of Apache children (MaxClients).
If all front-end requests result in a call to a back-end Tomcat (or several Tomcats), then
the (total) number of threads available in all the Tomcat(s) should not be lower than the
MaxClients on your front-end server. (In a simple Tomcat configuration, that would be the
total of all the maxThreads of all your AJP Connectors).
Of course, the total number of Apache children and Tomcat threads that can be active at
the same time, depends on the resources of your host(s), and on your application.
I find that a simple picture often helps :
browser --> HTTP --> Apache httpd --> AJP --> Tomcat Connector --> thread
browser (child) --> AJP --> Tomcat Connector --> thread
browser --> AJP --> Tomcat Connector --> thread
...
One browser request results in one HTTP connection to Apache httpd, and one AJP connection
from httpd to a Tomcat Connector, which fires one thread to handle the request. The
request takes a certain time to be processed and answered, during which time other browser
requests arrive at the front-end.
The point is to balance all these elements. If one of them is too large, you are wasting
resources; if one of them is too small, you are creating a bottleneck.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Frequently Getting Service Temporaily availbale message from backend tomcat servers
Posted by Amol Puglia <am...@yahoo.com>.
Hi,
Thanks for the information.
Is it advisable to use worker mpm or prefork mpm?
If i use worker mpm will it resolve this issue?
Please let me know what shall i do if i have to use worker mpm.
kindly Let me know your inputs.
________________________________
From: André Warnier <aw...@ice-sa.com>
To: Tomcat Users List <us...@tomcat.apache.org>
Sent: Monday, February 6, 2012 4:17 PM
Subject: Re: Frequently Getting Service Temporaily availbale message from backend tomcat servers
Hi.
Amol Puglia wrote:
> Hello ,
>
> Thanks for the response.
>
> I am using following worker and no of threads.
>
> Worker is Prefork and
> ThreadsPerChild 25
>
In the context of a pre-fork Apache, ThreadsPerChild has no meaning and is not used.
> Below is the details for the same.
>
>
> Server version: Apache/2.2.16 (Unix)
> Server built: Sep 27 2010 12:48:44
> Server's Module Magic Number: 20051115:24
> Server loaded: APR 1.4.2, APR-Util 1.3.9
> Compiled using: APR 1.4.2, APR-Util 1.3.9
> Architecture: 32-bit
> Server MPM: Prefork
> threaded: no
> forked: yes (variable process count)
>
The following configuration section will be ignored, because the MPM is not "worker", it is "Prefork". So you need to look at the section <IfModule prefork.c>, not the one below.
> <IfModule worker.c>
> StartServers 2
> MaxClients 250
> MinSpareThreads 25
> MaxSpareThreads 75
> ThreadsPerChild 25
> MaxRequestsPerChild 0
> </IfModule>
>
The following look ok and resonable.
> Timeout 300
> KeepAlive On
> MaxKeepAliveRequests 100
> KeepAliveTimeout 15
> HostnameLookups Off
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Frequently Getting Service Temporaily availbale message from
backend tomcat servers
Posted by André Warnier <aw...@ice-sa.com>.
Hi.
Amol Puglia wrote:
> Hello ,
>
> Thanks for the response.
>
> I am using following worker and no of threads.
>
> Worker is Prefork and
>
> ThreadsPerChild 25
>
In the context of a pre-fork Apache, ThreadsPerChild has no meaning and is not used.
> Below is the details for the same.
>
>
> Server version: Apache/2.2.16 (Unix)
> Server built: Sep 27 2010 12:48:44
> Server's Module Magic Number: 20051115:24
> Server loaded: APR 1.4.2, APR-Util 1.3.9
> Compiled using: APR 1.4.2, APR-Util 1.3.9
> Architecture: 32-bit
> Server MPM: Prefork
> threaded: no
> forked: yes (variable process count)
>
The following configuration section will be ignored, because the MPM is not "worker", it
is "Prefork". So you need to look at the section <IfModule prefork.c>, not the one below.
> <IfModule worker.c>
> StartServers 2
> MaxClients 250
> MinSpareThreads 25
> MaxSpareThreads 75
> ThreadsPerChild 25
> MaxRequestsPerChild 0
> </IfModule>
>
The following look ok and resonable.
> Timeout 300
> KeepAlive On
> MaxKeepAliveRequests 100
> KeepAliveTimeout 15
> HostnameLookups Off
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Frequently Getting Service Temporaily availbale message from
backend tomcat servers
Posted by Pid <pi...@pidster.com>.
On 06/02/2012 10:26, Amol Puglia wrote:
> Hello ,
>
> Thanks for the response.
>
> I am using following worker and no of threads.
>
> Worker is Prefork and
>
> ThreadsPerChild 25
>
> Below is the details for the same.
Please post your reply below the relevant part of the previous email.
In most email clients this is a case of scrolling down a bit. Not doing
so leads to 'top-posting' which makes following the conversation flow
harder.
> Server version: Apache/2.2.16 (Unix)
> Server built: Sep 27 2010 12:48:44
> Server's Module Magic Number: 20051115:24
> Server loaded: APR 1.4.2, APR-Util 1.3.9
> Compiled using: APR 1.4.2, APR-Util 1.3.9
> Architecture: 32-bit
> Server MPM: Prefork
> threaded: no
> forked: yes (variable process count)
>
> <IfModule worker.c>
> StartServers 2
> MaxClients 250
> MinSpareThreads 25
> MaxSpareThreads 75
> ThreadsPerChild 25
> MaxRequestsPerChild 0
> </IfModule>
Please post prefork.c instead, as that's what you're using.
p
> Timeout 300
> KeepAlive On
> MaxKeepAliveRequests 100
> KeepAliveTimeout 15
> HostnameLookups Off
> ________________________________
> From: Pid <pi...@pidster.com>
> To: Tomcat Users List <us...@tomcat.apache.org>
> Sent: Monday, February 6, 2012 3:44 PM
> Subject: Re: Frequently Getting Service Temporaily availbale message from backend tomcat servers
>
> On 06/02/2012 08:58, Amol Puglia wrote:
>> Hello Team,
>>
>> I have configured apache to load balanced six tomcat instances in the backend using mod_proxy_balance and mod_proxy_ajp modules.
>>
>> I am frequently getting service temporary unavailable message and following error in the apache error logs.
>>
>> [Mon Feb 06 09:07:28 2012] [error] proxy: BALANCER: (balancer://cluster). All workers are in error state for route (marsprod_rmiserver_3)
>> [Mon Feb 06 09:07:32 2012] [error] proxy: BALANCER: (balancer://cluster). All workers are in error state for route (marsprod_rmiserver_3)
>> [Mon Feb 06 09:07:36 2012] [error] proxy: BALANCER: (balancer://cluster). All workers are in error state for route (marsprod_rmiserver_3)
>> [Mon Feb 06 09:07:37 2012] [error] proxy: BALANCER: (balancer://cluster). All workers are in error state for route (marsprod_rmiserver_3)
>> [Mon Feb 06 09:07:42 2012] [error] proxy: BALANCER: (balancer://cluster). All workers are in error state for route (marsprod_rmiserver_3)
>> [Mon Feb 06 09:07:45 2012] [error] proxy: BALANCER: (balancer://cluster). All workers are in error state for route (marsprod_rmiserver_3)
>> [Mon Feb 06 09:08:02 2012] [error] proxy: BALANCER: (balancer://cluster). All workers are in error state for route (marsprod_rmiserver_3)
>>
>>
>> I am using following version of apache and tomcat.
>>
>> Apache version :- Apache/2.2.16
>>
>> Tomcat :- 6.0.26
>>
>> Jdk :-1.6.0_24
>
> Thanks for providing the information above. You should plan an upgrade
> of each of those ASAP.
>
>
>> Below is the configuration of my apache and tomcat.
>>
>> # Port 80
>> Listen server_name:80
>>
>> <VirtualHost _default_:80>
>> ServerName server_name
>> ServerAlias server_name
>> ServerAlias server_name
>>
>>
>> <Directory /eMatrix>
>>
>> Order deny,allow
>>
>> Deny from all
>>
>> Allow from 153.88.251.174 153.88.251.160 153.88.251.165 153.88.251.170 153.88.251.212 153.88.251.199
>>
>> </Directory>
>>
>> Timeout 1800
>> ProxyTimeout 1800
>> ProxyRequests Off
>>
>>
>> ProxyPass /eMatrix balancer://cluster stickysession=JSESSIONID|jsessionid nofailover=On
>
> OK - here is your balancer definition, you'll need to change it to:
>
> ProxyPass /eMatrix balancer://cluster/eMatrix
> stickysession=JSESSIONID|jsessionid nofailover=On
>
>> #ProxyPass /eMatrix balancer://cluster stickysession=JSESSIONID nofailover=On
>>
>> ProxyPreserveHost On
>>
>>
>> ProxyPass /eMatrix ajp://server_name:8009/eMatrix
>>
>> ProxyPass /eMatrix ajp://server_name:8010/eMatrix
>>
>> ProxyPass /eMatrix ajp://server_name:8011/eMatrix
>>
>> ProxyPass /eMatrix ajp://server_name:9009/eMatrix
>>
>> ProxyPass /eMatrix ajp://server_name:9010/eMatrix
>>
>> ProxyPass /eMatrix ajp://server_name:9011/eMatrix
>
> If you have these lines, they will each override the one above. So your
> balancer is overridden.
>
>
>> ProxyPassReverse /eMatrix ajp://server_name:8009/eMatrix
>>
>> ProxyPassReverse /eMatrix ajp://server_name:8010/eMatrix
>>
>> ProxyPassReverse /eMatrix ajp://server_name:8011/eMatrix
>>
>> ProxyPassReverse /eMatrix ajp://server_name:9009/eMatrix
>>
>> ProxyPassReverse /eMatrix ajp://server_name:9010/eMatrix
>>
>> ProxyPassReverse /eMatrix ajp://server_name:9011/eMatrix
>
> The ProxyPassReverse won't be needed.
>
>> <Proxy balancer://cluster>
>> BalancerMember ajp://server_name:8009/eMatrix route=marsprod_rmiserver_1 loadfactor=33 retry=60
>> BalancerMember ajp://server_name:8010/eMatrix route=marsprod_rmiserver_2 loadfactor=33 retry=60
>> BalancerMember ajp://server_name:8011/eMatrix route=marsprod_rmiserver_3 loadfactor=33 retry=60
>> BalancerMember ajp://server_name:9009/eMatrix route=marsprod_rmiserver_4 loadfactor=33 retry=60
>> BalancerMember ajp://server_name:9010/eMatrix route=marsprod_rmiserver_5 loadfactor=33 retry=60
>
> Remove the app path from the balancer members:
>
> BalancerMember ajp://server_name:9010 route=marsprod_rmiserver_5
> loadfactor=33 retry=60
>
>
>> #load balancing performed based on number of user requests
>> #ProxySet lbmethod=byrequests
>> # Report server is for Report purpose only so balancing is required now.
>> BalancerMember ajp://server_name:9011/eMatrix route=marsprod_rmiserver_6 loadfactor=33 retry=60
>> ProxySet lbmethod=byrequests
>> </Proxy>
>>
>> #Status page for balancer
>>
>> <Location /balancer>
>> SetHandler balancer-manager
>> Order Deny,Allow
>> Deny from all
>> Allow from all
>> </Location>
>>
>> ProxyStatus On
>>
>> <Location /status>
>> SetHandler server-status
>> Order Deny,Allow
>> Deny from all
>> Allow from all
>> </Location>
>>
>> ErrorLog /opt/web/apache/app/mxora/logs/error.log
>>
>> </VirtualHost>
>>
>>
>> Tomcat Configuration Server.xml
>>
>> <!-- for Native tomcat setup-->
>> <Connector port="8082" protocol="org.apache.coyote.http11.Http11AprProtocol"
>> maxHttpHeaderSize="8192"
>> maxThreads="60"
>> maxPostSize="0"
>> enableLookups="false"
>> redirectPort="8443"
>> acceptCount="150"
>> compression="on"
>> connectionTimeout="60000"
>> disableUploadTimeout="true"
>> URIEncoding="UTF-8"/>
>>
>>
>> <!-- Define an AJP 1.3 Connector on port 9084 -->
>> <Connector port="8010"
>> protocol="AJP/1.3"
>> redirectPort="8443"
>> maxThreads="60"
>> enableLookups="false"
>> maxPostSize="0"
>> connectionTimeout="60000"
>> URIEncoding="UTF-8"/>
>
> You have 60 threads per connector. How did you determine this value?
>
> In order to calculate how many connections you need, we need to see more
> details of the HTTPD config for your server. Specifically, we'll need
> to work out whether you're using the Prefork or Worker MPM and what the
> value of ThreadsPerChild is.
>
>
>> Kindly help me to trobleshoot the issue.
>
>
> Please include the Engine definition as well.
>
>
> p
>
--
[key:62590808]
Re: Frequently Getting Service Temporaily availbale message from backend tomcat servers
Posted by Amol Puglia <am...@yahoo.com>.
Hello ,
Thanks for the response.
I am using following worker and no of threads.
Worker is Prefork and
ThreadsPerChild 25
Below is the details for the same.
Server version: Apache/2.2.16 (Unix)
Server built: Sep 27 2010 12:48:44
Server's Module Magic Number: 20051115:24
Server loaded: APR 1.4.2, APR-Util 1.3.9
Compiled using: APR 1.4.2, APR-Util 1.3.9
Architecture: 32-bit
Server MPM: Prefork
threaded: no
forked: yes (variable process count)
<IfModule worker.c>
StartServers 2
MaxClients 250
MinSpareThreads 25
MaxSpareThreads 75
ThreadsPerChild 25
MaxRequestsPerChild 0
</IfModule>
Timeout 300
KeepAlive On
MaxKeepAliveRequests 100
KeepAliveTimeout 15
HostnameLookups Off
________________________________
From: Pid <pi...@pidster.com>
To: Tomcat Users List <us...@tomcat.apache.org>
Sent: Monday, February 6, 2012 3:44 PM
Subject: Re: Frequently Getting Service Temporaily availbale message from backend tomcat servers
On 06/02/2012 08:58, Amol Puglia wrote:
> Hello Team,
>
> I have configured apache to load balanced six tomcat instances in the backend using mod_proxy_balance and mod_proxy_ajp modules.
>
> I am frequently getting service temporary unavailable message and following error in the apache error logs.
>
> [Mon Feb 06 09:07:28 2012] [error] proxy: BALANCER: (balancer://cluster). All workers are in error state for route (marsprod_rmiserver_3)
> [Mon Feb 06 09:07:32 2012] [error] proxy: BALANCER: (balancer://cluster). All workers are in error state for route (marsprod_rmiserver_3)
> [Mon Feb 06 09:07:36 2012] [error] proxy: BALANCER: (balancer://cluster). All workers are in error state for route (marsprod_rmiserver_3)
> [Mon Feb 06 09:07:37 2012] [error] proxy: BALANCER: (balancer://cluster). All workers are in error state for route (marsprod_rmiserver_3)
> [Mon Feb 06 09:07:42 2012] [error] proxy: BALANCER: (balancer://cluster). All workers are in error state for route (marsprod_rmiserver_3)
> [Mon Feb 06 09:07:45 2012] [error] proxy: BALANCER: (balancer://cluster). All workers are in error state for route (marsprod_rmiserver_3)
> [Mon Feb 06 09:08:02 2012] [error] proxy: BALANCER: (balancer://cluster). All workers are in error state for route (marsprod_rmiserver_3)
>
>
> I am using following version of apache and tomcat.
>
> Apache version :- Apache/2.2.16
>
> Tomcat :- 6.0.26
>
> Jdk :-1.6.0_24
Thanks for providing the information above. You should plan an upgrade
of each of those ASAP.
> Below is the configuration of my apache and tomcat.
>
> # Port 80
> Listen server_name:80
>
> <VirtualHost _default_:80>
> ServerName server_name
> ServerAlias server_name
> ServerAlias server_name
>
>
> <Directory /eMatrix>
>
> Order deny,allow
>
> Deny from all
>
> Allow from 153.88.251.174 153.88.251.160 153.88.251.165 153.88.251.170 153.88.251.212 153.88.251.199
>
> </Directory>
>
> Timeout 1800
> ProxyTimeout 1800
> ProxyRequests Off
>
>
> ProxyPass /eMatrix balancer://cluster stickysession=JSESSIONID|jsessionid nofailover=On
OK - here is your balancer definition, you'll need to change it to:
ProxyPass /eMatrix balancer://cluster/eMatrix
stickysession=JSESSIONID|jsessionid nofailover=On
> #ProxyPass /eMatrix balancer://cluster stickysession=JSESSIONID nofailover=On
>
> ProxyPreserveHost On
>
>
> ProxyPass /eMatrix ajp://server_name:8009/eMatrix
>
> ProxyPass /eMatrix ajp://server_name:8010/eMatrix
>
> ProxyPass /eMatrix ajp://server_name:8011/eMatrix
>
> ProxyPass /eMatrix ajp://server_name:9009/eMatrix
>
> ProxyPass /eMatrix ajp://server_name:9010/eMatrix
>
> ProxyPass /eMatrix ajp://server_name:9011/eMatrix
If you have these lines, they will each override the one above. So your
balancer is overridden.
> ProxyPassReverse /eMatrix ajp://server_name:8009/eMatrix
>
> ProxyPassReverse /eMatrix ajp://server_name:8010/eMatrix
>
> ProxyPassReverse /eMatrix ajp://server_name:8011/eMatrix
>
> ProxyPassReverse /eMatrix ajp://server_name:9009/eMatrix
>
> ProxyPassReverse /eMatrix ajp://server_name:9010/eMatrix
>
> ProxyPassReverse /eMatrix ajp://server_name:9011/eMatrix
The ProxyPassReverse won't be needed.
> <Proxy balancer://cluster>
> BalancerMember ajp://server_name:8009/eMatrix route=marsprod_rmiserver_1 loadfactor=33 retry=60
> BalancerMember ajp://server_name:8010/eMatrix route=marsprod_rmiserver_2 loadfactor=33 retry=60
> BalancerMember ajp://server_name:8011/eMatrix route=marsprod_rmiserver_3 loadfactor=33 retry=60
> BalancerMember ajp://server_name:9009/eMatrix route=marsprod_rmiserver_4 loadfactor=33 retry=60
> BalancerMember ajp://server_name:9010/eMatrix route=marsprod_rmiserver_5 loadfactor=33 retry=60
Remove the app path from the balancer members:
BalancerMember ajp://server_name:9010 route=marsprod_rmiserver_5
loadfactor=33 retry=60
> #load balancing performed based on number of user requests
> #ProxySet lbmethod=byrequests
> # Report server is for Report purpose only so balancing is required now.
> BalancerMember ajp://server_name:9011/eMatrix route=marsprod_rmiserver_6 loadfactor=33 retry=60
> ProxySet lbmethod=byrequests
> </Proxy>
>
> #Status page for balancer
>
> <Location /balancer>
> SetHandler balancer-manager
> Order Deny,Allow
> Deny from all
> Allow from all
> </Location>
>
> ProxyStatus On
>
> <Location /status>
> SetHandler server-status
> Order Deny,Allow
> Deny from all
> Allow from all
> </Location>
>
> ErrorLog /opt/web/apache/app/mxora/logs/error.log
>
> </VirtualHost>
>
>
> Tomcat Configuration Server.xml
>
> <!-- for Native tomcat setup-->
> <Connector port="8082" protocol="org.apache.coyote.http11.Http11AprProtocol"
> maxHttpHeaderSize="8192"
> maxThreads="60"
> maxPostSize="0"
> enableLookups="false"
> redirectPort="8443"
> acceptCount="150"
> compression="on"
> connectionTimeout="60000"
> disableUploadTimeout="true"
> URIEncoding="UTF-8"/>
>
>
> <!-- Define an AJP 1.3 Connector on port 9084 -->
> <Connector port="8010"
> protocol="AJP/1.3"
> redirectPort="8443"
> maxThreads="60"
> enableLookups="false"
> maxPostSize="0"
> connectionTimeout="60000"
> URIEncoding="UTF-8"/>
You have 60 threads per connector. How did you determine this value?
In order to calculate how many connections you need, we need to see more
details of the HTTPD config for your server. Specifically, we'll need
to work out whether you're using the Prefork or Worker MPM and what the
value of ThreadsPerChild is.
> Kindly help me to trobleshoot the issue.
Please include the Engine definition as well.
p
--
[key:62590808]
Re: Frequently Getting Service Temporaily availbale message from
backend tomcat servers
Posted by Pid <pi...@pidster.com>.
On 06/02/2012 08:58, Amol Puglia wrote:
> Hello Team,
>
> I have configured apache to load balanced six tomcat instances in the backend using mod_proxy_balance and mod_proxy_ajp modules.
>
> I am frequently getting service temporary unavailable message and following error in the apache error logs.
>
> [Mon Feb 06 09:07:28 2012] [error] proxy: BALANCER: (balancer://cluster). All workers are in error state for route (marsprod_rmiserver_3)
> [Mon Feb 06 09:07:32 2012] [error] proxy: BALANCER: (balancer://cluster). All workers are in error state for route (marsprod_rmiserver_3)
> [Mon Feb 06 09:07:36 2012] [error] proxy: BALANCER: (balancer://cluster). All workers are in error state for route (marsprod_rmiserver_3)
> [Mon Feb 06 09:07:37 2012] [error] proxy: BALANCER: (balancer://cluster). All workers are in error state for route (marsprod_rmiserver_3)
> [Mon Feb 06 09:07:42 2012] [error] proxy: BALANCER: (balancer://cluster). All workers are in error state for route (marsprod_rmiserver_3)
> [Mon Feb 06 09:07:45 2012] [error] proxy: BALANCER: (balancer://cluster). All workers are in error state for route (marsprod_rmiserver_3)
> [Mon Feb 06 09:08:02 2012] [error] proxy: BALANCER: (balancer://cluster). All workers are in error state for route (marsprod_rmiserver_3)
>
>
> I am using following version of apache and tomcat.
>
> Apache version :- Apache/2.2.16
>
> Tomcat :- 6.0.26
>
> Jdk :-1.6.0_24
Thanks for providing the information above. You should plan an upgrade
of each of those ASAP.
> Below is the configuration of my apache and tomcat.
>
> # Port 80
> Listen server_name:80
>
> <VirtualHost _default_:80>
> ServerName server_name
> ServerAlias server_name
> ServerAlias server_name
>
>
> <Directory /eMatrix>
>
> Order deny,allow
>
> Deny from all
>
> Allow from 153.88.251.174 153.88.251.160 153.88.251.165 153.88.251.170 153.88.251.212 153.88.251.199
>
> </Directory>
>
> Timeout 1800
> ProxyTimeout 1800
> ProxyRequests Off
>
>
> ProxyPass /eMatrix balancer://cluster stickysession=JSESSIONID|jsessionid nofailover=On
OK - here is your balancer definition, you'll need to change it to:
ProxyPass /eMatrix balancer://cluster/eMatrix
stickysession=JSESSIONID|jsessionid nofailover=On
> #ProxyPass /eMatrix balancer://cluster stickysession=JSESSIONID nofailover=On
>
> ProxyPreserveHost On
>
>
> ProxyPass /eMatrix ajp://server_name:8009/eMatrix
>
> ProxyPass /eMatrix ajp://server_name:8010/eMatrix
>
> ProxyPass /eMatrix ajp://server_name:8011/eMatrix
>
> ProxyPass /eMatrix ajp://server_name:9009/eMatrix
>
> ProxyPass /eMatrix ajp://server_name:9010/eMatrix
>
> ProxyPass /eMatrix ajp://server_name:9011/eMatrix
If you have these lines, they will each override the one above. So your
balancer is overridden.
> ProxyPassReverse /eMatrix ajp://server_name:8009/eMatrix
>
> ProxyPassReverse /eMatrix ajp://server_name:8010/eMatrix
>
> ProxyPassReverse /eMatrix ajp://server_name:8011/eMatrix
>
> ProxyPassReverse /eMatrix ajp://server_name:9009/eMatrix
>
> ProxyPassReverse /eMatrix ajp://server_name:9010/eMatrix
>
> ProxyPassReverse /eMatrix ajp://server_name:9011/eMatrix
The ProxyPassReverse won't be needed.
> <Proxy balancer://cluster>
> BalancerMember ajp://server_name:8009/eMatrix route=marsprod_rmiserver_1 loadfactor=33 retry=60
> BalancerMember ajp://server_name:8010/eMatrix route=marsprod_rmiserver_2 loadfactor=33 retry=60
> BalancerMember ajp://server_name:8011/eMatrix route=marsprod_rmiserver_3 loadfactor=33 retry=60
> BalancerMember ajp://server_name:9009/eMatrix route=marsprod_rmiserver_4 loadfactor=33 retry=60
> BalancerMember ajp://server_name:9010/eMatrix route=marsprod_rmiserver_5 loadfactor=33 retry=60
Remove the app path from the balancer members:
BalancerMember ajp://server_name:9010 route=marsprod_rmiserver_5
loadfactor=33 retry=60
> #load balancing performed based on number of user requests
> #ProxySet lbmethod=byrequests
> # Report server is for Report purpose only so balancing is required now.
> BalancerMember ajp://server_name:9011/eMatrix route=marsprod_rmiserver_6 loadfactor=33 retry=60
> ProxySet lbmethod=byrequests
> </Proxy>
>
> #Status page for balancer
>
> <Location /balancer>
> SetHandler balancer-manager
> Order Deny,Allow
> Deny from all
> Allow from all
> </Location>
>
> ProxyStatus On
>
> <Location /status>
> SetHandler server-status
> Order Deny,Allow
> Deny from all
> Allow from all
> </Location>
>
> ErrorLog /opt/web/apache/app/mxora/logs/error.log
>
> </VirtualHost>
>
>
> Tomcat Configuration Server.xml
>
> <!-- for Native tomcat setup-->
> <Connector port="8082" protocol="org.apache.coyote.http11.Http11AprProtocol"
> maxHttpHeaderSize="8192"
> maxThreads="60"
> maxPostSize="0"
> enableLookups="false"
> redirectPort="8443"
> acceptCount="150"
> compression="on"
> connectionTimeout="60000"
> disableUploadTimeout="true"
> URIEncoding="UTF-8"/>
>
>
> <!-- Define an AJP 1.3 Connector on port 9084 -->
> <Connector port="8010"
> protocol="AJP/1.3"
> redirectPort="8443"
> maxThreads="60"
> enableLookups="false"
> maxPostSize="0"
> connectionTimeout="60000"
> URIEncoding="UTF-8"/>
You have 60 threads per connector. How did you determine this value?
In order to calculate how many connections you need, we need to see more
details of the HTTPD config for your server. Specifically, we'll need
to work out whether you're using the Prefork or Worker MPM and what the
value of ThreadsPerChild is.
> Kindly help me to trobleshoot the issue.
Please include the Engine definition as well.
p
--
[key:62590808]