You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@httpd.apache.org by 张文东 <zw...@gmail.com> on 2009/03/24 09:37:18 UTC
[users@httpd] (mod_jk load balancer) could not get free endpoint for worker xxx
Hi all.
I'm using httpd 2.2 and websphere application community edition
(wasce) 2.1.1 (you may treat it as tomcat if you are not familiar with
wasce).
My goal is: the httpd server delivers 3000 requests per second to the
wasce servers. I use the mod_jk module, and the workers.properties
conf like this:
...
worker.node5.port=8009
worker.node5.host=9.186.96.71
worker.node5.type=ajp13
worker.node5.connection_pool_size=50
worker.node5.connection_pool_minsize=25
worker.node5.connection_pool_timeout=600
worker.node5.lbfactor=1
worker.loadbalancer.type=lb
worker.loadbalancer.balance_workers=node1, node2,node3,node4,node5
worker.loadbalancer.sticky_session=1
worker.loadbalancer.sticky_session_force=0
now the question is, I'm using mod_jk as the load balancer server.
when users' requests increase (more than 1000requests per second),
there are many warnings in the mod_jk.log:
[Tue Mar 24 15:36:37 2009] [14425:735282400] [warn]
ajp_get_endpoint::jk_ajp_common.c (2946): Unable to get the free
endpoint for worker node3 from 50 slots
[Tue Mar 24 15:36:37 2009] [14425:4059268320] [info]
service::jk_lb_worker.c (1161): could not get free endpoint for worker
node3 (0 retries)
and the performance of httpd reduce.
I want to know is there anybody having the SAME SITUATION or someone
knows is there any error in my worker.properties configuation (which
causes the could not get free endpoing error).
Thanks.
--
Best Regards!
Wen Dong
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] (mod_jk load balancer) could not get free endpoint
for worker xxx
Posted by Eric Covener <co...@gmail.com>.
2009/3/25 张文东 <zw...@gmail.com>:
>
> I think maybe the httpd server could not hold so many pool size
> (cache?). I want to increase the httpd's ThreadsPerChild. but I found
> it cannot set larger than 400 (this ThreadLimit is seted too). when
>>400, the httpd error_log output: "[alert] (12)Cannot allocate
> memory: apr_thread_create: unable to create worker thread".
Set ThreadStackSize or ulimit -s to something small (128k, 256k, etc).
--
Eric Covener
covener@gmail.com
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] (mod_jk load balancer) could not get free endpoint
for worker xxx
Posted by 张文东 <zw...@gmail.com>.
Hi André Warnier,
I modified the ajp connection configuation in wasce, such as ajp
maxthreads, but it seems that it's not the reason for the "could not
get free endpoint for worker" error.
I set a series pool size and test the differences between them. (one node case)
pool_size errors
25-50 5927
50-100 275
80-160 828
100-200 2
100-200 45
80-160 132
the test results show that the performance turns to be better when
pool_size increase. but when I add 2 nodes, the pool_size if 80~160
for each one (160~320 total) the httpd server occurs many 503 errors.
I think maybe the httpd server could not hold so many pool size
(cache?). I want to increase the httpd's ThreadsPerChild. but I found
it cannot set larger than 400 (this ThreadLimit is seted too). when
>400, the httpd error_log output: "[alert] (12)Cannot allocate
memory: apr_thread_create: unable to create worker thread".
btw, I'm using IBM HTTP SERVER 7 (whose kernal is based on
apache.2.2). And the Machine for httpd server is PPC (4cpus & 32G
RAM), the usage for httpd server is very low.
I'm trying to fix this problem, and further progress I will post to
you all. thanks for concern about this.
2009/3/24 André Warnier <aw...@ice-sa.com>:
> 张文东 wrote:
>>
>> Hi all.
>>
>> I'm using httpd 2.2 and websphere application community edition
>> (wasce) 2.1.1 (you may treat it as tomcat if you are not familiar with
>> wasce).
>> My goal is: the httpd server delivers 3000 requests per second to the
>> wasce servers. I use the mod_jk module, and the workers.properties
>> conf like this:
>>
>> ...
>> worker.node5.port=8009
>> worker.node5.host=9.186.96.71
>> worker.node5.type=ajp13
>> worker.node5.connection_pool_size=50
>> worker.node5.connection_pool_minsize=25
>> worker.node5.connection_pool_timeout=600
>> worker.node5.lbfactor=1
>>
>> worker.loadbalancer.type=lb
>> worker.loadbalancer.balance_workers=node1, node2,node3,node4,node5
>> worker.loadbalancer.sticky_session=1
>> worker.loadbalancer.sticky_session_force=0
>>
>> now the question is, I'm using mod_jk as the load balancer server.
>> when users' requests increase (more than 1000requests per second),
>> there are many warnings in the mod_jk.log:
>> [Tue Mar 24 15:36:37 2009] [14425:735282400] [warn]
>> ajp_get_endpoint::jk_ajp_common.c (2946): Unable to get the free
>> endpoint for worker node3 from 50 slots
>> [Tue Mar 24 15:36:37 2009] [14425:4059268320] [info]
>> service::jk_lb_worker.c (1161): could not get free endpoint for worker
>> node3 (0 retries)
>>
>> and the performance of httpd reduce.
>>
>> I want to know is there anybody having the SAME SITUATION or someone
>> knows is there any error in my worker.properties configuation (which
>> causes the could not get free endpoing error).
>>
> Not the ultimate specialist here, but what is the configuration of your
> back-end Connectors ? I mean, what it sounds like to me from the error
> messages above, is that the worker is just not able to connect to a free
> thread at the back-end level, meaning that the back-end nodes are saturated,
> not your front-end mod_jk.
>
>
>
> ---------------------------------------------------------------------
> The official User-To-User support forum of the Apache HTTP Server Project.
> See <URL:http://httpd.apache.org/userslist.html> for more info.
> To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
> " from the digest: users-digest-unsubscribe@httpd.apache.org
> For additional commands, e-mail: users-help@httpd.apache.org
>
>
--
Best Regards!
Wen Dong
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] (mod_jk load balancer) could not get free endpoint
for worker xxx
Posted by André Warnier <aw...@ice-sa.com>.
张文东 wrote:
> Hi all.
>
> I'm using httpd 2.2 and websphere application community edition
> (wasce) 2.1.1 (you may treat it as tomcat if you are not familiar with
> wasce).
> My goal is: the httpd server delivers 3000 requests per second to the
> wasce servers. I use the mod_jk module, and the workers.properties
> conf like this:
>
> ...
> worker.node5.port=8009
> worker.node5.host=9.186.96.71
> worker.node5.type=ajp13
> worker.node5.connection_pool_size=50
> worker.node5.connection_pool_minsize=25
> worker.node5.connection_pool_timeout=600
> worker.node5.lbfactor=1
>
> worker.loadbalancer.type=lb
> worker.loadbalancer.balance_workers=node1, node2,node3,node4,node5
> worker.loadbalancer.sticky_session=1
> worker.loadbalancer.sticky_session_force=0
>
> now the question is, I'm using mod_jk as the load balancer server.
> when users' requests increase (more than 1000requests per second),
> there are many warnings in the mod_jk.log:
> [Tue Mar 24 15:36:37 2009] [14425:735282400] [warn]
> ajp_get_endpoint::jk_ajp_common.c (2946): Unable to get the free
> endpoint for worker node3 from 50 slots
> [Tue Mar 24 15:36:37 2009] [14425:4059268320] [info]
> service::jk_lb_worker.c (1161): could not get free endpoint for worker
> node3 (0 retries)
>
> and the performance of httpd reduce.
>
> I want to know is there anybody having the SAME SITUATION or someone
> knows is there any error in my worker.properties configuation (which
> causes the could not get free endpoing error).
>
Not the ultimate specialist here, but what is the configuration of your
back-end Connectors ? I mean, what it sounds like to me from the error
messages above, is that the worker is just not able to connect to a free
thread at the back-end level, meaning that the back-end nodes are
saturated, not your front-end mod_jk.
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org