You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@tomcat.apache.org by Ruslan Gainutdinov <ru...@gmail.com> on 2010/09/06 17:54:31 UTC
Tomcat Connectors - Dynamic reloading of workers
Hello!
I am trying to implement dynamic reloading of workers on top of mod_jk
1.2.30 + apache 2.2.
However, I am getting strange errors related to shared memory and others.
== Initial design decisions ==
- Reloading is checked during watchdog, wc_maintain phase. This way
all connections can be freed, and w_maintain call ensures it is single
run only.
- mtime is checked, just as in jk_uri_worker_map.c
- wc_close/wc_open calls is made, if mtime is changed.
- No JkWorker* directives must be allowed, except JkWorkerFile
- No JkMount directives must be allowed, except JkMountFile
== Errors ==
close_workers::jk_worker.c (252): Closing existing 4 workers
close_workers::jk_worker.c (260): close_workers will destroy worker SATURN-1
ajp_destroy::jk_ajp_common.c (2947): up to 64 endpoints to close
ajp_close_endpoint::jk_ajp_common.c (775): closing endpoint with sd =
4294967295 (socket shutdown)
... 64 lines (note 4294967295 == 0xFFFFFFFF, somehow such messages
does not occur during standard shutdown procedure)
....
build_worker_map::jk_worker.c (284): creating worker SATURN-1
wc_create_worker::jk_worker.c (185): about to create instance SATURN-1 of ajp13
ajp_worker_factory::jk_ajp_common.c (2929): allocating ajp worker
record from shared memory
wc_create_worker::jk_worker.c (190): factory for ajp13 failed for SATURN-1
build_worker_map::jk_worker.c (304): failed to create worker SATURN-1
close_workers::jk_worker.c (252): Closing existing 0 workers
wc_create_worker::jk_worker.c (190): factory for ajp13 failed for SATURN-1
this error is particularly confusing and afaik, occurs
in call to jk_shm_alloc_ajp_worker, which in turn fails in
jk_shm.c:768 - jk_shm_alloc(p, JK_SHM_AJP_WORKER_SIZE);
i wonder,
maybe shared memory/or pool is readonly after initial setup phase?
is there other things I should watch for?
Here is the patch.
http://huksley.sdot.ru/wp-content/uploads/2010/09/mod_jk-1.2.30-reload-workers-patch.zip
I haven`t modified rest of source tree, related to IIS, Apache1X or
other webservers, so it wont compile.
With kindest personal regards,
Ruslan Gainutdinov
<ru...@gmail.com>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Tomcat Connectors - Dynamic reloading of workers
Posted by Rainer Jung <ra...@kippdata.de>.
Can you please add an enhancement issue in bugzilla and attach the
patch. That way it doesn't get lost and we can easily find it.
Thanks!
Rainer
On 07.09.2010 19:36, Ruslan Gainutdinov wrote:
> Thanks, JkShmSize helps.
>
> However this solution does not seems to be flexible enough. No way to
> tell how much memory needed beforehand.
> Is there any way to reinit shared memory?
> Seems to be a lot of such code on apache22 connector side...
>
> Haven`t looked into solution to reserve space by adding disabled workers.
> I believe it will be complicated, because it is not known whether
> actual workers will be simple nodes or lbcluster.
>
> I`ve made another patch.
> Added JkWorkersFileReload apache setting, by default disabled.
> It works if using JkWatchdogInterval only.
>
> Adding a call to wc_reload to jk_status.c (just like mount file reloading)
> sometimes crashes apache process.
>
> http://huksley.sdot.ru/wp-content/uploads/2010/09/mod_jk-1.2.30-reload-workers-patch2.zip
>
> With kindest personal regards,
> Ruslan Gainutdinov
> <ru...@gmail.com>
>
> On Tue, Sep 7, 2010 at 1:54 AM, Rainer Jung<ra...@kippdata.de> wrote:
>> On 06.09.2010 22:04, Ruslan Gainutdinov wrote:
>>>
>>> Thanks for info, I`ll check it out tomorrow.
>>> AFAIK, on windows, it is not shared memory but some sort of pool.
>>> Isn`t it dynamically grows when needed?
>>
>> No. Until version 1.2.26 you had to configure a fixed max size, from which
>> the needed memory was allocated, but we didn't give you any indication about
>> how much you would need.
>>
>> Starting with 1.2.27 we check how many load balancer and ajp workers are
>> configured and allocate shared memory (or plain memory on Windows) of
>> exactly the needed size.
>>
>> As Mladen says: you can still set a size explicitely using JkShmSize, which
>> should disable the automatic calculation. You will get a warning though,
>> that you don't need to set JkShmSize (that's true for the current
>> functionality in mod_jk).
>>
>> Regards,
>>
>> Rainer
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Tomcat Connectors - Dynamic reloading of workers
Posted by Ruslan Gainutdinov <ru...@gmail.com>.
Thanks, JkShmSize helps.
However this solution does not seems to be flexible enough. No way to
tell how much memory needed beforehand.
Is there any way to reinit shared memory?
Seems to be a lot of such code on apache22 connector side...
Haven`t looked into solution to reserve space by adding disabled workers.
I believe it will be complicated, because it is not known whether
actual workers will be simple nodes or lbcluster.
I`ve made another patch.
Added JkWorkersFileReload apache setting, by default disabled.
It works if using JkWatchdogInterval only.
Adding a call to wc_reload to jk_status.c (just like mount file reloading)
sometimes crashes apache process.
http://huksley.sdot.ru/wp-content/uploads/2010/09/mod_jk-1.2.30-reload-workers-patch2.zip
With kindest personal regards,
Ruslan Gainutdinov
<ru...@gmail.com>
On Tue, Sep 7, 2010 at 1:54 AM, Rainer Jung <ra...@kippdata.de> wrote:
> On 06.09.2010 22:04, Ruslan Gainutdinov wrote:
>>
>> Thanks for info, I`ll check it out tomorrow.
>> AFAIK, on windows, it is not shared memory but some sort of pool.
>> Isn`t it dynamically grows when needed?
>
> No. Until version 1.2.26 you had to configure a fixed max size, from which
> the needed memory was allocated, but we didn't give you any indication about
> how much you would need.
>
> Starting with 1.2.27 we check how many load balancer and ajp workers are
> configured and allocate shared memory (or plain memory on Windows) of
> exactly the needed size.
>
> As Mladen says: you can still set a size explicitely using JkShmSize, which
> should disable the automatic calculation. You will get a warning though,
> that you don't need to set JkShmSize (that's true for the current
> functionality in mod_jk).
>
> Regards,
>
> Rainer
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Tomcat Connectors - Dynamic reloading of workers
Posted by Rainer Jung <ra...@kippdata.de>.
On 06.09.2010 22:04, Ruslan Gainutdinov wrote:
> Thanks for info, I`ll check it out tomorrow.
> AFAIK, on windows, it is not shared memory but some sort of pool.
> Isn`t it dynamically grows when needed?
No. Until version 1.2.26 you had to configure a fixed max size, from
which the needed memory was allocated, but we didn't give you any
indication about how much you would need.
Starting with 1.2.27 we check how many load balancer and ajp workers are
configured and allocate shared memory (or plain memory on Windows) of
exactly the needed size.
As Mladen says: you can still set a size explicitely using JkShmSize,
which should disable the automatic calculation. You will get a warning
though, that you don't need to set JkShmSize (that's true for the
current functionality in mod_jk).
Regards,
Rainer
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Tomcat Connectors - Dynamic reloading of workers
Posted by Ruslan Gainutdinov <ru...@gmail.com>.
Thanks for info, I`ll check it out tomorrow.
AFAIK, on windows, it is not shared memory but some sort of pool.
Isn`t it dynamically grows when needed?
I am trying to create dynamic discovery of workers.
I know there is mod_cluster, however, it is not very stable in Tomcat.
And it does not give me direct control on which web server, which
node/cluster application I want to see.
The only thing missing from mod_jk is reloadable workers file (mounts
already reloaded).
If done properly and well tested, IMHO, reloadable workers file will
be good for everyone.
For example, right now, timeouts or activation can be either set at
startup time
or modified through jkmanager interface. However, when modified
through the jkmanager,
the same settings must be duplicated into workers file - this is
tedious and error-prone.
With kindest personal regards,
Ruslan Gainutdinov
<ru...@gmail.com>
On Mon, Sep 6, 2010 at 9:34 PM, Mladen Turk <mt...@apache.org> wrote:
> On 09/06/2010 05:54 PM, Ruslan Gainutdinov wrote:
>>
>> Hello!
>>
>>
>> this error is particularly confusing and afaik, occurs
>> in call to jk_shm_alloc_ajp_worker, which in turn fails in
>> jk_shm.c:768 - jk_shm_alloc(p, JK_SHM_AJP_WORKER_SIZE);
>>
>> i wonder,
>> maybe shared memory/or pool is readonly after initial setup phase?
>> is there other things I should watch for?
>>
>
> You have run out of shared memory.
> By default jk will create shm of size which is
> enough to store the initial size of workers. If you
> wish to add more, you will have to set JkShmSize
> to some value greater then initial number of workers
> plus some space that could be used for dynamic ones.
>
> However the best would be to create a number
> of disabled workers, and then fill them up with
> the runtime data, but then it's already there and
> usable.
>
> What are you trying to accomplish?
>
>
>
> Regards
> --
> ^TM
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Tomcat Connectors - Dynamic reloading of workers
Posted by Mladen Turk <mt...@apache.org>.
On 09/06/2010 05:54 PM, Ruslan Gainutdinov wrote:
> Hello!
>
>
> this error is particularly confusing and afaik, occurs
> in call to jk_shm_alloc_ajp_worker, which in turn fails in
> jk_shm.c:768 - jk_shm_alloc(p, JK_SHM_AJP_WORKER_SIZE);
>
> i wonder,
> maybe shared memory/or pool is readonly after initial setup phase?
> is there other things I should watch for?
>
You have run out of shared memory.
By default jk will create shm of size which is
enough to store the initial size of workers. If you
wish to add more, you will have to set JkShmSize
to some value greater then initial number of workers
plus some space that could be used for dynamic ones.
However the best would be to create a number
of disabled workers, and then fill them up with
the runtime data, but then it's already there and
usable.
What are you trying to accomplish?
Regards
--
^TM
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org