[C2] sitemap creation threading issue ?

[Marcus Crafter <cr...@fztig938.bank.dresdner.net>]

Hi *,

	Hope all is well.

	Michael and I have spent the day testing our C2 application with
	LoadRunner and have potentially uncovered a threading problem during
	sitemap creation.

	We're not experts with the code but from our understanding the
	following is happening, please let us know if we are right/wrong:

	There seems to be a problem with the getHandler() method, located in
	the sitemap Manager class (line 154). getHandler() attempts to access a
	sitemap handler object for each request for processing. If the handler
	object is not available it creates one, causing the sitemap to be
	generated.

	We've noticed under load, that many handler objects are created for
	the same sitemap. This is because getHandler() does not protect the
	following lines:

		Handler sitemapHandler = (Handler)sitemaps.get(source);

	and

		sitemaps.put(source, sitemapHandler);

	as a critical area.

	If multiple concurrent threads pass through getHandler() which are
	requests for resources from the same sitemap, the first line
	above will return null multiple times causing the same sitemap to be
	compiled several times, each by individual Handler objects.
	
	This happens because sitemaps.put() executes after each sitemap handler
	object is created (which can take time for large sitemaps), and cannot
	prevent other incoming threads from waiting until it adds the newly
	created handler object into the 'sitemaps' hashmap.

	When we synchronized the getHandler method to protect the
	getting/setting of the sitemaps hashmap, we saw that the sitemap
	handler object was created only once, and that the application
	performed much better under load. Previously the same sitemap handler
	object was created as many times as we had simultaneous threads
	make requests.

	Attached is a diff of the change we made. There might be a better
	solution as the Handler class seems to be built to handle this, it's
	just that the allocation of a new Handler objects per sitemap, defeats
	it's internal multi-thread logic.

	Any comments/thoughts/suggestions ?

	Cheers,

	Marcus

-- 
        .....
     ,,$$$$$$$$$,      Marcus Crafter
    ;$'      '$$$$:    Computer Systems Engineer
    $:         $$$$:   Open Software Associates GmbH
     $       o_)$$$:   82-84 Mainzer Landstrasse
     ;$,    _/\ &&:'   60327 Frankfurt Germany
       '     /( &&&
           \_&&&&'     Email : Marcus.Crafter@osa.de
          &&&&.        Business Hours : +49 69 9757 200
    &&&&&&&:

AW: [C2] sitemap creation threading issue ?

[Carsten Ziegeler <cz...@sundn.de>]

Hi Marcus,

thanks for your patch. I applied it, please cross check :-)

Carsten

> -----Ursprungliche Nachricht-----
> Von: Marcus Crafter [mailto:crafterm@fztig938.bank.dresdner.net]
> Gesendet: Samstag, 18. August 2001 22:41
> An: Cocoon Developers Mailing List
> Betreff: [C2] sitemap creation threading issue ?
> 
> 
> Hi *,
> 
> 	Hope all is well.
> 
> 	Michael and I have spent the day testing our C2 application with
> 	LoadRunner and have potentially uncovered a threading problem during
> 	sitemap creation.
> 
> 	We're not experts with the code but from our understanding the
> 	following is happening, please let us know if we are right/wrong:
> 
> 	There seems to be a problem with the getHandler() method, located in
> 	the sitemap Manager class (line 154). getHandler() attempts 
> to access a
> 	sitemap handler object for each request for processing. If 
> the handler
> 	object is not available it creates one, causing the sitemap to be
> 	generated.
> 
> 	We've noticed under load, that many handler objects are created for
> 	the same sitemap. This is because getHandler() does not protect the
> 	following lines:
> 
> 		Handler sitemapHandler = (Handler)sitemaps.get(source);
> 
> 	and
> 
> 		sitemaps.put(source, sitemapHandler);
> 
> 	as a critical area.
> 
> 	If multiple concurrent threads pass through getHandler() which are
> 	requests for resources from the same sitemap, the first line
> 	above will return null multiple times causing the same sitemap to be
> 	compiled several times, each by individual Handler objects.
> 	
> 	This happens because sitemaps.put() executes after each 
> sitemap handler
> 	object is created (which can take time for large sitemaps), 
> and cannot
> 	prevent other incoming threads from waiting until it adds the newly
> 	created handler object into the 'sitemaps' hashmap.
> 
> 	When we synchronized the getHandler method to protect the
> 	getting/setting of the sitemaps hashmap, we saw that the sitemap
> 	handler object was created only once, and that the application
> 	performed much better under load. Previously the same 
> sitemap handler
> 	object was created as many times as we had simultaneous threads
> 	make requests.
> 
> 	Attached is a diff of the change we made. There might be a better
> 	solution as the Handler class seems to be built to handle this, it's
> 	just that the allocation of a new Handler objects per 
> sitemap, defeats
> 	it's internal multi-thread logic.
> 
> 	Any comments/thoughts/suggestions ?
> 
> 	Cheers,
> 
> 	Marcus
> 
> -- 
>         .....
>      ,,$$$$$$$$$,      Marcus Crafter
>     ;$'      '$$$$:    Computer Systems Engineer
>     $:         $$$$:   Open Software Associates GmbH
>      $       o_)$$$:   82-84 Mainzer Landstrasse
>      ;$,    _/\ &&:'   60327 Frankfurt Germany
>        '     /( &&&
>            \_&&&&'     Email : Marcus.Crafter@osa.de
>           &&&&.        Business Hours : +49 69 9757 200
>     &&&&&&&:
> 

---------------------------------------------------------------------
To unsubscribe, e-mail: cocoon-dev-unsubscribe@xml.apache.org
For additional commands, email: cocoon-dev-help@xml.apache.org