jUDDI load testing update: Search testing

[Jeremi Thebeau <j....@xceptance.de>]

Search performance testing

A series of test were run to determine how quickly searches for 
businesses and business services were executed given the same number of 
services in a node while varying the number of business under which the 
services were published.

This was done by repeatedly filling an empty node with the following 
numbers of businesses and services/business:

# of Businesses
	# of Services/Business
100,000
	1
10,000
	10
1,000
	100
100
	1000

We see that the total number of services is consistently 100,000 for 
every instance of the node. Each business and service was published 
under a lone unique names so that every search only has one result. Load 
test repeating the TFindServiceBM scenario was done to measure mean 
response times. The TFindServiceBM scenario searches the node by name 
for a random service. The name is supplied by a previously generated 
file containing the names of all services. The mean times for each ~ 
30min load test is illustrated in the table below (the tests were run 
twice on each node instance):

Arrival rate \ businesses 	100,000 	10,000 	1,000 	100
100,000 	247ms 
<http://xlt.xceptance.de/download/results/juddi/20090821-130234/> 	250ms 
<http://xlt.xceptance.de/download/results/juddi/20090819-183924/> 	327ms 
<http://xlt.xceptance.de/download/results/juddi/20090821-093748/> 	231ms 
<http://xlt.xceptance.de/download/results/juddi/20090821-164739/>
100,000 	186ms 
<http://xlt.xceptance.de/download/results/juddi/20090825-131253/> 	242ms 
<http://xlt.xceptance.de/download/results/juddi/20090824-181227/> 	243ms 
<http://xlt.xceptance.de/download/results/juddi/20090821-093748/> 	238ms 
<http://xlt.xceptance.de/download/results/juddi/20090824-102539/>


Clicking on any of the mean times above will link to the XLT report of 
run. We can see that the resulting mean times are almost always ~ 245ms 
except for a couple of outliers. This would suggest that searches for 
services are independent of how they are distributed over businesses.

As for find_business operations, a 100,000 transaction/hour constant 
arrival rate could not be maintained for all the different distributions 
of services. Instead an arrival rate of 6,000 t/h was used since this 
seemed to be close to highest load the slowest configuration (100 
businesses) could take. The table below shows the mean times for the 
TFindBusinessBM runs:

Arrival rate \ businesses 	100,000 	10,000 	1,000 	100
6,000 	64ms 
<http://xlt.xceptance.de/download/results/juddi/20090825-122527/> 	27ms 
<http://xlt.xceptance.de/download/results/juddi/20090824-173745/> 	91ms 
<http://xlt.xceptance.de/download/results/juddi/20090824-160655/> 	869ms 
<http://xlt.xceptance.de/download/results/juddi/20090824-121054/>


TFindBusinessBM finds and returns a random business much in the same way 
as TFindServiceBM, by searching for a business name supplied by a 
previously generated file containing the names of all businesses in the 
node. Here we see an increase in mean response time which seams to be 
related to the increase in services /business. This is probably cause by 
the lengthening returned  SOAP message that includes not only details 
off the business but also a list of all the services associated with it. 
The first value of the node with 100,000 businesses with just one 
service each, does not seem to support this trend but that may be 
because of the relatively large number of businesses that must be 
searched through. More testing would be needed to determine this.

Here are trend reports showing performance differences between service 
searches 
<http://xlt.xceptance.de/download/results/juddi/trendreport-20090826-131138/requests.html#request-summary> 
and businesses searches 
<http://xlt.xceptance.de/download/results/juddi/trendreport-20090826-130737/>.

System setup:

This 
<http://people.apache.org/repo/m2-snapshot-repository/org/apache/juddi/juddi-portal-bundle/3.0.0.SNAPSHOT/juddi-portal-bundle-3.0.0.20090723.201427-7.zip> 
release of jUDDI ran on the following:

Processor: Intel(R) Core(TM)2 CPU     6400  @ 2.13GHz
RAM: 2011 MB
OS: Ubuntu 9.04, linux kernel 2.6.28-13-generic

This 
<http://xlt.xceptance.de/download/results/juddi/testsuites/20090821-juddiTestSuite.zip> 
test suite was used to run the above test.


Please let me know if there are any specific scenarios you, the jUDDI 
developers, would like to see.

Jeremi