You are viewing a plain text version of this content. The canonical link for it is here.
Posted to java-dev@axis.apache.org by Aleksander Slominski <as...@cs.indiana.edu> on 2004/06/14 18:34:49 UTC
Re: more on AXIS-Java/C++ server side performance
Susantha Kumara wrote:
>Hi Alek
>
>You are doing a great work...!
>
>Which parser did you use ?. Xerces or Expat.
>
expat.
>Last time we did
>performance tests we found that Axis C++ is faster with Expat. 3 times
>faster with Expat than Xerces.
>
>
ha tis ehat i expected as expat is supposed ot be the fastest XML parser
on C.
i will post more results and a graph soon.
thanks,
alek
>>-----Original Message-----
>>From: Aleksander Slominski [mailto:aslom@cs.indiana.edu]
>>Sent: Saturday, June 12, 2004 5:34 PM
>>To: Apache AXIS C Developers List; 'axis-dev@ws.apache.org'
>>Subject: more on AXIS-Java/C++ server side performance
>>
>>Davanum Srinivas wrote:
>>
>>
>>
>>>Alek,
>>>
>>>Where is the sample code for your XSOAP server-side code? does it
>>>create a java object from the soap request?
>>>
>>>
>>>
>>hi dims,
>>
>>first let me emphasize that i do not look on this benchmark as only to
>>indicate performance but also a tool
>>
>>now that i said that i think i can demonstrate usefulness of such
>>
>>
>tests:
>
>
>>to my big surprise after running tests i can see that AXIS-C++ is two
>>times slower than AXIS-Java (in this sense the test work to help
>>
>>
>detect
>
>
>>possible huge performance bottlenecks may ultimately lead to much
>>
>>
>better
>
>
>>implementation).
>>
>>anyway this is really strange - i would expect C++ to be faster :)
>>
>>below is everything you need to reproduce test results yourself and
>>maybe somebody will have an idea what is going on with AXIS-C++ ...
>>
>>the test driver (client side) is still not fully finished but here is
>>working version:
>>http://www.extreme.indiana.edu/~aslom/bnp/wsperf/driver/
>>
>>after downloading do source classpath.sh (or classpath.bat on windows)
>>to set CLASSPATH
>>
>>run self test:
>>java -Dserver.name=SELF_TEST soap_bench.BenchDriver
>>
>>
>http://localhost:3434
>
>
>>and here is full description of command line:
>>
>>java -Dserver.name=SERVICE_NAME soap_bench.Bench URL total
>>{rsea}{bdisva} [arraySize][,arraySize][,...]
>>
>>parameters in order:
>>1. URL of service
>>2. total number of elements to send (default 10K)
>>3. specification of what to send : two letters
>> [rse] means receive, send, or echo (a == all)
>> [bdisva] means base64, double, int, string, void (only applies to
>>echo), a == all methods;
>>4. list of arraySizes (optional for void) - default to 10
>>and you can set test metadata such as -Dmachine.name=...
>>-Dserver.name=... (it is included in tabulated results good for
>>spreadsheet input and to draw graphs)
>>
>>those are optional system properties:
>>-Dsmoke_test - will show you what messages were received and check
>>
>>
>that
>
>
>>all works fine
>>-DAXISCPP - will add SOAPAction header and xsi:type for BASE64 (so
>>AXIS-C++ is happy)
>>-Dlog=trace.xsul.http.client:ALL -will show all messages and HTTP
>>headers (good for debugging)
>>-Dlog=:ALL - if you are doing desperate debugging
>>
>>use smoke test to get code to work with driver - it will just send 3
>>messages and will print what it received - combine it with -Dlog=...
>>
>>
>to
>
>
>>get more fine grained information on what is happening and if/when
>>driver is failing.
>>
>>the way i would do to get test driver to work with a SOAP service is
>>
>>
>to
>
>
>>it in multiple esteps and to get first to work different echo methods
>>
>>a) echoVoid
>>
>>java -Dsmoke_test -Dserver.name=FOO soap_bench.BenchClient <URL> 200
>>
>>
>ev
>
>
>>10,100
>>
>>echoInts
>>
>>java -Dsmoke_test -Dserver.name=FOO soap_bench.BenchClient <URL> 200
>>
>>
>ei
>
>
>>10,100
>>
>>echoDoubles
>>
>>java -Dsmoke_test -Dserver.name=FOO soap_bench.BenchClient <URL> 200
>>
>>
>ed
>
>
>>10,100
>>
>>echoStrings
>>
>>java -Dsmoke_test -Dserver.name=FOO soap_bench.BenchClient <URL> 200
>>
>>
>es
>
>
>>10,100
>>
>>echoBytes (as Base64)
>>
>>java -Dsmoke_test -Dserver.name=FOO soap_bench.BenchClient <URL> 200
>>
>>
>eb
>
>
>>10,100
>>
>>b) all echo methods
>>
>>java -Dsmoke_test -Dserver.name=FOO soap_bench.BenchClient <URL> 200
>>
>>
>ea
>
>
>>10,100
>>
>>c) all methods
>>
>>java -Dsmoke_test -Dserver.name=FOO soap_bench.BenchClient <URL> 200
>>
>>
>aa
>
>
>>10,100
>>
>>and then run real tests
>>
>>----------------------
>>For AXIS-Java
>>
>>get code from
>>
>>
>http://www.extreme.indiana.edu/~aslom/bnp/wsperf/axis_java/
>
>
>>and deploy benchmark service in tomcat - to do it just modify
>>build.properties and do ant all
>>
>>run test
>>java -Dmachine.name=YOUR_MACHINE_NAME -Dserver.name=AXIS_1_2_CVS
>>soap_bench.BenchClient http://server:8080/axis/services/Benchmark1
>>200000 aa 10,1000,10000
>>
>>----------------------
>>For AXIS-C++
>>
>>get code from
>>
>>
>http://www.extreme.indiana.edu/~aslom/bnp/wsperf/axis_cpp/
>
>
>>copy soap_bench/ to samples/server (and client respectively)
>>modify top level Makefile.am to include soap_bench (see example)
>>rebuilt .so library
>>add to list of deployed services (see deploy.wsdd in soap_bench)
>>restart apache server
>>
>>run test (make sure -DAXISCPP parameter is included!!!!!)
>>java -DAXISCPP -Dmachine.name=YOUR_MACHINE_NAME
>>-Dserver.name=AXISCPP_1_0_CVS
>>soap_bench.BenchClient http://server:8080/axis/Benchmark1 200000 ea
>>10,1000,10000
>>
>>NOTE: 'aa' will not work as all send tests do not work (and i do not
>>know why). instead try ea (echo tests) and sa (send tests) - if you
>>
>>
>want
>
>
>>to see what is not working use sa and enable logging (-Dlog=:ALL)
>>
>>--------------------------
>>For XSOAP4
>>
>>start service: java soap_bench.BenchServer 8077
>>
>>run test driver:
>>java -Dmachine.name=YOUR_MACHINE_NAME -Dserver.name=XSOAP4
>>soap_bench.BenchClient http://server:8077 200000 aa 10,1000,10000
>>
>>----------------------
>>
>>comments are welcome especially concerning performance differences
>>between Java and C++ and how to get send tests to work with AXIS
>>
>>
>C++...
>
>
>>thanks,
>>
>>alek
>>
>>
>>
>>>On Thu, 03 Jun 2004 12:58:41 -0500, Aleksander Slominski
>>><as...@cs.indiana.edu> wrote:
>>>
>>>
>>>
>>>
>>>>Davanum Srinivas wrote:
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>>You need to turn streaming on the client-side as well. see
>>>>>samples/perf in latest cvs.
>>>>>
>>>>>
>>>>>
>>>>>
>>binding._setProperty(org.apache.axis.client.Call.STREAMING_PROPERTY,
>>
>>
>>>>>Boolean.TRUE);
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>i am interested in testing server side performance (as server side
>>>>
>>>>
>is
>
>
>>>>typically the bottleneck) so i use the same client to access all
>>>>services to measure only factors that are effecting server side
>>>>performance (CPU, memory footprint, scalability).
>>>>
>>>>hopefully soon i will have AXIS-C++ numbers - i am really close now
>>>>
>>>>
>to
>
>
>>>>get them after few weeks of struggling with C++ (Java configuration
>>>>
>>>>
>and
>
>
>>>>build tools are much easier even when dealing with JAR hell ...)
>>>>
>>>>so it should be interesting to see how AXIS-Java and AXIS-C++
>>>>
>>>>
>compare.
>
>
>>>>however i am also *very* interested in very long running web
>>>>
>>>>
>services so
>
>
>>>>Peter post about _huge_ memory leaks that are easily reproducible
>>>>
>>>>
>looked
>
>
>>>>very interesting to me :) is it a property of data graph he send or
>>>>
>>>>
>is
>
>
>>>>it just a bug ...
>>>>
>>>>thanks,
>>>>
>>>>alek
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>>-- dims
>>>>>
>>>>>On Thu, 03 Jun 2004 12:27:18 -0500, Aleksander Slominski
>>>>><as...@cs.indiana.edu> wrote:
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>>Peter,
>>>>>>
>>>>>>please post more details how to reproduce the memory leak and i
>>>>>>
>>>>>>
>will
>
>
>>add
>>
>>
>>>>>>to what i have for WS/SOAP performance tests at:
>>>>>>http://www.extreme.indiana.edu/~aslom/bnp/wsperf/
>>>>>>
>>>>>>thanks,
>>>>>>
>>>>>>alek
>>>>>>
>>>>>>Peter Molettiere wrote:
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>>We're currently looking into a massive memory leak in axis, that
>>>>>>>
>>>>>>>
>no
>
>
>>>>>>>one here seems willing to admit. We're close to having a minimal
>>>>>>>
>>>>>>>
>>test
>>
>>
>>>>>>>case which demonstrates the leak we're seeing. I don't think
>>>>>>>
>>>>>>>
>it's
>
>
>>the
>>
>>
>>>>>>>same one you're seeing however, as our leak happens 1) when the
>>>>>>>server is sitting doing nothing (but only after axis has been
>>>>>>>loaded), and 2) every time we serialize a large message. The
>>>>>>>suggestions on the list about switching to using attachments
>>>>>>>
>>>>>>>
>doesn't
>
>
>>>>>>>fix the leak sadly. And setting a larger heap size only delays
>>>>>>>
>>>>>>>
>the
>
>
>>>>>>>inevitable Out Of Memory Exception.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>>From our testing, it looks like it takes on the order of 700MB to
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>>serialize a 30MB java object graph into a 3MB SOAP stream.
>>>>>>>
>>>>>>>I've posted about this before, but no one has responded.
>>>>>>>
>>>>>>>
>Hopefully
>
>
>>>>>>>when we post the test case someone will actually take a look at
>>>>>>>
>>>>>>>
>it.
>
>
>>>>>>>There are other threads with test cases which currently have
>>>>>>>
>>>>>>>
>"we're
>
>
>>>>>>>looking at this now" as the only response, but as they are all
>>>>>>>
>>>>>>>
>>pretty
>>
>>
>>>>>>>old threads, my guess is that this problem either isn't believed
>>>>>>>
>>>>>>>
>or
>
>
>>>>>>>is being swept under the carpet. Maybe we're the only people
>>>>>>>
>>>>>>>
>using
>
>
>>>>>>>axis for large messages.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>--
>>>>>>>
>>>>>>>On Jun 3, 2004, at 9:38 AM, jira@apache.org wrote:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>>The following comment has been added to this issue:
>>>>>>>>
>>>>>>>> Author: Jennifer Jackson
>>>>>>>> Created: Thu, 3 Jun 2004 9:37 AM
>>>>>>>> Body:
>>>>>>>>Has anyone looked into this? I haven't used Axis because of
>>>>>>>>
>>>>>>>>
>this
>
>
>>>>>>>>problem and doing a quick search it seems others have come
>>>>>>>>
>>>>>>>>
>across
>
>
>>>>>>>>memory leaks also. I narrowed it down to the Call object not
>>>>>>>>cleaning out its member vars (the whole Deserialization
>>>>>>>>
>>>>>>>>
>process).
>
>
>>>>>>>>If you do that the only thing stuck in memory is the call
>>>>>>>>
>>>>>>>>
>object
>
>
>>>>>>>>since its part of the ThreadLocal object.
>>>>>>>>
>>>>>>>>
>>>>>>>------------------------------------------------------------------
>>>>>>>
>>>>>>>
>--
>
>
>>-
>>
>>
>>>>>>>>View this comment:
>>>>>>>> http://issues.apache.org/jira/browse/AXIS-935?
>>>>>>>>page=comments#action_35890
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>------------------------------------------------------------------
>>>>>>>
>>>>>>>
>--
>
>
>>-
>>
>>
>>>>>>>>View the issue:
>>>>>>>>http://issues.apache.org/jira/browse/AXIS-935
>>>>>>>>
>>>>>>>>Here is an overview of the issue:
>>>>>>>>
>>>>>>>>
>>>>>>>------------------------------------------------------------------
>>>>>>>
>>>>>>>
>--
>
>
>>-
>>
>>
>>>>>>>> Key: AXIS-935
>>>>>>>> Summary: Using ThreadLocal Call objects in a servlet memory
>>>>>>>>
>>>>>>>>
>leak
>
>
>>>>>>>>both on client and server side
>>>>>>>> Type: Bug
>>>>>>>>
>>>>>>>> Status: Open
>>>>>>>>
>>>>>>>> Project: Axis
>>>>>>>>Components:
>>>>>>>> Basic Architecture
>>>>>>>> Versions:
>>>>>>>> 1.1rc2
>>>>>>>>
>>>>>>>> Assignee: Axis Developers Mailing List
>>>>>>>> Reporter: Jennifer Jackson
>>>>>>>>
>>>>>>>> Created: Mon, 30 Jun 2003 12:12 PM
>>>>>>>> Updated: Thu, 3 Jun 2004 9:37 AM
>>>>>>>>Environment: Operating System: All
>>>>>>>>Platform: All
>>>>>>>>
>>>>>>>>Description:
>>>>>>>>I discovered using Axis in tomcat causes a memory leak.
>>>>>>>>
>>>>>>>>It seems using threadlocal in the classes causes a memory leak
>>>>>>>>
>>>>>>>>
>for
>
>
>>>>>>>>each Call
>>>>>>>>that is created. Threadlocal sticks around until the calling
>>>>>>>>
>>>>>>>>
>thread
>
>
>>>>>>>>dies, and
>>>>>>>>since tomcat keeps threads alive to answer new calls, the
>>>>>>>>threadlocal data
>>>>>>>>never goes away. The Call object gets stuck in the threadlocal
>>>>>>>>data, which
>>>>>>>>contains the entire soap response and all objects. This is a
>>>>>>>>
>>>>>>>>
>BIG
>
>
>>>>>>>>memory leak.
>>>>>>>>I initially cleared it by forcing all Call objects to clean
>>>>>>>>
>>>>>>>>
>their
>
>
>>>>>>>>member vars
>>>>>>>>after I was done with a call, then I realized the source was
>>>>>>>>
>>>>>>>>
>from
>
>
>>>>>>>>the thread
>>>>>>>>local class. Here is some info I found about threadlocal:
>>>>>>>>
>>>>>>>>This is stated in the Sun Javadocs for ThreadLocal:
>>>>>>>>
>>>>>>>> Each thread holds an implicit reference to its copy of a
>>>>>>>> ThreadLocal as long as the thread is alive and the
>>>>>>>> ThreadLocal object is accessible; after a thread goes
>>>>>>>> away, all of its copies of ThreadLocal variables are
>>>>>>>> subject to garbage collection (unless other references
>>>>>>>> to these copies exist).
>>>>>>>>
>>>>>>>>So, this means that ANY APPLICATION that uses PreparedStatements
>>>>>>>>
>>>>>>>>
>in
>
>
>>>>>>>>a thread
>>>>>>>>that 1) either does a lot of PreparedStatements or 2) never dies
>>>>>>>>(i.e., a
>>>>>>>>main thread) will ALWAYS eventually have an out of memory error.
>>>>>>>>Simply put,
>>>>>>>>this is a MEMORY LEAK. I imagine that the leak is very small,
>>>>>>>>
>>>>>>>>
>the
>
>
>>>>>>>>ThreadLocal object only contains one member variable, maybe 64
>>>>>>>>
>>>>>>>>
>bytes
>
>
>>>>>>>>or less
>>>>>>>>(depending on the VM implementation). So, our 60,000
>>>>>>>>PreparedStatements of 2
>>>>>>>>ThreadLocals each times 64 bytes (my wild guess) is 7.5MB.
>>>>>>>>
>>>>>>>>Ideas? I've never used threadlocal myself so this is new to me.
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>------------------------------------------------------------------
>>>>>>>
>>>>>>>
>--
>
>
>>-
>>
>>
>>>>>>>>JIRA INFORMATION:
>>>>>>>>This message is automatically generated by JIRA.
>>>>>>>>
>>>>>>>>If you think it was sent incorrectly contact one of the
>>>>>>>>
>>>>>>>>
>>administrators:
>>
>>
>>>>>>>> http://issues.apache.org/jira/secure/Administrators.jspa
>>>>>>>>
>>>>>>>>If you want more information on JIRA, or have a bug to report
>>>>>>>>
>>>>>>>>
>see:
>
>
>>>>>>>> http://www.atlassian.com/software/jira
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>--
>>>>>>The best way to predict the future is to invent it - Alan Kay
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>--
>>>>The best way to predict the future is to invent it - Alan Kay
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>--
>>The best way to predict the future is to invent it - Alan Kay
>>
>>
>
>
>
>
--
The best way to predict the future is to invent it - Alan Kay