Profiling DirServer with TPTP

[Ole Ersoy <ol...@yahoo.com>]

Hey Guys,

I tried playing with TPTP a little.

Amazing tool.  It produces UML2 class and thread
interaction diagrams that highlight in red application
hot spots.

I hit some rough spots just going by the TPTP
tutorial, and I'll mention those + solution further
down in the email.

So now I'm trying to think what the right strategy is
for testing the DS?

So suppose I created a JNDI client that created say 10
threads that all hammered DS with lookup requests,
should that produce the concurrency issue?

Now when I did the TPTP tutorial I profiled the
Product example...So I profiled a Java Application. 
With the approach above I would be profiling a server
using a client.

That's a little trickier.  I'd most likely have to
write a Probe (Which the Eclipse help is very good
about explaining under "Monitoring and Analyzing
Performance > Collecting Runtime Data With User
Defined Probes") and then use it to collect data on DS
while the client is running.

The simpler way though would be to run DS as a
application and just collect performance data that
way.

This would require me to check out the DS code, and
then figure out how to run methods on it directly from
a client sitting in the same project directory.

I don't think I would want to go through JNDI, since
that means I'll be tunneling through a lot of JNDI
method calls...assuming there's a more direct way of
achieving the usage pattern...?

In that case how would I write that client so that I'm
calling DS methods directly and which methods should
be profiled first?

OK - Here's some key notes from the TPTP experience.

The tutorial:

http://www.eclipse.org/articles/Article-TPTP-Profiling-Tool/tptpProfilingArticle.html

is very helpful, however there's no mention of THIS:

Here's the context:
You right click on the Product class
Profile As > Java Application

This brings up the "Edit Launch and Configuration
Properties"

Here you have to do what he says, BUT you also 
have to go to the 

Environment tab

and create the variable "PATH"

which needs to be set to the bin directory of the 

Raserver...

The Raserver is what collects data...you need to
install it as part of the TPTP installation...very
straight forward though.

So to get the tutorial working all I did was 

Install TPTP via the update site

Install the Raserver...unzipped and used the
getting_started.html install instructions in the root
install directory to configure and run...

And now it works...

If anyone has any questions or needs help with it I'll
be glad to do more research.  

Cheers,
- Ole


__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 

Re: Profiling DirServer with TPTP

[Ole Ersoy <ol...@yahoo.com>]

Hey Emmanuel,

Yeah I know - If only there were 48 hours in a day!
I can associate with the hardware need too - So far
and my old windows 2000 machine, which crashes
whenever there's a breeze.

I saw IBM is one of the customers of YourKit...looks
like they must have hit some rough spots in trying out
the tutorial as well :-).

Is there a central place where testing data can be put
at safehaus or should we just attach scenarios to
JIRA?

Cheers,
- Ole

--- Emmanuel Lecharny <el...@gmail.com> wrote:

> Ole Ersoy a écrit :
> 
> >Hey Guys,
> >  
> >
> Hi Ole,
> 
> ><snip/>
> >So now I'm trying to think what the right strategy
> is
> >for testing the DS?
> >  
> >
> Well, you have many.
> 
> >So suppose I created a JNDI client that created say
> 10
> >threads that all hammered DS with lookup requests,
> >should that produce the concurrency issue?
> >  
> >
> I think that the problem has been solved by trustin
> last week. It was 
> not really a concurrency problem, but pretty much a
> problem with thread 
> pooling : I finished one test with more that 5000
> threads ...
> 
> ><snip/>
> >I don't think I would want to go through JNDI,
> since
> >that means I'll be tunneling through a lot of JNDI
> >method calls...assuming there's a more direct way
> of
> >achieving the usage pattern...?
> >  
> >
> There is no problem with JNDI assuming your client
> is *not* on the same 
> computer than the server. From the server side point
> of view, JNDI is 
> just a wrapping API, which cost virtually nothing.
> 
> >In that case how would I write that client so that
> I'm
> >calling DS methods directly and which methods
> should
> >be profiled first?
> >  
> >
> I don't think that is a good idea. You will spend a
> lot of time for very 
> limited improvment. Except if you want to test
> specific parts of 
> ApacheDS like the backend, for instance, this is not
> really an approach 
> that will help a lot finding hot-spots.
> 
> ><snip/>
> >OK - Here's some key notes from the TPTP
> experience.
> >  
> >
> Thanks a lot for the TPTP feedback, it will be very
> usefull !!
> 
> FYI, we are using YourKit profiling tool to do some
> profiling. It has 
> given us some very valuable information about where
> are the hotspots. So 
> far, we know that Dn parsing is the major
> bottleneck. This is quite 
> surprising, because it's a pure CPU operation, and
> one could think that 
> backend operations might have been the main
> bottleneck. However, the 
> tests were biased :
> - first, we did it with very little amount of data
> in the backend, so 
> the chane are that all those data are cached
> - second, the test just grab always the same data,
> so now you can be 
> sure that data is cached !
> 
> It would be very interesting to build a real sample,
> with thousands of 
> data, and real tests (scenario) that we can
> reproduce. This is something 
> I have in mind for months, and I'm trying to build
> right now, but lack 
> of time kills me. We also need the infrastructure to
> run those tests (at 
> least three computers : a server and two clients).
> 
> I would be very please if we can gather our efforts
> to build these test 
> scenarii and do all the smoke tests needed to prove
> that ADS is solid, 
> fast - and furious ;). Of course, then, we will be
> able to work on the 
> hot-spot to improve ADS.
> 
> Emmanuel Lécharny
> 


__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 

Re: Profiling DirServer with TPTP

[Emmanuel Lecharny <el...@gmail.com>]

Ole Ersoy a écrit :

>Hey Guys,
>  
>
Hi Ole,

><snip/>
>So now I'm trying to think what the right strategy is
>for testing the DS?
>  
>
Well, you have many.

>So suppose I created a JNDI client that created say 10
>threads that all hammered DS with lookup requests,
>should that produce the concurrency issue?
>  
>
I think that the problem has been solved by trustin last week. It was 
not really a concurrency problem, but pretty much a problem with thread 
pooling : I finished one test with more that 5000 threads ...

><snip/>
>I don't think I would want to go through JNDI, since
>that means I'll be tunneling through a lot of JNDI
>method calls...assuming there's a more direct way of
>achieving the usage pattern...?
>  
>
There is no problem with JNDI assuming your client is *not* on the same 
computer than the server. From the server side point of view, JNDI is 
just a wrapping API, which cost virtually nothing.

>In that case how would I write that client so that I'm
>calling DS methods directly and which methods should
>be profiled first?
>  
>
I don't think that is a good idea. You will spend a lot of time for very 
limited improvment. Except if you want to test specific parts of 
ApacheDS like the backend, for instance, this is not really an approach 
that will help a lot finding hot-spots.

><snip/>
>OK - Here's some key notes from the TPTP experience.
>  
>
Thanks a lot for the TPTP feedback, it will be very usefull !!

FYI, we are using YourKit profiling tool to do some profiling. It has 
given us some very valuable information about where are the hotspots. So 
far, we know that Dn parsing is the major bottleneck. This is quite 
surprising, because it's a pure CPU operation, and one could think that 
backend operations might have been the main bottleneck. However, the 
tests were biased :
- first, we did it with very little amount of data in the backend, so 
the chane are that all those data are cached
- second, the test just grab always the same data, so now you can be 
sure that data is cached !

It would be very interesting to build a real sample, with thousands of 
data, and real tests (scenario) that we can reproduce. This is something 
I have in mind for months, and I'm trying to build right now, but lack 
of time kills me. We also need the infrastructure to run those tests (at 
least three computers : a server and two clients).

I would be very please if we can gather our efforts to build these test 
scenarii and do all the smoke tests needed to prove that ADS is solid, 
fast - and furious ;). Of course, then, we will be able to work on the 
hot-spot to improve ADS.

Emmanuel Lécharny

Re: Profiling DirServer with TPTP

[Ole Ersoy <ol...@yahoo.com>]

Hey Ersin,

Yeah - The embedding rocks!  

Thanks for the safehaus link and tips - I'll check it
out.

Great to hear that the thread pool issue was solved.

Cheers,
- Ole

--- Ersin Er <er...@gmail.com> wrote:

> Ole Ersoy wrote:
> > Hey Guys,
> >   
> Hi Ole,
> > I tried playing with TPTP a little.
> >
> > Amazing tool.  It produces UML2 class and thread
> > interaction diagrams that highlight in red
> application
> > hot spots.
> >   
> I had a glance at it. It really looks cool.
> > I hit some rough spots just going by the TPTP
> > tutorial, and I'll mention those + solution
> further
> > down in the email.
> >
> > So now I'm trying to think what the right strategy
> is
> > for testing the DS?
> >   
> I think there are two strategies which I'll mention
> below.
> > So suppose I created a JNDI client that created
> say 10
> > threads that all hammered DS with lookup requests,
> > should that produce the concurrency issue?
> >   
> I don't think that we have the concurrency issue any
> more.
> > Now when I did the TPTP tutorial I profiled the
> > Product example...So I profiled a Java
> Application. 
> > With the approach above I would be profiling a
> server
> > using a client.
> >
> > That's a little trickier.  I'd most likely have to
> > write a Probe (Which the Eclipse help is very good
> > about explaining under "Monitoring and Analyzing
> > Performance > Collecting Runtime Data With User
> > Defined Probes") and then use it to collect data
> on DS
> > while the client is running.
> >
> > The simpler way though would be to run DS as a
> > application and just collect performance data that
> > way.
> >
> > This would require me to check out the DS code,
> and
> > then figure out how to run methods on it directly
> from
> > a client sitting in the same project directory.
> >
> > I don't think I would want to go through JNDI,
> since
> > that means I'll be tunneling through a lot of JNDI
> > method calls...assuming there's a more direct way
> of
> > achieving the usage pattern...?
> >   
> "Direct way" of performing operations on ApacheDS is
> again JNDI. Because 
> ApacheDS core API is provided as JNDI. So
> programming to ApacheDS over 
> wire or embedding it are same, cool eh ? ;-)
> 
> So, to use the "direct way" which I mean embeddeding
> ApacheDS and making 
> the profiling within the embedder application
> "without going on the 
> wire", you just need to start ApacheDS within your
> application and do 
> some extensive bind/search/unbind operations on the
> JNDI Context which 
> is really the core. So when you right click on this
> Application in 
> Eclipse and select Profile As Java Application
> everything will go as in 
> the tutorial, I wish :)
> 
> This way requires setting the
> Context.INITIAL_CONTEXT_FACTORY as 
>
"org.apache.directory.server.core.jndi.CoreContextFactory"
> in 
> environment variables configuration as you can see
> in the example.
> 
> To have an idea about how to embed ApacheDS you may
> check out here:
>
https://svn.safehaus.org/repos/sandbox/apachecon-m2/simple/
> 
> The second way which is "the real way", "the on-wire
> way", which will 
> also show the network and related performance is to
> run (Profile As Java 
> Application) ApacheDS standalone with network
> support. And than you can 
> connect to it with JNDI over wire with a second
> application to do some 
> extensive bind/search/unbind (or whatever else)
> operations (with 
> possibly more than one thread if you want to see the
> concurrent 
> performance).
> 
> This way requires setting the
> Context.INITIAL_CONTEXT_FACTORY as 
>
"org.apache.directory.server.jndi.ServerContextFactory"
> in environment 
> variables configuration as you can see in the
> example. To do this you 
> may again embed ApacheDS within a simple application
> or you may start 
> ApacheDS within Eclipse with server.xml on your
> classpath where you can 
> configure which services will be started and which
> will not. For 
> example, you would not want to start Kerberos
> service probably.
> 
> The embedding+ldap service support example is here:
>
https://svn.safehaus.org/repos/sandbox/apachecon/ldap/
> > In that case how would I write that client so that
> I'm
> > calling DS methods directly and which methods
> should
> > be profiled first?
> >   
> I think the whole server can be tested at once and
> the most time taking 
> parts will be shown by Eclipse so there will be no
> need to go deep into 
> the server code.
> > OK - Here's some key notes from the TPTP
> experience.
> >
> > The tutorial:
> >
> >
>
http://www.eclipse.org/articles/Article-TPTP-Profiling-Tool/tptpProfilingArticle.html
> >
> > is very helpful, however there's no mention of
> THIS:
> >
> > Here's the context:
> > You right click on the Product class
> > Profile As > Java Application
> >
> > This brings up the "Edit Launch and Configuration
> > Properties"
> >
> > Here you have to do what he says, BUT you also 
> > have to go to the 
> >
> > Environment tab
> >
> > and create the variable "PATH"
> >
> > which needs to be set to the bin directory of the 
> >
> > Raserver...
> >
> > The Raserver is what collects data...you need to
> > install it as part of the TPTP installation...very
> > straight forward though.
> >
> > So to get the tutorial working all I did was 
> >
> > Install TPTP via the update site
> >
> > Install the Raserver...unzipped and used the
> > getting_started.html install instructions in the
> root
> > install directory to configure and run...
> >
> > And now it works...
> Sorry, I did not have time to try this but I'll do.
> > If anyone has any questions or needs help with it
> I'll
> > be glad to do more research.  
> >
> > Cheers,
> > - Ole
> HTH,
> 
> -- 
> Ersin
> 


__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 

Re: Profiling DirServer with TPTP

[Ersin Er <er...@gmail.com>]

Ole Ersoy wrote:
> Hey Guys,
>   
Hi Ole,
> I tried playing with TPTP a little.
>
> Amazing tool.  It produces UML2 class and thread
> interaction diagrams that highlight in red application
> hot spots.
>   
I had a glance at it. It really looks cool.
> I hit some rough spots just going by the TPTP
> tutorial, and I'll mention those + solution further
> down in the email.
>
> So now I'm trying to think what the right strategy is
> for testing the DS?
>   
I think there are two strategies which I'll mention below.
> So suppose I created a JNDI client that created say 10
> threads that all hammered DS with lookup requests,
> should that produce the concurrency issue?
>   
I don't think that we have the concurrency issue any more.
> Now when I did the TPTP tutorial I profiled the
> Product example...So I profiled a Java Application. 
> With the approach above I would be profiling a server
> using a client.
>
> That's a little trickier.  I'd most likely have to
> write a Probe (Which the Eclipse help is very good
> about explaining under "Monitoring and Analyzing
> Performance > Collecting Runtime Data With User
> Defined Probes") and then use it to collect data on DS
> while the client is running.
>
> The simpler way though would be to run DS as a
> application and just collect performance data that
> way.
>
> This would require me to check out the DS code, and
> then figure out how to run methods on it directly from
> a client sitting in the same project directory.
>
> I don't think I would want to go through JNDI, since
> that means I'll be tunneling through a lot of JNDI
> method calls...assuming there's a more direct way of
> achieving the usage pattern...?
>   
"Direct way" of performing operations on ApacheDS is again JNDI. Because 
ApacheDS core API is provided as JNDI. So programming to ApacheDS over 
wire or embedding it are same, cool eh ? ;-)

So, to use the "direct way" which I mean embeddeding ApacheDS and making 
the profiling within the embedder application "without going on the 
wire", you just need to start ApacheDS within your application and do 
some extensive bind/search/unbind operations on the JNDI Context which 
is really the core. So when you right click on this Application in 
Eclipse and select Profile As Java Application everything will go as in 
the tutorial, I wish :)

This way requires setting the Context.INITIAL_CONTEXT_FACTORY as 
"org.apache.directory.server.core.jndi.CoreContextFactory" in 
environment variables configuration as you can see in the example.

To have an idea about how to embed ApacheDS you may check out here:
https://svn.safehaus.org/repos/sandbox/apachecon-m2/simple/

The second way which is "the real way", "the on-wire way", which will 
also show the network and related performance is to run (Profile As Java 
Application) ApacheDS standalone with network support. And than you can 
connect to it with JNDI over wire with a second application to do some 
extensive bind/search/unbind (or whatever else) operations (with 
possibly more than one thread if you want to see the concurrent 
performance).

This way requires setting the Context.INITIAL_CONTEXT_FACTORY as 
"org.apache.directory.server.jndi.ServerContextFactory" in environment 
variables configuration as you can see in the example. To do this you 
may again embed ApacheDS within a simple application or you may start 
ApacheDS within Eclipse with server.xml on your classpath where you can 
configure which services will be started and which will not. For 
example, you would not want to start Kerberos service probably.

The embedding+ldap service support example is here:
https://svn.safehaus.org/repos/sandbox/apachecon/ldap/
> In that case how would I write that client so that I'm
> calling DS methods directly and which methods should
> be profiled first?
>   
I think the whole server can be tested at once and the most time taking 
parts will be shown by Eclipse so there will be no need to go deep into 
the server code.
> OK - Here's some key notes from the TPTP experience.
>
> The tutorial:
>
> http://www.eclipse.org/articles/Article-TPTP-Profiling-Tool/tptpProfilingArticle.html
>
> is very helpful, however there's no mention of THIS:
>
> Here's the context:
> You right click on the Product class
> Profile As > Java Application
>
> This brings up the "Edit Launch and Configuration
> Properties"
>
> Here you have to do what he says, BUT you also 
> have to go to the 
>
> Environment tab
>
> and create the variable "PATH"
>
> which needs to be set to the bin directory of the 
>
> Raserver...
>
> The Raserver is what collects data...you need to
> install it as part of the TPTP installation...very
> straight forward though.
>
> So to get the tutorial working all I did was 
>
> Install TPTP via the update site
>
> Install the Raserver...unzipped and used the
> getting_started.html install instructions in the root
> install directory to configure and run...
>
> And now it works...
Sorry, I did not have time to try this but I'll do.
> If anyone has any questions or needs help with it I'll
> be glad to do more research.  
>
> Cheers,
> - Ole
HTH,

-- 
Ersin