You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@directory.apache.org by Stefan Zoerner <st...@labeo.de> on 2005/08/18 19:13:40 UTC
Protocol Testsuite: Mission Statement (Proposal)
Hi all,
here is my first draft for a Mission statement for the test suite
subproject, Alex suggested. Any comments/corrections -- please note that
English is not my first language (and not my second, which is LDIF) --
are welcome!
Greetings from sunny Hamburg,
Stefan
---
Protocol Testsuite: Mission Statement (Draft)
The success of Apache Directory Server (as of any LDAP server) depends
heavily on interoperability. May it be used as a user repository of an
arbitrary J2EE application server (or web server, or mail server, or ...
You name it)? Can you plug it into your favorite mail tool as an address
book?
Thus the primary aim of this subproject is to provide a test suite which
is used to ensure that Apache Directory Server behaves like a 08/15
(i.e. an ordinary, bulk commodity) LDAP client would expect. Therefore
it concentrates on the functionality which is undisputable among
different LDAP vendors/providers. We run the test suite against major
directory solutions (commercial and Open Source), just to be sure that
we check common sense.
For a start JUnit and JNDI are used, there is no dependency on code
which is provided by the Apache Directory Project (we do black box
tests). While JUnit is beyond dispute, JNDI is not. The JNDI framework
is an abstraction for naming and directory services, and some LDAP
operations are not straight forward to perform with JNDI (e.g. compare,
abandon). But the option to create a test suite which does not depend on
an "explicit" LDAP library (like Netscape SDK) and the fact that a lot
of the Apache Directory Server users will choose JNDI for integration
lead us to the decision to start with it.
Later on this subproject may support the main development of the Apache
Directory Project with tests for topics which are not rock-solid
standardized (e.g. access control lists). Or performance tests. Or ... .
But as long as any client may complain about the behavior, we will
concentrate on basic operations. Client experience is key.
External links, if placed on the site:
Black box testing: http://en.wikipedia.org/wiki/Black_box_testing
JUnit: http://junit.org/
JNDI: http://java.sun.com/products/jndi/
Netscape SDK: http://www.mozilla.org/directory/javasdk.html
Re: Protocol Testsuite: Mission Statement (Proposal)
Posted by Chris Betts <ch...@pegacat.com>.
Hi Guys,
It sounds like you're well on your way already, so feel free to
ignore what's below :-).
The JXplorer project has a simple test suite based on LDIF
change files. It was written to test the functionality of JX, but of
course it also checks the server at the same time. It is a command
line tool that runs through an arbitrary LDIF change file, and
reports back on how it went. It's used to test the standard copy/
delete/add/modify operations, as well as some nasty stuff to do with
multi valued RDNs, binary attribute values and so forth.
It used to use junit, but we stopped using it because we wanted
to be able to run the JX client side tests stand alone, and these
were (for us) 'system tests'. But it could be fitted back into the
junit mold if we wanted - the trick though is that because it uses an
LDIF file rather than a series of JUnit like test methods, there
might need to be some adjustment.
Anyway, even though it shouldn't be, it keeps sneaking into the
standard JXplorer release, so it can in fact be used immediately with
the current 3.2b2 version of JX - go to the /jars directory, and type
>java -cp jxplorer.jar com.ca.commons.jndi.TestJNDIOps -h and it
will print out its usage parameters 'n stuff. I suspect we'd need to
work on it a bit for the use of ApacheDS, but it's a current, working
implementation similar to that used in the overnight build tests at CA.
It may be that there is room for multiple test tools to run in
an automated fashion - it sounds like what people are talking about
below is much more low level; this tool basically imitates a java
jndi client. It is convenient, since it just uses the standard LDIF
change file format and new tests simply involve adding a new snippet
of LDIF, but other tools may be able to more rigourously exercise the
full functionality of the Apache DS.
cheers,
Chris
P.S. The code is licensed under the 'CA open source license', which
is equivalent to Apache (and which I can't change :-( ); since we
wouldn't be distributing the code though there is no reason why we
can't use it for testing without formal acknowledgement...
P.P.S. The source files are in the JX cvs tree (com/ca/commons/jndi/
TestJNDIOps) but not (yet) in the standard source distribution
packages...
On 19/08/2005, at 6:41 AM, Stefan Zoerner wrote:
> Emmanuel Lecharny wrote:
>
>
>> On Thu, 2005-08-18 at 19:13 +0200, Stefan Zoerner wrote:
>>
>>
>>> Hi all,
>>>
>>>
>> Hi Stefan!
>>
>>
> Thanks for your reply, Emmanuel!
>
>
>> <snip/>
>>
>>
>>> For a start JUnit and JNDI are used, there is no dependency on
>>> code which is provided by the Apache Directory Project (we do
>>> black box tests). While JUnit is beyond dispute, JNDI is not.
>>>
>>
>> JNDI is very commonly used. So it doesn't arm to use it. Better
>> have a
>> way to test ApacheDS than arguing about the best tool to do it :)
>>
>> <snip/>
>>
>>
> You are right, I will rework this passage.
>
>
>> <snip/>
>>
>> I'm still on my way to create a simple test framework based on XML
>> files. I just lack of time to finalize it, but I may found it
>> soon! The
>> idea is to use the asn1-codec to send request and to test expected
>> responses. The XML file will store the data and the expected
>> response.
>> The tool will just launch those tests and give back a result. It
>> will be
>> easy to add a new test, as the structure will be controlled by a XSD
>> (the XSD already exists).
>> Another way to store tests could also to use the proxy to capture
>> PDUs
>> and replay them. you will then be able to create scenarios and tests
>> them against different kind of LDAP server (the proxy already exists,
>> and have been tested against ApacheDS and IDS, with LdapBrowser and
>> JXplorer - snapshot attached). Of course, I need to improve this
>> proxy
>> to dump XML structures to disk, but that's not very complicated.
>> Except
>> that I have to find time to do it ;)
>>
>>
> Especially the proxy seems to be a very useful and interesting
> thing! Even in other scenarios. It may help me to ensure that JNDI
> really sends the operations I want it to (currently I run the tests
> against Sun DS and look in its logs ...). And it would definitly be
> helpful in tutorials to teach people LDAP/JNDI (I give an
> introduction to LDAP for Java developers in November in Munich on a
> conference, for example). Are the sources for the proxy GUI already
> in the Apache DS svn? It may also be interesting in integration
> tasks, e.g. for research if someone fails to integrate an LDAP
> server in a middleware, because the latter is poorly documentated.
> Have you ever thought about offering it in the client section of
> the Apache Directory Project?
>
>
>> Jmeter could also be an opton. I've done some very simply tests once
>> upon a time using it, just to check how ApacheDS was doing against
>> OpenLdap. The answer was : almost as fast. But it was over
>> simplistic.
>> It's very interesting to improve those tests to do some benchmarks !
>>
>>
> Yes, I thought about JMeter as well. Especially because it gives us
> remote JMeter engines on different machines for heavy load tests.
> Nice task for my little test center here (I collect old Sun
> workstations ...). But first I would like to complete the test
> cases for the little test suite. JNDI biggest advantage will be
> that Alex can integrate theme easily in the sercer test suite (at
> least If we find a way to do this wothout copying to much code). He
> may use your internal JNDI provider, while the testsuite run the
> original LDAP provider included in the J2SE.
>
>
>> so, we have a lot of things to do, and we are very few... Welcome
>> Stefan !
>>
>>
> Thank you to welcome me aboard. The more users we have, the more
> helping hands will appear. This is why I am so interested in
> improving end user experience (and this starts with standard
> compliance and includes documentation, which I am also interested
> to contribute). If a user fails to use the server (e.g. because his/
> her clients refuse to work correctly with it), s/he will choose
> another ... .
>
> Bye, Stefan
>
>
Re: Protocol Testsuite: Mission Statement (Proposal)
Posted by Emmanuel Lecharny <el...@apache.org>.
> Are the sources for the proxy GUI already in the Apache DS svn? It may also
> be interesting in integration tasks, e.g. for research if someone fails
> to integrate an LDAP server in a middleware, because the latter is
> poorly documentated. Have you ever thought about offering it in the
> client section of the Apache Directory Project?
It's not in the client section, as it's in the sandbox !
http://svn.apache.org/viewcvs.cgi/directory/sandbox/trunk/asn1-new-
codec/src/java/org/apache/ldap/proxy/
This is very disgusting code, but it works.
It's quite easy to use, you just have to launch MainFrame with three
parameters : the listening port, the LDAP server host and port.
I must admit that it bloody need some packaging, but it met my needs
when I used it : testing the new ASN1 codec.
> Yes, I thought about JMeter as well. Especially because it gives us
> remote JMeter engines on different machines for heavy load tests. Nice
> task for my little test center here (I collect old Sun workstations
> ...). But first I would like to complete the test cases for the little
> test suite.
You are perfectly correct. As soon as tests are completed, it will be a
piece of cake to integrate them in JMeter
> JNDI biggest advantage will be that Alex can integrate theme
> easily in the sercer test suite (at least If we find a way to do this
> wothout copying to much code).
Don't forget to mention that they are helpfull to debug the code ;)
> Thank you to welcome me aboard. The more users we have, the more helping
> hands will appear. This is why I am so interested in improving end user
> experience (and this starts with standard compliance and includes
> documentation, which I am also interested to contribute). If a user
> fails to use the server (e.g. because his/her clients refuse to work
> correctly with it), s/he will choose another ... .
The good point is that there are not a lot of candidates out there ;-)
>
> Bye, Stefan
>
> ---------------------------------------------------------------------------------------
> Wanadoo vous informe que cet e-mail a ete controle par l'anti-virus mail.
> Aucun virus connu a ce jour par nos services n'a ete detecte.
>
>
>
Re: Protocol Testsuite: Mission Statement (Proposal)
Posted by Stefan Zoerner <st...@labeo.de>.
Emmanuel Lecharny wrote:
>On Thu, 2005-08-18 at 19:13 +0200, Stefan Zoerner wrote:
>
>
>>Hi all,
>>
>>
>Hi Stefan!
>
>
Thanks for your reply, Emmanuel!
><snip/>
>
>
>>For a start JUnit and JNDI are used, there is no dependency on code
>>which is provided by the Apache Directory Project (we do black box
>>tests). While JUnit is beyond dispute, JNDI is not.
>>
>>
>
>JNDI is very commonly used. So it doesn't arm to use it. Better have a
>way to test ApacheDS than arguing about the best tool to do it :)
>
><snip/>
>
>
You are right, I will rework this passage.
><snip/>
>
>I'm still on my way to create a simple test framework based on XML
>files. I just lack of time to finalize it, but I may found it soon! The
>idea is to use the asn1-codec to send request and to test expected
>responses. The XML file will store the data and the expected response.
>The tool will just launch those tests and give back a result. It will be
>easy to add a new test, as the structure will be controlled by a XSD
>(the XSD already exists).
>
>Another way to store tests could also to use the proxy to capture PDUs
>and replay them. you will then be able to create scenarios and tests
>them against different kind of LDAP server (the proxy already exists,
>and have been tested against ApacheDS and IDS, with LdapBrowser and
>JXplorer - snapshot attached). Of course, I need to improve this proxy
>to dump XML structures to disk, but that's not very complicated. Except
>that I have to find time to do it ;)
>
>
Especially the proxy seems to be a very useful and interesting thing!
Even in other scenarios. It may help me to ensure that JNDI really sends
the operations I want it to (currently I run the tests against Sun DS
and look in its logs ...). And it would definitly be helpful in
tutorials to teach people LDAP/JNDI (I give an introduction to LDAP for
Java developers in November in Munich on a conference, for example). Are
the sources for the proxy GUI already in the Apache DS svn? It may also
be interesting in integration tasks, e.g. for research if someone fails
to integrate an LDAP server in a middleware, because the latter is
poorly documentated. Have you ever thought about offering it in the
client section of the Apache Directory Project?
>Jmeter could also be an opton. I've done some very simply tests once
>upon a time using it, just to check how ApacheDS was doing against
>OpenLdap. The answer was : almost as fast. But it was over simplistic.
>It's very interesting to improve those tests to do some benchmarks !
>
>
Yes, I thought about JMeter as well. Especially because it gives us
remote JMeter engines on different machines for heavy load tests. Nice
task for my little test center here (I collect old Sun workstations
...). But first I would like to complete the test cases for the little
test suite. JNDI biggest advantage will be that Alex can integrate theme
easily in the sercer test suite (at least If we find a way to do this
wothout copying to much code). He may use your internal JNDI provider,
while the testsuite run the original LDAP provider included in the J2SE.
>so, we have a lot of things to do, and we are very few... Welcome
>Stefan !
>
>
Thank you to welcome me aboard. The more users we have, the more helping
hands will appear. This is why I am so interested in improving end user
experience (and this starts with standard compliance and includes
documentation, which I am also interested to contribute). If a user
fails to use the server (e.g. because his/her clients refuse to work
correctly with it), s/he will choose another ... .
Bye, Stefan
Re: Protocol Testsuite: Mission Statement (Proposal)
Posted by Emmanuel Lecharny <el...@gmail.com>.
On Thu, 2005-08-18 at 19:13 +0200, Stefan Zoerner wrote:
> Hi all,
Hi Stefan!
<snip/>
> For a start JUnit and JNDI are used, there is no dependency on code
> which is provided by the Apache Directory Project (we do black box
> tests). While JUnit is beyond dispute, JNDI is not.
JNDI is very commonly used. So it doesn't arm to use it. Better have a
way to test ApacheDS than arguing about the best tool to do it :)
<snip/>
> Later on this subproject may support the main development of the Apache
> Directory Project with tests for topics which are not rock-solid
> standardized (e.g. access control lists). Or performance tests. Or ... .
> But as long as any client may complain about the behavior, we will
> concentrate on basic operations. Client experience is key.
+ infinite :)
<snip/>
I'm still on my way to create a simple test framework based on XML
files. I just lack of time to finalize it, but I may found it soon! The
idea is to use the asn1-codec to send request and to test expected
responses. The XML file will store the data and the expected response.
The tool will just launch those tests and give back a result. It will be
easy to add a new test, as the structure will be controlled by a XSD
(the XSD already exists).
Another way to store tests could also to use the proxy to capture PDUs
and replay them. you will then be able to create scenarios and tests
them against different kind of LDAP server (the proxy already exists,
and have been tested against ApacheDS and IDS, with LdapBrowser and
JXplorer - snapshot attached). Of course, I need to improve this proxy
to dump XML structures to disk, but that's not very complicated. Except
that I have to find time to do it ;)
Jmeter could also be an opton. I've done some very simply tests once
upon a time using it, just to check how ApacheDS was doing against
OpenLdap. The answer was : almost as fast. But it was over simplistic.
It's very interesting to improve those tests to do some benchmarks !
so, we have a lot of things to do, and we are very few... Welcome
Stefan !