You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@directory.apache.org by Emmanuel Lecharny <el...@apache.org> on 2009/02/11 17:41:14 UTC
Cleanup levels& Mode for tests
Hi,
following all the mails about the tests, here is a small analysis I
have doen regarding the Mode used and the consequences on the tests.
I'm considering only three modes here :
- RESTART : the server will be restarted but the base will remain on
the disk (usefull when trying to see if we can read the data after
having stopped the server, or for replication)
- PRISTINE : the server will be stopped, data will be cleaned and the
server will be restarted
- ROLLBACK : the changes will be rollbacked, the server remains started
The other mode (CUMMULATIVE and NOSERVICE) are not included in this
small study as the CUMMULATIVE seems to be useless, as we have no way
to guarantee the test orders and for NOSERVICE, as we don't care about
the directory service being up or down, we won't do anything special.
We have three cleanup levels : SUITE, CLASS and METHOD
We have three Modes : RESTART, PRISTINE and ROLLBACK
let's see what we will have to do for each combinaison of those modes
and cleanup levels, assuming we can also avoid declaring a cleanup
level, hence defining a forth cleanup level, which is the inherited
one.
suite class method action
N/A N/A restart server shutdown and restarted for each test
N/A N/A pristine server shutdown, cleaned and started
for each test
restart retstart rollback server shutdown and restarted for every class
restart retstart N/A server shutdown and restarted for every class
restart pristine rollback server shutdown, cleanup and restart for every class
restart pristine N/A server shutdown, cleanup and restart for
every class
everything else : server started once
As we can see, if the Method has a RESTART or PRISTINE mode, we don't
care about the SUITE and CLASS mode. On the other side, if the Method
does not define a mode or is using ROLLBACK, we will restart the
server accordingly to the Class mode if it's PRISTINE or RESTART, and
last, not least, in any other case, we start the server once, and run
each test with a rollback the test's modifications after each test.
More to come...
--
Regards,
Cordialement,
Emmanuel Lécharny
www.iktek.com
Re: Cleanup levels& Mode for tests
Posted by Emmanuel Lecharny <el...@apache.org>.
>> In any case, we can deduce the clean level from the mode.
>>
>>
>
> Right I see that with your examples clearly now. Even though RESTART
> requires some handling contrary to what you wrote it does not impact the
> point you're making here.
>
I meant revert or LDIF inject when I wrote 'don't do anything'. I should
have been more precise though.
> Thanks,
> Alex
>
>
--
--
cordialement, regards,
Emmanuel Lécharny
www.iktek.com
directory.apache.org
Re: Cleanup levels& Mode for tests
Posted by Alex Karasulu <ak...@gmail.com>.
On Fri, Feb 13, 2009 at 11:42 AM, Emmanuel Lecharny <el...@apache.org>wrote:
> On Fri, Feb 13, 2009 at 5:22 PM, Alex Karasulu <ak...@gmail.com>
> wrote:
> > Ahh so you're saying we can infer when to cleanup from the mode and don't
> > need to specify explicity when to cleanup?
>
> yes. Let me explain.
>
> if we have a test which is not the first one in a class,
> 1-1) If we have a test with mode = PRISTINE, we don't care : the
> server is wept out
> 1-2) If the mode is RESTART, CUMULATIVE or NOSERVICE, we don't do anything
With RESTART we do have to stop and start the server.
>
> 1-3) If the mode is ROLLBACK, we rollback the method changes
> (including the LDIFs)
> 1-4) If the test does not define a mode, then step 2:
>
> step 2
> 2-1) if the class mode is PRISTINE, we wipeout the server
> 2-2) if the class mode is RESTART, CUMULATIVE or NOSERVICE, we don't do
> anything
Again RESTART will require a stop the start before conducting the test.
>
> 2-3) if the class mode is ROLLBACK, we rollback the method changes and
> LDIFs AND the class LDIFs
> 2-4) if the class does not define a mode, then step 3
>
> step 3
> 3-1) if the suite mode is PRISTINE, we wipe the server
> 3-2) if the suite mode is RESTART, CUMULATIVE or NOSERVICE, we don't do
> anything
> 3-3) if the suite mode is ROLLBACK, we rollback the the method changes
Same as above.
>
> and LDIFs AND the class LDIFs AND the suite LDIFs
> 3-4) if the class does not define a mode, then assume that the mode is
> ROLLBACK for the test, and go back to (1-3)
>
> If the test is the first one for a class which is not the first class
> for the suite, then we have to check the class mode :
> 4-1) if the class mode is PRISTINE, wipe the data
> 4-2) if the class mode is RESTART, CUMULATIVE or NOSERVICE, we don't do
> anything
> 4-3) if the class mode is ROLLBACK, then we revert the class LDIFs
> 4-4) if the class does not define a mode, then step 4
>
> step 4
> 5-1) if the suite mode is PRISTINE, wipe the data
> 5-2) if the suite mode is RESTART, CUMULATIVE or NOSERVICE, we don't do
> anything
> 5-3) if the suite mode is ROLLBACK, then we revert the class LDIFs
> 5-4) if the suite does not define a mode, then assume that the class
> mode is ROLLBACK and go back to (4-3)
>
> last, not least, if a test is first one of teh first class of the
> suite, we just start the server from scratch
>
> In any case, we can deduce the clean level from the mode.
>
Right I see that with your examples clearly now. Even though RESTART
requires some handling contrary to what you wrote it does not impact the
point you're making here.
Thanks,
Alex
Re: Cleanup levels& Mode for tests
Posted by Emmanuel Lecharny <el...@apache.org>.
On Fri, Feb 13, 2009 at 5:22 PM, Alex Karasulu <ak...@gmail.com> wrote:
> Ahh so you're saying we can infer when to cleanup from the mode and don't
> need to specify explicity when to cleanup?
yes. Let me explain.
if we have a test which is not the first one in a class,
1-1) If we have a test with mode = PRISTINE, we don't care : the
server is wept out
1-2) If the mode is RESTART, CUMULATIVE or NOSERVICE, we don't do anything
1-3) If the mode is ROLLBACK, we rollback the method changes
(including the LDIFs)
1-4) If the test does not define a mode, then step 2:
step 2
2-1) if the class mode is PRISTINE, we wipeout the server
2-2) if the class mode is RESTART, CUMULATIVE or NOSERVICE, we don't do anything
2-3) if the class mode is ROLLBACK, we rollback the method changes and
LDIFs AND the class LDIFs
2-4) if the class does not define a mode, then step 3
step 3
3-1) if the suite mode is PRISTINE, we wipe the server
3-2) if the suite mode is RESTART, CUMULATIVE or NOSERVICE, we don't do anything
3-3) if the suite mode is ROLLBACK, we rollback the the method changes
and LDIFs AND the class LDIFs AND the suite LDIFs
3-4) if the class does not define a mode, then assume that the mode is
ROLLBACK for the test, and go back to (1-3)
If the test is the first one for a class which is not the first class
for the suite, then we have to check the class mode :
4-1) if the class mode is PRISTINE, wipe the data
4-2) if the class mode is RESTART, CUMULATIVE or NOSERVICE, we don't do anything
4-3) if the class mode is ROLLBACK, then we revert the class LDIFs
4-4) if the class does not define a mode, then step 4
step 4
5-1) if the suite mode is PRISTINE, wipe the data
5-2) if the suite mode is RESTART, CUMULATIVE or NOSERVICE, we don't do anything
5-3) if the suite mode is ROLLBACK, then we revert the class LDIFs
5-4) if the suite does not define a mode, then assume that the class
mode is ROLLBACK and go back to (4-3)
last, not least, if a test is first one of teh first class of the
suite, we just start the server from scratch
In any case, we can deduce the clean level from the mode.
--
Regards,
Cordialement,
Emmanuel Lécharny
www.iktek.com
Re: Cleanup levels& Mode for tests
Posted by Alex Karasulu <ak...@gmail.com>.
Ahh so you're saying we can infer when to cleanup from the mode and don't
need to specify explicity when to cleanup?
On Fri, Feb 13, 2009 at 11:04 AM, Emmanuel Lecharny <el...@apache.org>wrote:
> I reviewed the modes, and I now think we can keep all of them, at some
> point, they all make some sense. However, I'm more annoyed about the
> cleanLevel, I think they are a kind of redondant annotation, as we
> know what to do using the Mode.
>
> --
> Regards,
> Cordialement,
> Emmanuel Lécharny
> www.iktek.com
>
Re: Cleanup levels& Mode for tests
Posted by Emmanuel Lecharny <el...@apache.org>.
I reviewed the modes, and I now think we can keep all of them, at some
point, they all make some sense. However, I'm more annoyed about the
cleanLevel, I think they are a kind of redondant annotation, as we
know what to do using the Mode.
--
Regards,
Cordialement,
Emmanuel Lécharny
www.iktek.com
Re: Cleanup levels& Mode for tests
Posted by Alex Karasulu <ak...@gmail.com>.
Thanks for this Emm. BTW I think CUMULATIVE was put in so if one test set
something up the other can use it. But this situation is seldom if at all
encountered.
Alex
On Wed, Feb 11, 2009 at 11:41 AM, Emmanuel Lecharny <el...@apache.org>wrote:
> Hi,
>
> following all the mails about the tests, here is a small analysis I
> have doen regarding the Mode used and the consequences on the tests.
> I'm considering only three modes here :
> - RESTART : the server will be restarted but the base will remain on
> the disk (usefull when trying to see if we can read the data after
> having stopped the server, or for replication)
> - PRISTINE : the server will be stopped, data will be cleaned and the
> server will be restarted
> - ROLLBACK : the changes will be rollbacked, the server remains started
>
> The other mode (CUMMULATIVE and NOSERVICE) are not included in this
> small study as the CUMMULATIVE seems to be useless, as we have no way
> to guarantee the test orders and for NOSERVICE, as we don't care about
> the directory service being up or down, we won't do anything special.
>
> We have three cleanup levels : SUITE, CLASS and METHOD
> We have three Modes : RESTART, PRISTINE and ROLLBACK
>
> let's see what we will have to do for each combinaison of those modes
> and cleanup levels, assuming we can also avoid declaring a cleanup
> level, hence defining a forth cleanup level, which is the inherited
> one.
>
> suite class method action
> N/A N/A restart server shutdown and restarted for
> each test
> N/A N/A pristine server shutdown, cleaned
> and started
> for each test
> restart retstart rollback server shutdown and restarted for
> every class
> restart retstart N/A server shutdown and restarted for
> every class
> restart pristine rollback server shutdown, cleanup and
> restart for every class
> restart pristine N/A server shutdown, cleanup and
> restart for
> every class
> everything else : server started once
>
> As we can see, if the Method has a RESTART or PRISTINE mode, we don't
> care about the SUITE and CLASS mode. On the other side, if the Method
> does not define a mode or is using ROLLBACK, we will restart the
> server accordingly to the Class mode if it's PRISTINE or RESTART, and
> last, not least, in any other case, we start the server once, and run
> each test with a rollback the test's modifications after each test.
>
> More to come...
>
>
> --
> Regards,
> Cordialement,
> Emmanuel Lécharny
> www.iktek.com
>