Re: CamelTestSupport for junit 4 (re)starting context per class instead of per method

[Christian Schneider <ch...@die-schneider.net>]

Sure you can get strange behaviour in some cases but I think if the 
Developer of the test knows that his context is per class it should be ok.
I am not proposing to change the default as current tests sometimes will 
rely on running isolated. I rather propose to write new tests
with one context per class where possible and of course to rewrite old 
tests over time.

While you are right that per method will avoid most side effects I think 
the test time could be the more important argument. Currently the
whole camel build takes way tool long and perhaps this is one way to 
bring the time down.

Greetings

Christian



Am 05.03.2010 12:16, schrieb Claus Ibsen:
> -1
>
> All tests should run isolated.
>
> If you start to keep the JVM up for each test method you can really
> get some strange and subtle side effects with testing.
>
>
> On Fri, Mar 5, 2010 at 11:52 AM, Christian Schneider
> <ch...@die-schneider.net>  wrote:
>    
>> Hi all,
>>
>> I am currently debugging a little in the tests for jetty because I got an
>> error in a test.
>> While doing so I found out that the camel context or even the spring context
>> is (re)started for each test method. Wouldn´t it be much more efficient to
>> do this only per class?
>> I guess the tests could take less then half the time after this change. What
>> do you think?
>>
>> Greetings
>>
>> Christian
>>
>> --
>>
>> Christian Schneider
>> ---
>> http://www.liquid-reality.de
>>
>>
>>      
>
>
>    


-- 

Christian Schneider
---
http://www.liquid-reality.de

Re: CamelTestSupport for junit 4 (re)starting context per class instead of per method

[Norman Maurer <no...@googlemail.com>]

I think Claus is right.. Its just "safer" to create a new context everytime..

Just my 2 cent as camel user ;)

Bye,
Norman


2010/3/5 Claus Ibsen <cl...@gmail.com>:
> I am still -1 on this.
>
> Testing camel-spring is just a little tip of the iceberg.
> I would rather have consistent test which newer make it out on a ghost
> chase because of side effects of running multiple tests in the same
> JVM.
>
> With Camel we embed many different frameworks and test together in the
> JVM and you can't rely on the JVM being 100% left in a consistent and
> clean state between each method that is tested.
>
> Yes Camel takes long time to test, but it has 5500+ tests.
>
> And we got CI servers which test it throughly every day.
>
> And frankly Spring is somewhat slow to start and hence why Camel
> Spring is taking its time.
> On top of that JMX adds around 15-20% as well.
>
>
>
> On Fri, Mar 5, 2010 at 12:34 PM, Christian Schneider
> <ch...@die-schneider.net> wrote:
>> Sure you can get strange behaviour in some cases but I think if the
>> Developer of the test knows that his context is per class it should be ok.
>> I am not proposing to change the default as current tests sometimes will
>> rely on running isolated. I rather propose to write new tests
>> with one context per class where possible and of course to rewrite old tests
>> over time.
>>
>> While you are right that per method will avoid most side effects I think the
>> test time could be the more important argument. Currently the
>> whole camel build takes way tool long and perhaps this is one way to bring
>> the time down.
>>
>> Greetings
>>
>> Christian
>>
>>
>>
>> Am 05.03.2010 12:16, schrieb Claus Ibsen:
>>>
>>> -1
>>>
>>> All tests should run isolated.
>>>
>>> If you start to keep the JVM up for each test method you can really
>>> get some strange and subtle side effects with testing.
>>>
>>>
>>> On Fri, Mar 5, 2010 at 11:52 AM, Christian Schneider
>>> <ch...@die-schneider.net>  wrote:
>>>
>>>>
>>>> Hi all,
>>>>
>>>> I am currently debugging a little in the tests for jetty because I got an
>>>> error in a test.
>>>> While doing so I found out that the camel context or even the spring
>>>> context
>>>> is (re)started for each test method. Wouldn´t it be much more efficient
>>>> to
>>>> do this only per class?
>>>> I guess the tests could take less then half the time after this change.
>>>> What
>>>> do you think?
>>>>
>>>> Greetings
>>>>
>>>> Christian
>>>>
>>>> --
>>>>
>>>> Christian Schneider
>>>> ---
>>>> http://www.liquid-reality.de
>>>>
>>>>
>>>>
>>>
>>>
>>>
>>
>>
>> --
>>
>> Christian Schneider
>> ---
>> http://www.liquid-reality.de
>>
>>
>
>
>
> --
> Claus Ibsen
> Apache Camel Committer
>
> Author of Camel in Action: http://www.manning.com/ibsen/
> Open Source Integration: http://fusesource.com
> Blog: http://davsclaus.blogspot.com/
> Twitter: http://twitter.com/davsclaus
>

Re: CamelTestSupport for junit 4 (re)starting context per class instead of per method

[Claus Ibsen <cl...@gmail.com>]

Hi

I wonder if you can change in the pom.xml how junit is run? I recon
you can configure it to fork per test or per class?
Maybe we can add a maven profile or something to select the "other"
kind of testing.

But the default tests should be run as they do now, per test and in
100% isolation.


On Fri, Mar 5, 2010 at 8:42 PM, Stephen Gargan <st...@gmail.com> wrote:
> It would be very useful to have both particularly for integration
> testing with spring. I regularly end up writing only a single
> 'testable' method so that the context is only created once. This
> runner method delegates to the actual test methods. It works, but the
> granularity of the test reporting suffers. You only ever see one test
> succeed and if one of the sub tests fails well you need to go hunting
> through the logs.
>
> I agree though that you can end up with strange behaviour when
> assumptions you made for isolated tests don't turn out to be the case
> when they share a VM. If the feature was present it could be made
> clear that this mode is unsupported, use at your peril. The isolated
> case would always be the default and log warnings and the Support
> documenation would state, in no uncertain terms, that you are on your
> own.
>
> I'd defintely use it if it was there.
>
> rgds,
>
> ste
>
>
> On Fri, Mar 5, 2010 at 4:40 AM, Willem Jiang <wi...@gmail.com> wrote:
>> Current camel unit tests take lots of time, and in most case the time are
>> spent on create the camel context.
>>
>> If we don't change the routes dynamically, I think it should be safe for us
>> to run the test method on the same routes. But if you have other resource
>> need to clean, such as file resource , or DB tables, that could  be another
>> issue.
>>
>> The main reason that I add the CamelTestSupport for JUnit4 is we can get the
>> benefit to start the cxf back end services or JMS broker per class, which
>> will save us lots of test time and avoid the port conflict when running the
>> test in the same JVM.
>>
>> Just my 2 cents.
>>
>> Willem
>>
>>
>> Claus Ibsen wrote:
>>>
>>> I am still -1 on this.
>>>
>>> Testing camel-spring is just a little tip of the iceberg.
>>> I would rather have consistent test which newer make it out on a ghost
>>> chase because of side effects of running multiple tests in the same
>>> JVM.
>>>
>>> With Camel we embed many different frameworks and test together in the
>>> JVM and you can't rely on the JVM being 100% left in a consistent and
>>> clean state between each method that is tested.
>>>
>>> Yes Camel takes long time to test, but it has 5500+ tests.
>>>
>>> And we got CI servers which test it throughly every day.
>>>
>>> And frankly Spring is somewhat slow to start and hence why Camel
>>> Spring is taking its time.
>>> On top of that JMX adds around 15-20% as well.
>>>
>>>
>>>
>>> On Fri, Mar 5, 2010 at 12:34 PM, Christian Schneider
>>> <ch...@die-schneider.net> wrote:
>>>>
>>>> Sure you can get strange behaviour in some cases but I think if the
>>>> Developer of the test knows that his context is per class it should be
>>>> ok.
>>>> I am not proposing to change the default as current tests sometimes will
>>>> rely on running isolated. I rather propose to write new tests
>>>> with one context per class where possible and of course to rewrite old
>>>> tests
>>>> over time.
>>>>
>>>> While you are right that per method will avoid most side effects I think
>>>> the
>>>> test time could be the more important argument. Currently the
>>>> whole camel build takes way tool long and perhaps this is one way to
>>>> bring
>>>> the time down.
>>>>
>>>> Greetings
>>>>
>>>> Christian
>>>>
>>>>
>>>>
>>>> Am 05.03.2010 12:16, schrieb Claus Ibsen:
>>>>>
>>>>> -1
>>>>>
>>>>> All tests should run isolated.
>>>>>
>>>>> If you start to keep the JVM up for each test method you can really
>>>>> get some strange and subtle side effects with testing.
>>>>>
>>>>>
>>>>> On Fri, Mar 5, 2010 at 11:52 AM, Christian Schneider
>>>>> <ch...@die-schneider.net>  wrote:
>>>>>
>>>>>> Hi all,
>>>>>>
>>>>>> I am currently debugging a little in the tests for jetty because I got
>>>>>> an
>>>>>> error in a test.
>>>>>> While doing so I found out that the camel context or even the spring
>>>>>> context
>>>>>> is (re)started for each test method. Wouldn´t it be much more efficient
>>>>>> to
>>>>>> do this only per class?
>>>>>> I guess the tests could take less then half the time after this change.
>>>>>> What
>>>>>> do you think?
>>>>>>
>>>>>> Greetings
>>>>>>
>>>>>> Christian
>>>>>>
>>>>>> --
>>>>>>
>>>>>> Christian Schneider
>>>>>> ---
>>>>>> http://www.liquid-reality.de
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>
>>>> --
>>>>
>>>> Christian Schneider
>>>> ---
>>>> http://www.liquid-reality.de
>>>>
>>>>
>>>
>>>
>>>
>>
>>
>



-- 
Claus Ibsen
Apache Camel Committer

Author of Camel in Action: http://www.manning.com/ibsen/
Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Twitter: http://twitter.com/davsclaus

Re: CamelTestSupport for junit 4 (re)starting context per class instead of per method

[Stephen Gargan <st...@gmail.com>]

It would be very useful to have both particularly for integration
testing with spring. I regularly end up writing only a single
'testable' method so that the context is only created once. This
runner method delegates to the actual test methods. It works, but the
granularity of the test reporting suffers. You only ever see one test
succeed and if one of the sub tests fails well you need to go hunting
through the logs.

I agree though that you can end up with strange behaviour when
assumptions you made for isolated tests don't turn out to be the case
when they share a VM. If the feature was present it could be made
clear that this mode is unsupported, use at your peril. The isolated
case would always be the default and log warnings and the Support
documenation would state, in no uncertain terms, that you are on your
own.

I'd defintely use it if it was there.

rgds,

ste


On Fri, Mar 5, 2010 at 4:40 AM, Willem Jiang <wi...@gmail.com> wrote:
> Current camel unit tests take lots of time, and in most case the time are
> spent on create the camel context.
>
> If we don't change the routes dynamically, I think it should be safe for us
> to run the test method on the same routes. But if you have other resource
> need to clean, such as file resource , or DB tables, that could  be another
> issue.
>
> The main reason that I add the CamelTestSupport for JUnit4 is we can get the
> benefit to start the cxf back end services or JMS broker per class, which
> will save us lots of test time and avoid the port conflict when running the
> test in the same JVM.
>
> Just my 2 cents.
>
> Willem
>
>
> Claus Ibsen wrote:
>>
>> I am still -1 on this.
>>
>> Testing camel-spring is just a little tip of the iceberg.
>> I would rather have consistent test which newer make it out on a ghost
>> chase because of side effects of running multiple tests in the same
>> JVM.
>>
>> With Camel we embed many different frameworks and test together in the
>> JVM and you can't rely on the JVM being 100% left in a consistent and
>> clean state between each method that is tested.
>>
>> Yes Camel takes long time to test, but it has 5500+ tests.
>>
>> And we got CI servers which test it throughly every day.
>>
>> And frankly Spring is somewhat slow to start and hence why Camel
>> Spring is taking its time.
>> On top of that JMX adds around 15-20% as well.
>>
>>
>>
>> On Fri, Mar 5, 2010 at 12:34 PM, Christian Schneider
>> <ch...@die-schneider.net> wrote:
>>>
>>> Sure you can get strange behaviour in some cases but I think if the
>>> Developer of the test knows that his context is per class it should be
>>> ok.
>>> I am not proposing to change the default as current tests sometimes will
>>> rely on running isolated. I rather propose to write new tests
>>> with one context per class where possible and of course to rewrite old
>>> tests
>>> over time.
>>>
>>> While you are right that per method will avoid most side effects I think
>>> the
>>> test time could be the more important argument. Currently the
>>> whole camel build takes way tool long and perhaps this is one way to
>>> bring
>>> the time down.
>>>
>>> Greetings
>>>
>>> Christian
>>>
>>>
>>>
>>> Am 05.03.2010 12:16, schrieb Claus Ibsen:
>>>>
>>>> -1
>>>>
>>>> All tests should run isolated.
>>>>
>>>> If you start to keep the JVM up for each test method you can really
>>>> get some strange and subtle side effects with testing.
>>>>
>>>>
>>>> On Fri, Mar 5, 2010 at 11:52 AM, Christian Schneider
>>>> <ch...@die-schneider.net>  wrote:
>>>>
>>>>> Hi all,
>>>>>
>>>>> I am currently debugging a little in the tests for jetty because I got
>>>>> an
>>>>> error in a test.
>>>>> While doing so I found out that the camel context or even the spring
>>>>> context
>>>>> is (re)started for each test method. Wouldn´t it be much more efficient
>>>>> to
>>>>> do this only per class?
>>>>> I guess the tests could take less then half the time after this change.
>>>>> What
>>>>> do you think?
>>>>>
>>>>> Greetings
>>>>>
>>>>> Christian
>>>>>
>>>>> --
>>>>>
>>>>> Christian Schneider
>>>>> ---
>>>>> http://www.liquid-reality.de
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>
>>> --
>>>
>>> Christian Schneider
>>> ---
>>> http://www.liquid-reality.de
>>>
>>>
>>
>>
>>
>
>

Re: CamelTestSupport for junit 4 (re)starting context per class instead of per method

[Willem Jiang <wi...@gmail.com>]

Current camel unit tests take lots of time, and in most case the time 
are spent on create the camel context.

If we don't change the routes dynamically, I think it should be safe for 
us to run the test method on the same routes. But if you have other 
resource need to clean, such as file resource , or DB tables, that could 
  be another issue.

The main reason that I add the CamelTestSupport for JUnit4 is we can get 
the benefit to start the cxf back end services or JMS broker per class, 
which will save us lots of test time and avoid the port conflict when 
running the test in the same JVM.

Just my 2 cents.

Willem


Claus Ibsen wrote:
> I am still -1 on this.
> 
> Testing camel-spring is just a little tip of the iceberg.
> I would rather have consistent test which newer make it out on a ghost
> chase because of side effects of running multiple tests in the same
> JVM.
> 
> With Camel we embed many different frameworks and test together in the
> JVM and you can't rely on the JVM being 100% left in a consistent and
> clean state between each method that is tested.
> 
> Yes Camel takes long time to test, but it has 5500+ tests.
> 
> And we got CI servers which test it throughly every day.
> 
> And frankly Spring is somewhat slow to start and hence why Camel
> Spring is taking its time.
> On top of that JMX adds around 15-20% as well.
> 
> 
> 
> On Fri, Mar 5, 2010 at 12:34 PM, Christian Schneider
> <ch...@die-schneider.net> wrote:
>> Sure you can get strange behaviour in some cases but I think if the
>> Developer of the test knows that his context is per class it should be ok.
>> I am not proposing to change the default as current tests sometimes will
>> rely on running isolated. I rather propose to write new tests
>> with one context per class where possible and of course to rewrite old tests
>> over time.
>>
>> While you are right that per method will avoid most side effects I think the
>> test time could be the more important argument. Currently the
>> whole camel build takes way tool long and perhaps this is one way to bring
>> the time down.
>>
>> Greetings
>>
>> Christian
>>
>>
>>
>> Am 05.03.2010 12:16, schrieb Claus Ibsen:
>>> -1
>>>
>>> All tests should run isolated.
>>>
>>> If you start to keep the JVM up for each test method you can really
>>> get some strange and subtle side effects with testing.
>>>
>>>
>>> On Fri, Mar 5, 2010 at 11:52 AM, Christian Schneider
>>> <ch...@die-schneider.net>  wrote:
>>>
>>>> Hi all,
>>>>
>>>> I am currently debugging a little in the tests for jetty because I got an
>>>> error in a test.
>>>> While doing so I found out that the camel context or even the spring
>>>> context
>>>> is (re)started for each test method. Wouldn´t it be much more efficient
>>>> to
>>>> do this only per class?
>>>> I guess the tests could take less then half the time after this change.
>>>> What
>>>> do you think?
>>>>
>>>> Greetings
>>>>
>>>> Christian
>>>>
>>>> --
>>>>
>>>> Christian Schneider
>>>> ---
>>>> http://www.liquid-reality.de
>>>>
>>>>
>>>>
>>>
>>>
>>
>> --
>>
>> Christian Schneider
>> ---
>> http://www.liquid-reality.de
>>
>>
> 
> 
> 

Re: CamelTestSupport for junit 4 (re)starting context per class instead of per method

[Claus Ibsen <cl...@gmail.com>]

I am still -1 on this.

Testing camel-spring is just a little tip of the iceberg.
I would rather have consistent test which newer make it out on a ghost
chase because of side effects of running multiple tests in the same
JVM.

With Camel we embed many different frameworks and test together in the
JVM and you can't rely on the JVM being 100% left in a consistent and
clean state between each method that is tested.

Yes Camel takes long time to test, but it has 5500+ tests.

And we got CI servers which test it throughly every day.

And frankly Spring is somewhat slow to start and hence why Camel
Spring is taking its time.
On top of that JMX adds around 15-20% as well.



On Fri, Mar 5, 2010 at 12:34 PM, Christian Schneider
<ch...@die-schneider.net> wrote:
> Sure you can get strange behaviour in some cases but I think if the
> Developer of the test knows that his context is per class it should be ok.
> I am not proposing to change the default as current tests sometimes will
> rely on running isolated. I rather propose to write new tests
> with one context per class where possible and of course to rewrite old tests
> over time.
>
> While you are right that per method will avoid most side effects I think the
> test time could be the more important argument. Currently the
> whole camel build takes way tool long and perhaps this is one way to bring
> the time down.
>
> Greetings
>
> Christian
>
>
>
> Am 05.03.2010 12:16, schrieb Claus Ibsen:
>>
>> -1
>>
>> All tests should run isolated.
>>
>> If you start to keep the JVM up for each test method you can really
>> get some strange and subtle side effects with testing.
>>
>>
>> On Fri, Mar 5, 2010 at 11:52 AM, Christian Schneider
>> <ch...@die-schneider.net>  wrote:
>>
>>>
>>> Hi all,
>>>
>>> I am currently debugging a little in the tests for jetty because I got an
>>> error in a test.
>>> While doing so I found out that the camel context or even the spring
>>> context
>>> is (re)started for each test method. Wouldn´t it be much more efficient
>>> to
>>> do this only per class?
>>> I guess the tests could take less then half the time after this change.
>>> What
>>> do you think?
>>>
>>> Greetings
>>>
>>> Christian
>>>
>>> --
>>>
>>> Christian Schneider
>>> ---
>>> http://www.liquid-reality.de
>>>
>>>
>>>
>>
>>
>>
>
>
> --
>
> Christian Schneider
> ---
> http://www.liquid-reality.de
>
>



-- 
Claus Ibsen
Apache Camel Committer

Author of Camel in Action: http://www.manning.com/ibsen/
Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Twitter: http://twitter.com/davsclaus