Limited memory *only* in integration tests

[Ignasi Barrera <ig...@gmail.com>]

Out of curiosity, Does anyone know why we are explicitly configuring
the memory for the "integration" tests [1], but not for the "unit" and
"live" tests?

Can we just remove that line?


[1] https://github.com/jclouds/jclouds/blob/master/project/pom.xml#L519

Re: Limited memory *only* in integration tests

[Jeremy Daggett <je...@gmail.com>]

Ignasi, have you by chance timed the differences on the tests? I can run
them here, but just wanted to see if you had any data first. It is great to
get a speed increase on just about anything! ;)


On Wed, Dec 18, 2013 at 7:08 AM, Ignasi Barrera <ig...@gmail.com>wrote:

> Yes, removing this it speeds up a bit the tests, and also improves
> some tests that perform operations on big files (such as an upload).
> I think we could safely remove that (given that neither unit nor live
> tests use those options), but I'd prefer to ask first for opinions.
>
> On 18 December 2013 16:04, Andrew Phillips <ap...@qrmedia.com> wrote:
> >> Out of curiosity, Does anyone know why we are explicitly configuring
> >> the memory for the "integration" tests [1], but not for the "unit" and
> >> "live" tests?
> >>
> >> Can we just remove that line?
> >
> >
> > Do you have any information on what effect that change with have? Does it
> > speed things up? Are there any problems that the line is causing for you
> > right now..?
> >
> > ap
>

Re: Limited memory *only* in integration tests

[Andrew Phillips <ap...@qrmedia.com>]

> Just want to know if someone knows the reason for that configuration
> being there. There is no need to remove it at all. I just want to make
> sure that configuration is there for a reason, not just because "it
> was already there" :)

I suspect it might be one of those "reasons lost in the dawn of time".  
Especially if things improve, even if only slightly, if it's removed,  
I'd say let's submit a PR to get rid of it and see what happens.

After all, we can always get it back if necessary...

ap

Re: Limited memory *only* in integration tests

[Ignasi Barrera <ig...@gmail.com>]

I've changed the upload tests to upload smaller files. Following
Andrew's example, the test now will upload a file up between 1 and 32
Mb. Previously it was uploading much bigger files, but I think the
test has the same value with the new smaller size.

Regarding the memory thing, I was just asking if someone knows why
integration tests use a explicit memory configuration. One thing that
comes to my mind is that integration tests usually perform the
operations against the Transient in-memory providers, so perhaps it is
good to preconfigure the maximum memory to use, but that is only a
guess.

Just want to know if someone knows the reason for that configuration
being there. There is no need to remove it at all. I just want to make
sure that configuration is there for a reason, not just because "it
was already there" :) even more when unit and live tests don't use
that configuration.



On 18 December 2013 23:30, Andrew Gaul <ga...@apache.org> wrote:
> Let's change the tests that use heap size to size their input then
> re-run the experiment.  Also a few months ago I changed some integration
> tests to use smaller inputs in 0d6ef0607617f9968c9fe1267b6fef48e5d329ab
> and e27ae6117cc813b8c39f4514908541d839fb461a which might have also
> improved our memory use.  Otherwise we can remove the explicit size.
>
> On Wed, Dec 18, 2013 at 04:08:14PM +0100, Ignasi Barrera wrote:
>> Yes, removing this it speeds up a bit the tests, and also improves
>> some tests that perform operations on big files (such as an upload).
>> I think we could safely remove that (given that neither unit nor live
>> tests use those options), but I'd prefer to ask first for opinions.
>>
>> On 18 December 2013 16:04, Andrew Phillips <ap...@qrmedia.com> wrote:
>> >> Out of curiosity, Does anyone know why we are explicitly configuring
>> >> the memory for the "integration" tests [1], but not for the "unit" and
>> >> "live" tests?
>> >>
>> >> Can we just remove that line?
>> >
>> >
>> > Do you have any information on what effect that change with have? Does it
>> > speed things up? Are there any problems that the line is causing for you
>> > right now..?
>> >
>> > ap
>
> --
> Andrew Gaul
> http://gaul.org/

Re: Limited memory *only* in integration tests

[Andrew Gaul <ga...@apache.org>]

Let's change the tests that use heap size to size their input then
re-run the experiment.  Also a few months ago I changed some integration
tests to use smaller inputs in 0d6ef0607617f9968c9fe1267b6fef48e5d329ab
and e27ae6117cc813b8c39f4514908541d839fb461a which might have also
improved our memory use.  Otherwise we can remove the explicit size.

On Wed, Dec 18, 2013 at 04:08:14PM +0100, Ignasi Barrera wrote:
> Yes, removing this it speeds up a bit the tests, and also improves
> some tests that perform operations on big files (such as an upload).
> I think we could safely remove that (given that neither unit nor live
> tests use those options), but I'd prefer to ask first for opinions.
> 
> On 18 December 2013 16:04, Andrew Phillips <ap...@qrmedia.com> wrote:
> >> Out of curiosity, Does anyone know why we are explicitly configuring
> >> the memory for the "integration" tests [1], but not for the "unit" and
> >> "live" tests?
> >>
> >> Can we just remove that line?
> >
> >
> > Do you have any information on what effect that change with have? Does it
> > speed things up? Are there any problems that the line is causing for you
> > right now..?
> >
> > ap

-- 
Andrew Gaul
http://gaul.org/

Re: Limited memory *only* in integration tests

[Ignasi Barrera <ig...@gmail.com>]

Yes, removing this it speeds up a bit the tests, and also improves
some tests that perform operations on big files (such as an upload).
I think we could safely remove that (given that neither unit nor live
tests use those options), but I'd prefer to ask first for opinions.

On 18 December 2013 16:04, Andrew Phillips <ap...@qrmedia.com> wrote:
>> Out of curiosity, Does anyone know why we are explicitly configuring
>> the memory for the "integration" tests [1], but not for the "unit" and
>> "live" tests?
>>
>> Can we just remove that line?
>
>
> Do you have any information on what effect that change with have? Does it
> speed things up? Are there any problems that the line is causing for you
> right now..?
>
> ap

Re: Limited memory *only* in integration tests

[Andrew Phillips <ap...@qrmedia.com>]

> Out of curiosity, Does anyone know why we are explicitly configuring
> the memory for the "integration" tests [1], but not for the "unit" and
> "live" tests?
>
> Can we just remove that line?

Do you have any information on what effect that change with have? Does  
it speed things up? Are there any problems that the line is causing  
for you right now..?

ap