You are viewing a plain text version of this content. The canonical link for it is here.
Posted to server-dev@james.apache.org by Manuel Carrasco Moñino <ma...@apache.org> on 2011/10/16 16:58:54 UTC
MailBox Manager Tests performance
Hi all,
Is the Manager Test useful to measure performance?
Testing the the couchdb implementation Pepijn is working on, I've
released that some implementations takes a long to run the tests (see
attached screenshot):
InMemory 0.7s
MailDir 2.4s
JPA 6.0s
JCR 21.1s
CouchDB 40.4s
HBase 81.8s
JPA uses a ddbb in memory, CouchDB is not optimized at all, but Is
there any reason why jcr and hbase takes so long?
- Manolo
Re: MailBox Manager Tests performance
Posted by Eric Charles <er...@u-mangate.com>.
On 17/10/11 13:42, Manuel Carrasco Moñino wrote:
...
> Maybe this test could be expanded to test more mailbox operations and
> not only append 200 simple messages, for instance large attachments,
> read, delete, folder manipulation etc.
>
Makes sense.
We just need to take care to not replicate the same kind of tests in
different base tests.
Eric
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
Re: MailBox Manager Tests performance
Posted by Manuel Carrasco Moñino <ma...@apache.org>.
Ok, clear enough.
So I think we could use the AbstractStressTest to check performance
for each implementation and realize if there is any performance issue.
Maybe this test could be expanded to test more mailbox operations and
not only append 200 simple messages, for instance large attachments,
read, delete, folder manipulation etc.
Thanks
- Manolo
On Mon, Oct 17, 2011 at 10:03 AM, Eric Charles
<er...@u-mangate.com> wrote:
> For hbase, we need to reduce the number of minihbasecluster start/stop.
>
> Each James independent tests suppose a clean persistent store, this is why
> we need to start from scratch (easy for memory and file, takes more time for
> other persistence technologies).
>
> btw, this is also a topic on which the hbase project works for its own tests
> (running a full test suite on hbase src takes hours, mainly due to those
> numerous minihbasecluster start/stop).
>
>
> Thx,
> Eric
>
>
> On 16/10/11 17:14, Manuel Carrasco Moñino wrote:
>>
>> Thanks Stefano
>>
>> I just run once each test.
>>
>> I know there is a overhead when running setup and teardown, what I do
>> not know is if the test could be useful for performance, subtracting
>> the setup/teardown times and looping it.
>>
>> - Manolo
>>
>> On Sun, Oct 16, 2011 at 5:05 PM, Stefano Bagnara<ap...@bago.org> wrote:
>>>
>>> 2011/10/16 Manuel Carrasco Moñino<ma...@apache.org>:
>>>>
>>>> Hi all,
>>>>
>>>> Is the Manager Test useful to measure performance?
>>>>
>>>> Testing the the couchdb implementation Pepijn is working on, I've
>>>> released that some implementations takes a long to run the tests (see
>>>> attached screenshot):
>>>> InMemory 0.7s
>>>> MailDir 2.4s
>>>> JPA 6.0s
>>>> JCR 21.1s
>>>> CouchDB 40.4s
>>>> HBase 81.8s
>>>>
>>>> JPA uses a ddbb in memory, CouchDB is not optimized at all, but Is
>>>> there any reason why jcr and hbase takes so long?
>>>
>>> Don't know how you test this, but if you include startup and warmup
>>> times then it is expected that more complex layers will take longer to
>>> startup.
>>> Maybe you should run the tests 1000 times or make sure you at least
>>> run them twice and only measure timings for the second time (and
>>> exclude startup times).
>>>
>>> Does this make sense?
>>>
>>> Stefano
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
>>> For additional commands, e-mail: server-dev-help@james.apache.org
>>>
>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
>> For additional commands, e-mail: server-dev-help@james.apache.org
>>
>
> --
> Eric
> http://about.echarles.net
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
> For additional commands, e-mail: server-dev-help@james.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
Re: MailBox Manager Tests performance
Posted by Eric Charles <er...@u-mangate.com>.
For hbase, we need to reduce the number of minihbasecluster start/stop.
Each James independent tests suppose a clean persistent store, this is
why we need to start from scratch (easy for memory and file, takes more
time for other persistence technologies).
btw, this is also a topic on which the hbase project works for its own
tests (running a full test suite on hbase src takes hours, mainly due to
those numerous minihbasecluster start/stop).
Thx,
Eric
On 16/10/11 17:14, Manuel Carrasco Moñino wrote:
> Thanks Stefano
>
> I just run once each test.
>
> I know there is a overhead when running setup and teardown, what I do
> not know is if the test could be useful for performance, subtracting
> the setup/teardown times and looping it.
>
> - Manolo
>
> On Sun, Oct 16, 2011 at 5:05 PM, Stefano Bagnara<ap...@bago.org> wrote:
>> 2011/10/16 Manuel Carrasco Moñino<ma...@apache.org>:
>>> Hi all,
>>>
>>> Is the Manager Test useful to measure performance?
>>>
>>> Testing the the couchdb implementation Pepijn is working on, I've
>>> released that some implementations takes a long to run the tests (see
>>> attached screenshot):
>>> InMemory 0.7s
>>> MailDir 2.4s
>>> JPA 6.0s
>>> JCR 21.1s
>>> CouchDB 40.4s
>>> HBase 81.8s
>>>
>>> JPA uses a ddbb in memory, CouchDB is not optimized at all, but Is
>>> there any reason why jcr and hbase takes so long?
>>
>> Don't know how you test this, but if you include startup and warmup
>> times then it is expected that more complex layers will take longer to
>> startup.
>> Maybe you should run the tests 1000 times or make sure you at least
>> run them twice and only measure timings for the second time (and
>> exclude startup times).
>>
>> Does this make sense?
>>
>> Stefano
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
>> For additional commands, e-mail: server-dev-help@james.apache.org
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
> For additional commands, e-mail: server-dev-help@james.apache.org
>
--
Eric
http://about.echarles.net
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
Re: MailBox Manager Tests performance
Posted by Manuel Carrasco Moñino <ma...@apache.org>.
Thanks Stefano
I just run once each test.
I know there is a overhead when running setup and teardown, what I do
not know is if the test could be useful for performance, subtracting
the setup/teardown times and looping it.
- Manolo
On Sun, Oct 16, 2011 at 5:05 PM, Stefano Bagnara <ap...@bago.org> wrote:
> 2011/10/16 Manuel Carrasco Moñino <ma...@apache.org>:
>> Hi all,
>>
>> Is the Manager Test useful to measure performance?
>>
>> Testing the the couchdb implementation Pepijn is working on, I've
>> released that some implementations takes a long to run the tests (see
>> attached screenshot):
>> InMemory 0.7s
>> MailDir 2.4s
>> JPA 6.0s
>> JCR 21.1s
>> CouchDB 40.4s
>> HBase 81.8s
>>
>> JPA uses a ddbb in memory, CouchDB is not optimized at all, but Is
>> there any reason why jcr and hbase takes so long?
>
> Don't know how you test this, but if you include startup and warmup
> times then it is expected that more complex layers will take longer to
> startup.
> Maybe you should run the tests 1000 times or make sure you at least
> run them twice and only measure timings for the second time (and
> exclude startup times).
>
> Does this make sense?
>
> Stefano
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
> For additional commands, e-mail: server-dev-help@james.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
Re: MailBox Manager Tests performance
Posted by Stefano Bagnara <ap...@bago.org>.
2011/10/16 Manuel Carrasco Moñino <ma...@apache.org>:
> Hi all,
>
> Is the Manager Test useful to measure performance?
>
> Testing the the couchdb implementation Pepijn is working on, I've
> released that some implementations takes a long to run the tests (see
> attached screenshot):
> InMemory 0.7s
> MailDir 2.4s
> JPA 6.0s
> JCR 21.1s
> CouchDB 40.4s
> HBase 81.8s
>
> JPA uses a ddbb in memory, CouchDB is not optimized at all, but Is
> there any reason why jcr and hbase takes so long?
Don't know how you test this, but if you include startup and warmup
times then it is expected that more complex layers will take longer to
startup.
Maybe you should run the tests 1000 times or make sure you at least
run them twice and only measure timings for the second time (and
exclude startup times).
Does this make sense?
Stefano
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org