looking for opinions on reasonable hardware requirements for tests in standard derby suite

[Mike Matrigali <mi...@sbcglobal.net>]

What is a reasonable maximum size of a table to be included in
a the standard derby all test suite?  For most tests size of
the table does not matter much, but for a few tests I am looking
at writing I need there to be multiple allocation pages which
then leads to fairly large tables.  For instance I just found
an issue that I didn't see before, but showed up when I used
a 250meg table.

I think there is also a need for a separate test suite which is
meant to be run on machines with large resources, but is not
reasonable to include in a test that every developer runs.  For
instance there should be a test which verifies derby functions
correctly on > 4 gig tables (to be run on OS/JVM's which support
> 4 gig files).  There should be some tests that
test upper bound of clob and blob.

Another category would be long running stress tests.  It would
be nice to have a set of tests that run longer than seconds
to minutes, but not reasonable to expect everyone to include
10 hour stress test runs in their nightly checkins.

I would still like almost all tests to be included in derbyall,
but it would seem reasonable for there to be a small suite
which hopefully could be set up to be automatically run on
some interval - worst case at least before a release.

Re: looking for opinions on reasonable hardware requirements for tests in standard derby suite

[Army <qo...@sbcglobal.net>]

Øystein Grøvlen wrote:

>>>>>>"MM" == Mike Matrigali <mi...@sbcglobal.net> writes:
> 
>     MM> I would still like almost all tests to be included in derbyall,
>     MM> but it would seem reasonable for there to be a small suite
>     MM> which hopefully could be set up to be automatically run on
>     MM> some interval - worst case at least before a release.
> 
> I agree with you.  We need a testsuite for long-running tests and
> large data volumes.  I have a feeling that due to the way we do
> testing, the robustness of Derby for large data volumes are not the
> best.  I know David had some recovery problems when doing a large data
> volume test.
> 

I agree, as well.  And in fact, the need for such a suite came up when I was 
working with DERBY-121.  The test case that I wrote for DERBY-121 requires a 
rather large LOB to be passed to Network Server, which means the server JVM 
needs more heap than usual.  While this probably isn't a "large data volume" 
like what you were envisioning, it nonetheless is a similar sort of 
thing--namely, it's a test that requires extra machine resources and thus 
shouldn't be run as part of "derbyall".

As part of my patch for DERBY-121, I created a new suite "largeData" that is 
fairly generic and that can (hopefully) be expanded/extended to help satisfy the 
requirements described by Mike in this thread.  See the following email for that 
patch and a description of what I did with the new suite:

http://thread.gmane.org/gmane.comp.apache.db.derby.devel/4843

And please make comments if you think the new suite could be organized better to 
fit the needs described in this thread (at least, the "large data volumne" 
needs).  Remember: the new suite is only meant to be a starting place; it's not 
(yet) intended to satsify all of the large data/long-running test requirements...

Army

Re: looking for opinions on reasonable hardware requirements for tests in standard derby suite

[Øystein Grøvlen <Oy...@Sun.COM>]

>>>>> "MM" == Mike Matrigali <mi...@sbcglobal.net> writes:

    MM> Another category would be long running stress tests.  It would
    MM> be nice to have a set of tests that run longer than seconds
    MM> to minutes, but not reasonable to expect everyone to include
    MM> 10 hour stress test runs in their nightly checkins.

    MM> I would still like almost all tests to be included in derbyall,
    MM> but it would seem reasonable for there to be a small suite
    MM> which hopefully could be set up to be automatically run on
    MM> some interval - worst case at least before a release.

I agree with you.  We need a testsuite for long-running tests and
large data volumes.  I have a feeling that due to the way we do
testing, the robustness of Derby for large data volumes are not the
best.  I know David had some recovery problems when doing a large data
volume test.

Hopefully, we can get someone to host the running of such a suite at
least every week.

-- 
Øystein