You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ofbiz.apache.org by Brett Palmer <br...@gmail.com> on 2006/10/24 05:21:23 UTC
Re: Database Test Suite
Sorry, there has not been a lot of work done on the JXUnit stuff
since its original creation. The JXUnit project did not get a lot of
support on sourceforge and the original contributors moved on to other
projects.
I continued to use the ofbiz jxunit code for some of my own projects
but creating data driven tests were still difficult to create and
maintain.
Over the past year, I have been working with a different framework
called grinder (http://grinder.sourceforge.net/). There have been a
few posts on the subject from David Jones and Al Byers. Al has
recently been working on it and we have shared some ideas on how to
make it effective for ofbiz.
Here are some of the features that I like about grinder:
- Uses Jython as a the scripting language to implement tests. This
makes it easier to create new tests without having to compile and
deploy each time.
- TCP Proxy listener to capture web requests and generates a test
script. This script can then be modified to be more dynamic
- Ability to simulate multiple clients with various load and frequency options
Here are some things that are missing and what Al and I have been discussing:
- Need to be able to separate data from the test
- Need to develop some jython utilities that can read in a common XML
(or something else) format for a test and pass the data to the jython
test script
Al and David can probably share some of their ideas on testing. We
need to have more test tools but it is always a matter of priorities
and everyone is always busy with other tasks.
Thoughts?
Brett
On 10/23/06, Tim Ruppert <ti...@hotwaxmedia.com> wrote:
> I was a committer for DbUnit back in the day, which does stage
> information nicely into the database for running tests. We used it in
> conjunction with WebTest to drive all testing from ant. It has worked
> well in the past, but might not be the most up to date solution.
>
> The only concern that I would have on this front is that there may be
> too much data to stage, With large amounts of data, the DbUnit approach
> could run into some performance issues with certain data sets. It might
> still be a viable approach for you John depending on your situation.
>
> I'd still love to hear more about Brett's JXUnit experience whenever he
> gets a chance.
>
> Cheers,
> Tim
>
> --
> Tim Ruppert
> HotWax Media
> http://www.hotwaxmedia.com
>
> o:801.649.6594
> f:801.649.6595
>
> John Martin wrote:
> > I'm looking for a good way of testing service methods that do CRUD on
> > various tables in the database.
> >
> > I reviewed the framework/testtools and it didn't seem that there was
> > much if anything going on there nor did it appear to address
> > manipulation of test database data.
> >
> > While poking around I ran into ProposedOFBizTestFramework document by
> > Brett Palmer that uses JXUnit
> > (http://ofbizwiki.go-integral.com/Wiki.jsp?page=ProposedOFBizTestFramework
> >
> > ).
> >
> > This document looks quite promising and seems like it is exactly what
> > I am looking for but the ofbiz-test that it referenced no longer
> > exists at SourceForge. Is this a dead project? If not, where is it
> > now located?
> >
> > What is the best practices for writing a test suite for new services
> > that are manipulating data?
> >
> > Thanks!
> >
> > John
>
Re: Database Test Suite
Posted by Tim Ruppert <ti...@hotwaxmedia.com>.
John, this is exactly what I used DbUnit (http://www.dbunit.org/) for in
the past. You can drive this from Ant or thru code, so it's as flexible
as you need it to be.
Check out some of the information up on their site - as well as this
(slightly dated) article written about them -
http://www.onjava.com/pub/a/onjava/2004/01/21/dbunit.html . What we did
was extend their DatabaseTestCase and load and clear data based upon the
test case or test suite that we were running - using dynamically
generated resource paths based upon the package and class names of the
files.
If you're interested and I can be of help - I'm here to answer more Q's
Cheers,
Tim
--
Tim Ruppert
HotWax Media
http://www.hotwaxmedia.com
o:801.649.6594
f:801.649.6595
John Martin wrote:
> Grinder does look really interesting and appears to me to be geared
> more for testing the application for a UI and performance perspective.
>
> What I am really after is something that is more database oriented to
> allow a test suite to initialize data appropriately, run the test, and
> then clean out the test data. I'm writing new services to
> insert/update data and want to have a simple way of building tests to
> make sure that my code is working correctly.
>
> Currently, I write a test script that invokes the service and I
> inspect the results and walk through the code in the debugger. I
> agree with Brett that "creating data driven tests were still difficult
> to create and maintain." There's got to be an easier way...
>
> Brett - I would be interested in the JXUnit integration with ofBiz
> since that appears to be inline with what I'm trying to do. Can point
> me to where I can get what I would need to use it?
>
> Thanks,
>
> John
Re: Database Test Suite
Posted by John Martin <pb...@gmail.com>.
Intelligent meaning that the teardown logic could refer to the test
data XML and perform the cleanup. The XML probably would need some
syntax for the clean up. It would be a nice feature if you could
disable the teardown so that you could visually inspect the data and
then invoke the teardown on demand to do the cleanup.
Re: Database Test Suite
Posted by David E Jones <jo...@undersunconsulting.com>.
On Oct 24, 2006, at 6:42 PM, John Martin wrote:
> Setting up the service and using transactions sound like a good idea.
> The only negative that I see initially is that some times inspecting
> the data visually is quite helpful to see what is going on. In that
> case, you have to let the transaction commit and then manually clean
> up the mess.
>
> Perhaps we could use the XML data import from with the build up method
> and the tear-down could intellegently remove data from the XML. Any
> thoughts on that?
Just one: depending on the approach you take it's either really hard
to do, or requires a very large overhead and only somewhat hard to do
(which might be okay for test environments). Either way, it requires
defining exactly what you want, including the granularity of changes
you want to deal with. The tough part is the "intelligent" part...
what would that mean?
-David
Re: Database Test Suite
Posted by John Martin <pb...@gmail.com>.
David,
Setting up the service and using transactions sound like a good idea.
The only negative that I see initially is that some times inspecting
the data visually is quite helpful to see what is going on. In that
case, you have to let the transaction commit and then manually clean
up the mess.
Perhaps we could use the XML data import from with the build up method
and the tear-down could intellegently remove data from the XML. Any
thoughts on that?
John
Re: Database Test Suite
Posted by David E Jones <jo...@undersunconsulting.com>.
John,
Okay, that's a good distinction. You're not looking for regression or
system wide tests, which generally take enough time to run that
building a test database (about 3 minutes on Derby on my laptop) is
not a significant amount of time.
For quick tests like this I'd recommend creating test services that
do the build up, code run, and results test all at once and then just
roll back the transaction to do the cleanup. This tends to work
fairly well and leverages existing transaction functionality that is
pretty solid and does the "undo" easily. This is a lot easier than
trying to save the state of the db and restoring it in the tear down.
Of course, if you're not worried about developer efficiency, then
code away and have a good time... ;)
-David
On Oct 24, 2006, at 6:07 PM, John Martin wrote:
> Hi David,
>
> I think that the "throw away" database idea is probably out of the
> question for my needs. I code in an iterative process and need to
> repeat the tests through the coding/debugging cycle. When the code is
> complete, so is the test suite for that service or set of services
> which is then available for regression testing in the system.
>
> During the development cycle I could run the tests 10-30 or more times
> depending on the complexity. If I understand your solution, one would
> need to repopulate the database after each iteration which would be
> very time consuming. In addition, I could see issues with the XML
> entity comparision with createXXX and lastUpdatedXXX datetime columns
> as well as the autoincrement id columns.
>
> What I am really looking for is the junit functionality where the test
> suite has a build up and tear down section that is called before/after
> the tests. If this build up / tear down could be driven with XML,
> that would be best.
>
> It seems that the dbUnit project is more in line with what I am
> looking for. I'm going to explore this option and will colaborate
> with Tim on this.
>
> Thanks,
>
> John
>
> On 10/24/06, David E Jones <jo...@undersunconsulting.com> wrote:
>>
>> John,
>>
>> Part of the test plan for OFBiz (that has been around for well over a
>> year has not seen much attention) is to be able to easily to data
>> level testing. We plan to use the OOTB demo and seed data as the
>> initial database state, and then after tests are done the test
>> database would simply be thrown away (avoiding the need to "undo" all
>> test changes).
>>
>> To validate results on the data level there is an existing feature in
>> the simple-method that we plan to make easier to use in the test
>> config XML files. What it does is read an entity engine XML file
>> (which can exported from WebTools, or created manually) and validate
>> it against the database by asserting that each element in the XML
>> file matches a database record. It reports records that are missing
>> and all fields that are different for existing records.
>>
>> Is that more or less what you are looking for? The intent with these
>> is to be intermixed with UI level tests and test services and such to
>> provide testing facilities for the various architectural layers in
>> OFBiz.
>>
>> -David
>>
>>
>> On Oct 24, 2006, at 6:54 AM, John Martin wrote:
>>
>> > Grinder does look really interesting and appears to me to be geared
>> > more for testing the application for a UI and performance
>> perspective.
>> >
>> > What I am really after is something that is more database
>> oriented to
>> > allow a test suite to initialize data appropriately, run the
>> test, and
>> > then clean out the test data. I'm writing new services to
>> > insert/update data and want to have a simple way of building
>> tests to
>> > make sure that my code is working correctly.
>> >
>> > Currently, I write a test script that invokes the service and I
>> > inspect the results and walk through the code in the debugger. I
>> > agree with Brett that "creating data driven tests were still
>> difficult
>> > to create and maintain." There's got to be an easier way...
>> >
>> > Brett - I would be interested in the JXUnit integration with ofBiz
>> > since that appears to be inline with what I'm trying to do. Can
>> point
>> > me to where I can get what I would need to use it?
>> >
>> > Thanks,
>> >
>> > John
>>
>>
Re: Database Test Suite
Posted by John Martin <pb...@gmail.com>.
Hi David,
I think that the "throw away" database idea is probably out of the
question for my needs. I code in an iterative process and need to
repeat the tests through the coding/debugging cycle. When the code is
complete, so is the test suite for that service or set of services
which is then available for regression testing in the system.
During the development cycle I could run the tests 10-30 or more times
depending on the complexity. If I understand your solution, one would
need to repopulate the database after each iteration which would be
very time consuming. In addition, I could see issues with the XML
entity comparision with createXXX and lastUpdatedXXX datetime columns
as well as the autoincrement id columns.
What I am really looking for is the junit functionality where the test
suite has a build up and tear down section that is called before/after
the tests. If this build up / tear down could be driven with XML,
that would be best.
It seems that the dbUnit project is more in line with what I am
looking for. I'm going to explore this option and will colaborate
with Tim on this.
Thanks,
John
On 10/24/06, David E Jones <jo...@undersunconsulting.com> wrote:
>
> John,
>
> Part of the test plan for OFBiz (that has been around for well over a
> year has not seen much attention) is to be able to easily to data
> level testing. We plan to use the OOTB demo and seed data as the
> initial database state, and then after tests are done the test
> database would simply be thrown away (avoiding the need to "undo" all
> test changes).
>
> To validate results on the data level there is an existing feature in
> the simple-method that we plan to make easier to use in the test
> config XML files. What it does is read an entity engine XML file
> (which can exported from WebTools, or created manually) and validate
> it against the database by asserting that each element in the XML
> file matches a database record. It reports records that are missing
> and all fields that are different for existing records.
>
> Is that more or less what you are looking for? The intent with these
> is to be intermixed with UI level tests and test services and such to
> provide testing facilities for the various architectural layers in
> OFBiz.
>
> -David
>
>
> On Oct 24, 2006, at 6:54 AM, John Martin wrote:
>
> > Grinder does look really interesting and appears to me to be geared
> > more for testing the application for a UI and performance perspective.
> >
> > What I am really after is something that is more database oriented to
> > allow a test suite to initialize data appropriately, run the test, and
> > then clean out the test data. I'm writing new services to
> > insert/update data and want to have a simple way of building tests to
> > make sure that my code is working correctly.
> >
> > Currently, I write a test script that invokes the service and I
> > inspect the results and walk through the code in the debugger. I
> > agree with Brett that "creating data driven tests were still difficult
> > to create and maintain." There's got to be an easier way...
> >
> > Brett - I would be interested in the JXUnit integration with ofBiz
> > since that appears to be inline with what I'm trying to do. Can point
> > me to where I can get what I would need to use it?
> >
> > Thanks,
> >
> > John
>
>
Re: Database Test Suite
Posted by David E Jones <jo...@undersunconsulting.com>.
John,
Part of the test plan for OFBiz (that has been around for well over a
year has not seen much attention) is to be able to easily to data
level testing. We plan to use the OOTB demo and seed data as the
initial database state, and then after tests are done the test
database would simply be thrown away (avoiding the need to "undo" all
test changes).
To validate results on the data level there is an existing feature in
the simple-method that we plan to make easier to use in the test
config XML files. What it does is read an entity engine XML file
(which can exported from WebTools, or created manually) and validate
it against the database by asserting that each element in the XML
file matches a database record. It reports records that are missing
and all fields that are different for existing records.
Is that more or less what you are looking for? The intent with these
is to be intermixed with UI level tests and test services and such to
provide testing facilities for the various architectural layers in
OFBiz.
-David
On Oct 24, 2006, at 6:54 AM, John Martin wrote:
> Grinder does look really interesting and appears to me to be geared
> more for testing the application for a UI and performance perspective.
>
> What I am really after is something that is more database oriented to
> allow a test suite to initialize data appropriately, run the test, and
> then clean out the test data. I'm writing new services to
> insert/update data and want to have a simple way of building tests to
> make sure that my code is working correctly.
>
> Currently, I write a test script that invokes the service and I
> inspect the results and walk through the code in the debugger. I
> agree with Brett that "creating data driven tests were still difficult
> to create and maintain." There's got to be an easier way...
>
> Brett - I would be interested in the JXUnit integration with ofBiz
> since that appears to be inline with what I'm trying to do. Can point
> me to where I can get what I would need to use it?
>
> Thanks,
>
> John
Re: Database Test Suite
Posted by John Martin <pb...@gmail.com>.
Grinder does look really interesting and appears to me to be geared
more for testing the application for a UI and performance perspective.
What I am really after is something that is more database oriented to
allow a test suite to initialize data appropriately, run the test, and
then clean out the test data. I'm writing new services to
insert/update data and want to have a simple way of building tests to
make sure that my code is working correctly.
Currently, I write a test script that invokes the service and I
inspect the results and walk through the code in the debugger. I
agree with Brett that "creating data driven tests were still difficult
to create and maintain." There's got to be an easier way...
Brett - I would be interested in the JXUnit integration with ofBiz
since that appears to be inline with what I'm trying to do. Can point
me to where I can get what I would need to use it?
Thanks,
John
Re: Database Test Suite
Posted by Tim Ruppert <ti...@hotwaxmedia.com>.
Brett, I've looked into Grinder earlier this year (albeit only for an
hour or so) and was pleasantly surprised by it. We used something
called qftestJUI (http://www.qfs.de/en/qftestJUI/index.html) while
developing a swing front end to a J2EE app - which had a lot of the
record and playback stuff that we were looking for.
Grinder seems to give you a lot of that, but some additional load
testing stuff as well. Is that about right? Does anyone know how
Grinder compares to Selinium and it's new Firefox plugin?
Cheers,
Tim
--
Tim Ruppert
HotWax Media
http://www.hotwaxmedia.com
o:801.649.6594
f:801.649.6595
Brett Palmer wrote:
> Sorry, there has not been a lot of work done on the JXUnit stuff
> since its original creation. The JXUnit project did not get a lot of
> support on sourceforge and the original contributors moved on to other
> projects.
>
> I continued to use the ofbiz jxunit code for some of my own projects
> but creating data driven tests were still difficult to create and
> maintain.
>
> Over the past year, I have been working with a different framework
> called grinder (http://grinder.sourceforge.net/). There have been a
> few posts on the subject from David Jones and Al Byers. Al has
> recently been working on it and we have shared some ideas on how to
> make it effective for ofbiz.
>
> Here are some of the features that I like about grinder:
>
> - Uses Jython as a the scripting language to implement tests. This
> makes it easier to create new tests without having to compile and
> deploy each time.
> - TCP Proxy listener to capture web requests and generates a test
> script. This script can then be modified to be more dynamic
> - Ability to simulate multiple clients with various load and frequency
> options
>
>
> Here are some things that are missing and what Al and I have been
> discussing:
>
> - Need to be able to separate data from the test
> - Need to develop some jython utilities that can read in a common XML
> (or something else) format for a test and pass the data to the jython
> test script
>
>
> Al and David can probably share some of their ideas on testing. We
> need to have more test tools but it is always a matter of priorities
> and everyone is always busy with other tasks.
>
> Thoughts?
>
>
> Brett
>
> On 10/23/06, Tim Ruppert <ti...@hotwaxmedia.com> wrote:
>> I was a committer for DbUnit back in the day, which does stage
>> information nicely into the database for running tests. We used it in
>> conjunction with WebTest to drive all testing from ant. It has worked
>> well in the past, but might not be the most up to date solution.
>>
>> The only concern that I would have on this front is that there may be
>> too much data to stage, With large amounts of data, the DbUnit approach
>> could run into some performance issues with certain data sets. It might
>> still be a viable approach for you John depending on your situation.
>>
>> I'd still love to hear more about Brett's JXUnit experience whenever he
>> gets a chance.
>>
>> Cheers,
>> Tim
>>
>> --
>> Tim Ruppert
>> HotWax Media
>> http://www.hotwaxmedia.com
>>
>> o:801.649.6594
>> f:801.649.6595
>>
>> John Martin wrote:
>> > I'm looking for a good way of testing service methods that do CRUD on
>> > various tables in the database.
>> >
>> > I reviewed the framework/testtools and it didn't seem that there was
>> > much if anything going on there nor did it appear to address
>> > manipulation of test database data.
>> >
>> > While poking around I ran into ProposedOFBizTestFramework document by
>> > Brett Palmer that uses JXUnit
>> >
>> (http://ofbizwiki.go-integral.com/Wiki.jsp?page=ProposedOFBizTestFramework
>>
>> >
>> > ).
>> >
>> > This document looks quite promising and seems like it is exactly what
>> > I am looking for but the ofbiz-test that it referenced no longer
>> > exists at SourceForge. Is this a dead project? If not, where is it
>> > now located?
>> >
>> > What is the best practices for writing a test suite for new services
>> > that are manipulating data?
>> >
>> > Thanks!
>> >
>> > John
>>