More on testing

[Al Byers <by...@automationgroups.com>]

I would like to see discussion about testing kept alive, so I have
taken the unprecedented step of doing some investigation on my own -
but not much; I am counting on the community picking this up and
helping us get to a useful OFBiz standard solution.

As I learn more, I see that the capture/playback mode of Grinder is
very limited. The next step up seems to be WebTest with its ability to
script tests. WebTest, essentially, emulates a browser and looks like
it would be very eary to use - especially for acceptance testing.
Grinder may be a better tool for performance testing.

Selenium goes one step further and, instead of emulating a browser, it
installs a Javascript shell in the browser (after it starts the
browser automatically from the selenium server) and receives requests
from the server (request that you send to the server, either
interactively or via scripts), causes the browser, itself, to execute
those requests, thereby testing the web application under test.

There seems to be much more to getting a selenium environment working,
but it would appear that the testing would be more rigourous. This is
where the size of the OFBiz community should help. Whereas, it may not
be cost-effective for one party to get selenium working for their own
use, we should be able to tap into the experience of the entire
community to get a ready cookbook for setting up selenium and for
writing tests for various standard modules. Also, there would be many
reusable subtest and techniques that could be shared.

It would seem that Selenium would be the way to go as OFBiz interfaces
start using more AJAX interactions - because it uses actual browsers
to run the tests.

Of course, I am getting ahead of myself - annointing a winner before
there has been a consensus.

Anyone have some experience/s to share?

-Al

Re: More on testing

[Tim Ruppert <ti...@hotwaxmedia.com>]

As many of you know, I have been planning on doing these tests for a 
while and have just been swamped.  I promise to get back to them at the 
beginning of next week!.  David Welton makes a good point about wanting 
to be able to test the non-presentation layer components outside of the 
browser - both Webtest and Selenium can help on this - but we should be 
able to put some sort of xUnit tests in place to fill in the gaps that 
you're seeing.

Al, you are right about the setup on Selenium - it does take a little 
bit longer to get one's head around and will probably be a big help with 
some of the AJAX components - but in my experience it's been less than 
intuitive to get going and using.

Personally, I still think that Webtest should be able to manage most of 
the stuff that we need to do as it's ant driven, emulates a browser, and 
potentially could deal with AJAX in a clean way - you just won't be 
changing pages.  It has a Javascript engine built in, but this is where 
I ran into problems a few years ago and unfortunately stopped using it 
every day :(

In short, I will pick up this ball again and get some information out to 
everyone ASAP.

Cheers,
Tim

--
Tim Ruppert
HotWax Media
http://www.hotwaxmedia.com

o:801.649.6594
f:801.649.6595

Al Byers wrote:
> I would like to see discussion about testing kept alive, so I have
> taken the unprecedented step of doing some investigation on my own -
> but not much; I am counting on the community picking this up and
> helping us get to a useful OFBiz standard solution.
>
> As I learn more, I see that the capture/playback mode of Grinder is
> very limited. The next step up seems to be WebTest with its ability to
> script tests. WebTest, essentially, emulates a browser and looks like
> it would be very eary to use - especially for acceptance testing.
> Grinder may be a better tool for performance testing.
>
> Selenium goes one step further and, instead of emulating a browser, it
> installs a Javascript shell in the browser (after it starts the
> browser automatically from the selenium server) and receives requests
> from the server (request that you send to the server, either
> interactively or via scripts), causes the browser, itself, to execute
> those requests, thereby testing the web application under test.
>
> There seems to be much more to getting a selenium environment working,
> but it would appear that the testing would be more rigourous. This is
> where the size of the OFBiz community should help. Whereas, it may not
> be cost-effective for one party to get selenium working for their own
> use, we should be able to tap into the experience of the entire
> community to get a ready cookbook for setting up selenium and for
> writing tests for various standard modules. Also, there would be many
> reusable subtest and techniques that could be shared.
>
> It would seem that Selenium would be the way to go as OFBiz interfaces
> start using more AJAX interactions - because it uses actual browsers
> to run the tests.
>
> Of course, I am getting ahead of myself - annointing a winner before
> there has been a consensus.
>
> Anyone have some experience/s to share?
>
> -Al

Re: More on testing

[Tim Ruppert <ti...@hotwaxmedia.com>]

Brett, this is all great news.  We should chat about these sometime as I 
have a framework for making these kind of things easy in the build 
process.  It won't get around the startup issues, but I may be able to 
be of service.

Cheers,
Tim

--
Tim Ruppert
HotWax Media
http://www.hotwaxmedia.com

o:801.649.6594
f:801.649.6595

Brett Palmer wrote:
> David,
>
> I agree that there should be a method for testing the business logic
> layer separate from the presentation layer.  Here are some of my
> suggestions:
>
> Presentation Layer Testing
>
> - Use a tool like WebTest, Selenium, or grinder
> - The key to these tests should be ease to setup and execute.  If they
> are too difficult no one will use them
>
> Business Logic Layer
>
> - In the ofbiz world, this is the service layer.
> - The easiest method I have found to test this is to use regular JUnit
> tests and export the service you want to tests.  Some of the drawbacks
> to this approach are as follows:
> ---> requires the service to be set as exportable which is means you
> have to manually switch them back to non-exportable if you want to run
> them in production
> ---> RMI Serialization incompatibilities when you compile ofbiz from
> Ant but run your JUnit test from an IDE like eclipse.  You can work
> around these issues but that is a topic for further discussion
>
>
> Unit Testing
>
> - These would be JUnit tests at the component level like entity and
> service engine.  I haven't found this level of testing that productive
> as the framework is mature.  It would be  a good idea to have the
> community develop key unit tests for new features added to these
> components.
>
>
> Automated JUnit test with Ant
>
> OFBiz has a mechanism to run automated tests in the ant build.  Here
> are my thoughts and feedback on this type of testing.
>
> Good
> - These are very helpful for automated build tests and for sharing
> tests within the community.
> - They are also helpful if you want to run something like
> CruiseControl each night to verify that your builds are working
> correctly
>
> Bad
> - They take too long to execute because the test has to run through an
> entire build, then startup up the ofbiz app server, and then execute
> the tests.
> - One of the key objectives of XP is to be able to test frequently and
> often and this means they have to execute quickly.
>
> I am currently working on some JUnit tests that will work in the
> automated build but can also be run as a standalone client to tests
> against an exported service.
>
>
> Feedback is always welcome.
>
>
> Brett
>
> On 12/2/06, David Welton <da...@gmail.com> wrote:
>> Quick thought of my own:
>>
>> I'd suggest having a look at something that can be run from the
>> command line to test the controller logic with the default store's
>> data, which could be added to as needs to be.  It would IMO be handy
>> to be able to skip the presentation layer in order to test the logic
>> without worrying about the browser.
>>
>> -- 
>> David N. Welton
>>  - http://www.dedasys.com/davidw/
>>
>> Linux, Open Source Consulting
>>  - http://www.dedasys.com/
>>

Re: More on testing

[Brett Palmer <br...@gmail.com>]

David,

I agree that there should be a method for testing the business logic
layer separate from the presentation layer.  Here are some of my
suggestions:

Presentation Layer Testing

- Use a tool like WebTest, Selenium, or grinder
- The key to these tests should be ease to setup and execute.  If they
are too difficult no one will use them

Business Logic Layer

- In the ofbiz world, this is the service layer.
- The easiest method I have found to test this is to use regular JUnit
tests and export the service you want to tests.  Some of the drawbacks
to this approach are as follows:
---> requires the service to be set as exportable which is means you
have to manually switch them back to non-exportable if you want to run
them in production
---> RMI Serialization incompatibilities when you compile ofbiz from
Ant but run your JUnit test from an IDE like eclipse.  You can work
around these issues but that is a topic for further discussion


Unit Testing

- These would be JUnit tests at the component level like entity and
service engine.  I haven't found this level of testing that productive
as the framework is mature.  It would be  a good idea to have the
community develop key unit tests for new features added to these
components.


Automated JUnit test with Ant

OFBiz has a mechanism to run automated tests in the ant build.  Here
are my thoughts and feedback on this type of testing.

Good
- These are very helpful for automated build tests and for sharing
tests within the community.
- They are also helpful if you want to run something like
CruiseControl each night to verify that your builds are working
correctly

Bad
- They take too long to execute because the test has to run through an
entire build, then startup up the ofbiz app server, and then execute
the tests.
- One of the key objectives of XP is to be able to test frequently and
often and this means they have to execute quickly.

I am currently working on some JUnit tests that will work in the
automated build but can also be run as a standalone client to tests
against an exported service.


Feedback is always welcome.


Brett

On 12/2/06, David Welton <da...@gmail.com> wrote:
> Quick thought of my own:
>
> I'd suggest having a look at something that can be run from the
> command line to test the controller logic with the default store's
> data, which could be added to as needs to be.  It would IMO be handy
> to be able to skip the presentation layer in order to test the logic
> without worrying about the browser.
>
> --
> David N. Welton
>  - http://www.dedasys.com/davidw/
>
> Linux, Open Source Consulting
>  - http://www.dedasys.com/
>

Re: More on testing

[David Welton <da...@gmail.com>]

Quick thought of my own:

I'd suggest having a look at something that can be run from the
command line to test the controller logic with the default store's
data, which could be added to as needs to be.  It would IMO be handy
to be able to skip the presentation layer in order to test the logic
without worrying about the browser.

-- 
David N. Welton
 - http://www.dedasys.com/davidw/

Linux, Open Source Consulting
 - http://www.dedasys.com/