Proposal to modify the testing framework for OFBiz

[Taher Alkhateeb <sl...@gmail.com>]

Hello Everyone,

After refactoring the start component and while starting on the base
component I realized that the testing framework for OFBiz is not good. You
cannot do real test driven development or red-green-refactor with the
current setup, hence my proposal to change it. I explain below:

Problem with current design
----------------------------------------
- What we have right now is not unit tests, it's really integration tests.
You have to start the framework, the database, the service engine, the
entity engine and pretty much everything.
- Testing is very slow, because it's an integration test as I mentioned
above. 10 minutes on a good computer!
- There is zero mocking! We actually have to --load-data for things to
work. Again, these are integration tests.
- Too complex: Integration tests by their nature are grabbing too much.
Mind you, I am not objecting to integration tests (I actually like them)
but I am objecting to not having real unit-tests. Unit tests should all run
in a few seconds.

Proposed solution
--------------------------
- We keep what is considered real integration tests the way they are right
now and keep using them
- We move what should be unit tests into simple JUnit classes, and we do
not run them using java -jar ofbiz.jar --test, but instead run them
directly from the build.xml script, so these files are not identified in
any XML document, but are simply called immediately from the build scripts.
- We clearly mark the difference between integration tests and unit tests
(inside the source files or in the suite declarations).
- We change the run-tests target in build.xml to run both unit tests and
integration tests.

I intend to heavily refactor the framework and I would feel better about
introducing this change while refactoring. What do you guys think? Ideas?
Suggestions? Approvals and thumbs up?

Regards,

Taher Alkhateeb

Re: Proposal to modify the testing framework for OFBiz

[gil portenseigne <gi...@nereide.fr>]

Same here : +1

Gil

On 08/06/2016 11:24, Jacques Le Roux wrote:
> That sounds like a more than reasonable proposition to me
>
> + 1
>
> Jacques
>
>
> Le 08/06/2016 � 11:11, Taher Alkhateeb a �crit :
>> Hello Everyone,
>>
>> After refactoring the start component and while starting on the base
>> component I realized that the testing framework for OFBiz is not 
>> good. You
>> cannot do real test driven development or red-green-refactor with the
>> current setup, hence my proposal to change it. I explain below:
>>
>> Problem with current design
>> ----------------------------------------
>> - What we have right now is not unit tests, it's really integration 
>> tests.
>> You have to start the framework, the database, the service engine, the
>> entity engine and pretty much everything.
>> - Testing is very slow, because it's an integration test as I mentioned
>> above. 10 minutes on a good computer!
>> - There is zero mocking! We actually have to --load-data for things to
>> work. Again, these are integration tests.
>> - Too complex: Integration tests by their nature are grabbing too much.
>> Mind you, I am not objecting to integration tests (I actually like them)
>> but I am objecting to not having real unit-tests. Unit tests should 
>> all run
>> in a few seconds.
>>
>> Proposed solution
>> --------------------------
>> - We keep what is considered real integration tests the way they are 
>> right
>> now and keep using them
>> - We move what should be unit tests into simple JUnit classes, and we do
>> not run them using java -jar ofbiz.jar --test, but instead run them
>> directly from the build.xml script, so these files are not identified in
>> any XML document, but are simply called immediately from the build 
>> scripts.
>> - We clearly mark the difference between integration tests and unit 
>> tests
>> (inside the source files or in the suite declarations).
>> - We change the run-tests target in build.xml to run both unit tests and
>> integration tests.
>>
>> I intend to heavily refactor the framework and I would feel better about
>> introducing this change while refactoring. What do you guys think? 
>> Ideas?
>> Suggestions? Approvals and thumbs up?
>>
>> Regards,
>>
>> Taher Alkhateeb
>>
>

Re: Proposal to modify the testing framework for OFBiz

[Jacques Le Roux <ja...@les7arts.com>]

That sounds like a more than reasonable proposition to me

+ 1

Jacques


Le 08/06/2016 � 11:11, Taher Alkhateeb a �crit :
> Hello Everyone,
>
> After refactoring the start component and while starting on the base
> component I realized that the testing framework for OFBiz is not good. You
> cannot do real test driven development or red-green-refactor with the
> current setup, hence my proposal to change it. I explain below:
>
> Problem with current design
> ----------------------------------------
> - What we have right now is not unit tests, it's really integration tests.
> You have to start the framework, the database, the service engine, the
> entity engine and pretty much everything.
> - Testing is very slow, because it's an integration test as I mentioned
> above. 10 minutes on a good computer!
> - There is zero mocking! We actually have to --load-data for things to
> work. Again, these are integration tests.
> - Too complex: Integration tests by their nature are grabbing too much.
> Mind you, I am not objecting to integration tests (I actually like them)
> but I am objecting to not having real unit-tests. Unit tests should all run
> in a few seconds.
>
> Proposed solution
> --------------------------
> - We keep what is considered real integration tests the way they are right
> now and keep using them
> - We move what should be unit tests into simple JUnit classes, and we do
> not run them using java -jar ofbiz.jar --test, but instead run them
> directly from the build.xml script, so these files are not identified in
> any XML document, but are simply called immediately from the build scripts.
> - We clearly mark the difference between integration tests and unit tests
> (inside the source files or in the suite declarations).
> - We change the run-tests target in build.xml to run both unit tests and
> integration tests.
>
> I intend to heavily refactor the framework and I would feel better about
> introducing this change while refactoring. What do you guys think? Ideas?
> Suggestions? Approvals and thumbs up?
>
> Regards,
>
> Taher Alkhateeb
>

Re: Proposal to modify the testing framework for OFBiz

[Jacopo Cappellato <ja...@hotwaxsystems.com>]

Totally agree that what we have right now is a set of integration tests,
not unit tests.
Big +1 to the proposal to implement unit tests and mocking.

Jacopo

On Wed, Jun 8, 2016 at 11:11 AM, Taher Alkhateeb <slidingfilaments@gmail.com
> wrote:

> Hello Everyone,
>
> After refactoring the start component and while starting on the base
> component I realized that the testing framework for OFBiz is not good. You
> cannot do real test driven development or red-green-refactor with the
> current setup, hence my proposal to change it. I explain below:
>
> Problem with current design
> ----------------------------------------
> - What we have right now is not unit tests, it's really integration tests.
> You have to start the framework, the database, the service engine, the
> entity engine and pretty much everything.
> - Testing is very slow, because it's an integration test as I mentioned
> above. 10 minutes on a good computer!
> - There is zero mocking! We actually have to --load-data for things to
> work. Again, these are integration tests.
> - Too complex: Integration tests by their nature are grabbing too much.
> Mind you, I am not objecting to integration tests (I actually like them)
> but I am objecting to not having real unit-tests. Unit tests should all run
> in a few seconds.
>
> Proposed solution
> --------------------------
> - We keep what is considered real integration tests the way they are right
> now and keep using them
> - We move what should be unit tests into simple JUnit classes, and we do
> not run them using java -jar ofbiz.jar --test, but instead run them
> directly from the build.xml script, so these files are not identified in
> any XML document, but are simply called immediately from the build scripts.
> - We clearly mark the difference between integration tests and unit tests
> (inside the source files or in the suite declarations).
> - We change the run-tests target in build.xml to run both unit tests and
> integration tests.
>
> I intend to heavily refactor the framework and I would feel better about
> introducing this change while refactoring. What do you guys think? Ideas?
> Suggestions? Approvals and thumbs up?
>
> Regards,
>
> Taher Alkhateeb
>

Re: Proposal to modify the testing framework for OFBiz

[Taher Alkhateeb <sl...@gmail.com>]

Hi Divesh,

Yes correct. To give you a metaphor, think of integration tests as taking
your car and driving it to make sure it works, and think of unit tests like
checking your car engine piece by piece in isolation by measuring and
inspecting every piece by itself.

You can read here for more information:
http://www.jamesshore.com/Blog/Red-Green-Refactor.html
https://en.wikipedia.org/wiki/Test-driven_development

Regards,

Taher Alkhateeb

On Wed, Jun 8, 2016 at 1:07 PM, Divesh Dutta <divesh.dutta@hotwaxsystems.com
> wrote:

> Hi Taher,
>
> What I understand from your proposal is, we will like to test a small unit
> which can be a Java method as well. And framework does not allows that. And
> you want to introduce such support in framework . Is that what you mean ?
>
> And if this what you are thinking, then I think its nice proposal.
>
> Also this article by Martin Fowler on Unit test is good read:
> http://martinfowler.com/bliki/UnitTest.html
>
>
> Thanks
> --
> Divesh Dutta.
>
> On Wed, Jun 8, 2016 at 3:02 PM, Taher Alkhateeb <
> slidingfilaments@gmail.com>
> wrote:
>
> > Hi Pranay,
> >
> > What you are talking about is integration tests (ones that you can run
> per
> > component)
> >
> > Unit tests do not need to be run per component. The entire unit test
> > library for the entire framework should run in probably less than 5
> > seconds. So there is no need to make them run per component.
> >
> > My proposal is to simply have a target like run-unit-tests which would
> run
> > all the unit tests. These tests do not talk to the database or server or
> > anything, You can run them while flying in an airplane 30,000 feet above
> > the ocean.
> >
> > Mind you I will not remove the integration tests, I'm just adding unit
> > tests (and probably moving some integration tests to unit tests).
> >
> > Naturally, my proposition means that those tests are only JUnit tests
> (for
> > now) but that is a quick solution that I think is rather clean because
> > we're not really radically modifying anything, we're just adding JUnit
> > classes and running them.
> >
> > Regards,
> >
> > Taher Alkhateeb
> >
> > On Wed, Jun 8, 2016 at 12:23 PM, Pranay Pandey <
> > pranay.pandey@hotwaxsystems.com> wrote:
> >
> > > Hi Taher,
> > >
> > > There is one thing that I want to confirm based my understanding, as
> long
> > > as I know, we can run tests independently as well e.g.
> > >
> > > *Running tests for a component: *
> > >
> > > java -jar ofbiz.jar -test component=‘order’
> > >
> > > *Running a specific test Suite from a component:*
> > >
> > > ant run-single-test-suite -Dtest.component=‘order’ -Dtest.suiteName='
> > > ordertests'
> > >
> > > *Running a test case from a component:*
> > >
> > > java -jar ofbiz.jar -test -component='order' -case='salesOrder-test'
> > >
> > > I may be completely wrong here but just want to confirm and understand
> > your
> > > proposal.
> > >
> > > Best regards,
> > >
> > > Pranay Pandey
> > > HotWax Systems
> > > http://www.hotwaxsystems.com/
> > >
> > > On Wed, Jun 8, 2016 at 2:41 PM, Taher Alkhateeb <
> > > slidingfilaments@gmail.com>
> > > wrote:
> > >
> > > > Hello Everyone,
> > > >
> > > > After refactoring the start component and while starting on the base
> > > > component I realized that the testing framework for OFBiz is not
> good.
> > > You
> > > > cannot do real test driven development or red-green-refactor with the
> > > > current setup, hence my proposal to change it. I explain below:
> > > >
> > > > Problem with current design
> > > > ----------------------------------------
> > > > - What we have right now is not unit tests, it's really integration
> > > tests.
> > > > You have to start the framework, the database, the service engine,
> the
> > > > entity engine and pretty much everything.
> > > > - Testing is very slow, because it's an integration test as I
> mentioned
> > > > above. 10 minutes on a good computer!
> > > > - There is zero mocking! We actually have to --load-data for things
> to
> > > > work. Again, these are integration tests.
> > > > - Too complex: Integration tests by their nature are grabbing too
> much.
> > > > Mind you, I am not objecting to integration tests (I actually like
> > them)
> > > > but I am objecting to not having real unit-tests. Unit tests should
> all
> > > run
> > > > in a few seconds.
> > > >
> > > > Proposed solution
> > > > --------------------------
> > > > - We keep what is considered real integration tests the way they are
> > > right
> > > > now and keep using them
> > > > - We move what should be unit tests into simple JUnit classes, and we
> > do
> > > > not run them using java -jar ofbiz.jar --test, but instead run them
> > > > directly from the build.xml script, so these files are not identified
> > in
> > > > any XML document, but are simply called immediately from the build
> > > scripts.
> > > > - We clearly mark the difference between integration tests and unit
> > tests
> > > > (inside the source files or in the suite declarations).
> > > > - We change the run-tests target in build.xml to run both unit tests
> > and
> > > > integration tests.
> > > >
> > > > I intend to heavily refactor the framework and I would feel better
> > about
> > > > introducing this change while refactoring. What do you guys think?
> > Ideas?
> > > > Suggestions? Approvals and thumbs up?
> > > >
> > > > Regards,
> > > >
> > > > Taher Alkhateeb
> > > >
> > >
> >
>

Re: Proposal to modify the testing framework for OFBiz

[Divesh Dutta <di...@hotwaxsystems.com>]

Hi Taher,

What I understand from your proposal is, we will like to test a small unit
which can be a Java method as well. And framework does not allows that. And
you want to introduce such support in framework . Is that what you mean ?

And if this what you are thinking, then I think its nice proposal.

Also this article by Martin Fowler on Unit test is good read:
http://martinfowler.com/bliki/UnitTest.html


Thanks
--
Divesh Dutta.

On Wed, Jun 8, 2016 at 3:02 PM, Taher Alkhateeb <sl...@gmail.com>
wrote:

> Hi Pranay,
>
> What you are talking about is integration tests (ones that you can run per
> component)
>
> Unit tests do not need to be run per component. The entire unit test
> library for the entire framework should run in probably less than 5
> seconds. So there is no need to make them run per component.
>
> My proposal is to simply have a target like run-unit-tests which would run
> all the unit tests. These tests do not talk to the database or server or
> anything, You can run them while flying in an airplane 30,000 feet above
> the ocean.
>
> Mind you I will not remove the integration tests, I'm just adding unit
> tests (and probably moving some integration tests to unit tests).
>
> Naturally, my proposition means that those tests are only JUnit tests (for
> now) but that is a quick solution that I think is rather clean because
> we're not really radically modifying anything, we're just adding JUnit
> classes and running them.
>
> Regards,
>
> Taher Alkhateeb
>
> On Wed, Jun 8, 2016 at 12:23 PM, Pranay Pandey <
> pranay.pandey@hotwaxsystems.com> wrote:
>
> > Hi Taher,
> >
> > There is one thing that I want to confirm based my understanding, as long
> > as I know, we can run tests independently as well e.g.
> >
> > *Running tests for a component: *
> >
> > java -jar ofbiz.jar -test component=‘order’
> >
> > *Running a specific test Suite from a component:*
> >
> > ant run-single-test-suite -Dtest.component=‘order’ -Dtest.suiteName='
> > ordertests'
> >
> > *Running a test case from a component:*
> >
> > java -jar ofbiz.jar -test -component='order' -case='salesOrder-test'
> >
> > I may be completely wrong here but just want to confirm and understand
> your
> > proposal.
> >
> > Best regards,
> >
> > Pranay Pandey
> > HotWax Systems
> > http://www.hotwaxsystems.com/
> >
> > On Wed, Jun 8, 2016 at 2:41 PM, Taher Alkhateeb <
> > slidingfilaments@gmail.com>
> > wrote:
> >
> > > Hello Everyone,
> > >
> > > After refactoring the start component and while starting on the base
> > > component I realized that the testing framework for OFBiz is not good.
> > You
> > > cannot do real test driven development or red-green-refactor with the
> > > current setup, hence my proposal to change it. I explain below:
> > >
> > > Problem with current design
> > > ----------------------------------------
> > > - What we have right now is not unit tests, it's really integration
> > tests.
> > > You have to start the framework, the database, the service engine, the
> > > entity engine and pretty much everything.
> > > - Testing is very slow, because it's an integration test as I mentioned
> > > above. 10 minutes on a good computer!
> > > - There is zero mocking! We actually have to --load-data for things to
> > > work. Again, these are integration tests.
> > > - Too complex: Integration tests by their nature are grabbing too much.
> > > Mind you, I am not objecting to integration tests (I actually like
> them)
> > > but I am objecting to not having real unit-tests. Unit tests should all
> > run
> > > in a few seconds.
> > >
> > > Proposed solution
> > > --------------------------
> > > - We keep what is considered real integration tests the way they are
> > right
> > > now and keep using them
> > > - We move what should be unit tests into simple JUnit classes, and we
> do
> > > not run them using java -jar ofbiz.jar --test, but instead run them
> > > directly from the build.xml script, so these files are not identified
> in
> > > any XML document, but are simply called immediately from the build
> > scripts.
> > > - We clearly mark the difference between integration tests and unit
> tests
> > > (inside the source files or in the suite declarations).
> > > - We change the run-tests target in build.xml to run both unit tests
> and
> > > integration tests.
> > >
> > > I intend to heavily refactor the framework and I would feel better
> about
> > > introducing this change while refactoring. What do you guys think?
> Ideas?
> > > Suggestions? Approvals and thumbs up?
> > >
> > > Regards,
> > >
> > > Taher Alkhateeb
> > >
> >
>

Re: Proposal to modify the testing framework for OFBiz

[Pierre Smits <pi...@gmail.com>]

+1

I suggest the first thing to start with is bring the proposed solution over
to an improvement issue in JIRA and work from there.

Best regards,

Pierre Smits

ORRTIZ.COM <http://www.orrtiz.com>
OFBiz based solutions & services

OFBiz Extensions Marketplace
http://oem.ofbizci.net/oci-2/

On Wed, Jun 8, 2016 at 1:45 PM, Nicolas Malin <ni...@nereide.fr>
wrote:

> I'm on the same case, +1
>
>
> Le 08/06/2016 13:42, Pranay Pandey a écrit :
>
>> thanks so much Taher for the explaining it further, makes perfect sense.
>>
>>
>> Best regards,
>>
>> Pranay Pandey
>> HotWax Systems
>> http://www.hotwaxsystems.com/
>>
>> On Wed, Jun 8, 2016 at 3:02 PM, Taher Alkhateeb <
>> slidingfilaments@gmail.com>
>> wrote:
>>
>> Hi Pranay,
>>>
>>> What you are talking about is integration tests (ones that you can run
>>> per
>>> component)
>>>
>>> Unit tests do not need to be run per component. The entire unit test
>>> library for the entire framework should run in probably less than 5
>>> seconds. So there is no need to make them run per component.
>>>
>>> My proposal is to simply have a target like run-unit-tests which would
>>> run
>>> all the unit tests. These tests do not talk to the database or server or
>>> anything, You can run them while flying in an airplane 30,000 feet above
>>> the ocean.
>>>
>>> Mind you I will not remove the integration tests, I'm just adding unit
>>> tests (and probably moving some integration tests to unit tests).
>>>
>>> Naturally, my proposition means that those tests are only JUnit tests
>>> (for
>>> now) but that is a quick solution that I think is rather clean because
>>> we're not really radically modifying anything, we're just adding JUnit
>>> classes and running them.
>>>
>>> Regards,
>>>
>>> Taher Alkhateeb
>>>
>>> On Wed, Jun 8, 2016 at 12:23 PM, Pranay Pandey <
>>> pranay.pandey@hotwaxsystems.com> wrote:
>>>
>>> Hi Taher,
>>>>
>>>> There is one thing that I want to confirm based my understanding, as
>>>> long
>>>> as I know, we can run tests independently as well e.g.
>>>>
>>>> *Running tests for a component: *
>>>>
>>>> java -jar ofbiz.jar -test component=‘order’
>>>>
>>>> *Running a specific test Suite from a component:*
>>>>
>>>> ant run-single-test-suite -Dtest.component=‘order’ -Dtest.suiteName='
>>>> ordertests'
>>>>
>>>> *Running a test case from a component:*
>>>>
>>>> java -jar ofbiz.jar -test -component='order' -case='salesOrder-test'
>>>>
>>>> I may be completely wrong here but just want to confirm and understand
>>>>
>>> your
>>>
>>>> proposal.
>>>>
>>>> Best regards,
>>>>
>>>> Pranay Pandey
>>>> HotWax Systems
>>>> http://www.hotwaxsystems.com/
>>>>
>>>> On Wed, Jun 8, 2016 at 2:41 PM, Taher Alkhateeb <
>>>> slidingfilaments@gmail.com>
>>>> wrote:
>>>>
>>>> Hello Everyone,
>>>>>
>>>>> After refactoring the start component and while starting on the base
>>>>> component I realized that the testing framework for OFBiz is not good.
>>>>>
>>>> You
>>>>
>>>>> cannot do real test driven development or red-green-refactor with the
>>>>> current setup, hence my proposal to change it. I explain below:
>>>>>
>>>>> Problem with current design
>>>>> ----------------------------------------
>>>>> - What we have right now is not unit tests, it's really integration
>>>>>
>>>> tests.
>>>>
>>>>> You have to start the framework, the database, the service engine, the
>>>>> entity engine and pretty much everything.
>>>>> - Testing is very slow, because it's an integration test as I mentioned
>>>>> above. 10 minutes on a good computer!
>>>>> - There is zero mocking! We actually have to --load-data for things to
>>>>> work. Again, these are integration tests.
>>>>> - Too complex: Integration tests by their nature are grabbing too much.
>>>>> Mind you, I am not objecting to integration tests (I actually like
>>>>>
>>>> them)
>>>
>>>> but I am objecting to not having real unit-tests. Unit tests should all
>>>>>
>>>> run
>>>>
>>>>> in a few seconds.
>>>>>
>>>>> Proposed solution
>>>>> --------------------------
>>>>> - We keep what is considered real integration tests the way they are
>>>>>
>>>> right
>>>>
>>>>> now and keep using them
>>>>> - We move what should be unit tests into simple JUnit classes, and we
>>>>>
>>>> do
>>>
>>>> not run them using java -jar ofbiz.jar --test, but instead run them
>>>>> directly from the build.xml script, so these files are not identified
>>>>>
>>>> in
>>>
>>>> any XML document, but are simply called immediately from the build
>>>>>
>>>> scripts.
>>>>
>>>>> - We clearly mark the difference between integration tests and unit
>>>>>
>>>> tests
>>>
>>>> (inside the source files or in the suite declarations).
>>>>> - We change the run-tests target in build.xml to run both unit tests
>>>>>
>>>> and
>>>
>>>> integration tests.
>>>>>
>>>>> I intend to heavily refactor the framework and I would feel better
>>>>>
>>>> about
>>>
>>>> introducing this change while refactoring. What do you guys think?
>>>>>
>>>> Ideas?
>>>
>>>> Suggestions? Approvals and thumbs up?
>>>>>
>>>>> Regards,
>>>>>
>>>>> Taher Alkhateeb
>>>>>
>>>>>
>

Re: Proposal to modify the testing framework for OFBiz

[Nicolas Malin <ni...@nereide.fr>]

I'm on the same case, +1

Le 08/06/2016 13:42, Pranay Pandey a �crit :
> thanks so much Taher for the explaining it further, makes perfect sense.
>
>
> Best regards,
>
> Pranay Pandey
> HotWax Systems
> http://www.hotwaxsystems.com/
>
> On Wed, Jun 8, 2016 at 3:02 PM, Taher Alkhateeb <sl...@gmail.com>
> wrote:
>
>> Hi Pranay,
>>
>> What you are talking about is integration tests (ones that you can run per
>> component)
>>
>> Unit tests do not need to be run per component. The entire unit test
>> library for the entire framework should run in probably less than 5
>> seconds. So there is no need to make them run per component.
>>
>> My proposal is to simply have a target like run-unit-tests which would run
>> all the unit tests. These tests do not talk to the database or server or
>> anything, You can run them while flying in an airplane 30,000 feet above
>> the ocean.
>>
>> Mind you I will not remove the integration tests, I'm just adding unit
>> tests (and probably moving some integration tests to unit tests).
>>
>> Naturally, my proposition means that those tests are only JUnit tests (for
>> now) but that is a quick solution that I think is rather clean because
>> we're not really radically modifying anything, we're just adding JUnit
>> classes and running them.
>>
>> Regards,
>>
>> Taher Alkhateeb
>>
>> On Wed, Jun 8, 2016 at 12:23 PM, Pranay Pandey <
>> pranay.pandey@hotwaxsystems.com> wrote:
>>
>>> Hi Taher,
>>>
>>> There is one thing that I want to confirm based my understanding, as long
>>> as I know, we can run tests independently as well e.g.
>>>
>>> *Running tests for a component: *
>>>
>>> java -jar ofbiz.jar -test component=\u2018order\u2019
>>>
>>> *Running a specific test Suite from a component:*
>>>
>>> ant run-single-test-suite -Dtest.component=\u2018order\u2019 -Dtest.suiteName='
>>> ordertests'
>>>
>>> *Running a test case from a component:*
>>>
>>> java -jar ofbiz.jar -test -component='order' -case='salesOrder-test'
>>>
>>> I may be completely wrong here but just want to confirm and understand
>> your
>>> proposal.
>>>
>>> Best regards,
>>>
>>> Pranay Pandey
>>> HotWax Systems
>>> http://www.hotwaxsystems.com/
>>>
>>> On Wed, Jun 8, 2016 at 2:41 PM, Taher Alkhateeb <
>>> slidingfilaments@gmail.com>
>>> wrote:
>>>
>>>> Hello Everyone,
>>>>
>>>> After refactoring the start component and while starting on the base
>>>> component I realized that the testing framework for OFBiz is not good.
>>> You
>>>> cannot do real test driven development or red-green-refactor with the
>>>> current setup, hence my proposal to change it. I explain below:
>>>>
>>>> Problem with current design
>>>> ----------------------------------------
>>>> - What we have right now is not unit tests, it's really integration
>>> tests.
>>>> You have to start the framework, the database, the service engine, the
>>>> entity engine and pretty much everything.
>>>> - Testing is very slow, because it's an integration test as I mentioned
>>>> above. 10 minutes on a good computer!
>>>> - There is zero mocking! We actually have to --load-data for things to
>>>> work. Again, these are integration tests.
>>>> - Too complex: Integration tests by their nature are grabbing too much.
>>>> Mind you, I am not objecting to integration tests (I actually like
>> them)
>>>> but I am objecting to not having real unit-tests. Unit tests should all
>>> run
>>>> in a few seconds.
>>>>
>>>> Proposed solution
>>>> --------------------------
>>>> - We keep what is considered real integration tests the way they are
>>> right
>>>> now and keep using them
>>>> - We move what should be unit tests into simple JUnit classes, and we
>> do
>>>> not run them using java -jar ofbiz.jar --test, but instead run them
>>>> directly from the build.xml script, so these files are not identified
>> in
>>>> any XML document, but are simply called immediately from the build
>>> scripts.
>>>> - We clearly mark the difference between integration tests and unit
>> tests
>>>> (inside the source files or in the suite declarations).
>>>> - We change the run-tests target in build.xml to run both unit tests
>> and
>>>> integration tests.
>>>>
>>>> I intend to heavily refactor the framework and I would feel better
>> about
>>>> introducing this change while refactoring. What do you guys think?
>> Ideas?
>>>> Suggestions? Approvals and thumbs up?
>>>>
>>>> Regards,
>>>>
>>>> Taher Alkhateeb
>>>>

Re: Proposal to modify the testing framework for OFBiz

[Pranay Pandey <pr...@hotwaxsystems.com>]

thanks so much Taher for the explaining it further, makes perfect sense.


Best regards,

Pranay Pandey
HotWax Systems
http://www.hotwaxsystems.com/

On Wed, Jun 8, 2016 at 3:02 PM, Taher Alkhateeb <sl...@gmail.com>
wrote:

> Hi Pranay,
>
> What you are talking about is integration tests (ones that you can run per
> component)
>
> Unit tests do not need to be run per component. The entire unit test
> library for the entire framework should run in probably less than 5
> seconds. So there is no need to make them run per component.
>
> My proposal is to simply have a target like run-unit-tests which would run
> all the unit tests. These tests do not talk to the database or server or
> anything, You can run them while flying in an airplane 30,000 feet above
> the ocean.
>
> Mind you I will not remove the integration tests, I'm just adding unit
> tests (and probably moving some integration tests to unit tests).
>
> Naturally, my proposition means that those tests are only JUnit tests (for
> now) but that is a quick solution that I think is rather clean because
> we're not really radically modifying anything, we're just adding JUnit
> classes and running them.
>
> Regards,
>
> Taher Alkhateeb
>
> On Wed, Jun 8, 2016 at 12:23 PM, Pranay Pandey <
> pranay.pandey@hotwaxsystems.com> wrote:
>
> > Hi Taher,
> >
> > There is one thing that I want to confirm based my understanding, as long
> > as I know, we can run tests independently as well e.g.
> >
> > *Running tests for a component: *
> >
> > java -jar ofbiz.jar -test component=‘order’
> >
> > *Running a specific test Suite from a component:*
> >
> > ant run-single-test-suite -Dtest.component=‘order’ -Dtest.suiteName='
> > ordertests'
> >
> > *Running a test case from a component:*
> >
> > java -jar ofbiz.jar -test -component='order' -case='salesOrder-test'
> >
> > I may be completely wrong here but just want to confirm and understand
> your
> > proposal.
> >
> > Best regards,
> >
> > Pranay Pandey
> > HotWax Systems
> > http://www.hotwaxsystems.com/
> >
> > On Wed, Jun 8, 2016 at 2:41 PM, Taher Alkhateeb <
> > slidingfilaments@gmail.com>
> > wrote:
> >
> > > Hello Everyone,
> > >
> > > After refactoring the start component and while starting on the base
> > > component I realized that the testing framework for OFBiz is not good.
> > You
> > > cannot do real test driven development or red-green-refactor with the
> > > current setup, hence my proposal to change it. I explain below:
> > >
> > > Problem with current design
> > > ----------------------------------------
> > > - What we have right now is not unit tests, it's really integration
> > tests.
> > > You have to start the framework, the database, the service engine, the
> > > entity engine and pretty much everything.
> > > - Testing is very slow, because it's an integration test as I mentioned
> > > above. 10 minutes on a good computer!
> > > - There is zero mocking! We actually have to --load-data for things to
> > > work. Again, these are integration tests.
> > > - Too complex: Integration tests by their nature are grabbing too much.
> > > Mind you, I am not objecting to integration tests (I actually like
> them)
> > > but I am objecting to not having real unit-tests. Unit tests should all
> > run
> > > in a few seconds.
> > >
> > > Proposed solution
> > > --------------------------
> > > - We keep what is considered real integration tests the way they are
> > right
> > > now and keep using them
> > > - We move what should be unit tests into simple JUnit classes, and we
> do
> > > not run them using java -jar ofbiz.jar --test, but instead run them
> > > directly from the build.xml script, so these files are not identified
> in
> > > any XML document, but are simply called immediately from the build
> > scripts.
> > > - We clearly mark the difference between integration tests and unit
> tests
> > > (inside the source files or in the suite declarations).
> > > - We change the run-tests target in build.xml to run both unit tests
> and
> > > integration tests.
> > >
> > > I intend to heavily refactor the framework and I would feel better
> about
> > > introducing this change while refactoring. What do you guys think?
> Ideas?
> > > Suggestions? Approvals and thumbs up?
> > >
> > > Regards,
> > >
> > > Taher Alkhateeb
> > >
> >
>

Re: Proposal to modify the testing framework for OFBiz

[Taher Alkhateeb <sl...@gmail.com>]

Hi Pranay,

What you are talking about is integration tests (ones that you can run per
component)

Unit tests do not need to be run per component. The entire unit test
library for the entire framework should run in probably less than 5
seconds. So there is no need to make them run per component.

My proposal is to simply have a target like run-unit-tests which would run
all the unit tests. These tests do not talk to the database or server or
anything, You can run them while flying in an airplane 30,000 feet above
the ocean.

Mind you I will not remove the integration tests, I'm just adding unit
tests (and probably moving some integration tests to unit tests).

Naturally, my proposition means that those tests are only JUnit tests (for
now) but that is a quick solution that I think is rather clean because
we're not really radically modifying anything, we're just adding JUnit
classes and running them.

Regards,

Taher Alkhateeb

On Wed, Jun 8, 2016 at 12:23 PM, Pranay Pandey <
pranay.pandey@hotwaxsystems.com> wrote:

> Hi Taher,
>
> There is one thing that I want to confirm based my understanding, as long
> as I know, we can run tests independently as well e.g.
>
> *Running tests for a component: *
>
> java -jar ofbiz.jar -test component=‘order’
>
> *Running a specific test Suite from a component:*
>
> ant run-single-test-suite -Dtest.component=‘order’ -Dtest.suiteName='
> ordertests'
>
> *Running a test case from a component:*
>
> java -jar ofbiz.jar -test -component='order' -case='salesOrder-test'
>
> I may be completely wrong here but just want to confirm and understand your
> proposal.
>
> Best regards,
>
> Pranay Pandey
> HotWax Systems
> http://www.hotwaxsystems.com/
>
> On Wed, Jun 8, 2016 at 2:41 PM, Taher Alkhateeb <
> slidingfilaments@gmail.com>
> wrote:
>
> > Hello Everyone,
> >
> > After refactoring the start component and while starting on the base
> > component I realized that the testing framework for OFBiz is not good.
> You
> > cannot do real test driven development or red-green-refactor with the
> > current setup, hence my proposal to change it. I explain below:
> >
> > Problem with current design
> > ----------------------------------------
> > - What we have right now is not unit tests, it's really integration
> tests.
> > You have to start the framework, the database, the service engine, the
> > entity engine and pretty much everything.
> > - Testing is very slow, because it's an integration test as I mentioned
> > above. 10 minutes on a good computer!
> > - There is zero mocking! We actually have to --load-data for things to
> > work. Again, these are integration tests.
> > - Too complex: Integration tests by their nature are grabbing too much.
> > Mind you, I am not objecting to integration tests (I actually like them)
> > but I am objecting to not having real unit-tests. Unit tests should all
> run
> > in a few seconds.
> >
> > Proposed solution
> > --------------------------
> > - We keep what is considered real integration tests the way they are
> right
> > now and keep using them
> > - We move what should be unit tests into simple JUnit classes, and we do
> > not run them using java -jar ofbiz.jar --test, but instead run them
> > directly from the build.xml script, so these files are not identified in
> > any XML document, but are simply called immediately from the build
> scripts.
> > - We clearly mark the difference between integration tests and unit tests
> > (inside the source files or in the suite declarations).
> > - We change the run-tests target in build.xml to run both unit tests and
> > integration tests.
> >
> > I intend to heavily refactor the framework and I would feel better about
> > introducing this change while refactoring. What do you guys think? Ideas?
> > Suggestions? Approvals and thumbs up?
> >
> > Regards,
> >
> > Taher Alkhateeb
> >
>

Re: Proposal to modify the testing framework for OFBiz

[Pranay Pandey <pr...@hotwaxsystems.com>]

Hi Taher,

There is one thing that I want to confirm based my understanding, as long
as I know, we can run tests independently as well e.g.

*Running tests for a component: *

java -jar ofbiz.jar -test component=‘order’

*Running a specific test Suite from a component:*

ant run-single-test-suite -Dtest.component=‘order’ -Dtest.suiteName='
ordertests'

*Running a test case from a component:*

java -jar ofbiz.jar -test -component='order' -case='salesOrder-test'

I may be completely wrong here but just want to confirm and understand your
proposal.

Best regards,

Pranay Pandey
HotWax Systems
http://www.hotwaxsystems.com/

On Wed, Jun 8, 2016 at 2:41 PM, Taher Alkhateeb <sl...@gmail.com>
wrote:

> Hello Everyone,
>
> After refactoring the start component and while starting on the base
> component I realized that the testing framework for OFBiz is not good. You
> cannot do real test driven development or red-green-refactor with the
> current setup, hence my proposal to change it. I explain below:
>
> Problem with current design
> ----------------------------------------
> - What we have right now is not unit tests, it's really integration tests.
> You have to start the framework, the database, the service engine, the
> entity engine and pretty much everything.
> - Testing is very slow, because it's an integration test as I mentioned
> above. 10 minutes on a good computer!
> - There is zero mocking! We actually have to --load-data for things to
> work. Again, these are integration tests.
> - Too complex: Integration tests by their nature are grabbing too much.
> Mind you, I am not objecting to integration tests (I actually like them)
> but I am objecting to not having real unit-tests. Unit tests should all run
> in a few seconds.
>
> Proposed solution
> --------------------------
> - We keep what is considered real integration tests the way they are right
> now and keep using them
> - We move what should be unit tests into simple JUnit classes, and we do
> not run them using java -jar ofbiz.jar --test, but instead run them
> directly from the build.xml script, so these files are not identified in
> any XML document, but are simply called immediately from the build scripts.
> - We clearly mark the difference between integration tests and unit tests
> (inside the source files or in the suite declarations).
> - We change the run-tests target in build.xml to run both unit tests and
> integration tests.
>
> I intend to heavily refactor the framework and I would feel better about
> introducing this change while refactoring. What do you guys think? Ideas?
> Suggestions? Approvals and thumbs up?
>
> Regards,
>
> Taher Alkhateeb
>

Re: Proposal to modify the testing framework for OFBiz

[Hans Bakker <h....@antwebsystems.com>]

great initiative Taher, finally someone who values automated tests to 
enable TDD but also leading to continuous improvement and finally 
continuous deployment....

Regards,
Hans Bakker
CEO AntWebsystems.com

On 18/07/16 12:06, Taher Alkhateeb wrote:
> Hello Everyone,
>
> In reference to this thread and the Jira OFBIZ-7254, I'm very happy to
> announce that OFBiz is now ready for applying unit tests with only 8 new
> lines of code in the build script (r1753143) :)
>
> I recommend we do the following moving forward:
>
> 1- Introduce unit tests as much as we can to all components
> 2- Migrate most of the integration tests we currently have to unit tests
> (since they are designed to do very little integration).
>
> This is a great chance for us to practice real TDD in OFBiz
>
> Cheers,
>
> Taher Alkhateeb
>
> On Mon, Jun 13, 2016 at 7:43 AM, Pranay Pandey <
> pranay.pandey@hotwaxsystems.com> wrote:
>
>> +1
>>
>> Best regards,
>>
>> Pranay Pandey
>> HotWax Systems
>> http://www.hotwaxsystems.com/
>>
>> On Fri, Jun 10, 2016 at 7:46 PM, Taher Alkhateeb <
>> slidingfilaments@gmail.com
>>> wrote:
>>
>>> Hello Everyone,
>>>
>>> I was able to get a few tests running and this is very doable. But I
>> faced
>>> a big problem in designing the testing framework because of ANT.
>>>
>>> The problem
>>> ----------------
>>> The way the build scripts are designed in OFBiz are very complex. A
>> master
>>> file calls other files which call other files. And in the middle you have
>>> external libraries (ant-contrib) and macros, and variables, and class
>> path
>>> declarations, and and and ....
>>>
>>> I cannot declare the tests programmatically (with JUnit test suites)
>>> because this means lower level components would depend on higher level
>>> components. So I have to do it in ANT, by navigating this maze of build
>>> scripts, and it was a headache for me just to read them, let alone modify
>>> them to create a testing framework.
>>>
>>> Suggested Solution
>>> ------------------------
>>> I suggest to implement the testing framework in Gradle, and simply call
>> it
>>> from within ant. This is a middle solution that sustains ant for now, but
>>> can allow us to switch out later.
>>>
>>> This means I will just add one more file called build.gradle in the top
>>> level directory, and figure out the business logic for calling the test
>>> suites from that file
>>>
>>> I look forward to your feedback.
>>>
>>> Regards,
>>>
>>> Taher Alkhateeb
>>>
>>> On Wed, Jun 8, 2016 at 6:00 PM, Taher Alkhateeb <
>>> slidingfilaments@gmail.com>
>>> wrote:
>>>
>>>> Hi Everyone,
>>>>
>>>> Thank you all for your support, JIRA created in
>>>> https://issues.apache.org/jira/browse/OFBIZ-7254
>>>>
>>>> I will start working on it and try to implement ASAP to get my focus
>> back
>>>> on refactoring.
>>>>
>>>> Cheers!
>>>>
>>>> Taher Alkhateeb
>>>>
>>>> On Wed, Jun 8, 2016 at 4:58 PM, Deepak Dixit <
>>>> deepak.dixit@hotwaxsystems.com> wrote:
>>>>
>>>>> +1
>>>>>
>>>>> Thanks & Regards
>>>>> --
>>>>> Deepak Dixit
>>>>> www.hotwaxsystems.com
>>>>>
>>>>> On Wed, Jun 8, 2016 at 7:12 PM, Mridul Pathak <
>>>>> mridul.pathak@hotwaxsystems.com> wrote:
>>>>>
>>>>>> +1
>>>>>>
>>>>>> Makes perfect sense.
>>>>>>
>>>>>> --
>>>>>> Thanks & Regards,
>>>>>> Mridul Pathak
>>>>>> Senior Manager
>>>>>> HotWax Systems
>>>>>> http://www.hotwaxsystems.com
>>>>>>
>>>>>>> On Jun 8, 2016, at 2:41 PM, Taher Alkhateeb <
>>>>> slidingfilaments@gmail.com>
>>>>>> wrote:
>>>>>>>
>>>>>>> Hello Everyone,
>>>>>>>
>>>>>>> After refactoring the start component and while starting on the
>> base
>>>>>>> component I realized that the testing framework for OFBiz is not
>>> good.
>>>>>> You
>>>>>>> cannot do real test driven development or red-green-refactor with
>>> the
>>>>>>> current setup, hence my proposal to change it. I explain below:
>>>>>>>
>>>>>>> Problem with current design
>>>>>>> ----------------------------------------
>>>>>>> - What we have right now is not unit tests, it's really
>> integration
>>>>>> tests.
>>>>>>> You have to start the framework, the database, the service engine,
>>> the
>>>>>>> entity engine and pretty much everything.
>>>>>>> - Testing is very slow, because it's an integration test as I
>>>>> mentioned
>>>>>>> above. 10 minutes on a good computer!
>>>>>>> - There is zero mocking! We actually have to --load-data for
>> things
>>> to
>>>>>>> work. Again, these are integration tests.
>>>>>>> - Too complex: Integration tests by their nature are grabbing too
>>>>> much.
>>>>>>> Mind you, I am not objecting to integration tests (I actually like
>>>>> them)
>>>>>>> but I am objecting to not having real unit-tests. Unit tests
>> should
>>>>> all
>>>>>> run
>>>>>>> in a few seconds.
>>>>>>>
>>>>>>> Proposed solution
>>>>>>> --------------------------
>>>>>>> - We keep what is considered real integration tests the way they
>> are
>>>>>> right
>>>>>>> now and keep using them
>>>>>>> - We move what should be unit tests into simple JUnit classes, and
>>> we
>>>>> do
>>>>>>> not run them using java -jar ofbiz.jar --test, but instead run
>> them
>>>>>>> directly from the build.xml script, so these files are not
>>> identified
>>>>> in
>>>>>>> any XML document, but are simply called immediately from the build
>>>>>> scripts.
>>>>>>> - We clearly mark the difference between integration tests and
>> unit
>>>>> tests
>>>>>>> (inside the source files or in the suite declarations).
>>>>>>> - We change the run-tests target in build.xml to run both unit
>> tests
>>>>> and
>>>>>>> integration tests.
>>>>>>>
>>>>>>> I intend to heavily refactor the framework and I would feel better
>>>>> about
>>>>>>> introducing this change while refactoring. What do you guys think?
>>>>> Ideas?
>>>>>>> Suggestions? Approvals and thumbs up?
>>>>>>>
>>>>>>> Regards,
>>>>>>>
>>>>>>> Taher Alkhateeb
>>>>>>
>>>>>>
>>>>>
>>>>
>>>>
>>>
>>
>

-- 

Regards,

Hans Bakker
CEO, http://antwebsystems.com

Re: Proposal to modify the testing framework for OFBiz

[Ron Wheeler <rw...@artifact-software.com>]

You are probably right that integration testing has a lot more value.
However, unit tests are still useful in their own right and may 
encourage people to write smaller classes that can be tested easily.

As you point out, integration testing can be done at several levels with 
mock implementations used to limit the scope of the tests.

A full set of integration tests would take a long time to run and even 
longer to construct.

You may also be right that building a lot of new unit tests to test old 
features may not be very valuable but getting developers to move towards 
TDD might reduce the number of regressions.
Unit testing does, at least, give positive proof of what has been tested 
when new code is presented for review.
It also demonstrates the use of a class which may add to the specifications.

It is usually possible to exclude tests that are redundant or not 
relative to the new features being added.
There is not a lot of point in running low-level unit tests repeatedly 
in a development cycle if you are not changing the classes being tested.
This is trivial to manage in Maven with jUnit and Surefire.
Not sure about the Gradle build.

Proper integration tests that cover load testing, synchronization, 
caching and locking under tests of all UI features would be wonderful to 
have but would take a long time to construct given the number of feature 
combinations, stack configurations and use cases.
A full test would take a long time to run and would be something that 
would be useful only a few times in each release cycle.
Very useful if versions are going to be released with any sort of 
implied warranty about quality.

The current activity is a big step forward and Taher's leadership in 
modernizing the framework seems to have inspired a lot of new activity 
and a more modern way of thinking about software development.

Well done.


Ron


On 18/07/2016 10:19 PM, Scott Gray wrote:
> I know I'm late to the party here, but I just want to say that I think
> integration tests have far greater value to OFBiz than unit tests.  Mostly
> because we tend to have quite a low number of tests and integration tests
> give us much better coverage per line of test code and the tests are much
> closer to the real world scenarios the application might encounter.
>
> I don't really see how unit tests could be applied to non-framework testing
> in a useful manner, could you expand on your vision in that regard?  I mean
> would we be testing something smaller than a service as the 'unit'?  What
> would we mock? Would transaction management still be active? What happens
> when the service calls another service, I guess we mock the response from
> that service (how)?
>
> It just seems a very complicated method to achieve a less thorough but
> albeit faster (maybe) test result.
>
> A build, data load and full test run takes 4m 9s on my laptop.  Excluding
> build, data load and framework tests: the application level tests take 35s,
> not very expensive IMO.  The data load time can be reduced to practically
> nothing by copying a clean slate database into runtime for each run.
>
> I'm mostly just suggesting we be wary of adding complicated testing
> procedures in the hope of achieving some 'best practice' result which in
> reality will provide minimal benefits.
>
> Regards
> Scott
>
> On 18 July 2016 at 18:57, Taher Alkhateeb <sl...@gmail.com>
> wrote:
>
>> Hello Akash,
>>
>> Fantastic, I have a few unit tests almost done to be included in the start
>> component. I will create a new subtask under OFBIZ-1463 to commit the tests
>> so you can use them as a reference if you like to.
>>
>> I also recommend that you follow the same directory structure between the
>> test code and production code. So for example:
>>
>> Production code: framework/start/src/main/java/org/apache/ofbiz/base/start
>> Test code: framework/start/src/test/java/org/apache/ofbiz/base/start
>>
>> The benefit of this hierarchy is that you can access non-public (package
>> protected) methods for testing. This is in fact exactly what I needed to be
>> able to apply some of the tests.
>>
>> Cheers,
>>
>> Taher Alkhateeb
>>
>> On Mon, Jul 18, 2016 at 9:22 AM, Akash Jain <ak...@hotwaxsystems.com>
>> wrote:
>>
>>> Thanks Taher for nice initiative!
>>>
>>> We are planning to written unit tests to all components under OFBIZ-1463
>>>
>>> Thanks and Regards
>>> --
>>> Akash Jain
>>>
>>> On Mon, Jul 18, 2016 at 10:36 AM, Taher Alkhateeb <
>>> slidingfilaments@gmail.com> wrote:
>>>
>>>> Hello Everyone,
>>>>
>>>> In reference to this thread and the Jira OFBIZ-7254, I'm very happy to
>>>> announce that OFBiz is now ready for applying unit tests with only 8
>> new
>>>> lines of code in the build script (r1753143) :)
>>>>
>>>> I recommend we do the following moving forward:
>>>>
>>>> 1- Introduce unit tests as much as we can to all components
>>>> 2- Migrate most of the integration tests we currently have to unit
>> tests
>>>> (since they are designed to do very little integration).
>>>>
>>>> This is a great chance for us to practice real TDD in OFBiz
>>>>
>>>> Cheers,
>>>>
>>>> Taher Alkhateeb
>>>>
>>>> On Mon, Jun 13, 2016 at 7:43 AM, Pranay Pandey <
>>>> pranay.pandey@hotwaxsystems.com> wrote:
>>>>
>>>>> +1
>>>>>
>>>>> Best regards,
>>>>>
>>>>> Pranay Pandey
>>>>> HotWax Systems
>>>>> http://www.hotwaxsystems.com/
>>>>>
>>>>> On Fri, Jun 10, 2016 at 7:46 PM, Taher Alkhateeb <
>>>>> slidingfilaments@gmail.com
>>>>>> wrote:
>>>>>> Hello Everyone,
>>>>>>
>>>>>> I was able to get a few tests running and this is very doable. But
>> I
>>>>> faced
>>>>>> a big problem in designing the testing framework because of ANT.
>>>>>>
>>>>>> The problem
>>>>>> ----------------
>>>>>> The way the build scripts are designed in OFBiz are very complex. A
>>>>> master
>>>>>> file calls other files which call other files. And in the middle
>> you
>>>> have
>>>>>> external libraries (ant-contrib) and macros, and variables, and
>> class
>>>>> path
>>>>>> declarations, and and and ....
>>>>>>
>>>>>> I cannot declare the tests programmatically (with JUnit test
>> suites)
>>>>>> because this means lower level components would depend on higher
>>> level
>>>>>> components. So I have to do it in ANT, by navigating this maze of
>>> build
>>>>>> scripts, and it was a headache for me just to read them, let alone
>>>> modify
>>>>>> them to create a testing framework.
>>>>>>
>>>>>> Suggested Solution
>>>>>> ------------------------
>>>>>> I suggest to implement the testing framework in Gradle, and simply
>>> call
>>>>> it
>>>>>> from within ant. This is a middle solution that sustains ant for
>> now,
>>>> but
>>>>>> can allow us to switch out later.
>>>>>>
>>>>>> This means I will just add one more file called build.gradle in the
>>> top
>>>>>> level directory, and figure out the business logic for calling the
>>> test
>>>>>> suites from that file
>>>>>>
>>>>>> I look forward to your feedback.
>>>>>>
>>>>>> Regards,
>>>>>>
>>>>>> Taher Alkhateeb
>>>>>>
>>>>>> On Wed, Jun 8, 2016 at 6:00 PM, Taher Alkhateeb <
>>>>>> slidingfilaments@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Hi Everyone,
>>>>>>>
>>>>>>> Thank you all for your support, JIRA created in
>>>>>>> https://issues.apache.org/jira/browse/OFBIZ-7254
>>>>>>>
>>>>>>> I will start working on it and try to implement ASAP to get my
>>> focus
>>>>> back
>>>>>>> on refactoring.
>>>>>>>
>>>>>>> Cheers!
>>>>>>>
>>>>>>> Taher Alkhateeb
>>>>>>>
>>>>>>> On Wed, Jun 8, 2016 at 4:58 PM, Deepak Dixit <
>>>>>>> deepak.dixit@hotwaxsystems.com> wrote:
>>>>>>>
>>>>>>>> +1
>>>>>>>>
>>>>>>>> Thanks & Regards
>>>>>>>> --
>>>>>>>> Deepak Dixit
>>>>>>>> www.hotwaxsystems.com
>>>>>>>>
>>>>>>>> On Wed, Jun 8, 2016 at 7:12 PM, Mridul Pathak <
>>>>>>>> mridul.pathak@hotwaxsystems.com> wrote:
>>>>>>>>
>>>>>>>>> +1
>>>>>>>>>
>>>>>>>>> Makes perfect sense.
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Thanks & Regards,
>>>>>>>>> Mridul Pathak
>>>>>>>>> Senior Manager
>>>>>>>>> HotWax Systems
>>>>>>>>> http://www.hotwaxsystems.com
>>>>>>>>>
>>>>>>>>>> On Jun 8, 2016, at 2:41 PM, Taher Alkhateeb <
>>>>>>>> slidingfilaments@gmail.com>
>>>>>>>>> wrote:
>>>>>>>>>> Hello Everyone,
>>>>>>>>>>
>>>>>>>>>> After refactoring the start component and while starting on
>>> the
>>>>> base
>>>>>>>>>> component I realized that the testing framework for OFBiz is
>>> not
>>>>>> good.
>>>>>>>>> You
>>>>>>>>>> cannot do real test driven development or red-green-refactor
>>>> with
>>>>>> the
>>>>>>>>>> current setup, hence my proposal to change it. I explain
>>> below:
>>>>>>>>>> Problem with current design
>>>>>>>>>> ----------------------------------------
>>>>>>>>>> - What we have right now is not unit tests, it's really
>>>>> integration
>>>>>>>>> tests.
>>>>>>>>>> You have to start the framework, the database, the service
>>>> engine,
>>>>>> the
>>>>>>>>>> entity engine and pretty much everything.
>>>>>>>>>> - Testing is very slow, because it's an integration test as
>> I
>>>>>>>> mentioned
>>>>>>>>>> above. 10 minutes on a good computer!
>>>>>>>>>> - There is zero mocking! We actually have to --load-data for
>>>>> things
>>>>>> to
>>>>>>>>>> work. Again, these are integration tests.
>>>>>>>>>> - Too complex: Integration tests by their nature are
>> grabbing
>>>> too
>>>>>>>> much.
>>>>>>>>>> Mind you, I am not objecting to integration tests (I
>> actually
>>>> like
>>>>>>>> them)
>>>>>>>>>> but I am objecting to not having real unit-tests. Unit tests
>>>>> should
>>>>>>>> all
>>>>>>>>> run
>>>>>>>>>> in a few seconds.
>>>>>>>>>>
>>>>>>>>>> Proposed solution
>>>>>>>>>> --------------------------
>>>>>>>>>> - We keep what is considered real integration tests the way
>>> they
>>>>> are
>>>>>>>>> right
>>>>>>>>>> now and keep using them
>>>>>>>>>> - We move what should be unit tests into simple JUnit
>> classes,
>>>> and
>>>>>> we
>>>>>>>> do
>>>>>>>>>> not run them using java -jar ofbiz.jar --test, but instead
>> run
>>>>> them
>>>>>>>>>> directly from the build.xml script, so these files are not
>>>>>> identified
>>>>>>>> in
>>>>>>>>>> any XML document, but are simply called immediately from the
>>>> build
>>>>>>>>> scripts.
>>>>>>>>>> - We clearly mark the difference between integration tests
>> and
>>>>> unit
>>>>>>>> tests
>>>>>>>>>> (inside the source files or in the suite declarations).
>>>>>>>>>> - We change the run-tests target in build.xml to run both
>> unit
>>>>> tests
>>>>>>>> and
>>>>>>>>>> integration tests.
>>>>>>>>>>
>>>>>>>>>> I intend to heavily refactor the framework and I would feel
>>>> better
>>>>>>>> about
>>>>>>>>>> introducing this change while refactoring. What do you guys
>>>> think?
>>>>>>>> Ideas?
>>>>>>>>>> Suggestions? Approvals and thumbs up?
>>>>>>>>>>
>>>>>>>>>> Regards,
>>>>>>>>>>
>>>>>>>>>> Taher Alkhateeb
>>>>>>>>>
>>>>>>>


-- 
Ron Wheeler
President
Artifact Software Inc
email: rwheeler@artifact-software.com
skype: ronaldmwheeler
phone: 866-970-2435, ext 102

Re: Proposal to modify the testing framework for OFBiz

[Ron Wheeler <rw...@artifact-software.com>]

I think that your point was well received.
I am just happy that testing is getting attention.

Ron

On 19/07/2016 7:53 PM, Scott Gray wrote:
> I'm not looking for any sort of response really.  I was just speaking in
> favor of integration tests because I think they provide better coverage at
> the cost of a little bit of speed (the framework has to be running to run
> them).  Most of the responses coming in were of the form "unit tests are
> great because [insert a reason that applies to any type of test]".  So I
> was merely pointing out that the arguments weren't specific to unit tests
> and hence weren't really adding value to the discussion.
>
> Regards
> Scott
>
> On 20 July 2016 at 02:05, Ron Wheeler <rw...@artifact-software.com>
> wrote:
>
>> It is not clear what you are expecting as a specific response.
>>
>> As you move up the food chain, the definition of "units" changes and the
>> tests get more complicated and the design of the tests gets more complex
>> (mock objects, testing race conditions).
>>
>> It is hard to make specific comments about what kind of testing will give
>> the highest level of confidence at each level but we are going in the right
>> direction.
>>
>> The fact that we are considering tests as a major development activity is
>> a big step forward.
>>
>> We can argue about specific cases (scope, tools, value, etc) as we get to
>> them.
>>
>> I think that we are at the "good idea" stage rather than at the "policy"
>> stage at this point.
>>
>> Ron
>>
>>
>>
>> On 19/07/2016 4:23 AM, Scott Gray wrote:
>>
>>> I think people are missing my point because they keep replying with
>>> generic
>>> statements that aren't specific to unit tests.
>>>
>>> On 19 July 2016 at 20:08, Montalbano Florian <
>>> florian.montalbano@nereide.fr>
>>> wrote:
>>>
>>> Hi everyone,
>>>> unit tests and integration tests are complementary and with test, the
>>>> more
>>>> the better.
>>>> Implementing unit tests will be a great load of work but it is the same
>>>> with every kind of test. As Taher said, we will gain a lot from those
>>>> unit
>>>> tests for avoiding regression while refactoring the framework. From the
>>>> start component scope, it has already be proved useful.
>>>> And what Ron said about encouraging the creation of "smaller class that
>>>> can be tested easily" is very important too. It can be a first step to
>>>> reach TDD in the OFBiz development. TDD can help to give confidence to
>>>> newcomer and veteran when updating some part of the code and as Hans
>>>> said,
>>>> this lead to "continuous improvement and finally continuous deployment".
>>>>
>>>> I'm aware that this does not solve the technical implementation stuff,
>>>> but
>>>> I think this is an improvement that need to be supported.
>>>>
>>>> Have a nice day,
>>>>
>>>> Florian Montalbano
>>>>
>>>>
>>>> Le 19/07/2016 09:04, Pierre Smits a �crit :
>>>>
>>>> As with anything, the law of diminishing returns also applies to OFBiz
>>>>> and
>>>>> tests. This is not true for unit tests and system integration tests, but
>>>>> also for user acceptance and performance tests.
>>>>>
>>>>> Nevertheless, the work done up to now is a good start and - I feel
>>>>> confident - appreciated. And unit tests are certainly valuable in the
>>>>> framework stack. How it will be for functions (regarding components in
>>>>> application and special purpose stack) needs to be addressed when we
>>>>> reach
>>>>> that bridge.
>>>>>
>>>>>
>>>>> Best regards,
>>>>>
>>>>> Pierre Smits
>>>>>
>>>>> ORRTIZ.COM <http://www.orrtiz.com>
>>>>> OFBiz based solutions & services
>>>>>
>>>>> OFBiz Extensions Marketplace
>>>>> http://oem.ofbizci.net/oci-2/
>>>>>
>>>>> On Tue, Jul 19, 2016 at 8:22 AM, Taher Alkhateeb <
>>>>> slidingfilaments@gmail.com
>>>>>
>>>>> wrote:
>>>>>> Hi Scott,
>>>>>>
>>>>>> Well spoken thank you. Okay may I suggest that for any such work we
>>>>>> will
>>>>>> discuss it here see its Merit and if it makes sense then we take it in.
>>>>>> It
>>>>>> is a little early to discuss it right now because we did not yet go to
>>>>>> the
>>>>>> higher-level components. Once we do I'll be sure to have a conversation
>>>>>> about this in here and would appreciate your input to it.
>>>>>>
>>>>>> Regards,
>>>>>>
>>>>>> Taher Alkhateeb
>>>>>>
>>>>>> On Jul 19, 2016 9:00 AM, "Scott Gray" <sc...@hotwaxsystems.com>
>>>>>> wrote:
>>>>>>
>>>>>> Yeah I'm sure unit tests probably worked well for the start component
>>>>>> given
>>>>>>
>>>>>> it is the lowest level component in the system and closest to a basic
>>>>>>> java
>>>>>> app.  I just think the value proposition might decrease the further up
>>>>>>> the
>>>>>> stack we move with them.  I'm not against unit tests when they prove
>>>>>>> useful, but further up the stack I think we should prove the case for
>>>>>>>
>>>>>>> them
>>>>>> before doing much work to support the mocking that will be required to
>>>>>>> keep
>>>>>> them inline with 'best practices'.
>>>>>>> In OFBiz "bad or unwanted" tends to come mostly in the way of
>>>>>>> increasing
>>>>>>> complexity from features that require more effort to maintain than the
>>>>>>> value they provide to the community.  I think there's a chance
>>>>>>> attempting
>>>>>>> to mock service tests could fall into that bucket.  I could be wrong,
>>>>>>>
>>>>>>> but I
>>>>>> think it's worth looking into before we declare that unit tests are the
>>>>>>> best form of testing for OFBiz.
>>>>>>>
>>>>>>> Regards
>>>>>>> Scott
>>>>>>>
>>>>>>> On 19 July 2016 at 17:37, Taher Alkhateeb <slidingfilaments@gmail.com
>>>>>>> wrote:
>>>>>>>
>>>>>>> Hi Scott,
>>>>>>>
>>>>>>>> Thank you for the feedback. To be focused exactly on integration vs
>>>>>>>>
>>>>>>>> unit, I
>>>>>>> already mentioned above that at least for me the main objective is to
>>>>>>>> confidently and quickly run the tests in short bursts of red-green
>>>>>>>> refactor. This allows me to refactor code without waiting in front of
>>>>>>>>
>>>>>>>> the
>>>>>>> screen in between test cycles thus giving me immediate feedback on any
>>>>>>>
>>>>>>>> errors I made. Perhaps my intro was too long so this is the squeezed
>>>>>>>>
>>>>>>>> out
>>>>>>> version of it.
>>>>>>>
>>>>>>>> I already had one round of testing in the start component which was
>>>>>>>>
>>>>>>>> much
>>>>>>> faster that way and had an immediate impact. Oh and by the way, you
>>>>>>> cannot
>>>>>>>
>>>>>>> test the start component with integration tests for example unless you
>>>>>>>> do
>>>>>>> it from an external component which cannot access package protected
>>>>>>> items.
>>>>>>>
>>>>>>> This style of coding is applicable I think to any software project
>>>>>>>> inclusive of OFBiz. Maybe in certain components more than others or
>>>>>>>>
>>>>>>>> certain
>>>>>>> areas more than others. But I can't see how it could be bad or
>>>>>>>> unwanted.
>>>>>>> Taher Alkhateeb
>>>>>>>
>>>>>>>> On Jul 19, 2016 8:18 AM, "Scott Gray" <sc...@hotwaxsystems.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>> Thanks Ron and Taher for your responses, I appreciate them but I
>>>>>>>> don't
>>>>>>>>
>>>>>>> hear
>>>>>>>
>>>>>>>> much in the conversation that speaks to the value of unit tests over
>>>>>>>>> integration tests.  Most of the thoughts shared speaks more to the
>>>>>>>>>
>>>>>>>>> value
>>>>>>>> of
>>>>>>>>
>>>>>>>> tests rather than differences in the type of tests.
>>>>>>>>> Speed: At an application level we have ~685 tests that run in 35
>>>>>>>>>
>>>>>>>>> seconds
>>>>>>>> (excluding build and data load).  Another point is that there isn't
>>>>>>>> much
>>>>>>>> reason why tests can't be run in parallel rather than sequentially.
>>>>>>>>
>>>>>>>>> TDD: Integration tests in OFBiz are as simple if not simpler to
>>>>>>>>> write
>>>>>>>>>
>>>>>>>>> than
>>>>>>>> unit tests and just as useful for TDD.
>>>>>>>>> I'm not sure if I missed any other points raised regarding
>>>>>>>>>
>>>>>>>>> integration
>>>>>>> vs.
>>>>>>>
>>>>>>>> unit tests?
>>>>>>>>> I'm not looking to start a big long debate on the topic, I just
>>>>>>>>>
>>>>>>>>> wanted
>>>>>>> to
>>>>>>>
>>>>>>> speak out that someone out there (me) doesn't think unit tests are
>>>>>>>> the
>>>>>>>>
>>>>>>> best
>>>>>>>
>>>>>>>> solution for testing OFBiz applications.
>>>>>>>>> Regards
>>>>>>>>> Scott
>>>>>>>>>
>>>>>>>>> On 19 July 2016 at 16:52, Taher Alkhateeb <
>>>>>>>>>
>>>>>>>>> slidingfilaments@gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Hi Scott,
>>>>>>>>>> Thank you for your input. Your ideas are thought provoking and
>>>>>>>>>>
>>>>>>>>>> enriching
>>>>>>>>> to
>>>>>>>>>
>>>>>>>>> the conversation.
>>>>>>>>>> The way I look at it, the general rule is usually many unit tests,
>>>>>>>>>>
>>>>>>>>>> less
>>>>>>>> integration tests, lesser functional tests. So we are not excluding
>>>>>>>>
>>>>>>>>> any
>>>>>>>>>
>>>>>>>> types of test, all of them are important in certain areas for
>>>>>>>>
>>>>>>>>> certain
>>>>>>>>>
>>>>>>>> purposes with certain quantity. Usually integration tests are less
>>>>>>>> because
>>>>>>>>> as you said they just grab more. I like the picture below as a
>>>>>>>>>> general
>>>>>>>> guideline
>>>>>>>>
>>>>>>>>> http://i.stack.imgur.com/fjQvQ.png
>>>>>>>>>> As you already mentioned unit tests are useful for the framework. I
>>>>>>>>>> discovered errors in code that I wrote which I was very very
>>>>>>>>>>
>>>>>>>>>> careful
>>>>>>>> with.
>>>>>>>> I immediately learned the lesson that humans are not designed to
>>>>>>>>>> code,
>>>>>>>> and
>>>>>>>>
>>>>>>>>> TDD gives you confidence as you build your code with those short
>>>>>>>>>> 30-60
>>>>>>>> second red-green refactors. I feel much much safer and more
>>>>>>>>
>>>>>>>>> confident
>>>>>>>>>
>>>>>>>> writing code that way, the test also documents how to use the api,
>>>>>>>> refactoring and feature change becomes less terrifying. Also messy
>>>>>>>>>> code
>>>>>>>> is
>>>>>>>>
>>>>>>>>> usually not test friendly thus refactoring and unit tests go
>>>>>>>>>> hand-in-hand
>>>>>>>>> for improving the code base.
>>>>>>>>>
>>>>>>>>>> Also, I am sorry to say that the framework code is rather messy and
>>>>>>>>>>
>>>>>>>>>> brittle
>>>>>>>>> in many places. I think you probably encountered this yourself. The
>>>>>>>>>> same
>>>>>>>>> is
>>>>>>>>>
>>>>>>>>> said for the applications. Now If we start refactoring without unit
>>>>>>>>>> tests
>>>>>>>>> then we are back to scary business. So much can go wrong so fast
>>>>>>>>> and
>>>>>>>>>
>>>>>>>> break
>>>>>>>> things you never expected to break. The framework with all
>>>>>>>>>> applications
>>>>>>>> require heavy refactoring of things like massive ugly methods,
>>>>>>>>
>>>>>>>>> sandwiched
>>>>>>>>> logic, heavy shared mutable state, hidden dependencies, poor
>>>>>>>>>
>>>>>>>>>> interfaces, and much more. Every time I touch something I get
>>>>>>>>>>
>>>>>>>>>> shocked
>>>>>>>> at
>>>>>>>> how bad it looks, spaghetti logic to no end. If you refactor with
>>>>>>>>>> integration tests instead of unit tests then you will come down to
>>>>>>>>>>
>>>>>>>>>> a
>>>>>>>> screeching halt as you wait between these test cycles. My computer
>>>>>>>> actually
>>>>>>>>> takes 10 minutes or more for a full clean load data and testing.
>>>>>>>>>> Now talking about mocking you raise some interesting points, what
>>>>>>>>>>
>>>>>>>>>> can
>>>>>>>> you
>>>>>>>> mock vs utilizing integration tests? This is an excellent question
>>>>>>>>> that
>>>>>>>>>
>>>>>>>> does not have a simple answer, and you mostly learn as you code,
>>>>>>>>
>>>>>>>>> difficult
>>>>>>>>>
>>>>>>>>> to envision without coding. However simple things like Java
>>>>>>>>>> services
>>>>>>>> can
>>>>>>>> be
>>>>>>>>> mocked with a standard mocking class that you use everywhere.
>>>>>>>>>> Transactions
>>>>>>>>> are difficult to mock and better left to integration tests I think.
>>>>>>>>>> SECAs
>>>>>>>>> might be mocked by simply passing output to input as chaining
>>>>>>>>> methods.
>>>>>>>>>
>>>>>>>> Mind
>>>>>>>>
>>>>>>>>> you I am not 100% sure of all of this, coding is the ultimate guide
>>>>>>>>>> to
>>>>>>>> what
>>>>>>>>
>>>>>>>>> can or cannot be done.
>>>>>>>>>> So I am not suggesting unit tests as a best-practice (even though
>>>>>>>>>>
>>>>>>>>>> it
>>>>>>>> is).
>>>>>>>> Instead I suggest it as something that I and others did and got a
>>>>>>>>> huge
>>>>>>>>>
>>>>>>>> psychological relief and confidence and comfort from. Swimming in
>>>>>>>>
>>>>>>>>> code
>>>>>>>>>
>>>>>>>> without tests is a terrifying business, made more terrifying if the
>>>>>>>>
>>>>>>>>> code
>>>>>>>>> is
>>>>>>>>>
>>>>>>>>> bad. Short bursts of red/green/red/green makes you feel good as you
>>>>>>>>>> build
>>>>>>>>> up your logic. And I don't have an exact vision of how to do it,
>>>>>>>>> because
>>>>>>>>> the details always win.
>>>>>>>>>
>>>>>>>>>> I look forward to hearing your thoughts and thank you for enriching
>>>>>>>>>>
>>>>>>>>>> this
>>>>>>>>> conversation.
>>>>>>>>>
>>>>>>>>>> Taher Alkhateeb
>>>>>>>>>>
>>>>>>>>>> On Tuesday, 19 July 2016, Scott Gray <scott.gray@hotwaxsystems.com
>>>>>>>>>>
>>>>>>>>>> wrote:
>>>>>>>>> I know I'm late to the party here, but I just want to say that I
>>>>>>>>>> think
>>>>>>>>>>
>>>>>>>>> integration tests have far greater value to OFBiz than unit
>>>>>>>>>
>>>>>>>>>> tests.
>>>>>>>>>>
>>>>>>>>> Mostly
>>>>>>>> because we tend to have quite a low number of tests and
>>>>>>>>>>> integration
>>>>>>>>> tests
>>>>>>>> give us much better coverage per line of test code and the tests
>>>>>>>>>> are
>>>>>>>>>>
>>>>>>>>> much
>>>>>>>>> closer to the real world scenarios the application might
>>>>>>>>>> encounter.
>>>>>>>>>>
>>>>>>>>> I don't really see how unit tests could be applied to
>>>>>>>> non-framework
>>>>>>>>> testing
>>>>>>>> in a useful manner, could you expand on your vision in that
>>>>>>>>>>> regard?
>>>>>>>>> I
>>>>>>>> mean
>>>>>>>>>> would we be testing something smaller than a service as the
>>>>>>>>>>> 'unit'?
>>>>>>>>> What
>>>>>>>> would we mock? Would transaction management still be active? What
>>>>>>>>>> happens
>>>>>>>>>> when the service calls another service, I guess we mock the
>>>>>>>>>> response
>>>>>>>>>>
>>>>>>>>> from
>>>>>>>>> that service (how)?
>>>>>>>>>>> It just seems a very complicated method to achieve a less
>>>>>>>>>>>
>>>>>>>>>>> thorough
>>>>>>>>> but
>>>>>>>> albeit faster (maybe) test result.
>>>>>>>>>> A build, data load and full test run takes 4m 9s on my laptop.
>>>>>>>>>>> Excluding
>>>>>>>>>> build, data load and framework tests: the application level tests
>>>>>>>>>> take
>>>>>>>>>>
>>>>>>>>> 35s,
>>>>>>>>>
>>>>>>>>>> not very expensive IMO.  The data load time can be reduced to
>>>>>>>>>>> practically
>>>>>>>>>> nothing by copying a clean slate database into runtime for each
>>>>>>>>>> run.
>>>>>>>>>>
>>>>>>>>> I'm mostly just suggesting we be wary of adding complicated
>>>>>>>>> testing
>>>>>>>>> procedures in the hope of achieving some 'best practice' result
>>>>>>>> which
>>>>>>>>> in
>>>>>>>>> reality will provide minimal benefits.
>>>>>>>>>>> Regards
>>>>>>>>>>> Scott
>>>>>>>>>>>
>>>>>>>>>>> On 18 July 2016 at 18:57, Taher Alkhateeb <
>>>>>>>>>>>
>>>>>>>>>>> slidingfilaments@gmail.com
>>>>>>>>> <javascript:;>>
>>>>>>>>>
>>>>>>>>>> wrote:
>>>>>>>>>>> Hello Akash,
>>>>>>>>>>>
>>>>>>>>>>>> Fantastic, I have a few unit tests almost done to be included
>>>>>>>>>>>>
>>>>>>>>>>>> in
>>>>>>>>>> the
>>>>>>>> start
>>>>>>>>>> component. I will create a new subtask under OFBIZ-1463 to
>>>>>>>>>>>> commit
>>>>>>>>>> the
>>>>>>>> tests
>>>>>>>>>> so you can use them as a reference if you like to.
>>>>>>>>>>>> I also recommend that you follow the same directory structure
>>>>>>>>>>>>
>>>>>>>>>>>> between
>>>>>>>>>> the
>>>>>>>>>> test code and production code. So for example:
>>>>>>>>>>>> Production code:
>>>>>>>>>>>>
>>>>>>>>>>>> framework/start/src/main/java/org/apache/ofbiz/base/start
>>>>>>>>>>> Test code:
>>>>>>>>>>>> framework/start/src/test/java/org/apache/ofbiz/base/start
>>>>>>>>>> The benefit of this hierarchy is that you can access non-public
>>>>>>>>>> (package
>>>>>>>>>>> protected) methods for testing. This is in fact exactly what I
>>>>>>>>>>> needed
>>>>>>>>>>>
>>>>>>>>>> to
>>>>>>>>>> be
>>>>>>>>>>> able to apply some of the tests.
>>>>>>>>>>>> Cheers,
>>>>>>>>>>>>
>>>>>>>>>>>> Taher Alkhateeb
>>>>>>>>>>>>
>>>>>>>>>>>> On Mon, Jul 18, 2016 at 9:22 AM, Akash Jain <
>>>>>>>>>>>>
>>>>>>>>>>>> akash.jain@hotwaxsystems.com <javascript:;>>
>>>>>>>>>>> wrote:
>>>>>>>>>>>> Thanks Taher for nice initiative!
>>>>>>>>>>>>
>>>>>>>>>>>>> We are planning to written unit tests to all components under
>>>>>>>>>>>>>
>>>>>>>>>>>>> OFBIZ-1463
>>>>>>>>>>>> Thanks and Regards
>>>>>>>>>>>>
>>>>>>>>>>>>> --
>>>>>>>>>>>>> Akash Jain
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Mon, Jul 18, 2016 at 10:36 AM, Taher Alkhateeb <
>>>>>>>>>>>>> slidingfilaments@gmail.com <javascript:;>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>> Hello Everyone,
>>>>>>>>>>>>>
>>>>>>>>>>>>>> In reference to this thread and the Jira OFBIZ-7254, I'm
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> very
>>>>>>>>>>>> happy
>>>>>>>> to
>>>>>>>>>>> announce that OFBiz is now ready for applying unit tests
>>>>>>>>>>>>> with
>>>>>>>>>>>>>
>>>>>>>>>>>> only
>>>>>>>> 8
>>>>>>>>>> new
>>>>>>>>>>>> lines of code in the build script (r1753143) :)
>>>>>>>>>>>>>> I recommend we do the following moving forward:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> 1- Introduce unit tests as much as we can to all components
>>>>>>>>>>>>>> 2- Migrate most of the integration tests we currently have
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> to
>>>>>>>>>>>> unit
>>>>>>>> tests
>>>>>>>>>>> (since they are designed to do very little integration).
>>>>>>>>>>>>>> This is a great chance for us to practice real TDD in OFBiz
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Cheers,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Taher Alkhateeb
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Mon, Jun 13, 2016 at 7:43 AM, Pranay Pandey <
>>>>>>>>>>>>>> pranay.pandey@hotwaxsystems.com <javascript:;>> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> +1
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Best regards,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Pranay Pandey
>>>>>>>>>>>>>>> HotWax Systems
>>>>>>>>>>>>>>> http://www.hotwaxsystems.com/
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Fri, Jun 10, 2016 at 7:46 PM, Taher Alkhateeb <
>>>>>>>>>>>>>>> slidingfilaments@gmail.com <javascript:;>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>> Hello Everyone,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I was able to get a few tests running and this is very
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> doable.
>>>>>>>>>>>>>> But
>>>>>>>>>>> I
>>>>>>>>>>>> faced
>>>>>>>>>>>>>> a big problem in designing the testing framework
>>>>>>>>>>>>>>>> because
>>>>>>>>>>>>>> of
>>>>>>> ANT.
>>>>>>>>> The problem
>>>>>>>>>>>> ----------------
>>>>>>>>>>>>>>>> The way the build scripts are designed in OFBiz are
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> very
>>>>>>>>>>>>>> complex. A
>>>>>>>> master
>>>>>>>>>>>>> file calls other files which call other files. And in
>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>> middle
>>>>>>>> you
>>>>>>>>>>>> have
>>>>>>>>>>>>>> external libraries (ant-contrib) and macros, and
>>>>>>>>>>>>>>> variables,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> and
>>>>>>>>> class
>>>>>>>>>>> path
>>>>>>>>>>>>>> declarations, and and and ....
>>>>>>>>>>>>>>>> I cannot declare the tests programmatically (with JUnit
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> test
>>>>>>>>>>>>>> suites)
>>>>>>>>>> because this means lower level components would depend
>>>>>>>>>>>>>> on
>>>>>>>>>>>>>> higher
>>>>>>>> level
>>>>>>>>>>>> components. So I have to do it in ANT, by navigating
>>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> maze
>>>>>>>> of
>>>>>>>>>> build
>>>>>>>>>>>> scripts, and it was a headache for me just to read
>>>>>>>>>>>>>>> them,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> let
>>>>>>>> alone
>>>>>>>>>> modify
>>>>>>>>>>>>> them to create a testing framework.
>>>>>>>>>>>>>>>> Suggested Solution
>>>>>>>>>>>>>>>> ------------------------
>>>>>>>>>>>>>>>> I suggest to implement the testing framework in Gradle,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>> simply
>>>>>>>>> call
>>>>>>>>>>>>> it
>>>>>>>>>>>>>>> from within ant. This is a middle solution that
>>>>>>>>>>>>>>>> sustains
>>>>>>>>>>>>>> ant
>>>>>>>> for
>>>>>>>>>> now,
>>>>>>>>>>>> but
>>>>>>>>>>>>>> can allow us to switch out later.
>>>>>>>>>>>>>>>> This means I will just add one more file called
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> build.gradle
>>>>>>>>>>>>>> in
>>>>>>>>> the
>>>>>>>>>>> top
>>>>>>>>>>>>> level directory, and figure out the business logic for
>>>>>>>>>>>>>>> calling
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> the
>>>>>>>>>>> test
>>>>>>>>>>>>> suites from that file
>>>>>>>>>>>>>>> I look forward to your feedback.
>>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Taher Alkhateeb
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Wed, Jun 8, 2016 at 6:00 PM, Taher Alkhateeb <
>>>>>>>>>>>>>>>> slidingfilaments@gmail.com <javascript:;>>
>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Hi Everyone,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Thank you all for your support, JIRA created in
>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OFBIZ-7254
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> I will start working on it and try to implement ASAP
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>> get
>>>>>>>> my
>>>>>>>>>> focus
>>>>>>>>>>>> back
>>>>>>>>>>>>>>> on refactoring.
>>>>>>>>>>>>>>>>> Cheers!
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Taher Alkhateeb
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On Wed, Jun 8, 2016 at 4:58 PM, Deepak Dixit <
>>>>>>>>>>>>>>>>> deepak.dixit@hotwaxsystems.com <javascript:;>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>> +1
>>>>>>>> Thanks & Regards
>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>> Deepak Dixit
>>>>>>>>>>>>>>>>>> www.hotwaxsystems.com
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> On Wed, Jun 8, 2016 at 7:12 PM, Mridul Pathak <
>>>>>>>>>>>>>>>>>> mridul.pathak@hotwaxsystems.com <javascript:;>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>> +1
>>>>>>>>> Makes perfect sense.
>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>> Thanks & Regards,
>>>>>>>>>>>>>>>>>>> Mridul Pathak
>>>>>>>>>>>>>>>>>>> Senior Manager
>>>>>>>>>>>>>>>>>>> HotWax Systems
>>>>>>>>>>>>>>>>>>> http://www.hotwaxsystems.com
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> On Jun 8, 2016, at 2:41 PM, Taher Alkhateeb <
>>>>>>>>>>>>>>>>>>> slidingfilaments@gmail.com <javascript:;>>
>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Hello Everyone,
>>>>>>>>>>>>>>>>>>>> After refactoring the start component and while
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> starting
>>>>>>>>>>>>>>>>>> on
>>>>>>>>>>> the
>>>>>>>>>>>>> base
>>>>>>>>>>>>>>> component I realized that the testing framework
>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>> OFBiz
>>>>>>>>> is
>>>>>>>>>>> not
>>>>>>>>>>>>> good.
>>>>>>>>>>>>>>> You
>>>>>>>>>>>>>>>>>> cannot do real test driven development or
>>>>>>>>>>>>>>>>>>>> red-green-refactor
>>>>>>>>>>>>>>>>>> with
>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>> current setup, hence my proposal to change it. I
>>>>>>>>>>>>>>>>>> explain
>>>>>>>>>>>>>>>>>> below:
>>>>>>>>>>> Problem with current design
>>>>>>>>>>>>>>> ----------------------------------------
>>>>>>>>>>>>>>>>>>>> - What we have right now is not unit tests, it's
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> really
>>>>>>>>>>>>>>>>>> integration
>>>>>>>>>>> tests.
>>>>>>>>>>>>>>>>> You have to start the framework, the database,
>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>> service
>>>>>>>> engine,
>>>>>>>>>>>> the
>>>>>>>>>>>>>>>> entity engine and pretty much everything.
>>>>>>>>>>>>>>>>>> - Testing is very slow, because it's an
>>>>>>>>>>>>>>>>>>>> integration
>>>>>>>>>>>>>>>>>> test
>>>>>>>>> as
>>>>>>>>>>> I
>>>>>>>>>>>> mentioned
>>>>>>>>>>>>>> above. 10 minutes on a good computer!
>>>>>>>>>>>>>>>>>>>> - There is zero mocking! We actually have to
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> --load-data
>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>> things
>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>> work. Again, these are integration tests.
>>>>>>>>>>>>>>>>>> - Too complex: Integration tests by their nature
>>>>>>>>>>>>>>>>>>>> are
>>>>>>>>>>>>>>>>>> grabbing
>>>>>>>>> too
>>>>>>>>>>>>>> much.
>>>>>>>>>>>>>>>> Mind you, I am not objecting to integration
>>>>>>>>>>>>>>>>>>> tests
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> (I
>>>>>>> actually
>>>>>>>>> like
>>>>>>>>>>>>>> them)
>>>>>>>>>>>>>>>> but I am objecting to not having real
>>>>>>>>>>>>>>>>>>> unit-tests.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Unit
>>>>>>>> tests
>>>>>>>>>> should
>>>>>>>>>>>>> all
>>>>>>>>>>>>>>>>> run
>>>>>>>>>>>>>>>>>>> in a few seconds.
>>>>>>>>>>>>>>>>>>>> Proposed solution
>>>>>>>>>>>>>>>>>>>> --------------------------
>>>>>>>>>>>>>>>>>>>> - We keep what is considered real integration
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> tests
>>>>>>>>>>>>>>>>>> the
>>>>>>>>> way
>>>>>>>>>>> they
>>>>>>>>>>>>> are
>>>>>>>>>>>>>>> right
>>>>>>>>>>>>>>>>> now and keep using them
>>>>>>>>>>>>>>>>>>>> - We move what should be unit tests into simple
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> JUnit
>>>>>>>>>>>>>>>>>> classes,
>>>>>>>>>> and
>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>> do
>>>>>>>>>>>>>>>>>> not run them using java -jar ofbiz.jar --test,
>>>>>>>>>>>>>>>>>>> but
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> instead
>>>>>>>> run
>>>>>>>>>>>> them
>>>>>>>>>>>>>> directly from the build.xml script, so these
>>>>>>>>>>>>>>>>> files
>>>>>>>>>>>>>>>>>> are
>>>>>>>> not
>>>>>>>>>> identified
>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>> any XML document, but are simply called
>>>>>>>>>>>>>>>>>>> immediately
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> from
>>>>>>>>> the
>>>>>>>>>>> build
>>>>>>>>>>>>> scripts.
>>>>>>>>>>>>>>>> - We clearly mark the difference between
>>>>>>>>>>>>>>>>>>>> integration
>>>>>>>>>>>>>>>>>> tests
>>>>>>>>> and
>>>>>>>>>>>> unit
>>>>>>>>>>>>>> tests
>>>>>>>>>>>>>>>>> (inside the source files or in the suite
>>>>>>>>>>>>>>>>>>> declarations).
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> - We change the run-tests target in build.xml to
>>>>>>>>>>> run
>>>>>>>>>>>>>>>>>> both
>>>>>>>>> unit
>>>>>>>>>>>> tests
>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>> integration tests.
>>>>>>>>>>>>>>>>>>>> I intend to heavily refactor the framework and I
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> would
>>>>>>>>>>>>>>>>>> feel
>>>>>>>>>> better
>>>>>>>>>>>>> about
>>>>>>>>>>>>>>>> introducing this change while refactoring. What
>>>>>>>>>>>>>>>>>>> do
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> you
>>>>>>>> guys
>>>>>>>>>> think?
>>>>>>>>>>>>> Ideas?
>>>>>>>>>>>>>>>> Suggestions? Approvals and thumbs up?
>>>>>>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Taher Alkhateeb
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>> --
>> Ron Wheeler
>> President
>> Artifact Software Inc
>> email: rwheeler@artifact-software.com
>> skype: ronaldmwheeler
>> phone: 866-970-2435, ext 102
>>
>>


-- 
Ron Wheeler
President
Artifact Software Inc
email: rwheeler@artifact-software.com
skype: ronaldmwheeler
phone: 866-970-2435, ext 102

Re: Proposal to modify the testing framework for OFBiz

[Scott Gray <sc...@hotwaxsystems.com>]

I'm not looking for any sort of response really.  I was just speaking in
favor of integration tests because I think they provide better coverage at
the cost of a little bit of speed (the framework has to be running to run
them).  Most of the responses coming in were of the form "unit tests are
great because [insert a reason that applies to any type of test]".  So I
was merely pointing out that the arguments weren't specific to unit tests
and hence weren't really adding value to the discussion.

Regards
Scott

On 20 July 2016 at 02:05, Ron Wheeler <rw...@artifact-software.com>
wrote:

> It is not clear what you are expecting as a specific response.
>
> As you move up the food chain, the definition of "units" changes and the
> tests get more complicated and the design of the tests gets more complex
> (mock objects, testing race conditions).
>
> It is hard to make specific comments about what kind of testing will give
> the highest level of confidence at each level but we are going in the right
> direction.
>
> The fact that we are considering tests as a major development activity is
> a big step forward.
>
> We can argue about specific cases (scope, tools, value, etc) as we get to
> them.
>
> I think that we are at the "good idea" stage rather than at the "policy"
> stage at this point.
>
> Ron
>
>
>
> On 19/07/2016 4:23 AM, Scott Gray wrote:
>
>> I think people are missing my point because they keep replying with
>> generic
>> statements that aren't specific to unit tests.
>>
>> On 19 July 2016 at 20:08, Montalbano Florian <
>> florian.montalbano@nereide.fr>
>> wrote:
>>
>> Hi everyone,
>>>
>>> unit tests and integration tests are complementary and with test, the
>>> more
>>> the better.
>>> Implementing unit tests will be a great load of work but it is the same
>>> with every kind of test. As Taher said, we will gain a lot from those
>>> unit
>>> tests for avoiding regression while refactoring the framework. From the
>>> start component scope, it has already be proved useful.
>>> And what Ron said about encouraging the creation of "smaller class that
>>> can be tested easily" is very important too. It can be a first step to
>>> reach TDD in the OFBiz development. TDD can help to give confidence to
>>> newcomer and veteran when updating some part of the code and as Hans
>>> said,
>>> this lead to "continuous improvement and finally continuous deployment".
>>>
>>> I'm aware that this does not solve the technical implementation stuff,
>>> but
>>> I think this is an improvement that need to be supported.
>>>
>>> Have a nice day,
>>>
>>> Florian Montalbano
>>>
>>>
>>> Le 19/07/2016 09:04, Pierre Smits a écrit :
>>>
>>> As with anything, the law of diminishing returns also applies to OFBiz
>>>> and
>>>> tests. This is not true for unit tests and system integration tests, but
>>>> also for user acceptance and performance tests.
>>>>
>>>> Nevertheless, the work done up to now is a good start and - I feel
>>>> confident - appreciated. And unit tests are certainly valuable in the
>>>> framework stack. How it will be for functions (regarding components in
>>>> application and special purpose stack) needs to be addressed when we
>>>> reach
>>>> that bridge.
>>>>
>>>>
>>>> Best regards,
>>>>
>>>> Pierre Smits
>>>>
>>>> ORRTIZ.COM <http://www.orrtiz.com>
>>>> OFBiz based solutions & services
>>>>
>>>> OFBiz Extensions Marketplace
>>>> http://oem.ofbizci.net/oci-2/
>>>>
>>>> On Tue, Jul 19, 2016 at 8:22 AM, Taher Alkhateeb <
>>>> slidingfilaments@gmail.com
>>>>
>>>> wrote:
>>>>> Hi Scott,
>>>>>
>>>>> Well spoken thank you. Okay may I suggest that for any such work we
>>>>> will
>>>>> discuss it here see its Merit and if it makes sense then we take it in.
>>>>> It
>>>>> is a little early to discuss it right now because we did not yet go to
>>>>> the
>>>>> higher-level components. Once we do I'll be sure to have a conversation
>>>>> about this in here and would appreciate your input to it.
>>>>>
>>>>> Regards,
>>>>>
>>>>> Taher Alkhateeb
>>>>>
>>>>> On Jul 19, 2016 9:00 AM, "Scott Gray" <sc...@hotwaxsystems.com>
>>>>> wrote:
>>>>>
>>>>> Yeah I'm sure unit tests probably worked well for the start component
>>>>> given
>>>>>
>>>>> it is the lowest level component in the system and closest to a basic
>>>>>>
>>>>>> java
>>>>>
>>>>> app.  I just think the value proposition might decrease the further up
>>>>>>
>>>>>> the
>>>>>
>>>>> stack we move with them.  I'm not against unit tests when they prove
>>>>>> useful, but further up the stack I think we should prove the case for
>>>>>>
>>>>>> them
>>>>>
>>>>> before doing much work to support the mocking that will be required to
>>>>>>
>>>>>> keep
>>>>>
>>>>> them inline with 'best practices'.
>>>>>>
>>>>>> In OFBiz "bad or unwanted" tends to come mostly in the way of
>>>>>> increasing
>>>>>> complexity from features that require more effort to maintain than the
>>>>>> value they provide to the community.  I think there's a chance
>>>>>> attempting
>>>>>> to mock service tests could fall into that bucket.  I could be wrong,
>>>>>>
>>>>>> but I
>>>>>
>>>>> think it's worth looking into before we declare that unit tests are the
>>>>>> best form of testing for OFBiz.
>>>>>>
>>>>>> Regards
>>>>>> Scott
>>>>>>
>>>>>> On 19 July 2016 at 17:37, Taher Alkhateeb <slidingfilaments@gmail.com
>>>>>> >
>>>>>> wrote:
>>>>>>
>>>>>> Hi Scott,
>>>>>>
>>>>>>> Thank you for the feedback. To be focused exactly on integration vs
>>>>>>>
>>>>>>> unit, I
>>>>>>
>>>>>> already mentioned above that at least for me the main objective is to
>>>>>>> confidently and quickly run the tests in short bursts of red-green
>>>>>>> refactor. This allows me to refactor code without waiting in front of
>>>>>>>
>>>>>>> the
>>>>>> screen in between test cycles thus giving me immediate feedback on any
>>>>>>
>>>>>>> errors I made. Perhaps my intro was too long so this is the squeezed
>>>>>>>
>>>>>>> out
>>>>>> version of it.
>>>>>>
>>>>>>> I already had one round of testing in the start component which was
>>>>>>>
>>>>>>> much
>>>>>> faster that way and had an immediate impact. Oh and by the way, you
>>>>>> cannot
>>>>>>
>>>>>> test the start component with integration tests for example unless you
>>>>>>>
>>>>>>> do
>>>>>> it from an external component which cannot access package protected
>>>>>> items.
>>>>>>
>>>>>> This style of coding is applicable I think to any software project
>>>>>>> inclusive of OFBiz. Maybe in certain components more than others or
>>>>>>>
>>>>>>> certain
>>>>>>
>>>>>> areas more than others. But I can't see how it could be bad or
>>>>>>>
>>>>>>> unwanted.
>>>>>> Taher Alkhateeb
>>>>>>
>>>>>>> On Jul 19, 2016 8:18 AM, "Scott Gray" <sc...@hotwaxsystems.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>> Thanks Ron and Taher for your responses, I appreciate them but I
>>>>>>> don't
>>>>>>>
>>>>>> hear
>>>>>>
>>>>>>> much in the conversation that speaks to the value of unit tests over
>>>>>>>> integration tests.  Most of the thoughts shared speaks more to the
>>>>>>>>
>>>>>>>> value
>>>>>>> of
>>>>>>>
>>>>>>> tests rather than differences in the type of tests.
>>>>>>>>
>>>>>>>> Speed: At an application level we have ~685 tests that run in 35
>>>>>>>>
>>>>>>>> seconds
>>>>>>> (excluding build and data load).  Another point is that there isn't
>>>>>>> much
>>>>>>> reason why tests can't be run in parallel rather than sequentially.
>>>>>>>
>>>>>>>> TDD: Integration tests in OFBiz are as simple if not simpler to
>>>>>>>> write
>>>>>>>>
>>>>>>>> than
>>>>>>>
>>>>>>> unit tests and just as useful for TDD.
>>>>>>>> I'm not sure if I missed any other points raised regarding
>>>>>>>>
>>>>>>>> integration
>>>>>>>
>>>>>> vs.
>>>>>>
>>>>>>> unit tests?
>>>>>>>>
>>>>>>>> I'm not looking to start a big long debate on the topic, I just
>>>>>>>>
>>>>>>>> wanted
>>>>>>>
>>>>>> to
>>>>>>
>>>>>> speak out that someone out there (me) doesn't think unit tests are
>>>>>>> the
>>>>>>>
>>>>>> best
>>>>>>
>>>>>>> solution for testing OFBiz applications.
>>>>>>>>
>>>>>>>> Regards
>>>>>>>> Scott
>>>>>>>>
>>>>>>>> On 19 July 2016 at 16:52, Taher Alkhateeb <
>>>>>>>>
>>>>>>>> slidingfilaments@gmail.com>
>>>>>>>
>>>>>> wrote:
>>>>>>
>>>>>>> Hi Scott,
>>>>>>>>
>>>>>>>>> Thank you for your input. Your ideas are thought provoking and
>>>>>>>>>
>>>>>>>>> enriching
>>>>>>>> to
>>>>>>>>
>>>>>>>> the conversation.
>>>>>>>>>
>>>>>>>>> The way I look at it, the general rule is usually many unit tests,
>>>>>>>>>
>>>>>>>>> less
>>>>>>>>
>>>>>>> integration tests, lesser functional tests. So we are not excluding
>>>>>>>
>>>>>>>> any
>>>>>>>>
>>>>>>> types of test, all of them are important in certain areas for
>>>>>>>
>>>>>>>> certain
>>>>>>>>
>>>>>>> purposes with certain quantity. Usually integration tests are less
>>>>>>
>>>>>>> because
>>>>>>>>
>>>>>>>> as you said they just grab more. I like the picture below as a
>>>>>>>>>
>>>>>>>>> general
>>>>>>>>
>>>>>>> guideline
>>>>>>>
>>>>>>>> http://i.stack.imgur.com/fjQvQ.png
>>>>>>>>>
>>>>>>>>> As you already mentioned unit tests are useful for the framework. I
>>>>>>>>> discovered errors in code that I wrote which I was very very
>>>>>>>>>
>>>>>>>>> careful
>>>>>>>>
>>>>>>> with.
>>>>>>
>>>>>>> I immediately learned the lesson that humans are not designed to
>>>>>>>>>
>>>>>>>>> code,
>>>>>>>>
>>>>>>> and
>>>>>>>
>>>>>>>> TDD gives you confidence as you build your code with those short
>>>>>>>>>
>>>>>>>>> 30-60
>>>>>>>>
>>>>>>> second red-green refactors. I feel much much safer and more
>>>>>>>
>>>>>>>> confident
>>>>>>>>
>>>>>>> writing code that way, the test also documents how to use the api,
>>>>>>
>>>>>>> refactoring and feature change becomes less terrifying. Also messy
>>>>>>>>>
>>>>>>>>> code
>>>>>>>>
>>>>>>> is
>>>>>>>
>>>>>>>> usually not test friendly thus refactoring and unit tests go
>>>>>>>>>
>>>>>>>>> hand-in-hand
>>>>>>>> for improving the code base.
>>>>>>>>
>>>>>>>>> Also, I am sorry to say that the framework code is rather messy and
>>>>>>>>>
>>>>>>>>> brittle
>>>>>>>>
>>>>>>>> in many places. I think you probably encountered this yourself. The
>>>>>>>>>
>>>>>>>>> same
>>>>>>>> is
>>>>>>>>
>>>>>>>> said for the applications. Now If we start refactoring without unit
>>>>>>>>>
>>>>>>>>> tests
>>>>>>>> then we are back to scary business. So much can go wrong so fast
>>>>>>>> and
>>>>>>>>
>>>>>>> break
>>>>>>
>>>>>>> things you never expected to break. The framework with all
>>>>>>>>>
>>>>>>>>> applications
>>>>>>>>
>>>>>>> require heavy refactoring of things like massive ugly methods,
>>>>>>>
>>>>>>>> sandwiched
>>>>>>>> logic, heavy shared mutable state, hidden dependencies, poor
>>>>>>>>
>>>>>>>>> interfaces, and much more. Every time I touch something I get
>>>>>>>>>
>>>>>>>>> shocked
>>>>>>>>
>>>>>>> at
>>>>>>
>>>>>>> how bad it looks, spaghetti logic to no end. If you refactor with
>>>>>>>>
>>>>>>>>> integration tests instead of unit tests then you will come down to
>>>>>>>>>
>>>>>>>>> a
>>>>>>>>
>>>>>>> screeching halt as you wait between these test cycles. My computer
>>>>>>
>>>>>>> actually
>>>>>>>>
>>>>>>>> takes 10 minutes or more for a full clean load data and testing.
>>>>>>>>>
>>>>>>>>> Now talking about mocking you raise some interesting points, what
>>>>>>>>>
>>>>>>>>> can
>>>>>>>>
>>>>>>> you
>>>>>>
>>>>>>> mock vs utilizing integration tests? This is an excellent question
>>>>>>>> that
>>>>>>>>
>>>>>>> does not have a simple answer, and you mostly learn as you code,
>>>>>>>
>>>>>>>> difficult
>>>>>>>>
>>>>>>>> to envision without coding. However simple things like Java
>>>>>>>>>
>>>>>>>>> services
>>>>>>>>
>>>>>>> can
>>>>>>
>>>>>>> be
>>>>>>>>
>>>>>>>> mocked with a standard mocking class that you use everywhere.
>>>>>>>>>
>>>>>>>>> Transactions
>>>>>>>>
>>>>>>>> are difficult to mock and better left to integration tests I think.
>>>>>>>>>
>>>>>>>>> SECAs
>>>>>>>> might be mocked by simply passing output to input as chaining
>>>>>>>> methods.
>>>>>>>>
>>>>>>> Mind
>>>>>>>
>>>>>>>> you I am not 100% sure of all of this, coding is the ultimate guide
>>>>>>>>>
>>>>>>>>> to
>>>>>>>>
>>>>>>> what
>>>>>>>
>>>>>>>> can or cannot be done.
>>>>>>>>>
>>>>>>>>> So I am not suggesting unit tests as a best-practice (even though
>>>>>>>>>
>>>>>>>>> it
>>>>>>>>
>>>>>>> is).
>>>>>>
>>>>>>> Instead I suggest it as something that I and others did and got a
>>>>>>>> huge
>>>>>>>>
>>>>>>> psychological relief and confidence and comfort from. Swimming in
>>>>>>>
>>>>>>>> code
>>>>>>>>
>>>>>>> without tests is a terrifying business, made more terrifying if the
>>>>>>>
>>>>>>>> code
>>>>>>>> is
>>>>>>>>
>>>>>>>> bad. Short bursts of red/green/red/green makes you feel good as you
>>>>>>>>>
>>>>>>>>> build
>>>>>>>> up your logic. And I don't have an exact vision of how to do it,
>>>>>>>> because
>>>>>>>> the details always win.
>>>>>>>>
>>>>>>>>> I look forward to hearing your thoughts and thank you for enriching
>>>>>>>>>
>>>>>>>>> this
>>>>>>>> conversation.
>>>>>>>>
>>>>>>>>> Taher Alkhateeb
>>>>>>>>>
>>>>>>>>> On Tuesday, 19 July 2016, Scott Gray <scott.gray@hotwaxsystems.com
>>>>>>>>>
>>>>>>>>> wrote:
>>>>>>>>
>>>>>>>> I know I'm late to the party here, but I just want to say that I
>>>>>>>>> think
>>>>>>>>>
>>>>>>>> integration tests have far greater value to OFBiz than unit
>>>>>>>>
>>>>>>>>> tests.
>>>>>>>>>
>>>>>>>> Mostly
>>>>>>
>>>>>>> because we tend to have quite a low number of tests and
>>>>>>>>>>
>>>>>>>>>> integration
>>>>>>>>>
>>>>>>>> tests
>>>>>>
>>>>>>> give us much better coverage per line of test code and the tests
>>>>>>>>> are
>>>>>>>>>
>>>>>>>> much
>>>>>>>
>>>>>>>> closer to the real world scenarios the application might
>>>>>>>>> encounter.
>>>>>>>>>
>>>>>>>> I don't really see how unit tests could be applied to
>>>>>>
>>>>>>> non-framework
>>>>>>>>>
>>>>>>>> testing
>>>>>>
>>>>>>> in a useful manner, could you expand on your vision in that
>>>>>>>>>>
>>>>>>>>>> regard?
>>>>>>>>>
>>>>>>>> I
>>>>>>
>>>>>>> mean
>>>>>>>>
>>>>>>>>> would we be testing something smaller than a service as the
>>>>>>>>>>
>>>>>>>>>> 'unit'?
>>>>>>>>>
>>>>>>>> What
>>>>>>
>>>>>>> would we mock? Would transaction management still be active? What
>>>>>>>>> happens
>>>>>>>>> when the service calls another service, I guess we mock the
>>>>>>>>> response
>>>>>>>>>
>>>>>>>> from
>>>>>>>
>>>>>>>> that service (how)?
>>>>>>>>>
>>>>>>>>>> It just seems a very complicated method to achieve a less
>>>>>>>>>>
>>>>>>>>>> thorough
>>>>>>>>>
>>>>>>>> but
>>>>>>
>>>>>>> albeit faster (maybe) test result.
>>>>>>>>
>>>>>>>>> A build, data load and full test run takes 4m 9s on my laptop.
>>>>>>>>>>
>>>>>>>>>> Excluding
>>>>>>>>> build, data load and framework tests: the application level tests
>>>>>>>>> take
>>>>>>>>>
>>>>>>>> 35s,
>>>>>>>>
>>>>>>>>> not very expensive IMO.  The data load time can be reduced to
>>>>>>>>>>
>>>>>>>>>> practically
>>>>>>>>> nothing by copying a clean slate database into runtime for each
>>>>>>>>> run.
>>>>>>>>>
>>>>>>>> I'm mostly just suggesting we be wary of adding complicated
>>>>>>>
>>>>>>>> testing
>>>>>>>>>
>>>>>>>> procedures in the hope of achieving some 'best practice' result
>>>>>>
>>>>>>> which
>>>>>>>>>
>>>>>>>> in
>>>>>>>
>>>>>>>> reality will provide minimal benefits.
>>>>>>>>>
>>>>>>>>>> Regards
>>>>>>>>>> Scott
>>>>>>>>>>
>>>>>>>>>> On 18 July 2016 at 18:57, Taher Alkhateeb <
>>>>>>>>>>
>>>>>>>>>> slidingfilaments@gmail.com
>>>>>>>>>
>>>>>>>> <javascript:;>>
>>>>>>>>
>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>> Hello Akash,
>>>>>>>>>>
>>>>>>>>>>> Fantastic, I have a few unit tests almost done to be included
>>>>>>>>>>>
>>>>>>>>>>> in
>>>>>>>>>>
>>>>>>>>> the
>>>>>>
>>>>>>> start
>>>>>>>>
>>>>>>>>> component. I will create a new subtask under OFBIZ-1463 to
>>>>>>>>>>>
>>>>>>>>>>> commit
>>>>>>>>>>
>>>>>>>>> the
>>>>>>
>>>>>>> tests
>>>>>>>>
>>>>>>>>> so you can use them as a reference if you like to.
>>>>>>>>>>>
>>>>>>>>>>> I also recommend that you follow the same directory structure
>>>>>>>>>>>
>>>>>>>>>>> between
>>>>>>>>>>
>>>>>>>>> the
>>>>>>>>
>>>>>>>>> test code and production code. So for example:
>>>>>>>>>>
>>>>>>>>>>> Production code:
>>>>>>>>>>>
>>>>>>>>>>> framework/start/src/main/java/org/apache/ofbiz/base/start
>>>>>>>>>>
>>>>>>>>>> Test code:
>>>>>>>>>>>
>>>>>>>>>>> framework/start/src/test/java/org/apache/ofbiz/base/start
>>>>>>>>>>
>>>>>>>>> The benefit of this hierarchy is that you can access non-public
>>>>>>>>
>>>>>>>>> (package
>>>>>>>>>> protected) methods for testing. This is in fact exactly what I
>>>>>>>>>> needed
>>>>>>>>>>
>>>>>>>>> to
>>>>>>>>
>>>>>>>>> be
>>>>>>>>>>
>>>>>>>>>> able to apply some of the tests.
>>>>>>>>>>>
>>>>>>>>>>> Cheers,
>>>>>>>>>>>
>>>>>>>>>>> Taher Alkhateeb
>>>>>>>>>>>
>>>>>>>>>>> On Mon, Jul 18, 2016 at 9:22 AM, Akash Jain <
>>>>>>>>>>>
>>>>>>>>>>> akash.jain@hotwaxsystems.com <javascript:;>>
>>>>>>>>>>
>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>> Thanks Taher for nice initiative!
>>>>>>>>>>>
>>>>>>>>>>>> We are planning to written unit tests to all components under
>>>>>>>>>>>>
>>>>>>>>>>>> OFBIZ-1463
>>>>>>>>>>> Thanks and Regards
>>>>>>>>>>>
>>>>>>>>>>>> --
>>>>>>>>>>>> Akash Jain
>>>>>>>>>>>>
>>>>>>>>>>>> On Mon, Jul 18, 2016 at 10:36 AM, Taher Alkhateeb <
>>>>>>>>>>>> slidingfilaments@gmail.com <javascript:;>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>> Hello Everyone,
>>>>>>>>>>>>
>>>>>>>>>>>>> In reference to this thread and the Jira OFBIZ-7254, I'm
>>>>>>>>>>>>>
>>>>>>>>>>>>> very
>>>>>>>>>>>>
>>>>>>>>>>> happy
>>>>>>
>>>>>>> to
>>>>>>>>>
>>>>>>>>>> announce that OFBiz is now ready for applying unit tests
>>>>>>>>>>>
>>>>>>>>>>>> with
>>>>>>>>>>>>
>>>>>>>>>>> only
>>>>>>
>>>>>>> 8
>>>>>>>>>
>>>>>>>>> new
>>>>>>>>>>
>>>>>>>>>>> lines of code in the build script (r1753143) :)
>>>>>>>>>>>>
>>>>>>>>>>>>> I recommend we do the following moving forward:
>>>>>>>>>>>>>
>>>>>>>>>>>>> 1- Introduce unit tests as much as we can to all components
>>>>>>>>>>>>> 2- Migrate most of the integration tests we currently have
>>>>>>>>>>>>>
>>>>>>>>>>>>> to
>>>>>>>>>>>>
>>>>>>>>>>> unit
>>>>>>
>>>>>>> tests
>>>>>>>>>
>>>>>>>>>> (since they are designed to do very little integration).
>>>>>>>>>>>>
>>>>>>>>>>>>> This is a great chance for us to practice real TDD in OFBiz
>>>>>>>>>>>>>
>>>>>>>>>>>>> Cheers,
>>>>>>>>>>>>>
>>>>>>>>>>>>> Taher Alkhateeb
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Mon, Jun 13, 2016 at 7:43 AM, Pranay Pandey <
>>>>>>>>>>>>> pranay.pandey@hotwaxsystems.com <javascript:;>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>> +1
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Best regards,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Pranay Pandey
>>>>>>>>>>>>>> HotWax Systems
>>>>>>>>>>>>>> http://www.hotwaxsystems.com/
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Fri, Jun 10, 2016 at 7:46 PM, Taher Alkhateeb <
>>>>>>>>>>>>>> slidingfilaments@gmail.com <javascript:;>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>> Hello Everyone,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I was able to get a few tests running and this is very
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> doable.
>>>>>>>>>>>>>>
>>>>>>>>>>>>> But
>>>>>>>>>
>>>>>>>>>> I
>>>>>>>>>>>
>>>>>>>>>>> faced
>>>>>>>>>>>>
>>>>>>>>>>>>> a big problem in designing the testing framework
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> because
>>>>>>>>>>>>>>
>>>>>>>>>>>>> of
>>>>>>
>>>>>> ANT.
>>>>>>>
>>>>>>>> The problem
>>>>>>>>>>
>>>>>>>>>>> ----------------
>>>>>>>>>>>>>>> The way the build scripts are designed in OFBiz are
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> very
>>>>>>>>>>>>>>
>>>>>>>>>>>>> complex. A
>>>>>>
>>>>>>> master
>>>>>>>>>>>
>>>>>>>>>>>> file calls other files which call other files. And in
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>
>>>>>>>>>>>>> middle
>>>>>>
>>>>>>> you
>>>>>>>>>>
>>>>>>>>>>> have
>>>>>>>>>>>>
>>>>>>>>>>>>> external libraries (ant-contrib) and macros, and
>>>>>>>>>>>>>> variables,
>>>>>>>>>>>>>>
>>>>>>>>>>>>> and
>>>>>>>
>>>>>>>> class
>>>>>>>>>
>>>>>>>>>> path
>>>>>>>>>>>>
>>>>>>>>>>>>> declarations, and and and ....
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I cannot declare the tests programmatically (with JUnit
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> test
>>>>>>>>>>>>>>
>>>>>>>>>>>>> suites)
>>>>>>>>
>>>>>>>>> because this means lower level components would depend
>>>>>>>>>>>>
>>>>>>>>>>>>> on
>>>>>>>>>>>>>>
>>>>>>>>>>>>> higher
>>>>>>
>>>>>>> level
>>>>>>>>>>
>>>>>>>>>>> components. So I have to do it in ANT, by navigating
>>>>>>>>>>>>>
>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>
>>>>>>>>>>>>> maze
>>>>>>
>>>>>>> of
>>>>>>>>>
>>>>>>>>> build
>>>>>>>>>>
>>>>>>>>>>> scripts, and it was a headache for me just to read
>>>>>>>>>>>>>
>>>>>>>>>>>>>> them,
>>>>>>>>>>>>>>
>>>>>>>>>>>>> let
>>>>>>
>>>>>>> alone
>>>>>>>>
>>>>>>>>> modify
>>>>>>>>>>>
>>>>>>>>>>>> them to create a testing framework.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Suggested Solution
>>>>>>>>>>>>>>> ------------------------
>>>>>>>>>>>>>>> I suggest to implement the testing framework in Gradle,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>
>>>>>>>>>>>>> simply
>>>>>>>
>>>>>>>> call
>>>>>>>>>>>
>>>>>>>>>>>> it
>>>>>>>>>>>>>
>>>>>>>>>>>>>> from within ant. This is a middle solution that
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> sustains
>>>>>>>>>>>>>>
>>>>>>>>>>>>> ant
>>>>>>
>>>>>>> for
>>>>>>>>
>>>>>>>>> now,
>>>>>>>>>>
>>>>>>>>>>> but
>>>>>>>>>>>>
>>>>>>>>>>>>> can allow us to switch out later.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> This means I will just add one more file called
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> build.gradle
>>>>>>>>>>>>>>
>>>>>>>>>>>>> in
>>>>>>>>
>>>>>>>> the
>>>>>>>>>
>>>>>>>>>> top
>>>>>>>>>>>
>>>>>>>>>>>> level directory, and figure out the business logic for
>>>>>>>>>>>>>
>>>>>>>>>>>>>> calling
>>>>>>>>>>>>>>
>>>>>>>>>>>>> the
>>>>>>>>>
>>>>>>>>>> test
>>>>>>>>>>>
>>>>>>>>>>>> suites from that file
>>>>>>>>>>>>>
>>>>>>>>>>>>>> I look forward to your feedback.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Taher Alkhateeb
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Wed, Jun 8, 2016 at 6:00 PM, Taher Alkhateeb <
>>>>>>>>>>>>>>> slidingfilaments@gmail.com <javascript:;>>
>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Hi Everyone,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Thank you all for your support, JIRA created in
>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OFBIZ-7254
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I will start working on it and try to implement ASAP
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> get
>>>>>>
>>>>>>> my
>>>>>>>>
>>>>>>>>> focus
>>>>>>>>>>
>>>>>>>>>>> back
>>>>>>>>>>>>>
>>>>>>>>>>>>>> on refactoring.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Cheers!
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Taher Alkhateeb
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Wed, Jun 8, 2016 at 4:58 PM, Deepak Dixit <
>>>>>>>>>>>>>>>> deepak.dixit@hotwaxsystems.com <javascript:;>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> +1
>>>>>>
>>>>>>> Thanks & Regards
>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>> Deepak Dixit
>>>>>>>>>>>>>>>>> www.hotwaxsystems.com
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On Wed, Jun 8, 2016 at 7:12 PM, Mridul Pathak <
>>>>>>>>>>>>>>>>> mridul.pathak@hotwaxsystems.com <javascript:;>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> +1
>>>>>>>
>>>>>>>> Makes perfect sense.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>> Thanks & Regards,
>>>>>>>>>>>>>>>>>> Mridul Pathak
>>>>>>>>>>>>>>>>>> Senior Manager
>>>>>>>>>>>>>>>>>> HotWax Systems
>>>>>>>>>>>>>>>>>> http://www.hotwaxsystems.com
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> On Jun 8, 2016, at 2:41 PM, Taher Alkhateeb <
>>>>>>>>>>>>>>>>>> slidingfilaments@gmail.com <javascript:;>>
>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Hello Everyone,
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> After refactoring the start component and while
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> starting
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> on
>>>>>>>>>
>>>>>>>>>> the
>>>>>>>>>>>
>>>>>>>>>>>> base
>>>>>>>>>>>>>
>>>>>>>>>>>>>> component I realized that the testing framework
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> OFBiz
>>>>>>>
>>>>>>>> is
>>>>>>>>>>
>>>>>>>>>> not
>>>>>>>>>>>
>>>>>>>>>>>> good.
>>>>>>>>>>>>>
>>>>>>>>>>>>>> You
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> cannot do real test driven development or
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> red-green-refactor
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> with
>>>>>>>>>>>
>>>>>>>>>>>> the
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> current setup, hence my proposal to change it. I
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> explain
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> below:
>>>>>>>>>
>>>>>>>>>> Problem with current design
>>>>>>>>>>>>>
>>>>>>>>>>>>>> ----------------------------------------
>>>>>>>>>>>>>>>>>>> - What we have right now is not unit tests, it's
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> really
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> integration
>>>>>>>>>
>>>>>>>>>> tests.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> You have to start the framework, the database,
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> service
>>>>>>
>>>>>>> engine,
>>>>>>>>>>
>>>>>>>>>>> the
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> entity engine and pretty much everything.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> - Testing is very slow, because it's an
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> integration
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> test
>>>>>>>
>>>>>>>> as
>>>>>>>>>
>>>>>>>>>> I
>>>>>>>>>>>
>>>>>>>>>>> mentioned
>>>>>>>>>>>>
>>>>>>>>>>>>> above. 10 minutes on a good computer!
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> - There is zero mocking! We actually have to
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> --load-data
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> for
>>>>>>>>>
>>>>>>>>>> things
>>>>>>>>>>>
>>>>>>>>>>>> to
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> work. Again, these are integration tests.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> - Too complex: Integration tests by their nature
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> are
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> grabbing
>>>>>>>
>>>>>>>> too
>>>>>>>>>>>>
>>>>>>>>>>>>> much.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Mind you, I am not objecting to integration
>>>>>>>>>>>>>>>>>> tests
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> (I
>>>>>>
>>>>>> actually
>>>>>>>
>>>>>>>> like
>>>>>>>>>>>>
>>>>>>>>>>>>> them)
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> but I am objecting to not having real
>>>>>>>>>>>>>>>>>> unit-tests.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Unit
>>>>>>
>>>>>>> tests
>>>>>>>>
>>>>>>>>> should
>>>>>>>>>>>
>>>>>>>>>>>> all
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> run
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> in a few seconds.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Proposed solution
>>>>>>>>>>>>>>>>>>> --------------------------
>>>>>>>>>>>>>>>>>>> - We keep what is considered real integration
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> tests
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> the
>>>>>>>
>>>>>>>> way
>>>>>>>>>
>>>>>>>>>> they
>>>>>>>>>>>
>>>>>>>>>>>> are
>>>>>>>>>>>>>
>>>>>>>>>>>>>> right
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> now and keep using them
>>>>>>>>>>>>>>>>>>> - We move what should be unit tests into simple
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> JUnit
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> classes,
>>>>>>>>
>>>>>>>>> and
>>>>>>>>>>>>
>>>>>>>>>>>>> we
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> do
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> not run them using java -jar ofbiz.jar --test,
>>>>>>>>>>>>>>>>>> but
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> instead
>>>>>>
>>>>>>> run
>>>>>>>>>>
>>>>>>>>>>> them
>>>>>>>>>>>>
>>>>>>>>>>>>> directly from the build.xml script, so these
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> files
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> are
>>>>>>
>>>>>>> not
>>>>>>>>
>>>>>>>>> identified
>>>>>>>>>>
>>>>>>>>>>> in
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> any XML document, but are simply called
>>>>>>>>>>>>>>>>>> immediately
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> from
>>>>>>>
>>>>>>>> the
>>>>>>>>>
>>>>>>>>>> build
>>>>>>>>>>>
>>>>>>>>>>>> scripts.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> - We clearly mark the difference between
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> integration
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> tests
>>>>>>>
>>>>>>>> and
>>>>>>>>>>
>>>>>>>>>>> unit
>>>>>>>>>>>>
>>>>>>>>>>>>> tests
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> (inside the source files or in the suite
>>>>>>>>>>>>>>>>>> declarations).
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> - We change the run-tests target in build.xml to
>>>>>>>>>
>>>>>>>>>> run
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> both
>>>>>>>
>>>>>>>> unit
>>>>>>>>>>
>>>>>>>>>>> tests
>>>>>>>>>>>>
>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> integration tests.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> I intend to heavily refactor the framework and I
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> would
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> feel
>>>>>>>>
>>>>>>>>> better
>>>>>>>>>>>
>>>>>>>>>>>> about
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> introducing this change while refactoring. What
>>>>>>>>>>>>>>>>>> do
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> you
>>>>>>
>>>>>>> guys
>>>>>>>>
>>>>>>>>> think?
>>>>>>>>>>>
>>>>>>>>>>>> Ideas?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Suggestions? Approvals and thumbs up?
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Taher Alkhateeb
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>
> --
> Ron Wheeler
> President
> Artifact Software Inc
> email: rwheeler@artifact-software.com
> skype: ronaldmwheeler
> phone: 866-970-2435, ext 102
>
>

Re: Proposal to modify the testing framework for OFBiz

[Ron Wheeler <rw...@artifact-software.com>]

It is not clear what you are expecting as a specific response.

As you move up the food chain, the definition of "units" changes and the 
tests get more complicated and the design of the tests gets more complex 
(mock objects, testing race conditions).

It is hard to make specific comments about what kind of testing will 
give the highest level of confidence at each level but we are going in 
the right direction.

The fact that we are considering tests as a major development activity 
is a big step forward.

We can argue about specific cases (scope, tools, value, etc) as we get 
to them.

I think that we are at the "good idea" stage rather than at the "policy" 
stage at this point.

Ron


On 19/07/2016 4:23 AM, Scott Gray wrote:
> I think people are missing my point because they keep replying with generic
> statements that aren't specific to unit tests.
>
> On 19 July 2016 at 20:08, Montalbano Florian <fl...@nereide.fr>
> wrote:
>
>> Hi everyone,
>>
>> unit tests and integration tests are complementary and with test, the more
>> the better.
>> Implementing unit tests will be a great load of work but it is the same
>> with every kind of test. As Taher said, we will gain a lot from those unit
>> tests for avoiding regression while refactoring the framework. From the
>> start component scope, it has already be proved useful.
>> And what Ron said about encouraging the creation of "smaller class that
>> can be tested easily" is very important too. It can be a first step to
>> reach TDD in the OFBiz development. TDD can help to give confidence to
>> newcomer and veteran when updating some part of the code and as Hans said,
>> this lead to "continuous improvement and finally continuous deployment".
>>
>> I'm aware that this does not solve the technical implementation stuff, but
>> I think this is an improvement that need to be supported.
>>
>> Have a nice day,
>>
>> Florian Montalbano
>>
>>
>> Le 19/07/2016 09:04, Pierre Smits a �crit :
>>
>>> As with anything, the law of diminishing returns also applies to OFBiz and
>>> tests. This is not true for unit tests and system integration tests, but
>>> also for user acceptance and performance tests.
>>>
>>> Nevertheless, the work done up to now is a good start and - I feel
>>> confident - appreciated. And unit tests are certainly valuable in the
>>> framework stack. How it will be for functions (regarding components in
>>> application and special purpose stack) needs to be addressed when we reach
>>> that bridge.
>>>
>>>
>>> Best regards,
>>>
>>> Pierre Smits
>>>
>>> ORRTIZ.COM <http://www.orrtiz.com>
>>> OFBiz based solutions & services
>>>
>>> OFBiz Extensions Marketplace
>>> http://oem.ofbizci.net/oci-2/
>>>
>>> On Tue, Jul 19, 2016 at 8:22 AM, Taher Alkhateeb <
>>> slidingfilaments@gmail.com
>>>
>>>> wrote:
>>>> Hi Scott,
>>>>
>>>> Well spoken thank you. Okay may I suggest that for any such work we will
>>>> discuss it here see its Merit and if it makes sense then we take it in.
>>>> It
>>>> is a little early to discuss it right now because we did not yet go to
>>>> the
>>>> higher-level components. Once we do I'll be sure to have a conversation
>>>> about this in here and would appreciate your input to it.
>>>>
>>>> Regards,
>>>>
>>>> Taher Alkhateeb
>>>>
>>>> On Jul 19, 2016 9:00 AM, "Scott Gray" <sc...@hotwaxsystems.com>
>>>> wrote:
>>>>
>>>> Yeah I'm sure unit tests probably worked well for the start component
>>>> given
>>>>
>>>>> it is the lowest level component in the system and closest to a basic
>>>>>
>>>> java
>>>>
>>>>> app.  I just think the value proposition might decrease the further up
>>>>>
>>>> the
>>>>
>>>>> stack we move with them.  I'm not against unit tests when they prove
>>>>> useful, but further up the stack I think we should prove the case for
>>>>>
>>>> them
>>>>
>>>>> before doing much work to support the mocking that will be required to
>>>>>
>>>> keep
>>>>
>>>>> them inline with 'best practices'.
>>>>>
>>>>> In OFBiz "bad or unwanted" tends to come mostly in the way of increasing
>>>>> complexity from features that require more effort to maintain than the
>>>>> value they provide to the community.  I think there's a chance
>>>>> attempting
>>>>> to mock service tests could fall into that bucket.  I could be wrong,
>>>>>
>>>> but I
>>>>
>>>>> think it's worth looking into before we declare that unit tests are the
>>>>> best form of testing for OFBiz.
>>>>>
>>>>> Regards
>>>>> Scott
>>>>>
>>>>> On 19 July 2016 at 17:37, Taher Alkhateeb <sl...@gmail.com>
>>>>> wrote:
>>>>>
>>>>> Hi Scott,
>>>>>> Thank you for the feedback. To be focused exactly on integration vs
>>>>>>
>>>>> unit, I
>>>>>
>>>>>> already mentioned above that at least for me the main objective is to
>>>>>> confidently and quickly run the tests in short bursts of red-green
>>>>>> refactor. This allows me to refactor code without waiting in front of
>>>>>>
>>>>> the
>>>>> screen in between test cycles thus giving me immediate feedback on any
>>>>>> errors I made. Perhaps my intro was too long so this is the squeezed
>>>>>>
>>>>> out
>>>>> version of it.
>>>>>> I already had one round of testing in the start component which was
>>>>>>
>>>>> much
>>>>> faster that way and had an immediate impact. Oh and by the way, you
>>>>> cannot
>>>>>
>>>>>> test the start component with integration tests for example unless you
>>>>>>
>>>>> do
>>>>> it from an external component which cannot access package protected
>>>>> items.
>>>>>
>>>>>> This style of coding is applicable I think to any software project
>>>>>> inclusive of OFBiz. Maybe in certain components more than others or
>>>>>>
>>>>> certain
>>>>>
>>>>>> areas more than others. But I can't see how it could be bad or
>>>>>>
>>>>> unwanted.
>>>>> Taher Alkhateeb
>>>>>> On Jul 19, 2016 8:18 AM, "Scott Gray" <sc...@hotwaxsystems.com>
>>>>>> wrote:
>>>>>>
>>>>>> Thanks Ron and Taher for your responses, I appreciate them but I
>>>>>> don't
>>>>> hear
>>>>>>> much in the conversation that speaks to the value of unit tests over
>>>>>>> integration tests.  Most of the thoughts shared speaks more to the
>>>>>>>
>>>>>> value
>>>>>> of
>>>>>>
>>>>>>> tests rather than differences in the type of tests.
>>>>>>>
>>>>>>> Speed: At an application level we have ~685 tests that run in 35
>>>>>>>
>>>>>> seconds
>>>>>> (excluding build and data load).  Another point is that there isn't
>>>>>> much
>>>>>> reason why tests can't be run in parallel rather than sequentially.
>>>>>>> TDD: Integration tests in OFBiz are as simple if not simpler to write
>>>>>>>
>>>>>> than
>>>>>>
>>>>>>> unit tests and just as useful for TDD.
>>>>>>> I'm not sure if I missed any other points raised regarding
>>>>>>>
>>>>>> integration
>>>>> vs.
>>>>>>> unit tests?
>>>>>>>
>>>>>>> I'm not looking to start a big long debate on the topic, I just
>>>>>>>
>>>>>> wanted
>>>>> to
>>>>>
>>>>>> speak out that someone out there (me) doesn't think unit tests are
>>>>>> the
>>>>> best
>>>>>>> solution for testing OFBiz applications.
>>>>>>>
>>>>>>> Regards
>>>>>>> Scott
>>>>>>>
>>>>>>> On 19 July 2016 at 16:52, Taher Alkhateeb <
>>>>>>>
>>>>>> slidingfilaments@gmail.com>
>>>>> wrote:
>>>>>>> Hi Scott,
>>>>>>>> Thank you for your input. Your ideas are thought provoking and
>>>>>>>>
>>>>>>> enriching
>>>>>>> to
>>>>>>>
>>>>>>>> the conversation.
>>>>>>>>
>>>>>>>> The way I look at it, the general rule is usually many unit tests,
>>>>>>>>
>>>>>>> less
>>>>>> integration tests, lesser functional tests. So we are not excluding
>>>>>>> any
>>>>>> types of test, all of them are important in certain areas for
>>>>>>> certain
>>>>> purposes with certain quantity. Usually integration tests are less
>>>>>>> because
>>>>>>>
>>>>>>>> as you said they just grab more. I like the picture below as a
>>>>>>>>
>>>>>>> general
>>>>>> guideline
>>>>>>>> http://i.stack.imgur.com/fjQvQ.png
>>>>>>>>
>>>>>>>> As you already mentioned unit tests are useful for the framework. I
>>>>>>>> discovered errors in code that I wrote which I was very very
>>>>>>>>
>>>>>>> careful
>>>>> with.
>>>>>>>> I immediately learned the lesson that humans are not designed to
>>>>>>>>
>>>>>>> code,
>>>>>> and
>>>>>>>> TDD gives you confidence as you build your code with those short
>>>>>>>>
>>>>>>> 30-60
>>>>>> second red-green refactors. I feel much much safer and more
>>>>>>> confident
>>>>> writing code that way, the test also documents how to use the api,
>>>>>>>> refactoring and feature change becomes less terrifying. Also messy
>>>>>>>>
>>>>>>> code
>>>>>> is
>>>>>>>> usually not test friendly thus refactoring and unit tests go
>>>>>>>>
>>>>>>> hand-in-hand
>>>>>>> for improving the code base.
>>>>>>>> Also, I am sorry to say that the framework code is rather messy and
>>>>>>>>
>>>>>>> brittle
>>>>>>>
>>>>>>>> in many places. I think you probably encountered this yourself. The
>>>>>>>>
>>>>>>> same
>>>>>>> is
>>>>>>>
>>>>>>>> said for the applications. Now If we start refactoring without unit
>>>>>>>>
>>>>>>> tests
>>>>>>> then we are back to scary business. So much can go wrong so fast
>>>>>>> and
>>>>> break
>>>>>>>> things you never expected to break. The framework with all
>>>>>>>>
>>>>>>> applications
>>>>>> require heavy refactoring of things like massive ugly methods,
>>>>>>> sandwiched
>>>>>>> logic, heavy shared mutable state, hidden dependencies, poor
>>>>>>>> interfaces, and much more. Every time I touch something I get
>>>>>>>>
>>>>>>> shocked
>>>>> at
>>>>>>> how bad it looks, spaghetti logic to no end. If you refactor with
>>>>>>>> integration tests instead of unit tests then you will come down to
>>>>>>>>
>>>>>>> a
>>>>> screeching halt as you wait between these test cycles. My computer
>>>>>>> actually
>>>>>>>
>>>>>>>> takes 10 minutes or more for a full clean load data and testing.
>>>>>>>>
>>>>>>>> Now talking about mocking you raise some interesting points, what
>>>>>>>>
>>>>>>> can
>>>>> you
>>>>>>> mock vs utilizing integration tests? This is an excellent question
>>>>>>> that
>>>>>> does not have a simple answer, and you mostly learn as you code,
>>>>>>> difficult
>>>>>>>
>>>>>>>> to envision without coding. However simple things like Java
>>>>>>>>
>>>>>>> services
>>>>> can
>>>>>>> be
>>>>>>>
>>>>>>>> mocked with a standard mocking class that you use everywhere.
>>>>>>>>
>>>>>>> Transactions
>>>>>>>
>>>>>>>> are difficult to mock and better left to integration tests I think.
>>>>>>>>
>>>>>>> SECAs
>>>>>>> might be mocked by simply passing output to input as chaining
>>>>>>> methods.
>>>>>> Mind
>>>>>>>> you I am not 100% sure of all of this, coding is the ultimate guide
>>>>>>>>
>>>>>>> to
>>>>>> what
>>>>>>>> can or cannot be done.
>>>>>>>>
>>>>>>>> So I am not suggesting unit tests as a best-practice (even though
>>>>>>>>
>>>>>>> it
>>>>> is).
>>>>>>> Instead I suggest it as something that I and others did and got a
>>>>>>> huge
>>>>>> psychological relief and confidence and comfort from. Swimming in
>>>>>>> code
>>>>>> without tests is a terrifying business, made more terrifying if the
>>>>>>> code
>>>>>>> is
>>>>>>>
>>>>>>>> bad. Short bursts of red/green/red/green makes you feel good as you
>>>>>>>>
>>>>>>> build
>>>>>>> up your logic. And I don't have an exact vision of how to do it,
>>>>>>> because
>>>>>>> the details always win.
>>>>>>>> I look forward to hearing your thoughts and thank you for enriching
>>>>>>>>
>>>>>>> this
>>>>>>> conversation.
>>>>>>>> Taher Alkhateeb
>>>>>>>>
>>>>>>>> On Tuesday, 19 July 2016, Scott Gray <scott.gray@hotwaxsystems.com
>>>>>>>>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> I know I'm late to the party here, but I just want to say that I
>>>>>>>> think
>>>>>>> integration tests have far greater value to OFBiz than unit
>>>>>>>> tests.
>>>>> Mostly
>>>>>>>>> because we tend to have quite a low number of tests and
>>>>>>>>>
>>>>>>>> integration
>>>>> tests
>>>>>>>> give us much better coverage per line of test code and the tests
>>>>>>>> are
>>>>>> much
>>>>>>>> closer to the real world scenarios the application might
>>>>>>>> encounter.
>>>>> I don't really see how unit tests could be applied to
>>>>>>>> non-framework
>>>>> testing
>>>>>>>>> in a useful manner, could you expand on your vision in that
>>>>>>>>>
>>>>>>>> regard?
>>>>> I
>>>>>>> mean
>>>>>>>>> would we be testing something smaller than a service as the
>>>>>>>>>
>>>>>>>> 'unit'?
>>>>> What
>>>>>>>> would we mock? Would transaction management still be active? What
>>>>>>>> happens
>>>>>>>> when the service calls another service, I guess we mock the
>>>>>>>> response
>>>>>> from
>>>>>>>> that service (how)?
>>>>>>>>> It just seems a very complicated method to achieve a less
>>>>>>>>>
>>>>>>>> thorough
>>>>> but
>>>>>>> albeit faster (maybe) test result.
>>>>>>>>> A build, data load and full test run takes 4m 9s on my laptop.
>>>>>>>>>
>>>>>>>> Excluding
>>>>>>>> build, data load and framework tests: the application level tests
>>>>>>>> take
>>>>>>> 35s,
>>>>>>>>> not very expensive IMO.  The data load time can be reduced to
>>>>>>>>>
>>>>>>>> practically
>>>>>>>> nothing by copying a clean slate database into runtime for each
>>>>>>>> run.
>>>>>> I'm mostly just suggesting we be wary of adding complicated
>>>>>>>> testing
>>>>> procedures in the hope of achieving some 'best practice' result
>>>>>>>> which
>>>>>> in
>>>>>>>> reality will provide minimal benefits.
>>>>>>>>> Regards
>>>>>>>>> Scott
>>>>>>>>>
>>>>>>>>> On 18 July 2016 at 18:57, Taher Alkhateeb <
>>>>>>>>>
>>>>>>>> slidingfilaments@gmail.com
>>>>>>> <javascript:;>>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>> Hello Akash,
>>>>>>>>>> Fantastic, I have a few unit tests almost done to be included
>>>>>>>>>>
>>>>>>>>> in
>>>>> the
>>>>>>> start
>>>>>>>>>> component. I will create a new subtask under OFBIZ-1463 to
>>>>>>>>>>
>>>>>>>>> commit
>>>>> the
>>>>>>> tests
>>>>>>>>>> so you can use them as a reference if you like to.
>>>>>>>>>>
>>>>>>>>>> I also recommend that you follow the same directory structure
>>>>>>>>>>
>>>>>>>>> between
>>>>>>> the
>>>>>>>>> test code and production code. So for example:
>>>>>>>>>> Production code:
>>>>>>>>>>
>>>>>>>>> framework/start/src/main/java/org/apache/ofbiz/base/start
>>>>>>>>>
>>>>>>>>>> Test code:
>>>>>>>>>>
>>>>>>>>> framework/start/src/test/java/org/apache/ofbiz/base/start
>>>>>>> The benefit of this hierarchy is that you can access non-public
>>>>>>>>> (package
>>>>>>>>> protected) methods for testing. This is in fact exactly what I
>>>>>>>>> needed
>>>>>>> to
>>>>>>>>> be
>>>>>>>>>
>>>>>>>>>> able to apply some of the tests.
>>>>>>>>>>
>>>>>>>>>> Cheers,
>>>>>>>>>>
>>>>>>>>>> Taher Alkhateeb
>>>>>>>>>>
>>>>>>>>>> On Mon, Jul 18, 2016 at 9:22 AM, Akash Jain <
>>>>>>>>>>
>>>>>>>>> akash.jain@hotwaxsystems.com <javascript:;>>
>>>>>>>>>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>> Thanks Taher for nice initiative!
>>>>>>>>>>> We are planning to written unit tests to all components under
>>>>>>>>>>>
>>>>>>>>>> OFBIZ-1463
>>>>>>>>>> Thanks and Regards
>>>>>>>>>>> --
>>>>>>>>>>> Akash Jain
>>>>>>>>>>>
>>>>>>>>>>> On Mon, Jul 18, 2016 at 10:36 AM, Taher Alkhateeb <
>>>>>>>>>>> slidingfilaments@gmail.com <javascript:;>> wrote:
>>>>>>>>>>>
>>>>>>>>>>> Hello Everyone,
>>>>>>>>>>>> In reference to this thread and the Jira OFBIZ-7254, I'm
>>>>>>>>>>>>
>>>>>>>>>>> very
>>>>> happy
>>>>>>>> to
>>>>>>>>>> announce that OFBiz is now ready for applying unit tests
>>>>>>>>>>> with
>>>>> only
>>>>>>>> 8
>>>>>>>>
>>>>>>>>> new
>>>>>>>>>>> lines of code in the build script (r1753143) :)
>>>>>>>>>>>> I recommend we do the following moving forward:
>>>>>>>>>>>>
>>>>>>>>>>>> 1- Introduce unit tests as much as we can to all components
>>>>>>>>>>>> 2- Migrate most of the integration tests we currently have
>>>>>>>>>>>>
>>>>>>>>>>> to
>>>>> unit
>>>>>>>> tests
>>>>>>>>>>> (since they are designed to do very little integration).
>>>>>>>>>>>> This is a great chance for us to practice real TDD in OFBiz
>>>>>>>>>>>>
>>>>>>>>>>>> Cheers,
>>>>>>>>>>>>
>>>>>>>>>>>> Taher Alkhateeb
>>>>>>>>>>>>
>>>>>>>>>>>> On Mon, Jun 13, 2016 at 7:43 AM, Pranay Pandey <
>>>>>>>>>>>> pranay.pandey@hotwaxsystems.com <javascript:;>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>> +1
>>>>>>>>>>>>> Best regards,
>>>>>>>>>>>>>
>>>>>>>>>>>>> Pranay Pandey
>>>>>>>>>>>>> HotWax Systems
>>>>>>>>>>>>> http://www.hotwaxsystems.com/
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Fri, Jun 10, 2016 at 7:46 PM, Taher Alkhateeb <
>>>>>>>>>>>>> slidingfilaments@gmail.com <javascript:;>
>>>>>>>>>>>>>
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>> Hello Everyone,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I was able to get a few tests running and this is very
>>>>>>>>>>>>>>
>>>>>>>>>>>>> doable.
>>>>>>>> But
>>>>>>>>>> I
>>>>>>>>>>
>>>>>>>>>>> faced
>>>>>>>>>>>>>> a big problem in designing the testing framework
>>>>>>>>>>>>>>
>>>>>>>>>>>>> because
>>>>> of
>>>>>
>>>>>> ANT.
>>>>>>>>> The problem
>>>>>>>>>>>>>> ----------------
>>>>>>>>>>>>>> The way the build scripts are designed in OFBiz are
>>>>>>>>>>>>>>
>>>>>>>>>>>>> very
>>>>> complex. A
>>>>>>>>>> master
>>>>>>>>>>>>>> file calls other files which call other files. And in
>>>>>>>>>>>>>>
>>>>>>>>>>>>> the
>>>>> middle
>>>>>>>>> you
>>>>>>>>>>> have
>>>>>>>>>>>>> external libraries (ant-contrib) and macros, and
>>>>>>>>>>>>> variables,
>>>>>> and
>>>>>>>> class
>>>>>>>>>>> path
>>>>>>>>>>>>>> declarations, and and and ....
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I cannot declare the tests programmatically (with JUnit
>>>>>>>>>>>>>>
>>>>>>>>>>>>> test
>>>>>>> suites)
>>>>>>>>>>> because this means lower level components would depend
>>>>>>>>>>>>> on
>>>>> higher
>>>>>>>>> level
>>>>>>>>>>>> components. So I have to do it in ANT, by navigating
>>>>>>>>>>>>> this
>>>>> maze
>>>>>>>> of
>>>>>>>>
>>>>>>>>> build
>>>>>>>>>>>> scripts, and it was a headache for me just to read
>>>>>>>>>>>>> them,
>>>>> let
>>>>>>> alone
>>>>>>>>>> modify
>>>>>>>>>>>>> them to create a testing framework.
>>>>>>>>>>>>>> Suggested Solution
>>>>>>>>>>>>>> ------------------------
>>>>>>>>>>>>>> I suggest to implement the testing framework in Gradle,
>>>>>>>>>>>>>>
>>>>>>>>>>>>> and
>>>>>> simply
>>>>>>>>>> call
>>>>>>>>>>>> it
>>>>>>>>>>>>>> from within ant. This is a middle solution that
>>>>>>>>>>>>>>
>>>>>>>>>>>>> sustains
>>>>> ant
>>>>>>> for
>>>>>>>>> now,
>>>>>>>>>>> but
>>>>>>>>>>>>> can allow us to switch out later.
>>>>>>>>>>>>>> This means I will just add one more file called
>>>>>>>>>>>>>>
>>>>>>>>>>>>> build.gradle
>>>>>>> in
>>>>>>>
>>>>>>>> the
>>>>>>>>>> top
>>>>>>>>>>>> level directory, and figure out the business logic for
>>>>>>>>>>>>> calling
>>>>>>>> the
>>>>>>>>>> test
>>>>>>>>>>>> suites from that file
>>>>>>>>>>>>>> I look forward to your feedback.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Taher Alkhateeb
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Wed, Jun 8, 2016 at 6:00 PM, Taher Alkhateeb <
>>>>>>>>>>>>>> slidingfilaments@gmail.com <javascript:;>>
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hi Everyone,
>>>>>>>>>>>>>>> Thank you all for your support, JIRA created in
>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OFBIZ-7254
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I will start working on it and try to implement ASAP
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> to
>>>>> get
>>>>>>> my
>>>>>>>>> focus
>>>>>>>>>>>> back
>>>>>>>>>>>>>> on refactoring.
>>>>>>>>>>>>>>> Cheers!
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Taher Alkhateeb
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Wed, Jun 8, 2016 at 4:58 PM, Deepak Dixit <
>>>>>>>>>>>>>>> deepak.dixit@hotwaxsystems.com <javascript:;>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> wrote:
>>>>> +1
>>>>>>>>>>>>>>>> Thanks & Regards
>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>> Deepak Dixit
>>>>>>>>>>>>>>>> www.hotwaxsystems.com
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Wed, Jun 8, 2016 at 7:12 PM, Mridul Pathak <
>>>>>>>>>>>>>>>> mridul.pathak@hotwaxsystems.com <javascript:;>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> wrote:
>>>>>> +1
>>>>>>>>>>>>>>>>> Makes perfect sense.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>> Thanks & Regards,
>>>>>>>>>>>>>>>>> Mridul Pathak
>>>>>>>>>>>>>>>>> Senior Manager
>>>>>>>>>>>>>>>>> HotWax Systems
>>>>>>>>>>>>>>>>> http://www.hotwaxsystems.com
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On Jun 8, 2016, at 2:41 PM, Taher Alkhateeb <
>>>>>>>>>>>>>>>>> slidingfilaments@gmail.com <javascript:;>>
>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Hello Everyone,
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> After refactoring the start component and while
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> starting
>>>>>>>> on
>>>>>>>>>> the
>>>>>>>>>>>> base
>>>>>>>>>>>>>> component I realized that the testing framework
>>>>>>>>>>>>>>>>> for
>>>>>> OFBiz
>>>>>>>>> is
>>>>>>>>>
>>>>>>>>>> not
>>>>>>>>>>>> good.
>>>>>>>>>>>>>>> You
>>>>>>>>>>>>>>>>>> cannot do real test driven development or
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> red-green-refactor
>>>>>>>>>> with
>>>>>>>>>>>>> the
>>>>>>>>>>>>>>> current setup, hence my proposal to change it. I
>>>>>>>>>>>>>>>>> explain
>>>>>>>> below:
>>>>>>>>>>>> Problem with current design
>>>>>>>>>>>>>>>>>> ----------------------------------------
>>>>>>>>>>>>>>>>>> - What we have right now is not unit tests, it's
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> really
>>>>>>>> integration
>>>>>>>>>>>>>> tests.
>>>>>>>>>>>>>>>>>> You have to start the framework, the database,
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> the
>>>>> service
>>>>>>>>> engine,
>>>>>>>>>>>>> the
>>>>>>>>>>>>>>> entity engine and pretty much everything.
>>>>>>>>>>>>>>>>>> - Testing is very slow, because it's an
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> integration
>>>>>> test
>>>>>>>> as
>>>>>>>>>> I
>>>>>>>>>>
>>>>>>>>>>> mentioned
>>>>>>>>>>>>>>>>> above. 10 minutes on a good computer!
>>>>>>>>>>>>>>>>>> - There is zero mocking! We actually have to
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> --load-data
>>>>>>>> for
>>>>>>>>>> things
>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> work. Again, these are integration tests.
>>>>>>>>>>>>>>>>>> - Too complex: Integration tests by their nature
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> are
>>>>>> grabbing
>>>>>>>>>>> too
>>>>>>>>>>>>> much.
>>>>>>>>>>>>>>>>> Mind you, I am not objecting to integration
>>>>>>>>>>>>>>>>> tests
>>>>> (I
>>>>>
>>>>>> actually
>>>>>>>>>>> like
>>>>>>>>>>>>> them)
>>>>>>>>>>>>>>>>> but I am objecting to not having real
>>>>>>>>>>>>>>>>> unit-tests.
>>>>> Unit
>>>>>>> tests
>>>>>>>>>> should
>>>>>>>>>>>>>> all
>>>>>>>>>>>>>>>>> run
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> in a few seconds.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Proposed solution
>>>>>>>>>>>>>>>>>> --------------------------
>>>>>>>>>>>>>>>>>> - We keep what is considered real integration
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> tests
>>>>>> the
>>>>>>>> way
>>>>>>>>>> they
>>>>>>>>>>>> are
>>>>>>>>>>>>>> right
>>>>>>>>>>>>>>>>>> now and keep using them
>>>>>>>>>>>>>>>>>> - We move what should be unit tests into simple
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> JUnit
>>>>>>> classes,
>>>>>>>>>>> and
>>>>>>>>>>>>> we
>>>>>>>>>>>>>>> do
>>>>>>>>>>>>>>>>> not run them using java -jar ofbiz.jar --test,
>>>>>>>>>>>>>>>>> but
>>>>> instead
>>>>>>>>> run
>>>>>>>>>>> them
>>>>>>>>>>>>>> directly from the build.xml script, so these
>>>>>>>>>>>>>>>>> files
>>>>> are
>>>>>>> not
>>>>>>>>> identified
>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>> any XML document, but are simply called
>>>>>>>>>>>>>>>>> immediately
>>>>>> from
>>>>>>>> the
>>>>>>>>>> build
>>>>>>>>>>>>> scripts.
>>>>>>>>>>>>>>>>>> - We clearly mark the difference between
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> integration
>>>>>> tests
>>>>>>>>> and
>>>>>>>>>>> unit
>>>>>>>>>>>>>> tests
>>>>>>>>>>>>>>>>> (inside the source files or in the suite
>>>>>>>>>>>>>>>>> declarations).
>>>>>>>> - We change the run-tests target in build.xml to
>>>>>>>>>>>>>>>>> run
>>>>>> both
>>>>>>>>> unit
>>>>>>>>>>> tests
>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>> integration tests.
>>>>>>>>>>>>>>>>>> I intend to heavily refactor the framework and I
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> would
>>>>>>> feel
>>>>>>>>>> better
>>>>>>>>>>>>> about
>>>>>>>>>>>>>>>>> introducing this change while refactoring. What
>>>>>>>>>>>>>>>>> do
>>>>> you
>>>>>>> guys
>>>>>>>>>> think?
>>>>>>>>>>>>> Ideas?
>>>>>>>>>>>>>>>>> Suggestions? Approvals and thumbs up?
>>>>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Taher Alkhateeb
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>


-- 
Ron Wheeler
President
Artifact Software Inc
email: rwheeler@artifact-software.com
skype: ronaldmwheeler
phone: 866-970-2435, ext 102

Re: Proposal to modify the testing framework for OFBiz

[Scott Gray <sc...@hotwaxsystems.com>]

I think people are missing my point because they keep replying with generic
statements that aren't specific to unit tests.

On 19 July 2016 at 20:08, Montalbano Florian <fl...@nereide.fr>
wrote:

> Hi everyone,
>
> unit tests and integration tests are complementary and with test, the more
> the better.
> Implementing unit tests will be a great load of work but it is the same
> with every kind of test. As Taher said, we will gain a lot from those unit
> tests for avoiding regression while refactoring the framework. From the
> start component scope, it has already be proved useful.
> And what Ron said about encouraging the creation of "smaller class that
> can be tested easily" is very important too. It can be a first step to
> reach TDD in the OFBiz development. TDD can help to give confidence to
> newcomer and veteran when updating some part of the code and as Hans said,
> this lead to "continuous improvement and finally continuous deployment".
>
> I'm aware that this does not solve the technical implementation stuff, but
> I think this is an improvement that need to be supported.
>
> Have a nice day,
>
> Florian Montalbano
>
>
> Le 19/07/2016 09:04, Pierre Smits a écrit :
>
>> As with anything, the law of diminishing returns also applies to OFBiz and
>> tests. This is not true for unit tests and system integration tests, but
>> also for user acceptance and performance tests.
>>
>> Nevertheless, the work done up to now is a good start and - I feel
>> confident - appreciated. And unit tests are certainly valuable in the
>> framework stack. How it will be for functions (regarding components in
>> application and special purpose stack) needs to be addressed when we reach
>> that bridge.
>>
>>
>> Best regards,
>>
>> Pierre Smits
>>
>> ORRTIZ.COM <http://www.orrtiz.com>
>> OFBiz based solutions & services
>>
>> OFBiz Extensions Marketplace
>> http://oem.ofbizci.net/oci-2/
>>
>> On Tue, Jul 19, 2016 at 8:22 AM, Taher Alkhateeb <
>> slidingfilaments@gmail.com
>>
>>> wrote:
>>> Hi Scott,
>>>
>>> Well spoken thank you. Okay may I suggest that for any such work we will
>>> discuss it here see its Merit and if it makes sense then we take it in.
>>> It
>>> is a little early to discuss it right now because we did not yet go to
>>> the
>>> higher-level components. Once we do I'll be sure to have a conversation
>>> about this in here and would appreciate your input to it.
>>>
>>> Regards,
>>>
>>> Taher Alkhateeb
>>>
>>> On Jul 19, 2016 9:00 AM, "Scott Gray" <sc...@hotwaxsystems.com>
>>> wrote:
>>>
>>> Yeah I'm sure unit tests probably worked well for the start component
>>>>
>>> given
>>>
>>>> it is the lowest level component in the system and closest to a basic
>>>>
>>> java
>>>
>>>> app.  I just think the value proposition might decrease the further up
>>>>
>>> the
>>>
>>>> stack we move with them.  I'm not against unit tests when they prove
>>>> useful, but further up the stack I think we should prove the case for
>>>>
>>> them
>>>
>>>> before doing much work to support the mocking that will be required to
>>>>
>>> keep
>>>
>>>> them inline with 'best practices'.
>>>>
>>>> In OFBiz "bad or unwanted" tends to come mostly in the way of increasing
>>>> complexity from features that require more effort to maintain than the
>>>> value they provide to the community.  I think there's a chance
>>>> attempting
>>>> to mock service tests could fall into that bucket.  I could be wrong,
>>>>
>>> but I
>>>
>>>> think it's worth looking into before we declare that unit tests are the
>>>> best form of testing for OFBiz.
>>>>
>>>> Regards
>>>> Scott
>>>>
>>>> On 19 July 2016 at 17:37, Taher Alkhateeb <sl...@gmail.com>
>>>> wrote:
>>>>
>>>> Hi Scott,
>>>>>
>>>>> Thank you for the feedback. To be focused exactly on integration vs
>>>>>
>>>> unit, I
>>>>
>>>>> already mentioned above that at least for me the main objective is to
>>>>> confidently and quickly run the tests in short bursts of red-green
>>>>> refactor. This allows me to refactor code without waiting in front of
>>>>>
>>>> the
>>>
>>>> screen in between test cycles thus giving me immediate feedback on any
>>>>> errors I made. Perhaps my intro was too long so this is the squeezed
>>>>>
>>>> out
>>>
>>>> version of it.
>>>>>
>>>>> I already had one round of testing in the start component which was
>>>>>
>>>> much
>>>
>>>> faster that way and had an immediate impact. Oh and by the way, you
>>>>>
>>>> cannot
>>>>
>>>>> test the start component with integration tests for example unless you
>>>>>
>>>> do
>>>
>>>> it from an external component which cannot access package protected
>>>>>
>>>> items.
>>>>
>>>>> This style of coding is applicable I think to any software project
>>>>> inclusive of OFBiz. Maybe in certain components more than others or
>>>>>
>>>> certain
>>>>
>>>>> areas more than others. But I can't see how it could be bad or
>>>>>
>>>> unwanted.
>>>
>>>> Taher Alkhateeb
>>>>>
>>>>> On Jul 19, 2016 8:18 AM, "Scott Gray" <sc...@hotwaxsystems.com>
>>>>> wrote:
>>>>>
>>>>> Thanks Ron and Taher for your responses, I appreciate them but I
>>>>>>
>>>>> don't
>>>
>>>> hear
>>>>>
>>>>>> much in the conversation that speaks to the value of unit tests over
>>>>>> integration tests.  Most of the thoughts shared speaks more to the
>>>>>>
>>>>> value
>>>>
>>>>> of
>>>>>
>>>>>> tests rather than differences in the type of tests.
>>>>>>
>>>>>> Speed: At an application level we have ~685 tests that run in 35
>>>>>>
>>>>> seconds
>>>>
>>>>> (excluding build and data load).  Another point is that there isn't
>>>>>>
>>>>> much
>>>>
>>>>> reason why tests can't be run in parallel rather than sequentially.
>>>>>> TDD: Integration tests in OFBiz are as simple if not simpler to write
>>>>>>
>>>>> than
>>>>>
>>>>>> unit tests and just as useful for TDD.
>>>>>> I'm not sure if I missed any other points raised regarding
>>>>>>
>>>>> integration
>>>
>>>> vs.
>>>>>
>>>>>> unit tests?
>>>>>>
>>>>>> I'm not looking to start a big long debate on the topic, I just
>>>>>>
>>>>> wanted
>>>
>>>> to
>>>>
>>>>> speak out that someone out there (me) doesn't think unit tests are
>>>>>>
>>>>> the
>>>
>>>> best
>>>>>
>>>>>> solution for testing OFBiz applications.
>>>>>>
>>>>>> Regards
>>>>>> Scott
>>>>>>
>>>>>> On 19 July 2016 at 16:52, Taher Alkhateeb <
>>>>>>
>>>>> slidingfilaments@gmail.com>
>>>
>>>> wrote:
>>>>>>
>>>>>> Hi Scott,
>>>>>>>
>>>>>>> Thank you for your input. Your ideas are thought provoking and
>>>>>>>
>>>>>> enriching
>>>>>
>>>>>> to
>>>>>>
>>>>>>> the conversation.
>>>>>>>
>>>>>>> The way I look at it, the general rule is usually many unit tests,
>>>>>>>
>>>>>> less
>>>>
>>>>> integration tests, lesser functional tests. So we are not excluding
>>>>>>>
>>>>>> any
>>>>
>>>>> types of test, all of them are important in certain areas for
>>>>>>>
>>>>>> certain
>>>
>>>> purposes with certain quantity. Usually integration tests are less
>>>>>>>
>>>>>> because
>>>>>>
>>>>>>> as you said they just grab more. I like the picture below as a
>>>>>>>
>>>>>> general
>>>>
>>>>> guideline
>>>>>>>
>>>>>>> http://i.stack.imgur.com/fjQvQ.png
>>>>>>>
>>>>>>> As you already mentioned unit tests are useful for the framework. I
>>>>>>> discovered errors in code that I wrote which I was very very
>>>>>>>
>>>>>> careful
>>>
>>>> with.
>>>>>>
>>>>>>> I immediately learned the lesson that humans are not designed to
>>>>>>>
>>>>>> code,
>>>>
>>>>> and
>>>>>>
>>>>>>> TDD gives you confidence as you build your code with those short
>>>>>>>
>>>>>> 30-60
>>>>
>>>>> second red-green refactors. I feel much much safer and more
>>>>>>>
>>>>>> confident
>>>
>>>> writing code that way, the test also documents how to use the api,
>>>>>>> refactoring and feature change becomes less terrifying. Also messy
>>>>>>>
>>>>>> code
>>>>
>>>>> is
>>>>>>
>>>>>>> usually not test friendly thus refactoring and unit tests go
>>>>>>>
>>>>>> hand-in-hand
>>>>>
>>>>>> for improving the code base.
>>>>>>>
>>>>>>> Also, I am sorry to say that the framework code is rather messy and
>>>>>>>
>>>>>> brittle
>>>>>>
>>>>>>> in many places. I think you probably encountered this yourself. The
>>>>>>>
>>>>>> same
>>>>>
>>>>>> is
>>>>>>
>>>>>>> said for the applications. Now If we start refactoring without unit
>>>>>>>
>>>>>> tests
>>>>>
>>>>>> then we are back to scary business. So much can go wrong so fast
>>>>>>>
>>>>>> and
>>>
>>>> break
>>>>>>
>>>>>>> things you never expected to break. The framework with all
>>>>>>>
>>>>>> applications
>>>>
>>>>> require heavy refactoring of things like massive ugly methods,
>>>>>>>
>>>>>> sandwiched
>>>>>
>>>>>> logic, heavy shared mutable state, hidden dependencies, poor
>>>>>>> interfaces, and much more. Every time I touch something I get
>>>>>>>
>>>>>> shocked
>>>
>>>> at
>>>>>
>>>>>> how bad it looks, spaghetti logic to no end. If you refactor with
>>>>>>> integration tests instead of unit tests then you will come down to
>>>>>>>
>>>>>> a
>>>
>>>> screeching halt as you wait between these test cycles. My computer
>>>>>>>
>>>>>> actually
>>>>>>
>>>>>>> takes 10 minutes or more for a full clean load data and testing.
>>>>>>>
>>>>>>> Now talking about mocking you raise some interesting points, what
>>>>>>>
>>>>>> can
>>>
>>>> you
>>>>>
>>>>>> mock vs utilizing integration tests? This is an excellent question
>>>>>>>
>>>>>> that
>>>>
>>>>> does not have a simple answer, and you mostly learn as you code,
>>>>>>>
>>>>>> difficult
>>>>>>
>>>>>>> to envision without coding. However simple things like Java
>>>>>>>
>>>>>> services
>>>
>>>> can
>>>>>
>>>>>> be
>>>>>>
>>>>>>> mocked with a standard mocking class that you use everywhere.
>>>>>>>
>>>>>> Transactions
>>>>>>
>>>>>>> are difficult to mock and better left to integration tests I think.
>>>>>>>
>>>>>> SECAs
>>>>>
>>>>>> might be mocked by simply passing output to input as chaining
>>>>>>>
>>>>>> methods.
>>>>
>>>>> Mind
>>>>>>
>>>>>>> you I am not 100% sure of all of this, coding is the ultimate guide
>>>>>>>
>>>>>> to
>>>>
>>>>> what
>>>>>>
>>>>>>> can or cannot be done.
>>>>>>>
>>>>>>> So I am not suggesting unit tests as a best-practice (even though
>>>>>>>
>>>>>> it
>>>
>>>> is).
>>>>>
>>>>>> Instead I suggest it as something that I and others did and got a
>>>>>>>
>>>>>> huge
>>>>
>>>>> psychological relief and confidence and comfort from. Swimming in
>>>>>>>
>>>>>> code
>>>>
>>>>> without tests is a terrifying business, made more terrifying if the
>>>>>>>
>>>>>> code
>>>>>
>>>>>> is
>>>>>>
>>>>>>> bad. Short bursts of red/green/red/green makes you feel good as you
>>>>>>>
>>>>>> build
>>>>>
>>>>>> up your logic. And I don't have an exact vision of how to do it,
>>>>>>>
>>>>>> because
>>>>>
>>>>>> the details always win.
>>>>>>>
>>>>>>> I look forward to hearing your thoughts and thank you for enriching
>>>>>>>
>>>>>> this
>>>>>
>>>>>> conversation.
>>>>>>>
>>>>>>> Taher Alkhateeb
>>>>>>>
>>>>>>> On Tuesday, 19 July 2016, Scott Gray <scott.gray@hotwaxsystems.com
>>>>>>>
>>>>>> wrote:
>>>>>>
>>>>>>> I know I'm late to the party here, but I just want to say that I
>>>>>>>>
>>>>>>> think
>>>>>
>>>>>> integration tests have far greater value to OFBiz than unit
>>>>>>>>
>>>>>>> tests.
>>>
>>>> Mostly
>>>>>>>
>>>>>>>> because we tend to have quite a low number of tests and
>>>>>>>>
>>>>>>> integration
>>>
>>>> tests
>>>>>>
>>>>>>> give us much better coverage per line of test code and the tests
>>>>>>>>
>>>>>>> are
>>>>
>>>>> much
>>>>>>
>>>>>>> closer to the real world scenarios the application might
>>>>>>>>
>>>>>>> encounter.
>>>
>>>> I don't really see how unit tests could be applied to
>>>>>>>>
>>>>>>> non-framework
>>>
>>>> testing
>>>>>>>
>>>>>>>> in a useful manner, could you expand on your vision in that
>>>>>>>>
>>>>>>> regard?
>>>
>>>> I
>>>>>
>>>>>> mean
>>>>>>>
>>>>>>>> would we be testing something smaller than a service as the
>>>>>>>>
>>>>>>> 'unit'?
>>>
>>>> What
>>>>>>
>>>>>>> would we mock? Would transaction management still be active? What
>>>>>>>>
>>>>>>> happens
>>>>>>
>>>>>>> when the service calls another service, I guess we mock the
>>>>>>>>
>>>>>>> response
>>>>
>>>>> from
>>>>>>
>>>>>>> that service (how)?
>>>>>>>>
>>>>>>>> It just seems a very complicated method to achieve a less
>>>>>>>>
>>>>>>> thorough
>>>
>>>> but
>>>>>
>>>>>> albeit faster (maybe) test result.
>>>>>>>>
>>>>>>>> A build, data load and full test run takes 4m 9s on my laptop.
>>>>>>>>
>>>>>>> Excluding
>>>>>>
>>>>>>> build, data load and framework tests: the application level tests
>>>>>>>>
>>>>>>> take
>>>>>
>>>>>> 35s,
>>>>>>>
>>>>>>>> not very expensive IMO.  The data load time can be reduced to
>>>>>>>>
>>>>>>> practically
>>>>>>
>>>>>>> nothing by copying a clean slate database into runtime for each
>>>>>>>>
>>>>>>> run.
>>>>
>>>>> I'm mostly just suggesting we be wary of adding complicated
>>>>>>>>
>>>>>>> testing
>>>
>>>> procedures in the hope of achieving some 'best practice' result
>>>>>>>>
>>>>>>> which
>>>>
>>>>> in
>>>>>>
>>>>>>> reality will provide minimal benefits.
>>>>>>>>
>>>>>>>> Regards
>>>>>>>> Scott
>>>>>>>>
>>>>>>>> On 18 July 2016 at 18:57, Taher Alkhateeb <
>>>>>>>>
>>>>>>> slidingfilaments@gmail.com
>>>>>
>>>>>> <javascript:;>>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>> Hello Akash,
>>>>>>>>>
>>>>>>>>> Fantastic, I have a few unit tests almost done to be included
>>>>>>>>>
>>>>>>>> in
>>>
>>>> the
>>>>>
>>>>>> start
>>>>>>>>
>>>>>>>>> component. I will create a new subtask under OFBIZ-1463 to
>>>>>>>>>
>>>>>>>> commit
>>>
>>>> the
>>>>>
>>>>>> tests
>>>>>>>>
>>>>>>>>> so you can use them as a reference if you like to.
>>>>>>>>>
>>>>>>>>> I also recommend that you follow the same directory structure
>>>>>>>>>
>>>>>>>> between
>>>>>
>>>>>> the
>>>>>>>
>>>>>>>> test code and production code. So for example:
>>>>>>>>>
>>>>>>>>> Production code:
>>>>>>>>>
>>>>>>>> framework/start/src/main/java/org/apache/ofbiz/base/start
>>>>>>>>
>>>>>>>>> Test code:
>>>>>>>>>
>>>>>>>> framework/start/src/test/java/org/apache/ofbiz/base/start
>>>>>
>>>>>> The benefit of this hierarchy is that you can access non-public
>>>>>>>>>
>>>>>>>> (package
>>>>>>>
>>>>>>>> protected) methods for testing. This is in fact exactly what I
>>>>>>>>>
>>>>>>>> needed
>>>>>
>>>>>> to
>>>>>>>
>>>>>>>> be
>>>>>>>>
>>>>>>>>> able to apply some of the tests.
>>>>>>>>>
>>>>>>>>> Cheers,
>>>>>>>>>
>>>>>>>>> Taher Alkhateeb
>>>>>>>>>
>>>>>>>>> On Mon, Jul 18, 2016 at 9:22 AM, Akash Jain <
>>>>>>>>>
>>>>>>>> akash.jain@hotwaxsystems.com <javascript:;>>
>>>>>>>>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>> Thanks Taher for nice initiative!
>>>>>>>>>>
>>>>>>>>>> We are planning to written unit tests to all components under
>>>>>>>>>>
>>>>>>>>> OFBIZ-1463
>>>>>>>>
>>>>>>>>> Thanks and Regards
>>>>>>>>>> --
>>>>>>>>>> Akash Jain
>>>>>>>>>>
>>>>>>>>>> On Mon, Jul 18, 2016 at 10:36 AM, Taher Alkhateeb <
>>>>>>>>>> slidingfilaments@gmail.com <javascript:;>> wrote:
>>>>>>>>>>
>>>>>>>>>> Hello Everyone,
>>>>>>>>>>>
>>>>>>>>>>> In reference to this thread and the Jira OFBIZ-7254, I'm
>>>>>>>>>>>
>>>>>>>>>> very
>>>
>>>> happy
>>>>>>
>>>>>>> to
>>>>>>>>
>>>>>>>>> announce that OFBiz is now ready for applying unit tests
>>>>>>>>>>>
>>>>>>>>>> with
>>>
>>>> only
>>>>>>
>>>>>>> 8
>>>>>>>
>>>>>>>> new
>>>>>>>>>
>>>>>>>>>> lines of code in the build script (r1753143) :)
>>>>>>>>>>>
>>>>>>>>>>> I recommend we do the following moving forward:
>>>>>>>>>>>
>>>>>>>>>>> 1- Introduce unit tests as much as we can to all components
>>>>>>>>>>> 2- Migrate most of the integration tests we currently have
>>>>>>>>>>>
>>>>>>>>>> to
>>>
>>>> unit
>>>>>>
>>>>>>> tests
>>>>>>>>>
>>>>>>>>>> (since they are designed to do very little integration).
>>>>>>>>>>>
>>>>>>>>>>> This is a great chance for us to practice real TDD in OFBiz
>>>>>>>>>>>
>>>>>>>>>>> Cheers,
>>>>>>>>>>>
>>>>>>>>>>> Taher Alkhateeb
>>>>>>>>>>>
>>>>>>>>>>> On Mon, Jun 13, 2016 at 7:43 AM, Pranay Pandey <
>>>>>>>>>>> pranay.pandey@hotwaxsystems.com <javascript:;>> wrote:
>>>>>>>>>>>
>>>>>>>>>>> +1
>>>>>>>>>>>>
>>>>>>>>>>>> Best regards,
>>>>>>>>>>>>
>>>>>>>>>>>> Pranay Pandey
>>>>>>>>>>>> HotWax Systems
>>>>>>>>>>>> http://www.hotwaxsystems.com/
>>>>>>>>>>>>
>>>>>>>>>>>> On Fri, Jun 10, 2016 at 7:46 PM, Taher Alkhateeb <
>>>>>>>>>>>> slidingfilaments@gmail.com <javascript:;>
>>>>>>>>>>>>
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>> Hello Everyone,
>>>>>>>>>>>>>
>>>>>>>>>>>>> I was able to get a few tests running and this is very
>>>>>>>>>>>>>
>>>>>>>>>>>> doable.
>>>>>>
>>>>>>> But
>>>>>>>>
>>>>>>>>> I
>>>>>>>>>
>>>>>>>>>> faced
>>>>>>>>>>>>
>>>>>>>>>>>>> a big problem in designing the testing framework
>>>>>>>>>>>>>
>>>>>>>>>>>> because
>>>
>>>> of
>>>>
>>>>> ANT.
>>>>>>>
>>>>>>>> The problem
>>>>>>>>>>>>> ----------------
>>>>>>>>>>>>> The way the build scripts are designed in OFBiz are
>>>>>>>>>>>>>
>>>>>>>>>>>> very
>>>
>>>> complex. A
>>>>>>>>
>>>>>>>>> master
>>>>>>>>>>>>
>>>>>>>>>>>>> file calls other files which call other files. And in
>>>>>>>>>>>>>
>>>>>>>>>>>> the
>>>
>>>> middle
>>>>>>>
>>>>>>>> you
>>>>>>>>>
>>>>>>>>>> have
>>>>>>>>>>>
>>>>>>>>>>>> external libraries (ant-contrib) and macros, and
>>>>>>>>>>>>>
>>>>>>>>>>>> variables,
>>>>
>>>>> and
>>>>>>
>>>>>>> class
>>>>>>>>>
>>>>>>>>>> path
>>>>>>>>>>>>
>>>>>>>>>>>>> declarations, and and and ....
>>>>>>>>>>>>>
>>>>>>>>>>>>> I cannot declare the tests programmatically (with JUnit
>>>>>>>>>>>>>
>>>>>>>>>>>> test
>>>>>
>>>>>> suites)
>>>>>>>>>
>>>>>>>>>> because this means lower level components would depend
>>>>>>>>>>>>>
>>>>>>>>>>>> on
>>>
>>>> higher
>>>>>>>
>>>>>>>> level
>>>>>>>>>>
>>>>>>>>>>> components. So I have to do it in ANT, by navigating
>>>>>>>>>>>>>
>>>>>>>>>>>> this
>>>
>>>> maze
>>>>>>
>>>>>>> of
>>>>>>>
>>>>>>>> build
>>>>>>>>>>
>>>>>>>>>>> scripts, and it was a headache for me just to read
>>>>>>>>>>>>>
>>>>>>>>>>>> them,
>>>
>>>> let
>>>>>
>>>>>> alone
>>>>>>>>
>>>>>>>>> modify
>>>>>>>>>>>
>>>>>>>>>>>> them to create a testing framework.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Suggested Solution
>>>>>>>>>>>>> ------------------------
>>>>>>>>>>>>> I suggest to implement the testing framework in Gradle,
>>>>>>>>>>>>>
>>>>>>>>>>>> and
>>>>
>>>>> simply
>>>>>>>>
>>>>>>>>> call
>>>>>>>>>>
>>>>>>>>>>> it
>>>>>>>>>>>>
>>>>>>>>>>>>> from within ant. This is a middle solution that
>>>>>>>>>>>>>
>>>>>>>>>>>> sustains
>>>
>>>> ant
>>>>>
>>>>>> for
>>>>>>>
>>>>>>>> now,
>>>>>>>>>
>>>>>>>>>> but
>>>>>>>>>>>
>>>>>>>>>>>> can allow us to switch out later.
>>>>>>>>>>>>>
>>>>>>>>>>>>> This means I will just add one more file called
>>>>>>>>>>>>>
>>>>>>>>>>>> build.gradle
>>>>>
>>>>>> in
>>>>>>
>>>>>>> the
>>>>>>>>
>>>>>>>>> top
>>>>>>>>>>
>>>>>>>>>>> level directory, and figure out the business logic for
>>>>>>>>>>>>>
>>>>>>>>>>>> calling
>>>>>>
>>>>>>> the
>>>>>>>>
>>>>>>>>> test
>>>>>>>>>>
>>>>>>>>>>> suites from that file
>>>>>>>>>>>>>
>>>>>>>>>>>>> I look forward to your feedback.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>
>>>>>>>>>>>>> Taher Alkhateeb
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Wed, Jun 8, 2016 at 6:00 PM, Taher Alkhateeb <
>>>>>>>>>>>>> slidingfilaments@gmail.com <javascript:;>>
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>> Hi Everyone,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thank you all for your support, JIRA created in
>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OFBIZ-7254
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I will start working on it and try to implement ASAP
>>>>>>>>>>>>>>
>>>>>>>>>>>>> to
>>>
>>>> get
>>>>>
>>>>>> my
>>>>>>>
>>>>>>>> focus
>>>>>>>>>>
>>>>>>>>>>> back
>>>>>>>>>>>>
>>>>>>>>>>>>> on refactoring.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Cheers!
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Taher Alkhateeb
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Wed, Jun 8, 2016 at 4:58 PM, Deepak Dixit <
>>>>>>>>>>>>>> deepak.dixit@hotwaxsystems.com <javascript:;>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>> wrote:
>>>
>>>> +1
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Thanks & Regards
>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>> Deepak Dixit
>>>>>>>>>>>>>>> www.hotwaxsystems.com
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Wed, Jun 8, 2016 at 7:12 PM, Mridul Pathak <
>>>>>>>>>>>>>>> mridul.pathak@hotwaxsystems.com <javascript:;>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> wrote:
>>>>
>>>>> +1
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Makes perfect sense.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>> Thanks & Regards,
>>>>>>>>>>>>>>>> Mridul Pathak
>>>>>>>>>>>>>>>> Senior Manager
>>>>>>>>>>>>>>>> HotWax Systems
>>>>>>>>>>>>>>>> http://www.hotwaxsystems.com
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Jun 8, 2016, at 2:41 PM, Taher Alkhateeb <
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> slidingfilaments@gmail.com <javascript:;>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Hello Everyone,
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> After refactoring the start component and while
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> starting
>>>>>>
>>>>>>> on
>>>>>>>>
>>>>>>>>> the
>>>>>>>>>>
>>>>>>>>>>> base
>>>>>>>>>>>>
>>>>>>>>>>>>> component I realized that the testing framework
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> for
>>>>
>>>>> OFBiz
>>>>>>>
>>>>>>>> is
>>>>>>>>
>>>>>>>>> not
>>>>>>>>>>
>>>>>>>>>>> good.
>>>>>>>>>>>>>
>>>>>>>>>>>>>> You
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> cannot do real test driven development or
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> red-green-refactor
>>>>>>>>
>>>>>>>>> with
>>>>>>>>>>>
>>>>>>>>>>>> the
>>>>>>>>>>>>>
>>>>>>>>>>>>>> current setup, hence my proposal to change it. I
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> explain
>>>>>>
>>>>>>> below:
>>>>>>>>>>
>>>>>>>>>>> Problem with current design
>>>>>>>>>>>>>>>>> ----------------------------------------
>>>>>>>>>>>>>>>>> - What we have right now is not unit tests, it's
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> really
>>>>>>
>>>>>>> integration
>>>>>>>>>>>>
>>>>>>>>>>>>> tests.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> You have to start the framework, the database,
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> the
>>>
>>>> service
>>>>>>>
>>>>>>>> engine,
>>>>>>>>>>>
>>>>>>>>>>>> the
>>>>>>>>>>>>>
>>>>>>>>>>>>>> entity engine and pretty much everything.
>>>>>>>>>>>>>>>>> - Testing is very slow, because it's an
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> integration
>>>>
>>>>> test
>>>>>>
>>>>>>> as
>>>>>>>>
>>>>>>>>> I
>>>>>>>>>
>>>>>>>>>> mentioned
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> above. 10 minutes on a good computer!
>>>>>>>>>>>>>>>>> - There is zero mocking! We actually have to
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> --load-data
>>>>>>
>>>>>>> for
>>>>>>>>
>>>>>>>>> things
>>>>>>>>>>>>
>>>>>>>>>>>>> to
>>>>>>>>>>>>>
>>>>>>>>>>>>>> work. Again, these are integration tests.
>>>>>>>>>>>>>>>>> - Too complex: Integration tests by their nature
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> are
>>>>
>>>>> grabbing
>>>>>>>>>
>>>>>>>>>> too
>>>>>>>>>>>
>>>>>>>>>>>> much.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Mind you, I am not objecting to integration
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> tests
>>>
>>>> (I
>>>>
>>>>> actually
>>>>>>>>>
>>>>>>>>>> like
>>>>>>>>>>>
>>>>>>>>>>>> them)
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> but I am objecting to not having real
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> unit-tests.
>>>
>>>> Unit
>>>>>
>>>>>> tests
>>>>>>>>
>>>>>>>>> should
>>>>>>>>>>>>
>>>>>>>>>>>>> all
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> run
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> in a few seconds.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Proposed solution
>>>>>>>>>>>>>>>>> --------------------------
>>>>>>>>>>>>>>>>> - We keep what is considered real integration
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> tests
>>>>
>>>>> the
>>>>>>
>>>>>>> way
>>>>>>>>
>>>>>>>>> they
>>>>>>>>>>
>>>>>>>>>>> are
>>>>>>>>>>>>
>>>>>>>>>>>>> right
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> now and keep using them
>>>>>>>>>>>>>>>>> - We move what should be unit tests into simple
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> JUnit
>>>>>
>>>>>> classes,
>>>>>>>>>
>>>>>>>>>> and
>>>>>>>>>>>
>>>>>>>>>>>> we
>>>>>>>>>>>>>
>>>>>>>>>>>>>> do
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> not run them using java -jar ofbiz.jar --test,
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> but
>>>
>>>> instead
>>>>>>>
>>>>>>>> run
>>>>>>>>>
>>>>>>>>>> them
>>>>>>>>>>>>
>>>>>>>>>>>>> directly from the build.xml script, so these
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> files
>>>
>>>> are
>>>>>
>>>>>> not
>>>>>>>
>>>>>>>> identified
>>>>>>>>>>>>>
>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> any XML document, but are simply called
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> immediately
>>>>
>>>>> from
>>>>>>
>>>>>>> the
>>>>>>>>
>>>>>>>>> build
>>>>>>>>>>>
>>>>>>>>>>>> scripts.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> - We clearly mark the difference between
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> integration
>>>>
>>>>> tests
>>>>>>>
>>>>>>>> and
>>>>>>>>>
>>>>>>>>>> unit
>>>>>>>>>>>>
>>>>>>>>>>>>> tests
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> (inside the source files or in the suite
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> declarations).
>>>>>>
>>>>>>> - We change the run-tests target in build.xml to
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> run
>>>>
>>>>> both
>>>>>>>
>>>>>>>> unit
>>>>>>>>>
>>>>>>>>>> tests
>>>>>>>>>>>>
>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> integration tests.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> I intend to heavily refactor the framework and I
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> would
>>>>>
>>>>>> feel
>>>>>>>>
>>>>>>>>> better
>>>>>>>>>>>
>>>>>>>>>>>> about
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> introducing this change while refactoring. What
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> do
>>>
>>>> you
>>>>>
>>>>>> guys
>>>>>>>>
>>>>>>>>> think?
>>>>>>>>>>>
>>>>>>>>>>>> Ideas?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Suggestions? Approvals and thumbs up?
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Taher Alkhateeb
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>

Re: Proposal to modify the testing framework for OFBiz

[Montalbano Florian <fl...@nereide.fr>]

Hi everyone,

unit tests and integration tests are complementary and with test, the 
more the better.
Implementing unit tests will be a great load of work but it is the same 
with every kind of test. As Taher said, we will gain a lot from those 
unit tests for avoiding regression while refactoring the framework. From 
the start component scope, it has already be proved useful.
And what Ron said about encouraging the creation of "smaller class that 
can be tested easily" is very important too. It can be a first step to 
reach TDD in the OFBiz development. TDD can help to give confidence to 
newcomer and veteran when updating some part of the code and as Hans 
said, this lead to "continuous improvement and finally continuous 
deployment".

I'm aware that this does not solve the technical implementation stuff, 
but I think this is an improvement that need to be supported.

Have a nice day,

Florian Montalbano

Le 19/07/2016 09:04, Pierre Smits a �crit :
> As with anything, the law of diminishing returns also applies to OFBiz and
> tests. This is not true for unit tests and system integration tests, but
> also for user acceptance and performance tests.
>
> Nevertheless, the work done up to now is a good start and - I feel
> confident - appreciated. And unit tests are certainly valuable in the
> framework stack. How it will be for functions (regarding components in
> application and special purpose stack) needs to be addressed when we reach
> that bridge.
>
>
> Best regards,
>
> Pierre Smits
>
> ORRTIZ.COM <http://www.orrtiz.com>
> OFBiz based solutions & services
>
> OFBiz Extensions Marketplace
> http://oem.ofbizci.net/oci-2/
>
> On Tue, Jul 19, 2016 at 8:22 AM, Taher Alkhateeb <slidingfilaments@gmail.com
>> wrote:
>> Hi Scott,
>>
>> Well spoken thank you. Okay may I suggest that for any such work we will
>> discuss it here see its Merit and if it makes sense then we take it in. It
>> is a little early to discuss it right now because we did not yet go to the
>> higher-level components. Once we do I'll be sure to have a conversation
>> about this in here and would appreciate your input to it.
>>
>> Regards,
>>
>> Taher Alkhateeb
>>
>> On Jul 19, 2016 9:00 AM, "Scott Gray" <sc...@hotwaxsystems.com>
>> wrote:
>>
>>> Yeah I'm sure unit tests probably worked well for the start component
>> given
>>> it is the lowest level component in the system and closest to a basic
>> java
>>> app.  I just think the value proposition might decrease the further up
>> the
>>> stack we move with them.  I'm not against unit tests when they prove
>>> useful, but further up the stack I think we should prove the case for
>> them
>>> before doing much work to support the mocking that will be required to
>> keep
>>> them inline with 'best practices'.
>>>
>>> In OFBiz "bad or unwanted" tends to come mostly in the way of increasing
>>> complexity from features that require more effort to maintain than the
>>> value they provide to the community.  I think there's a chance attempting
>>> to mock service tests could fall into that bucket.  I could be wrong,
>> but I
>>> think it's worth looking into before we declare that unit tests are the
>>> best form of testing for OFBiz.
>>>
>>> Regards
>>> Scott
>>>
>>> On 19 July 2016 at 17:37, Taher Alkhateeb <sl...@gmail.com>
>>> wrote:
>>>
>>>> Hi Scott,
>>>>
>>>> Thank you for the feedback. To be focused exactly on integration vs
>>> unit, I
>>>> already mentioned above that at least for me the main objective is to
>>>> confidently and quickly run the tests in short bursts of red-green
>>>> refactor. This allows me to refactor code without waiting in front of
>> the
>>>> screen in between test cycles thus giving me immediate feedback on any
>>>> errors I made. Perhaps my intro was too long so this is the squeezed
>> out
>>>> version of it.
>>>>
>>>> I already had one round of testing in the start component which was
>> much
>>>> faster that way and had an immediate impact. Oh and by the way, you
>>> cannot
>>>> test the start component with integration tests for example unless you
>> do
>>>> it from an external component which cannot access package protected
>>> items.
>>>> This style of coding is applicable I think to any software project
>>>> inclusive of OFBiz. Maybe in certain components more than others or
>>> certain
>>>> areas more than others. But I can't see how it could be bad or
>> unwanted.
>>>> Taher Alkhateeb
>>>>
>>>> On Jul 19, 2016 8:18 AM, "Scott Gray" <sc...@hotwaxsystems.com>
>>>> wrote:
>>>>
>>>>> Thanks Ron and Taher for your responses, I appreciate them but I
>> don't
>>>> hear
>>>>> much in the conversation that speaks to the value of unit tests over
>>>>> integration tests.  Most of the thoughts shared speaks more to the
>>> value
>>>> of
>>>>> tests rather than differences in the type of tests.
>>>>>
>>>>> Speed: At an application level we have ~685 tests that run in 35
>>> seconds
>>>>> (excluding build and data load).  Another point is that there isn't
>>> much
>>>>> reason why tests can't be run in parallel rather than sequentially.
>>>>> TDD: Integration tests in OFBiz are as simple if not simpler to write
>>>> than
>>>>> unit tests and just as useful for TDD.
>>>>> I'm not sure if I missed any other points raised regarding
>> integration
>>>> vs.
>>>>> unit tests?
>>>>>
>>>>> I'm not looking to start a big long debate on the topic, I just
>> wanted
>>> to
>>>>> speak out that someone out there (me) doesn't think unit tests are
>> the
>>>> best
>>>>> solution for testing OFBiz applications.
>>>>>
>>>>> Regards
>>>>> Scott
>>>>>
>>>>> On 19 July 2016 at 16:52, Taher Alkhateeb <
>> slidingfilaments@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Hi Scott,
>>>>>>
>>>>>> Thank you for your input. Your ideas are thought provoking and
>>>> enriching
>>>>> to
>>>>>> the conversation.
>>>>>>
>>>>>> The way I look at it, the general rule is usually many unit tests,
>>> less
>>>>>> integration tests, lesser functional tests. So we are not excluding
>>> any
>>>>>> types of test, all of them are important in certain areas for
>> certain
>>>>>> purposes with certain quantity. Usually integration tests are less
>>>>> because
>>>>>> as you said they just grab more. I like the picture below as a
>>> general
>>>>>> guideline
>>>>>>
>>>>>> http://i.stack.imgur.com/fjQvQ.png
>>>>>>
>>>>>> As you already mentioned unit tests are useful for the framework. I
>>>>>> discovered errors in code that I wrote which I was very very
>> careful
>>>>> with.
>>>>>> I immediately learned the lesson that humans are not designed to
>>> code,
>>>>> and
>>>>>> TDD gives you confidence as you build your code with those short
>>> 30-60
>>>>>> second red-green refactors. I feel much much safer and more
>> confident
>>>>>> writing code that way, the test also documents how to use the api,
>>>>>> refactoring and feature change becomes less terrifying. Also messy
>>> code
>>>>> is
>>>>>> usually not test friendly thus refactoring and unit tests go
>>>> hand-in-hand
>>>>>> for improving the code base.
>>>>>>
>>>>>> Also, I am sorry to say that the framework code is rather messy and
>>>>> brittle
>>>>>> in many places. I think you probably encountered this yourself. The
>>>> same
>>>>> is
>>>>>> said for the applications. Now If we start refactoring without unit
>>>> tests
>>>>>> then we are back to scary business. So much can go wrong so fast
>> and
>>>>> break
>>>>>> things you never expected to break. The framework with all
>>> applications
>>>>>> require heavy refactoring of things like massive ugly methods,
>>>> sandwiched
>>>>>> logic, heavy shared mutable state, hidden dependencies, poor
>>>>>> interfaces, and much more. Every time I touch something I get
>> shocked
>>>> at
>>>>>> how bad it looks, spaghetti logic to no end. If you refactor with
>>>>>> integration tests instead of unit tests then you will come down to
>> a
>>>>>> screeching halt as you wait between these test cycles. My computer
>>>>> actually
>>>>>> takes 10 minutes or more for a full clean load data and testing.
>>>>>>
>>>>>> Now talking about mocking you raise some interesting points, what
>> can
>>>> you
>>>>>> mock vs utilizing integration tests? This is an excellent question
>>> that
>>>>>> does not have a simple answer, and you mostly learn as you code,
>>>>> difficult
>>>>>> to envision without coding. However simple things like Java
>> services
>>>> can
>>>>> be
>>>>>> mocked with a standard mocking class that you use everywhere.
>>>>> Transactions
>>>>>> are difficult to mock and better left to integration tests I think.
>>>> SECAs
>>>>>> might be mocked by simply passing output to input as chaining
>>> methods.
>>>>> Mind
>>>>>> you I am not 100% sure of all of this, coding is the ultimate guide
>>> to
>>>>> what
>>>>>> can or cannot be done.
>>>>>>
>>>>>> So I am not suggesting unit tests as a best-practice (even though
>> it
>>>> is).
>>>>>> Instead I suggest it as something that I and others did and got a
>>> huge
>>>>>> psychological relief and confidence and comfort from. Swimming in
>>> code
>>>>>> without tests is a terrifying business, made more terrifying if the
>>>> code
>>>>> is
>>>>>> bad. Short bursts of red/green/red/green makes you feel good as you
>>>> build
>>>>>> up your logic. And I don't have an exact vision of how to do it,
>>>> because
>>>>>> the details always win.
>>>>>>
>>>>>> I look forward to hearing your thoughts and thank you for enriching
>>>> this
>>>>>> conversation.
>>>>>>
>>>>>> Taher Alkhateeb
>>>>>>
>>>>>> On Tuesday, 19 July 2016, Scott Gray <scott.gray@hotwaxsystems.com
>>>>> wrote:
>>>>>>> I know I'm late to the party here, but I just want to say that I
>>>> think
>>>>>>> integration tests have far greater value to OFBiz than unit
>> tests.
>>>>>> Mostly
>>>>>>> because we tend to have quite a low number of tests and
>> integration
>>>>> tests
>>>>>>> give us much better coverage per line of test code and the tests
>>> are
>>>>> much
>>>>>>> closer to the real world scenarios the application might
>> encounter.
>>>>>>> I don't really see how unit tests could be applied to
>> non-framework
>>>>>> testing
>>>>>>> in a useful manner, could you expand on your vision in that
>> regard?
>>>> I
>>>>>> mean
>>>>>>> would we be testing something smaller than a service as the
>> 'unit'?
>>>>> What
>>>>>>> would we mock? Would transaction management still be active? What
>>>>> happens
>>>>>>> when the service calls another service, I guess we mock the
>>> response
>>>>> from
>>>>>>> that service (how)?
>>>>>>>
>>>>>>> It just seems a very complicated method to achieve a less
>> thorough
>>>> but
>>>>>>> albeit faster (maybe) test result.
>>>>>>>
>>>>>>> A build, data load and full test run takes 4m 9s on my laptop.
>>>>> Excluding
>>>>>>> build, data load and framework tests: the application level tests
>>>> take
>>>>>> 35s,
>>>>>>> not very expensive IMO.  The data load time can be reduced to
>>>>> practically
>>>>>>> nothing by copying a clean slate database into runtime for each
>>> run.
>>>>>>> I'm mostly just suggesting we be wary of adding complicated
>> testing
>>>>>>> procedures in the hope of achieving some 'best practice' result
>>> which
>>>>> in
>>>>>>> reality will provide minimal benefits.
>>>>>>>
>>>>>>> Regards
>>>>>>> Scott
>>>>>>>
>>>>>>> On 18 July 2016 at 18:57, Taher Alkhateeb <
>>>> slidingfilaments@gmail.com
>>>>>>> <javascript:;>>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Hello Akash,
>>>>>>>>
>>>>>>>> Fantastic, I have a few unit tests almost done to be included
>> in
>>>> the
>>>>>>> start
>>>>>>>> component. I will create a new subtask under OFBIZ-1463 to
>> commit
>>>> the
>>>>>>> tests
>>>>>>>> so you can use them as a reference if you like to.
>>>>>>>>
>>>>>>>> I also recommend that you follow the same directory structure
>>>> between
>>>>>> the
>>>>>>>> test code and production code. So for example:
>>>>>>>>
>>>>>>>> Production code:
>>>>>>> framework/start/src/main/java/org/apache/ofbiz/base/start
>>>>>>>> Test code:
>>>> framework/start/src/test/java/org/apache/ofbiz/base/start
>>>>>>>> The benefit of this hierarchy is that you can access non-public
>>>>>> (package
>>>>>>>> protected) methods for testing. This is in fact exactly what I
>>>> needed
>>>>>> to
>>>>>>> be
>>>>>>>> able to apply some of the tests.
>>>>>>>>
>>>>>>>> Cheers,
>>>>>>>>
>>>>>>>> Taher Alkhateeb
>>>>>>>>
>>>>>>>> On Mon, Jul 18, 2016 at 9:22 AM, Akash Jain <
>>>>>>> akash.jain@hotwaxsystems.com <javascript:;>>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Thanks Taher for nice initiative!
>>>>>>>>>
>>>>>>>>> We are planning to written unit tests to all components under
>>>>>>> OFBIZ-1463
>>>>>>>>> Thanks and Regards
>>>>>>>>> --
>>>>>>>>> Akash Jain
>>>>>>>>>
>>>>>>>>> On Mon, Jul 18, 2016 at 10:36 AM, Taher Alkhateeb <
>>>>>>>>> slidingfilaments@gmail.com <javascript:;>> wrote:
>>>>>>>>>
>>>>>>>>>> Hello Everyone,
>>>>>>>>>>
>>>>>>>>>> In reference to this thread and the Jira OFBIZ-7254, I'm
>> very
>>>>> happy
>>>>>>> to
>>>>>>>>>> announce that OFBiz is now ready for applying unit tests
>> with
>>>>> only
>>>>>> 8
>>>>>>>> new
>>>>>>>>>> lines of code in the build script (r1753143) :)
>>>>>>>>>>
>>>>>>>>>> I recommend we do the following moving forward:
>>>>>>>>>>
>>>>>>>>>> 1- Introduce unit tests as much as we can to all components
>>>>>>>>>> 2- Migrate most of the integration tests we currently have
>> to
>>>>> unit
>>>>>>>> tests
>>>>>>>>>> (since they are designed to do very little integration).
>>>>>>>>>>
>>>>>>>>>> This is a great chance for us to practice real TDD in OFBiz
>>>>>>>>>>
>>>>>>>>>> Cheers,
>>>>>>>>>>
>>>>>>>>>> Taher Alkhateeb
>>>>>>>>>>
>>>>>>>>>> On Mon, Jun 13, 2016 at 7:43 AM, Pranay Pandey <
>>>>>>>>>> pranay.pandey@hotwaxsystems.com <javascript:;>> wrote:
>>>>>>>>>>
>>>>>>>>>>> +1
>>>>>>>>>>>
>>>>>>>>>>> Best regards,
>>>>>>>>>>>
>>>>>>>>>>> Pranay Pandey
>>>>>>>>>>> HotWax Systems
>>>>>>>>>>> http://www.hotwaxsystems.com/
>>>>>>>>>>>
>>>>>>>>>>> On Fri, Jun 10, 2016 at 7:46 PM, Taher Alkhateeb <
>>>>>>>>>>> slidingfilaments@gmail.com <javascript:;>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>> Hello Everyone,
>>>>>>>>>>>>
>>>>>>>>>>>> I was able to get a few tests running and this is very
>>>>> doable.
>>>>>>> But
>>>>>>>> I
>>>>>>>>>>> faced
>>>>>>>>>>>> a big problem in designing the testing framework
>> because
>>> of
>>>>>> ANT.
>>>>>>>>>>>> The problem
>>>>>>>>>>>> ----------------
>>>>>>>>>>>> The way the build scripts are designed in OFBiz are
>> very
>>>>>>> complex. A
>>>>>>>>>>> master
>>>>>>>>>>>> file calls other files which call other files. And in
>> the
>>>>>> middle
>>>>>>>> you
>>>>>>>>>> have
>>>>>>>>>>>> external libraries (ant-contrib) and macros, and
>>> variables,
>>>>> and
>>>>>>>> class
>>>>>>>>>>> path
>>>>>>>>>>>> declarations, and and and ....
>>>>>>>>>>>>
>>>>>>>>>>>> I cannot declare the tests programmatically (with JUnit
>>>> test
>>>>>>>> suites)
>>>>>>>>>>>> because this means lower level components would depend
>> on
>>>>>> higher
>>>>>>>>> level
>>>>>>>>>>>> components. So I have to do it in ANT, by navigating
>> this
>>>>> maze
>>>>>> of
>>>>>>>>> build
>>>>>>>>>>>> scripts, and it was a headache for me just to read
>> them,
>>>> let
>>>>>>> alone
>>>>>>>>>> modify
>>>>>>>>>>>> them to create a testing framework.
>>>>>>>>>>>>
>>>>>>>>>>>> Suggested Solution
>>>>>>>>>>>> ------------------------
>>>>>>>>>>>> I suggest to implement the testing framework in Gradle,
>>> and
>>>>>>> simply
>>>>>>>>> call
>>>>>>>>>>> it
>>>>>>>>>>>> from within ant. This is a middle solution that
>> sustains
>>>> ant
>>>>>> for
>>>>>>>> now,
>>>>>>>>>> but
>>>>>>>>>>>> can allow us to switch out later.
>>>>>>>>>>>>
>>>>>>>>>>>> This means I will just add one more file called
>>>> build.gradle
>>>>> in
>>>>>>> the
>>>>>>>>> top
>>>>>>>>>>>> level directory, and figure out the business logic for
>>>>> calling
>>>>>>> the
>>>>>>>>> test
>>>>>>>>>>>> suites from that file
>>>>>>>>>>>>
>>>>>>>>>>>> I look forward to your feedback.
>>>>>>>>>>>>
>>>>>>>>>>>> Regards,
>>>>>>>>>>>>
>>>>>>>>>>>> Taher Alkhateeb
>>>>>>>>>>>>
>>>>>>>>>>>> On Wed, Jun 8, 2016 at 6:00 PM, Taher Alkhateeb <
>>>>>>>>>>>> slidingfilaments@gmail.com <javascript:;>>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Hi Everyone,
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thank you all for your support, JIRA created in
>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OFBIZ-7254
>>>>>>>>>>>>>
>>>>>>>>>>>>> I will start working on it and try to implement ASAP
>> to
>>>> get
>>>>>> my
>>>>>>>>> focus
>>>>>>>>>>> back
>>>>>>>>>>>>> on refactoring.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Cheers!
>>>>>>>>>>>>>
>>>>>>>>>>>>> Taher Alkhateeb
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Wed, Jun 8, 2016 at 4:58 PM, Deepak Dixit <
>>>>>>>>>>>>> deepak.dixit@hotwaxsystems.com <javascript:;>>
>> wrote:
>>>>>>>>>>>>>> +1
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thanks & Regards
>>>>>>>>>>>>>> --
>>>>>>>>>>>>>> Deepak Dixit
>>>>>>>>>>>>>> www.hotwaxsystems.com
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Wed, Jun 8, 2016 at 7:12 PM, Mridul Pathak <
>>>>>>>>>>>>>> mridul.pathak@hotwaxsystems.com <javascript:;>>
>>> wrote:
>>>>>>>>>>>>>>> +1
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Makes perfect sense.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>> Thanks & Regards,
>>>>>>>>>>>>>>> Mridul Pathak
>>>>>>>>>>>>>>> Senior Manager
>>>>>>>>>>>>>>> HotWax Systems
>>>>>>>>>>>>>>> http://www.hotwaxsystems.com
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Jun 8, 2016, at 2:41 PM, Taher Alkhateeb <
>>>>>>>>>>>>>> slidingfilaments@gmail.com <javascript:;>>
>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>> Hello Everyone,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> After refactoring the start component and while
>>>>> starting
>>>>>>> on
>>>>>>>>> the
>>>>>>>>>>> base
>>>>>>>>>>>>>>>> component I realized that the testing framework
>>> for
>>>>>> OFBiz
>>>>>>> is
>>>>>>>>> not
>>>>>>>>>>>> good.
>>>>>>>>>>>>>>> You
>>>>>>>>>>>>>>>> cannot do real test driven development or
>>>>>>> red-green-refactor
>>>>>>>>>> with
>>>>>>>>>>>> the
>>>>>>>>>>>>>>>> current setup, hence my proposal to change it. I
>>>>> explain
>>>>>>>>> below:
>>>>>>>>>>>>>>>> Problem with current design
>>>>>>>>>>>>>>>> ----------------------------------------
>>>>>>>>>>>>>>>> - What we have right now is not unit tests, it's
>>>>> really
>>>>>>>>>>> integration
>>>>>>>>>>>>>>> tests.
>>>>>>>>>>>>>>>> You have to start the framework, the database,
>> the
>>>>>> service
>>>>>>>>>> engine,
>>>>>>>>>>>> the
>>>>>>>>>>>>>>>> entity engine and pretty much everything.
>>>>>>>>>>>>>>>> - Testing is very slow, because it's an
>>> integration
>>>>> test
>>>>>>> as
>>>>>>>> I
>>>>>>>>>>>>>> mentioned
>>>>>>>>>>>>>>>> above. 10 minutes on a good computer!
>>>>>>>>>>>>>>>> - There is zero mocking! We actually have to
>>>>> --load-data
>>>>>>> for
>>>>>>>>>>> things
>>>>>>>>>>>> to
>>>>>>>>>>>>>>>> work. Again, these are integration tests.
>>>>>>>>>>>>>>>> - Too complex: Integration tests by their nature
>>> are
>>>>>>>> grabbing
>>>>>>>>>> too
>>>>>>>>>>>>>> much.
>>>>>>>>>>>>>>>> Mind you, I am not objecting to integration
>> tests
>>> (I
>>>>>>>> actually
>>>>>>>>>> like
>>>>>>>>>>>>>> them)
>>>>>>>>>>>>>>>> but I am objecting to not having real
>> unit-tests.
>>>> Unit
>>>>>>> tests
>>>>>>>>>>> should
>>>>>>>>>>>>>> all
>>>>>>>>>>>>>>> run
>>>>>>>>>>>>>>>> in a few seconds.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Proposed solution
>>>>>>>>>>>>>>>> --------------------------
>>>>>>>>>>>>>>>> - We keep what is considered real integration
>>> tests
>>>>> the
>>>>>>> way
>>>>>>>>> they
>>>>>>>>>>> are
>>>>>>>>>>>>>>> right
>>>>>>>>>>>>>>>> now and keep using them
>>>>>>>>>>>>>>>> - We move what should be unit tests into simple
>>>> JUnit
>>>>>>>> classes,
>>>>>>>>>> and
>>>>>>>>>>>> we
>>>>>>>>>>>>>> do
>>>>>>>>>>>>>>>> not run them using java -jar ofbiz.jar --test,
>> but
>>>>>> instead
>>>>>>>> run
>>>>>>>>>>> them
>>>>>>>>>>>>>>>> directly from the build.xml script, so these
>> files
>>>> are
>>>>>> not
>>>>>>>>>>>> identified
>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>> any XML document, but are simply called
>>> immediately
>>>>> from
>>>>>>> the
>>>>>>>>>> build
>>>>>>>>>>>>>>> scripts.
>>>>>>>>>>>>>>>> - We clearly mark the difference between
>>> integration
>>>>>> tests
>>>>>>>> and
>>>>>>>>>>> unit
>>>>>>>>>>>>>> tests
>>>>>>>>>>>>>>>> (inside the source files or in the suite
>>>>> declarations).
>>>>>>>>>>>>>>>> - We change the run-tests target in build.xml to
>>> run
>>>>>> both
>>>>>>>> unit
>>>>>>>>>>> tests
>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>> integration tests.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I intend to heavily refactor the framework and I
>>>> would
>>>>>>> feel
>>>>>>>>>> better
>>>>>>>>>>>>>> about
>>>>>>>>>>>>>>>> introducing this change while refactoring. What
>> do
>>>> you
>>>>>>> guys
>>>>>>>>>> think?
>>>>>>>>>>>>>> Ideas?
>>>>>>>>>>>>>>>> Suggestions? Approvals and thumbs up?
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Taher Alkhateeb
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>

Re: Proposal to modify the testing framework for OFBiz

[Pierre Smits <pi...@gmail.com>]

As with anything, the law of diminishing returns also applies to OFBiz and
tests. This is not true for unit tests and system integration tests, but
also for user acceptance and performance tests.

Nevertheless, the work done up to now is a good start and - I feel
confident - appreciated. And unit tests are certainly valuable in the
framework stack. How it will be for functions (regarding components in
application and special purpose stack) needs to be addressed when we reach
that bridge.


Best regards,

Pierre Smits

ORRTIZ.COM <http://www.orrtiz.com>
OFBiz based solutions & services

OFBiz Extensions Marketplace
http://oem.ofbizci.net/oci-2/

On Tue, Jul 19, 2016 at 8:22 AM, Taher Alkhateeb <slidingfilaments@gmail.com
> wrote:

> Hi Scott,
>
> Well spoken thank you. Okay may I suggest that for any such work we will
> discuss it here see its Merit and if it makes sense then we take it in. It
> is a little early to discuss it right now because we did not yet go to the
> higher-level components. Once we do I'll be sure to have a conversation
> about this in here and would appreciate your input to it.
>
> Regards,
>
> Taher Alkhateeb
>
> On Jul 19, 2016 9:00 AM, "Scott Gray" <sc...@hotwaxsystems.com>
> wrote:
>
> > Yeah I'm sure unit tests probably worked well for the start component
> given
> > it is the lowest level component in the system and closest to a basic
> java
> > app.  I just think the value proposition might decrease the further up
> the
> > stack we move with them.  I'm not against unit tests when they prove
> > useful, but further up the stack I think we should prove the case for
> them
> > before doing much work to support the mocking that will be required to
> keep
> > them inline with 'best practices'.
> >
> > In OFBiz "bad or unwanted" tends to come mostly in the way of increasing
> > complexity from features that require more effort to maintain than the
> > value they provide to the community.  I think there's a chance attempting
> > to mock service tests could fall into that bucket.  I could be wrong,
> but I
> > think it's worth looking into before we declare that unit tests are the
> > best form of testing for OFBiz.
> >
> > Regards
> > Scott
> >
> > On 19 July 2016 at 17:37, Taher Alkhateeb <sl...@gmail.com>
> > wrote:
> >
> > > Hi Scott,
> > >
> > > Thank you for the feedback. To be focused exactly on integration vs
> > unit, I
> > > already mentioned above that at least for me the main objective is to
> > > confidently and quickly run the tests in short bursts of red-green
> > > refactor. This allows me to refactor code without waiting in front of
> the
> > > screen in between test cycles thus giving me immediate feedback on any
> > > errors I made. Perhaps my intro was too long so this is the squeezed
> out
> > > version of it.
> > >
> > > I already had one round of testing in the start component which was
> much
> > > faster that way and had an immediate impact. Oh and by the way, you
> > cannot
> > > test the start component with integration tests for example unless you
> do
> > > it from an external component which cannot access package protected
> > items.
> > >
> > > This style of coding is applicable I think to any software project
> > > inclusive of OFBiz. Maybe in certain components more than others or
> > certain
> > > areas more than others. But I can't see how it could be bad or
> unwanted.
> > >
> > > Taher Alkhateeb
> > >
> > > On Jul 19, 2016 8:18 AM, "Scott Gray" <sc...@hotwaxsystems.com>
> > > wrote:
> > >
> > > > Thanks Ron and Taher for your responses, I appreciate them but I
> don't
> > > hear
> > > > much in the conversation that speaks to the value of unit tests over
> > > > integration tests.  Most of the thoughts shared speaks more to the
> > value
> > > of
> > > > tests rather than differences in the type of tests.
> > > >
> > > > Speed: At an application level we have ~685 tests that run in 35
> > seconds
> > > > (excluding build and data load).  Another point is that there isn't
> > much
> > > > reason why tests can't be run in parallel rather than sequentially.
> > > > TDD: Integration tests in OFBiz are as simple if not simpler to write
> > > than
> > > > unit tests and just as useful for TDD.
> > > > I'm not sure if I missed any other points raised regarding
> integration
> > > vs.
> > > > unit tests?
> > > >
> > > > I'm not looking to start a big long debate on the topic, I just
> wanted
> > to
> > > > speak out that someone out there (me) doesn't think unit tests are
> the
> > > best
> > > > solution for testing OFBiz applications.
> > > >
> > > > Regards
> > > > Scott
> > > >
> > > > On 19 July 2016 at 16:52, Taher Alkhateeb <
> slidingfilaments@gmail.com>
> > > > wrote:
> > > >
> > > > > Hi Scott,
> > > > >
> > > > > Thank you for your input. Your ideas are thought provoking and
> > > enriching
> > > > to
> > > > > the conversation.
> > > > >
> > > > > The way I look at it, the general rule is usually many unit tests,
> > less
> > > > > integration tests, lesser functional tests. So we are not excluding
> > any
> > > > > types of test, all of them are important in certain areas for
> certain
> > > > > purposes with certain quantity. Usually integration tests are less
> > > > because
> > > > > as you said they just grab more. I like the picture below as a
> > general
> > > > > guideline
> > > > >
> > > > > http://i.stack.imgur.com/fjQvQ.png
> > > > >
> > > > > As you already mentioned unit tests are useful for the framework. I
> > > > > discovered errors in code that I wrote which I was very very
> careful
> > > > with.
> > > > > I immediately learned the lesson that humans are not designed to
> > code,
> > > > and
> > > > > TDD gives you confidence as you build your code with those short
> > 30-60
> > > > > second red-green refactors. I feel much much safer and more
> confident
> > > > > writing code that way, the test also documents how to use the api,
> > > > > refactoring and feature change becomes less terrifying. Also messy
> > code
> > > > is
> > > > > usually not test friendly thus refactoring and unit tests go
> > > hand-in-hand
> > > > > for improving the code base.
> > > > >
> > > > > Also, I am sorry to say that the framework code is rather messy and
> > > > brittle
> > > > > in many places. I think you probably encountered this yourself. The
> > > same
> > > > is
> > > > > said for the applications. Now If we start refactoring without unit
> > > tests
> > > > > then we are back to scary business. So much can go wrong so fast
> and
> > > > break
> > > > > things you never expected to break. The framework with all
> > applications
> > > > > require heavy refactoring of things like massive ugly methods,
> > > sandwiched
> > > > > logic, heavy shared mutable state, hidden dependencies, poor
> > > > > interfaces, and much more. Every time I touch something I get
> shocked
> > > at
> > > > > how bad it looks, spaghetti logic to no end. If you refactor with
> > > > > integration tests instead of unit tests then you will come down to
> a
> > > > > screeching halt as you wait between these test cycles. My computer
> > > > actually
> > > > > takes 10 minutes or more for a full clean load data and testing.
> > > > >
> > > > > Now talking about mocking you raise some interesting points, what
> can
> > > you
> > > > > mock vs utilizing integration tests? This is an excellent question
> > that
> > > > > does not have a simple answer, and you mostly learn as you code,
> > > > difficult
> > > > > to envision without coding. However simple things like Java
> services
> > > can
> > > > be
> > > > > mocked with a standard mocking class that you use everywhere.
> > > > Transactions
> > > > > are difficult to mock and better left to integration tests I think.
> > > SECAs
> > > > > might be mocked by simply passing output to input as chaining
> > methods.
> > > > Mind
> > > > > you I am not 100% sure of all of this, coding is the ultimate guide
> > to
> > > > what
> > > > > can or cannot be done.
> > > > >
> > > > > So I am not suggesting unit tests as a best-practice (even though
> it
> > > is).
> > > > > Instead I suggest it as something that I and others did and got a
> > huge
> > > > > psychological relief and confidence and comfort from. Swimming in
> > code
> > > > > without tests is a terrifying business, made more terrifying if the
> > > code
> > > > is
> > > > > bad. Short bursts of red/green/red/green makes you feel good as you
> > > build
> > > > > up your logic. And I don't have an exact vision of how to do it,
> > > because
> > > > > the details always win.
> > > > >
> > > > > I look forward to hearing your thoughts and thank you for enriching
> > > this
> > > > > conversation.
> > > > >
> > > > > Taher Alkhateeb
> > > > >
> > > > > On Tuesday, 19 July 2016, Scott Gray <scott.gray@hotwaxsystems.com
> >
> > > > wrote:
> > > > >
> > > > > > I know I'm late to the party here, but I just want to say that I
> > > think
> > > > > > integration tests have far greater value to OFBiz than unit
> tests.
> > > > > Mostly
> > > > > > because we tend to have quite a low number of tests and
> integration
> > > > tests
> > > > > > give us much better coverage per line of test code and the tests
> > are
> > > > much
> > > > > > closer to the real world scenarios the application might
> encounter.
> > > > > >
> > > > > > I don't really see how unit tests could be applied to
> non-framework
> > > > > testing
> > > > > > in a useful manner, could you expand on your vision in that
> regard?
> > > I
> > > > > mean
> > > > > > would we be testing something smaller than a service as the
> 'unit'?
> > > > What
> > > > > > would we mock? Would transaction management still be active? What
> > > > happens
> > > > > > when the service calls another service, I guess we mock the
> > response
> > > > from
> > > > > > that service (how)?
> > > > > >
> > > > > > It just seems a very complicated method to achieve a less
> thorough
> > > but
> > > > > > albeit faster (maybe) test result.
> > > > > >
> > > > > > A build, data load and full test run takes 4m 9s on my laptop.
> > > > Excluding
> > > > > > build, data load and framework tests: the application level tests
> > > take
> > > > > 35s,
> > > > > > not very expensive IMO.  The data load time can be reduced to
> > > > practically
> > > > > > nothing by copying a clean slate database into runtime for each
> > run.
> > > > > >
> > > > > > I'm mostly just suggesting we be wary of adding complicated
> testing
> > > > > > procedures in the hope of achieving some 'best practice' result
> > which
> > > > in
> > > > > > reality will provide minimal benefits.
> > > > > >
> > > > > > Regards
> > > > > > Scott
> > > > > >
> > > > > > On 18 July 2016 at 18:57, Taher Alkhateeb <
> > > slidingfilaments@gmail.com
> > > > > > <javascript:;>>
> > > > > > wrote:
> > > > > >
> > > > > > > Hello Akash,
> > > > > > >
> > > > > > > Fantastic, I have a few unit tests almost done to be included
> in
> > > the
> > > > > > start
> > > > > > > component. I will create a new subtask under OFBIZ-1463 to
> commit
> > > the
> > > > > > tests
> > > > > > > so you can use them as a reference if you like to.
> > > > > > >
> > > > > > > I also recommend that you follow the same directory structure
> > > between
> > > > > the
> > > > > > > test code and production code. So for example:
> > > > > > >
> > > > > > > Production code:
> > > > > > framework/start/src/main/java/org/apache/ofbiz/base/start
> > > > > > > Test code:
> > > framework/start/src/test/java/org/apache/ofbiz/base/start
> > > > > > >
> > > > > > > The benefit of this hierarchy is that you can access non-public
> > > > > (package
> > > > > > > protected) methods for testing. This is in fact exactly what I
> > > needed
> > > > > to
> > > > > > be
> > > > > > > able to apply some of the tests.
> > > > > > >
> > > > > > > Cheers,
> > > > > > >
> > > > > > > Taher Alkhateeb
> > > > > > >
> > > > > > > On Mon, Jul 18, 2016 at 9:22 AM, Akash Jain <
> > > > > > akash.jain@hotwaxsystems.com <javascript:;>>
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Thanks Taher for nice initiative!
> > > > > > > >
> > > > > > > > We are planning to written unit tests to all components under
> > > > > > OFBIZ-1463
> > > > > > > >
> > > > > > > > Thanks and Regards
> > > > > > > > --
> > > > > > > > Akash Jain
> > > > > > > >
> > > > > > > > On Mon, Jul 18, 2016 at 10:36 AM, Taher Alkhateeb <
> > > > > > > > slidingfilaments@gmail.com <javascript:;>> wrote:
> > > > > > > >
> > > > > > > > > Hello Everyone,
> > > > > > > > >
> > > > > > > > > In reference to this thread and the Jira OFBIZ-7254, I'm
> very
> > > > happy
> > > > > > to
> > > > > > > > > announce that OFBiz is now ready for applying unit tests
> with
> > > > only
> > > > > 8
> > > > > > > new
> > > > > > > > > lines of code in the build script (r1753143) :)
> > > > > > > > >
> > > > > > > > > I recommend we do the following moving forward:
> > > > > > > > >
> > > > > > > > > 1- Introduce unit tests as much as we can to all components
> > > > > > > > > 2- Migrate most of the integration tests we currently have
> to
> > > > unit
> > > > > > > tests
> > > > > > > > > (since they are designed to do very little integration).
> > > > > > > > >
> > > > > > > > > This is a great chance for us to practice real TDD in OFBiz
> > > > > > > > >
> > > > > > > > > Cheers,
> > > > > > > > >
> > > > > > > > > Taher Alkhateeb
> > > > > > > > >
> > > > > > > > > On Mon, Jun 13, 2016 at 7:43 AM, Pranay Pandey <
> > > > > > > > > pranay.pandey@hotwaxsystems.com <javascript:;>> wrote:
> > > > > > > > >
> > > > > > > > > > +1
> > > > > > > > > >
> > > > > > > > > > Best regards,
> > > > > > > > > >
> > > > > > > > > > Pranay Pandey
> > > > > > > > > > HotWax Systems
> > > > > > > > > > http://www.hotwaxsystems.com/
> > > > > > > > > >
> > > > > > > > > > On Fri, Jun 10, 2016 at 7:46 PM, Taher Alkhateeb <
> > > > > > > > > > slidingfilaments@gmail.com <javascript:;>
> > > > > > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > > Hello Everyone,
> > > > > > > > > > >
> > > > > > > > > > > I was able to get a few tests running and this is very
> > > > doable.
> > > > > > But
> > > > > > > I
> > > > > > > > > > faced
> > > > > > > > > > > a big problem in designing the testing framework
> because
> > of
> > > > > ANT.
> > > > > > > > > > >
> > > > > > > > > > > The problem
> > > > > > > > > > > ----------------
> > > > > > > > > > > The way the build scripts are designed in OFBiz are
> very
> > > > > > complex. A
> > > > > > > > > > master
> > > > > > > > > > > file calls other files which call other files. And in
> the
> > > > > middle
> > > > > > > you
> > > > > > > > > have
> > > > > > > > > > > external libraries (ant-contrib) and macros, and
> > variables,
> > > > and
> > > > > > > class
> > > > > > > > > > path
> > > > > > > > > > > declarations, and and and ....
> > > > > > > > > > >
> > > > > > > > > > > I cannot declare the tests programmatically (with JUnit
> > > test
> > > > > > > suites)
> > > > > > > > > > > because this means lower level components would depend
> on
> > > > > higher
> > > > > > > > level
> > > > > > > > > > > components. So I have to do it in ANT, by navigating
> this
> > > > maze
> > > > > of
> > > > > > > > build
> > > > > > > > > > > scripts, and it was a headache for me just to read
> them,
> > > let
> > > > > > alone
> > > > > > > > > modify
> > > > > > > > > > > them to create a testing framework.
> > > > > > > > > > >
> > > > > > > > > > > Suggested Solution
> > > > > > > > > > > ------------------------
> > > > > > > > > > > I suggest to implement the testing framework in Gradle,
> > and
> > > > > > simply
> > > > > > > > call
> > > > > > > > > > it
> > > > > > > > > > > from within ant. This is a middle solution that
> sustains
> > > ant
> > > > > for
> > > > > > > now,
> > > > > > > > > but
> > > > > > > > > > > can allow us to switch out later.
> > > > > > > > > > >
> > > > > > > > > > > This means I will just add one more file called
> > > build.gradle
> > > > in
> > > > > > the
> > > > > > > > top
> > > > > > > > > > > level directory, and figure out the business logic for
> > > > calling
> > > > > > the
> > > > > > > > test
> > > > > > > > > > > suites from that file
> > > > > > > > > > >
> > > > > > > > > > > I look forward to your feedback.
> > > > > > > > > > >
> > > > > > > > > > > Regards,
> > > > > > > > > > >
> > > > > > > > > > > Taher Alkhateeb
> > > > > > > > > > >
> > > > > > > > > > > On Wed, Jun 8, 2016 at 6:00 PM, Taher Alkhateeb <
> > > > > > > > > > > slidingfilaments@gmail.com <javascript:;>>
> > > > > > > > > > > wrote:
> > > > > > > > > > >
> > > > > > > > > > > > Hi Everyone,
> > > > > > > > > > > >
> > > > > > > > > > > > Thank you all for your support, JIRA created in
> > > > > > > > > > > > https://issues.apache.org/jira/browse/OFBIZ-7254
> > > > > > > > > > > >
> > > > > > > > > > > > I will start working on it and try to implement ASAP
> to
> > > get
> > > > > my
> > > > > > > > focus
> > > > > > > > > > back
> > > > > > > > > > > > on refactoring.
> > > > > > > > > > > >
> > > > > > > > > > > > Cheers!
> > > > > > > > > > > >
> > > > > > > > > > > > Taher Alkhateeb
> > > > > > > > > > > >
> > > > > > > > > > > > On Wed, Jun 8, 2016 at 4:58 PM, Deepak Dixit <
> > > > > > > > > > > > deepak.dixit@hotwaxsystems.com <javascript:;>>
> wrote:
> > > > > > > > > > > >
> > > > > > > > > > > >> +1
> > > > > > > > > > > >>
> > > > > > > > > > > >> Thanks & Regards
> > > > > > > > > > > >> --
> > > > > > > > > > > >> Deepak Dixit
> > > > > > > > > > > >> www.hotwaxsystems.com
> > > > > > > > > > > >>
> > > > > > > > > > > >> On Wed, Jun 8, 2016 at 7:12 PM, Mridul Pathak <
> > > > > > > > > > > >> mridul.pathak@hotwaxsystems.com <javascript:;>>
> > wrote:
> > > > > > > > > > > >>
> > > > > > > > > > > >> > +1
> > > > > > > > > > > >> >
> > > > > > > > > > > >> > Makes perfect sense.
> > > > > > > > > > > >> >
> > > > > > > > > > > >> > --
> > > > > > > > > > > >> > Thanks & Regards,
> > > > > > > > > > > >> > Mridul Pathak
> > > > > > > > > > > >> > Senior Manager
> > > > > > > > > > > >> > HotWax Systems
> > > > > > > > > > > >> > http://www.hotwaxsystems.com
> > > > > > > > > > > >> >
> > > > > > > > > > > >> > > On Jun 8, 2016, at 2:41 PM, Taher Alkhateeb <
> > > > > > > > > > > >> slidingfilaments@gmail.com <javascript:;>>
> > > > > > > > > > > >> > wrote:
> > > > > > > > > > > >> > >
> > > > > > > > > > > >> > > Hello Everyone,
> > > > > > > > > > > >> > >
> > > > > > > > > > > >> > > After refactoring the start component and while
> > > > starting
> > > > > > on
> > > > > > > > the
> > > > > > > > > > base
> > > > > > > > > > > >> > > component I realized that the testing framework
> > for
> > > > > OFBiz
> > > > > > is
> > > > > > > > not
> > > > > > > > > > > good.
> > > > > > > > > > > >> > You
> > > > > > > > > > > >> > > cannot do real test driven development or
> > > > > > red-green-refactor
> > > > > > > > > with
> > > > > > > > > > > the
> > > > > > > > > > > >> > > current setup, hence my proposal to change it. I
> > > > explain
> > > > > > > > below:
> > > > > > > > > > > >> > >
> > > > > > > > > > > >> > > Problem with current design
> > > > > > > > > > > >> > > ----------------------------------------
> > > > > > > > > > > >> > > - What we have right now is not unit tests, it's
> > > > really
> > > > > > > > > > integration
> > > > > > > > > > > >> > tests.
> > > > > > > > > > > >> > > You have to start the framework, the database,
> the
> > > > > service
> > > > > > > > > engine,
> > > > > > > > > > > the
> > > > > > > > > > > >> > > entity engine and pretty much everything.
> > > > > > > > > > > >> > > - Testing is very slow, because it's an
> > integration
> > > > test
> > > > > > as
> > > > > > > I
> > > > > > > > > > > >> mentioned
> > > > > > > > > > > >> > > above. 10 minutes on a good computer!
> > > > > > > > > > > >> > > - There is zero mocking! We actually have to
> > > > --load-data
> > > > > > for
> > > > > > > > > > things
> > > > > > > > > > > to
> > > > > > > > > > > >> > > work. Again, these are integration tests.
> > > > > > > > > > > >> > > - Too complex: Integration tests by their nature
> > are
> > > > > > > grabbing
> > > > > > > > > too
> > > > > > > > > > > >> much.
> > > > > > > > > > > >> > > Mind you, I am not objecting to integration
> tests
> > (I
> > > > > > > actually
> > > > > > > > > like
> > > > > > > > > > > >> them)
> > > > > > > > > > > >> > > but I am objecting to not having real
> unit-tests.
> > > Unit
> > > > > > tests
> > > > > > > > > > should
> > > > > > > > > > > >> all
> > > > > > > > > > > >> > run
> > > > > > > > > > > >> > > in a few seconds.
> > > > > > > > > > > >> > >
> > > > > > > > > > > >> > > Proposed solution
> > > > > > > > > > > >> > > --------------------------
> > > > > > > > > > > >> > > - We keep what is considered real integration
> > tests
> > > > the
> > > > > > way
> > > > > > > > they
> > > > > > > > > > are
> > > > > > > > > > > >> > right
> > > > > > > > > > > >> > > now and keep using them
> > > > > > > > > > > >> > > - We move what should be unit tests into simple
> > > JUnit
> > > > > > > classes,
> > > > > > > > > and
> > > > > > > > > > > we
> > > > > > > > > > > >> do
> > > > > > > > > > > >> > > not run them using java -jar ofbiz.jar --test,
> but
> > > > > instead
> > > > > > > run
> > > > > > > > > > them
> > > > > > > > > > > >> > > directly from the build.xml script, so these
> files
> > > are
> > > > > not
> > > > > > > > > > > identified
> > > > > > > > > > > >> in
> > > > > > > > > > > >> > > any XML document, but are simply called
> > immediately
> > > > from
> > > > > > the
> > > > > > > > > build
> > > > > > > > > > > >> > scripts.
> > > > > > > > > > > >> > > - We clearly mark the difference between
> > integration
> > > > > tests
> > > > > > > and
> > > > > > > > > > unit
> > > > > > > > > > > >> tests
> > > > > > > > > > > >> > > (inside the source files or in the suite
> > > > declarations).
> > > > > > > > > > > >> > > - We change the run-tests target in build.xml to
> > run
> > > > > both
> > > > > > > unit
> > > > > > > > > > tests
> > > > > > > > > > > >> and
> > > > > > > > > > > >> > > integration tests.
> > > > > > > > > > > >> > >
> > > > > > > > > > > >> > > I intend to heavily refactor the framework and I
> > > would
> > > > > > feel
> > > > > > > > > better
> > > > > > > > > > > >> about
> > > > > > > > > > > >> > > introducing this change while refactoring. What
> do
> > > you
> > > > > > guys
> > > > > > > > > think?
> > > > > > > > > > > >> Ideas?
> > > > > > > > > > > >> > > Suggestions? Approvals and thumbs up?
> > > > > > > > > > > >> > >
> > > > > > > > > > > >> > > Regards,
> > > > > > > > > > > >> > >
> > > > > > > > > > > >> > > Taher Alkhateeb
> > > > > > > > > > > >> >
> > > > > > > > > > > >> >
> > > > > > > > > > > >>
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: Proposal to modify the testing framework for OFBiz

[Taher Alkhateeb <sl...@gmail.com>]

Hi Scott,

Well spoken thank you. Okay may I suggest that for any such work we will
discuss it here see its Merit and if it makes sense then we take it in. It
is a little early to discuss it right now because we did not yet go to the
higher-level components. Once we do I'll be sure to have a conversation
about this in here and would appreciate your input to it.

Regards,

Taher Alkhateeb

On Jul 19, 2016 9:00 AM, "Scott Gray" <sc...@hotwaxsystems.com> wrote:

> Yeah I'm sure unit tests probably worked well for the start component given
> it is the lowest level component in the system and closest to a basic java
> app.  I just think the value proposition might decrease the further up the
> stack we move with them.  I'm not against unit tests when they prove
> useful, but further up the stack I think we should prove the case for them
> before doing much work to support the mocking that will be required to keep
> them inline with 'best practices'.
>
> In OFBiz "bad or unwanted" tends to come mostly in the way of increasing
> complexity from features that require more effort to maintain than the
> value they provide to the community.  I think there's a chance attempting
> to mock service tests could fall into that bucket.  I could be wrong, but I
> think it's worth looking into before we declare that unit tests are the
> best form of testing for OFBiz.
>
> Regards
> Scott
>
> On 19 July 2016 at 17:37, Taher Alkhateeb <sl...@gmail.com>
> wrote:
>
> > Hi Scott,
> >
> > Thank you for the feedback. To be focused exactly on integration vs
> unit, I
> > already mentioned above that at least for me the main objective is to
> > confidently and quickly run the tests in short bursts of red-green
> > refactor. This allows me to refactor code without waiting in front of the
> > screen in between test cycles thus giving me immediate feedback on any
> > errors I made. Perhaps my intro was too long so this is the squeezed out
> > version of it.
> >
> > I already had one round of testing in the start component which was much
> > faster that way and had an immediate impact. Oh and by the way, you
> cannot
> > test the start component with integration tests for example unless you do
> > it from an external component which cannot access package protected
> items.
> >
> > This style of coding is applicable I think to any software project
> > inclusive of OFBiz. Maybe in certain components more than others or
> certain
> > areas more than others. But I can't see how it could be bad or unwanted.
> >
> > Taher Alkhateeb
> >
> > On Jul 19, 2016 8:18 AM, "Scott Gray" <sc...@hotwaxsystems.com>
> > wrote:
> >
> > > Thanks Ron and Taher for your responses, I appreciate them but I don't
> > hear
> > > much in the conversation that speaks to the value of unit tests over
> > > integration tests.  Most of the thoughts shared speaks more to the
> value
> > of
> > > tests rather than differences in the type of tests.
> > >
> > > Speed: At an application level we have ~685 tests that run in 35
> seconds
> > > (excluding build and data load).  Another point is that there isn't
> much
> > > reason why tests can't be run in parallel rather than sequentially.
> > > TDD: Integration tests in OFBiz are as simple if not simpler to write
> > than
> > > unit tests and just as useful for TDD.
> > > I'm not sure if I missed any other points raised regarding integration
> > vs.
> > > unit tests?
> > >
> > > I'm not looking to start a big long debate on the topic, I just wanted
> to
> > > speak out that someone out there (me) doesn't think unit tests are the
> > best
> > > solution for testing OFBiz applications.
> > >
> > > Regards
> > > Scott
> > >
> > > On 19 July 2016 at 16:52, Taher Alkhateeb <sl...@gmail.com>
> > > wrote:
> > >
> > > > Hi Scott,
> > > >
> > > > Thank you for your input. Your ideas are thought provoking and
> > enriching
> > > to
> > > > the conversation.
> > > >
> > > > The way I look at it, the general rule is usually many unit tests,
> less
> > > > integration tests, lesser functional tests. So we are not excluding
> any
> > > > types of test, all of them are important in certain areas for certain
> > > > purposes with certain quantity. Usually integration tests are less
> > > because
> > > > as you said they just grab more. I like the picture below as a
> general
> > > > guideline
> > > >
> > > > http://i.stack.imgur.com/fjQvQ.png
> > > >
> > > > As you already mentioned unit tests are useful for the framework. I
> > > > discovered errors in code that I wrote which I was very very careful
> > > with.
> > > > I immediately learned the lesson that humans are not designed to
> code,
> > > and
> > > > TDD gives you confidence as you build your code with those short
> 30-60
> > > > second red-green refactors. I feel much much safer and more confident
> > > > writing code that way, the test also documents how to use the api,
> > > > refactoring and feature change becomes less terrifying. Also messy
> code
> > > is
> > > > usually not test friendly thus refactoring and unit tests go
> > hand-in-hand
> > > > for improving the code base.
> > > >
> > > > Also, I am sorry to say that the framework code is rather messy and
> > > brittle
> > > > in many places. I think you probably encountered this yourself. The
> > same
> > > is
> > > > said for the applications. Now If we start refactoring without unit
> > tests
> > > > then we are back to scary business. So much can go wrong so fast and
> > > break
> > > > things you never expected to break. The framework with all
> applications
> > > > require heavy refactoring of things like massive ugly methods,
> > sandwiched
> > > > logic, heavy shared mutable state, hidden dependencies, poor
> > > > interfaces, and much more. Every time I touch something I get shocked
> > at
> > > > how bad it looks, spaghetti logic to no end. If you refactor with
> > > > integration tests instead of unit tests then you will come down to a
> > > > screeching halt as you wait between these test cycles. My computer
> > > actually
> > > > takes 10 minutes or more for a full clean load data and testing.
> > > >
> > > > Now talking about mocking you raise some interesting points, what can
> > you
> > > > mock vs utilizing integration tests? This is an excellent question
> that
> > > > does not have a simple answer, and you mostly learn as you code,
> > > difficult
> > > > to envision without coding. However simple things like Java services
> > can
> > > be
> > > > mocked with a standard mocking class that you use everywhere.
> > > Transactions
> > > > are difficult to mock and better left to integration tests I think.
> > SECAs
> > > > might be mocked by simply passing output to input as chaining
> methods.
> > > Mind
> > > > you I am not 100% sure of all of this, coding is the ultimate guide
> to
> > > what
> > > > can or cannot be done.
> > > >
> > > > So I am not suggesting unit tests as a best-practice (even though it
> > is).
> > > > Instead I suggest it as something that I and others did and got a
> huge
> > > > psychological relief and confidence and comfort from. Swimming in
> code
> > > > without tests is a terrifying business, made more terrifying if the
> > code
> > > is
> > > > bad. Short bursts of red/green/red/green makes you feel good as you
> > build
> > > > up your logic. And I don't have an exact vision of how to do it,
> > because
> > > > the details always win.
> > > >
> > > > I look forward to hearing your thoughts and thank you for enriching
> > this
> > > > conversation.
> > > >
> > > > Taher Alkhateeb
> > > >
> > > > On Tuesday, 19 July 2016, Scott Gray <sc...@hotwaxsystems.com>
> > > wrote:
> > > >
> > > > > I know I'm late to the party here, but I just want to say that I
> > think
> > > > > integration tests have far greater value to OFBiz than unit tests.
> > > > Mostly
> > > > > because we tend to have quite a low number of tests and integration
> > > tests
> > > > > give us much better coverage per line of test code and the tests
> are
> > > much
> > > > > closer to the real world scenarios the application might encounter.
> > > > >
> > > > > I don't really see how unit tests could be applied to non-framework
> > > > testing
> > > > > in a useful manner, could you expand on your vision in that regard?
> > I
> > > > mean
> > > > > would we be testing something smaller than a service as the 'unit'?
> > > What
> > > > > would we mock? Would transaction management still be active? What
> > > happens
> > > > > when the service calls another service, I guess we mock the
> response
> > > from
> > > > > that service (how)?
> > > > >
> > > > > It just seems a very complicated method to achieve a less thorough
> > but
> > > > > albeit faster (maybe) test result.
> > > > >
> > > > > A build, data load and full test run takes 4m 9s on my laptop.
> > > Excluding
> > > > > build, data load and framework tests: the application level tests
> > take
> > > > 35s,
> > > > > not very expensive IMO.  The data load time can be reduced to
> > > practically
> > > > > nothing by copying a clean slate database into runtime for each
> run.
> > > > >
> > > > > I'm mostly just suggesting we be wary of adding complicated testing
> > > > > procedures in the hope of achieving some 'best practice' result
> which
> > > in
> > > > > reality will provide minimal benefits.
> > > > >
> > > > > Regards
> > > > > Scott
> > > > >
> > > > > On 18 July 2016 at 18:57, Taher Alkhateeb <
> > slidingfilaments@gmail.com
> > > > > <javascript:;>>
> > > > > wrote:
> > > > >
> > > > > > Hello Akash,
> > > > > >
> > > > > > Fantastic, I have a few unit tests almost done to be included in
> > the
> > > > > start
> > > > > > component. I will create a new subtask under OFBIZ-1463 to commit
> > the
> > > > > tests
> > > > > > so you can use them as a reference if you like to.
> > > > > >
> > > > > > I also recommend that you follow the same directory structure
> > between
> > > > the
> > > > > > test code and production code. So for example:
> > > > > >
> > > > > > Production code:
> > > > > framework/start/src/main/java/org/apache/ofbiz/base/start
> > > > > > Test code:
> > framework/start/src/test/java/org/apache/ofbiz/base/start
> > > > > >
> > > > > > The benefit of this hierarchy is that you can access non-public
> > > > (package
> > > > > > protected) methods for testing. This is in fact exactly what I
> > needed
> > > > to
> > > > > be
> > > > > > able to apply some of the tests.
> > > > > >
> > > > > > Cheers,
> > > > > >
> > > > > > Taher Alkhateeb
> > > > > >
> > > > > > On Mon, Jul 18, 2016 at 9:22 AM, Akash Jain <
> > > > > akash.jain@hotwaxsystems.com <javascript:;>>
> > > > > > wrote:
> > > > > >
> > > > > > > Thanks Taher for nice initiative!
> > > > > > >
> > > > > > > We are planning to written unit tests to all components under
> > > > > OFBIZ-1463
> > > > > > >
> > > > > > > Thanks and Regards
> > > > > > > --
> > > > > > > Akash Jain
> > > > > > >
> > > > > > > On Mon, Jul 18, 2016 at 10:36 AM, Taher Alkhateeb <
> > > > > > > slidingfilaments@gmail.com <javascript:;>> wrote:
> > > > > > >
> > > > > > > > Hello Everyone,
> > > > > > > >
> > > > > > > > In reference to this thread and the Jira OFBIZ-7254, I'm very
> > > happy
> > > > > to
> > > > > > > > announce that OFBiz is now ready for applying unit tests with
> > > only
> > > > 8
> > > > > > new
> > > > > > > > lines of code in the build script (r1753143) :)
> > > > > > > >
> > > > > > > > I recommend we do the following moving forward:
> > > > > > > >
> > > > > > > > 1- Introduce unit tests as much as we can to all components
> > > > > > > > 2- Migrate most of the integration tests we currently have to
> > > unit
> > > > > > tests
> > > > > > > > (since they are designed to do very little integration).
> > > > > > > >
> > > > > > > > This is a great chance for us to practice real TDD in OFBiz
> > > > > > > >
> > > > > > > > Cheers,
> > > > > > > >
> > > > > > > > Taher Alkhateeb
> > > > > > > >
> > > > > > > > On Mon, Jun 13, 2016 at 7:43 AM, Pranay Pandey <
> > > > > > > > pranay.pandey@hotwaxsystems.com <javascript:;>> wrote:
> > > > > > > >
> > > > > > > > > +1
> > > > > > > > >
> > > > > > > > > Best regards,
> > > > > > > > >
> > > > > > > > > Pranay Pandey
> > > > > > > > > HotWax Systems
> > > > > > > > > http://www.hotwaxsystems.com/
> > > > > > > > >
> > > > > > > > > On Fri, Jun 10, 2016 at 7:46 PM, Taher Alkhateeb <
> > > > > > > > > slidingfilaments@gmail.com <javascript:;>
> > > > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > > Hello Everyone,
> > > > > > > > > >
> > > > > > > > > > I was able to get a few tests running and this is very
> > > doable.
> > > > > But
> > > > > > I
> > > > > > > > > faced
> > > > > > > > > > a big problem in designing the testing framework because
> of
> > > > ANT.
> > > > > > > > > >
> > > > > > > > > > The problem
> > > > > > > > > > ----------------
> > > > > > > > > > The way the build scripts are designed in OFBiz are very
> > > > > complex. A
> > > > > > > > > master
> > > > > > > > > > file calls other files which call other files. And in the
> > > > middle
> > > > > > you
> > > > > > > > have
> > > > > > > > > > external libraries (ant-contrib) and macros, and
> variables,
> > > and
> > > > > > class
> > > > > > > > > path
> > > > > > > > > > declarations, and and and ....
> > > > > > > > > >
> > > > > > > > > > I cannot declare the tests programmatically (with JUnit
> > test
> > > > > > suites)
> > > > > > > > > > because this means lower level components would depend on
> > > > higher
> > > > > > > level
> > > > > > > > > > components. So I have to do it in ANT, by navigating this
> > > maze
> > > > of
> > > > > > > build
> > > > > > > > > > scripts, and it was a headache for me just to read them,
> > let
> > > > > alone
> > > > > > > > modify
> > > > > > > > > > them to create a testing framework.
> > > > > > > > > >
> > > > > > > > > > Suggested Solution
> > > > > > > > > > ------------------------
> > > > > > > > > > I suggest to implement the testing framework in Gradle,
> and
> > > > > simply
> > > > > > > call
> > > > > > > > > it
> > > > > > > > > > from within ant. This is a middle solution that sustains
> > ant
> > > > for
> > > > > > now,
> > > > > > > > but
> > > > > > > > > > can allow us to switch out later.
> > > > > > > > > >
> > > > > > > > > > This means I will just add one more file called
> > build.gradle
> > > in
> > > > > the
> > > > > > > top
> > > > > > > > > > level directory, and figure out the business logic for
> > > calling
> > > > > the
> > > > > > > test
> > > > > > > > > > suites from that file
> > > > > > > > > >
> > > > > > > > > > I look forward to your feedback.
> > > > > > > > > >
> > > > > > > > > > Regards,
> > > > > > > > > >
> > > > > > > > > > Taher Alkhateeb
> > > > > > > > > >
> > > > > > > > > > On Wed, Jun 8, 2016 at 6:00 PM, Taher Alkhateeb <
> > > > > > > > > > slidingfilaments@gmail.com <javascript:;>>
> > > > > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > > Hi Everyone,
> > > > > > > > > > >
> > > > > > > > > > > Thank you all for your support, JIRA created in
> > > > > > > > > > > https://issues.apache.org/jira/browse/OFBIZ-7254
> > > > > > > > > > >
> > > > > > > > > > > I will start working on it and try to implement ASAP to
> > get
> > > > my
> > > > > > > focus
> > > > > > > > > back
> > > > > > > > > > > on refactoring.
> > > > > > > > > > >
> > > > > > > > > > > Cheers!
> > > > > > > > > > >
> > > > > > > > > > > Taher Alkhateeb
> > > > > > > > > > >
> > > > > > > > > > > On Wed, Jun 8, 2016 at 4:58 PM, Deepak Dixit <
> > > > > > > > > > > deepak.dixit@hotwaxsystems.com <javascript:;>> wrote:
> > > > > > > > > > >
> > > > > > > > > > >> +1
> > > > > > > > > > >>
> > > > > > > > > > >> Thanks & Regards
> > > > > > > > > > >> --
> > > > > > > > > > >> Deepak Dixit
> > > > > > > > > > >> www.hotwaxsystems.com
> > > > > > > > > > >>
> > > > > > > > > > >> On Wed, Jun 8, 2016 at 7:12 PM, Mridul Pathak <
> > > > > > > > > > >> mridul.pathak@hotwaxsystems.com <javascript:;>>
> wrote:
> > > > > > > > > > >>
> > > > > > > > > > >> > +1
> > > > > > > > > > >> >
> > > > > > > > > > >> > Makes perfect sense.
> > > > > > > > > > >> >
> > > > > > > > > > >> > --
> > > > > > > > > > >> > Thanks & Regards,
> > > > > > > > > > >> > Mridul Pathak
> > > > > > > > > > >> > Senior Manager
> > > > > > > > > > >> > HotWax Systems
> > > > > > > > > > >> > http://www.hotwaxsystems.com
> > > > > > > > > > >> >
> > > > > > > > > > >> > > On Jun 8, 2016, at 2:41 PM, Taher Alkhateeb <
> > > > > > > > > > >> slidingfilaments@gmail.com <javascript:;>>
> > > > > > > > > > >> > wrote:
> > > > > > > > > > >> > >
> > > > > > > > > > >> > > Hello Everyone,
> > > > > > > > > > >> > >
> > > > > > > > > > >> > > After refactoring the start component and while
> > > starting
> > > > > on
> > > > > > > the
> > > > > > > > > base
> > > > > > > > > > >> > > component I realized that the testing framework
> for
> > > > OFBiz
> > > > > is
> > > > > > > not
> > > > > > > > > > good.
> > > > > > > > > > >> > You
> > > > > > > > > > >> > > cannot do real test driven development or
> > > > > red-green-refactor
> > > > > > > > with
> > > > > > > > > > the
> > > > > > > > > > >> > > current setup, hence my proposal to change it. I
> > > explain
> > > > > > > below:
> > > > > > > > > > >> > >
> > > > > > > > > > >> > > Problem with current design
> > > > > > > > > > >> > > ----------------------------------------
> > > > > > > > > > >> > > - What we have right now is not unit tests, it's
> > > really
> > > > > > > > > integration
> > > > > > > > > > >> > tests.
> > > > > > > > > > >> > > You have to start the framework, the database, the
> > > > service
> > > > > > > > engine,
> > > > > > > > > > the
> > > > > > > > > > >> > > entity engine and pretty much everything.
> > > > > > > > > > >> > > - Testing is very slow, because it's an
> integration
> > > test
> > > > > as
> > > > > > I
> > > > > > > > > > >> mentioned
> > > > > > > > > > >> > > above. 10 minutes on a good computer!
> > > > > > > > > > >> > > - There is zero mocking! We actually have to
> > > --load-data
> > > > > for
> > > > > > > > > things
> > > > > > > > > > to
> > > > > > > > > > >> > > work. Again, these are integration tests.
> > > > > > > > > > >> > > - Too complex: Integration tests by their nature
> are
> > > > > > grabbing
> > > > > > > > too
> > > > > > > > > > >> much.
> > > > > > > > > > >> > > Mind you, I am not objecting to integration tests
> (I
> > > > > > actually
> > > > > > > > like
> > > > > > > > > > >> them)
> > > > > > > > > > >> > > but I am objecting to not having real unit-tests.
> > Unit
> > > > > tests
> > > > > > > > > should
> > > > > > > > > > >> all
> > > > > > > > > > >> > run
> > > > > > > > > > >> > > in a few seconds.
> > > > > > > > > > >> > >
> > > > > > > > > > >> > > Proposed solution
> > > > > > > > > > >> > > --------------------------
> > > > > > > > > > >> > > - We keep what is considered real integration
> tests
> > > the
> > > > > way
> > > > > > > they
> > > > > > > > > are
> > > > > > > > > > >> > right
> > > > > > > > > > >> > > now and keep using them
> > > > > > > > > > >> > > - We move what should be unit tests into simple
> > JUnit
> > > > > > classes,
> > > > > > > > and
> > > > > > > > > > we
> > > > > > > > > > >> do
> > > > > > > > > > >> > > not run them using java -jar ofbiz.jar --test, but
> > > > instead
> > > > > > run
> > > > > > > > > them
> > > > > > > > > > >> > > directly from the build.xml script, so these files
> > are
> > > > not
> > > > > > > > > > identified
> > > > > > > > > > >> in
> > > > > > > > > > >> > > any XML document, but are simply called
> immediately
> > > from
> > > > > the
> > > > > > > > build
> > > > > > > > > > >> > scripts.
> > > > > > > > > > >> > > - We clearly mark the difference between
> integration
> > > > tests
> > > > > > and
> > > > > > > > > unit
> > > > > > > > > > >> tests
> > > > > > > > > > >> > > (inside the source files or in the suite
> > > declarations).
> > > > > > > > > > >> > > - We change the run-tests target in build.xml to
> run
> > > > both
> > > > > > unit
> > > > > > > > > tests
> > > > > > > > > > >> and
> > > > > > > > > > >> > > integration tests.
> > > > > > > > > > >> > >
> > > > > > > > > > >> > > I intend to heavily refactor the framework and I
> > would
> > > > > feel
> > > > > > > > better
> > > > > > > > > > >> about
> > > > > > > > > > >> > > introducing this change while refactoring. What do
> > you
> > > > > guys
> > > > > > > > think?
> > > > > > > > > > >> Ideas?
> > > > > > > > > > >> > > Suggestions? Approvals and thumbs up?
> > > > > > > > > > >> > >
> > > > > > > > > > >> > > Regards,
> > > > > > > > > > >> > >
> > > > > > > > > > >> > > Taher Alkhateeb
> > > > > > > > > > >> >
> > > > > > > > > > >> >
> > > > > > > > > > >>
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: Proposal to modify the testing framework for OFBiz

[Scott Gray <sc...@hotwaxsystems.com>]

Yeah I'm sure unit tests probably worked well for the start component given
it is the lowest level component in the system and closest to a basic java
app.  I just think the value proposition might decrease the further up the
stack we move with them.  I'm not against unit tests when they prove
useful, but further up the stack I think we should prove the case for them
before doing much work to support the mocking that will be required to keep
them inline with 'best practices'.

In OFBiz "bad or unwanted" tends to come mostly in the way of increasing
complexity from features that require more effort to maintain than the
value they provide to the community.  I think there's a chance attempting
to mock service tests could fall into that bucket.  I could be wrong, but I
think it's worth looking into before we declare that unit tests are the
best form of testing for OFBiz.

Regards
Scott

On 19 July 2016 at 17:37, Taher Alkhateeb <sl...@gmail.com>
wrote:

> Hi Scott,
>
> Thank you for the feedback. To be focused exactly on integration vs unit, I
> already mentioned above that at least for me the main objective is to
> confidently and quickly run the tests in short bursts of red-green
> refactor. This allows me to refactor code without waiting in front of the
> screen in between test cycles thus giving me immediate feedback on any
> errors I made. Perhaps my intro was too long so this is the squeezed out
> version of it.
>
> I already had one round of testing in the start component which was much
> faster that way and had an immediate impact. Oh and by the way, you cannot
> test the start component with integration tests for example unless you do
> it from an external component which cannot access package protected items.
>
> This style of coding is applicable I think to any software project
> inclusive of OFBiz. Maybe in certain components more than others or certain
> areas more than others. But I can't see how it could be bad or unwanted.
>
> Taher Alkhateeb
>
> On Jul 19, 2016 8:18 AM, "Scott Gray" <sc...@hotwaxsystems.com>
> wrote:
>
> > Thanks Ron and Taher for your responses, I appreciate them but I don't
> hear
> > much in the conversation that speaks to the value of unit tests over
> > integration tests.  Most of the thoughts shared speaks more to the value
> of
> > tests rather than differences in the type of tests.
> >
> > Speed: At an application level we have ~685 tests that run in 35 seconds
> > (excluding build and data load).  Another point is that there isn't much
> > reason why tests can't be run in parallel rather than sequentially.
> > TDD: Integration tests in OFBiz are as simple if not simpler to write
> than
> > unit tests and just as useful for TDD.
> > I'm not sure if I missed any other points raised regarding integration
> vs.
> > unit tests?
> >
> > I'm not looking to start a big long debate on the topic, I just wanted to
> > speak out that someone out there (me) doesn't think unit tests are the
> best
> > solution for testing OFBiz applications.
> >
> > Regards
> > Scott
> >
> > On 19 July 2016 at 16:52, Taher Alkhateeb <sl...@gmail.com>
> > wrote:
> >
> > > Hi Scott,
> > >
> > > Thank you for your input. Your ideas are thought provoking and
> enriching
> > to
> > > the conversation.
> > >
> > > The way I look at it, the general rule is usually many unit tests, less
> > > integration tests, lesser functional tests. So we are not excluding any
> > > types of test, all of them are important in certain areas for certain
> > > purposes with certain quantity. Usually integration tests are less
> > because
> > > as you said they just grab more. I like the picture below as a general
> > > guideline
> > >
> > > http://i.stack.imgur.com/fjQvQ.png
> > >
> > > As you already mentioned unit tests are useful for the framework. I
> > > discovered errors in code that I wrote which I was very very careful
> > with.
> > > I immediately learned the lesson that humans are not designed to code,
> > and
> > > TDD gives you confidence as you build your code with those short 30-60
> > > second red-green refactors. I feel much much safer and more confident
> > > writing code that way, the test also documents how to use the api,
> > > refactoring and feature change becomes less terrifying. Also messy code
> > is
> > > usually not test friendly thus refactoring and unit tests go
> hand-in-hand
> > > for improving the code base.
> > >
> > > Also, I am sorry to say that the framework code is rather messy and
> > brittle
> > > in many places. I think you probably encountered this yourself. The
> same
> > is
> > > said for the applications. Now If we start refactoring without unit
> tests
> > > then we are back to scary business. So much can go wrong so fast and
> > break
> > > things you never expected to break. The framework with all applications
> > > require heavy refactoring of things like massive ugly methods,
> sandwiched
> > > logic, heavy shared mutable state, hidden dependencies, poor
> > > interfaces, and much more. Every time I touch something I get shocked
> at
> > > how bad it looks, spaghetti logic to no end. If you refactor with
> > > integration tests instead of unit tests then you will come down to a
> > > screeching halt as you wait between these test cycles. My computer
> > actually
> > > takes 10 minutes or more for a full clean load data and testing.
> > >
> > > Now talking about mocking you raise some interesting points, what can
> you
> > > mock vs utilizing integration tests? This is an excellent question that
> > > does not have a simple answer, and you mostly learn as you code,
> > difficult
> > > to envision without coding. However simple things like Java services
> can
> > be
> > > mocked with a standard mocking class that you use everywhere.
> > Transactions
> > > are difficult to mock and better left to integration tests I think.
> SECAs
> > > might be mocked by simply passing output to input as chaining methods.
> > Mind
> > > you I am not 100% sure of all of this, coding is the ultimate guide to
> > what
> > > can or cannot be done.
> > >
> > > So I am not suggesting unit tests as a best-practice (even though it
> is).
> > > Instead I suggest it as something that I and others did and got a huge
> > > psychological relief and confidence and comfort from. Swimming in code
> > > without tests is a terrifying business, made more terrifying if the
> code
> > is
> > > bad. Short bursts of red/green/red/green makes you feel good as you
> build
> > > up your logic. And I don't have an exact vision of how to do it,
> because
> > > the details always win.
> > >
> > > I look forward to hearing your thoughts and thank you for enriching
> this
> > > conversation.
> > >
> > > Taher Alkhateeb
> > >
> > > On Tuesday, 19 July 2016, Scott Gray <sc...@hotwaxsystems.com>
> > wrote:
> > >
> > > > I know I'm late to the party here, but I just want to say that I
> think
> > > > integration tests have far greater value to OFBiz than unit tests.
> > > Mostly
> > > > because we tend to have quite a low number of tests and integration
> > tests
> > > > give us much better coverage per line of test code and the tests are
> > much
> > > > closer to the real world scenarios the application might encounter.
> > > >
> > > > I don't really see how unit tests could be applied to non-framework
> > > testing
> > > > in a useful manner, could you expand on your vision in that regard?
> I
> > > mean
> > > > would we be testing something smaller than a service as the 'unit'?
> > What
> > > > would we mock? Would transaction management still be active? What
> > happens
> > > > when the service calls another service, I guess we mock the response
> > from
> > > > that service (how)?
> > > >
> > > > It just seems a very complicated method to achieve a less thorough
> but
> > > > albeit faster (maybe) test result.
> > > >
> > > > A build, data load and full test run takes 4m 9s on my laptop.
> > Excluding
> > > > build, data load and framework tests: the application level tests
> take
> > > 35s,
> > > > not very expensive IMO.  The data load time can be reduced to
> > practically
> > > > nothing by copying a clean slate database into runtime for each run.
> > > >
> > > > I'm mostly just suggesting we be wary of adding complicated testing
> > > > procedures in the hope of achieving some 'best practice' result which
> > in
> > > > reality will provide minimal benefits.
> > > >
> > > > Regards
> > > > Scott
> > > >
> > > > On 18 July 2016 at 18:57, Taher Alkhateeb <
> slidingfilaments@gmail.com
> > > > <javascript:;>>
> > > > wrote:
> > > >
> > > > > Hello Akash,
> > > > >
> > > > > Fantastic, I have a few unit tests almost done to be included in
> the
> > > > start
> > > > > component. I will create a new subtask under OFBIZ-1463 to commit
> the
> > > > tests
> > > > > so you can use them as a reference if you like to.
> > > > >
> > > > > I also recommend that you follow the same directory structure
> between
> > > the
> > > > > test code and production code. So for example:
> > > > >
> > > > > Production code:
> > > > framework/start/src/main/java/org/apache/ofbiz/base/start
> > > > > Test code:
> framework/start/src/test/java/org/apache/ofbiz/base/start
> > > > >
> > > > > The benefit of this hierarchy is that you can access non-public
> > > (package
> > > > > protected) methods for testing. This is in fact exactly what I
> needed
> > > to
> > > > be
> > > > > able to apply some of the tests.
> > > > >
> > > > > Cheers,
> > > > >
> > > > > Taher Alkhateeb
> > > > >
> > > > > On Mon, Jul 18, 2016 at 9:22 AM, Akash Jain <
> > > > akash.jain@hotwaxsystems.com <javascript:;>>
> > > > > wrote:
> > > > >
> > > > > > Thanks Taher for nice initiative!
> > > > > >
> > > > > > We are planning to written unit tests to all components under
> > > > OFBIZ-1463
> > > > > >
> > > > > > Thanks and Regards
> > > > > > --
> > > > > > Akash Jain
> > > > > >
> > > > > > On Mon, Jul 18, 2016 at 10:36 AM, Taher Alkhateeb <
> > > > > > slidingfilaments@gmail.com <javascript:;>> wrote:
> > > > > >
> > > > > > > Hello Everyone,
> > > > > > >
> > > > > > > In reference to this thread and the Jira OFBIZ-7254, I'm very
> > happy
> > > > to
> > > > > > > announce that OFBiz is now ready for applying unit tests with
> > only
> > > 8
> > > > > new
> > > > > > > lines of code in the build script (r1753143) :)
> > > > > > >
> > > > > > > I recommend we do the following moving forward:
> > > > > > >
> > > > > > > 1- Introduce unit tests as much as we can to all components
> > > > > > > 2- Migrate most of the integration tests we currently have to
> > unit
> > > > > tests
> > > > > > > (since they are designed to do very little integration).
> > > > > > >
> > > > > > > This is a great chance for us to practice real TDD in OFBiz
> > > > > > >
> > > > > > > Cheers,
> > > > > > >
> > > > > > > Taher Alkhateeb
> > > > > > >
> > > > > > > On Mon, Jun 13, 2016 at 7:43 AM, Pranay Pandey <
> > > > > > > pranay.pandey@hotwaxsystems.com <javascript:;>> wrote:
> > > > > > >
> > > > > > > > +1
> > > > > > > >
> > > > > > > > Best regards,
> > > > > > > >
> > > > > > > > Pranay Pandey
> > > > > > > > HotWax Systems
> > > > > > > > http://www.hotwaxsystems.com/
> > > > > > > >
> > > > > > > > On Fri, Jun 10, 2016 at 7:46 PM, Taher Alkhateeb <
> > > > > > > > slidingfilaments@gmail.com <javascript:;>
> > > > > > > > > wrote:
> > > > > > > >
> > > > > > > > > Hello Everyone,
> > > > > > > > >
> > > > > > > > > I was able to get a few tests running and this is very
> > doable.
> > > > But
> > > > > I
> > > > > > > > faced
> > > > > > > > > a big problem in designing the testing framework because of
> > > ANT.
> > > > > > > > >
> > > > > > > > > The problem
> > > > > > > > > ----------------
> > > > > > > > > The way the build scripts are designed in OFBiz are very
> > > > complex. A
> > > > > > > > master
> > > > > > > > > file calls other files which call other files. And in the
> > > middle
> > > > > you
> > > > > > > have
> > > > > > > > > external libraries (ant-contrib) and macros, and variables,
> > and
> > > > > class
> > > > > > > > path
> > > > > > > > > declarations, and and and ....
> > > > > > > > >
> > > > > > > > > I cannot declare the tests programmatically (with JUnit
> test
> > > > > suites)
> > > > > > > > > because this means lower level components would depend on
> > > higher
> > > > > > level
> > > > > > > > > components. So I have to do it in ANT, by navigating this
> > maze
> > > of
> > > > > > build
> > > > > > > > > scripts, and it was a headache for me just to read them,
> let
> > > > alone
> > > > > > > modify
> > > > > > > > > them to create a testing framework.
> > > > > > > > >
> > > > > > > > > Suggested Solution
> > > > > > > > > ------------------------
> > > > > > > > > I suggest to implement the testing framework in Gradle, and
> > > > simply
> > > > > > call
> > > > > > > > it
> > > > > > > > > from within ant. This is a middle solution that sustains
> ant
> > > for
> > > > > now,
> > > > > > > but
> > > > > > > > > can allow us to switch out later.
> > > > > > > > >
> > > > > > > > > This means I will just add one more file called
> build.gradle
> > in
> > > > the
> > > > > > top
> > > > > > > > > level directory, and figure out the business logic for
> > calling
> > > > the
> > > > > > test
> > > > > > > > > suites from that file
> > > > > > > > >
> > > > > > > > > I look forward to your feedback.
> > > > > > > > >
> > > > > > > > > Regards,
> > > > > > > > >
> > > > > > > > > Taher Alkhateeb
> > > > > > > > >
> > > > > > > > > On Wed, Jun 8, 2016 at 6:00 PM, Taher Alkhateeb <
> > > > > > > > > slidingfilaments@gmail.com <javascript:;>>
> > > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > > Hi Everyone,
> > > > > > > > > >
> > > > > > > > > > Thank you all for your support, JIRA created in
> > > > > > > > > > https://issues.apache.org/jira/browse/OFBIZ-7254
> > > > > > > > > >
> > > > > > > > > > I will start working on it and try to implement ASAP to
> get
> > > my
> > > > > > focus
> > > > > > > > back
> > > > > > > > > > on refactoring.
> > > > > > > > > >
> > > > > > > > > > Cheers!
> > > > > > > > > >
> > > > > > > > > > Taher Alkhateeb
> > > > > > > > > >
> > > > > > > > > > On Wed, Jun 8, 2016 at 4:58 PM, Deepak Dixit <
> > > > > > > > > > deepak.dixit@hotwaxsystems.com <javascript:;>> wrote:
> > > > > > > > > >
> > > > > > > > > >> +1
> > > > > > > > > >>
> > > > > > > > > >> Thanks & Regards
> > > > > > > > > >> --
> > > > > > > > > >> Deepak Dixit
> > > > > > > > > >> www.hotwaxsystems.com
> > > > > > > > > >>
> > > > > > > > > >> On Wed, Jun 8, 2016 at 7:12 PM, Mridul Pathak <
> > > > > > > > > >> mridul.pathak@hotwaxsystems.com <javascript:;>> wrote:
> > > > > > > > > >>
> > > > > > > > > >> > +1
> > > > > > > > > >> >
> > > > > > > > > >> > Makes perfect sense.
> > > > > > > > > >> >
> > > > > > > > > >> > --
> > > > > > > > > >> > Thanks & Regards,
> > > > > > > > > >> > Mridul Pathak
> > > > > > > > > >> > Senior Manager
> > > > > > > > > >> > HotWax Systems
> > > > > > > > > >> > http://www.hotwaxsystems.com
> > > > > > > > > >> >
> > > > > > > > > >> > > On Jun 8, 2016, at 2:41 PM, Taher Alkhateeb <
> > > > > > > > > >> slidingfilaments@gmail.com <javascript:;>>
> > > > > > > > > >> > wrote:
> > > > > > > > > >> > >
> > > > > > > > > >> > > Hello Everyone,
> > > > > > > > > >> > >
> > > > > > > > > >> > > After refactoring the start component and while
> > starting
> > > > on
> > > > > > the
> > > > > > > > base
> > > > > > > > > >> > > component I realized that the testing framework for
> > > OFBiz
> > > > is
> > > > > > not
> > > > > > > > > good.
> > > > > > > > > >> > You
> > > > > > > > > >> > > cannot do real test driven development or
> > > > red-green-refactor
> > > > > > > with
> > > > > > > > > the
> > > > > > > > > >> > > current setup, hence my proposal to change it. I
> > explain
> > > > > > below:
> > > > > > > > > >> > >
> > > > > > > > > >> > > Problem with current design
> > > > > > > > > >> > > ----------------------------------------
> > > > > > > > > >> > > - What we have right now is not unit tests, it's
> > really
> > > > > > > > integration
> > > > > > > > > >> > tests.
> > > > > > > > > >> > > You have to start the framework, the database, the
> > > service
> > > > > > > engine,
> > > > > > > > > the
> > > > > > > > > >> > > entity engine and pretty much everything.
> > > > > > > > > >> > > - Testing is very slow, because it's an integration
> > test
> > > > as
> > > > > I
> > > > > > > > > >> mentioned
> > > > > > > > > >> > > above. 10 minutes on a good computer!
> > > > > > > > > >> > > - There is zero mocking! We actually have to
> > --load-data
> > > > for
> > > > > > > > things
> > > > > > > > > to
> > > > > > > > > >> > > work. Again, these are integration tests.
> > > > > > > > > >> > > - Too complex: Integration tests by their nature are
> > > > > grabbing
> > > > > > > too
> > > > > > > > > >> much.
> > > > > > > > > >> > > Mind you, I am not objecting to integration tests (I
> > > > > actually
> > > > > > > like
> > > > > > > > > >> them)
> > > > > > > > > >> > > but I am objecting to not having real unit-tests.
> Unit
> > > > tests
> > > > > > > > should
> > > > > > > > > >> all
> > > > > > > > > >> > run
> > > > > > > > > >> > > in a few seconds.
> > > > > > > > > >> > >
> > > > > > > > > >> > > Proposed solution
> > > > > > > > > >> > > --------------------------
> > > > > > > > > >> > > - We keep what is considered real integration tests
> > the
> > > > way
> > > > > > they
> > > > > > > > are
> > > > > > > > > >> > right
> > > > > > > > > >> > > now and keep using them
> > > > > > > > > >> > > - We move what should be unit tests into simple
> JUnit
> > > > > classes,
> > > > > > > and
> > > > > > > > > we
> > > > > > > > > >> do
> > > > > > > > > >> > > not run them using java -jar ofbiz.jar --test, but
> > > instead
> > > > > run
> > > > > > > > them
> > > > > > > > > >> > > directly from the build.xml script, so these files
> are
> > > not
> > > > > > > > > identified
> > > > > > > > > >> in
> > > > > > > > > >> > > any XML document, but are simply called immediately
> > from
> > > > the
> > > > > > > build
> > > > > > > > > >> > scripts.
> > > > > > > > > >> > > - We clearly mark the difference between integration
> > > tests
> > > > > and
> > > > > > > > unit
> > > > > > > > > >> tests
> > > > > > > > > >> > > (inside the source files or in the suite
> > declarations).
> > > > > > > > > >> > > - We change the run-tests target in build.xml to run
> > > both
> > > > > unit
> > > > > > > > tests
> > > > > > > > > >> and
> > > > > > > > > >> > > integration tests.
> > > > > > > > > >> > >
> > > > > > > > > >> > > I intend to heavily refactor the framework and I
> would
> > > > feel
> > > > > > > better
> > > > > > > > > >> about
> > > > > > > > > >> > > introducing this change while refactoring. What do
> you
> > > > guys
> > > > > > > think?
> > > > > > > > > >> Ideas?
> > > > > > > > > >> > > Suggestions? Approvals and thumbs up?
> > > > > > > > > >> > >
> > > > > > > > > >> > > Regards,
> > > > > > > > > >> > >
> > > > > > > > > >> > > Taher Alkhateeb
> > > > > > > > > >> >
> > > > > > > > > >> >
> > > > > > > > > >>
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: Proposal to modify the testing framework for OFBiz

[Taher Alkhateeb <sl...@gmail.com>]

Hi Scott,

Thank you for the feedback. To be focused exactly on integration vs unit, I
already mentioned above that at least for me the main objective is to
confidently and quickly run the tests in short bursts of red-green
refactor. This allows me to refactor code without waiting in front of the
screen in between test cycles thus giving me immediate feedback on any
errors I made. Perhaps my intro was too long so this is the squeezed out
version of it.

I already had one round of testing in the start component which was much
faster that way and had an immediate impact. Oh and by the way, you cannot
test the start component with integration tests for example unless you do
it from an external component which cannot access package protected items.

This style of coding is applicable I think to any software project
inclusive of OFBiz. Maybe in certain components more than others or certain
areas more than others. But I can't see how it could be bad or unwanted.

Taher Alkhateeb

On Jul 19, 2016 8:18 AM, "Scott Gray" <sc...@hotwaxsystems.com> wrote:

> Thanks Ron and Taher for your responses, I appreciate them but I don't hear
> much in the conversation that speaks to the value of unit tests over
> integration tests.  Most of the thoughts shared speaks more to the value of
> tests rather than differences in the type of tests.
>
> Speed: At an application level we have ~685 tests that run in 35 seconds
> (excluding build and data load).  Another point is that there isn't much
> reason why tests can't be run in parallel rather than sequentially.
> TDD: Integration tests in OFBiz are as simple if not simpler to write than
> unit tests and just as useful for TDD.
> I'm not sure if I missed any other points raised regarding integration vs.
> unit tests?
>
> I'm not looking to start a big long debate on the topic, I just wanted to
> speak out that someone out there (me) doesn't think unit tests are the best
> solution for testing OFBiz applications.
>
> Regards
> Scott
>
> On 19 July 2016 at 16:52, Taher Alkhateeb <sl...@gmail.com>
> wrote:
>
> > Hi Scott,
> >
> > Thank you for your input. Your ideas are thought provoking and enriching
> to
> > the conversation.
> >
> > The way I look at it, the general rule is usually many unit tests, less
> > integration tests, lesser functional tests. So we are not excluding any
> > types of test, all of them are important in certain areas for certain
> > purposes with certain quantity. Usually integration tests are less
> because
> > as you said they just grab more. I like the picture below as a general
> > guideline
> >
> > http://i.stack.imgur.com/fjQvQ.png
> >
> > As you already mentioned unit tests are useful for the framework. I
> > discovered errors in code that I wrote which I was very very careful
> with.
> > I immediately learned the lesson that humans are not designed to code,
> and
> > TDD gives you confidence as you build your code with those short 30-60
> > second red-green refactors. I feel much much safer and more confident
> > writing code that way, the test also documents how to use the api,
> > refactoring and feature change becomes less terrifying. Also messy code
> is
> > usually not test friendly thus refactoring and unit tests go hand-in-hand
> > for improving the code base.
> >
> > Also, I am sorry to say that the framework code is rather messy and
> brittle
> > in many places. I think you probably encountered this yourself. The same
> is
> > said for the applications. Now If we start refactoring without unit tests
> > then we are back to scary business. So much can go wrong so fast and
> break
> > things you never expected to break. The framework with all applications
> > require heavy refactoring of things like massive ugly methods, sandwiched
> > logic, heavy shared mutable state, hidden dependencies, poor
> > interfaces, and much more. Every time I touch something I get shocked at
> > how bad it looks, spaghetti logic to no end. If you refactor with
> > integration tests instead of unit tests then you will come down to a
> > screeching halt as you wait between these test cycles. My computer
> actually
> > takes 10 minutes or more for a full clean load data and testing.
> >
> > Now talking about mocking you raise some interesting points, what can you
> > mock vs utilizing integration tests? This is an excellent question that
> > does not have a simple answer, and you mostly learn as you code,
> difficult
> > to envision without coding. However simple things like Java services can
> be
> > mocked with a standard mocking class that you use everywhere.
> Transactions
> > are difficult to mock and better left to integration tests I think. SECAs
> > might be mocked by simply passing output to input as chaining methods.
> Mind
> > you I am not 100% sure of all of this, coding is the ultimate guide to
> what
> > can or cannot be done.
> >
> > So I am not suggesting unit tests as a best-practice (even though it is).
> > Instead I suggest it as something that I and others did and got a huge
> > psychological relief and confidence and comfort from. Swimming in code
> > without tests is a terrifying business, made more terrifying if the code
> is
> > bad. Short bursts of red/green/red/green makes you feel good as you build
> > up your logic. And I don't have an exact vision of how to do it, because
> > the details always win.
> >
> > I look forward to hearing your thoughts and thank you for enriching this
> > conversation.
> >
> > Taher Alkhateeb
> >
> > On Tuesday, 19 July 2016, Scott Gray <sc...@hotwaxsystems.com>
> wrote:
> >
> > > I know I'm late to the party here, but I just want to say that I think
> > > integration tests have far greater value to OFBiz than unit tests.
> > Mostly
> > > because we tend to have quite a low number of tests and integration
> tests
> > > give us much better coverage per line of test code and the tests are
> much
> > > closer to the real world scenarios the application might encounter.
> > >
> > > I don't really see how unit tests could be applied to non-framework
> > testing
> > > in a useful manner, could you expand on your vision in that regard?  I
> > mean
> > > would we be testing something smaller than a service as the 'unit'?
> What
> > > would we mock? Would transaction management still be active? What
> happens
> > > when the service calls another service, I guess we mock the response
> from
> > > that service (how)?
> > >
> > > It just seems a very complicated method to achieve a less thorough but
> > > albeit faster (maybe) test result.
> > >
> > > A build, data load and full test run takes 4m 9s on my laptop.
> Excluding
> > > build, data load and framework tests: the application level tests take
> > 35s,
> > > not very expensive IMO.  The data load time can be reduced to
> practically
> > > nothing by copying a clean slate database into runtime for each run.
> > >
> > > I'm mostly just suggesting we be wary of adding complicated testing
> > > procedures in the hope of achieving some 'best practice' result which
> in
> > > reality will provide minimal benefits.
> > >
> > > Regards
> > > Scott
> > >
> > > On 18 July 2016 at 18:57, Taher Alkhateeb <slidingfilaments@gmail.com
> > > <javascript:;>>
> > > wrote:
> > >
> > > > Hello Akash,
> > > >
> > > > Fantastic, I have a few unit tests almost done to be included in the
> > > start
> > > > component. I will create a new subtask under OFBIZ-1463 to commit the
> > > tests
> > > > so you can use them as a reference if you like to.
> > > >
> > > > I also recommend that you follow the same directory structure between
> > the
> > > > test code and production code. So for example:
> > > >
> > > > Production code:
> > > framework/start/src/main/java/org/apache/ofbiz/base/start
> > > > Test code: framework/start/src/test/java/org/apache/ofbiz/base/start
> > > >
> > > > The benefit of this hierarchy is that you can access non-public
> > (package
> > > > protected) methods for testing. This is in fact exactly what I needed
> > to
> > > be
> > > > able to apply some of the tests.
> > > >
> > > > Cheers,
> > > >
> > > > Taher Alkhateeb
> > > >
> > > > On Mon, Jul 18, 2016 at 9:22 AM, Akash Jain <
> > > akash.jain@hotwaxsystems.com <javascript:;>>
> > > > wrote:
> > > >
> > > > > Thanks Taher for nice initiative!
> > > > >
> > > > > We are planning to written unit tests to all components under
> > > OFBIZ-1463
> > > > >
> > > > > Thanks and Regards
> > > > > --
> > > > > Akash Jain
> > > > >
> > > > > On Mon, Jul 18, 2016 at 10:36 AM, Taher Alkhateeb <
> > > > > slidingfilaments@gmail.com <javascript:;>> wrote:
> > > > >
> > > > > > Hello Everyone,
> > > > > >
> > > > > > In reference to this thread and the Jira OFBIZ-7254, I'm very
> happy
> > > to
> > > > > > announce that OFBiz is now ready for applying unit tests with
> only
> > 8
> > > > new
> > > > > > lines of code in the build script (r1753143) :)
> > > > > >
> > > > > > I recommend we do the following moving forward:
> > > > > >
> > > > > > 1- Introduce unit tests as much as we can to all components
> > > > > > 2- Migrate most of the integration tests we currently have to
> unit
> > > > tests
> > > > > > (since they are designed to do very little integration).
> > > > > >
> > > > > > This is a great chance for us to practice real TDD in OFBiz
> > > > > >
> > > > > > Cheers,
> > > > > >
> > > > > > Taher Alkhateeb
> > > > > >
> > > > > > On Mon, Jun 13, 2016 at 7:43 AM, Pranay Pandey <
> > > > > > pranay.pandey@hotwaxsystems.com <javascript:;>> wrote:
> > > > > >
> > > > > > > +1
> > > > > > >
> > > > > > > Best regards,
> > > > > > >
> > > > > > > Pranay Pandey
> > > > > > > HotWax Systems
> > > > > > > http://www.hotwaxsystems.com/
> > > > > > >
> > > > > > > On Fri, Jun 10, 2016 at 7:46 PM, Taher Alkhateeb <
> > > > > > > slidingfilaments@gmail.com <javascript:;>
> > > > > > > > wrote:
> > > > > > >
> > > > > > > > Hello Everyone,
> > > > > > > >
> > > > > > > > I was able to get a few tests running and this is very
> doable.
> > > But
> > > > I
> > > > > > > faced
> > > > > > > > a big problem in designing the testing framework because of
> > ANT.
> > > > > > > >
> > > > > > > > The problem
> > > > > > > > ----------------
> > > > > > > > The way the build scripts are designed in OFBiz are very
> > > complex. A
> > > > > > > master
> > > > > > > > file calls other files which call other files. And in the
> > middle
> > > > you
> > > > > > have
> > > > > > > > external libraries (ant-contrib) and macros, and variables,
> and
> > > > class
> > > > > > > path
> > > > > > > > declarations, and and and ....
> > > > > > > >
> > > > > > > > I cannot declare the tests programmatically (with JUnit test
> > > > suites)
> > > > > > > > because this means lower level components would depend on
> > higher
> > > > > level
> > > > > > > > components. So I have to do it in ANT, by navigating this
> maze
> > of
> > > > > build
> > > > > > > > scripts, and it was a headache for me just to read them, let
> > > alone
> > > > > > modify
> > > > > > > > them to create a testing framework.
> > > > > > > >
> > > > > > > > Suggested Solution
> > > > > > > > ------------------------
> > > > > > > > I suggest to implement the testing framework in Gradle, and
> > > simply
> > > > > call
> > > > > > > it
> > > > > > > > from within ant. This is a middle solution that sustains ant
> > for
> > > > now,
> > > > > > but
> > > > > > > > can allow us to switch out later.
> > > > > > > >
> > > > > > > > This means I will just add one more file called build.gradle
> in
> > > the
> > > > > top
> > > > > > > > level directory, and figure out the business logic for
> calling
> > > the
> > > > > test
> > > > > > > > suites from that file
> > > > > > > >
> > > > > > > > I look forward to your feedback.
> > > > > > > >
> > > > > > > > Regards,
> > > > > > > >
> > > > > > > > Taher Alkhateeb
> > > > > > > >
> > > > > > > > On Wed, Jun 8, 2016 at 6:00 PM, Taher Alkhateeb <
> > > > > > > > slidingfilaments@gmail.com <javascript:;>>
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > > > Hi Everyone,
> > > > > > > > >
> > > > > > > > > Thank you all for your support, JIRA created in
> > > > > > > > > https://issues.apache.org/jira/browse/OFBIZ-7254
> > > > > > > > >
> > > > > > > > > I will start working on it and try to implement ASAP to get
> > my
> > > > > focus
> > > > > > > back
> > > > > > > > > on refactoring.
> > > > > > > > >
> > > > > > > > > Cheers!
> > > > > > > > >
> > > > > > > > > Taher Alkhateeb
> > > > > > > > >
> > > > > > > > > On Wed, Jun 8, 2016 at 4:58 PM, Deepak Dixit <
> > > > > > > > > deepak.dixit@hotwaxsystems.com <javascript:;>> wrote:
> > > > > > > > >
> > > > > > > > >> +1
> > > > > > > > >>
> > > > > > > > >> Thanks & Regards
> > > > > > > > >> --
> > > > > > > > >> Deepak Dixit
> > > > > > > > >> www.hotwaxsystems.com
> > > > > > > > >>
> > > > > > > > >> On Wed, Jun 8, 2016 at 7:12 PM, Mridul Pathak <
> > > > > > > > >> mridul.pathak@hotwaxsystems.com <javascript:;>> wrote:
> > > > > > > > >>
> > > > > > > > >> > +1
> > > > > > > > >> >
> > > > > > > > >> > Makes perfect sense.
> > > > > > > > >> >
> > > > > > > > >> > --
> > > > > > > > >> > Thanks & Regards,
> > > > > > > > >> > Mridul Pathak
> > > > > > > > >> > Senior Manager
> > > > > > > > >> > HotWax Systems
> > > > > > > > >> > http://www.hotwaxsystems.com
> > > > > > > > >> >
> > > > > > > > >> > > On Jun 8, 2016, at 2:41 PM, Taher Alkhateeb <
> > > > > > > > >> slidingfilaments@gmail.com <javascript:;>>
> > > > > > > > >> > wrote:
> > > > > > > > >> > >
> > > > > > > > >> > > Hello Everyone,
> > > > > > > > >> > >
> > > > > > > > >> > > After refactoring the start component and while
> starting
> > > on
> > > > > the
> > > > > > > base
> > > > > > > > >> > > component I realized that the testing framework for
> > OFBiz
> > > is
> > > > > not
> > > > > > > > good.
> > > > > > > > >> > You
> > > > > > > > >> > > cannot do real test driven development or
> > > red-green-refactor
> > > > > > with
> > > > > > > > the
> > > > > > > > >> > > current setup, hence my proposal to change it. I
> explain
> > > > > below:
> > > > > > > > >> > >
> > > > > > > > >> > > Problem with current design
> > > > > > > > >> > > ----------------------------------------
> > > > > > > > >> > > - What we have right now is not unit tests, it's
> really
> > > > > > > integration
> > > > > > > > >> > tests.
> > > > > > > > >> > > You have to start the framework, the database, the
> > service
> > > > > > engine,
> > > > > > > > the
> > > > > > > > >> > > entity engine and pretty much everything.
> > > > > > > > >> > > - Testing is very slow, because it's an integration
> test
> > > as
> > > > I
> > > > > > > > >> mentioned
> > > > > > > > >> > > above. 10 minutes on a good computer!
> > > > > > > > >> > > - There is zero mocking! We actually have to
> --load-data
> > > for
> > > > > > > things
> > > > > > > > to
> > > > > > > > >> > > work. Again, these are integration tests.
> > > > > > > > >> > > - Too complex: Integration tests by their nature are
> > > > grabbing
> > > > > > too
> > > > > > > > >> much.
> > > > > > > > >> > > Mind you, I am not objecting to integration tests (I
> > > > actually
> > > > > > like
> > > > > > > > >> them)
> > > > > > > > >> > > but I am objecting to not having real unit-tests. Unit
> > > tests
> > > > > > > should
> > > > > > > > >> all
> > > > > > > > >> > run
> > > > > > > > >> > > in a few seconds.
> > > > > > > > >> > >
> > > > > > > > >> > > Proposed solution
> > > > > > > > >> > > --------------------------
> > > > > > > > >> > > - We keep what is considered real integration tests
> the
> > > way
> > > > > they
> > > > > > > are
> > > > > > > > >> > right
> > > > > > > > >> > > now and keep using them
> > > > > > > > >> > > - We move what should be unit tests into simple JUnit
> > > > classes,
> > > > > > and
> > > > > > > > we
> > > > > > > > >> do
> > > > > > > > >> > > not run them using java -jar ofbiz.jar --test, but
> > instead
> > > > run
> > > > > > > them
> > > > > > > > >> > > directly from the build.xml script, so these files are
> > not
> > > > > > > > identified
> > > > > > > > >> in
> > > > > > > > >> > > any XML document, but are simply called immediately
> from
> > > the
> > > > > > build
> > > > > > > > >> > scripts.
> > > > > > > > >> > > - We clearly mark the difference between integration
> > tests
> > > > and
> > > > > > > unit
> > > > > > > > >> tests
> > > > > > > > >> > > (inside the source files or in the suite
> declarations).
> > > > > > > > >> > > - We change the run-tests target in build.xml to run
> > both
> > > > unit
> > > > > > > tests
> > > > > > > > >> and
> > > > > > > > >> > > integration tests.
> > > > > > > > >> > >
> > > > > > > > >> > > I intend to heavily refactor the framework and I would
> > > feel
> > > > > > better
> > > > > > > > >> about
> > > > > > > > >> > > introducing this change while refactoring. What do you
> > > guys
> > > > > > think?
> > > > > > > > >> Ideas?
> > > > > > > > >> > > Suggestions? Approvals and thumbs up?
> > > > > > > > >> > >
> > > > > > > > >> > > Regards,
> > > > > > > > >> > >
> > > > > > > > >> > > Taher Alkhateeb
> > > > > > > > >> >
> > > > > > > > >> >
> > > > > > > > >>
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: Proposal to modify the testing framework for OFBiz

[Scott Gray <sc...@hotwaxsystems.com>]

Thanks Ron and Taher for your responses, I appreciate them but I don't hear
much in the conversation that speaks to the value of unit tests over
integration tests.  Most of the thoughts shared speaks more to the value of
tests rather than differences in the type of tests.

Speed: At an application level we have ~685 tests that run in 35 seconds
(excluding build and data load).  Another point is that there isn't much
reason why tests can't be run in parallel rather than sequentially.
TDD: Integration tests in OFBiz are as simple if not simpler to write than
unit tests and just as useful for TDD.
I'm not sure if I missed any other points raised regarding integration vs.
unit tests?

I'm not looking to start a big long debate on the topic, I just wanted to
speak out that someone out there (me) doesn't think unit tests are the best
solution for testing OFBiz applications.

Regards
Scott

On 19 July 2016 at 16:52, Taher Alkhateeb <sl...@gmail.com>
wrote:

> Hi Scott,
>
> Thank you for your input. Your ideas are thought provoking and enriching to
> the conversation.
>
> The way I look at it, the general rule is usually many unit tests, less
> integration tests, lesser functional tests. So we are not excluding any
> types of test, all of them are important in certain areas for certain
> purposes with certain quantity. Usually integration tests are less because
> as you said they just grab more. I like the picture below as a general
> guideline
>
> http://i.stack.imgur.com/fjQvQ.png
>
> As you already mentioned unit tests are useful for the framework. I
> discovered errors in code that I wrote which I was very very careful with.
> I immediately learned the lesson that humans are not designed to code, and
> TDD gives you confidence as you build your code with those short 30-60
> second red-green refactors. I feel much much safer and more confident
> writing code that way, the test also documents how to use the api,
> refactoring and feature change becomes less terrifying. Also messy code is
> usually not test friendly thus refactoring and unit tests go hand-in-hand
> for improving the code base.
>
> Also, I am sorry to say that the framework code is rather messy and brittle
> in many places. I think you probably encountered this yourself. The same is
> said for the applications. Now If we start refactoring without unit tests
> then we are back to scary business. So much can go wrong so fast and break
> things you never expected to break. The framework with all applications
> require heavy refactoring of things like massive ugly methods, sandwiched
> logic, heavy shared mutable state, hidden dependencies, poor
> interfaces, and much more. Every time I touch something I get shocked at
> how bad it looks, spaghetti logic to no end. If you refactor with
> integration tests instead of unit tests then you will come down to a
> screeching halt as you wait between these test cycles. My computer actually
> takes 10 minutes or more for a full clean load data and testing.
>
> Now talking about mocking you raise some interesting points, what can you
> mock vs utilizing integration tests? This is an excellent question that
> does not have a simple answer, and you mostly learn as you code, difficult
> to envision without coding. However simple things like Java services can be
> mocked with a standard mocking class that you use everywhere. Transactions
> are difficult to mock and better left to integration tests I think. SECAs
> might be mocked by simply passing output to input as chaining methods. Mind
> you I am not 100% sure of all of this, coding is the ultimate guide to what
> can or cannot be done.
>
> So I am not suggesting unit tests as a best-practice (even though it is).
> Instead I suggest it as something that I and others did and got a huge
> psychological relief and confidence and comfort from. Swimming in code
> without tests is a terrifying business, made more terrifying if the code is
> bad. Short bursts of red/green/red/green makes you feel good as you build
> up your logic. And I don't have an exact vision of how to do it, because
> the details always win.
>
> I look forward to hearing your thoughts and thank you for enriching this
> conversation.
>
> Taher Alkhateeb
>
> On Tuesday, 19 July 2016, Scott Gray <sc...@hotwaxsystems.com> wrote:
>
> > I know I'm late to the party here, but I just want to say that I think
> > integration tests have far greater value to OFBiz than unit tests.
> Mostly
> > because we tend to have quite a low number of tests and integration tests
> > give us much better coverage per line of test code and the tests are much
> > closer to the real world scenarios the application might encounter.
> >
> > I don't really see how unit tests could be applied to non-framework
> testing
> > in a useful manner, could you expand on your vision in that regard?  I
> mean
> > would we be testing something smaller than a service as the 'unit'?  What
> > would we mock? Would transaction management still be active? What happens
> > when the service calls another service, I guess we mock the response from
> > that service (how)?
> >
> > It just seems a very complicated method to achieve a less thorough but
> > albeit faster (maybe) test result.
> >
> > A build, data load and full test run takes 4m 9s on my laptop.  Excluding
> > build, data load and framework tests: the application level tests take
> 35s,
> > not very expensive IMO.  The data load time can be reduced to practically
> > nothing by copying a clean slate database into runtime for each run.
> >
> > I'm mostly just suggesting we be wary of adding complicated testing
> > procedures in the hope of achieving some 'best practice' result which in
> > reality will provide minimal benefits.
> >
> > Regards
> > Scott
> >
> > On 18 July 2016 at 18:57, Taher Alkhateeb <slidingfilaments@gmail.com
> > <javascript:;>>
> > wrote:
> >
> > > Hello Akash,
> > >
> > > Fantastic, I have a few unit tests almost done to be included in the
> > start
> > > component. I will create a new subtask under OFBIZ-1463 to commit the
> > tests
> > > so you can use them as a reference if you like to.
> > >
> > > I also recommend that you follow the same directory structure between
> the
> > > test code and production code. So for example:
> > >
> > > Production code:
> > framework/start/src/main/java/org/apache/ofbiz/base/start
> > > Test code: framework/start/src/test/java/org/apache/ofbiz/base/start
> > >
> > > The benefit of this hierarchy is that you can access non-public
> (package
> > > protected) methods for testing. This is in fact exactly what I needed
> to
> > be
> > > able to apply some of the tests.
> > >
> > > Cheers,
> > >
> > > Taher Alkhateeb
> > >
> > > On Mon, Jul 18, 2016 at 9:22 AM, Akash Jain <
> > akash.jain@hotwaxsystems.com <javascript:;>>
> > > wrote:
> > >
> > > > Thanks Taher for nice initiative!
> > > >
> > > > We are planning to written unit tests to all components under
> > OFBIZ-1463
> > > >
> > > > Thanks and Regards
> > > > --
> > > > Akash Jain
> > > >
> > > > On Mon, Jul 18, 2016 at 10:36 AM, Taher Alkhateeb <
> > > > slidingfilaments@gmail.com <javascript:;>> wrote:
> > > >
> > > > > Hello Everyone,
> > > > >
> > > > > In reference to this thread and the Jira OFBIZ-7254, I'm very happy
> > to
> > > > > announce that OFBiz is now ready for applying unit tests with only
> 8
> > > new
> > > > > lines of code in the build script (r1753143) :)
> > > > >
> > > > > I recommend we do the following moving forward:
> > > > >
> > > > > 1- Introduce unit tests as much as we can to all components
> > > > > 2- Migrate most of the integration tests we currently have to unit
> > > tests
> > > > > (since they are designed to do very little integration).
> > > > >
> > > > > This is a great chance for us to practice real TDD in OFBiz
> > > > >
> > > > > Cheers,
> > > > >
> > > > > Taher Alkhateeb
> > > > >
> > > > > On Mon, Jun 13, 2016 at 7:43 AM, Pranay Pandey <
> > > > > pranay.pandey@hotwaxsystems.com <javascript:;>> wrote:
> > > > >
> > > > > > +1
> > > > > >
> > > > > > Best regards,
> > > > > >
> > > > > > Pranay Pandey
> > > > > > HotWax Systems
> > > > > > http://www.hotwaxsystems.com/
> > > > > >
> > > > > > On Fri, Jun 10, 2016 at 7:46 PM, Taher Alkhateeb <
> > > > > > slidingfilaments@gmail.com <javascript:;>
> > > > > > > wrote:
> > > > > >
> > > > > > > Hello Everyone,
> > > > > > >
> > > > > > > I was able to get a few tests running and this is very doable.
> > But
> > > I
> > > > > > faced
> > > > > > > a big problem in designing the testing framework because of
> ANT.
> > > > > > >
> > > > > > > The problem
> > > > > > > ----------------
> > > > > > > The way the build scripts are designed in OFBiz are very
> > complex. A
> > > > > > master
> > > > > > > file calls other files which call other files. And in the
> middle
> > > you
> > > > > have
> > > > > > > external libraries (ant-contrib) and macros, and variables, and
> > > class
> > > > > > path
> > > > > > > declarations, and and and ....
> > > > > > >
> > > > > > > I cannot declare the tests programmatically (with JUnit test
> > > suites)
> > > > > > > because this means lower level components would depend on
> higher
> > > > level
> > > > > > > components. So I have to do it in ANT, by navigating this maze
> of
> > > > build
> > > > > > > scripts, and it was a headache for me just to read them, let
> > alone
> > > > > modify
> > > > > > > them to create a testing framework.
> > > > > > >
> > > > > > > Suggested Solution
> > > > > > > ------------------------
> > > > > > > I suggest to implement the testing framework in Gradle, and
> > simply
> > > > call
> > > > > > it
> > > > > > > from within ant. This is a middle solution that sustains ant
> for
> > > now,
> > > > > but
> > > > > > > can allow us to switch out later.
> > > > > > >
> > > > > > > This means I will just add one more file called build.gradle in
> > the
> > > > top
> > > > > > > level directory, and figure out the business logic for calling
> > the
> > > > test
> > > > > > > suites from that file
> > > > > > >
> > > > > > > I look forward to your feedback.
> > > > > > >
> > > > > > > Regards,
> > > > > > >
> > > > > > > Taher Alkhateeb
> > > > > > >
> > > > > > > On Wed, Jun 8, 2016 at 6:00 PM, Taher Alkhateeb <
> > > > > > > slidingfilaments@gmail.com <javascript:;>>
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Hi Everyone,
> > > > > > > >
> > > > > > > > Thank you all for your support, JIRA created in
> > > > > > > > https://issues.apache.org/jira/browse/OFBIZ-7254
> > > > > > > >
> > > > > > > > I will start working on it and try to implement ASAP to get
> my
> > > > focus
> > > > > > back
> > > > > > > > on refactoring.
> > > > > > > >
> > > > > > > > Cheers!
> > > > > > > >
> > > > > > > > Taher Alkhateeb
> > > > > > > >
> > > > > > > > On Wed, Jun 8, 2016 at 4:58 PM, Deepak Dixit <
> > > > > > > > deepak.dixit@hotwaxsystems.com <javascript:;>> wrote:
> > > > > > > >
> > > > > > > >> +1
> > > > > > > >>
> > > > > > > >> Thanks & Regards
> > > > > > > >> --
> > > > > > > >> Deepak Dixit
> > > > > > > >> www.hotwaxsystems.com
> > > > > > > >>
> > > > > > > >> On Wed, Jun 8, 2016 at 7:12 PM, Mridul Pathak <
> > > > > > > >> mridul.pathak@hotwaxsystems.com <javascript:;>> wrote:
> > > > > > > >>
> > > > > > > >> > +1
> > > > > > > >> >
> > > > > > > >> > Makes perfect sense.
> > > > > > > >> >
> > > > > > > >> > --
> > > > > > > >> > Thanks & Regards,
> > > > > > > >> > Mridul Pathak
> > > > > > > >> > Senior Manager
> > > > > > > >> > HotWax Systems
> > > > > > > >> > http://www.hotwaxsystems.com
> > > > > > > >> >
> > > > > > > >> > > On Jun 8, 2016, at 2:41 PM, Taher Alkhateeb <
> > > > > > > >> slidingfilaments@gmail.com <javascript:;>>
> > > > > > > >> > wrote:
> > > > > > > >> > >
> > > > > > > >> > > Hello Everyone,
> > > > > > > >> > >
> > > > > > > >> > > After refactoring the start component and while starting
> > on
> > > > the
> > > > > > base
> > > > > > > >> > > component I realized that the testing framework for
> OFBiz
> > is
> > > > not
> > > > > > > good.
> > > > > > > >> > You
> > > > > > > >> > > cannot do real test driven development or
> > red-green-refactor
> > > > > with
> > > > > > > the
> > > > > > > >> > > current setup, hence my proposal to change it. I explain
> > > > below:
> > > > > > > >> > >
> > > > > > > >> > > Problem with current design
> > > > > > > >> > > ----------------------------------------
> > > > > > > >> > > - What we have right now is not unit tests, it's really
> > > > > > integration
> > > > > > > >> > tests.
> > > > > > > >> > > You have to start the framework, the database, the
> service
> > > > > engine,
> > > > > > > the
> > > > > > > >> > > entity engine and pretty much everything.
> > > > > > > >> > > - Testing is very slow, because it's an integration test
> > as
> > > I
> > > > > > > >> mentioned
> > > > > > > >> > > above. 10 minutes on a good computer!
> > > > > > > >> > > - There is zero mocking! We actually have to --load-data
> > for
> > > > > > things
> > > > > > > to
> > > > > > > >> > > work. Again, these are integration tests.
> > > > > > > >> > > - Too complex: Integration tests by their nature are
> > > grabbing
> > > > > too
> > > > > > > >> much.
> > > > > > > >> > > Mind you, I am not objecting to integration tests (I
> > > actually
> > > > > like
> > > > > > > >> them)
> > > > > > > >> > > but I am objecting to not having real unit-tests. Unit
> > tests
> > > > > > should
> > > > > > > >> all
> > > > > > > >> > run
> > > > > > > >> > > in a few seconds.
> > > > > > > >> > >
> > > > > > > >> > > Proposed solution
> > > > > > > >> > > --------------------------
> > > > > > > >> > > - We keep what is considered real integration tests the
> > way
> > > > they
> > > > > > are
> > > > > > > >> > right
> > > > > > > >> > > now and keep using them
> > > > > > > >> > > - We move what should be unit tests into simple JUnit
> > > classes,
> > > > > and
> > > > > > > we
> > > > > > > >> do
> > > > > > > >> > > not run them using java -jar ofbiz.jar --test, but
> instead
> > > run
> > > > > > them
> > > > > > > >> > > directly from the build.xml script, so these files are
> not
> > > > > > > identified
> > > > > > > >> in
> > > > > > > >> > > any XML document, but are simply called immediately from
> > the
> > > > > build
> > > > > > > >> > scripts.
> > > > > > > >> > > - We clearly mark the difference between integration
> tests
> > > and
> > > > > > unit
> > > > > > > >> tests
> > > > > > > >> > > (inside the source files or in the suite declarations).
> > > > > > > >> > > - We change the run-tests target in build.xml to run
> both
> > > unit
> > > > > > tests
> > > > > > > >> and
> > > > > > > >> > > integration tests.
> > > > > > > >> > >
> > > > > > > >> > > I intend to heavily refactor the framework and I would
> > feel
> > > > > better
> > > > > > > >> about
> > > > > > > >> > > introducing this change while refactoring. What do you
> > guys
> > > > > think?
> > > > > > > >> Ideas?
> > > > > > > >> > > Suggestions? Approvals and thumbs up?
> > > > > > > >> > >
> > > > > > > >> > > Regards,
> > > > > > > >> > >
> > > > > > > >> > > Taher Alkhateeb
> > > > > > > >> >
> > > > > > > >> >
> > > > > > > >>
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: Proposal to modify the testing framework for OFBiz

[Taher Alkhateeb <sl...@gmail.com>]

Hi Scott,

Thank you for your input. Your ideas are thought provoking and enriching to
the conversation.

The way I look at it, the general rule is usually many unit tests, less
integration tests, lesser functional tests. So we are not excluding any
types of test, all of them are important in certain areas for certain
purposes with certain quantity. Usually integration tests are less because
as you said they just grab more. I like the picture below as a general
guideline

http://i.stack.imgur.com/fjQvQ.png

As you already mentioned unit tests are useful for the framework. I
discovered errors in code that I wrote which I was very very careful with.
I immediately learned the lesson that humans are not designed to code, and
TDD gives you confidence as you build your code with those short 30-60
second red-green refactors. I feel much much safer and more confident
writing code that way, the test also documents how to use the api,
refactoring and feature change becomes less terrifying. Also messy code is
usually not test friendly thus refactoring and unit tests go hand-in-hand
for improving the code base.

Also, I am sorry to say that the framework code is rather messy and brittle
in many places. I think you probably encountered this yourself. The same is
said for the applications. Now If we start refactoring without unit tests
then we are back to scary business. So much can go wrong so fast and break
things you never expected to break. The framework with all applications
require heavy refactoring of things like massive ugly methods, sandwiched
logic, heavy shared mutable state, hidden dependencies, poor
interfaces, and much more. Every time I touch something I get shocked at
how bad it looks, spaghetti logic to no end. If you refactor with
integration tests instead of unit tests then you will come down to a
screeching halt as you wait between these test cycles. My computer actually
takes 10 minutes or more for a full clean load data and testing.

Now talking about mocking you raise some interesting points, what can you
mock vs utilizing integration tests? This is an excellent question that
does not have a simple answer, and you mostly learn as you code, difficult
to envision without coding. However simple things like Java services can be
mocked with a standard mocking class that you use everywhere. Transactions
are difficult to mock and better left to integration tests I think. SECAs
might be mocked by simply passing output to input as chaining methods. Mind
you I am not 100% sure of all of this, coding is the ultimate guide to what
can or cannot be done.

So I am not suggesting unit tests as a best-practice (even though it is).
Instead I suggest it as something that I and others did and got a huge
psychological relief and confidence and comfort from. Swimming in code
without tests is a terrifying business, made more terrifying if the code is
bad. Short bursts of red/green/red/green makes you feel good as you build
up your logic. And I don't have an exact vision of how to do it, because
the details always win.

I look forward to hearing your thoughts and thank you for enriching this
conversation.

Taher Alkhateeb

On Tuesday, 19 July 2016, Scott Gray <sc...@hotwaxsystems.com> wrote:

> I know I'm late to the party here, but I just want to say that I think
> integration tests have far greater value to OFBiz than unit tests.  Mostly
> because we tend to have quite a low number of tests and integration tests
> give us much better coverage per line of test code and the tests are much
> closer to the real world scenarios the application might encounter.
>
> I don't really see how unit tests could be applied to non-framework testing
> in a useful manner, could you expand on your vision in that regard?  I mean
> would we be testing something smaller than a service as the 'unit'?  What
> would we mock? Would transaction management still be active? What happens
> when the service calls another service, I guess we mock the response from
> that service (how)?
>
> It just seems a very complicated method to achieve a less thorough but
> albeit faster (maybe) test result.
>
> A build, data load and full test run takes 4m 9s on my laptop.  Excluding
> build, data load and framework tests: the application level tests take 35s,
> not very expensive IMO.  The data load time can be reduced to practically
> nothing by copying a clean slate database into runtime for each run.
>
> I'm mostly just suggesting we be wary of adding complicated testing
> procedures in the hope of achieving some 'best practice' result which in
> reality will provide minimal benefits.
>
> Regards
> Scott
>
> On 18 July 2016 at 18:57, Taher Alkhateeb <slidingfilaments@gmail.com
> <javascript:;>>
> wrote:
>
> > Hello Akash,
> >
> > Fantastic, I have a few unit tests almost done to be included in the
> start
> > component. I will create a new subtask under OFBIZ-1463 to commit the
> tests
> > so you can use them as a reference if you like to.
> >
> > I also recommend that you follow the same directory structure between the
> > test code and production code. So for example:
> >
> > Production code:
> framework/start/src/main/java/org/apache/ofbiz/base/start
> > Test code: framework/start/src/test/java/org/apache/ofbiz/base/start
> >
> > The benefit of this hierarchy is that you can access non-public (package
> > protected) methods for testing. This is in fact exactly what I needed to
> be
> > able to apply some of the tests.
> >
> > Cheers,
> >
> > Taher Alkhateeb
> >
> > On Mon, Jul 18, 2016 at 9:22 AM, Akash Jain <
> akash.jain@hotwaxsystems.com <javascript:;>>
> > wrote:
> >
> > > Thanks Taher for nice initiative!
> > >
> > > We are planning to written unit tests to all components under
> OFBIZ-1463
> > >
> > > Thanks and Regards
> > > --
> > > Akash Jain
> > >
> > > On Mon, Jul 18, 2016 at 10:36 AM, Taher Alkhateeb <
> > > slidingfilaments@gmail.com <javascript:;>> wrote:
> > >
> > > > Hello Everyone,
> > > >
> > > > In reference to this thread and the Jira OFBIZ-7254, I'm very happy
> to
> > > > announce that OFBiz is now ready for applying unit tests with only 8
> > new
> > > > lines of code in the build script (r1753143) :)
> > > >
> > > > I recommend we do the following moving forward:
> > > >
> > > > 1- Introduce unit tests as much as we can to all components
> > > > 2- Migrate most of the integration tests we currently have to unit
> > tests
> > > > (since they are designed to do very little integration).
> > > >
> > > > This is a great chance for us to practice real TDD in OFBiz
> > > >
> > > > Cheers,
> > > >
> > > > Taher Alkhateeb
> > > >
> > > > On Mon, Jun 13, 2016 at 7:43 AM, Pranay Pandey <
> > > > pranay.pandey@hotwaxsystems.com <javascript:;>> wrote:
> > > >
> > > > > +1
> > > > >
> > > > > Best regards,
> > > > >
> > > > > Pranay Pandey
> > > > > HotWax Systems
> > > > > http://www.hotwaxsystems.com/
> > > > >
> > > > > On Fri, Jun 10, 2016 at 7:46 PM, Taher Alkhateeb <
> > > > > slidingfilaments@gmail.com <javascript:;>
> > > > > > wrote:
> > > > >
> > > > > > Hello Everyone,
> > > > > >
> > > > > > I was able to get a few tests running and this is very doable.
> But
> > I
> > > > > faced
> > > > > > a big problem in designing the testing framework because of ANT.
> > > > > >
> > > > > > The problem
> > > > > > ----------------
> > > > > > The way the build scripts are designed in OFBiz are very
> complex. A
> > > > > master
> > > > > > file calls other files which call other files. And in the middle
> > you
> > > > have
> > > > > > external libraries (ant-contrib) and macros, and variables, and
> > class
> > > > > path
> > > > > > declarations, and and and ....
> > > > > >
> > > > > > I cannot declare the tests programmatically (with JUnit test
> > suites)
> > > > > > because this means lower level components would depend on higher
> > > level
> > > > > > components. So I have to do it in ANT, by navigating this maze of
> > > build
> > > > > > scripts, and it was a headache for me just to read them, let
> alone
> > > > modify
> > > > > > them to create a testing framework.
> > > > > >
> > > > > > Suggested Solution
> > > > > > ------------------------
> > > > > > I suggest to implement the testing framework in Gradle, and
> simply
> > > call
> > > > > it
> > > > > > from within ant. This is a middle solution that sustains ant for
> > now,
> > > > but
> > > > > > can allow us to switch out later.
> > > > > >
> > > > > > This means I will just add one more file called build.gradle in
> the
> > > top
> > > > > > level directory, and figure out the business logic for calling
> the
> > > test
> > > > > > suites from that file
> > > > > >
> > > > > > I look forward to your feedback.
> > > > > >
> > > > > > Regards,
> > > > > >
> > > > > > Taher Alkhateeb
> > > > > >
> > > > > > On Wed, Jun 8, 2016 at 6:00 PM, Taher Alkhateeb <
> > > > > > slidingfilaments@gmail.com <javascript:;>>
> > > > > > wrote:
> > > > > >
> > > > > > > Hi Everyone,
> > > > > > >
> > > > > > > Thank you all for your support, JIRA created in
> > > > > > > https://issues.apache.org/jira/browse/OFBIZ-7254
> > > > > > >
> > > > > > > I will start working on it and try to implement ASAP to get my
> > > focus
> > > > > back
> > > > > > > on refactoring.
> > > > > > >
> > > > > > > Cheers!
> > > > > > >
> > > > > > > Taher Alkhateeb
> > > > > > >
> > > > > > > On Wed, Jun 8, 2016 at 4:58 PM, Deepak Dixit <
> > > > > > > deepak.dixit@hotwaxsystems.com <javascript:;>> wrote:
> > > > > > >
> > > > > > >> +1
> > > > > > >>
> > > > > > >> Thanks & Regards
> > > > > > >> --
> > > > > > >> Deepak Dixit
> > > > > > >> www.hotwaxsystems.com
> > > > > > >>
> > > > > > >> On Wed, Jun 8, 2016 at 7:12 PM, Mridul Pathak <
> > > > > > >> mridul.pathak@hotwaxsystems.com <javascript:;>> wrote:
> > > > > > >>
> > > > > > >> > +1
> > > > > > >> >
> > > > > > >> > Makes perfect sense.
> > > > > > >> >
> > > > > > >> > --
> > > > > > >> > Thanks & Regards,
> > > > > > >> > Mridul Pathak
> > > > > > >> > Senior Manager
> > > > > > >> > HotWax Systems
> > > > > > >> > http://www.hotwaxsystems.com
> > > > > > >> >
> > > > > > >> > > On Jun 8, 2016, at 2:41 PM, Taher Alkhateeb <
> > > > > > >> slidingfilaments@gmail.com <javascript:;>>
> > > > > > >> > wrote:
> > > > > > >> > >
> > > > > > >> > > Hello Everyone,
> > > > > > >> > >
> > > > > > >> > > After refactoring the start component and while starting
> on
> > > the
> > > > > base
> > > > > > >> > > component I realized that the testing framework for OFBiz
> is
> > > not
> > > > > > good.
> > > > > > >> > You
> > > > > > >> > > cannot do real test driven development or
> red-green-refactor
> > > > with
> > > > > > the
> > > > > > >> > > current setup, hence my proposal to change it. I explain
> > > below:
> > > > > > >> > >
> > > > > > >> > > Problem with current design
> > > > > > >> > > ----------------------------------------
> > > > > > >> > > - What we have right now is not unit tests, it's really
> > > > > integration
> > > > > > >> > tests.
> > > > > > >> > > You have to start the framework, the database, the service
> > > > engine,
> > > > > > the
> > > > > > >> > > entity engine and pretty much everything.
> > > > > > >> > > - Testing is very slow, because it's an integration test
> as
> > I
> > > > > > >> mentioned
> > > > > > >> > > above. 10 minutes on a good computer!
> > > > > > >> > > - There is zero mocking! We actually have to --load-data
> for
> > > > > things
> > > > > > to
> > > > > > >> > > work. Again, these are integration tests.
> > > > > > >> > > - Too complex: Integration tests by their nature are
> > grabbing
> > > > too
> > > > > > >> much.
> > > > > > >> > > Mind you, I am not objecting to integration tests (I
> > actually
> > > > like
> > > > > > >> them)
> > > > > > >> > > but I am objecting to not having real unit-tests. Unit
> tests
> > > > > should
> > > > > > >> all
> > > > > > >> > run
> > > > > > >> > > in a few seconds.
> > > > > > >> > >
> > > > > > >> > > Proposed solution
> > > > > > >> > > --------------------------
> > > > > > >> > > - We keep what is considered real integration tests the
> way
> > > they
> > > > > are
> > > > > > >> > right
> > > > > > >> > > now and keep using them
> > > > > > >> > > - We move what should be unit tests into simple JUnit
> > classes,
> > > > and
> > > > > > we
> > > > > > >> do
> > > > > > >> > > not run them using java -jar ofbiz.jar --test, but instead
> > run
> > > > > them
> > > > > > >> > > directly from the build.xml script, so these files are not
> > > > > > identified
> > > > > > >> in
> > > > > > >> > > any XML document, but are simply called immediately from
> the
> > > > build
> > > > > > >> > scripts.
> > > > > > >> > > - We clearly mark the difference between integration tests
> > and
> > > > > unit
> > > > > > >> tests
> > > > > > >> > > (inside the source files or in the suite declarations).
> > > > > > >> > > - We change the run-tests target in build.xml to run both
> > unit
> > > > > tests
> > > > > > >> and
> > > > > > >> > > integration tests.
> > > > > > >> > >
> > > > > > >> > > I intend to heavily refactor the framework and I would
> feel
> > > > better
> > > > > > >> about
> > > > > > >> > > introducing this change while refactoring. What do you
> guys
> > > > think?
> > > > > > >> Ideas?
> > > > > > >> > > Suggestions? Approvals and thumbs up?
> > > > > > >> > >
> > > > > > >> > > Regards,
> > > > > > >> > >
> > > > > > >> > > Taher Alkhateeb
> > > > > > >> >
> > > > > > >> >
> > > > > > >>
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: Proposal to modify the testing framework for OFBiz

[Scott Gray <sc...@hotwaxsystems.com>]

I know I'm late to the party here, but I just want to say that I think
integration tests have far greater value to OFBiz than unit tests.  Mostly
because we tend to have quite a low number of tests and integration tests
give us much better coverage per line of test code and the tests are much
closer to the real world scenarios the application might encounter.

I don't really see how unit tests could be applied to non-framework testing
in a useful manner, could you expand on your vision in that regard?  I mean
would we be testing something smaller than a service as the 'unit'?  What
would we mock? Would transaction management still be active? What happens
when the service calls another service, I guess we mock the response from
that service (how)?

It just seems a very complicated method to achieve a less thorough but
albeit faster (maybe) test result.

A build, data load and full test run takes 4m 9s on my laptop.  Excluding
build, data load and framework tests: the application level tests take 35s,
not very expensive IMO.  The data load time can be reduced to practically
nothing by copying a clean slate database into runtime for each run.

I'm mostly just suggesting we be wary of adding complicated testing
procedures in the hope of achieving some 'best practice' result which in
reality will provide minimal benefits.

Regards
Scott

On 18 July 2016 at 18:57, Taher Alkhateeb <sl...@gmail.com>
wrote:

> Hello Akash,
>
> Fantastic, I have a few unit tests almost done to be included in the start
> component. I will create a new subtask under OFBIZ-1463 to commit the tests
> so you can use them as a reference if you like to.
>
> I also recommend that you follow the same directory structure between the
> test code and production code. So for example:
>
> Production code: framework/start/src/main/java/org/apache/ofbiz/base/start
> Test code: framework/start/src/test/java/org/apache/ofbiz/base/start
>
> The benefit of this hierarchy is that you can access non-public (package
> protected) methods for testing. This is in fact exactly what I needed to be
> able to apply some of the tests.
>
> Cheers,
>
> Taher Alkhateeb
>
> On Mon, Jul 18, 2016 at 9:22 AM, Akash Jain <ak...@hotwaxsystems.com>
> wrote:
>
> > Thanks Taher for nice initiative!
> >
> > We are planning to written unit tests to all components under OFBIZ-1463
> >
> > Thanks and Regards
> > --
> > Akash Jain
> >
> > On Mon, Jul 18, 2016 at 10:36 AM, Taher Alkhateeb <
> > slidingfilaments@gmail.com> wrote:
> >
> > > Hello Everyone,
> > >
> > > In reference to this thread and the Jira OFBIZ-7254, I'm very happy to
> > > announce that OFBiz is now ready for applying unit tests with only 8
> new
> > > lines of code in the build script (r1753143) :)
> > >
> > > I recommend we do the following moving forward:
> > >
> > > 1- Introduce unit tests as much as we can to all components
> > > 2- Migrate most of the integration tests we currently have to unit
> tests
> > > (since they are designed to do very little integration).
> > >
> > > This is a great chance for us to practice real TDD in OFBiz
> > >
> > > Cheers,
> > >
> > > Taher Alkhateeb
> > >
> > > On Mon, Jun 13, 2016 at 7:43 AM, Pranay Pandey <
> > > pranay.pandey@hotwaxsystems.com> wrote:
> > >
> > > > +1
> > > >
> > > > Best regards,
> > > >
> > > > Pranay Pandey
> > > > HotWax Systems
> > > > http://www.hotwaxsystems.com/
> > > >
> > > > On Fri, Jun 10, 2016 at 7:46 PM, Taher Alkhateeb <
> > > > slidingfilaments@gmail.com
> > > > > wrote:
> > > >
> > > > > Hello Everyone,
> > > > >
> > > > > I was able to get a few tests running and this is very doable. But
> I
> > > > faced
> > > > > a big problem in designing the testing framework because of ANT.
> > > > >
> > > > > The problem
> > > > > ----------------
> > > > > The way the build scripts are designed in OFBiz are very complex. A
> > > > master
> > > > > file calls other files which call other files. And in the middle
> you
> > > have
> > > > > external libraries (ant-contrib) and macros, and variables, and
> class
> > > > path
> > > > > declarations, and and and ....
> > > > >
> > > > > I cannot declare the tests programmatically (with JUnit test
> suites)
> > > > > because this means lower level components would depend on higher
> > level
> > > > > components. So I have to do it in ANT, by navigating this maze of
> > build
> > > > > scripts, and it was a headache for me just to read them, let alone
> > > modify
> > > > > them to create a testing framework.
> > > > >
> > > > > Suggested Solution
> > > > > ------------------------
> > > > > I suggest to implement the testing framework in Gradle, and simply
> > call
> > > > it
> > > > > from within ant. This is a middle solution that sustains ant for
> now,
> > > but
> > > > > can allow us to switch out later.
> > > > >
> > > > > This means I will just add one more file called build.gradle in the
> > top
> > > > > level directory, and figure out the business logic for calling the
> > test
> > > > > suites from that file
> > > > >
> > > > > I look forward to your feedback.
> > > > >
> > > > > Regards,
> > > > >
> > > > > Taher Alkhateeb
> > > > >
> > > > > On Wed, Jun 8, 2016 at 6:00 PM, Taher Alkhateeb <
> > > > > slidingfilaments@gmail.com>
> > > > > wrote:
> > > > >
> > > > > > Hi Everyone,
> > > > > >
> > > > > > Thank you all for your support, JIRA created in
> > > > > > https://issues.apache.org/jira/browse/OFBIZ-7254
> > > > > >
> > > > > > I will start working on it and try to implement ASAP to get my
> > focus
> > > > back
> > > > > > on refactoring.
> > > > > >
> > > > > > Cheers!
> > > > > >
> > > > > > Taher Alkhateeb
> > > > > >
> > > > > > On Wed, Jun 8, 2016 at 4:58 PM, Deepak Dixit <
> > > > > > deepak.dixit@hotwaxsystems.com> wrote:
> > > > > >
> > > > > >> +1
> > > > > >>
> > > > > >> Thanks & Regards
> > > > > >> --
> > > > > >> Deepak Dixit
> > > > > >> www.hotwaxsystems.com
> > > > > >>
> > > > > >> On Wed, Jun 8, 2016 at 7:12 PM, Mridul Pathak <
> > > > > >> mridul.pathak@hotwaxsystems.com> wrote:
> > > > > >>
> > > > > >> > +1
> > > > > >> >
> > > > > >> > Makes perfect sense.
> > > > > >> >
> > > > > >> > --
> > > > > >> > Thanks & Regards,
> > > > > >> > Mridul Pathak
> > > > > >> > Senior Manager
> > > > > >> > HotWax Systems
> > > > > >> > http://www.hotwaxsystems.com
> > > > > >> >
> > > > > >> > > On Jun 8, 2016, at 2:41 PM, Taher Alkhateeb <
> > > > > >> slidingfilaments@gmail.com>
> > > > > >> > wrote:
> > > > > >> > >
> > > > > >> > > Hello Everyone,
> > > > > >> > >
> > > > > >> > > After refactoring the start component and while starting on
> > the
> > > > base
> > > > > >> > > component I realized that the testing framework for OFBiz is
> > not
> > > > > good.
> > > > > >> > You
> > > > > >> > > cannot do real test driven development or red-green-refactor
> > > with
> > > > > the
> > > > > >> > > current setup, hence my proposal to change it. I explain
> > below:
> > > > > >> > >
> > > > > >> > > Problem with current design
> > > > > >> > > ----------------------------------------
> > > > > >> > > - What we have right now is not unit tests, it's really
> > > > integration
> > > > > >> > tests.
> > > > > >> > > You have to start the framework, the database, the service
> > > engine,
> > > > > the
> > > > > >> > > entity engine and pretty much everything.
> > > > > >> > > - Testing is very slow, because it's an integration test as
> I
> > > > > >> mentioned
> > > > > >> > > above. 10 minutes on a good computer!
> > > > > >> > > - There is zero mocking! We actually have to --load-data for
> > > > things
> > > > > to
> > > > > >> > > work. Again, these are integration tests.
> > > > > >> > > - Too complex: Integration tests by their nature are
> grabbing
> > > too
> > > > > >> much.
> > > > > >> > > Mind you, I am not objecting to integration tests (I
> actually
> > > like
> > > > > >> them)
> > > > > >> > > but I am objecting to not having real unit-tests. Unit tests
> > > > should
> > > > > >> all
> > > > > >> > run
> > > > > >> > > in a few seconds.
> > > > > >> > >
> > > > > >> > > Proposed solution
> > > > > >> > > --------------------------
> > > > > >> > > - We keep what is considered real integration tests the way
> > they
> > > > are
> > > > > >> > right
> > > > > >> > > now and keep using them
> > > > > >> > > - We move what should be unit tests into simple JUnit
> classes,
> > > and
> > > > > we
> > > > > >> do
> > > > > >> > > not run them using java -jar ofbiz.jar --test, but instead
> run
> > > > them
> > > > > >> > > directly from the build.xml script, so these files are not
> > > > > identified
> > > > > >> in
> > > > > >> > > any XML document, but are simply called immediately from the
> > > build
> > > > > >> > scripts.
> > > > > >> > > - We clearly mark the difference between integration tests
> and
> > > > unit
> > > > > >> tests
> > > > > >> > > (inside the source files or in the suite declarations).
> > > > > >> > > - We change the run-tests target in build.xml to run both
> unit
> > > > tests
> > > > > >> and
> > > > > >> > > integration tests.
> > > > > >> > >
> > > > > >> > > I intend to heavily refactor the framework and I would feel
> > > better
> > > > > >> about
> > > > > >> > > introducing this change while refactoring. What do you guys
> > > think?
> > > > > >> Ideas?
> > > > > >> > > Suggestions? Approvals and thumbs up?
> > > > > >> > >
> > > > > >> > > Regards,
> > > > > >> > >
> > > > > >> > > Taher Alkhateeb
> > > > > >> >
> > > > > >> >
> > > > > >>
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: Proposal to modify the testing framework for OFBiz

[Taher Alkhateeb <sl...@gmail.com>]

Hello Akash,

Fantastic, I have a few unit tests almost done to be included in the start
component. I will create a new subtask under OFBIZ-1463 to commit the tests
so you can use them as a reference if you like to.

I also recommend that you follow the same directory structure between the
test code and production code. So for example:

Production code: framework/start/src/main/java/org/apache/ofbiz/base/start
Test code: framework/start/src/test/java/org/apache/ofbiz/base/start

The benefit of this hierarchy is that you can access non-public (package
protected) methods for testing. This is in fact exactly what I needed to be
able to apply some of the tests.

Cheers,

Taher Alkhateeb

On Mon, Jul 18, 2016 at 9:22 AM, Akash Jain <ak...@hotwaxsystems.com>
wrote:

> Thanks Taher for nice initiative!
>
> We are planning to written unit tests to all components under OFBIZ-1463
>
> Thanks and Regards
> --
> Akash Jain
>
> On Mon, Jul 18, 2016 at 10:36 AM, Taher Alkhateeb <
> slidingfilaments@gmail.com> wrote:
>
> > Hello Everyone,
> >
> > In reference to this thread and the Jira OFBIZ-7254, I'm very happy to
> > announce that OFBiz is now ready for applying unit tests with only 8 new
> > lines of code in the build script (r1753143) :)
> >
> > I recommend we do the following moving forward:
> >
> > 1- Introduce unit tests as much as we can to all components
> > 2- Migrate most of the integration tests we currently have to unit tests
> > (since they are designed to do very little integration).
> >
> > This is a great chance for us to practice real TDD in OFBiz
> >
> > Cheers,
> >
> > Taher Alkhateeb
> >
> > On Mon, Jun 13, 2016 at 7:43 AM, Pranay Pandey <
> > pranay.pandey@hotwaxsystems.com> wrote:
> >
> > > +1
> > >
> > > Best regards,
> > >
> > > Pranay Pandey
> > > HotWax Systems
> > > http://www.hotwaxsystems.com/
> > >
> > > On Fri, Jun 10, 2016 at 7:46 PM, Taher Alkhateeb <
> > > slidingfilaments@gmail.com
> > > > wrote:
> > >
> > > > Hello Everyone,
> > > >
> > > > I was able to get a few tests running and this is very doable. But I
> > > faced
> > > > a big problem in designing the testing framework because of ANT.
> > > >
> > > > The problem
> > > > ----------------
> > > > The way the build scripts are designed in OFBiz are very complex. A
> > > master
> > > > file calls other files which call other files. And in the middle you
> > have
> > > > external libraries (ant-contrib) and macros, and variables, and class
> > > path
> > > > declarations, and and and ....
> > > >
> > > > I cannot declare the tests programmatically (with JUnit test suites)
> > > > because this means lower level components would depend on higher
> level
> > > > components. So I have to do it in ANT, by navigating this maze of
> build
> > > > scripts, and it was a headache for me just to read them, let alone
> > modify
> > > > them to create a testing framework.
> > > >
> > > > Suggested Solution
> > > > ------------------------
> > > > I suggest to implement the testing framework in Gradle, and simply
> call
> > > it
> > > > from within ant. This is a middle solution that sustains ant for now,
> > but
> > > > can allow us to switch out later.
> > > >
> > > > This means I will just add one more file called build.gradle in the
> top
> > > > level directory, and figure out the business logic for calling the
> test
> > > > suites from that file
> > > >
> > > > I look forward to your feedback.
> > > >
> > > > Regards,
> > > >
> > > > Taher Alkhateeb
> > > >
> > > > On Wed, Jun 8, 2016 at 6:00 PM, Taher Alkhateeb <
> > > > slidingfilaments@gmail.com>
> > > > wrote:
> > > >
> > > > > Hi Everyone,
> > > > >
> > > > > Thank you all for your support, JIRA created in
> > > > > https://issues.apache.org/jira/browse/OFBIZ-7254
> > > > >
> > > > > I will start working on it and try to implement ASAP to get my
> focus
> > > back
> > > > > on refactoring.
> > > > >
> > > > > Cheers!
> > > > >
> > > > > Taher Alkhateeb
> > > > >
> > > > > On Wed, Jun 8, 2016 at 4:58 PM, Deepak Dixit <
> > > > > deepak.dixit@hotwaxsystems.com> wrote:
> > > > >
> > > > >> +1
> > > > >>
> > > > >> Thanks & Regards
> > > > >> --
> > > > >> Deepak Dixit
> > > > >> www.hotwaxsystems.com
> > > > >>
> > > > >> On Wed, Jun 8, 2016 at 7:12 PM, Mridul Pathak <
> > > > >> mridul.pathak@hotwaxsystems.com> wrote:
> > > > >>
> > > > >> > +1
> > > > >> >
> > > > >> > Makes perfect sense.
> > > > >> >
> > > > >> > --
> > > > >> > Thanks & Regards,
> > > > >> > Mridul Pathak
> > > > >> > Senior Manager
> > > > >> > HotWax Systems
> > > > >> > http://www.hotwaxsystems.com
> > > > >> >
> > > > >> > > On Jun 8, 2016, at 2:41 PM, Taher Alkhateeb <
> > > > >> slidingfilaments@gmail.com>
> > > > >> > wrote:
> > > > >> > >
> > > > >> > > Hello Everyone,
> > > > >> > >
> > > > >> > > After refactoring the start component and while starting on
> the
> > > base
> > > > >> > > component I realized that the testing framework for OFBiz is
> not
> > > > good.
> > > > >> > You
> > > > >> > > cannot do real test driven development or red-green-refactor
> > with
> > > > the
> > > > >> > > current setup, hence my proposal to change it. I explain
> below:
> > > > >> > >
> > > > >> > > Problem with current design
> > > > >> > > ----------------------------------------
> > > > >> > > - What we have right now is not unit tests, it's really
> > > integration
> > > > >> > tests.
> > > > >> > > You have to start the framework, the database, the service
> > engine,
> > > > the
> > > > >> > > entity engine and pretty much everything.
> > > > >> > > - Testing is very slow, because it's an integration test as I
> > > > >> mentioned
> > > > >> > > above. 10 minutes on a good computer!
> > > > >> > > - There is zero mocking! We actually have to --load-data for
> > > things
> > > > to
> > > > >> > > work. Again, these are integration tests.
> > > > >> > > - Too complex: Integration tests by their nature are grabbing
> > too
> > > > >> much.
> > > > >> > > Mind you, I am not objecting to integration tests (I actually
> > like
> > > > >> them)
> > > > >> > > but I am objecting to not having real unit-tests. Unit tests
> > > should
> > > > >> all
> > > > >> > run
> > > > >> > > in a few seconds.
> > > > >> > >
> > > > >> > > Proposed solution
> > > > >> > > --------------------------
> > > > >> > > - We keep what is considered real integration tests the way
> they
> > > are
> > > > >> > right
> > > > >> > > now and keep using them
> > > > >> > > - We move what should be unit tests into simple JUnit classes,
> > and
> > > > we
> > > > >> do
> > > > >> > > not run them using java -jar ofbiz.jar --test, but instead run
> > > them
> > > > >> > > directly from the build.xml script, so these files are not
> > > > identified
> > > > >> in
> > > > >> > > any XML document, but are simply called immediately from the
> > build
> > > > >> > scripts.
> > > > >> > > - We clearly mark the difference between integration tests and
> > > unit
> > > > >> tests
> > > > >> > > (inside the source files or in the suite declarations).
> > > > >> > > - We change the run-tests target in build.xml to run both unit
> > > tests
> > > > >> and
> > > > >> > > integration tests.
> > > > >> > >
> > > > >> > > I intend to heavily refactor the framework and I would feel
> > better
> > > > >> about
> > > > >> > > introducing this change while refactoring. What do you guys
> > think?
> > > > >> Ideas?
> > > > >> > > Suggestions? Approvals and thumbs up?
> > > > >> > >
> > > > >> > > Regards,
> > > > >> > >
> > > > >> > > Taher Alkhateeb
> > > > >> >
> > > > >> >
> > > > >>
> > > > >
> > > > >
> > > >
> > >
> >
>

Re: Proposal to modify the testing framework for OFBiz

[Jacques Le Roux <ja...@les7arts.com>]

It's very cool to see this old Jira revived :)

Jacques


Le 18/07/2016 � 08:22, Akash Jain a �crit :
> Thanks Taher for nice initiative!
>
> We are planning to written unit tests to all components under OFBIZ-1463
>
> Thanks and Regards
> --
> Akash Jain
>
> On Mon, Jul 18, 2016 at 10:36 AM, Taher Alkhateeb <
> slidingfilaments@gmail.com> wrote:
>
>> Hello Everyone,
>>
>> In reference to this thread and the Jira OFBIZ-7254, I'm very happy to
>> announce that OFBiz is now ready for applying unit tests with only 8 new
>> lines of code in the build script (r1753143) :)
>>
>> I recommend we do the following moving forward:
>>
>> 1- Introduce unit tests as much as we can to all components
>> 2- Migrate most of the integration tests we currently have to unit tests
>> (since they are designed to do very little integration).
>>
>> This is a great chance for us to practice real TDD in OFBiz
>>
>> Cheers,
>>
>> Taher Alkhateeb
>>
>> On Mon, Jun 13, 2016 at 7:43 AM, Pranay Pandey <
>> pranay.pandey@hotwaxsystems.com> wrote:
>>
>>> +1
>>>
>>> Best regards,
>>>
>>> Pranay Pandey
>>> HotWax Systems
>>> http://www.hotwaxsystems.com/
>>>
>>> On Fri, Jun 10, 2016 at 7:46 PM, Taher Alkhateeb <
>>> slidingfilaments@gmail.com
>>>> wrote:
>>>> Hello Everyone,
>>>>
>>>> I was able to get a few tests running and this is very doable. But I
>>> faced
>>>> a big problem in designing the testing framework because of ANT.
>>>>
>>>> The problem
>>>> ----------------
>>>> The way the build scripts are designed in OFBiz are very complex. A
>>> master
>>>> file calls other files which call other files. And in the middle you
>> have
>>>> external libraries (ant-contrib) and macros, and variables, and class
>>> path
>>>> declarations, and and and ....
>>>>
>>>> I cannot declare the tests programmatically (with JUnit test suites)
>>>> because this means lower level components would depend on higher level
>>>> components. So I have to do it in ANT, by navigating this maze of build
>>>> scripts, and it was a headache for me just to read them, let alone
>> modify
>>>> them to create a testing framework.
>>>>
>>>> Suggested Solution
>>>> ------------------------
>>>> I suggest to implement the testing framework in Gradle, and simply call
>>> it
>>>> from within ant. This is a middle solution that sustains ant for now,
>> but
>>>> can allow us to switch out later.
>>>>
>>>> This means I will just add one more file called build.gradle in the top
>>>> level directory, and figure out the business logic for calling the test
>>>> suites from that file
>>>>
>>>> I look forward to your feedback.
>>>>
>>>> Regards,
>>>>
>>>> Taher Alkhateeb
>>>>
>>>> On Wed, Jun 8, 2016 at 6:00 PM, Taher Alkhateeb <
>>>> slidingfilaments@gmail.com>
>>>> wrote:
>>>>
>>>>> Hi Everyone,
>>>>>
>>>>> Thank you all for your support, JIRA created in
>>>>> https://issues.apache.org/jira/browse/OFBIZ-7254
>>>>>
>>>>> I will start working on it and try to implement ASAP to get my focus
>>> back
>>>>> on refactoring.
>>>>>
>>>>> Cheers!
>>>>>
>>>>> Taher Alkhateeb
>>>>>
>>>>> On Wed, Jun 8, 2016 at 4:58 PM, Deepak Dixit <
>>>>> deepak.dixit@hotwaxsystems.com> wrote:
>>>>>
>>>>>> +1
>>>>>>
>>>>>> Thanks & Regards
>>>>>> --
>>>>>> Deepak Dixit
>>>>>> www.hotwaxsystems.com
>>>>>>
>>>>>> On Wed, Jun 8, 2016 at 7:12 PM, Mridul Pathak <
>>>>>> mridul.pathak@hotwaxsystems.com> wrote:
>>>>>>
>>>>>>> +1
>>>>>>>
>>>>>>> Makes perfect sense.
>>>>>>>
>>>>>>> --
>>>>>>> Thanks & Regards,
>>>>>>> Mridul Pathak
>>>>>>> Senior Manager
>>>>>>> HotWax Systems
>>>>>>> http://www.hotwaxsystems.com
>>>>>>>
>>>>>>>> On Jun 8, 2016, at 2:41 PM, Taher Alkhateeb <
>>>>>> slidingfilaments@gmail.com>
>>>>>>> wrote:
>>>>>>>> Hello Everyone,
>>>>>>>>
>>>>>>>> After refactoring the start component and while starting on the
>>> base
>>>>>>>> component I realized that the testing framework for OFBiz is not
>>>> good.
>>>>>>> You
>>>>>>>> cannot do real test driven development or red-green-refactor
>> with
>>>> the
>>>>>>>> current setup, hence my proposal to change it. I explain below:
>>>>>>>>
>>>>>>>> Problem with current design
>>>>>>>> ----------------------------------------
>>>>>>>> - What we have right now is not unit tests, it's really
>>> integration
>>>>>>> tests.
>>>>>>>> You have to start the framework, the database, the service
>> engine,
>>>> the
>>>>>>>> entity engine and pretty much everything.
>>>>>>>> - Testing is very slow, because it's an integration test as I
>>>>>> mentioned
>>>>>>>> above. 10 minutes on a good computer!
>>>>>>>> - There is zero mocking! We actually have to --load-data for
>>> things
>>>> to
>>>>>>>> work. Again, these are integration tests.
>>>>>>>> - Too complex: Integration tests by their nature are grabbing
>> too
>>>>>> much.
>>>>>>>> Mind you, I am not objecting to integration tests (I actually
>> like
>>>>>> them)
>>>>>>>> but I am objecting to not having real unit-tests. Unit tests
>>> should
>>>>>> all
>>>>>>> run
>>>>>>>> in a few seconds.
>>>>>>>>
>>>>>>>> Proposed solution
>>>>>>>> --------------------------
>>>>>>>> - We keep what is considered real integration tests the way they
>>> are
>>>>>>> right
>>>>>>>> now and keep using them
>>>>>>>> - We move what should be unit tests into simple JUnit classes,
>> and
>>>> we
>>>>>> do
>>>>>>>> not run them using java -jar ofbiz.jar --test, but instead run
>>> them
>>>>>>>> directly from the build.xml script, so these files are not
>>>> identified
>>>>>> in
>>>>>>>> any XML document, but are simply called immediately from the
>> build
>>>>>>> scripts.
>>>>>>>> - We clearly mark the difference between integration tests and
>>> unit
>>>>>> tests
>>>>>>>> (inside the source files or in the suite declarations).
>>>>>>>> - We change the run-tests target in build.xml to run both unit
>>> tests
>>>>>> and
>>>>>>>> integration tests.
>>>>>>>>
>>>>>>>> I intend to heavily refactor the framework and I would feel
>> better
>>>>>> about
>>>>>>>> introducing this change while refactoring. What do you guys
>> think?
>>>>>> Ideas?
>>>>>>>> Suggestions? Approvals and thumbs up?
>>>>>>>>
>>>>>>>> Regards,
>>>>>>>>
>>>>>>>> Taher Alkhateeb
>>>>>>>
>>>>>

Re: Proposal to modify the testing framework for OFBiz

[Akash Jain <ak...@hotwaxsystems.com>]

Thanks Taher for nice initiative!

We are planning to written unit tests to all components under OFBIZ-1463

Thanks and Regards
--
Akash Jain

On Mon, Jul 18, 2016 at 10:36 AM, Taher Alkhateeb <
slidingfilaments@gmail.com> wrote:

> Hello Everyone,
>
> In reference to this thread and the Jira OFBIZ-7254, I'm very happy to
> announce that OFBiz is now ready for applying unit tests with only 8 new
> lines of code in the build script (r1753143) :)
>
> I recommend we do the following moving forward:
>
> 1- Introduce unit tests as much as we can to all components
> 2- Migrate most of the integration tests we currently have to unit tests
> (since they are designed to do very little integration).
>
> This is a great chance for us to practice real TDD in OFBiz
>
> Cheers,
>
> Taher Alkhateeb
>
> On Mon, Jun 13, 2016 at 7:43 AM, Pranay Pandey <
> pranay.pandey@hotwaxsystems.com> wrote:
>
> > +1
> >
> > Best regards,
> >
> > Pranay Pandey
> > HotWax Systems
> > http://www.hotwaxsystems.com/
> >
> > On Fri, Jun 10, 2016 at 7:46 PM, Taher Alkhateeb <
> > slidingfilaments@gmail.com
> > > wrote:
> >
> > > Hello Everyone,
> > >
> > > I was able to get a few tests running and this is very doable. But I
> > faced
> > > a big problem in designing the testing framework because of ANT.
> > >
> > > The problem
> > > ----------------
> > > The way the build scripts are designed in OFBiz are very complex. A
> > master
> > > file calls other files which call other files. And in the middle you
> have
> > > external libraries (ant-contrib) and macros, and variables, and class
> > path
> > > declarations, and and and ....
> > >
> > > I cannot declare the tests programmatically (with JUnit test suites)
> > > because this means lower level components would depend on higher level
> > > components. So I have to do it in ANT, by navigating this maze of build
> > > scripts, and it was a headache for me just to read them, let alone
> modify
> > > them to create a testing framework.
> > >
> > > Suggested Solution
> > > ------------------------
> > > I suggest to implement the testing framework in Gradle, and simply call
> > it
> > > from within ant. This is a middle solution that sustains ant for now,
> but
> > > can allow us to switch out later.
> > >
> > > This means I will just add one more file called build.gradle in the top
> > > level directory, and figure out the business logic for calling the test
> > > suites from that file
> > >
> > > I look forward to your feedback.
> > >
> > > Regards,
> > >
> > > Taher Alkhateeb
> > >
> > > On Wed, Jun 8, 2016 at 6:00 PM, Taher Alkhateeb <
> > > slidingfilaments@gmail.com>
> > > wrote:
> > >
> > > > Hi Everyone,
> > > >
> > > > Thank you all for your support, JIRA created in
> > > > https://issues.apache.org/jira/browse/OFBIZ-7254
> > > >
> > > > I will start working on it and try to implement ASAP to get my focus
> > back
> > > > on refactoring.
> > > >
> > > > Cheers!
> > > >
> > > > Taher Alkhateeb
> > > >
> > > > On Wed, Jun 8, 2016 at 4:58 PM, Deepak Dixit <
> > > > deepak.dixit@hotwaxsystems.com> wrote:
> > > >
> > > >> +1
> > > >>
> > > >> Thanks & Regards
> > > >> --
> > > >> Deepak Dixit
> > > >> www.hotwaxsystems.com
> > > >>
> > > >> On Wed, Jun 8, 2016 at 7:12 PM, Mridul Pathak <
> > > >> mridul.pathak@hotwaxsystems.com> wrote:
> > > >>
> > > >> > +1
> > > >> >
> > > >> > Makes perfect sense.
> > > >> >
> > > >> > --
> > > >> > Thanks & Regards,
> > > >> > Mridul Pathak
> > > >> > Senior Manager
> > > >> > HotWax Systems
> > > >> > http://www.hotwaxsystems.com
> > > >> >
> > > >> > > On Jun 8, 2016, at 2:41 PM, Taher Alkhateeb <
> > > >> slidingfilaments@gmail.com>
> > > >> > wrote:
> > > >> > >
> > > >> > > Hello Everyone,
> > > >> > >
> > > >> > > After refactoring the start component and while starting on the
> > base
> > > >> > > component I realized that the testing framework for OFBiz is not
> > > good.
> > > >> > You
> > > >> > > cannot do real test driven development or red-green-refactor
> with
> > > the
> > > >> > > current setup, hence my proposal to change it. I explain below:
> > > >> > >
> > > >> > > Problem with current design
> > > >> > > ----------------------------------------
> > > >> > > - What we have right now is not unit tests, it's really
> > integration
> > > >> > tests.
> > > >> > > You have to start the framework, the database, the service
> engine,
> > > the
> > > >> > > entity engine and pretty much everything.
> > > >> > > - Testing is very slow, because it's an integration test as I
> > > >> mentioned
> > > >> > > above. 10 minutes on a good computer!
> > > >> > > - There is zero mocking! We actually have to --load-data for
> > things
> > > to
> > > >> > > work. Again, these are integration tests.
> > > >> > > - Too complex: Integration tests by their nature are grabbing
> too
> > > >> much.
> > > >> > > Mind you, I am not objecting to integration tests (I actually
> like
> > > >> them)
> > > >> > > but I am objecting to not having real unit-tests. Unit tests
> > should
> > > >> all
> > > >> > run
> > > >> > > in a few seconds.
> > > >> > >
> > > >> > > Proposed solution
> > > >> > > --------------------------
> > > >> > > - We keep what is considered real integration tests the way they
> > are
> > > >> > right
> > > >> > > now and keep using them
> > > >> > > - We move what should be unit tests into simple JUnit classes,
> and
> > > we
> > > >> do
> > > >> > > not run them using java -jar ofbiz.jar --test, but instead run
> > them
> > > >> > > directly from the build.xml script, so these files are not
> > > identified
> > > >> in
> > > >> > > any XML document, but are simply called immediately from the
> build
> > > >> > scripts.
> > > >> > > - We clearly mark the difference between integration tests and
> > unit
> > > >> tests
> > > >> > > (inside the source files or in the suite declarations).
> > > >> > > - We change the run-tests target in build.xml to run both unit
> > tests
> > > >> and
> > > >> > > integration tests.
> > > >> > >
> > > >> > > I intend to heavily refactor the framework and I would feel
> better
> > > >> about
> > > >> > > introducing this change while refactoring. What do you guys
> think?
> > > >> Ideas?
> > > >> > > Suggestions? Approvals and thumbs up?
> > > >> > >
> > > >> > > Regards,
> > > >> > >
> > > >> > > Taher Alkhateeb
> > > >> >
> > > >> >
> > > >>
> > > >
> > > >
> > >
> >
>

Re: Proposal to modify the testing framework for OFBiz

[Taher Alkhateeb <sl...@gmail.com>]

Hello Everyone,

In reference to this thread and the Jira OFBIZ-7254, I'm very happy to
announce that OFBiz is now ready for applying unit tests with only 8 new
lines of code in the build script (r1753143) :)

I recommend we do the following moving forward:

1- Introduce unit tests as much as we can to all components
2- Migrate most of the integration tests we currently have to unit tests
(since they are designed to do very little integration).

This is a great chance for us to practice real TDD in OFBiz

Cheers,

Taher Alkhateeb

On Mon, Jun 13, 2016 at 7:43 AM, Pranay Pandey <
pranay.pandey@hotwaxsystems.com> wrote:

> +1
>
> Best regards,
>
> Pranay Pandey
> HotWax Systems
> http://www.hotwaxsystems.com/
>
> On Fri, Jun 10, 2016 at 7:46 PM, Taher Alkhateeb <
> slidingfilaments@gmail.com
> > wrote:
>
> > Hello Everyone,
> >
> > I was able to get a few tests running and this is very doable. But I
> faced
> > a big problem in designing the testing framework because of ANT.
> >
> > The problem
> > ----------------
> > The way the build scripts are designed in OFBiz are very complex. A
> master
> > file calls other files which call other files. And in the middle you have
> > external libraries (ant-contrib) and macros, and variables, and class
> path
> > declarations, and and and ....
> >
> > I cannot declare the tests programmatically (with JUnit test suites)
> > because this means lower level components would depend on higher level
> > components. So I have to do it in ANT, by navigating this maze of build
> > scripts, and it was a headache for me just to read them, let alone modify
> > them to create a testing framework.
> >
> > Suggested Solution
> > ------------------------
> > I suggest to implement the testing framework in Gradle, and simply call
> it
> > from within ant. This is a middle solution that sustains ant for now, but
> > can allow us to switch out later.
> >
> > This means I will just add one more file called build.gradle in the top
> > level directory, and figure out the business logic for calling the test
> > suites from that file
> >
> > I look forward to your feedback.
> >
> > Regards,
> >
> > Taher Alkhateeb
> >
> > On Wed, Jun 8, 2016 at 6:00 PM, Taher Alkhateeb <
> > slidingfilaments@gmail.com>
> > wrote:
> >
> > > Hi Everyone,
> > >
> > > Thank you all for your support, JIRA created in
> > > https://issues.apache.org/jira/browse/OFBIZ-7254
> > >
> > > I will start working on it and try to implement ASAP to get my focus
> back
> > > on refactoring.
> > >
> > > Cheers!
> > >
> > > Taher Alkhateeb
> > >
> > > On Wed, Jun 8, 2016 at 4:58 PM, Deepak Dixit <
> > > deepak.dixit@hotwaxsystems.com> wrote:
> > >
> > >> +1
> > >>
> > >> Thanks & Regards
> > >> --
> > >> Deepak Dixit
> > >> www.hotwaxsystems.com
> > >>
> > >> On Wed, Jun 8, 2016 at 7:12 PM, Mridul Pathak <
> > >> mridul.pathak@hotwaxsystems.com> wrote:
> > >>
> > >> > +1
> > >> >
> > >> > Makes perfect sense.
> > >> >
> > >> > --
> > >> > Thanks & Regards,
> > >> > Mridul Pathak
> > >> > Senior Manager
> > >> > HotWax Systems
> > >> > http://www.hotwaxsystems.com
> > >> >
> > >> > > On Jun 8, 2016, at 2:41 PM, Taher Alkhateeb <
> > >> slidingfilaments@gmail.com>
> > >> > wrote:
> > >> > >
> > >> > > Hello Everyone,
> > >> > >
> > >> > > After refactoring the start component and while starting on the
> base
> > >> > > component I realized that the testing framework for OFBiz is not
> > good.
> > >> > You
> > >> > > cannot do real test driven development or red-green-refactor with
> > the
> > >> > > current setup, hence my proposal to change it. I explain below:
> > >> > >
> > >> > > Problem with current design
> > >> > > ----------------------------------------
> > >> > > - What we have right now is not unit tests, it's really
> integration
> > >> > tests.
> > >> > > You have to start the framework, the database, the service engine,
> > the
> > >> > > entity engine and pretty much everything.
> > >> > > - Testing is very slow, because it's an integration test as I
> > >> mentioned
> > >> > > above. 10 minutes on a good computer!
> > >> > > - There is zero mocking! We actually have to --load-data for
> things
> > to
> > >> > > work. Again, these are integration tests.
> > >> > > - Too complex: Integration tests by their nature are grabbing too
> > >> much.
> > >> > > Mind you, I am not objecting to integration tests (I actually like
> > >> them)
> > >> > > but I am objecting to not having real unit-tests. Unit tests
> should
> > >> all
> > >> > run
> > >> > > in a few seconds.
> > >> > >
> > >> > > Proposed solution
> > >> > > --------------------------
> > >> > > - We keep what is considered real integration tests the way they
> are
> > >> > right
> > >> > > now and keep using them
> > >> > > - We move what should be unit tests into simple JUnit classes, and
> > we
> > >> do
> > >> > > not run them using java -jar ofbiz.jar --test, but instead run
> them
> > >> > > directly from the build.xml script, so these files are not
> > identified
> > >> in
> > >> > > any XML document, but are simply called immediately from the build
> > >> > scripts.
> > >> > > - We clearly mark the difference between integration tests and
> unit
> > >> tests
> > >> > > (inside the source files or in the suite declarations).
> > >> > > - We change the run-tests target in build.xml to run both unit
> tests
> > >> and
> > >> > > integration tests.
> > >> > >
> > >> > > I intend to heavily refactor the framework and I would feel better
> > >> about
> > >> > > introducing this change while refactoring. What do you guys think?
> > >> Ideas?
> > >> > > Suggestions? Approvals and thumbs up?
> > >> > >
> > >> > > Regards,
> > >> > >
> > >> > > Taher Alkhateeb
> > >> >
> > >> >
> > >>
> > >
> > >
> >
>

Re: Proposal to modify the testing framework for OFBiz

[Pranay Pandey <pr...@hotwaxsystems.com>]

+1

Best regards,

Pranay Pandey
HotWax Systems
http://www.hotwaxsystems.com/

On Fri, Jun 10, 2016 at 7:46 PM, Taher Alkhateeb <slidingfilaments@gmail.com
> wrote:

> Hello Everyone,
>
> I was able to get a few tests running and this is very doable. But I faced
> a big problem in designing the testing framework because of ANT.
>
> The problem
> ----------------
> The way the build scripts are designed in OFBiz are very complex. A master
> file calls other files which call other files. And in the middle you have
> external libraries (ant-contrib) and macros, and variables, and class path
> declarations, and and and ....
>
> I cannot declare the tests programmatically (with JUnit test suites)
> because this means lower level components would depend on higher level
> components. So I have to do it in ANT, by navigating this maze of build
> scripts, and it was a headache for me just to read them, let alone modify
> them to create a testing framework.
>
> Suggested Solution
> ------------------------
> I suggest to implement the testing framework in Gradle, and simply call it
> from within ant. This is a middle solution that sustains ant for now, but
> can allow us to switch out later.
>
> This means I will just add one more file called build.gradle in the top
> level directory, and figure out the business logic for calling the test
> suites from that file
>
> I look forward to your feedback.
>
> Regards,
>
> Taher Alkhateeb
>
> On Wed, Jun 8, 2016 at 6:00 PM, Taher Alkhateeb <
> slidingfilaments@gmail.com>
> wrote:
>
> > Hi Everyone,
> >
> > Thank you all for your support, JIRA created in
> > https://issues.apache.org/jira/browse/OFBIZ-7254
> >
> > I will start working on it and try to implement ASAP to get my focus back
> > on refactoring.
> >
> > Cheers!
> >
> > Taher Alkhateeb
> >
> > On Wed, Jun 8, 2016 at 4:58 PM, Deepak Dixit <
> > deepak.dixit@hotwaxsystems.com> wrote:
> >
> >> +1
> >>
> >> Thanks & Regards
> >> --
> >> Deepak Dixit
> >> www.hotwaxsystems.com
> >>
> >> On Wed, Jun 8, 2016 at 7:12 PM, Mridul Pathak <
> >> mridul.pathak@hotwaxsystems.com> wrote:
> >>
> >> > +1
> >> >
> >> > Makes perfect sense.
> >> >
> >> > --
> >> > Thanks & Regards,
> >> > Mridul Pathak
> >> > Senior Manager
> >> > HotWax Systems
> >> > http://www.hotwaxsystems.com
> >> >
> >> > > On Jun 8, 2016, at 2:41 PM, Taher Alkhateeb <
> >> slidingfilaments@gmail.com>
> >> > wrote:
> >> > >
> >> > > Hello Everyone,
> >> > >
> >> > > After refactoring the start component and while starting on the base
> >> > > component I realized that the testing framework for OFBiz is not
> good.
> >> > You
> >> > > cannot do real test driven development or red-green-refactor with
> the
> >> > > current setup, hence my proposal to change it. I explain below:
> >> > >
> >> > > Problem with current design
> >> > > ----------------------------------------
> >> > > - What we have right now is not unit tests, it's really integration
> >> > tests.
> >> > > You have to start the framework, the database, the service engine,
> the
> >> > > entity engine and pretty much everything.
> >> > > - Testing is very slow, because it's an integration test as I
> >> mentioned
> >> > > above. 10 minutes on a good computer!
> >> > > - There is zero mocking! We actually have to --load-data for things
> to
> >> > > work. Again, these are integration tests.
> >> > > - Too complex: Integration tests by their nature are grabbing too
> >> much.
> >> > > Mind you, I am not objecting to integration tests (I actually like
> >> them)
> >> > > but I am objecting to not having real unit-tests. Unit tests should
> >> all
> >> > run
> >> > > in a few seconds.
> >> > >
> >> > > Proposed solution
> >> > > --------------------------
> >> > > - We keep what is considered real integration tests the way they are
> >> > right
> >> > > now and keep using them
> >> > > - We move what should be unit tests into simple JUnit classes, and
> we
> >> do
> >> > > not run them using java -jar ofbiz.jar --test, but instead run them
> >> > > directly from the build.xml script, so these files are not
> identified
> >> in
> >> > > any XML document, but are simply called immediately from the build
> >> > scripts.
> >> > > - We clearly mark the difference between integration tests and unit
> >> tests
> >> > > (inside the source files or in the suite declarations).
> >> > > - We change the run-tests target in build.xml to run both unit tests
> >> and
> >> > > integration tests.
> >> > >
> >> > > I intend to heavily refactor the framework and I would feel better
> >> about
> >> > > introducing this change while refactoring. What do you guys think?
> >> Ideas?
> >> > > Suggestions? Approvals and thumbs up?
> >> > >
> >> > > Regards,
> >> > >
> >> > > Taher Alkhateeb
> >> >
> >> >
> >>
> >
> >
>

Re: Proposal to modify the testing framework for OFBiz

[Mridul Pathak <mr...@hotwaxsystems.com>]

+1

--
Thanks & Regards,
Mridul Pathak
HotWax Systems
http://www.hotwaxsystems.com

> On Jun 10, 2016, at 7:46 PM, Taher Alkhateeb <sl...@gmail.com> wrote:
> 
> Hello Everyone,
> 
> I was able to get a few tests running and this is very doable. But I faced
> a big problem in designing the testing framework because of ANT.
> 
> The problem
> ----------------
> The way the build scripts are designed in OFBiz are very complex. A master
> file calls other files which call other files. And in the middle you have
> external libraries (ant-contrib) and macros, and variables, and class path
> declarations, and and and ....
> 
> I cannot declare the tests programmatically (with JUnit test suites)
> because this means lower level components would depend on higher level
> components. So I have to do it in ANT, by navigating this maze of build
> scripts, and it was a headache for me just to read them, let alone modify
> them to create a testing framework.
> 
> Suggested Solution
> ------------------------
> I suggest to implement the testing framework in Gradle, and simply call it
> from within ant. This is a middle solution that sustains ant for now, but
> can allow us to switch out later.
> 
> This means I will just add one more file called build.gradle in the top
> level directory, and figure out the business logic for calling the test
> suites from that file
> 
> I look forward to your feedback.
> 
> Regards,
> 
> Taher Alkhateeb
> 
> On Wed, Jun 8, 2016 at 6:00 PM, Taher Alkhateeb <sl...@gmail.com>
> wrote:
> 
>> Hi Everyone,
>> 
>> Thank you all for your support, JIRA created in
>> https://issues.apache.org/jira/browse/OFBIZ-7254
>> 
>> I will start working on it and try to implement ASAP to get my focus back
>> on refactoring.
>> 
>> Cheers!
>> 
>> Taher Alkhateeb
>> 
>> On Wed, Jun 8, 2016 at 4:58 PM, Deepak Dixit <
>> deepak.dixit@hotwaxsystems.com> wrote:
>> 
>>> +1
>>> 
>>> Thanks & Regards
>>> --
>>> Deepak Dixit
>>> www.hotwaxsystems.com
>>> 
>>> On Wed, Jun 8, 2016 at 7:12 PM, Mridul Pathak <
>>> mridul.pathak@hotwaxsystems.com> wrote:
>>> 
>>>> +1
>>>> 
>>>> Makes perfect sense.
>>>> 
>>>> --
>>>> Thanks & Regards,
>>>> Mridul Pathak
>>>> Senior Manager
>>>> HotWax Systems
>>>> http://www.hotwaxsystems.com
>>>> 
>>>>> On Jun 8, 2016, at 2:41 PM, Taher Alkhateeb <
>>> slidingfilaments@gmail.com>
>>>> wrote:
>>>>> 
>>>>> Hello Everyone,
>>>>> 
>>>>> After refactoring the start component and while starting on the base
>>>>> component I realized that the testing framework for OFBiz is not good.
>>>> You
>>>>> cannot do real test driven development or red-green-refactor with the
>>>>> current setup, hence my proposal to change it. I explain below:
>>>>> 
>>>>> Problem with current design
>>>>> ----------------------------------------
>>>>> - What we have right now is not unit tests, it's really integration
>>>> tests.
>>>>> You have to start the framework, the database, the service engine, the
>>>>> entity engine and pretty much everything.
>>>>> - Testing is very slow, because it's an integration test as I
>>> mentioned
>>>>> above. 10 minutes on a good computer!
>>>>> - There is zero mocking! We actually have to --load-data for things to
>>>>> work. Again, these are integration tests.
>>>>> - Too complex: Integration tests by their nature are grabbing too
>>> much.
>>>>> Mind you, I am not objecting to integration tests (I actually like
>>> them)
>>>>> but I am objecting to not having real unit-tests. Unit tests should
>>> all
>>>> run
>>>>> in a few seconds.
>>>>> 
>>>>> Proposed solution
>>>>> --------------------------
>>>>> - We keep what is considered real integration tests the way they are
>>>> right
>>>>> now and keep using them
>>>>> - We move what should be unit tests into simple JUnit classes, and we
>>> do
>>>>> not run them using java -jar ofbiz.jar --test, but instead run them
>>>>> directly from the build.xml script, so these files are not identified
>>> in
>>>>> any XML document, but are simply called immediately from the build
>>>> scripts.
>>>>> - We clearly mark the difference between integration tests and unit
>>> tests
>>>>> (inside the source files or in the suite declarations).
>>>>> - We change the run-tests target in build.xml to run both unit tests
>>> and
>>>>> integration tests.
>>>>> 
>>>>> I intend to heavily refactor the framework and I would feel better
>>> about
>>>>> introducing this change while refactoring. What do you guys think?
>>> Ideas?
>>>>> Suggestions? Approvals and thumbs up?
>>>>> 
>>>>> Regards,
>>>>> 
>>>>> Taher Alkhateeb
>>>> 
>>>> 
>>> 
>> 
>> 

Re: Proposal to modify the testing framework for OFBiz

[Sharan Foga <sh...@gmail.com>]

+1, I agree with Jacques and Jacopo.

Thanks
Sharan

On 10/06/16 16:43, Jacques Le Roux wrote:
> +1, we need to move ahead
>
> Jacques
>
> Le 10/06/2016 � 16:29, Jacopo Cappellato a �crit :
>> It sounds a reasonable approach to me.
>> Especially because it will let you proceed with the important work 
>> you are
>> contributing without loosing the momentum to study old configuration 
>> code
>> that would require some cleanups regardless.
>>
>> Thanks,
>>
>> Jacopo
>>
>> On Fri, Jun 10, 2016 at 4:16 PM, Taher Alkhateeb 
>> <slidingfilaments@gmail.com
>>> wrote:
>>> Hello Everyone,
>>>
>>> I was able to get a few tests running and this is very doable. But I 
>>> faced
>>> a big problem in designing the testing framework because of ANT.
>>>
>>> The problem
>>> ----------------
>>> The way the build scripts are designed in OFBiz are very complex. A 
>>> master
>>> file calls other files which call other files. And in the middle you 
>>> have
>>> external libraries (ant-contrib) and macros, and variables, and 
>>> class path
>>> declarations, and and and ....
>>>
>>> I cannot declare the tests programmatically (with JUnit test suites)
>>> because this means lower level components would depend on higher level
>>> components. So I have to do it in ANT, by navigating this maze of build
>>> scripts, and it was a headache for me just to read them, let alone 
>>> modify
>>> them to create a testing framework.
>>>
>>> Suggested Solution
>>> ------------------------
>>> I suggest to implement the testing framework in Gradle, and simply 
>>> call it
>>> from within ant. This is a middle solution that sustains ant for 
>>> now, but
>>> can allow us to switch out later.
>>>
>>> This means I will just add one more file called build.gradle in the top
>>> level directory, and figure out the business logic for calling the test
>>> suites from that file
>>>
>>> I look forward to your feedback.
>>>
>>> Regards,
>>>
>>> Taher Alkhateeb
>>>
>>> On Wed, Jun 8, 2016 at 6:00 PM, Taher Alkhateeb <
>>> slidingfilaments@gmail.com>
>>> wrote:
>>>
>>>> Hi Everyone,
>>>>
>>>> Thank you all for your support, JIRA created in
>>>> https://issues.apache.org/jira/browse/OFBIZ-7254
>>>>
>>>> I will start working on it and try to implement ASAP to get my 
>>>> focus back
>>>> on refactoring.
>>>>
>>>> Cheers!
>>>>
>>>> Taher Alkhateeb
>>>>
>>>> On Wed, Jun 8, 2016 at 4:58 PM, Deepak Dixit <
>>>> deepak.dixit@hotwaxsystems.com> wrote:
>>>>
>>>>> +1
>>>>>
>>>>> Thanks & Regards
>>>>> -- 
>>>>> Deepak Dixit
>>>>> www.hotwaxsystems.com
>>>>>
>>>>> On Wed, Jun 8, 2016 at 7:12 PM, Mridul Pathak <
>>>>> mridul.pathak@hotwaxsystems.com> wrote:
>>>>>
>>>>>> +1
>>>>>>
>>>>>> Makes perfect sense.
>>>>>>
>>>>>> -- 
>>>>>> Thanks & Regards,
>>>>>> Mridul Pathak
>>>>>> Senior Manager
>>>>>> HotWax Systems
>>>>>> http://www.hotwaxsystems.com
>>>>>>
>>>>>>> On Jun 8, 2016, at 2:41 PM, Taher Alkhateeb <
>>>>> slidingfilaments@gmail.com>
>>>>>> wrote:
>>>>>>> Hello Everyone,
>>>>>>>
>>>>>>> After refactoring the start component and while starting on the 
>>>>>>> base
>>>>>>> component I realized that the testing framework for OFBiz is not
>>> good.
>>>>>> You
>>>>>>> cannot do real test driven development or red-green-refactor with
>>> the
>>>>>>> current setup, hence my proposal to change it. I explain below:
>>>>>>>
>>>>>>> Problem with current design
>>>>>>> ----------------------------------------
>>>>>>> - What we have right now is not unit tests, it's really integration
>>>>>> tests.
>>>>>>> You have to start the framework, the database, the service engine,
>>> the
>>>>>>> entity engine and pretty much everything.
>>>>>>> - Testing is very slow, because it's an integration test as I
>>>>> mentioned
>>>>>>> above. 10 minutes on a good computer!
>>>>>>> - There is zero mocking! We actually have to --load-data for things
>>> to
>>>>>>> work. Again, these are integration tests.
>>>>>>> - Too complex: Integration tests by their nature are grabbing too
>>>>> much.
>>>>>>> Mind you, I am not objecting to integration tests (I actually like
>>>>> them)
>>>>>>> but I am objecting to not having real unit-tests. Unit tests should
>>>>> all
>>>>>> run
>>>>>>> in a few seconds.
>>>>>>>
>>>>>>> Proposed solution
>>>>>>> --------------------------
>>>>>>> - We keep what is considered real integration tests the way they 
>>>>>>> are
>>>>>> right
>>>>>>> now and keep using them
>>>>>>> - We move what should be unit tests into simple JUnit classes, and
>>> we
>>>>> do
>>>>>>> not run them using java -jar ofbiz.jar --test, but instead run them
>>>>>>> directly from the build.xml script, so these files are not
>>> identified
>>>>> in
>>>>>>> any XML document, but are simply called immediately from the build
>>>>>> scripts.
>>>>>>> - We clearly mark the difference between integration tests and unit
>>>>> tests
>>>>>>> (inside the source files or in the suite declarations).
>>>>>>> - We change the run-tests target in build.xml to run both unit 
>>>>>>> tests
>>>>> and
>>>>>>> integration tests.
>>>>>>>
>>>>>>> I intend to heavily refactor the framework and I would feel better
>>>>> about
>>>>>>> introducing this change while refactoring. What do you guys think?
>>>>> Ideas?
>>>>>>> Suggestions? Approvals and thumbs up?
>>>>>>>
>>>>>>> Regards,
>>>>>>>
>>>>>>> Taher Alkhateeb
>>>>>>
>>>>
>

Re: Proposal to modify the testing framework for OFBiz

[Jacques Le Roux <ja...@les7arts.com>]

+1, we need to move ahead

Jacques

Le 10/06/2016 � 16:29, Jacopo Cappellato a �crit :
> It sounds a reasonable approach to me.
> Especially because it will let you proceed with the important work you are
> contributing without loosing the momentum to study old configuration code
> that would require some cleanups regardless.
>
> Thanks,
>
> Jacopo
>
> On Fri, Jun 10, 2016 at 4:16 PM, Taher Alkhateeb <slidingfilaments@gmail.com
>> wrote:
>> Hello Everyone,
>>
>> I was able to get a few tests running and this is very doable. But I faced
>> a big problem in designing the testing framework because of ANT.
>>
>> The problem
>> ----------------
>> The way the build scripts are designed in OFBiz are very complex. A master
>> file calls other files which call other files. And in the middle you have
>> external libraries (ant-contrib) and macros, and variables, and class path
>> declarations, and and and ....
>>
>> I cannot declare the tests programmatically (with JUnit test suites)
>> because this means lower level components would depend on higher level
>> components. So I have to do it in ANT, by navigating this maze of build
>> scripts, and it was a headache for me just to read them, let alone modify
>> them to create a testing framework.
>>
>> Suggested Solution
>> ------------------------
>> I suggest to implement the testing framework in Gradle, and simply call it
>> from within ant. This is a middle solution that sustains ant for now, but
>> can allow us to switch out later.
>>
>> This means I will just add one more file called build.gradle in the top
>> level directory, and figure out the business logic for calling the test
>> suites from that file
>>
>> I look forward to your feedback.
>>
>> Regards,
>>
>> Taher Alkhateeb
>>
>> On Wed, Jun 8, 2016 at 6:00 PM, Taher Alkhateeb <
>> slidingfilaments@gmail.com>
>> wrote:
>>
>>> Hi Everyone,
>>>
>>> Thank you all for your support, JIRA created in
>>> https://issues.apache.org/jira/browse/OFBIZ-7254
>>>
>>> I will start working on it and try to implement ASAP to get my focus back
>>> on refactoring.
>>>
>>> Cheers!
>>>
>>> Taher Alkhateeb
>>>
>>> On Wed, Jun 8, 2016 at 4:58 PM, Deepak Dixit <
>>> deepak.dixit@hotwaxsystems.com> wrote:
>>>
>>>> +1
>>>>
>>>> Thanks & Regards
>>>> --
>>>> Deepak Dixit
>>>> www.hotwaxsystems.com
>>>>
>>>> On Wed, Jun 8, 2016 at 7:12 PM, Mridul Pathak <
>>>> mridul.pathak@hotwaxsystems.com> wrote:
>>>>
>>>>> +1
>>>>>
>>>>> Makes perfect sense.
>>>>>
>>>>> --
>>>>> Thanks & Regards,
>>>>> Mridul Pathak
>>>>> Senior Manager
>>>>> HotWax Systems
>>>>> http://www.hotwaxsystems.com
>>>>>
>>>>>> On Jun 8, 2016, at 2:41 PM, Taher Alkhateeb <
>>>> slidingfilaments@gmail.com>
>>>>> wrote:
>>>>>> Hello Everyone,
>>>>>>
>>>>>> After refactoring the start component and while starting on the base
>>>>>> component I realized that the testing framework for OFBiz is not
>> good.
>>>>> You
>>>>>> cannot do real test driven development or red-green-refactor with
>> the
>>>>>> current setup, hence my proposal to change it. I explain below:
>>>>>>
>>>>>> Problem with current design
>>>>>> ----------------------------------------
>>>>>> - What we have right now is not unit tests, it's really integration
>>>>> tests.
>>>>>> You have to start the framework, the database, the service engine,
>> the
>>>>>> entity engine and pretty much everything.
>>>>>> - Testing is very slow, because it's an integration test as I
>>>> mentioned
>>>>>> above. 10 minutes on a good computer!
>>>>>> - There is zero mocking! We actually have to --load-data for things
>> to
>>>>>> work. Again, these are integration tests.
>>>>>> - Too complex: Integration tests by their nature are grabbing too
>>>> much.
>>>>>> Mind you, I am not objecting to integration tests (I actually like
>>>> them)
>>>>>> but I am objecting to not having real unit-tests. Unit tests should
>>>> all
>>>>> run
>>>>>> in a few seconds.
>>>>>>
>>>>>> Proposed solution
>>>>>> --------------------------
>>>>>> - We keep what is considered real integration tests the way they are
>>>>> right
>>>>>> now and keep using them
>>>>>> - We move what should be unit tests into simple JUnit classes, and
>> we
>>>> do
>>>>>> not run them using java -jar ofbiz.jar --test, but instead run them
>>>>>> directly from the build.xml script, so these files are not
>> identified
>>>> in
>>>>>> any XML document, but are simply called immediately from the build
>>>>> scripts.
>>>>>> - We clearly mark the difference between integration tests and unit
>>>> tests
>>>>>> (inside the source files or in the suite declarations).
>>>>>> - We change the run-tests target in build.xml to run both unit tests
>>>> and
>>>>>> integration tests.
>>>>>>
>>>>>> I intend to heavily refactor the framework and I would feel better
>>>> about
>>>>>> introducing this change while refactoring. What do you guys think?
>>>> Ideas?
>>>>>> Suggestions? Approvals and thumbs up?
>>>>>>
>>>>>> Regards,
>>>>>>
>>>>>> Taher Alkhateeb
>>>>>
>>>

Re: Proposal to modify the testing framework for OFBiz

[Jacopo Cappellato <ja...@hotwaxsystems.com>]

It sounds a reasonable approach to me.
Especially because it will let you proceed with the important work you are
contributing without loosing the momentum to study old configuration code
that would require some cleanups regardless.

Thanks,

Jacopo

On Fri, Jun 10, 2016 at 4:16 PM, Taher Alkhateeb <slidingfilaments@gmail.com
> wrote:

> Hello Everyone,
>
> I was able to get a few tests running and this is very doable. But I faced
> a big problem in designing the testing framework because of ANT.
>
> The problem
> ----------------
> The way the build scripts are designed in OFBiz are very complex. A master
> file calls other files which call other files. And in the middle you have
> external libraries (ant-contrib) and macros, and variables, and class path
> declarations, and and and ....
>
> I cannot declare the tests programmatically (with JUnit test suites)
> because this means lower level components would depend on higher level
> components. So I have to do it in ANT, by navigating this maze of build
> scripts, and it was a headache for me just to read them, let alone modify
> them to create a testing framework.
>
> Suggested Solution
> ------------------------
> I suggest to implement the testing framework in Gradle, and simply call it
> from within ant. This is a middle solution that sustains ant for now, but
> can allow us to switch out later.
>
> This means I will just add one more file called build.gradle in the top
> level directory, and figure out the business logic for calling the test
> suites from that file
>
> I look forward to your feedback.
>
> Regards,
>
> Taher Alkhateeb
>
> On Wed, Jun 8, 2016 at 6:00 PM, Taher Alkhateeb <
> slidingfilaments@gmail.com>
> wrote:
>
> > Hi Everyone,
> >
> > Thank you all for your support, JIRA created in
> > https://issues.apache.org/jira/browse/OFBIZ-7254
> >
> > I will start working on it and try to implement ASAP to get my focus back
> > on refactoring.
> >
> > Cheers!
> >
> > Taher Alkhateeb
> >
> > On Wed, Jun 8, 2016 at 4:58 PM, Deepak Dixit <
> > deepak.dixit@hotwaxsystems.com> wrote:
> >
> >> +1
> >>
> >> Thanks & Regards
> >> --
> >> Deepak Dixit
> >> www.hotwaxsystems.com
> >>
> >> On Wed, Jun 8, 2016 at 7:12 PM, Mridul Pathak <
> >> mridul.pathak@hotwaxsystems.com> wrote:
> >>
> >> > +1
> >> >
> >> > Makes perfect sense.
> >> >
> >> > --
> >> > Thanks & Regards,
> >> > Mridul Pathak
> >> > Senior Manager
> >> > HotWax Systems
> >> > http://www.hotwaxsystems.com
> >> >
> >> > > On Jun 8, 2016, at 2:41 PM, Taher Alkhateeb <
> >> slidingfilaments@gmail.com>
> >> > wrote:
> >> > >
> >> > > Hello Everyone,
> >> > >
> >> > > After refactoring the start component and while starting on the base
> >> > > component I realized that the testing framework for OFBiz is not
> good.
> >> > You
> >> > > cannot do real test driven development or red-green-refactor with
> the
> >> > > current setup, hence my proposal to change it. I explain below:
> >> > >
> >> > > Problem with current design
> >> > > ----------------------------------------
> >> > > - What we have right now is not unit tests, it's really integration
> >> > tests.
> >> > > You have to start the framework, the database, the service engine,
> the
> >> > > entity engine and pretty much everything.
> >> > > - Testing is very slow, because it's an integration test as I
> >> mentioned
> >> > > above. 10 minutes on a good computer!
> >> > > - There is zero mocking! We actually have to --load-data for things
> to
> >> > > work. Again, these are integration tests.
> >> > > - Too complex: Integration tests by their nature are grabbing too
> >> much.
> >> > > Mind you, I am not objecting to integration tests (I actually like
> >> them)
> >> > > but I am objecting to not having real unit-tests. Unit tests should
> >> all
> >> > run
> >> > > in a few seconds.
> >> > >
> >> > > Proposed solution
> >> > > --------------------------
> >> > > - We keep what is considered real integration tests the way they are
> >> > right
> >> > > now and keep using them
> >> > > - We move what should be unit tests into simple JUnit classes, and
> we
> >> do
> >> > > not run them using java -jar ofbiz.jar --test, but instead run them
> >> > > directly from the build.xml script, so these files are not
> identified
> >> in
> >> > > any XML document, but are simply called immediately from the build
> >> > scripts.
> >> > > - We clearly mark the difference between integration tests and unit
> >> tests
> >> > > (inside the source files or in the suite declarations).
> >> > > - We change the run-tests target in build.xml to run both unit tests
> >> and
> >> > > integration tests.
> >> > >
> >> > > I intend to heavily refactor the framework and I would feel better
> >> about
> >> > > introducing this change while refactoring. What do you guys think?
> >> Ideas?
> >> > > Suggestions? Approvals and thumbs up?
> >> > >
> >> > > Regards,
> >> > >
> >> > > Taher Alkhateeb
> >> >
> >> >
> >>
> >
> >
>

Re: Proposal to modify the testing framework for OFBiz

[Taher Alkhateeb <sl...@gmail.com>]

Hello Everyone,

I was able to get a few tests running and this is very doable. But I faced
a big problem in designing the testing framework because of ANT.

The problem
----------------
The way the build scripts are designed in OFBiz are very complex. A master
file calls other files which call other files. And in the middle you have
external libraries (ant-contrib) and macros, and variables, and class path
declarations, and and and ....

I cannot declare the tests programmatically (with JUnit test suites)
because this means lower level components would depend on higher level
components. So I have to do it in ANT, by navigating this maze of build
scripts, and it was a headache for me just to read them, let alone modify
them to create a testing framework.

Suggested Solution
------------------------
I suggest to implement the testing framework in Gradle, and simply call it
from within ant. This is a middle solution that sustains ant for now, but
can allow us to switch out later.

This means I will just add one more file called build.gradle in the top
level directory, and figure out the business logic for calling the test
suites from that file

I look forward to your feedback.

Regards,

Taher Alkhateeb

On Wed, Jun 8, 2016 at 6:00 PM, Taher Alkhateeb <sl...@gmail.com>
wrote:

> Hi Everyone,
>
> Thank you all for your support, JIRA created in
> https://issues.apache.org/jira/browse/OFBIZ-7254
>
> I will start working on it and try to implement ASAP to get my focus back
> on refactoring.
>
> Cheers!
>
> Taher Alkhateeb
>
> On Wed, Jun 8, 2016 at 4:58 PM, Deepak Dixit <
> deepak.dixit@hotwaxsystems.com> wrote:
>
>> +1
>>
>> Thanks & Regards
>> --
>> Deepak Dixit
>> www.hotwaxsystems.com
>>
>> On Wed, Jun 8, 2016 at 7:12 PM, Mridul Pathak <
>> mridul.pathak@hotwaxsystems.com> wrote:
>>
>> > +1
>> >
>> > Makes perfect sense.
>> >
>> > --
>> > Thanks & Regards,
>> > Mridul Pathak
>> > Senior Manager
>> > HotWax Systems
>> > http://www.hotwaxsystems.com
>> >
>> > > On Jun 8, 2016, at 2:41 PM, Taher Alkhateeb <
>> slidingfilaments@gmail.com>
>> > wrote:
>> > >
>> > > Hello Everyone,
>> > >
>> > > After refactoring the start component and while starting on the base
>> > > component I realized that the testing framework for OFBiz is not good.
>> > You
>> > > cannot do real test driven development or red-green-refactor with the
>> > > current setup, hence my proposal to change it. I explain below:
>> > >
>> > > Problem with current design
>> > > ----------------------------------------
>> > > - What we have right now is not unit tests, it's really integration
>> > tests.
>> > > You have to start the framework, the database, the service engine, the
>> > > entity engine and pretty much everything.
>> > > - Testing is very slow, because it's an integration test as I
>> mentioned
>> > > above. 10 minutes on a good computer!
>> > > - There is zero mocking! We actually have to --load-data for things to
>> > > work. Again, these are integration tests.
>> > > - Too complex: Integration tests by their nature are grabbing too
>> much.
>> > > Mind you, I am not objecting to integration tests (I actually like
>> them)
>> > > but I am objecting to not having real unit-tests. Unit tests should
>> all
>> > run
>> > > in a few seconds.
>> > >
>> > > Proposed solution
>> > > --------------------------
>> > > - We keep what is considered real integration tests the way they are
>> > right
>> > > now and keep using them
>> > > - We move what should be unit tests into simple JUnit classes, and we
>> do
>> > > not run them using java -jar ofbiz.jar --test, but instead run them
>> > > directly from the build.xml script, so these files are not identified
>> in
>> > > any XML document, but are simply called immediately from the build
>> > scripts.
>> > > - We clearly mark the difference between integration tests and unit
>> tests
>> > > (inside the source files or in the suite declarations).
>> > > - We change the run-tests target in build.xml to run both unit tests
>> and
>> > > integration tests.
>> > >
>> > > I intend to heavily refactor the framework and I would feel better
>> about
>> > > introducing this change while refactoring. What do you guys think?
>> Ideas?
>> > > Suggestions? Approvals and thumbs up?
>> > >
>> > > Regards,
>> > >
>> > > Taher Alkhateeb
>> >
>> >
>>
>
>

Re: Proposal to modify the testing framework for OFBiz

[Taher Alkhateeb <sl...@gmail.com>]

Hi Everyone,

Thank you all for your support, JIRA created in
https://issues.apache.org/jira/browse/OFBIZ-7254

I will start working on it and try to implement ASAP to get my focus back
on refactoring.

Cheers!

Taher Alkhateeb

On Wed, Jun 8, 2016 at 4:58 PM, Deepak Dixit <deepak.dixit@hotwaxsystems.com
> wrote:

> +1
>
> Thanks & Regards
> --
> Deepak Dixit
> www.hotwaxsystems.com
>
> On Wed, Jun 8, 2016 at 7:12 PM, Mridul Pathak <
> mridul.pathak@hotwaxsystems.com> wrote:
>
> > +1
> >
> > Makes perfect sense.
> >
> > --
> > Thanks & Regards,
> > Mridul Pathak
> > Senior Manager
> > HotWax Systems
> > http://www.hotwaxsystems.com
> >
> > > On Jun 8, 2016, at 2:41 PM, Taher Alkhateeb <
> slidingfilaments@gmail.com>
> > wrote:
> > >
> > > Hello Everyone,
> > >
> > > After refactoring the start component and while starting on the base
> > > component I realized that the testing framework for OFBiz is not good.
> > You
> > > cannot do real test driven development or red-green-refactor with the
> > > current setup, hence my proposal to change it. I explain below:
> > >
> > > Problem with current design
> > > ----------------------------------------
> > > - What we have right now is not unit tests, it's really integration
> > tests.
> > > You have to start the framework, the database, the service engine, the
> > > entity engine and pretty much everything.
> > > - Testing is very slow, because it's an integration test as I mentioned
> > > above. 10 minutes on a good computer!
> > > - There is zero mocking! We actually have to --load-data for things to
> > > work. Again, these are integration tests.
> > > - Too complex: Integration tests by their nature are grabbing too much.
> > > Mind you, I am not objecting to integration tests (I actually like
> them)
> > > but I am objecting to not having real unit-tests. Unit tests should all
> > run
> > > in a few seconds.
> > >
> > > Proposed solution
> > > --------------------------
> > > - We keep what is considered real integration tests the way they are
> > right
> > > now and keep using them
> > > - We move what should be unit tests into simple JUnit classes, and we
> do
> > > not run them using java -jar ofbiz.jar --test, but instead run them
> > > directly from the build.xml script, so these files are not identified
> in
> > > any XML document, but are simply called immediately from the build
> > scripts.
> > > - We clearly mark the difference between integration tests and unit
> tests
> > > (inside the source files or in the suite declarations).
> > > - We change the run-tests target in build.xml to run both unit tests
> and
> > > integration tests.
> > >
> > > I intend to heavily refactor the framework and I would feel better
> about
> > > introducing this change while refactoring. What do you guys think?
> Ideas?
> > > Suggestions? Approvals and thumbs up?
> > >
> > > Regards,
> > >
> > > Taher Alkhateeb
> >
> >
>

Re: Proposal to modify the testing framework for OFBiz

[Deepak Dixit <de...@hotwaxsystems.com>]

+1

Thanks & Regards
--
Deepak Dixit
www.hotwaxsystems.com

On Wed, Jun 8, 2016 at 7:12 PM, Mridul Pathak <
mridul.pathak@hotwaxsystems.com> wrote:

> +1
>
> Makes perfect sense.
>
> --
> Thanks & Regards,
> Mridul Pathak
> Senior Manager
> HotWax Systems
> http://www.hotwaxsystems.com
>
> > On Jun 8, 2016, at 2:41 PM, Taher Alkhateeb <sl...@gmail.com>
> wrote:
> >
> > Hello Everyone,
> >
> > After refactoring the start component and while starting on the base
> > component I realized that the testing framework for OFBiz is not good.
> You
> > cannot do real test driven development or red-green-refactor with the
> > current setup, hence my proposal to change it. I explain below:
> >
> > Problem with current design
> > ----------------------------------------
> > - What we have right now is not unit tests, it's really integration
> tests.
> > You have to start the framework, the database, the service engine, the
> > entity engine and pretty much everything.
> > - Testing is very slow, because it's an integration test as I mentioned
> > above. 10 minutes on a good computer!
> > - There is zero mocking! We actually have to --load-data for things to
> > work. Again, these are integration tests.
> > - Too complex: Integration tests by their nature are grabbing too much.
> > Mind you, I am not objecting to integration tests (I actually like them)
> > but I am objecting to not having real unit-tests. Unit tests should all
> run
> > in a few seconds.
> >
> > Proposed solution
> > --------------------------
> > - We keep what is considered real integration tests the way they are
> right
> > now and keep using them
> > - We move what should be unit tests into simple JUnit classes, and we do
> > not run them using java -jar ofbiz.jar --test, but instead run them
> > directly from the build.xml script, so these files are not identified in
> > any XML document, but are simply called immediately from the build
> scripts.
> > - We clearly mark the difference between integration tests and unit tests
> > (inside the source files or in the suite declarations).
> > - We change the run-tests target in build.xml to run both unit tests and
> > integration tests.
> >
> > I intend to heavily refactor the framework and I would feel better about
> > introducing this change while refactoring. What do you guys think? Ideas?
> > Suggestions? Approvals and thumbs up?
> >
> > Regards,
> >
> > Taher Alkhateeb
>
>

Re: Proposal to modify the testing framework for OFBiz

[Mridul Pathak <mr...@hotwaxsystems.com>]

+1

Makes perfect sense.

--
Thanks & Regards,
Mridul Pathak
Senior Manager
HotWax Systems
http://www.hotwaxsystems.com

> On Jun 8, 2016, at 2:41 PM, Taher Alkhateeb <sl...@gmail.com> wrote:
> 
> Hello Everyone,
> 
> After refactoring the start component and while starting on the base
> component I realized that the testing framework for OFBiz is not good. You
> cannot do real test driven development or red-green-refactor with the
> current setup, hence my proposal to change it. I explain below:
> 
> Problem with current design
> ----------------------------------------
> - What we have right now is not unit tests, it's really integration tests.
> You have to start the framework, the database, the service engine, the
> entity engine and pretty much everything.
> - Testing is very slow, because it's an integration test as I mentioned
> above. 10 minutes on a good computer!
> - There is zero mocking! We actually have to --load-data for things to
> work. Again, these are integration tests.
> - Too complex: Integration tests by their nature are grabbing too much.
> Mind you, I am not objecting to integration tests (I actually like them)
> but I am objecting to not having real unit-tests. Unit tests should all run
> in a few seconds.
> 
> Proposed solution
> --------------------------
> - We keep what is considered real integration tests the way they are right
> now and keep using them
> - We move what should be unit tests into simple JUnit classes, and we do
> not run them using java -jar ofbiz.jar --test, but instead run them
> directly from the build.xml script, so these files are not identified in
> any XML document, but are simply called immediately from the build scripts.
> - We clearly mark the difference between integration tests and unit tests
> (inside the source files or in the suite declarations).
> - We change the run-tests target in build.xml to run both unit tests and
> integration tests.
> 
> I intend to heavily refactor the framework and I would feel better about
> introducing this change while refactoring. What do you guys think? Ideas?
> Suggestions? Approvals and thumbs up?
> 
> Regards,
> 
> Taher Alkhateeb