You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@ofbiz.apache.org by ja...@apache.org on 2012/04/18 09:57:46 UTC
svn commit: r1327411 - in /ofbiz/trunk: bin/ tools/functions.sh
tools/git-rebase-runner.sh
Author: jacopoc
Date: Wed Apr 18 07:57:46 2012
New Revision: 1327411
URL: http://svn.apache.org/viewvc?rev=1327411&view=rev
Log:
Removed bin folder and moved its content to the tools folder: this was discussed extensively in the dev list recently.
Added:
ofbiz/trunk/tools/functions.sh
- copied unchanged from r1327403, ofbiz/trunk/bin/functions.sh
ofbiz/trunk/tools/git-rebase-runner.sh
- copied unchanged from r1327403, ofbiz/trunk/bin/git-rebase-runner.sh
Removed:
ofbiz/trunk/bin/
Re: svn commit: r1327411 - in /ofbiz/trunk: bin/ tools/functions.sh tools/git-rebase-runner.sh
Posted by Jacques Le Roux <ja...@les7arts.com>.
Just buy a pair of SSDs, I can recommend Crucial M4 128Gb.
My old box (Intel Core Duo E8300 2.9 Mz from 2008) with XP and 4Go RAM is still happy, and it seems for long now... :o)
The disk subsystem has always been the bootleneck, not only on Windows...
The complete test takes me 32 minutes. I think with a faster/newer CPU could be twice less. I don't care, changing the machines does
not worth it (all that to move again, pfew I get too old, thanks SSDs...)
Jacques
From: "Adrian Crum" <ad...@sandglass-software.com>
> On 4/18/2012 5:43 PM, Adam Heath wrote:
>> On 04/18/2012 10:55 AM, Erwan de FERRIERES wrote:
>>> Le 18/04/2012 17:21, Adam Heath a écrit :
>>>> On 04/18/2012 02:57 AM, jacopoc@apache.org wrote:
>>>>> Author: jacopoc
>>>>> Date: Wed Apr 18 07:57:46 2012
>>>>> New Revision: 1327411
>>>>>
>>>>> URL: http://svn.apache.org/viewvc?rev=1327411&view=rev
>>>>> Log:
>>>>> Removed bin folder and moved its content to the tools folder: this
>>>>> was discussed extensively in the dev list recently.
>>>>>
>>>>> Added:
>>>>> ofbiz/trunk/tools/functions.sh
>>>>> - copied unchanged from r1327403, ofbiz/trunk/bin/functions.sh
>>>>> ofbiz/trunk/tools/git-rebase-runner.sh
>>>>> - copied unchanged from r1327403,
>>>>> ofbiz/trunk/bin/git-rebase-runner.sh
>>>>> Removed:
>>>>> ofbiz/trunk/bin/
>>>> Sorry, missed that. I've been away from ofbiz too long.
>>>>
>>>>
>>> So welcome back, then !
>> The main reason is the ofbiz community is not paying attention to how
>> slow the software has gotten. It currently takes my desktop 1 hour to
>> run a full test suite. clean, load-demo(30m), run-tests(30m). It
>> used to take 12-15m. This has caused me to not do much, as it takes
>> to long to do anything.
>
> Hard disk speed is what helps. With my high-speed SATA RAID array I can build a fresh checkout and load demo data in less than 5
> minutes. Ant clean-data plus ant load-demo takes 4 minutes. The tests take 25 minutes - but that is mainly due to the many timing
> loops involved. A faster computer will not solve that.
>
> -Adrian
>
>
Re: svn commit: r1327411 - in /ofbiz/trunk: bin/ tools/functions.sh tools/git-rebase-runner.sh
Posted by Jacques Le Roux <ja...@les7arts.com>.
From: "Adam Heath" <do...@brainfood.com>
> On 04/18/2012 11:56 AM, Adrian Crum wrote:
>> On 4/18/2012 5:43 PM, Adam Heath wrote:
>>> On 04/18/2012 10:55 AM, Erwan de FERRIERES wrote:
>>>> Le 18/04/2012 17:21, Adam Heath a écrit :
>>>>> On 04/18/2012 02:57 AM, jacopoc@apache.org wrote:
>>>>>> Author: jacopoc
>>>>>> Date: Wed Apr 18 07:57:46 2012
>>>>>> New Revision: 1327411
>>>>>>
>>>>>> URL: http://svn.apache.org/viewvc?rev=1327411&view=rev
>>>>>> Log:
>>>>>> Removed bin folder and moved its content to the tools folder: this
>>>>>> was discussed extensively in the dev list recently.
>>>>>>
>>>>>> Added:
>>>>>> ofbiz/trunk/tools/functions.sh
>>>>>> - copied unchanged from r1327403,
>>>>>> ofbiz/trunk/bin/functions.sh
>>>>>> ofbiz/trunk/tools/git-rebase-runner.sh
>>>>>> - copied unchanged from r1327403,
>>>>>> ofbiz/trunk/bin/git-rebase-runner.sh
>>>>>> Removed:
>>>>>> ofbiz/trunk/bin/
>>>>> Sorry, missed that. I've been away from ofbiz too long.
>>>>>
>>>>>
>>>> So welcome back, then !
>>> The main reason is the ofbiz community is not paying attention to how
>>> slow the software has gotten. It currently takes my desktop 1 hour to
>>> run a full test suite. clean, load-demo(30m), run-tests(30m). It
>>> used to take 12-15m. This has caused me to not do much, as it takes
>>> to long to do anything.
>>
>> Hard disk speed is what helps. With my high-speed SATA RAID array I
>> can build a fresh checkout and load demo data in less than 5 minutes.
>> Ant clean-data plus ant load-demo takes 4 minutes. The tests take 25
>> minutes - but that is mainly due to the many timing loops involved. A
>> faster computer will not solve that.
>
> Believe me, this is an *ofbiz* problem, not a hardware problem. It
> definately used to take 15m to do a full run(ant clean-all, ant
> install-demo, ant run-tests). 15m was sufficient for me to start a
> loop against a whole series of git commits, go away to lunch, or drive
> home, and log back in to see 10 test runs all happily finished.
>
> Things have changed in ofbiz, so that now it takes a *huge* amount of
> time. Some of that is related to a larger amount of code overflowing
> the cpu's L2 cache. Some of that is related to catalina startup, when
> it tries to seed the sessions randomness, and the local machine runs
> out of entropy.
>
> I suggest that every so often software gets run on older computers;
> this allows for poorly written software to be tested in a more
> real-world situation, and then moree problems can be detected and fixed.
>
> ps: I am going to switch <if> in ant *back* to javascript, *away* from
> ant-contrib. The latter is significantly slower when using a system
> installed ant.
Maybe simply because there are more things in OFBiz. Then the current slim-down iteration should help...
Jacques
Re: svn commit: r1327411 - in /ofbiz/trunk: bin/ tools/functions.sh tools/git-rebase-runner.sh
Posted by Scott Gray <sc...@hotwaxmedia.com>.
On 19/04/2012, at 5:54 AM, Adam Heath wrote:
> On 04/18/2012 12:46 PM, Jacques Le Roux wrote:
>> From: "Adam Heath" <do...@brainfood.com>
>>> On 04/18/2012 12:28 PM, Jacopo Cappellato wrote:
>>>> On Apr 18, 2012, at 7:20 PM, Adam Heath wrote:
>>>>
>>>>> ps: I am going to switch <if> in ant *back* to javascript, *away*
>>>>> from
>>>>> ant-contrib. The latter is significantly slower when using a system
>>>>> installed ant.
>>>>
>>>> No please, don't do that.
>
> You didn't give a reason as to why I shouldn't switch back. If it's
> because of things not working(at all), then list those problems. I've
> already discovered some cases of embedded jar paths in the build.xml
> files that are out-of-date(I can't do cd framework/base; ../../ant
> tests-cobertura, but using the system-ant does work(once I fix the
> broken paths).
Have you considered working with the ant project at all to fix the issue? The problem we have and I think we want to address is simply a case of way too much code in this project trying to do way too many things and being maintained by nowhere near enough people with the expertise to do so. We're attempting to take a good hard look at where we can pass the responsibility for maintaining that functionality off to more appropriate parties. Unless there is a very good reason for not contributing improvements back to ant then I can't see why it wouldn't more sense to do so.
>>> I've pointed out a problem, and I have a solution. Don't just respond
>>> saying don't do that, without a reason. That isn't enough to stop me.
>>>
>>> ant-contrib If-ant.js
>>> (system-ant)
>>> Apache Ant version 1.8.0 compiled on March 11 2010
>>> time ant clean 18.131s 8.291s
>>> time ant build 44.072s 34.749s
>>> time ant build 20.228s 10.559s
>>> time ant clean 18.222s 8.490s
>>> (local-ant)
>>> Apache Ant(TM) version 1.8.3 compiled on February 26 2012
>>> time ./ant clean 3.575s 3.218s
>>> time ./ant build 28.985s 28.909s
>>> time ./ant build 5.486s 5.232s
>>> time ./ant clean 3.876s 3.378s
>>>
>>> The first ant clean is run when the system is already cleaned. The
>>> second ant build is also run when everything is already built. It's a
>>> way to test what happens when nothing is done.
>>
>>> first ant clean
>> ie with (system-ant)?
>
> The first 4 runs are with the system ant. The second 4 are with the
> local ant. notice the './' on the second set. That's the clue.
>
>>> second ant build
>> ie with (local-ant)?
>>
>> I don't see much diff with (local-ant). why not using the embedded ant?
>
> They are all slightly slower. And that's not noise, it was consistent
> in my runs.
Re: svn commit: r1327411 - in /ofbiz/trunk: bin/ tools/functions.sh tools/git-rebase-runner.sh
Posted by Jacques Le Roux <ja...@les7arts.com>.
From: "Adam Heath" <do...@brainfood.com>
> On 04/18/2012 12:46 PM, Jacques Le Roux wrote:
>> From: "Adam Heath" <do...@brainfood.com>
>>> On 04/18/2012 12:28 PM, Jacopo Cappellato wrote:
>>>> On Apr 18, 2012, at 7:20 PM, Adam Heath wrote:
>>>>
>>>>> ps: I am going to switch <if> in ant *back* to javascript, *away*
>>>>> from
>>>>> ant-contrib. The latter is significantly slower when using a system
>>>>> installed ant.
>>>>
>>>> No please, don't do that.
>
> You didn't give a reason as to why I shouldn't switch back. If it's
> because of things not working(at all), then list those problems. I've
> already discovered some cases of embedded jar paths in the build.xml
> files that are out-of-date(I can't do cd framework/base; ../../ant
> tests-cobertura, but using the system-ant does work(once I fix the
> broken paths).
>
>>> I've pointed out a problem, and I have a solution. Don't just respond
>>> saying don't do that, without a reason. That isn't enough to stop me.
>>>
>>> ant-contrib If-ant.js
>>> (system-ant)
>>> Apache Ant version 1.8.0 compiled on March 11 2010
>>> time ant clean 18.131s 8.291s
>>> time ant build 44.072s 34.749s
>>> time ant build 20.228s 10.559s
>>> time ant clean 18.222s 8.490s
>>> (local-ant)
>>> Apache Ant(TM) version 1.8.3 compiled on February 26 2012
>>> time ./ant clean 3.575s 3.218s
>>> time ./ant build 28.985s 28.909s
>>> time ./ant build 5.486s 5.232s
>>> time ./ant clean 3.876s 3.378s
>>>
>>> The first ant clean is run when the system is already cleaned. The
>>> second ant build is also run when everything is already built. It's a
>>> way to test what happens when nothing is done.
>>
>>> first ant clean
>> ie with (system-ant)?
>
> The first 4 runs are with the system ant. The second 4 are with the
> local ant. notice the './' on the second set. That's the clue.
>
>>> second ant build
>> ie with (local-ant)?
>>
>> I don't see much diff with (local-ant). why not using the embedded ant?
>
> They are all slightly slower. And that's not noise, it was consistent
> in my runs.
I'd not worry about so low diff. respectively:
+11 %
+0.2 %
+4 %
+14 %
not significant enough to take action IMO (don't fix when it works)
I don't remember why I prefered ant-contrib,but I can digg deeper...
Jacques
Re: svn commit: r1327411 - in /ofbiz/trunk: bin/ tools/functions.sh
tools/git-rebase-runner.sh
Posted by Adam Heath <do...@brainfood.com>.
On 04/18/2012 12:46 PM, Jacques Le Roux wrote:
> From: "Adam Heath" <do...@brainfood.com>
>> On 04/18/2012 12:28 PM, Jacopo Cappellato wrote:
>>> On Apr 18, 2012, at 7:20 PM, Adam Heath wrote:
>>>
>>>> ps: I am going to switch <if> in ant *back* to javascript, *away*
>>>> from
>>>> ant-contrib. The latter is significantly slower when using a system
>>>> installed ant.
>>>
>>> No please, don't do that.
You didn't give a reason as to why I shouldn't switch back. If it's
because of things not working(at all), then list those problems. I've
already discovered some cases of embedded jar paths in the build.xml
files that are out-of-date(I can't do cd framework/base; ../../ant
tests-cobertura, but using the system-ant does work(once I fix the
broken paths).
>> I've pointed out a problem, and I have a solution. Don't just respond
>> saying don't do that, without a reason. That isn't enough to stop me.
>>
>> ant-contrib If-ant.js
>> (system-ant)
>> Apache Ant version 1.8.0 compiled on March 11 2010
>> time ant clean 18.131s 8.291s
>> time ant build 44.072s 34.749s
>> time ant build 20.228s 10.559s
>> time ant clean 18.222s 8.490s
>> (local-ant)
>> Apache Ant(TM) version 1.8.3 compiled on February 26 2012
>> time ./ant clean 3.575s 3.218s
>> time ./ant build 28.985s 28.909s
>> time ./ant build 5.486s 5.232s
>> time ./ant clean 3.876s 3.378s
>>
>> The first ant clean is run when the system is already cleaned. The
>> second ant build is also run when everything is already built. It's a
>> way to test what happens when nothing is done.
>
>> first ant clean
> ie with (system-ant)?
The first 4 runs are with the system ant. The second 4 are with the
local ant. notice the './' on the second set. That's the clue.
>> second ant build
> ie with (local-ant)?
>
> I don't see much diff with (local-ant). why not using the embedded ant?
They are all slightly slower. And that's not noise, it was consistent
in my runs.
Re: svn commit: r1327411 - in /ofbiz/trunk: bin/ tools/functions.sh tools/git-rebase-runner.sh
Posted by Jacques Le Roux <ja...@les7arts.com>.
From: "Adam Heath" <do...@brainfood.com>
> On 04/18/2012 12:28 PM, Jacopo Cappellato wrote:
>> On Apr 18, 2012, at 7:20 PM, Adam Heath wrote:
>>
>>> ps: I am going to switch <if> in ant *back* to javascript, *away* from
>>> ant-contrib. The latter is significantly slower when using a system
>>> installed ant.
>>
>> No please, don't do that.
>
> I've pointed out a problem, and I have a solution. Don't just respond
> saying don't do that, without a reason. That isn't enough to stop me.
>
> ant-contrib If-ant.js
> (system-ant)
> Apache Ant version 1.8.0 compiled on March 11 2010
> time ant clean 18.131s 8.291s
> time ant build 44.072s 34.749s
> time ant build 20.228s 10.559s
> time ant clean 18.222s 8.490s
> (local-ant)
> Apache Ant(TM) version 1.8.3 compiled on February 26 2012
> time ./ant clean 3.575s 3.218s
> time ./ant build 28.985s 28.909s
> time ./ant build 5.486s 5.232s
> time ./ant clean 3.876s 3.378s
>
> The first ant clean is run when the system is already cleaned. The
> second ant build is also run when everything is already built. It's a
> way to test what happens when nothing is done.
>first ant clean
ie with (system-ant)?
> second ant build
ie with (local-ant)?
I don't see much diff with (local-ant).
why not using the embedded ant?
Jacques
Re: svn commit: r1327411 - in /ofbiz/trunk: bin/ tools/functions.sh
tools/git-rebase-runner.sh
Posted by Adam Heath <do...@brainfood.com>.
On 04/18/2012 11:33 PM, Jacopo Cappellato wrote:
>
> On Apr 18, 2012, at 7:37 PM, Adam Heath wrote:
>
>> On 04/18/2012 12:28 PM, Jacopo Cappellato wrote:
>>> On Apr 18, 2012, at 7:20 PM, Adam Heath wrote:
>>>
>>>> ps: I am going to switch <if> in ant *back* to javascript, *away* from
>>>> ant-contrib. The latter is significantly slower when using a system
>>>> installed ant.
>>>
>>> No please, don't do that.
>>
>> I've pointed out a problem, and I have a solution. Don't just respond
>> saying don't do that, without a reason. That isn't enough to stop me.
>
> I thought that all the recent threads about code refactoring, moving out custom code etc... were enough to motivate my response; but it seems that you ignored all of them and now pretend a response from me, for such a trivial and small topic.
> The reason is that we want to keep code as clean as possible, as slim as possible (ok, I know it looks silly if you look at the code now... but we are determined to work on this): using an external tool for this instead of custom code is a step in this direction.
> The stats below show that the problem is trivial: it is only relevant when you use your own ant that is not what we recommend; and it is something that should be fixed with the ant community: in fact we want to increase the cooperation with other communities (especially if they are in the ASF world) rather than blobbing everything into OFBiz.
I didn't ignore them all, I've been away and haven't had time to keep
uptodate.
I haven't committed anything for this yet, once you said you had
issues with the switch. I have 130 commits in my local ofbiz tree,
keeping some out is *not* a problem (for instance, that salted-based
password change I've kept around for about 2 years).
This explanation is a *good* one. And yes, I do realize that the
problem is with ant, even system level shouldn't be running as slow as
it is.
Re: svn commit: r1327411 - in /ofbiz/trunk: bin/ tools/functions.sh tools/git-rebase-runner.sh
Posted by Jacopo Cappellato <ja...@hotwaxmedia.com>.
On Apr 18, 2012, at 7:37 PM, Adam Heath wrote:
> On 04/18/2012 12:28 PM, Jacopo Cappellato wrote:
>> On Apr 18, 2012, at 7:20 PM, Adam Heath wrote:
>>
>>> ps: I am going to switch <if> in ant *back* to javascript, *away* from
>>> ant-contrib. The latter is significantly slower when using a system
>>> installed ant.
>>
>> No please, don't do that.
>
> I've pointed out a problem, and I have a solution. Don't just respond
> saying don't do that, without a reason. That isn't enough to stop me.
I thought that all the recent threads about code refactoring, moving out custom code etc... were enough to motivate my response; but it seems that you ignored all of them and now pretend a response from me, for such a trivial and small topic.
The reason is that we want to keep code as clean as possible, as slim as possible (ok, I know it looks silly if you look at the code now... but we are determined to work on this): using an external tool for this instead of custom code is a step in this direction.
The stats below show that the problem is trivial: it is only relevant when you use your own ant that is not what we recommend; and it is something that should be fixed with the ant community: in fact we want to increase the cooperation with other communities (especially if they are in the ASF world) rather than blobbing everything into OFBiz.
Jacopo
>
> ant-contrib If-ant.js
> (system-ant)
> Apache Ant version 1.8.0 compiled on March 11 2010
> time ant clean 18.131s 8.291s
> time ant build 44.072s 34.749s
> time ant build 20.228s 10.559s
> time ant clean 18.222s 8.490s
> (local-ant)
> Apache Ant(TM) version 1.8.3 compiled on February 26 2012
> time ./ant clean 3.575s 3.218s
> time ./ant build 28.985s 28.909s
> time ./ant build 5.486s 5.232s
> time ./ant clean 3.876s 3.378s
>
> The first ant clean is run when the system is already cleaned. The
> second ant build is also run when everything is already built. It's a
> way to test what happens when nothing is done.
Re: svn commit: r1327411 - in /ofbiz/trunk: bin/ tools/functions.sh
tools/git-rebase-runner.sh
Posted by Adam Heath <do...@brainfood.com>.
On 04/18/2012 12:28 PM, Jacopo Cappellato wrote:
> On Apr 18, 2012, at 7:20 PM, Adam Heath wrote:
>
>> ps: I am going to switch <if> in ant *back* to javascript, *away* from
>> ant-contrib. The latter is significantly slower when using a system
>> installed ant.
>
> No please, don't do that.
I've pointed out a problem, and I have a solution. Don't just respond
saying don't do that, without a reason. That isn't enough to stop me.
ant-contrib If-ant.js
(system-ant)
Apache Ant version 1.8.0 compiled on March 11 2010
time ant clean 18.131s 8.291s
time ant build 44.072s 34.749s
time ant build 20.228s 10.559s
time ant clean 18.222s 8.490s
(local-ant)
Apache Ant(TM) version 1.8.3 compiled on February 26 2012
time ./ant clean 3.575s 3.218s
time ./ant build 28.985s 28.909s
time ./ant build 5.486s 5.232s
time ./ant clean 3.876s 3.378s
The first ant clean is run when the system is already cleaned. The
second ant build is also run when everything is already built. It's a
way to test what happens when nothing is done.
Re: svn commit: r1327411 - in /ofbiz/trunk: bin/ tools/functions.sh tools/git-rebase-runner.sh
Posted by Jacopo Cappellato <ja...@hotwaxmedia.com>.
On Apr 18, 2012, at 7:20 PM, Adam Heath wrote:
> ps: I am going to switch <if> in ant *back* to javascript, *away* from
> ant-contrib. The latter is significantly slower when using a system
> installed ant.
No please, don't do that.
Jacopo
Re: svn commit: r1327411 - in /ofbiz/trunk: bin/ tools/functions.sh
tools/git-rebase-runner.sh
Posted by Rajbir Saini <ra...@yahoo.com>.
We resolved per theme web app problem by moving the them header/footer
to CMS. In multi-tenant e-commerce applications it is possible to have
100's of themes and it is not possible to create web application for
each theme. What is needed is create different header/footer content for
each theme and create the theme data accordingly.
Raj
On Wednesday 18 April 2012 10:54 PM, Adam Heath wrote:
> On 04/18/2012 12:20 PM, Adam Heath wrote:
>> Things have changed in ofbiz, so that now it takes a *huge* amount of
>> time. Some of that is related to a larger amount of code overflowing
>> the cpu's L2 cache. Some of that is related to catalina startup, when
>> it tries to seed the sessions randomness, and the local machine runs
>> out of entropy.
> The method of placing every single theme as a *separate* catalina
> webapp context is also rather stupid. Each context drains more random
> entropy. It'd be better to have a single theme webapp, and then each
> theme is a sub-folder of that. Since themes are separate components,
> this is not immediately simple; perhaps a servlet filter that
> redirects paths to the appropriate theme folder.
>
>
Re: svn commit: r1327411 - in /ofbiz/trunk: bin/ tools/functions.sh
tools/git-rebase-runner.sh
Posted by Adam Heath <do...@brainfood.com>.
On 04/18/2012 12:20 PM, Adam Heath wrote:
> Things have changed in ofbiz, so that now it takes a *huge* amount of
> time. Some of that is related to a larger amount of code overflowing
> the cpu's L2 cache. Some of that is related to catalina startup, when
> it tries to seed the sessions randomness, and the local machine runs
> out of entropy.
The method of placing every single theme as a *separate* catalina
webapp context is also rather stupid. Each context drains more random
entropy. It'd be better to have a single theme webapp, and then each
theme is a sub-folder of that. Since themes are separate components,
this is not immediately simple; perhaps a servlet filter that
redirects paths to the appropriate theme folder.
Re: svn commit: r1327411 - in /ofbiz/trunk: bin/ tools/functions.sh
tools/git-rebase-runner.sh
Posted by Adam Heath <do...@brainfood.com>.
On 04/18/2012 11:56 AM, Adrian Crum wrote:
> On 4/18/2012 5:43 PM, Adam Heath wrote:
>> On 04/18/2012 10:55 AM, Erwan de FERRIERES wrote:
>>> Le 18/04/2012 17:21, Adam Heath a écrit :
>>>> On 04/18/2012 02:57 AM, jacopoc@apache.org wrote:
>>>>> Author: jacopoc
>>>>> Date: Wed Apr 18 07:57:46 2012
>>>>> New Revision: 1327411
>>>>>
>>>>> URL: http://svn.apache.org/viewvc?rev=1327411&view=rev
>>>>> Log:
>>>>> Removed bin folder and moved its content to the tools folder: this
>>>>> was discussed extensively in the dev list recently.
>>>>>
>>>>> Added:
>>>>> ofbiz/trunk/tools/functions.sh
>>>>> - copied unchanged from r1327403,
>>>>> ofbiz/trunk/bin/functions.sh
>>>>> ofbiz/trunk/tools/git-rebase-runner.sh
>>>>> - copied unchanged from r1327403,
>>>>> ofbiz/trunk/bin/git-rebase-runner.sh
>>>>> Removed:
>>>>> ofbiz/trunk/bin/
>>>> Sorry, missed that. I've been away from ofbiz too long.
>>>>
>>>>
>>> So welcome back, then !
>> The main reason is the ofbiz community is not paying attention to how
>> slow the software has gotten. It currently takes my desktop 1 hour to
>> run a full test suite. clean, load-demo(30m), run-tests(30m). It
>> used to take 12-15m. This has caused me to not do much, as it takes
>> to long to do anything.
>
> Hard disk speed is what helps. With my high-speed SATA RAID array I
> can build a fresh checkout and load demo data in less than 5 minutes.
> Ant clean-data plus ant load-demo takes 4 minutes. The tests take 25
> minutes - but that is mainly due to the many timing loops involved. A
> faster computer will not solve that.
Believe me, this is an *ofbiz* problem, not a hardware problem. It
definately used to take 15m to do a full run(ant clean-all, ant
install-demo, ant run-tests). 15m was sufficient for me to start a
loop against a whole series of git commits, go away to lunch, or drive
home, and log back in to see 10 test runs all happily finished.
Things have changed in ofbiz, so that now it takes a *huge* amount of
time. Some of that is related to a larger amount of code overflowing
the cpu's L2 cache. Some of that is related to catalina startup, when
it tries to seed the sessions randomness, and the local machine runs
out of entropy.
I suggest that every so often software gets run on older computers;
this allows for poorly written software to be tested in a more
real-world situation, and then moree problems can be detected and fixed.
ps: I am going to switch <if> in ant *back* to javascript, *away* from
ant-contrib. The latter is significantly slower when using a system
installed ant.
Re: svn commit: r1327411 - in /ofbiz/trunk: bin/ tools/functions.sh
tools/git-rebase-runner.sh
Posted by Adrian Crum <ad...@sandglass-software.com>.
On 4/18/2012 5:43 PM, Adam Heath wrote:
> On 04/18/2012 10:55 AM, Erwan de FERRIERES wrote:
>> Le 18/04/2012 17:21, Adam Heath a écrit :
>>> On 04/18/2012 02:57 AM, jacopoc@apache.org wrote:
>>>> Author: jacopoc
>>>> Date: Wed Apr 18 07:57:46 2012
>>>> New Revision: 1327411
>>>>
>>>> URL: http://svn.apache.org/viewvc?rev=1327411&view=rev
>>>> Log:
>>>> Removed bin folder and moved its content to the tools folder: this
>>>> was discussed extensively in the dev list recently.
>>>>
>>>> Added:
>>>> ofbiz/trunk/tools/functions.sh
>>>> - copied unchanged from r1327403, ofbiz/trunk/bin/functions.sh
>>>> ofbiz/trunk/tools/git-rebase-runner.sh
>>>> - copied unchanged from r1327403,
>>>> ofbiz/trunk/bin/git-rebase-runner.sh
>>>> Removed:
>>>> ofbiz/trunk/bin/
>>> Sorry, missed that. I've been away from ofbiz too long.
>>>
>>>
>> So welcome back, then !
> The main reason is the ofbiz community is not paying attention to how
> slow the software has gotten. It currently takes my desktop 1 hour to
> run a full test suite. clean, load-demo(30m), run-tests(30m). It
> used to take 12-15m. This has caused me to not do much, as it takes
> to long to do anything.
Hard disk speed is what helps. With my high-speed SATA RAID array I can
build a fresh checkout and load demo data in less than 5 minutes. Ant
clean-data plus ant load-demo takes 4 minutes. The tests take 25 minutes
- but that is mainly due to the many timing loops involved. A faster
computer will not solve that.
-Adrian
Re: svn commit: r1327411 - in /ofbiz/trunk: bin/ tools/functions.sh
tools/git-rebase-runner.sh
Posted by Adam Heath <do...@brainfood.com>.
On 04/19/2012 03:15 AM, Jacopo Cappellato wrote:
> Jacques,
>
> here are some details:
>
> Hardware Overview:
>
> Model Name: MacBook Pro
> Model Identifier: MacBookPro8,2
> Processor Name: Intel Core i7
> Processor Speed: 2.2 GHz
> Number of Processors: 1
> Total Number of Cores: 4
So 4 threads. That doesn't help that much, unless the test suite is
multithreaded now.
> L2 Cache (per Core): 256 KB
> L3 Cache: 6 MB
> Memory: 8 GB
>
> HD details are:
>
> Intel 6 Series Chipset:
>
> Vendor: Intel
> Product: 6 Series Chipset
> Link Speed: 6 Gigabit
> Negotiated Link Speed: 3 Gigabit
> Description: AHCI Version 1.30 Supported
>
> Hitachi HTS725050A9A362:
>
> Capacity: 500.11 GB (500,107,862,016 bytes)
> Model: Hitachi HTS725050A9A362
> Queue Depth: 32
> Rotational Rate: 7200
>
> Jacopo
>
> On Apr 19, 2012, at 9:58 AM, Jacques Le Roux wrote:
>
>> OK seems after all that I need a brand new CPU, hence a new box...
>> I must say with the SSDs when running run-tests my CPU is always at 100% (I put the run-tests process priority level to idle to be able to continue other tasks)
>> Do you have a SSD in your laptop Jacopo?
>>
>> Jacques
>>
>> From: "Jacopo Cappellato" <ja...@hotwaxmedia.com>
>>> On Apr 18, 2012, at 6:55 PM, Jacopo Cappellato wrote:
>>>
>>>> On Apr 18, 2012, at 6:43 PM, Adam Heath wrote:
>>>>
>>>>> It currently takes my desktop 1 hour to
>>>>> run a full test suite. clean, load-demo(30m), run-tests(30m). It
>>>>> used to take 12-15m.
>>>>
>>>> wow, it takes me 2-3 minutes in my laptop.
>>>>
>>>> Jacopo
>>>>
>>>
>>>
>>> To be more precise:
>>> (less than) 3 minutes for load-demo
>>> 4 minutes and 3 seconds for run-tests
>>>
>>> Jacopo
>
Re: svn commit: r1327411 - in /ofbiz/trunk: bin/ tools/functions.sh tools/git-rebase-runner.sh
Posted by Jacopo Cappellato <ja...@hotwaxmedia.com>.
Jacques,
here are some details:
Hardware Overview:
Model Name: MacBook Pro
Model Identifier: MacBookPro8,2
Processor Name: Intel Core i7
Processor Speed: 2.2 GHz
Number of Processors: 1
Total Number of Cores: 4
L2 Cache (per Core): 256 KB
L3 Cache: 6 MB
Memory: 8 GB
HD details are:
Intel 6 Series Chipset:
Vendor: Intel
Product: 6 Series Chipset
Link Speed: 6 Gigabit
Negotiated Link Speed: 3 Gigabit
Description: AHCI Version 1.30 Supported
Hitachi HTS725050A9A362:
Capacity: 500.11 GB (500,107,862,016 bytes)
Model: Hitachi HTS725050A9A362
Queue Depth: 32
Rotational Rate: 7200
Jacopo
On Apr 19, 2012, at 9:58 AM, Jacques Le Roux wrote:
> OK seems after all that I need a brand new CPU, hence a new box...
> I must say with the SSDs when running run-tests my CPU is always at 100% (I put the run-tests process priority level to idle to be able to continue other tasks)
> Do you have a SSD in your laptop Jacopo?
>
> Jacques
>
> From: "Jacopo Cappellato" <ja...@hotwaxmedia.com>
>> On Apr 18, 2012, at 6:55 PM, Jacopo Cappellato wrote:
>>
>>> On Apr 18, 2012, at 6:43 PM, Adam Heath wrote:
>>>
>>>> It currently takes my desktop 1 hour to
>>>> run a full test suite. clean, load-demo(30m), run-tests(30m). It
>>>> used to take 12-15m.
>>>
>>> wow, it takes me 2-3 minutes in my laptop.
>>>
>>> Jacopo
>>>
>>
>>
>> To be more precise:
>> (less than) 3 minutes for load-demo
>> 4 minutes and 3 seconds for run-tests
>>
>> Jacopo
Re: svn commit: r1327411 - in /ofbiz/trunk: bin/ tools/functions.sh tools/git-rebase-runner.sh
Posted by Jacques Le Roux <ja...@les7arts.com>.
OK seems after all that I need a brand new CPU, hence a new box...
I must say with the SSDs when running run-tests my CPU is always at 100% (I put the run-tests process priority level to idle to be
able to continue other tasks)
Do you have a SSD in your laptop Jacopo?
Jacques
From: "Jacopo Cappellato" <ja...@hotwaxmedia.com>
> On Apr 18, 2012, at 6:55 PM, Jacopo Cappellato wrote:
>
>> On Apr 18, 2012, at 6:43 PM, Adam Heath wrote:
>>
>>> It currently takes my desktop 1 hour to
>>> run a full test suite. clean, load-demo(30m), run-tests(30m). It
>>> used to take 12-15m.
>>
>> wow, it takes me 2-3 minutes in my laptop.
>>
>> Jacopo
>>
>
>
> To be more precise:
> (less than) 3 minutes for load-demo
> 4 minutes and 3 seconds for run-tests
>
> Jacopo
>
Re: svn commit: r1327411 - in /ofbiz/trunk: bin/ tools/functions.sh tools/git-rebase-runner.sh
Posted by Jacopo Cappellato <ja...@hotwaxmedia.com>.
On Apr 18, 2012, at 6:55 PM, Jacopo Cappellato wrote:
> On Apr 18, 2012, at 6:43 PM, Adam Heath wrote:
>
>> It currently takes my desktop 1 hour to
>> run a full test suite. clean, load-demo(30m), run-tests(30m). It
>> used to take 12-15m.
>
> wow, it takes me 2-3 minutes in my laptop.
>
> Jacopo
>
To be more precise:
(less than) 3 minutes for load-demo
4 minutes and 3 seconds for run-tests
Jacopo
Re: svn commit: r1327411 - in /ofbiz/trunk: bin/ tools/functions.sh tools/git-rebase-runner.sh
Posted by Jacopo Cappellato <ja...@hotwaxmedia.com>.
On Apr 18, 2012, at 6:43 PM, Adam Heath wrote:
> It currently takes my desktop 1 hour to
> run a full test suite. clean, load-demo(30m), run-tests(30m). It
> used to take 12-15m.
wow, it takes me 2-3 minutes in my laptop.
Jacopo
Re: svn commit: r1327411 - in /ofbiz/trunk: bin/ tools/functions.sh
tools/git-rebase-runner.sh
Posted by Erwan de FERRIERES <er...@nereide.fr>.
Le 18/04/2012 18:43, Adam Heath a écrit :
> On 04/18/2012 10:55 AM, Erwan de FERRIERES wrote:
>> Le 18/04/2012 17:21, Adam Heath a écrit :
>>> On 04/18/2012 02:57 AM, jacopoc@apache.org wrote:
>>>> Author: jacopoc
>>>> Date: Wed Apr 18 07:57:46 2012
>>>> New Revision: 1327411
>>>>
>>>> URL: http://svn.apache.org/viewvc?rev=1327411&view=rev
>>>> Log:
>>>> Removed bin folder and moved its content to the tools folder: this
>>>> was discussed extensively in the dev list recently.
>>>>
>>>> Added:
>>>> ofbiz/trunk/tools/functions.sh
>>>> - copied unchanged from r1327403, ofbiz/trunk/bin/functions.sh
>>>> ofbiz/trunk/tools/git-rebase-runner.sh
>>>> - copied unchanged from r1327403,
>>>> ofbiz/trunk/bin/git-rebase-runner.sh
>>>> Removed:
>>>> ofbiz/trunk/bin/
>>>
>>> Sorry, missed that. I've been away from ofbiz too long.
>>>
>>>
>> So welcome back, then !
>
> The main reason is the ofbiz community is not paying attention to how
> slow the software has gotten. It currently takes my desktop 1 hour to
> run a full test suite. clean, load-demo(30m), run-tests(30m). It
> used to take 12-15m. This has caused me to not do much, as it takes
> to long to do anything.
>
> The issue that helps the most is getting a larger l2-cache on the cpu.
> The load-demo time is greatly affected by derby being *way* to large,
> code-size wise.
>
> I've started doing work now, by using a virtualized instance that has
> more l2-cache, but the problem still remains.
>
Hi,
I had better stats with the new ant 1.8.3. But the times you have are
really impressive !
can't explain why...
--
Erwan de FERRIERES
www.nereide.biz
Re: svn commit: r1327411 - in /ofbiz/trunk: bin/ tools/functions.sh
tools/git-rebase-runner.sh
Posted by Adam Heath <do...@brainfood.com>.
On 04/18/2012 10:55 AM, Erwan de FERRIERES wrote:
> Le 18/04/2012 17:21, Adam Heath a écrit :
>> On 04/18/2012 02:57 AM, jacopoc@apache.org wrote:
>>> Author: jacopoc
>>> Date: Wed Apr 18 07:57:46 2012
>>> New Revision: 1327411
>>>
>>> URL: http://svn.apache.org/viewvc?rev=1327411&view=rev
>>> Log:
>>> Removed bin folder and moved its content to the tools folder: this
>>> was discussed extensively in the dev list recently.
>>>
>>> Added:
>>> ofbiz/trunk/tools/functions.sh
>>> - copied unchanged from r1327403, ofbiz/trunk/bin/functions.sh
>>> ofbiz/trunk/tools/git-rebase-runner.sh
>>> - copied unchanged from r1327403,
>>> ofbiz/trunk/bin/git-rebase-runner.sh
>>> Removed:
>>> ofbiz/trunk/bin/
>>
>> Sorry, missed that. I've been away from ofbiz too long.
>>
>>
> So welcome back, then !
The main reason is the ofbiz community is not paying attention to how
slow the software has gotten. It currently takes my desktop 1 hour to
run a full test suite. clean, load-demo(30m), run-tests(30m). It
used to take 12-15m. This has caused me to not do much, as it takes
to long to do anything.
The issue that helps the most is getting a larger l2-cache on the cpu.
The load-demo time is greatly affected by derby being *way* to large,
code-size wise.
I've started doing work now, by using a virtualized instance that has
more l2-cache, but the problem still remains.
Re: svn commit: r1327411 - in /ofbiz/trunk: bin/ tools/functions.sh
tools/git-rebase-runner.sh
Posted by Erwan de FERRIERES <er...@nereide.fr>.
Le 18/04/2012 17:21, Adam Heath a écrit :
> On 04/18/2012 02:57 AM, jacopoc@apache.org wrote:
>> Author: jacopoc
>> Date: Wed Apr 18 07:57:46 2012
>> New Revision: 1327411
>>
>> URL: http://svn.apache.org/viewvc?rev=1327411&view=rev
>> Log:
>> Removed bin folder and moved its content to the tools folder: this was discussed extensively in the dev list recently.
>>
>> Added:
>> ofbiz/trunk/tools/functions.sh
>> - copied unchanged from r1327403, ofbiz/trunk/bin/functions.sh
>> ofbiz/trunk/tools/git-rebase-runner.sh
>> - copied unchanged from r1327403, ofbiz/trunk/bin/git-rebase-runner.sh
>> Removed:
>> ofbiz/trunk/bin/
>
> Sorry, missed that. I've been away from ofbiz too long.
>
>
So welcome back, then !
--
Erwan de FERRIERES
www.nereide.biz
Re: svn commit: r1327411 - in /ofbiz/trunk: bin/ tools/functions.sh
tools/git-rebase-runner.sh
Posted by Adam Heath <do...@brainfood.com>.
On 04/18/2012 02:57 AM, jacopoc@apache.org wrote:
> Author: jacopoc
> Date: Wed Apr 18 07:57:46 2012
> New Revision: 1327411
>
> URL: http://svn.apache.org/viewvc?rev=1327411&view=rev
> Log:
> Removed bin folder and moved its content to the tools folder: this was discussed extensively in the dev list recently.
>
> Added:
> ofbiz/trunk/tools/functions.sh
> - copied unchanged from r1327403, ofbiz/trunk/bin/functions.sh
> ofbiz/trunk/tools/git-rebase-runner.sh
> - copied unchanged from r1327403, ofbiz/trunk/bin/git-rebase-runner.sh
> Removed:
> ofbiz/trunk/bin/
Sorry, missed that. I've been away from ofbiz too long.