You are viewing a plain text version of this content. The canonical link for it is here.
Posted to qa@openoffice.apache.org by Yi Xuan Liu <li...@gmail.com> on 2013/02/25 05:42:33 UTC
Call for volunteers for AOO 4.0 performance test.
Hi, all:
AOO 4.0 will release. Performance plays an important role in software
quality. Is there any volunteer who want to run performance test?
I've run AOO performance test on my own machines. I've tried on 3
platforms: Windows XP, Ubuntu 10.04, Mac Mac OS 10.7.3. The test configures
are as follows:
(1) W500; CPU:2.53 GHz; Mem: 3GB; OS: XP SP3
(2) Ubuntu; CPU: Interl® Core™ 2 Duo 2 GHz; Mem: 3GB; OS: Ubuntu 10.04
(3) Mac; CPU: Interl® Core™ 2 Duo 2 GHz; Mem: 3GB; OS: Mac OS 10.7.3
The test scenario include: AOO startup, file open, and save.
Volunteers could run performance test on other platforms.
All the automation scripts could be downloaded in AOO project. And it is
not difficult to set the automation environment. You could follow the guide:
http://wiki.openoffice.org/wiki/QA/test_automation_guide
For any questions, be free to contact with me :)
Re: Call for volunteers for AOO 4.0 performance test.
Posted by Yi Xuan Liu <li...@gmail.com>.
Great!
Thanks for your support. I haven't run project neither on other language
besides English nor on 64 bit machine.
It is a good trial. Expect for your test results :)
On Wed, Feb 27, 2013 at 11:07 PM, Marcolongo
<gi...@alice.it>wrote:
> I'm a italian volunteer. I would like to run performance tests for italian
> project.
> I have windows 8 pro 64.
> Gian Paolo Marcolongo
>
> Il 27/02/2013 15:49, Anders Kvibäck ha scritto:
>
> Hi!
>>
>> Yes, I would like to run performance tests. I've started to look into the
>> link about automated testing that you reffered to.
>> There it says that I have to disable color calibration. But is this
>> possible in Windows Vista.? I just can't find out how to do it. As far as
>> I
>> understand it this is only possible in Windows 7- which I don't have
>> unfortunately.
>>
>> Anders
>> 2013/2/26 Yi Xuan Liu <li...@gmail.com>
>>
>> Rob, I agree with you. The test environment configuration would impact
>>> the
>>> performance test results. Therefore, we should keep the test
>>> configuration
>>> unchanged between build to build.
>>>
>>> I checked performance bugs list in bugzilla. 5 performance bugs are found
>>> in automation performance test, such as memory leak and save performance
>>> issue.
>>>
>>> Lots of performance bugs are found in AOO daily usage. Therefore, for
>>> volunteers who are not willing to run automation performance, you can
>>> also
>>> report bugs for any performance issue and it will be helpful for us.
>>>
>>> On Mon, Feb 25, 2013 at 9:16 PM, Rob Weir <ro...@apache.org> wrote:
>>>
>>> On Sun, Feb 24, 2013 at 11:42 PM, Yi Xuan Liu <li...@gmail.com>
>>>> wrote:
>>>>
>>>>> Hi, all:
>>>>>
>>>>> AOO 4.0 will release. Performance plays an important role in software
>>>>> quality. Is there any volunteer who want to run performance test?
>>>>>
>>>>> I've run AOO performance test on my own machines. I've tried on 3
>>>>> platforms: Windows XP, Ubuntu 10.04, Mac Mac OS 10.7.3. The test
>>>>>
>>>> configures
>>>>
>>>>> are as follows:
>>>>>
>>>>> (1) W500; CPU:2.53 GHz; Mem: 3GB; OS: XP SP3
>>>>> (2) Ubuntu; CPU: Interl® Core™ 2 Duo 2 GHz; Mem: 3GB; OS: Ubuntu 10.04
>>>>> (3) Mac; CPU: Interl® Core™ 2 Duo 2 GHz; Mem: 3GB; OS: Mac OS 10.7.3
>>>>>
>>>>> I assume the volunteer does not need to have exactly the same machine
>>>> type as you had. But they need some stability in the configuration.
>>>> A performance test might be run first on the AOO 3.4.1 release to
>>>> establish a baseline. Then re-test on a current 4.0 snapshot build.
>>>> And then re-run on new dev snapshot builds, maybe once a week.
>>>>
>>>> The goal is to detect performance regressions early, so developers can
>>>>
>>> fix
>>>
>>>> it.
>>>>
>>>> The technical challenge here is to preserve a stable machine
>>>> configuration. If the machine changes, because of an OS upgrade, or a
>>>> changed hard drive, or a different network environment, or because of
>>>> a new anti-virus product, then that confuses things. We need to
>>>> "control all the variables".
>>>>
>>>> One approach to controlling all of the variables is to have a machine
>>>> that is used for nothing but performance testing. That way we know
>>>> the machine's base performance does not change.
>>>>
>>>> Another approach is re-run the baseline AOO 3.4.1 performance tests
>>>> each week. This is more tolerant of changes in machine configuration,
>>>> etc.
>>>>
>>>> -Rob
>>>>
>>>>
>>>> The test scenario include: AOO startup, file open, and save.
>>>>>
>>>>> Volunteers could run performance test on other platforms.
>>>>>
>>>>> All the automation scripts could be downloaded in AOO project. And it
>>>>>
>>>> is
>>>
>>>> not difficult to set the automation environment. You could follow the
>>>>>
>>>> guide:
>>>>
>>>>> http://wiki.openoffice.org/**wiki/QA/test_automation_guide<http://wiki.openoffice.org/wiki/QA/test_automation_guide>
>>>>>
>>>>> For any questions, be free to contact with me :)
>>>>>
>>>>
>
Re: Call for volunteers for AOO 4.0 performance test.
Posted by Marcolongo <gi...@alice.it>.
I'm a italian volunteer. I would like to run performance tests for
italian project.
I have windows 8 pro 64.
Gian Paolo Marcolongo
Il 27/02/2013 15:49, Anders Kvibäck ha scritto:
> Hi!
>
> Yes, I would like to run performance tests. I've started to look into the
> link about automated testing that you reffered to.
> There it says that I have to disable color calibration. But is this
> possible in Windows Vista.? I just can't find out how to do it. As far as I
> understand it this is only possible in Windows 7- which I don't have
> unfortunately.
>
> Anders
> 2013/2/26 Yi Xuan Liu <li...@gmail.com>
>
>> Rob, I agree with you. The test environment configuration would impact the
>> performance test results. Therefore, we should keep the test configuration
>> unchanged between build to build.
>>
>> I checked performance bugs list in bugzilla. 5 performance bugs are found
>> in automation performance test, such as memory leak and save performance
>> issue.
>>
>> Lots of performance bugs are found in AOO daily usage. Therefore, for
>> volunteers who are not willing to run automation performance, you can also
>> report bugs for any performance issue and it will be helpful for us.
>>
>> On Mon, Feb 25, 2013 at 9:16 PM, Rob Weir <ro...@apache.org> wrote:
>>
>>> On Sun, Feb 24, 2013 at 11:42 PM, Yi Xuan Liu <li...@gmail.com>
>>> wrote:
>>>> Hi, all:
>>>>
>>>> AOO 4.0 will release. Performance plays an important role in software
>>>> quality. Is there any volunteer who want to run performance test?
>>>>
>>>> I've run AOO performance test on my own machines. I've tried on 3
>>>> platforms: Windows XP, Ubuntu 10.04, Mac Mac OS 10.7.3. The test
>>> configures
>>>> are as follows:
>>>>
>>>> (1) W500; CPU:2.53 GHz; Mem: 3GB; OS: XP SP3
>>>> (2) Ubuntu; CPU: Interl® Core™ 2 Duo 2 GHz; Mem: 3GB; OS: Ubuntu 10.04
>>>> (3) Mac; CPU: Interl® Core™ 2 Duo 2 GHz; Mem: 3GB; OS: Mac OS 10.7.3
>>>>
>>> I assume the volunteer does not need to have exactly the same machine
>>> type as you had. But they need some stability in the configuration.
>>> A performance test might be run first on the AOO 3.4.1 release to
>>> establish a baseline. Then re-test on a current 4.0 snapshot build.
>>> And then re-run on new dev snapshot builds, maybe once a week.
>>>
>>> The goal is to detect performance regressions early, so developers can
>> fix
>>> it.
>>>
>>> The technical challenge here is to preserve a stable machine
>>> configuration. If the machine changes, because of an OS upgrade, or a
>>> changed hard drive, or a different network environment, or because of
>>> a new anti-virus product, then that confuses things. We need to
>>> "control all the variables".
>>>
>>> One approach to controlling all of the variables is to have a machine
>>> that is used for nothing but performance testing. That way we know
>>> the machine's base performance does not change.
>>>
>>> Another approach is re-run the baseline AOO 3.4.1 performance tests
>>> each week. This is more tolerant of changes in machine configuration,
>>> etc.
>>>
>>> -Rob
>>>
>>>
>>>> The test scenario include: AOO startup, file open, and save.
>>>>
>>>> Volunteers could run performance test on other platforms.
>>>>
>>>> All the automation scripts could be downloaded in AOO project. And it
>> is
>>>> not difficult to set the automation environment. You could follow the
>>> guide:
>>>> http://wiki.openoffice.org/wiki/QA/test_automation_guide
>>>>
>>>> For any questions, be free to contact with me :)
Re: Call for volunteers for AOO 4.0 performance test.
Posted by Li Feng Wang <ph...@gmail.com>.
(1)For PVT report you can record as wiki on
http://wiki.openoffice.org/wiki/QA/Report
* GUI PVT:*
1)open pvt.gui.Benchmark.xml with OO
2)compute test result
- Remove the minimum and maximum result
- Compute the average and standard deviation of the rest 5
results.* *Note:the
standard deviation is to reflect whether the results are stable.
* UNO PVT:*
1)Open pvt.uno.Conversion.xml with OO
2)Show your result without compute.
First of all, You need run on the AOO 3.4.1 release to establish a
baseline. To make your PVT result more meaningful.
(2)For test.uno.project, you need follow
http://wiki.openoffice.org/wiki/QA/test_automation_guide<http://wiki.openoffice.org/wiki/QA/test_automation_guide>
I've ran uno project on Eclipse and Command, it works fine.
From your error message, I know that your openoffice.home set with the
wrong path. openoffice.home is your default install directory. Below is the
list of default install path
Mac OS X: "*/Applications/OpenOffice.org.app/Contents"*
Linux: "*/opt/openoffice.org3*"
Windows: "*C:/Program Files/OpenOffice.org 3*"
Windows 64b: "*C:/Program Files (x86)/OpenOffice.org 3*"
2013/3/26 Li Feng Wang <ph...@gmail.com>
> (1)For PVT report you can record as wiki on
> http://wiki.openoffice.org/wiki/QA/Report
>
> * GUI PVT:*
> 1)open pvt.gui.Benchmark.xml with OO
> 2)compute test result
>
> - We run each test case 8 times.
> - Remove the first round result, and in the rest of 7 round results,
> remove the minimum and maximum result
> - Compute the average and standard deviation of the rest 5 results.* *Note:the
> standard deviation is to reflect whether the 8 round results are stable.
>
> * UNO PVT:*
> 1)Open pvt.uno.Conversion.xml with OO
> 2)Show your result without compute.
>
> First of all, You need run on the AOO 3.4.1 release to establish a
> baseline. To make your PVT result more meaningful.
>
> (2)For test.uno.project, you need follow
> http://wiki.openoffice.org/wiki/QA/test_automation_guide<http://wiki.openoffice.org/wiki/QA/test_automation_guide>
> I've ran uno project on Eclipse, it works fine.
> I will run uno project on command, try to find your env problem.
>
>
>
> 2013/3/26 Anders Kvibäck <ak...@gmail.com>
>
>> Hello!
>>
>> I uninstalled both 3.4.1 and 4.0 and language packs. Then I just installed
>> 4.0 and now it works again. Sorry , about that.
>>
>> I've done some sporadic testing on the test.gui project just to try it
>> out.
>> Basic function test works fine .I will come come back about that.
>>
>>
>> But now I can't run the test.uno project. I get an error message:
>>
>> "The archive: C:/Users/Anders/Desktop/OOo-dev 3.5 (en-US) Installation
>> Files/Basis/program/classes/unoil.jar which is referenced by the
>> classpath,
>> does not exist."
>>
>> So, it is something with the settings of my environmental variables here,
>> i
>> e classpath. What shall I type in there?
>>
>>
>> I see that in the class MixedTest it can't find the XTextDocument class;
>> I get the error messsages:
>>
>> "Multiple markers at this line
>> - The method newDocument(String) from the type UnoApp
>> refers to the missing type XComponent
>> - XTextDocument cannot be resolved to a type
>> - UnoRuntime cannot be resolved
>> - XTextDocument cannot be resolved to a type
>> - XTextDocument cannot be resolved to a type"
>>
>> Do you have any idea?
>>
>> And; how shall I report the perfomance test?
>>
>> Anders
>>
>>
>>
>>
>> 2013/3/25 Li Feng Wang <ph...@gmail.com>
>>
>> > Can you describe your steps more detailed?
>> >
>> > I will try to investigate the issue.
>> >
>> > 2013/3/22 Anders Kvibäck <ak...@gmail.com>
>> >
>> > > Hi!
>> > >
>> > > I've been trying to run some performance tests and I followed the
>> guide.
>> > I
>> > > started off with the class AOOTest and it workewd fine, so then I did
>> the
>> > > class smoke test and it worked fine plus some more.
>> > > But now all of a sudden I can't run the tests anymore. I just get the
>> > > message:
>> > >
>> > > *java.lang.RuntimeException: Can't connect to automation server!*
>> > >
>> > >
>> > > JUnit starts fine and I get the message in console:
>> > > *
>> > > INFO: Start running test method [smokeTest]*
>> > >
>> > >
>> > >
>> > > but after 30 sec it stops
>> > >
>> > > I do the same tests again but it doesn't work.
>> > >
>> > > I've spent an hour or two on google but I couldn't find anything
>> useful
>> > so
>> > > far.
>> > >
>> > > Does anyone have an idea how to solve this?
>> > >
>> > >
>> > >
>> > > Anders
>> > >
>> > >
>> > >
>> > > 2013/2/28 Yi Xuan Liu <li...@gmail.com>
>> > >
>> > > > Anders, Thanks for your trial. Did you follow this guide
>> > > > http://wiki.openoffice.org/wiki/QA/test_automation_guide ?
>> > > >
>> > > > I can't find "Disable color calibration" mentioned in the guide.
>> > > >
>> > > > On Wed, Feb 27, 2013 at 10:49 PM, Anders Kvibäck <
>> akva1270@gmail.com>
>> > > > wrote:
>> > > >
>> > > > > Hi!
>> > > > >
>> > > > > Yes, I would like to run performance tests. I've started to look
>> into
>> > > the
>> > > > > link about automated testing that you reffered to.
>> > > > > There it says that I have to disable color calibration. But is
>> this
>> > > > > possible in Windows Vista.? I just can't find out how to do it. As
>> > far
>> > > > as I
>> > > > > understand it this is only possible in Windows 7- which I don't
>> have
>> > > > > unfortunately.
>> > > > >
>> > > > > Anders
>> > > > > 2013/2/26 Yi Xuan Liu <li...@gmail.com>
>> > > > >
>> > > > > > Rob, I agree with you. The test environment configuration would
>> > > impact
>> > > > > the
>> > > > > > performance test results. Therefore, we should keep the test
>> > > > > configuration
>> > > > > > unchanged between build to build.
>> > > > > >
>> > > > > > I checked performance bugs list in bugzilla. 5 performance bugs
>> are
>> > > > found
>> > > > > > in automation performance test, such as memory leak and save
>> > > > performance
>> > > > > > issue.
>> > > > > >
>> > > > > > Lots of performance bugs are found in AOO daily usage.
>> Therefore,
>> > for
>> > > > > > volunteers who are not willing to run automation performance,
>> you
>> > can
>> > > > > also
>> > > > > > report bugs for any performance issue and it will be helpful for
>> > us.
>> > > > > >
>> > > > > > On Mon, Feb 25, 2013 at 9:16 PM, Rob Weir <ro...@apache.org>
>> > > wrote:
>> > > > > >
>> > > > > > > On Sun, Feb 24, 2013 at 11:42 PM, Yi Xuan Liu <
>> > > > liuyixuan.527@gmail.com
>> > > > > >
>> > > > > > > wrote:
>> > > > > > > > Hi, all:
>> > > > > > > >
>> > > > > > > > AOO 4.0 will release. Performance plays an important role in
>> > > > software
>> > > > > > > > quality. Is there any volunteer who want to run performance
>> > test?
>> > > > > > > >
>> > > > > > > > I've run AOO performance test on my own machines. I've tried
>> > on 3
>> > > > > > > > platforms: Windows XP, Ubuntu 10.04, Mac Mac OS 10.7.3. The
>> > test
>> > > > > > > configures
>> > > > > > > > are as follows:
>> > > > > > > >
>> > > > > > > > (1) W500; CPU:2.53 GHz; Mem: 3GB; OS: XP SP3
>> > > > > > > > (2) Ubuntu; CPU: Interl® Core™ 2 Duo 2 GHz; Mem: 3GB; OS:
>> > Ubuntu
>> > > > > 10.04
>> > > > > > > > (3) Mac; CPU: Interl® Core™ 2 Duo 2 GHz; Mem: 3GB; OS: Mac
>> OS
>> > > > 10.7.3
>> > > > > > > >
>> > > > > > >
>> > > > > > > I assume the volunteer does not need to have exactly the same
>> > > machine
>> > > > > > > type as you had. But they need some stability in the
>> > > configuration.
>> > > > > > > A performance test might be run first on the AOO 3.4.1
>> release
>> > to
>> > > > > > > establish a baseline. Then re-test on a current 4.0 snapshot
>> > > build.
>> > > > > > > And then re-run on new dev snapshot builds, maybe once a week.
>> > > > > > >
>> > > > > > > The goal is to detect performance regressions early, so
>> > developers
>> > > > can
>> > > > > > fix
>> > > > > > > it.
>> > > > > > >
>> > > > > > > The technical challenge here is to preserve a stable machine
>> > > > > > > configuration. If the machine changes, because of an OS
>> upgrade,
>> > > or
>> > > > a
>> > > > > > > changed hard drive, or a different network environment, or
>> > because
>> > > of
>> > > > > > > a new anti-virus product, then that confuses things. We need
>> to
>> > > > > > > "control all the variables".
>> > > > > > >
>> > > > > > > One approach to controlling all of the variables is to have a
>> > > machine
>> > > > > > > that is used for nothing but performance testing. That way we
>> > know
>> > > > > > > the machine's base performance does not change.
>> > > > > > >
>> > > > > > > Another approach is re-run the baseline AOO 3.4.1 performance
>> > tests
>> > > > > > > each week. This is more tolerant of changes in machine
>> > > > configuration,
>> > > > > > > etc.
>> > > > > > >
>> > > > > > > -Rob
>> > > > > > >
>> > > > > > >
>> > > > > > > > The test scenario include: AOO startup, file open, and save.
>> > > > > > > >
>> > > > > > > > Volunteers could run performance test on other platforms.
>> > > > > > > >
>> > > > > > > > All the automation scripts could be downloaded in AOO
>> project.
>> > > And
>> > > > it
>> > > > > > is
>> > > > > > > > not difficult to set the automation environment. You could
>> > follow
>> > > > the
>> > > > > > > guide:
>> > > > > > > > http://wiki.openoffice.org/wiki/QA/test_automation_guide
>> > > > > > > >
>> > > > > > > > For any questions, be free to contact with me :)
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> >
>> >
>> > --
>> > Best Wishes, LiFeng Wang
>> >
>>
>
>
>
> --
> Best Wishes, LiFeng Wang
>
--
Best Wishes, LiFeng Wang
Re: Call for volunteers for AOO 4.0 performance test.
Posted by Li Feng Wang <ph...@gmail.com>.
Thank you for reminder, pvt code was reconstructed to shield the impact of
cool startup and warm startup.
So now, no need to remove the first round result.
2013/3/26 colin mcdermott <co...@gmail.com>
> Can I ask a dumb question?
>
> >> - We run each test case 8 times.
> - Remove the first round result, and in the rest of 7 round results,
> remove the minimum and maximum result
>
> Naturally the first run will be loading all the libraries, dll,
> dependencies, etc into memory and be significantly different.
>
> So I understand why you would not include that run in the other tests. But
> what about first run data in and of itself?
>
> If OO takes a long time to start on first run, isn't that an issue?
>
> Cheers
>
> Colin
>
> On Tue, Mar 26, 2013 at 2:26 PM, Li Feng Wang <phoenix.wanglf@gmail.com
> >wrote:
>
> > (1)For PVT report you can record as wiki on
> > http://wiki.openoffice.org/wiki/QA/Report
> >
> > * GUI PVT:*
> > 1)open pvt.gui.Benchmark.xml with OO
> > 2)compute test result
> >
> > - We run each test case 8 times.
> > - Remove the first round result, and in the rest of 7 round results,
> > remove the minimum and maximum result
> > - Compute the average and standard deviation of the rest 5
> > results.* *Note:the
> > standard deviation is to reflect whether the 8 round results are
> stable.
> >
> > * UNO PVT:*
> > 1)Open pvt.uno.Conversion.xml with OO
> > 2)Show your result without compute.
> >
> > First of all, You need run on the AOO 3.4.1 release to establish a
> > baseline. To make your PVT result more meaningful.
> >
> > (2)For test.uno.project, you need follow
> > http://wiki.openoffice.org/wiki/QA/test_automation_guide<
> > http://wiki.openoffice.org/wiki/QA/test_automation_guide>
> > I've ran uno project on Eclipse, it works fine.
> > I will run uno project on command, try to find your env problem.
> >
> >
> > 2013/3/26 Anders Kvibäck <ak...@gmail.com>
> >
> > > Hello!
> > >
> > > I uninstalled both 3.4.1 and 4.0 and language packs. Then I just
> > installed
> > > 4.0 and now it works again. Sorry , about that.
> > >
> > > I've done some sporadic testing on the test.gui project just to try it
> > out.
> > > Basic function test works fine .I will come come back about that.
> > >
> > >
> > > But now I can't run the test.uno project. I get an error message:
> > >
> > > "The archive: C:/Users/Anders/Desktop/OOo-dev 3.5 (en-US) Installation
> > > Files/Basis/program/classes/unoil.jar which is referenced by the
> > classpath,
> > > does not exist."
> > >
> > > So, it is something with the settings of my environmental variables
> > here, i
> > > e classpath. What shall I type in there?
> > >
> > >
> > > I see that in the class MixedTest it can't find the XTextDocument
> class;
> > > I get the error messsages:
> > >
> > > "Multiple markers at this line
> > > - The method newDocument(String) from the type UnoApp
> > > refers to the missing type XComponent
> > > - XTextDocument cannot be resolved to a type
> > > - UnoRuntime cannot be resolved
> > > - XTextDocument cannot be resolved to a type
> > > - XTextDocument cannot be resolved to a type"
> > >
> > > Do you have any idea?
> > >
> > > And; how shall I report the perfomance test?
> > >
> > > Anders
> > >
> > >
> > >
> > >
> > > 2013/3/25 Li Feng Wang <ph...@gmail.com>
> > >
> > > > Can you describe your steps more detailed?
> > > >
> > > > I will try to investigate the issue.
> > > >
> > > > 2013/3/22 Anders Kvibäck <ak...@gmail.com>
> > > >
> > > > > Hi!
> > > > >
> > > > > I've been trying to run some performance tests and I followed the
> > > guide.
> > > > I
> > > > > started off with the class AOOTest and it workewd fine, so then I
> did
> > > the
> > > > > class smoke test and it worked fine plus some more.
> > > > > But now all of a sudden I can't run the tests anymore. I just get
> > the
> > > > > message:
> > > > >
> > > > > *java.lang.RuntimeException: Can't connect to automation server!*
> > > > >
> > > > >
> > > > > JUnit starts fine and I get the message in console:
> > > > > *
> > > > > INFO: Start running test method [smokeTest]*
> > > > >
> > > > >
> > > > >
> > > > > but after 30 sec it stops
> > > > >
> > > > > I do the same tests again but it doesn't work.
> > > > >
> > > > > I've spent an hour or two on google but I couldn't find anything
> > useful
> > > > so
> > > > > far.
> > > > >
> > > > > Does anyone have an idea how to solve this?
> > > > >
> > > > >
> > > > >
> > > > > Anders
> > > > >
> > > > >
> > > > >
> > > > > 2013/2/28 Yi Xuan Liu <li...@gmail.com>
> > > > >
> > > > > > Anders, Thanks for your trial. Did you follow this guide
> > > > > > http://wiki.openoffice.org/wiki/QA/test_automation_guide ?
> > > > > >
> > > > > > I can't find "Disable color calibration" mentioned in the guide.
> > > > > >
> > > > > > On Wed, Feb 27, 2013 at 10:49 PM, Anders Kvibäck <
> > akva1270@gmail.com
> > > >
> > > > > > wrote:
> > > > > >
> > > > > > > Hi!
> > > > > > >
> > > > > > > Yes, I would like to run performance tests. I've started to
> look
> > > into
> > > > > the
> > > > > > > link about automated testing that you reffered to.
> > > > > > > There it says that I have to disable color calibration. But is
> > this
> > > > > > > possible in Windows Vista.? I just can't find out how to do it.
> > As
> > > > far
> > > > > > as I
> > > > > > > understand it this is only possible in Windows 7- which I don't
> > > have
> > > > > > > unfortunately.
> > > > > > >
> > > > > > > Anders
> > > > > > > 2013/2/26 Yi Xuan Liu <li...@gmail.com>
> > > > > > >
> > > > > > > > Rob, I agree with you. The test environment configuration
> would
> > > > > impact
> > > > > > > the
> > > > > > > > performance test results. Therefore, we should keep the test
> > > > > > > configuration
> > > > > > > > unchanged between build to build.
> > > > > > > >
> > > > > > > > I checked performance bugs list in bugzilla. 5 performance
> bugs
> > > are
> > > > > > found
> > > > > > > > in automation performance test, such as memory leak and save
> > > > > > performance
> > > > > > > > issue.
> > > > > > > >
> > > > > > > > Lots of performance bugs are found in AOO daily usage.
> > Therefore,
> > > > for
> > > > > > > > volunteers who are not willing to run automation performance,
> > you
> > > > can
> > > > > > > also
> > > > > > > > report bugs for any performance issue and it will be helpful
> > for
> > > > us.
> > > > > > > >
> > > > > > > > On Mon, Feb 25, 2013 at 9:16 PM, Rob Weir <
> robweir@apache.org>
> > > > > wrote:
> > > > > > > >
> > > > > > > > > On Sun, Feb 24, 2013 at 11:42 PM, Yi Xuan Liu <
> > > > > > liuyixuan.527@gmail.com
> > > > > > > >
> > > > > > > > > wrote:
> > > > > > > > > > Hi, all:
> > > > > > > > > >
> > > > > > > > > > AOO 4.0 will release. Performance plays an important role
> > in
> > > > > > software
> > > > > > > > > > quality. Is there any volunteer who want to run
> performance
> > > > test?
> > > > > > > > > >
> > > > > > > > > > I've run AOO performance test on my own machines. I've
> > tried
> > > > on 3
> > > > > > > > > > platforms: Windows XP, Ubuntu 10.04, Mac Mac OS 10.7.3.
> The
> > > > test
> > > > > > > > > configures
> > > > > > > > > > are as follows:
> > > > > > > > > >
> > > > > > > > > > (1) W500; CPU:2.53 GHz; Mem: 3GB; OS: XP SP3
> > > > > > > > > > (2) Ubuntu; CPU: Interl® Core™ 2 Duo 2 GHz; Mem: 3GB; OS:
> > > > Ubuntu
> > > > > > > 10.04
> > > > > > > > > > (3) Mac; CPU: Interl® Core™ 2 Duo 2 GHz; Mem: 3GB; OS:
> Mac
> > OS
> > > > > > 10.7.3
> > > > > > > > > >
> > > > > > > > >
> > > > > > > > > I assume the volunteer does not need to have exactly the
> same
> > > > > machine
> > > > > > > > > type as you had. But they need some stability in the
> > > > > configuration.
> > > > > > > > > A performance test might be run first on the AOO 3.4.1
> > release
> > > > to
> > > > > > > > > establish a baseline. Then re-test on a current 4.0
> snapshot
> > > > > build.
> > > > > > > > > And then re-run on new dev snapshot builds, maybe once a
> > week.
> > > > > > > > >
> > > > > > > > > The goal is to detect performance regressions early, so
> > > > developers
> > > > > > can
> > > > > > > > fix
> > > > > > > > > it.
> > > > > > > > >
> > > > > > > > > The technical challenge here is to preserve a stable
> machine
> > > > > > > > > configuration. If the machine changes, because of an OS
> > > upgrade,
> > > > > or
> > > > > > a
> > > > > > > > > changed hard drive, or a different network environment, or
> > > > because
> > > > > of
> > > > > > > > > a new anti-virus product, then that confuses things. We
> need
> > > to
> > > > > > > > > "control all the variables".
> > > > > > > > >
> > > > > > > > > One approach to controlling all of the variables is to
> have a
> > > > > machine
> > > > > > > > > that is used for nothing but performance testing. That way
> > we
> > > > know
> > > > > > > > > the machine's base performance does not change.
> > > > > > > > >
> > > > > > > > > Another approach is re-run the baseline AOO 3.4.1
> performance
> > > > tests
> > > > > > > > > each week. This is more tolerant of changes in machine
> > > > > > configuration,
> > > > > > > > > etc.
> > > > > > > > >
> > > > > > > > > -Rob
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > > The test scenario include: AOO startup, file open, and
> > save.
> > > > > > > > > >
> > > > > > > > > > Volunteers could run performance test on other platforms.
> > > > > > > > > >
> > > > > > > > > > All the automation scripts could be downloaded in AOO
> > > project.
> > > > > And
> > > > > > it
> > > > > > > > is
> > > > > > > > > > not difficult to set the automation environment. You
> could
> > > > follow
> > > > > > the
> > > > > > > > > guide:
> > > > > > > > > > http://wiki.openoffice.org/wiki/QA/test_automation_guide
> > > > > > > > > >
> > > > > > > > > > For any questions, be free to contact with me :)
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > Best Wishes, LiFeng Wang
> > > >
> > >
> >
> >
> >
> > --
> > Best Wishes, LiFeng Wang
> >
>
>
>
> --
> Regards
>
> Colin McDermott
>
> 0433 400 256
>
--
Best Wishes, LiFeng Wang
Re: Call for volunteers for AOO 4.0 performance test.
Posted by colin mcdermott <co...@gmail.com>.
Can I ask a dumb question?
>> - We run each test case 8 times.
- Remove the first round result, and in the rest of 7 round results,
remove the minimum and maximum result
Naturally the first run will be loading all the libraries, dll,
dependencies, etc into memory and be significantly different.
So I understand why you would not include that run in the other tests. But
what about first run data in and of itself?
If OO takes a long time to start on first run, isn't that an issue?
Cheers
Colin
On Tue, Mar 26, 2013 at 2:26 PM, Li Feng Wang <ph...@gmail.com>wrote:
> (1)For PVT report you can record as wiki on
> http://wiki.openoffice.org/wiki/QA/Report
>
> * GUI PVT:*
> 1)open pvt.gui.Benchmark.xml with OO
> 2)compute test result
>
> - We run each test case 8 times.
> - Remove the first round result, and in the rest of 7 round results,
> remove the minimum and maximum result
> - Compute the average and standard deviation of the rest 5
> results.* *Note:the
> standard deviation is to reflect whether the 8 round results are stable.
>
> * UNO PVT:*
> 1)Open pvt.uno.Conversion.xml with OO
> 2)Show your result without compute.
>
> First of all, You need run on the AOO 3.4.1 release to establish a
> baseline. To make your PVT result more meaningful.
>
> (2)For test.uno.project, you need follow
> http://wiki.openoffice.org/wiki/QA/test_automation_guide<
> http://wiki.openoffice.org/wiki/QA/test_automation_guide>
> I've ran uno project on Eclipse, it works fine.
> I will run uno project on command, try to find your env problem.
>
>
> 2013/3/26 Anders Kvibäck <ak...@gmail.com>
>
> > Hello!
> >
> > I uninstalled both 3.4.1 and 4.0 and language packs. Then I just
> installed
> > 4.0 and now it works again. Sorry , about that.
> >
> > I've done some sporadic testing on the test.gui project just to try it
> out.
> > Basic function test works fine .I will come come back about that.
> >
> >
> > But now I can't run the test.uno project. I get an error message:
> >
> > "The archive: C:/Users/Anders/Desktop/OOo-dev 3.5 (en-US) Installation
> > Files/Basis/program/classes/unoil.jar which is referenced by the
> classpath,
> > does not exist."
> >
> > So, it is something with the settings of my environmental variables
> here, i
> > e classpath. What shall I type in there?
> >
> >
> > I see that in the class MixedTest it can't find the XTextDocument class;
> > I get the error messsages:
> >
> > "Multiple markers at this line
> > - The method newDocument(String) from the type UnoApp
> > refers to the missing type XComponent
> > - XTextDocument cannot be resolved to a type
> > - UnoRuntime cannot be resolved
> > - XTextDocument cannot be resolved to a type
> > - XTextDocument cannot be resolved to a type"
> >
> > Do you have any idea?
> >
> > And; how shall I report the perfomance test?
> >
> > Anders
> >
> >
> >
> >
> > 2013/3/25 Li Feng Wang <ph...@gmail.com>
> >
> > > Can you describe your steps more detailed?
> > >
> > > I will try to investigate the issue.
> > >
> > > 2013/3/22 Anders Kvibäck <ak...@gmail.com>
> > >
> > > > Hi!
> > > >
> > > > I've been trying to run some performance tests and I followed the
> > guide.
> > > I
> > > > started off with the class AOOTest and it workewd fine, so then I did
> > the
> > > > class smoke test and it worked fine plus some more.
> > > > But now all of a sudden I can't run the tests anymore. I just get
> the
> > > > message:
> > > >
> > > > *java.lang.RuntimeException: Can't connect to automation server!*
> > > >
> > > >
> > > > JUnit starts fine and I get the message in console:
> > > > *
> > > > INFO: Start running test method [smokeTest]*
> > > >
> > > >
> > > >
> > > > but after 30 sec it stops
> > > >
> > > > I do the same tests again but it doesn't work.
> > > >
> > > > I've spent an hour or two on google but I couldn't find anything
> useful
> > > so
> > > > far.
> > > >
> > > > Does anyone have an idea how to solve this?
> > > >
> > > >
> > > >
> > > > Anders
> > > >
> > > >
> > > >
> > > > 2013/2/28 Yi Xuan Liu <li...@gmail.com>
> > > >
> > > > > Anders, Thanks for your trial. Did you follow this guide
> > > > > http://wiki.openoffice.org/wiki/QA/test_automation_guide ?
> > > > >
> > > > > I can't find "Disable color calibration" mentioned in the guide.
> > > > >
> > > > > On Wed, Feb 27, 2013 at 10:49 PM, Anders Kvibäck <
> akva1270@gmail.com
> > >
> > > > > wrote:
> > > > >
> > > > > > Hi!
> > > > > >
> > > > > > Yes, I would like to run performance tests. I've started to look
> > into
> > > > the
> > > > > > link about automated testing that you reffered to.
> > > > > > There it says that I have to disable color calibration. But is
> this
> > > > > > possible in Windows Vista.? I just can't find out how to do it.
> As
> > > far
> > > > > as I
> > > > > > understand it this is only possible in Windows 7- which I don't
> > have
> > > > > > unfortunately.
> > > > > >
> > > > > > Anders
> > > > > > 2013/2/26 Yi Xuan Liu <li...@gmail.com>
> > > > > >
> > > > > > > Rob, I agree with you. The test environment configuration would
> > > > impact
> > > > > > the
> > > > > > > performance test results. Therefore, we should keep the test
> > > > > > configuration
> > > > > > > unchanged between build to build.
> > > > > > >
> > > > > > > I checked performance bugs list in bugzilla. 5 performance bugs
> > are
> > > > > found
> > > > > > > in automation performance test, such as memory leak and save
> > > > > performance
> > > > > > > issue.
> > > > > > >
> > > > > > > Lots of performance bugs are found in AOO daily usage.
> Therefore,
> > > for
> > > > > > > volunteers who are not willing to run automation performance,
> you
> > > can
> > > > > > also
> > > > > > > report bugs for any performance issue and it will be helpful
> for
> > > us.
> > > > > > >
> > > > > > > On Mon, Feb 25, 2013 at 9:16 PM, Rob Weir <ro...@apache.org>
> > > > wrote:
> > > > > > >
> > > > > > > > On Sun, Feb 24, 2013 at 11:42 PM, Yi Xuan Liu <
> > > > > liuyixuan.527@gmail.com
> > > > > > >
> > > > > > > > wrote:
> > > > > > > > > Hi, all:
> > > > > > > > >
> > > > > > > > > AOO 4.0 will release. Performance plays an important role
> in
> > > > > software
> > > > > > > > > quality. Is there any volunteer who want to run performance
> > > test?
> > > > > > > > >
> > > > > > > > > I've run AOO performance test on my own machines. I've
> tried
> > > on 3
> > > > > > > > > platforms: Windows XP, Ubuntu 10.04, Mac Mac OS 10.7.3. The
> > > test
> > > > > > > > configures
> > > > > > > > > are as follows:
> > > > > > > > >
> > > > > > > > > (1) W500; CPU:2.53 GHz; Mem: 3GB; OS: XP SP3
> > > > > > > > > (2) Ubuntu; CPU: Interl® Core™ 2 Duo 2 GHz; Mem: 3GB; OS:
> > > Ubuntu
> > > > > > 10.04
> > > > > > > > > (3) Mac; CPU: Interl® Core™ 2 Duo 2 GHz; Mem: 3GB; OS: Mac
> OS
> > > > > 10.7.3
> > > > > > > > >
> > > > > > > >
> > > > > > > > I assume the volunteer does not need to have exactly the same
> > > > machine
> > > > > > > > type as you had. But they need some stability in the
> > > > configuration.
> > > > > > > > A performance test might be run first on the AOO 3.4.1
> release
> > > to
> > > > > > > > establish a baseline. Then re-test on a current 4.0 snapshot
> > > > build.
> > > > > > > > And then re-run on new dev snapshot builds, maybe once a
> week.
> > > > > > > >
> > > > > > > > The goal is to detect performance regressions early, so
> > > developers
> > > > > can
> > > > > > > fix
> > > > > > > > it.
> > > > > > > >
> > > > > > > > The technical challenge here is to preserve a stable machine
> > > > > > > > configuration. If the machine changes, because of an OS
> > upgrade,
> > > > or
> > > > > a
> > > > > > > > changed hard drive, or a different network environment, or
> > > because
> > > > of
> > > > > > > > a new anti-virus product, then that confuses things. We need
> > to
> > > > > > > > "control all the variables".
> > > > > > > >
> > > > > > > > One approach to controlling all of the variables is to have a
> > > > machine
> > > > > > > > that is used for nothing but performance testing. That way
> we
> > > know
> > > > > > > > the machine's base performance does not change.
> > > > > > > >
> > > > > > > > Another approach is re-run the baseline AOO 3.4.1 performance
> > > tests
> > > > > > > > each week. This is more tolerant of changes in machine
> > > > > configuration,
> > > > > > > > etc.
> > > > > > > >
> > > > > > > > -Rob
> > > > > > > >
> > > > > > > >
> > > > > > > > > The test scenario include: AOO startup, file open, and
> save.
> > > > > > > > >
> > > > > > > > > Volunteers could run performance test on other platforms.
> > > > > > > > >
> > > > > > > > > All the automation scripts could be downloaded in AOO
> > project.
> > > > And
> > > > > it
> > > > > > > is
> > > > > > > > > not difficult to set the automation environment. You could
> > > follow
> > > > > the
> > > > > > > > guide:
> > > > > > > > > http://wiki.openoffice.org/wiki/QA/test_automation_guide
> > > > > > > > >
> > > > > > > > > For any questions, be free to contact with me :)
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> > >
> > >
> > > --
> > > Best Wishes, LiFeng Wang
> > >
> >
>
>
>
> --
> Best Wishes, LiFeng Wang
>
--
Regards
Colin McDermott
0433 400 256
Re: Call for volunteers for AOO 4.0 performance test.
Posted by Li Feng Wang <ph...@gmail.com>.
(1)For PVT report you can record as wiki on
http://wiki.openoffice.org/wiki/QA/Report
* GUI PVT:*
1)open pvt.gui.Benchmark.xml with OO
2)compute test result
- We run each test case 8 times.
- Remove the first round result, and in the rest of 7 round results,
remove the minimum and maximum result
- Compute the average and standard deviation of the rest 5
results.* *Note:the
standard deviation is to reflect whether the 8 round results are stable.
* UNO PVT:*
1)Open pvt.uno.Conversion.xml with OO
2)Show your result without compute.
First of all, You need run on the AOO 3.4.1 release to establish a
baseline. To make your PVT result more meaningful.
(2)For test.uno.project, you need follow
http://wiki.openoffice.org/wiki/QA/test_automation_guide<http://wiki.openoffice.org/wiki/QA/test_automation_guide>
I've ran uno project on Eclipse, it works fine.
I will run uno project on command, try to find your env problem.
2013/3/26 Anders Kvibäck <ak...@gmail.com>
> Hello!
>
> I uninstalled both 3.4.1 and 4.0 and language packs. Then I just installed
> 4.0 and now it works again. Sorry , about that.
>
> I've done some sporadic testing on the test.gui project just to try it out.
> Basic function test works fine .I will come come back about that.
>
>
> But now I can't run the test.uno project. I get an error message:
>
> "The archive: C:/Users/Anders/Desktop/OOo-dev 3.5 (en-US) Installation
> Files/Basis/program/classes/unoil.jar which is referenced by the classpath,
> does not exist."
>
> So, it is something with the settings of my environmental variables here, i
> e classpath. What shall I type in there?
>
>
> I see that in the class MixedTest it can't find the XTextDocument class;
> I get the error messsages:
>
> "Multiple markers at this line
> - The method newDocument(String) from the type UnoApp
> refers to the missing type XComponent
> - XTextDocument cannot be resolved to a type
> - UnoRuntime cannot be resolved
> - XTextDocument cannot be resolved to a type
> - XTextDocument cannot be resolved to a type"
>
> Do you have any idea?
>
> And; how shall I report the perfomance test?
>
> Anders
>
>
>
>
> 2013/3/25 Li Feng Wang <ph...@gmail.com>
>
> > Can you describe your steps more detailed?
> >
> > I will try to investigate the issue.
> >
> > 2013/3/22 Anders Kvibäck <ak...@gmail.com>
> >
> > > Hi!
> > >
> > > I've been trying to run some performance tests and I followed the
> guide.
> > I
> > > started off with the class AOOTest and it workewd fine, so then I did
> the
> > > class smoke test and it worked fine plus some more.
> > > But now all of a sudden I can't run the tests anymore. I just get the
> > > message:
> > >
> > > *java.lang.RuntimeException: Can't connect to automation server!*
> > >
> > >
> > > JUnit starts fine and I get the message in console:
> > > *
> > > INFO: Start running test method [smokeTest]*
> > >
> > >
> > >
> > > but after 30 sec it stops
> > >
> > > I do the same tests again but it doesn't work.
> > >
> > > I've spent an hour or two on google but I couldn't find anything useful
> > so
> > > far.
> > >
> > > Does anyone have an idea how to solve this?
> > >
> > >
> > >
> > > Anders
> > >
> > >
> > >
> > > 2013/2/28 Yi Xuan Liu <li...@gmail.com>
> > >
> > > > Anders, Thanks for your trial. Did you follow this guide
> > > > http://wiki.openoffice.org/wiki/QA/test_automation_guide ?
> > > >
> > > > I can't find "Disable color calibration" mentioned in the guide.
> > > >
> > > > On Wed, Feb 27, 2013 at 10:49 PM, Anders Kvibäck <akva1270@gmail.com
> >
> > > > wrote:
> > > >
> > > > > Hi!
> > > > >
> > > > > Yes, I would like to run performance tests. I've started to look
> into
> > > the
> > > > > link about automated testing that you reffered to.
> > > > > There it says that I have to disable color calibration. But is this
> > > > > possible in Windows Vista.? I just can't find out how to do it. As
> > far
> > > > as I
> > > > > understand it this is only possible in Windows 7- which I don't
> have
> > > > > unfortunately.
> > > > >
> > > > > Anders
> > > > > 2013/2/26 Yi Xuan Liu <li...@gmail.com>
> > > > >
> > > > > > Rob, I agree with you. The test environment configuration would
> > > impact
> > > > > the
> > > > > > performance test results. Therefore, we should keep the test
> > > > > configuration
> > > > > > unchanged between build to build.
> > > > > >
> > > > > > I checked performance bugs list in bugzilla. 5 performance bugs
> are
> > > > found
> > > > > > in automation performance test, such as memory leak and save
> > > > performance
> > > > > > issue.
> > > > > >
> > > > > > Lots of performance bugs are found in AOO daily usage. Therefore,
> > for
> > > > > > volunteers who are not willing to run automation performance, you
> > can
> > > > > also
> > > > > > report bugs for any performance issue and it will be helpful for
> > us.
> > > > > >
> > > > > > On Mon, Feb 25, 2013 at 9:16 PM, Rob Weir <ro...@apache.org>
> > > wrote:
> > > > > >
> > > > > > > On Sun, Feb 24, 2013 at 11:42 PM, Yi Xuan Liu <
> > > > liuyixuan.527@gmail.com
> > > > > >
> > > > > > > wrote:
> > > > > > > > Hi, all:
> > > > > > > >
> > > > > > > > AOO 4.0 will release. Performance plays an important role in
> > > > software
> > > > > > > > quality. Is there any volunteer who want to run performance
> > test?
> > > > > > > >
> > > > > > > > I've run AOO performance test on my own machines. I've tried
> > on 3
> > > > > > > > platforms: Windows XP, Ubuntu 10.04, Mac Mac OS 10.7.3. The
> > test
> > > > > > > configures
> > > > > > > > are as follows:
> > > > > > > >
> > > > > > > > (1) W500; CPU:2.53 GHz; Mem: 3GB; OS: XP SP3
> > > > > > > > (2) Ubuntu; CPU: Interl® Core™ 2 Duo 2 GHz; Mem: 3GB; OS:
> > Ubuntu
> > > > > 10.04
> > > > > > > > (3) Mac; CPU: Interl® Core™ 2 Duo 2 GHz; Mem: 3GB; OS: Mac OS
> > > > 10.7.3
> > > > > > > >
> > > > > > >
> > > > > > > I assume the volunteer does not need to have exactly the same
> > > machine
> > > > > > > type as you had. But they need some stability in the
> > > configuration.
> > > > > > > A performance test might be run first on the AOO 3.4.1 release
> > to
> > > > > > > establish a baseline. Then re-test on a current 4.0 snapshot
> > > build.
> > > > > > > And then re-run on new dev snapshot builds, maybe once a week.
> > > > > > >
> > > > > > > The goal is to detect performance regressions early, so
> > developers
> > > > can
> > > > > > fix
> > > > > > > it.
> > > > > > >
> > > > > > > The technical challenge here is to preserve a stable machine
> > > > > > > configuration. If the machine changes, because of an OS
> upgrade,
> > > or
> > > > a
> > > > > > > changed hard drive, or a different network environment, or
> > because
> > > of
> > > > > > > a new anti-virus product, then that confuses things. We need
> to
> > > > > > > "control all the variables".
> > > > > > >
> > > > > > > One approach to controlling all of the variables is to have a
> > > machine
> > > > > > > that is used for nothing but performance testing. That way we
> > know
> > > > > > > the machine's base performance does not change.
> > > > > > >
> > > > > > > Another approach is re-run the baseline AOO 3.4.1 performance
> > tests
> > > > > > > each week. This is more tolerant of changes in machine
> > > > configuration,
> > > > > > > etc.
> > > > > > >
> > > > > > > -Rob
> > > > > > >
> > > > > > >
> > > > > > > > The test scenario include: AOO startup, file open, and save.
> > > > > > > >
> > > > > > > > Volunteers could run performance test on other platforms.
> > > > > > > >
> > > > > > > > All the automation scripts could be downloaded in AOO
> project.
> > > And
> > > > it
> > > > > > is
> > > > > > > > not difficult to set the automation environment. You could
> > follow
> > > > the
> > > > > > > guide:
> > > > > > > > http://wiki.openoffice.org/wiki/QA/test_automation_guide
> > > > > > > >
> > > > > > > > For any questions, be free to contact with me :)
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> >
> >
> > --
> > Best Wishes, LiFeng Wang
> >
>
--
Best Wishes, LiFeng Wang
Re: Call for volunteers for AOO 4.0 performance test.
Posted by Anders Kvibäck <ak...@gmail.com>.
Hello!
I uninstalled both 3.4.1 and 4.0 and language packs. Then I just installed
4.0 and now it works again. Sorry , about that.
I've done some sporadic testing on the test.gui project just to try it out.
Basic function test works fine .I will come come back about that.
But now I can't run the test.uno project. I get an error message:
"The archive: C:/Users/Anders/Desktop/OOo-dev 3.5 (en-US) Installation
Files/Basis/program/classes/unoil.jar which is referenced by the classpath,
does not exist."
So, it is something with the settings of my environmental variables here, i
e classpath. What shall I type in there?
I see that in the class MixedTest it can't find the XTextDocument class;
I get the error messsages:
"Multiple markers at this line
- The method newDocument(String) from the type UnoApp
refers to the missing type XComponent
- XTextDocument cannot be resolved to a type
- UnoRuntime cannot be resolved
- XTextDocument cannot be resolved to a type
- XTextDocument cannot be resolved to a type"
Do you have any idea?
And; how shall I report the perfomance test?
Anders
2013/3/25 Li Feng Wang <ph...@gmail.com>
> Can you describe your steps more detailed?
>
> I will try to investigate the issue.
>
> 2013/3/22 Anders Kvibäck <ak...@gmail.com>
>
> > Hi!
> >
> > I've been trying to run some performance tests and I followed the guide.
> I
> > started off with the class AOOTest and it workewd fine, so then I did the
> > class smoke test and it worked fine plus some more.
> > But now all of a sudden I can't run the tests anymore. I just get the
> > message:
> >
> > *java.lang.RuntimeException: Can't connect to automation server!*
> >
> >
> > JUnit starts fine and I get the message in console:
> > *
> > INFO: Start running test method [smokeTest]*
> >
> >
> >
> > but after 30 sec it stops
> >
> > I do the same tests again but it doesn't work.
> >
> > I've spent an hour or two on google but I couldn't find anything useful
> so
> > far.
> >
> > Does anyone have an idea how to solve this?
> >
> >
> >
> > Anders
> >
> >
> >
> > 2013/2/28 Yi Xuan Liu <li...@gmail.com>
> >
> > > Anders, Thanks for your trial. Did you follow this guide
> > > http://wiki.openoffice.org/wiki/QA/test_automation_guide ?
> > >
> > > I can't find "Disable color calibration" mentioned in the guide.
> > >
> > > On Wed, Feb 27, 2013 at 10:49 PM, Anders Kvibäck <ak...@gmail.com>
> > > wrote:
> > >
> > > > Hi!
> > > >
> > > > Yes, I would like to run performance tests. I've started to look into
> > the
> > > > link about automated testing that you reffered to.
> > > > There it says that I have to disable color calibration. But is this
> > > > possible in Windows Vista.? I just can't find out how to do it. As
> far
> > > as I
> > > > understand it this is only possible in Windows 7- which I don't have
> > > > unfortunately.
> > > >
> > > > Anders
> > > > 2013/2/26 Yi Xuan Liu <li...@gmail.com>
> > > >
> > > > > Rob, I agree with you. The test environment configuration would
> > impact
> > > > the
> > > > > performance test results. Therefore, we should keep the test
> > > > configuration
> > > > > unchanged between build to build.
> > > > >
> > > > > I checked performance bugs list in bugzilla. 5 performance bugs are
> > > found
> > > > > in automation performance test, such as memory leak and save
> > > performance
> > > > > issue.
> > > > >
> > > > > Lots of performance bugs are found in AOO daily usage. Therefore,
> for
> > > > > volunteers who are not willing to run automation performance, you
> can
> > > > also
> > > > > report bugs for any performance issue and it will be helpful for
> us.
> > > > >
> > > > > On Mon, Feb 25, 2013 at 9:16 PM, Rob Weir <ro...@apache.org>
> > wrote:
> > > > >
> > > > > > On Sun, Feb 24, 2013 at 11:42 PM, Yi Xuan Liu <
> > > liuyixuan.527@gmail.com
> > > > >
> > > > > > wrote:
> > > > > > > Hi, all:
> > > > > > >
> > > > > > > AOO 4.0 will release. Performance plays an important role in
> > > software
> > > > > > > quality. Is there any volunteer who want to run performance
> test?
> > > > > > >
> > > > > > > I've run AOO performance test on my own machines. I've tried
> on 3
> > > > > > > platforms: Windows XP, Ubuntu 10.04, Mac Mac OS 10.7.3. The
> test
> > > > > > configures
> > > > > > > are as follows:
> > > > > > >
> > > > > > > (1) W500; CPU:2.53 GHz; Mem: 3GB; OS: XP SP3
> > > > > > > (2) Ubuntu; CPU: Interl® Core™ 2 Duo 2 GHz; Mem: 3GB; OS:
> Ubuntu
> > > > 10.04
> > > > > > > (3) Mac; CPU: Interl® Core™ 2 Duo 2 GHz; Mem: 3GB; OS: Mac OS
> > > 10.7.3
> > > > > > >
> > > > > >
> > > > > > I assume the volunteer does not need to have exactly the same
> > machine
> > > > > > type as you had. But they need some stability in the
> > configuration.
> > > > > > A performance test might be run first on the AOO 3.4.1 release
> to
> > > > > > establish a baseline. Then re-test on a current 4.0 snapshot
> > build.
> > > > > > And then re-run on new dev snapshot builds, maybe once a week.
> > > > > >
> > > > > > The goal is to detect performance regressions early, so
> developers
> > > can
> > > > > fix
> > > > > > it.
> > > > > >
> > > > > > The technical challenge here is to preserve a stable machine
> > > > > > configuration. If the machine changes, because of an OS upgrade,
> > or
> > > a
> > > > > > changed hard drive, or a different network environment, or
> because
> > of
> > > > > > a new anti-virus product, then that confuses things. We need to
> > > > > > "control all the variables".
> > > > > >
> > > > > > One approach to controlling all of the variables is to have a
> > machine
> > > > > > that is used for nothing but performance testing. That way we
> know
> > > > > > the machine's base performance does not change.
> > > > > >
> > > > > > Another approach is re-run the baseline AOO 3.4.1 performance
> tests
> > > > > > each week. This is more tolerant of changes in machine
> > > configuration,
> > > > > > etc.
> > > > > >
> > > > > > -Rob
> > > > > >
> > > > > >
> > > > > > > The test scenario include: AOO startup, file open, and save.
> > > > > > >
> > > > > > > Volunteers could run performance test on other platforms.
> > > > > > >
> > > > > > > All the automation scripts could be downloaded in AOO project.
> > And
> > > it
> > > > > is
> > > > > > > not difficult to set the automation environment. You could
> follow
> > > the
> > > > > > guide:
> > > > > > > http://wiki.openoffice.org/wiki/QA/test_automation_guide
> > > > > > >
> > > > > > > For any questions, be free to contact with me :)
> > > > > >
> > > > >
> > > >
> > >
> >
>
>
>
> --
> Best Wishes, LiFeng Wang
>
Re: Call for volunteers for AOO 4.0 performance test.
Posted by Li Feng Wang <ph...@gmail.com>.
Can you describe your steps more detailed?
I will try to investigate the issue.
2013/3/22 Anders Kvibäck <ak...@gmail.com>
> Hi!
>
> I've been trying to run some performance tests and I followed the guide. I
> started off with the class AOOTest and it workewd fine, so then I did the
> class smoke test and it worked fine plus some more.
> But now all of a sudden I can't run the tests anymore. I just get the
> message:
>
> *java.lang.RuntimeException: Can't connect to automation server!*
>
>
> JUnit starts fine and I get the message in console:
> *
> INFO: Start running test method [smokeTest]*
>
>
>
> but after 30 sec it stops
>
> I do the same tests again but it doesn't work.
>
> I've spent an hour or two on google but I couldn't find anything useful so
> far.
>
> Does anyone have an idea how to solve this?
>
>
>
> Anders
>
>
>
> 2013/2/28 Yi Xuan Liu <li...@gmail.com>
>
> > Anders, Thanks for your trial. Did you follow this guide
> > http://wiki.openoffice.org/wiki/QA/test_automation_guide ?
> >
> > I can't find "Disable color calibration" mentioned in the guide.
> >
> > On Wed, Feb 27, 2013 at 10:49 PM, Anders Kvibäck <ak...@gmail.com>
> > wrote:
> >
> > > Hi!
> > >
> > > Yes, I would like to run performance tests. I've started to look into
> the
> > > link about automated testing that you reffered to.
> > > There it says that I have to disable color calibration. But is this
> > > possible in Windows Vista.? I just can't find out how to do it. As far
> > as I
> > > understand it this is only possible in Windows 7- which I don't have
> > > unfortunately.
> > >
> > > Anders
> > > 2013/2/26 Yi Xuan Liu <li...@gmail.com>
> > >
> > > > Rob, I agree with you. The test environment configuration would
> impact
> > > the
> > > > performance test results. Therefore, we should keep the test
> > > configuration
> > > > unchanged between build to build.
> > > >
> > > > I checked performance bugs list in bugzilla. 5 performance bugs are
> > found
> > > > in automation performance test, such as memory leak and save
> > performance
> > > > issue.
> > > >
> > > > Lots of performance bugs are found in AOO daily usage. Therefore, for
> > > > volunteers who are not willing to run automation performance, you can
> > > also
> > > > report bugs for any performance issue and it will be helpful for us.
> > > >
> > > > On Mon, Feb 25, 2013 at 9:16 PM, Rob Weir <ro...@apache.org>
> wrote:
> > > >
> > > > > On Sun, Feb 24, 2013 at 11:42 PM, Yi Xuan Liu <
> > liuyixuan.527@gmail.com
> > > >
> > > > > wrote:
> > > > > > Hi, all:
> > > > > >
> > > > > > AOO 4.0 will release. Performance plays an important role in
> > software
> > > > > > quality. Is there any volunteer who want to run performance test?
> > > > > >
> > > > > > I've run AOO performance test on my own machines. I've tried on 3
> > > > > > platforms: Windows XP, Ubuntu 10.04, Mac Mac OS 10.7.3. The test
> > > > > configures
> > > > > > are as follows:
> > > > > >
> > > > > > (1) W500; CPU:2.53 GHz; Mem: 3GB; OS: XP SP3
> > > > > > (2) Ubuntu; CPU: Interl® Core™ 2 Duo 2 GHz; Mem: 3GB; OS: Ubuntu
> > > 10.04
> > > > > > (3) Mac; CPU: Interl® Core™ 2 Duo 2 GHz; Mem: 3GB; OS: Mac OS
> > 10.7.3
> > > > > >
> > > > >
> > > > > I assume the volunteer does not need to have exactly the same
> machine
> > > > > type as you had. But they need some stability in the
> configuration.
> > > > > A performance test might be run first on the AOO 3.4.1 release to
> > > > > establish a baseline. Then re-test on a current 4.0 snapshot
> build.
> > > > > And then re-run on new dev snapshot builds, maybe once a week.
> > > > >
> > > > > The goal is to detect performance regressions early, so developers
> > can
> > > > fix
> > > > > it.
> > > > >
> > > > > The technical challenge here is to preserve a stable machine
> > > > > configuration. If the machine changes, because of an OS upgrade,
> or
> > a
> > > > > changed hard drive, or a different network environment, or because
> of
> > > > > a new anti-virus product, then that confuses things. We need to
> > > > > "control all the variables".
> > > > >
> > > > > One approach to controlling all of the variables is to have a
> machine
> > > > > that is used for nothing but performance testing. That way we know
> > > > > the machine's base performance does not change.
> > > > >
> > > > > Another approach is re-run the baseline AOO 3.4.1 performance tests
> > > > > each week. This is more tolerant of changes in machine
> > configuration,
> > > > > etc.
> > > > >
> > > > > -Rob
> > > > >
> > > > >
> > > > > > The test scenario include: AOO startup, file open, and save.
> > > > > >
> > > > > > Volunteers could run performance test on other platforms.
> > > > > >
> > > > > > All the automation scripts could be downloaded in AOO project.
> And
> > it
> > > > is
> > > > > > not difficult to set the automation environment. You could follow
> > the
> > > > > guide:
> > > > > > http://wiki.openoffice.org/wiki/QA/test_automation_guide
> > > > > >
> > > > > > For any questions, be free to contact with me :)
> > > > >
> > > >
> > >
> >
>
--
Best Wishes, LiFeng Wang
Re: Call for volunteers for AOO 4.0 performance test.
Posted by Anders Kvibäck <ak...@gmail.com>.
Hi!
I've been trying to run some performance tests and I followed the guide. I
started off with the class AOOTest and it workewd fine, so then I did the
class smoke test and it worked fine plus some more.
But now all of a sudden I can't run the tests anymore. I just get the
message:
*java.lang.RuntimeException: Can't connect to automation server!*
JUnit starts fine and I get the message in console:
*
INFO: Start running test method [smokeTest]*
but after 30 sec it stops
I do the same tests again but it doesn't work.
I've spent an hour or two on google but I couldn't find anything useful so
far.
Does anyone have an idea how to solve this?
Anders
2013/2/28 Yi Xuan Liu <li...@gmail.com>
> Anders, Thanks for your trial. Did you follow this guide
> http://wiki.openoffice.org/wiki/QA/test_automation_guide ?
>
> I can't find "Disable color calibration" mentioned in the guide.
>
> On Wed, Feb 27, 2013 at 10:49 PM, Anders Kvibäck <ak...@gmail.com>
> wrote:
>
> > Hi!
> >
> > Yes, I would like to run performance tests. I've started to look into the
> > link about automated testing that you reffered to.
> > There it says that I have to disable color calibration. But is this
> > possible in Windows Vista.? I just can't find out how to do it. As far
> as I
> > understand it this is only possible in Windows 7- which I don't have
> > unfortunately.
> >
> > Anders
> > 2013/2/26 Yi Xuan Liu <li...@gmail.com>
> >
> > > Rob, I agree with you. The test environment configuration would impact
> > the
> > > performance test results. Therefore, we should keep the test
> > configuration
> > > unchanged between build to build.
> > >
> > > I checked performance bugs list in bugzilla. 5 performance bugs are
> found
> > > in automation performance test, such as memory leak and save
> performance
> > > issue.
> > >
> > > Lots of performance bugs are found in AOO daily usage. Therefore, for
> > > volunteers who are not willing to run automation performance, you can
> > also
> > > report bugs for any performance issue and it will be helpful for us.
> > >
> > > On Mon, Feb 25, 2013 at 9:16 PM, Rob Weir <ro...@apache.org> wrote:
> > >
> > > > On Sun, Feb 24, 2013 at 11:42 PM, Yi Xuan Liu <
> liuyixuan.527@gmail.com
> > >
> > > > wrote:
> > > > > Hi, all:
> > > > >
> > > > > AOO 4.0 will release. Performance plays an important role in
> software
> > > > > quality. Is there any volunteer who want to run performance test?
> > > > >
> > > > > I've run AOO performance test on my own machines. I've tried on 3
> > > > > platforms: Windows XP, Ubuntu 10.04, Mac Mac OS 10.7.3. The test
> > > > configures
> > > > > are as follows:
> > > > >
> > > > > (1) W500; CPU:2.53 GHz; Mem: 3GB; OS: XP SP3
> > > > > (2) Ubuntu; CPU: Interl® Core™ 2 Duo 2 GHz; Mem: 3GB; OS: Ubuntu
> > 10.04
> > > > > (3) Mac; CPU: Interl® Core™ 2 Duo 2 GHz; Mem: 3GB; OS: Mac OS
> 10.7.3
> > > > >
> > > >
> > > > I assume the volunteer does not need to have exactly the same machine
> > > > type as you had. But they need some stability in the configuration.
> > > > A performance test might be run first on the AOO 3.4.1 release to
> > > > establish a baseline. Then re-test on a current 4.0 snapshot build.
> > > > And then re-run on new dev snapshot builds, maybe once a week.
> > > >
> > > > The goal is to detect performance regressions early, so developers
> can
> > > fix
> > > > it.
> > > >
> > > > The technical challenge here is to preserve a stable machine
> > > > configuration. If the machine changes, because of an OS upgrade, or
> a
> > > > changed hard drive, or a different network environment, or because of
> > > > a new anti-virus product, then that confuses things. We need to
> > > > "control all the variables".
> > > >
> > > > One approach to controlling all of the variables is to have a machine
> > > > that is used for nothing but performance testing. That way we know
> > > > the machine's base performance does not change.
> > > >
> > > > Another approach is re-run the baseline AOO 3.4.1 performance tests
> > > > each week. This is more tolerant of changes in machine
> configuration,
> > > > etc.
> > > >
> > > > -Rob
> > > >
> > > >
> > > > > The test scenario include: AOO startup, file open, and save.
> > > > >
> > > > > Volunteers could run performance test on other platforms.
> > > > >
> > > > > All the automation scripts could be downloaded in AOO project. And
> it
> > > is
> > > > > not difficult to set the automation environment. You could follow
> the
> > > > guide:
> > > > > http://wiki.openoffice.org/wiki/QA/test_automation_guide
> > > > >
> > > > > For any questions, be free to contact with me :)
> > > >
> > >
> >
>
Re: Call for volunteers for AOO 4.0 performance test.
Posted by Yi Xuan Liu <li...@gmail.com>.
Anders, Thanks for your trial. Did you follow this guide
http://wiki.openoffice.org/wiki/QA/test_automation_guide ?
I can't find "Disable color calibration" mentioned in the guide.
On Wed, Feb 27, 2013 at 10:49 PM, Anders Kvibäck <ak...@gmail.com> wrote:
> Hi!
>
> Yes, I would like to run performance tests. I've started to look into the
> link about automated testing that you reffered to.
> There it says that I have to disable color calibration. But is this
> possible in Windows Vista.? I just can't find out how to do it. As far as I
> understand it this is only possible in Windows 7- which I don't have
> unfortunately.
>
> Anders
> 2013/2/26 Yi Xuan Liu <li...@gmail.com>
>
> > Rob, I agree with you. The test environment configuration would impact
> the
> > performance test results. Therefore, we should keep the test
> configuration
> > unchanged between build to build.
> >
> > I checked performance bugs list in bugzilla. 5 performance bugs are found
> > in automation performance test, such as memory leak and save performance
> > issue.
> >
> > Lots of performance bugs are found in AOO daily usage. Therefore, for
> > volunteers who are not willing to run automation performance, you can
> also
> > report bugs for any performance issue and it will be helpful for us.
> >
> > On Mon, Feb 25, 2013 at 9:16 PM, Rob Weir <ro...@apache.org> wrote:
> >
> > > On Sun, Feb 24, 2013 at 11:42 PM, Yi Xuan Liu <liuyixuan.527@gmail.com
> >
> > > wrote:
> > > > Hi, all:
> > > >
> > > > AOO 4.0 will release. Performance plays an important role in software
> > > > quality. Is there any volunteer who want to run performance test?
> > > >
> > > > I've run AOO performance test on my own machines. I've tried on 3
> > > > platforms: Windows XP, Ubuntu 10.04, Mac Mac OS 10.7.3. The test
> > > configures
> > > > are as follows:
> > > >
> > > > (1) W500; CPU:2.53 GHz; Mem: 3GB; OS: XP SP3
> > > > (2) Ubuntu; CPU: Interl® Core™ 2 Duo 2 GHz; Mem: 3GB; OS: Ubuntu
> 10.04
> > > > (3) Mac; CPU: Interl® Core™ 2 Duo 2 GHz; Mem: 3GB; OS: Mac OS 10.7.3
> > > >
> > >
> > > I assume the volunteer does not need to have exactly the same machine
> > > type as you had. But they need some stability in the configuration.
> > > A performance test might be run first on the AOO 3.4.1 release to
> > > establish a baseline. Then re-test on a current 4.0 snapshot build.
> > > And then re-run on new dev snapshot builds, maybe once a week.
> > >
> > > The goal is to detect performance regressions early, so developers can
> > fix
> > > it.
> > >
> > > The technical challenge here is to preserve a stable machine
> > > configuration. If the machine changes, because of an OS upgrade, or a
> > > changed hard drive, or a different network environment, or because of
> > > a new anti-virus product, then that confuses things. We need to
> > > "control all the variables".
> > >
> > > One approach to controlling all of the variables is to have a machine
> > > that is used for nothing but performance testing. That way we know
> > > the machine's base performance does not change.
> > >
> > > Another approach is re-run the baseline AOO 3.4.1 performance tests
> > > each week. This is more tolerant of changes in machine configuration,
> > > etc.
> > >
> > > -Rob
> > >
> > >
> > > > The test scenario include: AOO startup, file open, and save.
> > > >
> > > > Volunteers could run performance test on other platforms.
> > > >
> > > > All the automation scripts could be downloaded in AOO project. And it
> > is
> > > > not difficult to set the automation environment. You could follow the
> > > guide:
> > > > http://wiki.openoffice.org/wiki/QA/test_automation_guide
> > > >
> > > > For any questions, be free to contact with me :)
> > >
> >
>
Re: Call for volunteers for AOO 4.0 performance test.
Posted by Anders Kvibäck <ak...@gmail.com>.
Hi!
Yes, I would like to run performance tests. I've started to look into the
link about automated testing that you reffered to.
There it says that I have to disable color calibration. But is this
possible in Windows Vista.? I just can't find out how to do it. As far as I
understand it this is only possible in Windows 7- which I don't have
unfortunately.
Anders
2013/2/26 Yi Xuan Liu <li...@gmail.com>
> Rob, I agree with you. The test environment configuration would impact the
> performance test results. Therefore, we should keep the test configuration
> unchanged between build to build.
>
> I checked performance bugs list in bugzilla. 5 performance bugs are found
> in automation performance test, such as memory leak and save performance
> issue.
>
> Lots of performance bugs are found in AOO daily usage. Therefore, for
> volunteers who are not willing to run automation performance, you can also
> report bugs for any performance issue and it will be helpful for us.
>
> On Mon, Feb 25, 2013 at 9:16 PM, Rob Weir <ro...@apache.org> wrote:
>
> > On Sun, Feb 24, 2013 at 11:42 PM, Yi Xuan Liu <li...@gmail.com>
> > wrote:
> > > Hi, all:
> > >
> > > AOO 4.0 will release. Performance plays an important role in software
> > > quality. Is there any volunteer who want to run performance test?
> > >
> > > I've run AOO performance test on my own machines. I've tried on 3
> > > platforms: Windows XP, Ubuntu 10.04, Mac Mac OS 10.7.3. The test
> > configures
> > > are as follows:
> > >
> > > (1) W500; CPU:2.53 GHz; Mem: 3GB; OS: XP SP3
> > > (2) Ubuntu; CPU: Interl® Core™ 2 Duo 2 GHz; Mem: 3GB; OS: Ubuntu 10.04
> > > (3) Mac; CPU: Interl® Core™ 2 Duo 2 GHz; Mem: 3GB; OS: Mac OS 10.7.3
> > >
> >
> > I assume the volunteer does not need to have exactly the same machine
> > type as you had. But they need some stability in the configuration.
> > A performance test might be run first on the AOO 3.4.1 release to
> > establish a baseline. Then re-test on a current 4.0 snapshot build.
> > And then re-run on new dev snapshot builds, maybe once a week.
> >
> > The goal is to detect performance regressions early, so developers can
> fix
> > it.
> >
> > The technical challenge here is to preserve a stable machine
> > configuration. If the machine changes, because of an OS upgrade, or a
> > changed hard drive, or a different network environment, or because of
> > a new anti-virus product, then that confuses things. We need to
> > "control all the variables".
> >
> > One approach to controlling all of the variables is to have a machine
> > that is used for nothing but performance testing. That way we know
> > the machine's base performance does not change.
> >
> > Another approach is re-run the baseline AOO 3.4.1 performance tests
> > each week. This is more tolerant of changes in machine configuration,
> > etc.
> >
> > -Rob
> >
> >
> > > The test scenario include: AOO startup, file open, and save.
> > >
> > > Volunteers could run performance test on other platforms.
> > >
> > > All the automation scripts could be downloaded in AOO project. And it
> is
> > > not difficult to set the automation environment. You could follow the
> > guide:
> > > http://wiki.openoffice.org/wiki/QA/test_automation_guide
> > >
> > > For any questions, be free to contact with me :)
> >
>
Re: Call for volunteers for AOO 4.0 performance test.
Posted by Yi Xuan Liu <li...@gmail.com>.
Rob, I agree with you. The test environment configuration would impact the
performance test results. Therefore, we should keep the test configuration
unchanged between build to build.
I checked performance bugs list in bugzilla. 5 performance bugs are found
in automation performance test, such as memory leak and save performance
issue.
Lots of performance bugs are found in AOO daily usage. Therefore, for
volunteers who are not willing to run automation performance, you can also
report bugs for any performance issue and it will be helpful for us.
On Mon, Feb 25, 2013 at 9:16 PM, Rob Weir <ro...@apache.org> wrote:
> On Sun, Feb 24, 2013 at 11:42 PM, Yi Xuan Liu <li...@gmail.com>
> wrote:
> > Hi, all:
> >
> > AOO 4.0 will release. Performance plays an important role in software
> > quality. Is there any volunteer who want to run performance test?
> >
> > I've run AOO performance test on my own machines. I've tried on 3
> > platforms: Windows XP, Ubuntu 10.04, Mac Mac OS 10.7.3. The test
> configures
> > are as follows:
> >
> > (1) W500; CPU:2.53 GHz; Mem: 3GB; OS: XP SP3
> > (2) Ubuntu; CPU: Interl® Core™ 2 Duo 2 GHz; Mem: 3GB; OS: Ubuntu 10.04
> > (3) Mac; CPU: Interl® Core™ 2 Duo 2 GHz; Mem: 3GB; OS: Mac OS 10.7.3
> >
>
> I assume the volunteer does not need to have exactly the same machine
> type as you had. But they need some stability in the configuration.
> A performance test might be run first on the AOO 3.4.1 release to
> establish a baseline. Then re-test on a current 4.0 snapshot build.
> And then re-run on new dev snapshot builds, maybe once a week.
>
> The goal is to detect performance regressions early, so developers can fix
> it.
>
> The technical challenge here is to preserve a stable machine
> configuration. If the machine changes, because of an OS upgrade, or a
> changed hard drive, or a different network environment, or because of
> a new anti-virus product, then that confuses things. We need to
> "control all the variables".
>
> One approach to controlling all of the variables is to have a machine
> that is used for nothing but performance testing. That way we know
> the machine's base performance does not change.
>
> Another approach is re-run the baseline AOO 3.4.1 performance tests
> each week. This is more tolerant of changes in machine configuration,
> etc.
>
> -Rob
>
>
> > The test scenario include: AOO startup, file open, and save.
> >
> > Volunteers could run performance test on other platforms.
> >
> > All the automation scripts could be downloaded in AOO project. And it is
> > not difficult to set the automation environment. You could follow the
> guide:
> > http://wiki.openoffice.org/wiki/QA/test_automation_guide
> >
> > For any questions, be free to contact with me :)
>
Re: Call for volunteers for AOO 4.0 performance test.
Posted by Rob Weir <ro...@apache.org>.
On Mon, Feb 25, 2013 at 11:57 AM, Karen <el...@gmail.com> wrote:
> Rob,
>
> I thought I would be able to help with checking out bugs, but it is way more
> complicated than my skill level. Would you please remove me from the list
> of volunteers?
>
Hi Karen,
Note that the performance test stuff is more complicated. I wouldn't
expect everyone to be interested in that. But there are also other,
easier QA tasks, like manual test case execution.
But if you do want to be removed from the mailing list, you should
send an email to qa-unsubscribe@openoffice.apache.org. You'll then
receive a confirmation email, and once you respond that you will stop
receiving posts from the mailing list.
Regards,
-Rob
> Thank you,
>
> Karen Furber
>
>
> On 2/25/2013 5:16 AM, Rob Weir wrote:
>>
>> On Sun, Feb 24, 2013 at 11:42 PM, Yi Xuan Liu <li...@gmail.com>
>> wrote:
>>>
>>> Hi, all:
>>>
>>> AOO 4.0 will release. Performance plays an important role in software
>>> quality. Is there any volunteer who want to run performance test?
>>>
>>> I've run AOO performance test on my own machines. I've tried on 3
>>> platforms: Windows XP, Ubuntu 10.04, Mac Mac OS 10.7.3. The test
>>> configures
>>> are as follows:
>>>
>>> (1) W500; CPU:2.53 GHz; Mem: 3GB; OS: XP SP3
>>> (2) Ubuntu; CPU: Interl® Core™ 2 Duo 2 GHz; Mem: 3GB; OS: Ubuntu 10.04
>>> (3) Mac; CPU: Interl® Core™ 2 Duo 2 GHz; Mem: 3GB; OS: Mac OS 10.7.3
>>>
>> I assume the volunteer does not need to have exactly the same machine
>> type as you had. But they need some stability in the configuration.
>> A performance test might be run first on the AOO 3.4.1 release to
>> establish a baseline. Then re-test on a current 4.0 snapshot build.
>> And then re-run on new dev snapshot builds, maybe once a week.
>>
>> The goal is to detect performance regressions early, so developers can fix
>> it.
>>
>> The technical challenge here is to preserve a stable machine
>> configuration. If the machine changes, because of an OS upgrade, or a
>> changed hard drive, or a different network environment, or because of
>> a new anti-virus product, then that confuses things. We need to
>> "control all the variables".
>>
>> One approach to controlling all of the variables is to have a machine
>> that is used for nothing but performance testing. That way we know
>> the machine's base performance does not change.
>>
>> Another approach is re-run the baseline AOO 3.4.1 performance tests
>> each week. This is more tolerant of changes in machine configuration,
>> etc.
>>
>> -Rob
>>
>>
>>> The test scenario include: AOO startup, file open, and save.
>>>
>>> Volunteers could run performance test on other platforms.
>>>
>>> All the automation scripts could be downloaded in AOO project. And it is
>>> not difficult to set the automation environment. You could follow the
>>> guide:
>>> http://wiki.openoffice.org/wiki/QA/test_automation_guide
>>>
>>> For any questions, be free to contact with me :)
>
>
Re: Call for volunteers for AOO 4.0 performance test.
Posted by Karen <el...@gmail.com>.
Rob,
I thought I would be able to help with checking out bugs, but it is way
more complicated than my skill level. Would you please remove me from
the list of volunteers?
Thank you,
Karen Furber
On 2/25/2013 5:16 AM, Rob Weir wrote:
> On Sun, Feb 24, 2013 at 11:42 PM, Yi Xuan Liu <li...@gmail.com> wrote:
>> Hi, all:
>>
>> AOO 4.0 will release. Performance plays an important role in software
>> quality. Is there any volunteer who want to run performance test?
>>
>> I've run AOO performance test on my own machines. I've tried on 3
>> platforms: Windows XP, Ubuntu 10.04, Mac Mac OS 10.7.3. The test configures
>> are as follows:
>>
>> (1) W500; CPU:2.53 GHz; Mem: 3GB; OS: XP SP3
>> (2) Ubuntu; CPU: Interl® Core™ 2 Duo 2 GHz; Mem: 3GB; OS: Ubuntu 10.04
>> (3) Mac; CPU: Interl® Core™ 2 Duo 2 GHz; Mem: 3GB; OS: Mac OS 10.7.3
>>
> I assume the volunteer does not need to have exactly the same machine
> type as you had. But they need some stability in the configuration.
> A performance test might be run first on the AOO 3.4.1 release to
> establish a baseline. Then re-test on a current 4.0 snapshot build.
> And then re-run on new dev snapshot builds, maybe once a week.
>
> The goal is to detect performance regressions early, so developers can fix it.
>
> The technical challenge here is to preserve a stable machine
> configuration. If the machine changes, because of an OS upgrade, or a
> changed hard drive, or a different network environment, or because of
> a new anti-virus product, then that confuses things. We need to
> "control all the variables".
>
> One approach to controlling all of the variables is to have a machine
> that is used for nothing but performance testing. That way we know
> the machine's base performance does not change.
>
> Another approach is re-run the baseline AOO 3.4.1 performance tests
> each week. This is more tolerant of changes in machine configuration,
> etc.
>
> -Rob
>
>
>> The test scenario include: AOO startup, file open, and save.
>>
>> Volunteers could run performance test on other platforms.
>>
>> All the automation scripts could be downloaded in AOO project. And it is
>> not difficult to set the automation environment. You could follow the guide:
>> http://wiki.openoffice.org/wiki/QA/test_automation_guide
>>
>> For any questions, be free to contact with me :)
Re: Call for volunteers for AOO 4.0 performance test.
Posted by Rob Weir <ro...@apache.org>.
On Sun, Feb 24, 2013 at 11:42 PM, Yi Xuan Liu <li...@gmail.com> wrote:
> Hi, all:
>
> AOO 4.0 will release. Performance plays an important role in software
> quality. Is there any volunteer who want to run performance test?
>
> I've run AOO performance test on my own machines. I've tried on 3
> platforms: Windows XP, Ubuntu 10.04, Mac Mac OS 10.7.3. The test configures
> are as follows:
>
> (1) W500; CPU:2.53 GHz; Mem: 3GB; OS: XP SP3
> (2) Ubuntu; CPU: Interl® Core™ 2 Duo 2 GHz; Mem: 3GB; OS: Ubuntu 10.04
> (3) Mac; CPU: Interl® Core™ 2 Duo 2 GHz; Mem: 3GB; OS: Mac OS 10.7.3
>
I assume the volunteer does not need to have exactly the same machine
type as you had. But they need some stability in the configuration.
A performance test might be run first on the AOO 3.4.1 release to
establish a baseline. Then re-test on a current 4.0 snapshot build.
And then re-run on new dev snapshot builds, maybe once a week.
The goal is to detect performance regressions early, so developers can fix it.
The technical challenge here is to preserve a stable machine
configuration. If the machine changes, because of an OS upgrade, or a
changed hard drive, or a different network environment, or because of
a new anti-virus product, then that confuses things. We need to
"control all the variables".
One approach to controlling all of the variables is to have a machine
that is used for nothing but performance testing. That way we know
the machine's base performance does not change.
Another approach is re-run the baseline AOO 3.4.1 performance tests
each week. This is more tolerant of changes in machine configuration,
etc.
-Rob
> The test scenario include: AOO startup, file open, and save.
>
> Volunteers could run performance test on other platforms.
>
> All the automation scripts could be downloaded in AOO project. And it is
> not difficult to set the automation environment. You could follow the guide:
> http://wiki.openoffice.org/wiki/QA/test_automation_guide
>
> For any questions, be free to contact with me :)