You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@openoffice.apache.org by Ji Yan <ya...@cn.ibm.com> on 2011/11/02 01:58:54 UTC
Willing help on Test
Hi all,
This is Yan Ji from IBM Lotus Symphony test team. I've been working in
Symphony for almost five years . It's my honour to join the community. As a
QE of Symphony my major focus is FVT in both
WordProcessor/Spreadsheet/Presentation and Install. Now I'm willing to
contribute my effort on Apache OpenOffice. Please let me know if there is
anything I can help on OO 3.4 release.
Since most OO website are unreachable. Could someone help me to find OO
TCM or TCS, it will help me to understand how the OO test way works.
Looking forward to hear from you.
Thanks & Best Regards, Yan Ji
IBM Lotus Symphony QE, FVT & Integration
IBM China Software Development LAB, Beijing
Tel: 86-10-82454393 Notes ID: Ji Yan/China/IBM
Fax: 86-10-82451161 Email: yanji@cn.ibm.com
2/F, 28,ZhongGuanCun Software Park,No.8 Dong Bei Wang West Road, Haidian
District Beijing P.R.China 100193
Re: Willing help on Test
Posted by Raphael Bircher <r....@gmx.ch>.
Am 02.11.11 16:43, schrieb Wolfram Garten:
> Hi Raphael,
>
> Am 02.11.2011 03:30, schrieb Raphael Bircher:
>> Hi Ji Yan
>>
>> Am 02.11.11 01:58, schrieb Ji Yan:
>>>
>>> Hi all,
>>>
>>> This is Yan Ji from IBM Lotus Symphony test team. I've been
>>> working in
>>> Symphony for almost five years . It's my honour to join the
>>> community. As a
>>> QE of Symphony my major focus is FVT in both
>>> WordProcessor/Spreadsheet/Presentation and Install. Now I'm willing to
>>> contribute my effort on Apache OpenOffice. Please let me know if
>>> there is
>>> anything I can help on OO 3.4 release.
>> Welcome! At the moment we have no public builds, but you can build a
>> AOOo your self or maybe a other IBMer do it for you ;-) We have
>> different work to do:
>>
>> 1) Bugzilla editing unconfirmed issues
>> 2) Automation: Setup test boxes and make it ready to start
> eh...do you mean testmachines for automated testing ? In former times
> I can remember that ( I think it was) Maho who provided important
> files for the automated testers of the community outside from Oracle.
> Is he still doing so or did he left to LO? I do not know...
Maho was providing a compressed testautomation directory on the good-day
servers. At the moment, Maho search a new server, because the good-day
is no longer aviable for OpenOffice.org. Anyway, to get the
testautomation directory was never realy the key problem.
"testautomation" is still in the source, and everyone can checkout it.
The problem is more, that the TT never run in a productive mode outside
SUN/Oracle. Yes, same people like me experiment with the TT, but in fact
the TT was not used by the Community at all. I see different reasons,
why TT was never used outside SUN/Oracle
1) TT is complicate to use
2) SUN did this job, and there was no reason to do the same job again
3) SUN used tools who are not aviable for the Community (Testlauncher
for exemple)
So TT was in fact a more or less SUN internal tool. At SUN it was easy
(as I heared) to use the TT. This tests was hevy automated. SUN has
special VM's only dedicated for this tests. But we have no more access
to this infrastructure. The only thing we have is the TT and the
testscripts. If we will use TT then we have a load to rework.
Byside, we have not many people with TT experience. You can maybe help here.
Greetings Raphael
--
My private Homepage: http://www.raphaelbircher.ch/
Re: Willing help on Test
Posted by Wolfram Garten <wo...@googlemail.com>.
Hi Raphael,
Am 02.11.2011 03:30, schrieb Raphael Bircher:
> Hi Ji Yan
>
> Am 02.11.11 01:58, schrieb Ji Yan:
>>
>> Hi all,
>>
>> This is Yan Ji from IBM Lotus Symphony test team. I've been
>> working in
>> Symphony for almost five years . It's my honour to join the
>> community. As a
>> QE of Symphony my major focus is FVT in both
>> WordProcessor/Spreadsheet/Presentation and Install. Now I'm willing to
>> contribute my effort on Apache OpenOffice. Please let me know if
>> there is
>> anything I can help on OO 3.4 release.
> Welcome! At the moment we have no public builds, but you can build a
> AOOo your self or maybe a other IBMer do it for you ;-) We have
> different work to do:
>
> 1) Bugzilla editing unconfirmed issues
> 2) Automation: Setup test boxes and make it ready to start
eh...do you mean testmachines for automated testing ? In former times I
can remember that ( I think it was) Maho who provided important files
for the automated testers of the community outside from Oracle. Is he
still doing so or did he left to LO? I do not know...
> 3) Querbeet tests
hm. This might be a little bit hard to understand for people outside of
the german-speaking countries. Querbeet means random, all over the
place. Simply testing what you are interested in and what makes fun and
sense. Sorry, Raphael, I did not mean to be a know-it-all ;-)
>>
>> Since most OO website are unreachable. Could someone help me to
>> find OO
>> TCM or TCS, it will help me to understand how the OO test way works.
>> Looking forward to hear from you.
> Unfortionaly we have no TCM/TCS. Our old TCM was a proprietary tool,
> and we can't take over it. There was a plane to replace the TCM within
> QUASTe. But this one is still in Beta, and no one working on it since
> it was developed by Helge Delfs who was one of the Hamburg Oracle Team.
>
> But anyway. The workflow by OOo has changed, and we need to reorganize
> the QA. And first we should reorganize the QA and then we should talk
> about the needed tools. Thats my option.
>
> Greetings Raphael
Regards, Wolfram
Re: Willing help on Test
Posted by Mathias Bauer <Ma...@gmx.net>.
Am 02.11.2011 14:34, schrieb Rob Weir:
> So what do we have? What do we need?
>
> I have no idea how QA was done before for OpenOffice.org, but it make
> sense that you have basic elements like:
>
>
> 1) Unit tests that developers can execute before checking in code. We
> already have those, right? Are they working? Do they have good
> coverage? Would it be worth improving testing at that level?
Unit tests exist only for some low level libraries. We have some so
called "complex tests" and some simple API tests. All of them are
definitely worth to get improved, it's the best we could do.
We never investigated coverage, so no idea how much code is covered by
these tests.
Writing unit tests for most of the "higher level" OOo code is hard or
close to impossible, as the code refuses to be run in a test harness.
Too many code is depending on too much other code.
> 2) Manual scripted tests. This could be based on written test cases
> and test documents. These tests require some expertise to
> design/write, but once the test cases are written they can be tested
> by a much larger set of volunteers. Even power users could be helpful
> here. A good tester follows the test case, but also has skills in
> describing a bug in the defect report, with all necessary detail, but
> little extraneous detail. They know "how to think like a bug".
That might be comparable to what I called "complex test cases". As
writing unit tests is hard for many components, as mentioned above, this
is the kind of testing that gives us the most bang for the bug. The
build system has support for building and running them and basically all
of them could be run in parallel, if set up accordingly.
> 4) Scripted/automated testing via the GUI. Requires more effort and
> skill to write and maintain, but once done, it requires less effort
> to execute.
That depends on what you mean by "effort". The tests that we have run
awfully slow - even the most basic tests together sum up to a run time
of approximately 8 hours. If you wanted to run all tests that have been
written (and why wouldn't you want to do that?) you had to invest
several days for just one platform.
There are several reasons for that. It could be improved by running as
much tests in parallel as possible, using as much cores of the test
machine as possible. If the test infrastructure wouldn't have been so
byzantine and inflexible, we probably could have done that already.
The test tool and what we can still do with it might become a larger
topic, I will open a new thread for it.
Regards,
Mathias
Re: Willing help on Test
Posted by Andrea Pescetti <pe...@openoffice.org>.
On 02/11/2011 Rob Weir wrote:
> I have no idea how QA was done before for OpenOffice.org, but it make
> sense that you have basic elements like: ...
> 2) Manual scripted tests. This could be based on written test cases
> and test documents. These tests require some expertise to
> design/write, but once the test cases are written they can be tested
> by a much larger set of volunteers. Even power users could be helpful
The guess is mostly correct, but it misses the localization aspect. N-L
teams used to have coordinated, but independent, QA activites; and here
TCM was a great tool to have, since each team could assign a set of
tests (set "Writer-1", set "Writer-2", set "Calc-1" and so on) to
volunteers and make sure that important functionality was tested in all
languages.
This allowed to find localization bugs but also generic bugs, since
several teams were running the full testsuite and providing multiple
feedback.
The final version of OpenOffice.org was released when blocker bugs,
including those affecting only a specific localized version (like a
translation error that had broken styles management in Writer in the
Italian version) had been fixed.
Needless to say, localized testing is a great way to engage volunteers
from N-L communities.
Regards,
Andrea.
Re: Willing help on Test
Posted by Rob Weir <ro...@apache.org>.
On Wed, Nov 2, 2011 at 4:13 AM, Oliver-Rainer Wittmann
<or...@googlemail.com> wrote:
> Hi,
>
> On 02.11.2011 03:30, Raphael Bircher wrote:
>>
>> ...
>>
>> But anyway. The workflow by OOo has changed, and we need to reorganize
>> the QA. And first we should reorganize the QA and then we should talk
>> about the needed tools. Thats my option.
>>
>
> I agree here with Raphael that things have changed.
> Former stuff worked, but there is no need to recover everything.
>
> Just a developer's point of view on this.
>
So what do we have? What do we need?
I have no idea how QA was done before for OpenOffice.org, but it make
sense that you have basic elements like:
1) Unit tests that developers can execute before checking in code. We
already have those, right? Are they working? Do they have good
coverage? Would it be worth improving testing at that level?
2) Manual scripted tests. This could be based on written test cases
and test documents. These tests require some expertise to
design/write, but once the test cases are written they can be tested
by a much larger set of volunteers. Even power users could be helpful
here. A good tester follows the test case, but also has skills in
describing a bug in the defect report, with all necessary detail, but
little extraneous detail. They know "how to think like a bug".
3) Free form testing. Volunteers are asked to test a new build with
little additional direction. Maybe they are asked to "focus" on a
particular area, but they are not following a test script or plan.
This is not really an quality engineering approach since we have no
idea of test coverage or effort. So in practice this is not
sufficient, though it could supplement a QA plan.
4) Scripted/automated testing via the GUI. Requires more effort and
skill to write and maintain, but once done, it requires less effort
to execute.
For all of these, the question we should be asking as a product is --
is it ready to release? What is our confidence that the release does
not have a horrible bug some place? In other words, a key question
is test coverage? What portion of the product's features have been
tested?
So how do we get started on this? Do we have test plans that if we
found volunteers to execute the tests, even manually, they would give
high test coverage?
Also, with existing volunteers, what mix of skills do we have? Test
plan development? Test execution? Bugzilla issue verification? Test
automation design and coding?
-Rob
> Best regards, Oliver.
>
Re: Willing help on Test
Posted by Oliver-Rainer Wittmann <or...@googlemail.com>.
Hi,
On 02.11.2011 03:30, Raphael Bircher wrote:
> ...
>
> But anyway. The workflow by OOo has changed, and we need to reorganize
> the QA. And first we should reorganize the QA and then we should talk
> about the needed tools. Thats my option.
>
I agree here with Raphael that things have changed.
Former stuff worked, but there is no need to recover everything.
Just a developer's point of view on this.
Best regards, Oliver.
Re: Willing help on Test
Posted by Raphael Bircher <r....@gmx.ch>.
Hi Ji Yan
Am 02.11.11 01:58, schrieb Ji Yan:
>
> Hi all,
>
> This is Yan Ji from IBM Lotus Symphony test team. I've been working in
> Symphony for almost five years . It's my honour to join the community. As a
> QE of Symphony my major focus is FVT in both
> WordProcessor/Spreadsheet/Presentation and Install. Now I'm willing to
> contribute my effort on Apache OpenOffice. Please let me know if there is
> anything I can help on OO 3.4 release.
Welcome! At the moment we have no public builds, but you can build a
AOOo your self or maybe a other IBMer do it for you ;-) We have
different work to do:
1) Bugzilla editing unconfirmed issues
2) Automation: Setup test boxes and make it ready to start
3) Querbeet tests
>
> Since most OO website are unreachable. Could someone help me to find OO
> TCM or TCS, it will help me to understand how the OO test way works.
> Looking forward to hear from you.
Unfortionaly we have no TCM/TCS. Our old TCM was a proprietary tool, and
we can't take over it. There was a plane to replace the TCM within
QUASTe. But this one is still in Beta, and no one working on it since it
was developed by Helge Delfs who was one of the Hamburg Oracle Team.
But anyway. The workflow by OOo has changed, and we need to reorganize
the QA. And first we should reorganize the QA and then we should talk
about the needed tools. Thats my option.
Greetings Raphael
--
My private Homepage: http://www.raphaelbircher.ch/
Re: Willing help on Test
Posted by Wolfram Garten <wo...@googlemail.com>.
Hi Ji Yan,
welcome to the community.
as it was already said, a lot of Test-Specifications have been lost with
the Quaste server.
Maybe this here can help: http://specs.openoffice.org/ These
specifications were used often to generate the tesplans for manual and
automated testing.
I know this is not what you have searched for but it is getting pretty
close ;-)
Maybe this can help you.
Regards, Wolfram
At least Am 02.11.2011 01:58, schrieb Ji Yan:
>
> Hi all,
>
> This is Yan Ji from IBM Lotus Symphony test team. I've been working in
> Symphony for almost five years . It's my honour to join the community. As a
> QE of Symphony my major focus is FVT in both
> WordProcessor/Spreadsheet/Presentation and Install. Now I'm willing to
> contribute my effort on Apache OpenOffice. Please let me know if there is
> anything I can help on OO 3.4 release.
>
> Since most OO website are unreachable. Could someone help me to find OO
> TCM or TCS, it will help me to understand how the OO test way works.
> Looking forward to hear from you.
>
>
> Thanks& Best Regards, Yan Ji
>
> IBM Lotus Symphony QE, FVT& Integration
> IBM China Software Development LAB, Beijing
> Tel: 86-10-82454393 Notes ID: Ji Yan/China/IBM
> Fax: 86-10-82451161 Email: yanji@cn.ibm.com
> 2/F, 28,ZhongGuanCun Software Park,No.8 Dong Bei Wang West Road, Haidian
> District Beijing P.R.China 100193