You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@openoffice.apache.org by Peter Kovacs <pe...@apache.org> on 2021/10/24 12:31:58 UTC
QA automatisation in AOO
Hello all,
In the discussion on API, and releases Jörg has ask about how we could
improve QA.
The only way in general is to formalize. We need to define testcases and
automate those.
There are 2 ways of Automation:
1) is through Code
2) is through GUI.
Today both can be automated.
@1)
The Automation through code is already in place, but needs optimization
and improvements.
We lack KPIs like test coverage, We lack a reporting facility and we
have issues during the build.
@2)
What we have are test cases written on [1]. What we need is to extend them.
And then we need to automate them to get a good test coverage.
Also in this field we have no statistics. I think the last volunteer
looked into it has been Kai 3 years ago. I am not aware that we have
anyone who is looking into it now.
We have also a lot of Testcases in Bugzilla. Also some years ago
Mechtilde and I looked at some Test tool for Testers. But we did not
come to any conclusion.
Once we have testcases, we can again Automate them. These Tools are
called Robots, and remember in some ways in game bots. They click on the
Application window and check for results.
There are open sourced solution and Commercial solutions with community
editions.
That is what I have in mind when I talk in QA.
--
This is the Way! http://www.apache.org/theapacheway/index.html
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@openoffice.apache.org
For additional commands, e-mail: dev-help@openoffice.apache.org
Re: QA automatisation in AOO
Posted by Carl Marcum <cm...@apache.org>.
Hi Peter and All,
On 10/24/21 2:47 PM, Peter Kovacs wrote:
> Hi,
>
> On 24.10.21 20:39, Jörg Schmidt wrote:
>> Hello,
>>
>>> -----Original Message-----
>>> From: Peter Kovacs [mailto:petko@apache.org]
>>> Sent: Sunday, October 24, 2021 2:32 PM
>>> To: dev@openoffice.apache.org
>>> Subject: QA automatisation in AOO
>>>
>>> Hello all,
>>>
>>> In the discussion on API, and releases Jörg has ask about how
>>> we could
>>> improve QA.
>>>
>>> The only way in general is to formalize. We need to define
>>> testcases and
>>> automate those.
>> That sounds good, but we need specifications to define test cases.
>>
>> Not all tests have to run automatically (e.g. an installation test is
>> more likely to be done manually), but it has to be defined what the
>> minimum requirements are - that's what I mean by specifications.
> Well, no one will stop anyone to execute the test cases.
>>
>>> What we have are test cases written on [1].
>> Can you please post a link to this (or what does "[1]" mean?)
>
> Sorry, forgot to add the link.
>
> [1] https://www.openoffice.org/qa/testcase/index.html
>
> But note: these are the onbes I am aware of.
>
> Checkout the Mail that announced it:
>
> https://lists.apache.org/thread.html/dad44868cf67a2bbbbad627a0c27fde213e69cc3b6dfd8617e55e553%40%3Cdev.openoffice.apache.org%3E
>
I believe these were extracted from the old testlink system before we
lost it.
Many of these are included in the automated test suites as indicated by
the automation script names on the individual pages.
Not sure how many or what the coverage is and that would be good to
know. Also as a source of a TODO list for volunteers.
There are currently 867 functional tests and 49 build verification tests
as well as performance tests that I haven't looked at yet.
I'll also point out my current work on running the automated suites
separate from the need to build the office in the standalone-test branch
which I hope to merge sometime soon.
I'd be happy to collaborate if anyone wants to help.
We should move QA specific discussions to the QA list when the time comes.
Best regards,
Carl
>
>>
>>
>> greetings,
>> Jörg
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@openoffice.apache.org
>> For additional commands, e-mail: dev-help@openoffice.apache.org
>>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@openoffice.apache.org
For additional commands, e-mail: dev-help@openoffice.apache.org
Re: QA automatisation in AOO
Posted by Peter Kovacs <pe...@apache.org>.
Hi!
On 24.10.21 21:27, MENGUAL Jean-Philippe wrote:
> Hi,
>
> I am developing an infra for QA. Initially aimed for accessibility, I
> think the whole community can use this: they are based on Dogtail and
> track events from at-spi, on Linux, that enables to see if a key press
> or a mouse mouvement has the appropriate feedback from an event point
> of view. Perhaps it may be ported to other platforms.
>
> Anyway, I now have a high database for Writer scenarios. I can help
> providing such scenarios, translating those I created (there are
> several hunderds of them). Just need a wiki for this or something like
> this. If you want the example of usage of dogtail, see
> https://people.debian.org/~jpmengual/test.tar.gz
>
> Writer has already scenarios coded for LO 4.2, because it worked from
> an accessibilit piont of view, but these can be ported.
If you are fine with a wiki page, do you have an MWiki Account? If not
request one by sending a mail to dev list. We need the email address you
want to be associated with and a nickname. (Use a specific Subject like
"MWiki account needed" so we are aware on the need.)
I found a section for QA Test Automation at [1], which is interesting
for all that are interested in QA.
As a starting point I added [2] for test cases with dogtails. Feel free
to use the page or create additional pages.
HTH
All the best
Peter
[1] https://wiki.openoffice.org/wiki/QA/test_automation
[2] https://wiki.openoffice.org/wiki/QA/dogtail
--
This is the Way! http://www.apache.org/theapacheway/index.html
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@openoffice.apache.org
For additional commands, e-mail: dev-help@openoffice.apache.org
RE: QA automatisation in AOO
Posted by Jörg Schmidt <jo...@j-m-schmidt.de>.
> -----Original Message-----
> From: MENGUAL Jean-Philippe [mailto:mengualjeanphi@free.fr]
> Sent: Sunday, October 24, 2021 9:27 PM
> To: dev@openoffice.apache.org; Peter Kovacs
> Subject: Re: QA automatisation in AOO
>
> Hi,
>
> I am developing an infra for QA. Initially aimed for accessibility, I
> think the whole community can use this: they are based on Dogtail and
> track events from at-spi, on Linux, that enables to see if a
> key press
> or a mouse mouvement has the appropriate feedback from an
> event point of
> view. Perhaps it may be ported to other platforms.
That would be good and (I think) important for practice, because it is not uncommon for certain problems to occur only under one operating system.
For example, in the last 3 years or so, I have had API problems (or general programming problems) with a database project of one of my customers under UBUNTU/LO, which do not occur under Windows.
> Anyway, I now have a high database for Writer scenarios. I can help
> providing such scenarios, translating those I created (there
> are several
> hunderds of them). Just need a wiki for this or something
> like this. If
> you want the example of usage of dogtail, see
> https://people.debian.org/~jpmengual/test.tar.gz
That seems like a good offer to me. Thank you for that.
greetings,
Jörg
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@openoffice.apache.org
For additional commands, e-mail: dev-help@openoffice.apache.org
Re: QA automatisation in AOO
Posted by MENGUAL Jean-Philippe <me...@free.fr>.
Hi,
I am developing an infra for QA. Initially aimed for accessibility, I
think the whole community can use this: they are based on Dogtail and
track events from at-spi, on Linux, that enables to see if a key press
or a mouse mouvement has the appropriate feedback from an event point of
view. Perhaps it may be ported to other platforms.
Anyway, I now have a high database for Writer scenarios. I can help
providing such scenarios, translating those I created (there are several
hunderds of them). Just need a wiki for this or something like this. If
you want the example of usage of dogtail, see
https://people.debian.org/~jpmengual/test.tar.gz
Writer has already scenarios coded for LO 4.2, because it worked from an
accessibilit piont of view, but these can be ported.
Regards
Le 24/10/2021 à 20:47, Peter Kovacs a écrit :
> Hi,
>
> On 24.10.21 20:39, Jörg Schmidt wrote:
>> Hello,
>>
>>> -----Original Message-----
>>> From: Peter Kovacs [mailto:petko@apache.org]
>>> Sent: Sunday, October 24, 2021 2:32 PM
>>> To: dev@openoffice.apache.org
>>> Subject: QA automatisation in AOO
>>>
>>> Hello all,
>>>
>>> In the discussion on API, and releases Jörg has ask about how
>>> we could
>>> improve QA.
>>>
>>> The only way in general is to formalize. We need to define
>>> testcases and
>>> automate those.
>> That sounds good, but we need specifications to define test cases.
>>
>> Not all tests have to run automatically (e.g. an installation test is
>> more likely to be done manually), but it has to be defined what the
>> minimum requirements are - that's what I mean by specifications.
> Well, no one will stop anyone to execute the test cases.
>>
>>> What we have are test cases written on [1].
>> Can you please post a link to this (or what does "[1]" mean?)
>
> Sorry, forgot to add the link.
>
> [1] https://www.openoffice.org/qa/testcase/index.html
>
> But note: these are the onbes I am aware of.
>
> Checkout the Mail that announced it:
>
> https://lists.apache.org/thread.html/dad44868cf67a2bbbbad627a0c27fde213e69cc3b6dfd8617e55e553%40%3Cdev.openoffice.apache.org%3E
>
>>
>>
>> greetings,
>> Jörg
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@openoffice.apache.org
>> For additional commands, e-mail: dev-help@openoffice.apache.org
>>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@openoffice.apache.org
For additional commands, e-mail: dev-help@openoffice.apache.org
Re: QA automatisation in AOO
Posted by Peter Kovacs <pe...@apache.org>.
Hi,
On 24.10.21 20:39, Jörg Schmidt wrote:
> Hello,
>
>> -----Original Message-----
>> From: Peter Kovacs [mailto:petko@apache.org]
>> Sent: Sunday, October 24, 2021 2:32 PM
>> To: dev@openoffice.apache.org
>> Subject: QA automatisation in AOO
>>
>> Hello all,
>>
>> In the discussion on API, and releases Jörg has ask about how
>> we could
>> improve QA.
>>
>> The only way in general is to formalize. We need to define
>> testcases and
>> automate those.
> That sounds good, but we need specifications to define test cases.
>
> Not all tests have to run automatically (e.g. an installation test is more likely to be done manually), but it has to be defined what the minimum requirements are - that's what I mean by specifications.
Well, no one will stop anyone to execute the test cases.
>
>> What we have are test cases written on [1].
> Can you please post a link to this (or what does "[1]" mean?)
Sorry, forgot to add the link.
[1] https://www.openoffice.org/qa/testcase/index.html
But note: these are the onbes I am aware of.
Checkout the Mail that announced it:
https://lists.apache.org/thread.html/dad44868cf67a2bbbbad627a0c27fde213e69cc3b6dfd8617e55e553%40%3Cdev.openoffice.apache.org%3E
>
>
> greetings,
> Jörg
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@openoffice.apache.org
> For additional commands, e-mail: dev-help@openoffice.apache.org
>
--
This is the Way! http://www.apache.org/theapacheway/index.html
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@openoffice.apache.org
For additional commands, e-mail: dev-help@openoffice.apache.org
RE: QA automatisation in AOO
Posted by Jörg Schmidt <jo...@j-m-schmidt.de>.
Hello,
> -----Original Message-----
> From: Peter Kovacs [mailto:petko@apache.org]
> Sent: Sunday, October 24, 2021 2:32 PM
> To: dev@openoffice.apache.org
> Subject: QA automatisation in AOO
>
> Hello all,
>
> In the discussion on API, and releases Jörg has ask about how
> we could
> improve QA.
>
> The only way in general is to formalize. We need to define
> testcases and
> automate those.
That sounds good, but we need specifications to define test cases.
Not all tests have to run automatically (e.g. an installation test is more likely to be done manually), but it has to be defined what the minimum requirements are - that's what I mean by specifications.
> What we have are test cases written on [1].
Can you please post a link to this (or what does "[1]" mean?)
greetings,
Jörg
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@openoffice.apache.org
For additional commands, e-mail: dev-help@openoffice.apache.org