You are viewing a plain text version of this content. The canonical link for it is here.
Posted to qa@openoffice.apache.org by V Stuart Foote <VS...@utsa.edu> on 2013/05/07 17:46:19 UTC
RE: Start AOO 4.0 Full Regression Test - missing keyboard
navigation and accelerators in test cases
Shenfeng, Yuzhen, Liuping, Rob
Took a moment and browsed through the TestLink test cases for the Full Regression and earlier test plans against the trunk and sidebar branch.
Unless I completely missed seeing it, one noticeable omission from the test cases in general is a lack keyboard based navigation and use of accelerators. Would expect that to be a significant component of functional testing.
Only testing with mouse and menu based usage is unfortunately a way to miss the framework needed for accessibility and Assistive Technology support and impact all the supported OS builds. Not just the work on the IA2 branch.
Stuart
San Antonio, Texas
________________________________________
From: Shenfeng Liu [liushenf@gmail.com]
Sent: Tuesday, May 07, 2013 8:28 AM
To: qa@openoffice.apache.org; dev@openoffice.apache.org
Subject: Start AOO 4.0 Full Regression Test
Yu Zhen & Liu Ping,
Thanks very much for your help to prepare for the test guidance!
Now we have the latest dev snapshot build here:
https://cwiki.apache.org/confluence/display/OOOUSERS/Development+Snapshot+Builds#DevelopmentSnapshotBuilds-AOOSnapshotfullsets
And we have the "AOO 4.0 Full Regression Test" test plan created in
Testlink: http://aootesting.adfinis-sygroup.org/index.php .
And we have test guidance here:
https://cwiki.apache.org/confluence/display/OOOUSERS/AOO+Full+Regression+Test+Guidance
and here:
https://cwiki.apache.org/confluence/display/OOOUSERS/AOO+Defect+Verification+Guidance
So I suggest we start the AOO 4.0 Full Regression Test now !!!!!
The test focus of this test circle include the following:
(1) Basic Function Verification
(2) MS Interoperability Test
(3) 2nd phase of sidebar testing
(4) Defect verification
We will add more test cases (e.g. the Upgrade Check) in later circle.
We already got some volunteers replied in mail list. Thanks very much!
Will assign test cases in Testlink and send mail to you one by one.
And looking forward more people to join and help the testing!
- Shenfeng (Simon)
---------------------------------------------------------------------
To unsubscribe, e-mail: qa-unsubscribe@openoffice.apache.org
For additional commands, e-mail: qa-help@openoffice.apache.org
Re: Start AOO 4.0 Full Regression Test - missing keyboard navigation
and accelerators in test cases
Posted by janI <ja...@apache.org>.
On 7 May 2013 22:00, Rob Weir <ro...@apache.org> wrote:
> On Tue, May 7, 2013 at 3:42 PM, janI <ja...@apache.org> wrote:
> > On 7 May 2013 21:31, Rob Weir <ro...@apache.org> wrote:
> >
> >> On Tue, May 7, 2013 at 11:46 AM, V Stuart Foote <VStuart.Foote@utsa.edu
> >
> >> wrote:
> >> > Shenfeng, Yuzhen, Liuping, Rob
> >> >
> >> > Took a moment and browsed through the TestLink test cases for the Full
> >> Regression and earlier test plans against the trunk and sidebar branch.
> >> >
> >> > Unless I completely missed seeing it, one noticeable omission from the
> >> test cases in general is a lack keyboard based navigation and use of
> >> accelerators. Would expect that to be a significant component of
> >> functional testing.
> >> >
> >> > Only testing with mouse and menu based usage is unfortunately a way to
> >> miss the framework needed for accessibility and Assistive Technology
> >> support and impact all the supported OS builds. Not just the work on the
> >> IA2 branch.
> >> >
> >>
> >> So what would you recommend? Is this something you can help with, on
> >> the test definition or execution side?
> >>
> >> My naive approach would be, "Make sure you can perform all test cases
> >> with just the keyboard". Or is there a more targeted way to do this?
> >> A regression test (as opposed to a full functional test) tries to be
> >> smart about coverage and focuses on the areas that have changed in the
> >> most recent version. In other words, it puts the effort where the
> >> risk is greatest.
> >>
> >
> > Sounds like a good approach to me "just with the keyboard".
> >
> > I hope that for all automated testing we only use "full functional test".
> > With human resources involved regression testing is understandable.
> >
>
> The regression test is a manual test, not an automated one. Automated
> tests are "smoke tests" and have even less coverage than the
> regression tests. So automation, at least what we have now, is not
> going to solve this problem.
>
> >
> >>
> >> So, given what we know about AOO 3.4.1 keyboard support, and knowing
> >> what was changed in AOO 4.0, my guess would be the area to focus on
> >> would be the sidebar, yes?
> >>
> >> Translations are another area where I bet there is risk introduced.
> >> It is always possible for an accelerator collision to be introduced
> >> accidentally during translation. And the translations come in rather
> >> late in the process, after most testing. I wonder if we have any
> >> automated ways of detecting collisions?
> >>
> >
> > if/when we have keyboard automated testing, it would be an easy job to
> > write a script that maps the keyboard and accellerators to any language.
> > That would allow us to run the same tests in any language.
> >
>
> I was hoping for something closer to present capabilities. For
> example, accelerators are defined in resource files, yes? It should
> be possible to analyze the resources files directly to see if there is
> a collision within any menu item. In theory.
>
that is not just theory, I worked on a package earlier that did such an
analysis on PO files...actually it is quite easy. The benefit of doing it
on PO files is the consistency check (accelletor for a given menu item,
e.g. Open is identical in all product parts).
>
> > Based on my experience from international software, menus and
> accellerators
> > are common translation problems.
> >
>
> Yes.
>
> >
> >
> >>
> >> Any other areas that would make sense to focus on?
> >>
> >
> > Installation with a keyboard only, or automated installation
> >
>
> I didn't think the install UI changed in 4.0, did it?
>
> When I say "focus" I'm reasoning like this:
>
> 1) No product of the size and complexity of AOO is able to be fully
> tested, in any formal sense (line coverage, path coverage, etc.)
>
that is a statement...I have worked on bigger packages, where we had next
to full coverage (all automated), so the size is not the problem, our
restricted resources are, which is why we in the middleterm should focus on
automating as much a possible.
> 2) So we rely on a good sample of possible test cases that give good
> overall coverage, combined with more focused testing on areas where we
> think there is higher risk because of recent code changes.
>
for now that is the only thing we can do.
>
> 3) There are many areas that are important to specific users:
> performance, running on a server, users with bidirectional writing
> scripts, users with East Asian writing styles, users on older
> versions of Windows versus newest versions of Linux, etc., And
> keyboard use as well. We can't do every test in every combination.
> (What about Windows 8 Hebrew users without a mouse?) So we need to
> figure out the best combination of tests that give good overall
> coverage.
>
+1
>
> 4) To make the problem tractable areas that have not changed in AOO
> 4.0 will get less test coverage. Not the same as no coverage, since
> bugs can have strange interactions. But lacking 100% automated test
> suites with perfect coverage, infinite volunteers or infinite time, we
> need to figure out where to spend the time to find the most critical
> bugs. Where are those bugs? I'm thinking they are not in the
> install UI. But who knows...
>
+1
jan I.
>
> -Rob
>
> > rgds
> > Jan I.
> >
> >>
> >> Regards,
> >>
> >> -Rob
> >>
> >>
> >> > Stuart
> >> > San Antonio, Texas
> >> >
> >> >
> >> > ________________________________________
> >> > From: Shenfeng Liu [liushenf@gmail.com]
> >> > Sent: Tuesday, May 07, 2013 8:28 AM
> >> > To: qa@openoffice.apache.org; dev@openoffice.apache.org
> >> > Subject: Start AOO 4.0 Full Regression Test
> >> >
> >> > Yu Zhen & Liu Ping,
> >> > Thanks very much for your help to prepare for the test guidance!
> >> >
> >> > Now we have the latest dev snapshot build here:
> >> >
> >>
> https://cwiki.apache.org/confluence/display/OOOUSERS/Development+Snapshot+Builds#DevelopmentSnapshotBuilds-AOOSnapshotfullsets
> >> > And we have the "AOO 4.0 Full Regression Test" test plan created in
> >> > Testlink: http://aootesting.adfinis-sygroup.org/index.php .
> >> > And we have test guidance here:
> >> >
> >>
> https://cwiki.apache.org/confluence/display/OOOUSERS/AOO+Full+Regression+Test+Guidance
> >> > and here:
> >> >
> >>
> https://cwiki.apache.org/confluence/display/OOOUSERS/AOO+Defect+Verification+Guidance
> >> >
> >> > So I suggest we start the AOO 4.0 Full Regression Test now !!!!!
> >> >
> >> > The test focus of this test circle include the following:
> >> > (1) Basic Function Verification
> >> > (2) MS Interoperability Test
> >> > (3) 2nd phase of sidebar testing
> >> > (4) Defect verification
> >> >
> >> > We will add more test cases (e.g. the Upgrade Check) in later
> circle.
> >> >
> >> > We already got some volunteers replied in mail list. Thanks very
> much!
> >> > Will assign test cases in Testlink and send mail to you one by one.
> >> > And looking forward more people to join and help the testing!
> >> >
> >> > - Shenfeng (Simon)
> >> >
> >> >
> >> > ---------------------------------------------------------------------
> >> > To unsubscribe, e-mail: dev-unsubscribe@openoffice.apache.org
> >> > For additional commands, e-mail: dev-help@openoffice.apache.org
> >> >
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: qa-unsubscribe@openoffice.apache.org
> >> For additional commands, e-mail: qa-help@openoffice.apache.org
> >>
> >>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: qa-unsubscribe@openoffice.apache.org
> For additional commands, e-mail: qa-help@openoffice.apache.org
>
>
Re: Start AOO 4.0 Full Regression Test - missing keyboard navigation
and accelerators in test cases
Posted by janI <ja...@apache.org>.
On 7 May 2013 23:49, Andrea Pescetti <pe...@apache.org> wrote:
> Rob Weir wrote:
>
> On Tue, May 7, 2013 at 3:42 PM, janI wrote:
>>
>>> if/when we have keyboard automated testing, it would be an easy job to
>>> write a script that maps the keyboard and accellerators to any language.
>>> That would allow us to run the same tests in any language.
>>>
>> I was hoping for something closer to present capabilities. For
>> example, accelerators are defined in resource files, yes?
>>
>
> Accelerators are defined algorithmically by OpenOffice when not specified,
> and the algorithm avoids collisions. So the recommendation for translations
> is not to specify an accelerator unless you know what you are doing.
>
> Accelerators in PO files are marked by "~": "F~ormat" means that the
> accelerator is the letter O.
>
I found quite a lot, when I tested genLang, defined in the .src files. I
will try to make a grep later.
rgds
jan I.
>
> We do have an accelerator bug in the latest 4.x snapshot, feel free to
> post to Bugzilla if confirmed. In Calc, Alt+O opens F~ormat but, within
> "Format", there is no accelerator for the Flip command (and anything below
> it, apparently) since F, L, I, P are all taken by other commands. This
> might be one of the cases where we need manual definition (for example,
> mapping "AutoFormat" to "m"; or find and improve the algorithm!).
>
> Regards,
> Andrea.
>
>
> ------------------------------**------------------------------**---------
> To unsubscribe, e-mail: qa-unsubscribe@openoffice.**apache.org<qa...@openoffice.apache.org>
> For additional commands, e-mail: qa-help@openoffice.apache.org
>
>
Re: Start AOO 4.0 Full Regression Test - missing keyboard navigation
and accelerators in test cases
Posted by Andrea Pescetti <pe...@apache.org>.
On 07/05/2013 Andrea Pescetti wrote:
> We do have an accelerator bug in the latest 4.x snapshot, feel free to
> post to Bugzilla if confirmed. In Calc, Alt+O opens F~ormat but, within
> "Format", there is no accelerator for the Flip command (and anything
> below it, apparently) since F, L, I, P are all taken by other commands.
> This might be one of the cases where we need manual definition (for
> example, mapping "AutoFormat" to "m"; or find and improve the algorithm!).
Posted as https://issues.apache.org/ooo/show_bug.cgi?id=122288
Regards,
Andrea.
---------------------------------------------------------------------
To unsubscribe, e-mail: qa-unsubscribe@openoffice.apache.org
For additional commands, e-mail: qa-help@openoffice.apache.org
Re: Start AOO 4.0 Full Regression Test - missing keyboard navigation
and accelerators in test cases
Posted by Andrea Pescetti <pe...@apache.org>.
Rob Weir wrote:
> On Tue, May 7, 2013 at 3:42 PM, janI wrote:
>> if/when we have keyboard automated testing, it would be an easy job to
>> write a script that maps the keyboard and accellerators to any language.
>> That would allow us to run the same tests in any language.
> I was hoping for something closer to present capabilities. For
> example, accelerators are defined in resource files, yes?
Accelerators are defined algorithmically by OpenOffice when not
specified, and the algorithm avoids collisions. So the recommendation
for translations is not to specify an accelerator unless you know what
you are doing.
Accelerators in PO files are marked by "~": "F~ormat" means that the
accelerator is the letter O.
We do have an accelerator bug in the latest 4.x snapshot, feel free to
post to Bugzilla if confirmed. In Calc, Alt+O opens F~ormat but, within
"Format", there is no accelerator for the Flip command (and anything
below it, apparently) since F, L, I, P are all taken by other commands.
This might be one of the cases where we need manual definition (for
example, mapping "AutoFormat" to "m"; or find and improve the algorithm!).
Regards,
Andrea.
---------------------------------------------------------------------
To unsubscribe, e-mail: qa-unsubscribe@openoffice.apache.org
For additional commands, e-mail: qa-help@openoffice.apache.org
Re: Start AOO 4.0 Full Regression Test - missing keyboard navigation
and accelerators in test cases
Posted by Rob Weir <ro...@apache.org>.
On Tue, May 7, 2013 at 3:42 PM, janI <ja...@apache.org> wrote:
> On 7 May 2013 21:31, Rob Weir <ro...@apache.org> wrote:
>
>> On Tue, May 7, 2013 at 11:46 AM, V Stuart Foote <VS...@utsa.edu>
>> wrote:
>> > Shenfeng, Yuzhen, Liuping, Rob
>> >
>> > Took a moment and browsed through the TestLink test cases for the Full
>> Regression and earlier test plans against the trunk and sidebar branch.
>> >
>> > Unless I completely missed seeing it, one noticeable omission from the
>> test cases in general is a lack keyboard based navigation and use of
>> accelerators. Would expect that to be a significant component of
>> functional testing.
>> >
>> > Only testing with mouse and menu based usage is unfortunately a way to
>> miss the framework needed for accessibility and Assistive Technology
>> support and impact all the supported OS builds. Not just the work on the
>> IA2 branch.
>> >
>>
>> So what would you recommend? Is this something you can help with, on
>> the test definition or execution side?
>>
>> My naive approach would be, "Make sure you can perform all test cases
>> with just the keyboard". Or is there a more targeted way to do this?
>> A regression test (as opposed to a full functional test) tries to be
>> smart about coverage and focuses on the areas that have changed in the
>> most recent version. In other words, it puts the effort where the
>> risk is greatest.
>>
>
> Sounds like a good approach to me "just with the keyboard".
>
> I hope that for all automated testing we only use "full functional test".
> With human resources involved regression testing is understandable.
>
The regression test is a manual test, not an automated one. Automated
tests are "smoke tests" and have even less coverage than the
regression tests. So automation, at least what we have now, is not
going to solve this problem.
>
>>
>> So, given what we know about AOO 3.4.1 keyboard support, and knowing
>> what was changed in AOO 4.0, my guess would be the area to focus on
>> would be the sidebar, yes?
>>
>> Translations are another area where I bet there is risk introduced.
>> It is always possible for an accelerator collision to be introduced
>> accidentally during translation. And the translations come in rather
>> late in the process, after most testing. I wonder if we have any
>> automated ways of detecting collisions?
>>
>
> if/when we have keyboard automated testing, it would be an easy job to
> write a script that maps the keyboard and accellerators to any language.
> That would allow us to run the same tests in any language.
>
I was hoping for something closer to present capabilities. For
example, accelerators are defined in resource files, yes? It should
be possible to analyze the resources files directly to see if there is
a collision within any menu item. In theory.
> Based on my experience from international software, menus and accellerators
> are common translation problems.
>
Yes.
>
>
>>
>> Any other areas that would make sense to focus on?
>>
>
> Installation with a keyboard only, or automated installation
>
I didn't think the install UI changed in 4.0, did it?
When I say "focus" I'm reasoning like this:
1) No product of the size and complexity of AOO is able to be fully
tested, in any formal sense (line coverage, path coverage, etc.)
2) So we rely on a good sample of possible test cases that give good
overall coverage, combined with more focused testing on areas where we
think there is higher risk because of recent code changes.
3) There are many areas that are important to specific users:
performance, running on a server, users with bidirectional writing
scripts, users with East Asian writing styles, users on older
versions of Windows versus newest versions of Linux, etc., And
keyboard use as well. We can't do every test in every combination.
(What about Windows 8 Hebrew users without a mouse?) So we need to
figure out the best combination of tests that give good overall
coverage.
4) To make the problem tractable areas that have not changed in AOO
4.0 will get less test coverage. Not the same as no coverage, since
bugs can have strange interactions. But lacking 100% automated test
suites with perfect coverage, infinite volunteers or infinite time, we
need to figure out where to spend the time to find the most critical
bugs. Where are those bugs? I'm thinking they are not in the
install UI. But who knows...
-Rob
> rgds
> Jan I.
>
>>
>> Regards,
>>
>> -Rob
>>
>>
>> > Stuart
>> > San Antonio, Texas
>> >
>> >
>> > ________________________________________
>> > From: Shenfeng Liu [liushenf@gmail.com]
>> > Sent: Tuesday, May 07, 2013 8:28 AM
>> > To: qa@openoffice.apache.org; dev@openoffice.apache.org
>> > Subject: Start AOO 4.0 Full Regression Test
>> >
>> > Yu Zhen & Liu Ping,
>> > Thanks very much for your help to prepare for the test guidance!
>> >
>> > Now we have the latest dev snapshot build here:
>> >
>> https://cwiki.apache.org/confluence/display/OOOUSERS/Development+Snapshot+Builds#DevelopmentSnapshotBuilds-AOOSnapshotfullsets
>> > And we have the "AOO 4.0 Full Regression Test" test plan created in
>> > Testlink: http://aootesting.adfinis-sygroup.org/index.php .
>> > And we have test guidance here:
>> >
>> https://cwiki.apache.org/confluence/display/OOOUSERS/AOO+Full+Regression+Test+Guidance
>> > and here:
>> >
>> https://cwiki.apache.org/confluence/display/OOOUSERS/AOO+Defect+Verification+Guidance
>> >
>> > So I suggest we start the AOO 4.0 Full Regression Test now !!!!!
>> >
>> > The test focus of this test circle include the following:
>> > (1) Basic Function Verification
>> > (2) MS Interoperability Test
>> > (3) 2nd phase of sidebar testing
>> > (4) Defect verification
>> >
>> > We will add more test cases (e.g. the Upgrade Check) in later circle.
>> >
>> > We already got some volunteers replied in mail list. Thanks very much!
>> > Will assign test cases in Testlink and send mail to you one by one.
>> > And looking forward more people to join and help the testing!
>> >
>> > - Shenfeng (Simon)
>> >
>> >
>> > ---------------------------------------------------------------------
>> > To unsubscribe, e-mail: dev-unsubscribe@openoffice.apache.org
>> > For additional commands, e-mail: dev-help@openoffice.apache.org
>> >
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: qa-unsubscribe@openoffice.apache.org
>> For additional commands, e-mail: qa-help@openoffice.apache.org
>>
>>
---------------------------------------------------------------------
To unsubscribe, e-mail: qa-unsubscribe@openoffice.apache.org
For additional commands, e-mail: qa-help@openoffice.apache.org
Re: Start AOO 4.0 Full Regression Test - missing keyboard navigation
and accelerators in test cases
Posted by janI <ja...@apache.org>.
On 7 May 2013 21:31, Rob Weir <ro...@apache.org> wrote:
> On Tue, May 7, 2013 at 11:46 AM, V Stuart Foote <VS...@utsa.edu>
> wrote:
> > Shenfeng, Yuzhen, Liuping, Rob
> >
> > Took a moment and browsed through the TestLink test cases for the Full
> Regression and earlier test plans against the trunk and sidebar branch.
> >
> > Unless I completely missed seeing it, one noticeable omission from the
> test cases in general is a lack keyboard based navigation and use of
> accelerators. Would expect that to be a significant component of
> functional testing.
> >
> > Only testing with mouse and menu based usage is unfortunately a way to
> miss the framework needed for accessibility and Assistive Technology
> support and impact all the supported OS builds. Not just the work on the
> IA2 branch.
> >
>
> So what would you recommend? Is this something you can help with, on
> the test definition or execution side?
>
> My naive approach would be, "Make sure you can perform all test cases
> with just the keyboard". Or is there a more targeted way to do this?
> A regression test (as opposed to a full functional test) tries to be
> smart about coverage and focuses on the areas that have changed in the
> most recent version. In other words, it puts the effort where the
> risk is greatest.
>
Sounds like a good approach to me "just with the keyboard".
I hope that for all automated testing we only use "full functional test".
With human resources involved regression testing is understandable.
>
> So, given what we know about AOO 3.4.1 keyboard support, and knowing
> what was changed in AOO 4.0, my guess would be the area to focus on
> would be the sidebar, yes?
>
> Translations are another area where I bet there is risk introduced.
> It is always possible for an accelerator collision to be introduced
> accidentally during translation. And the translations come in rather
> late in the process, after most testing. I wonder if we have any
> automated ways of detecting collisions?
>
if/when we have keyboard automated testing, it would be an easy job to
write a script that maps the keyboard and accellerators to any language.
That would allow us to run the same tests in any language.
Based on my experience from international software, menus and accellerators
are common translation problems.
>
> Any other areas that would make sense to focus on?
>
Installation with a keyboard only, or automated installation
rgds
Jan I.
>
> Regards,
>
> -Rob
>
>
> > Stuart
> > San Antonio, Texas
> >
> >
> > ________________________________________
> > From: Shenfeng Liu [liushenf@gmail.com]
> > Sent: Tuesday, May 07, 2013 8:28 AM
> > To: qa@openoffice.apache.org; dev@openoffice.apache.org
> > Subject: Start AOO 4.0 Full Regression Test
> >
> > Yu Zhen & Liu Ping,
> > Thanks very much for your help to prepare for the test guidance!
> >
> > Now we have the latest dev snapshot build here:
> >
> https://cwiki.apache.org/confluence/display/OOOUSERS/Development+Snapshot+Builds#DevelopmentSnapshotBuilds-AOOSnapshotfullsets
> > And we have the "AOO 4.0 Full Regression Test" test plan created in
> > Testlink: http://aootesting.adfinis-sygroup.org/index.php .
> > And we have test guidance here:
> >
> https://cwiki.apache.org/confluence/display/OOOUSERS/AOO+Full+Regression+Test+Guidance
> > and here:
> >
> https://cwiki.apache.org/confluence/display/OOOUSERS/AOO+Defect+Verification+Guidance
> >
> > So I suggest we start the AOO 4.0 Full Regression Test now !!!!!
> >
> > The test focus of this test circle include the following:
> > (1) Basic Function Verification
> > (2) MS Interoperability Test
> > (3) 2nd phase of sidebar testing
> > (4) Defect verification
> >
> > We will add more test cases (e.g. the Upgrade Check) in later circle.
> >
> > We already got some volunteers replied in mail list. Thanks very much!
> > Will assign test cases in Testlink and send mail to you one by one.
> > And looking forward more people to join and help the testing!
> >
> > - Shenfeng (Simon)
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@openoffice.apache.org
> > For additional commands, e-mail: dev-help@openoffice.apache.org
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: qa-unsubscribe@openoffice.apache.org
> For additional commands, e-mail: qa-help@openoffice.apache.org
>
>
Re: Start AOO 4.0 Full Regression Test - missing keyboard navigation
and accelerators in test cases
Posted by Yuzhen Fan <fa...@gmail.com>.
I think basically we can use the approach "Make sure you can perform all
test cases with just the keyboard". To be more targeted, we can
1. Add a general test case in TestLink, saying "Make sure you can perform
all test cases with just the keyboard" and refer the expected
result(navigation and accelerators) to AOO product Help
2. Ask QA to run this test case but record the execution result in related
function test cases
If you all agree above approach, I can add the general test case and inform
QA about that when assign test cases to them.
Considering the track of the execution result, the perfect way could be add
this checkpoint "just with the keyboard" in every test case but we cannot
afford in this stage.
Regards,
Yu Zhen
On Wed, May 8, 2013 at 3:31 AM, Rob Weir <ro...@apache.org> wrote:
> On Tue, May 7, 2013 at 11:46 AM, V Stuart Foote <VS...@utsa.edu>
> wrote:
> > Shenfeng, Yuzhen, Liuping, Rob
> >
> > Took a moment and browsed through the TestLink test cases for the Full
> Regression and earlier test plans against the trunk and sidebar branch.
> >
> > Unless I completely missed seeing it, one noticeable omission from the
> test cases in general is a lack keyboard based navigation and use of
> accelerators. Would expect that to be a significant component of
> functional testing.
> >
> > Only testing with mouse and menu based usage is unfortunately a way to
> miss the framework needed for accessibility and Assistive Technology
> support and impact all the supported OS builds. Not just the work on the
> IA2 branch.
> >
>
> So what would you recommend? Is this something you can help with, on
> the test definition or execution side?
>
> My naive approach would be, "Make sure you can perform all test cases
> with just the keyboard". Or is there a more targeted way to do this?
> A regression test (as opposed to a full functional test) tries to be
> smart about coverage and focuses on the areas that have changed in the
> most recent version. In other words, it puts the effort where the
> risk is greatest.
>
> So, given what we know about AOO 3.4.1 keyboard support, and knowing
> what was changed in AOO 4.0, my guess would be the area to focus on
> would be the sidebar, yes?
>
> Translations are another area where I bet there is risk introduced.
> It is always possible for an accelerator collision to be introduced
> accidentally during translation. And the translations come in rather
> late in the process, after most testing. I wonder if we have any
> automated ways of detecting collisions?
>
> Any other areas that would make sense to focus on?
>
> Regards,
>
> -Rob
>
>
> > Stuart
> > San Antonio, Texas
> >
> >
> > ________________________________________
> > From: Shenfeng Liu [liushenf@gmail.com]
> > Sent: Tuesday, May 07, 2013 8:28 AM
> > To: qa@openoffice.apache.org; dev@openoffice.apache.org
> > Subject: Start AOO 4.0 Full Regression Test
> >
> > Yu Zhen & Liu Ping,
> > Thanks very much for your help to prepare for the test guidance!
> >
> > Now we have the latest dev snapshot build here:
> >
> https://cwiki.apache.org/confluence/display/OOOUSERS/Development+Snapshot+Builds#DevelopmentSnapshotBuilds-AOOSnapshotfullsets
> > And we have the "AOO 4.0 Full Regression Test" test plan created in
> > Testlink: http://aootesting.adfinis-sygroup.org/index.php .
> > And we have test guidance here:
> >
> https://cwiki.apache.org/confluence/display/OOOUSERS/AOO+Full+Regression+Test+Guidance
> > and here:
> >
> https://cwiki.apache.org/confluence/display/OOOUSERS/AOO+Defect+Verification+Guidance
> >
> > So I suggest we start the AOO 4.0 Full Regression Test now !!!!!
> >
> > The test focus of this test circle include the following:
> > (1) Basic Function Verification
> > (2) MS Interoperability Test
> > (3) 2nd phase of sidebar testing
> > (4) Defect verification
> >
> > We will add more test cases (e.g. the Upgrade Check) in later circle.
> >
> > We already got some volunteers replied in mail list. Thanks very much!
> > Will assign test cases in Testlink and send mail to you one by one.
> > And looking forward more people to join and help the testing!
> >
> > - Shenfeng (Simon)
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@openoffice.apache.org
> > For additional commands, e-mail: dev-help@openoffice.apache.org
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@openoffice.apache.org
> For additional commands, e-mail: dev-help@openoffice.apache.org
>
>
Re: Start AOO 4.0 Full Regression Test - missing keyboard navigation
and accelerators in test cases
Posted by Rob Weir <ro...@apache.org>.
On Tue, May 7, 2013 at 11:46 AM, V Stuart Foote <VS...@utsa.edu> wrote:
> Shenfeng, Yuzhen, Liuping, Rob
>
> Took a moment and browsed through the TestLink test cases for the Full Regression and earlier test plans against the trunk and sidebar branch.
>
> Unless I completely missed seeing it, one noticeable omission from the test cases in general is a lack keyboard based navigation and use of accelerators. Would expect that to be a significant component of functional testing.
>
> Only testing with mouse and menu based usage is unfortunately a way to miss the framework needed for accessibility and Assistive Technology support and impact all the supported OS builds. Not just the work on the IA2 branch.
>
So what would you recommend? Is this something you can help with, on
the test definition or execution side?
My naive approach would be, "Make sure you can perform all test cases
with just the keyboard". Or is there a more targeted way to do this?
A regression test (as opposed to a full functional test) tries to be
smart about coverage and focuses on the areas that have changed in the
most recent version. In other words, it puts the effort where the
risk is greatest.
So, given what we know about AOO 3.4.1 keyboard support, and knowing
what was changed in AOO 4.0, my guess would be the area to focus on
would be the sidebar, yes?
Translations are another area where I bet there is risk introduced.
It is always possible for an accelerator collision to be introduced
accidentally during translation. And the translations come in rather
late in the process, after most testing. I wonder if we have any
automated ways of detecting collisions?
Any other areas that would make sense to focus on?
Regards,
-Rob
> Stuart
> San Antonio, Texas
>
>
> ________________________________________
> From: Shenfeng Liu [liushenf@gmail.com]
> Sent: Tuesday, May 07, 2013 8:28 AM
> To: qa@openoffice.apache.org; dev@openoffice.apache.org
> Subject: Start AOO 4.0 Full Regression Test
>
> Yu Zhen & Liu Ping,
> Thanks very much for your help to prepare for the test guidance!
>
> Now we have the latest dev snapshot build here:
> https://cwiki.apache.org/confluence/display/OOOUSERS/Development+Snapshot+Builds#DevelopmentSnapshotBuilds-AOOSnapshotfullsets
> And we have the "AOO 4.0 Full Regression Test" test plan created in
> Testlink: http://aootesting.adfinis-sygroup.org/index.php .
> And we have test guidance here:
> https://cwiki.apache.org/confluence/display/OOOUSERS/AOO+Full+Regression+Test+Guidance
> and here:
> https://cwiki.apache.org/confluence/display/OOOUSERS/AOO+Defect+Verification+Guidance
>
> So I suggest we start the AOO 4.0 Full Regression Test now !!!!!
>
> The test focus of this test circle include the following:
> (1) Basic Function Verification
> (2) MS Interoperability Test
> (3) 2nd phase of sidebar testing
> (4) Defect verification
>
> We will add more test cases (e.g. the Upgrade Check) in later circle.
>
> We already got some volunteers replied in mail list. Thanks very much!
> Will assign test cases in Testlink and send mail to you one by one.
> And looking forward more people to join and help the testing!
>
> - Shenfeng (Simon)
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@openoffice.apache.org
> For additional commands, e-mail: dev-help@openoffice.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: qa-unsubscribe@openoffice.apache.org
For additional commands, e-mail: qa-help@openoffice.apache.org
Re: Start AOO 4.0 Full Regression Test - missing keyboard navigation
and accelerators in test cases
Posted by Rob Weir <ro...@apache.org>.
On Tue, May 7, 2013 at 11:46 AM, V Stuart Foote <VS...@utsa.edu> wrote:
> Shenfeng, Yuzhen, Liuping, Rob
>
> Took a moment and browsed through the TestLink test cases for the Full Regression and earlier test plans against the trunk and sidebar branch.
>
> Unless I completely missed seeing it, one noticeable omission from the test cases in general is a lack keyboard based navigation and use of accelerators. Would expect that to be a significant component of functional testing.
>
> Only testing with mouse and menu based usage is unfortunately a way to miss the framework needed for accessibility and Assistive Technology support and impact all the supported OS builds. Not just the work on the IA2 branch.
>
So what would you recommend? Is this something you can help with, on
the test definition or execution side?
My naive approach would be, "Make sure you can perform all test cases
with just the keyboard". Or is there a more targeted way to do this?
A regression test (as opposed to a full functional test) tries to be
smart about coverage and focuses on the areas that have changed in the
most recent version. In other words, it puts the effort where the
risk is greatest.
So, given what we know about AOO 3.4.1 keyboard support, and knowing
what was changed in AOO 4.0, my guess would be the area to focus on
would be the sidebar, yes?
Translations are another area where I bet there is risk introduced.
It is always possible for an accelerator collision to be introduced
accidentally during translation. And the translations come in rather
late in the process, after most testing. I wonder if we have any
automated ways of detecting collisions?
Any other areas that would make sense to focus on?
Regards,
-Rob
> Stuart
> San Antonio, Texas
>
>
> ________________________________________
> From: Shenfeng Liu [liushenf@gmail.com]
> Sent: Tuesday, May 07, 2013 8:28 AM
> To: qa@openoffice.apache.org; dev@openoffice.apache.org
> Subject: Start AOO 4.0 Full Regression Test
>
> Yu Zhen & Liu Ping,
> Thanks very much for your help to prepare for the test guidance!
>
> Now we have the latest dev snapshot build here:
> https://cwiki.apache.org/confluence/display/OOOUSERS/Development+Snapshot+Builds#DevelopmentSnapshotBuilds-AOOSnapshotfullsets
> And we have the "AOO 4.0 Full Regression Test" test plan created in
> Testlink: http://aootesting.adfinis-sygroup.org/index.php .
> And we have test guidance here:
> https://cwiki.apache.org/confluence/display/OOOUSERS/AOO+Full+Regression+Test+Guidance
> and here:
> https://cwiki.apache.org/confluence/display/OOOUSERS/AOO+Defect+Verification+Guidance
>
> So I suggest we start the AOO 4.0 Full Regression Test now !!!!!
>
> The test focus of this test circle include the following:
> (1) Basic Function Verification
> (2) MS Interoperability Test
> (3) 2nd phase of sidebar testing
> (4) Defect verification
>
> We will add more test cases (e.g. the Upgrade Check) in later circle.
>
> We already got some volunteers replied in mail list. Thanks very much!
> Will assign test cases in Testlink and send mail to you one by one.
> And looking forward more people to join and help the testing!
>
> - Shenfeng (Simon)
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@openoffice.apache.org
> For additional commands, e-mail: dev-help@openoffice.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@openoffice.apache.org
For additional commands, e-mail: dev-help@openoffice.apache.org