Regressions and testing

[Damjan Jovanovic <da...@apache.org>]

Hi

It seems to me that regressions that we've had with releases recently could
have been prevented with judicious testing.

We do already have considerable tests in our tree, but they are never run.

I've committed a patch that fixes use of junit with hamcrest in both gbuild
and dmake, and should allow us to run "subsequent tests" after a build.

Damjan

Re: Regressions and testing

[Damjan Jovanovic <da...@apache.org>]

We already have a bvt test that writes a spreadsheet with a chart:
https://wiki.openoffice.org/wiki/QA/BVT


On Friday, November 10, 2017, Damjan Jovanovic <da...@apache.org> wrote:

> In a module:
> OOO_SUBSEQUENT_TESTS=1 build
> Results go to main/solver/420/.../workdir/JunitTests for gbuild at least.
>
> There's a Perl script main/solenv/bin/subsequent_tests or something like
> that which runs the tests in all modules. It's currently broken at the
> first line in the file, but even when that's fixed, it complains about the
> gbuild module stack being corrupt.
>
> Subsequent tests have limits and may not help much though. The richest and
> most thorough tests that can check content round tripped through saving and
> reloading, with screenshots, are the bvt and fvt tests in the tests/
> directory (same level as main/). Please search for my previous emails where
> I explained more.
>
> On Friday, November 10, 2017, Patricia Shanahan <pats@acm.org
> <javascript:_e(%7B%7D,'cvml','pats@acm.org');>> wrote:
>
>> The more automated testing we can do, the better. I have a possible patch
>> for one of the regressions. What do I need to do to run the "subsequent
>> tests"?
>>
>> On 11/9/2017 9:46 PM, Damjan Jovanovic wrote:
>>
>>> Hi
>>>
>>> It seems to me that regressions that we've had with releases recently
>>> could
>>> have been prevented with judicious testing.
>>>
>>> We do already have considerable tests in our tree, but they are never
>>> run.
>>>
>>> I've committed a patch that fixes use of junit with hamcrest in both
>>> gbuild
>>> and dmake, and should allow us to run "subsequent tests" after a build.
>>>
>>> Damjan
>>>
>>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@openoffice.apache.org
>> For additional commands, e-mail: dev-help@openoffice.apache.org
>>
>>

Re: Regressions and testing

[Damjan Jovanovic <da...@apache.org>]

In a module:
OOO_SUBSEQUENT_TESTS=1 build
Results go to main/solver/420/.../workdir/JunitTests for gbuild at least.

There's a Perl script main/solenv/bin/subsequent_tests or something like
that which runs the tests in all modules. It's currently broken at the
first line in the file, but even when that's fixed, it complains about the
gbuild module stack being corrupt.

Subsequent tests have limits and may not help much though. The richest and
most thorough tests that can check content round tripped through saving and
reloading, with screenshots, are the bvt and fvt tests in the tests/
directory (same level as main/). Please search for my previous emails where
I explained more.

On Friday, November 10, 2017, Patricia Shanahan <pa...@acm.org> wrote:

> The more automated testing we can do, the better. I have a possible patch
> for one of the regressions. What do I need to do to run the "subsequent
> tests"?
>
> On 11/9/2017 9:46 PM, Damjan Jovanovic wrote:
>
>> Hi
>>
>> It seems to me that regressions that we've had with releases recently
>> could
>> have been prevented with judicious testing.
>>
>> We do already have considerable tests in our tree, but they are never run.
>>
>> I've committed a patch that fixes use of junit with hamcrest in both
>> gbuild
>> and dmake, and should allow us to run "subsequent tests" after a build.
>>
>> Damjan
>>
>>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@openoffice.apache.org
> For additional commands, e-mail: dev-help@openoffice.apache.org
>
>

Re: Regressions and testing

[Patricia Shanahan <pa...@acm.org>]

The more automated testing we can do, the better. I have a possible 
patch for one of the regressions. What do I need to do to run the 
"subsequent tests"?

On 11/9/2017 9:46 PM, Damjan Jovanovic wrote:
> Hi
> 
> It seems to me that regressions that we've had with releases recently could
> have been prevented with judicious testing.
> 
> We do already have considerable tests in our tree, but they are never run.
> 
> I've committed a patch that fixes use of junit with hamcrest in both gbuild
> and dmake, and should allow us to run "subsequent tests" after a build.
> 
> Damjan
> 

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@openoffice.apache.org
For additional commands, e-mail: dev-help@openoffice.apache.org