Unit testing needs to work against existing code

[sebb <se...@gmail.com>]

Good to see a start on setting up some unit tests.

However at present it does not work at all on the current release, as
it assumes changes to the API (in particular Archiver#compute_updates
has an extra parameter)

I think it's vital that the appropriate tests (i.e. non-dkim) should
also be run against the current ponymail without having to make
changes to the PonyMail code.

This means adding an option to change the compute_updates call, and
some way of ignoring tests for the dkim generator (or any other new
ones).

We should also be able to run the tests against earlier releases of PonyMail.
AIUI both 0.10 and 0.11 were used by public archives so at least MID
generation should be tested.

Sebb.

Re: Unit testing needs to work against existing code

[sebb <se...@gmail.com>]

Thanks, but the test code does not work with 0.11 or 0.10 because they
don't support the parse_html keyword argument which was recently added
to the current code.

Rather than change the existing code to work with the test suite, it
should be the other way around to allow for maximum coverage.

On Sat, 15 Aug 2020 at 07:27, Daniel Gruno <hu...@apache.org> wrote:
>
> On 15/08/2020 08.05, Daniel Gruno wrote:
> > On 14/08/2020 23.02, sebb wrote:
> >> Good to see a start on setting up some unit tests.
> >>
> >> However at present it does not work at all on the current release, as
> >> it assumes changes to the API (in particular Archiver#compute_updates
> >> has an extra parameter)
> >
> > I've made it use the inspect module to figure out which parameters are
> > expected and thus how to call compute_updates, so it should work with
> > both foal and 0.12/0.11 now. It is expected that this will fail on 76
> > tests for 0.12 and pass completely on foal due to a known bug in the
> > formatFlowed call in 0.12/master and earlier.
>
> This should drop to 0 with PR #518.
>
> >
> >>
> >> I think it's vital that the appropriate tests (i.e. non-dkim) should
> >> also be run against the current ponymail without having to make
> >> changes to the PonyMail code.
> >
> > I've made it skip tests when the appropriate generator could not be
> > found, hope this addresses the issue in the best possible way.
> >
> >>
> >> This means adding an option to change the compute_updates call, and
> >> some way of ignoring tests for the dkim generator (or any other new
> >> ones).
> >>
> >> We should also be able to run the tests against earlier releases of
> >> PonyMail.
> >> AIUI both 0.10 and 0.11 were used by public archives so at least MID
> >> generation should be tested.
> >>
> >> Sebb.
> >>
> >
>

Re: Unit testing needs to work against existing code

[Daniel Gruno <hu...@apache.org>]

On 15/08/2020 08.05, Daniel Gruno wrote:
> On 14/08/2020 23.02, sebb wrote:
>> Good to see a start on setting up some unit tests.
>>
>> However at present it does not work at all on the current release, as
>> it assumes changes to the API (in particular Archiver#compute_updates
>> has an extra parameter)
> 
> I've made it use the inspect module to figure out which parameters are 
> expected and thus how to call compute_updates, so it should work with 
> both foal and 0.12/0.11 now. It is expected that this will fail on 76 
> tests for 0.12 and pass completely on foal due to a known bug in the 
> formatFlowed call in 0.12/master and earlier.

This should drop to 0 with PR #518.

> 
>>
>> I think it's vital that the appropriate tests (i.e. non-dkim) should
>> also be run against the current ponymail without having to make
>> changes to the PonyMail code.
> 
> I've made it skip tests when the appropriate generator could not be 
> found, hope this addresses the issue in the best possible way.
> 
>>
>> This means adding an option to change the compute_updates call, and
>> some way of ignoring tests for the dkim generator (or any other new
>> ones).
>>
>> We should also be able to run the tests against earlier releases of 
>> PonyMail.
>> AIUI both 0.10 and 0.11 were used by public archives so at least MID
>> generation should be tested.
>>
>> Sebb.
>>
> 

Re: Unit testing needs to work against existing code

[Daniel Gruno <hu...@apache.org>]

On 14/08/2020 23.02, sebb wrote:
> Good to see a start on setting up some unit tests.
> 
> However at present it does not work at all on the current release, as
> it assumes changes to the API (in particular Archiver#compute_updates
> has an extra parameter)

I've made it use the inspect module to figure out which parameters are 
expected and thus how to call compute_updates, so it should work with 
both foal and 0.12/0.11 now. It is expected that this will fail on 76 
tests for 0.12 and pass completely on foal due to a known bug in the 
formatFlowed call in 0.12/master and earlier.

> 
> I think it's vital that the appropriate tests (i.e. non-dkim) should
> also be run against the current ponymail without having to make
> changes to the PonyMail code.

I've made it skip tests when the appropriate generator could not be 
found, hope this addresses the issue in the best possible way.

> 
> This means adding an option to change the compute_updates call, and
> some way of ignoring tests for the dkim generator (or any other new
> ones).
> 
> We should also be able to run the tests against earlier releases of PonyMail.
> AIUI both 0.10 and 0.11 were used by public archives so at least MID
> generation should be tested.
> 
> Sebb.
>