Regressions vs tsqa

[Leif Hedstrom <zw...@apache.org>]

Hi,

First, thanks to James, Thomas, Josh and everyone for taking these initiatives with tsqa. It's great. I'd like to make sure we don't forget the existing regression infrastructure.
Many times, it's IMO preferable and easier to add new tests to the regression tests, over tsqa.

I'd like us to do this in addition to all the new tsqa greatness:

1. Improve upon existing regression tests
2. Add new regression tests as appropriate 
3. Improve the regressions framework to make it easier to add new tests.

I know  a lot of work has been done on this over the years, which is great. I'm just writing this as a reminder that we have this in our testing arsenal. It's easy to run and it's fast, and both the CI and the individual contributors can run it quickly on every commit.

Cheers,

-- Leif 

Re: Regressions vs tsqa

[James Peach <jp...@apache.org>]

> On Jan 21, 2015, at 1:13 PM, Leif Hedstrom <zw...@apache.org> wrote:
> 
> 
>> On Jan 20, 2015, at 5:09 PM, Leif Hedstrom <zw...@apache.org> wrote:
>> 
>> Hi,
>> 
>> First, thanks to James, Thomas, Josh and everyone for taking these initiatives with tsqa. It's great. I'd like to make sure we don't forget the existing regression infrastructure.
>> Many times, it's IMO preferable and easier to add new tests to the regression tests, over tsqa.
>> 
>> I'd like us to do this in addition to all the new tsqa greatness:
>> 
>> 1. Improve upon existing regression tests
>> 2. Add new regression tests as appropriate 
>> 3. Improve the regressions framework to make it easier to add new tests.
> 
> 
> Oh, and I forgot (thanks Gancho for reminding me), we also have quite a number of regression tests running as part of the “make test” build target. This builds standalone applications the exercises certain libraries etc. Adding / improving on those is also a good thing IMO (they are also fast and easy to run before every commit).

The ./ci/regression script does that. Just run it before pushing your commits

J

Re: Regressions vs tsqa

[Leif Hedstrom <zw...@apache.org>]

> On Jan 20, 2015, at 5:09 PM, Leif Hedstrom <zw...@apache.org> wrote:
> 
> Hi,
> 
> First, thanks to James, Thomas, Josh and everyone for taking these initiatives with tsqa. It's great. I'd like to make sure we don't forget the existing regression infrastructure.
> Many times, it's IMO preferable and easier to add new tests to the regression tests, over tsqa.
> 
> I'd like us to do this in addition to all the new tsqa greatness:
> 
> 1. Improve upon existing regression tests
> 2. Add new regression tests as appropriate 
> 3. Improve the regressions framework to make it easier to add new tests.


Oh, and I forgot (thanks Gancho for reminding me), we also have quite a number of regression tests running as part of the “make test” build target. This builds standalone applications the exercises certain libraries etc. Adding / improving on those is also a good thing IMO (they are also fast and easy to run before every commit).

— Leif