Plugins for testing

[Alan Carroll <so...@yahoo-inc.com.INVALID>]

We (by which I mean Petar Penkovv) have been doing some work with plugins that are built for testing, rather than general use or for being coding examples. Ultimately we want to integrate them into a testing framework to be able to more fully test ATS. This means putting the code in the source tree but there has been some dissent about where to put them. I am strongly of the opinion these should be separated from the other plugins to avoid confusing people. The current suggestion is to put them in "tests/plugins" which I think is reasonable.

Re: Plugins for testing

[Alan Carroll <so...@yahoo-inc.com.INVALID>]

Haha, due to IRC discussions we're now planning on "ci/plugins" because that's where the testing code is now.
 

    On Friday, August 19, 2016 12:51 PM, Alan Carroll <so...@yahoo-inc.com.INVALID> wrote:
 

 We (by which I mean Petar Penkovv) have been doing some work with plugins that are built for testing, rather than general use or for being coding examples. Ultimately we want to integrate them into a testing framework to be able to more fully test ATS. This means putting the code in the source tree but there has been some dissent about where to put them. I am strongly of the opinion these should be separated from the other plugins to avoid confusing people. The current suggestion is to put them in "tests/plugins" which I think is reasonable.



   

Re: Plugins for testing

[Leif Hedstrom <zw...@apache.org>]

> On Aug 19, 2016, at 2:09 PM, Alan Carroll <so...@yahoo-inc.com.INVALID> wrote:
> 
> But it's not intended for general use, ever. It's not experimental in the sense that it's being previewed for promotion. I would like to actively discourage its use by other people until they're developing tests.



I don’t care strongly where they should be located, but I like the idea of keeping it out of normal builds / installs. That’s why I suggested examples, but e.g.  plugins/test sounds good too. Putting it into “ci” seems a bit of a stretch though, since that’s generally for “continuous integration” (i.e. automated tests and builds etc.).

Cheers,

— leif

Re: Plugins for testing

[Bryan Call <bc...@apache.org>]

Yes, I sounds good for having continuous integration plugins being under tests/plugins.  I don’t think the header frequency counter plugins would fall into this category. I would rather see that go under plugins/experimental.

-Bryan




> On Aug 19, 2016, at 11:33 AM, Alan Carroll <so...@yahoo-inc.com.INVALID> wrote:
> 
> Only new plugins, such as the START/CLOSE checker, the WKS counter, and some other stress testing plugins, would be put in this directory. No current plugin would be moved. But as time goes on I want to do a lot more plugin level testing. 
> 
> 

Re: Plugins for testing

[Alan Carroll <so...@yahoo-inc.com.INVALID>]

Only new plugins, such as the START/CLOSE checker, the WKS counter, and some other stress testing plugins, would be put in this directory. No current plugin would be moved. But as time goes on I want to do a lot more plugin level testing. 


   

Re: Plugins for testing

[Bryan Call <bc...@apache.org>]

Do you have a full list of plugins that would be moved into the proposed test directory?  If you are talking about moving existing plugins into another directory too, then it should have been on the original proposal.

-Bryan


> On Aug 19, 2016, at 11:25 AM, Alan Carroll <so...@yahoo-inc.com.INVALID> wrote:
> 
> What about a plugin that just counts SSN_START, SSN_CLOSE, TXN_START, TXN_CLOSE, and checks these are correctly ordered and match up? When would that be run except in a test environment?
> 
> 
>    On Friday, August 19, 2016 1:22 PM, Bryan Call <bc...@apache.org> wrote:
> 
> 
> It would be helpful for someone that is interested in seeing what headers are being transmitted.  I can see us at Yahoo periodically running this plugin on a couple servers to get an understanding of what headers we may be leaking to the outside world.
> 
> The distinction between test and experimental and having them in completely different directory hierarchies is more confusing and I can see end users asking questions about this.
> -Bryan
> 
> 
> 
> 
> On Aug 19, 2016, at 11:09 AM, Alan Carroll <so...@yahoo-inc.com.INVALID> wrote:
> But it's not intended for general use, ever. It's not experimental in the sense that it's being previewed for promotion. I would like to actively discourage its use by other people until they're developing tests.
> 
> 
> 
> 
> 
> 
> 

Re: Plugins for testing

[Alan Carroll <so...@yahoo-inc.com.INVALID>]

What about a plugin that just counts SSN_START, SSN_CLOSE, TXN_START, TXN_CLOSE, and checks these are correctly ordered and match up? When would that be run except in a test environment?
 

    On Friday, August 19, 2016 1:22 PM, Bryan Call <bc...@apache.org> wrote:
 

 It would be helpful for someone that is interested in seeing what headers are being transmitted.  I can see us at Yahoo periodically running this plugin on a couple servers to get an understanding of what headers we may be leaking to the outside world.

The distinction between test and experimental and having them in completely different directory hierarchies is more confusing and I can see end users asking questions about this.
-Bryan




On Aug 19, 2016, at 11:09 AM, Alan Carroll <so...@yahoo-inc.com.INVALID> wrote:
But it's not intended for general use, ever. It's not experimental in the sense that it's being previewed for promotion. I would like to actively discourage its use by other people until they're developing tests.







  

Re: Plugins for testing

[Bryan Call <bc...@apache.org>]

It would be helpful for someone that is interested in seeing what headers are being transmitted.  I can see us at Yahoo periodically running this plugin on a couple servers to get an understanding of what headers we may be leaking to the outside world.

The distinction between test and experimental and having them in completely different directory hierarchies is more confusing and I can see end users asking questions about this.

-Bryan




> On Aug 19, 2016, at 11:09 AM, Alan Carroll <so...@yahoo-inc.com.INVALID> wrote:
> 
> But it's not intended for general use, ever. It's not experimental in the sense that it's being previewed for promotion. I would like to actively discourage its use by other people until they're developing tests.
> 
> 
> 

Re: Plugins for testing

[Alan Carroll <so...@yahoo-inc.com.INVALID>]

But it's not intended for general use, ever. It's not experimental in the sense that it's being previewed for promotion. I would like to actively discourage its use by other people until they're developing tests.



   

Re: Plugins for testing

[Bryan Call <bc...@apache.org>]

I think it would be better to place it in the experimental directory since it is an experimental plugin (new untested plugin).  This plugin can be used by other people if they would like to use it and we will be using it a few times over the years.  I am also against creating more top level directories for plugins since we already have a plugins directory.

-Bryan




> On Aug 19, 2016, at 10:51 AM, Alan Carroll <so...@yahoo-inc.com.INVALID> wrote:
> 
> We (by which I mean Petar Penkovv) have been doing some work with plugins that are built for testing, rather than general use or for being coding examples. Ultimately we want to integrate them into a testing framework to be able to more fully test ATS. This means putting the code in the source tree but there has been some dissent about where to put them. I am strongly of the opinion these should be separated from the other plugins to avoid confusing people. The current suggestion is to put them in "tests/plugins" which I think is reasonable.
>