[GitHub] [lucenenet] NightOwl888 opened a new issue, #739: [Slow] attribute runs the code in BeforeClass() and AfterClass() even when the `tests:slow` property is set to `false`

[GitBox <gi...@apache.org>]

NightOwl888 opened a new issue, #739:
URL: https://github.com/apache/lucenenet/issues/739

   When the `SlowAttribute` in `Lucene.Net.TestFramework` is placed on a `LuceneTestCase`-derived class, it runs the code in the `BeforeClass()` and `AfterClass()` methods  (which are decorated with `[OneTimeSetUp]` and `[OneTimeTearDown]`) even when the system property `tests:slow` is set to `false`. This causes these disabled tests to run code, taking up extra testing time. Since the purpose of these settings is to optimize the test run, this undesired behavior is a bug.
   
   I suspect (but haven't confirmed) that the following other attributes also have a similar issue, since they all follow the same basic design.
   
   - `AwaitsFixAttribute`
   - `NightlyAttribute`
   - `WeeklyAttribute`
   
   Ideally, these would completely suppress any actions from occurring when applied at the class level when their associated [system property](https://github.com/apache/lucenenet/issues/307) is disabled. System properties are statically loaded by the test framework in [LuceneTestCase](https://github.com/apache/lucenenet/blob/c076e40b14d4c20e6fdfee4e28d0b3332cf6d0ce/.github/workflows/Lucene-Net-Tests-AllProjects.yml#L130).
   
   > Note: There is some better info [here](https://lucenenet.apache.org/docs/4.8.0-beta00016/api/test-framework/Lucene.Net.Util.LuceneTestCase.html#randomized-execution-and-test-facilities) on how to specify system properties and command line example [here](https://github.com/apache/lucenenet/blob/c076e40b14d4c20e6fdfee4e28d0b3332cf6d0ce/.github/workflows/Lucene-Net-Tests-AllProjects.yml#L130). No general docs for this yet.
   
   These attributes use NUnit's extensibility through attribute interfaces `IApplyToTest`, `IApplyToContext`, and `IWrapTestMethod` to function. NUnit scans for attributes that implement these interfaces in order to execute them.
   
   Ideally, we find a way to suppress the `[OneTimeSetUp]` and `[OneTimeTearDown]` calls somehow. But these execution calls are owned by NUnit.
   
   Failing that we may be able implement our own attributes with the same names that implement the same interfaces. When these attributes are placed as nested classes of `LuceneTestCase`, they effectively "override" the attributes in the `NUnit.Framework` namespace for `LuceneTestCase` or any class that inherits it.
   
   > Note: We use this approach already for our custom [`[TestFixtureAttribute]`](https://github.com/apache/lucenenet/blob/c076e40b14d4c20e6fdfee4e28d0b3332cf6d0ce/src/Lucene.Net.TestFramework/Support/Util/LuceneTestCase.TestFixtureAttribute.cs#L41). There may be some way to use that custom attribute to fix this where we have direct control over the code. It looks like we are already intercepting these calls [here](https://github.com/apache/lucenenet/blob/c076e40b14d4c20e6fdfee4e28d0b3332cf6d0ce/src/Lucene.Net.TestFramework/Support/Util/LuceneTestCase.SetUpFixture.cs#L70-L92), but I am not sure whether that gives us ownership over all of them.
   
   There is some [custom attribute documentation](https://docs.nunit.org/articles/nunit/extending-nunit/Custom-Attributes.html), but it doesn't actually include `IWrapTestMethod` or [other interfaces that are available in the source code](https://github.com/nunit/nunit/tree/b16d7540ea424d147847ce7af2a576e98a411209/src/NUnitFramework/framework/Interfaces).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@lucenenet.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org