[jira] [Created] (TINKERPOP-2013) Process tests that are auto-ignored stink

["stephen mallette (JIRA)" <ji...@apache.org>]

stephen mallette created TINKERPOP-2013:
-------------------------------------------

             Summary: Process tests that are auto-ignored stink
                 Key: TINKERPOP-2013
                 URL: https://issues.apache.org/jira/browse/TINKERPOP-2013
             Project: TinkerPop
          Issue Type: Improvement
          Components: test-suite
    Affects Versions: 3.2.9
            Reporter: stephen mallette
            Assignee: stephen mallette


I think I've had it with {{GremlinProcessRunner}} and any other related infrastructure that auto-ignores batches of tests unknowingly. The typical scenario is that a new test gets added, we build the project and everything works without failures. Then something starts misbehaving and we wonder why. After a lot of cycles and looking in the wrong places we realize that the tests that should have caught the problem are simply being ignored. 

There was a reason for making things work this way as we had assumed that certain tests failing in a certain way was expected behavior, but that is not always the case. We’ve learned that assumption doesn’t always apply and it would be better to be more explicit in dealing with those failures.

Not sure how this is going to be resolved, but I think that something needs to change. Perhaps we need to identify all the auto-ignored tests and do any of the following:

1. Make them work
2. Explicitly ignore them
3. Re-organize tests so that it's not assumed that they just are part of the process suite
4. Something else??



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)