[jira] [Commented] (TINKERPOP-2013) Process tests that are auto-ignored stink

["stephen mallette (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/TINKERPOP-2013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16718993#comment-16718993 ] 

stephen mallette commented on TINKERPOP-2013:
---------------------------------------------

just got bit again by this - so much wasted time. [~spmallette] do something about this!!!

> Process tests that are auto-ignored stink
> -----------------------------------------
>
>                 Key: TINKERPOP-2013
>                 URL: https://issues.apache.org/jira/browse/TINKERPOP-2013
>             Project: TinkerPop
>          Issue Type: Improvement
>          Components: test-suite
>    Affects Versions: 3.2.9
>            Reporter: stephen mallette
>            Assignee: stephen mallette
>            Priority: Major
>
> I think I've had it with {{GremlinProcessRunner}} and any other related infrastructure that auto-ignores batches of tests unknowingly. The typical scenario is that a new test gets added, we build the project and everything works without failures. Then something starts misbehaving and we wonder why. After a lot of cycles and looking in the wrong places we realize that the tests that should have caught the problem are simply being ignored. 
> There was a reason for making things work this way as we had assumed that certain tests failing in a certain way was expected behavior, but that is not always the case. We’ve learned that assumption doesn’t always apply and it would be better to be more explicit in dealing with those failures.
> Not sure how this is going to be resolved, but I think that something needs to change. Perhaps we need to identify all the auto-ignored tests and do any of the following:
> 1. Make them work
> 2. Explicitly ignore them
> 3. Re-organize tests so that it's not assumed that they just are part of the process suite
> 4. Something else??
> May also need to extend this into all the ignored GLV tests that aren't supported for various reasons. Perhaps the assertion language needs more expressiveness.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)