[jira] [Created] (LUCENE-10257) automation to help prevent slow tests from being added to the codebase

["Robert Muir (Jira)" <ji...@apache.org>]

Robert Muir created LUCENE-10257:
------------------------------------

             Summary: automation to help prevent slow tests from being added to the codebase
                 Key: LUCENE-10257
                 URL: https://issues.apache.org/jira/browse/LUCENE-10257
             Project: Lucene - Core
          Issue Type: Task
            Reporter: Robert Muir


We have thousands and thousands of unit tests.

The problem: when new PRs go in, the new test could take 30 seconds, but all the checks will pass, so it sneaks in. So nobody bothers to look at how long the new test takes.

After a year or so, test runtime tends to double, until the work is done to then fix all these slow long-tail tests to become reasonable. It becomes exhausting.

Can we improve the PR process to prevent these tests from making it in?
I don't know what it would be: more visibility into how long the unit test run took from the github action or something like that? At least maybe more visibility into the "top 10 slowest tests" that the build prints at the end?

I'm not trying to suggest hard limits, as that seems really annoying/frustrating. Just something better.





--
This message was sent by Atlassian Jira
(v8.20.1#820001)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org