[jira] [Comment Edited] (LUCENE-5755) Explore alternative build systems

["Uwe Schindler (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/LUCENE-5755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14049355#comment-14049355 ] 

Uwe Schindler edited comment on LUCENE-5755 at 7/1/14 9:42 PM:
---------------------------------------------------------------

Hi Matt,

I don't think that the junit4-runner needs to be included into gradle itsself. Just see it as a plugin for running tests. It is the same as Elasticsearch uses this runner instead of surefire (in their Maven build). And because gradle is able to execute Ant tasks without any problems, there is no need to make a "gradle" plugin out of it. Just use the gradle scripting to execute the ant task. 

In fact Lucene uses many other additional things to make the tests validate more: We use a special SecurityManager that also prevents to escape the test's working dir or to prevent that a tests calls System.halt(). This security manager relies on the Junit4-Runner.

Also the runner not only parallelizes, it also keeps statistics about the performance of tests to reorder them on next run, so slower tests don't leave the last JVM run longer just because the running tests take too long. Another important thing is that the runner randomizes the test execution order, whcih is important to prevent bugs that are caused by tests leaving state that influence others.

In any case, Lucene's build uses lots of other Ant tasks and Groovy scripts while building, e.g. forbidden-apis. Those are not available (and will never be as native gradle tasks), because you can invoke them as plain Ant tasks. You just have to declare a dependency in gradle and then invoke them.

We also have special tasks to build the documentation, including XSL transformations and Markdown processing. This is all implemented in groovy ("ant pegdown task"), so you can just copypaste the code to gradle (which is groovy). This is one reason why I prefer Gradle instead of buildr.

So my wish:
We should use Gradle to do the dependency management, but all tasks/targets and functionality that are available in Lucene/Solr's Ant build should be preserved in an identical way, using the same external build tools (mainly ant tasks) we currently use in Ant. Very important is also to *not* use transitive dependencies (Lucene prefers to declare every source code dependency explicit, the exemption are just build tools like Ant tasks loaded from Maven central)


was (Author: thetaphi):
Hi Matt,

I don't think that the junit4-runner needs to be included into gradle itsself. Just see it as a plugin for running tests. It is the same as Elasticsearch uses this runner instead of surefire (in their Maven build). And because gradle is able to execute Ant tasks without any problems, there is no need to make a "gradle" plugin out of it. Just use the gradle scripting to execute the ant task. 

In fact Lucene uses many other additional things to make the tests validate more: We use a special SecurityManager that also prevents to escape the test's working dir or to prevent that a tests calls System.halt(). This security manager relies on the Junit4-Runner.

Also the runner not only parallelizes, it also keeps statistics about the performance of tests to reorder them on next run, so slower tests don't leave the last JVM run longer just because the running tests take too long. Another important thing is that the runner randomizes the test execution order, whcih is important to prevent bugs that are caused by tests

In any case, Lucene's build uses lots of other Ant tasks and Groovy scripts while building, e.g. forbidden-apis. Those are not available (and will never be as native gradle tasks), because you can invoke them as plain Ant tasks. You just have to declare a dependency in gradle and then invoke them.

We also have special tasks to build the documentation, including XSL transformations and Markdown processing. This is all implemented in groovy ("ant pegdown task"), so you can just copypaste the code to gradle (which is groovy). This is one reason why I prefer Gradle instead of buildr.

So my wish:
We should use Gradle to do the dependency management, but all tasks/targets and functionality that are available in Lucene/Solr's Ant build should be preserved in an identical way, using the same external build tools (mainly ant tasks) we currently use in Ant. Very important is also to *not* use transitive dependencies (Lucene prefers to declare every source code dependency explicit, the exemption are just build tools like Ant tasks loaded from Maven central)

> Explore alternative build systems
> ---------------------------------
>
>                 Key: LUCENE-5755
>                 URL: https://issues.apache.org/jira/browse/LUCENE-5755
>             Project: Lucene - Core
>          Issue Type: Task
>            Reporter: Dawid Weiss
>            Assignee: Dawid Weiss
>            Priority: Minor
>
> I am dissatisfied with how ANT and submodules currently work in Lucene/ Solr. It's not even the tool's fault; it seems Lucene builds just hit the borders of what it can do, especially in terms of submodule dependencies etc.
> I don't think Maven will help much too, given certain things I'd like to have in the build (for example collect all tests globally for a single execution phase at the end of the build, to support better load-balancing).
> I'd like to explore Gradle as an alternative. This task is a notepad for thoughts and experiments.
> An example of a complex (?) gradle build is javafx, for example.
> http://hg.openjdk.java.net/openjfx/8/master/rt/file/f89b7dc932af/build.gradle



--
This message was sent by Atlassian JIRA
(v6.2#6252)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org