You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Steven Rowe (JIRA)" <ji...@apache.org> on 2010/09/20 16:40:32 UTC
[jira] Created: (LUCENE-2657) Auto-generate POM templates from Ant
builds
Auto-generate POM templates from Ant builds
-------------------------------------------
Key: LUCENE-2657
URL: https://issues.apache.org/jira/browse/LUCENE-2657
Project: Lucene - Java
Issue Type: Improvement
Components: Build
Reporter: Steven Rowe
Priority: Minor
Fix For: 3.1, 4.0
Lucene and Solr modules' POM templates are manually maintained, and so are not always in sync with the dependencies used by the Ant build.
It should be possible to auto-generate POM templates using build tools extending Ant's [BuildListener|http://api.dpml.net/ant/1.6.5/org/apache/tools/ant/BuildListener.html] interface, similarly to how the [ant2ide|http://gleamynode.net/articles/2234/] project generates eclipse project files.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Updated: (LUCENE-2657) Replace Maven POM templates with full
POMs, and change documentation accordingly
Posted by "Steven Rowe (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Steven Rowe updated LUCENE-2657:
--------------------------------
Attachment: LUCENE-2657.patch
First cut at Maven POMs for Lucene and Solr.
The following script, which installs non-mavenized artifacts into the user's local Maven repository, must be run before the Maven build works:
{code}
mvn install:install-file -DgroupId=com.ibm.icu -DartifactId=icu4j -Dversion=4.4 -Dpackaging=jar -Dfile=modules/analysis/icu/lib/icu4j-4_4.jar
mvn install:install-file -DgroupId=xerces -DartifactId=xercesImpl -Dversion=2.9.1-patched-XERCESJ-1257 -Dpackaging=jar -Dfile=lucene/contrib/benchmark/lib/xerces-2.9.1-patched-XERCESJ-1257.jar
mvn install:install-file -DgroupId=com.sleepycat -DartifactId=berkeleydb -Dversion=4.7.25 -Dpackaging=jar -Dfile=lucene/contrib/db/bdb/lib/db-4.7.25.jar
mvn install:install-file -DgroupId=com.sleepycat -DartifactId=berkeleydb-je -Dversion=3.3.93 -Dpackaging=jar -Dfile=lucene/contrib/db/bdb-je/lib/je-3.3.93.jar
mvn install:install-file -DgroupId=org.apache.solr -DartifactId=solr-commons-csv -Dversion=4.0-SNAPSHOT -Dpackaging=jar -Dfile=solr/lib/commons-csv-1.0-SNAPSHOT-r966014.jar
mvn install:install-file -DgroupId=org.apache.solr -DartifactId=solr-noggit -Dversion=r944541 -Dpackaging=jar -Dfile=solr/lib/apache-solr-noggit-r944541.jar
mvn install:install-file -DgroupId=org.apache.tika -DartifactId=tika-core -Dversion=0.8-SNAPSHOT -Dpackaging=jar -Dfile=solr/contrib/extraction/lib/tika-core-0.8-SNAPSHOT.jar
jar xvf solr/contrib/extraction/lib/tika-parsers-0.8-SNAPSHOT.jar META-INF/maven/org.apache.tika/tika-parsers/pom.xml
mvn install:install-file -Dfile=solr/contrib/extraction/lib/tika-parsers-0.8-SNAPSHOT.jar -DpomFile=META-INF/maven/org.apache.tika/tika-parsers/pom.xml
rm -rf META-INF
{code}
Everything compiles (using {{mvn -DskipTests clean install}}). Except for two Solr tests, all tests pass. The two tests that fail for me (on Win Vista 64, Sun JDK 1.6.0_13) are: JettyWebAppTest and Solr Cell's ExtractingRequestHandlerTest.testArabicPDF().
I have added configurations for running appassembler-maven-plugin (to produce run-time windows and linux scripts with lib/ dirs containing dependencies) for all classes with main() functions that are intended as tools rather than static tests.
I haven't tried using the POMs as project configuration for IntelliJ, so I'm not sure how that will go.
This is a work-in-progress. Please test and let me know if you find problems, or would like to see more stuff added, or if you think the design is all wrong.
One challenge worth noting is that Maven disallows aggregator POMs (those with a {{<modules>}} section directing nested Maven invocations) from producing any artifacts other than the POM itself. As a result, the 4 or 5 locations in the Lucene/Solr build that have "mixed content" (to borrow the XML term for mixed nested element and text content) have to be fudged; e.g. the Lucene and Solr core POMs are located in {{lucene/src/}} and {{solr/src/}}, because the aggregator POMs at {{lucene/}} and {{solr/}} can't produce binary artifacts. (The conventional Maven way to address this kind of issue is to make the "core" modules be siblings of the nested modules.)
> Replace Maven POM templates with full POMs, and change documentation accordingly
> --------------------------------------------------------------------------------
>
> Key: LUCENE-2657
> URL: https://issues.apache.org/jira/browse/LUCENE-2657
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Build
> Reporter: Steven Rowe
> Assignee: Steven Rowe
> Fix For: 3.1, 4.0
>
> Attachments: LUCENE-2657.patch
>
>
> The current Maven POM templates only contain dependency information, the bare bones necessary for uploading artifacts to the Maven repository.
> Full Maven POMs will include the information necessary to run a multi-module Maven build, in addition to serving the same purpose as the current POM templates.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Commented: (LUCENE-2657) Auto-generate POM templates from
Ant builds
Posted by "Chris Male (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12913445#action_12913445 ]
Chris Male commented on LUCENE-2657:
------------------------------------
bq. I don't see this issue as providing a complete Ant->Maven conversion system. The poms I envision will look just like the POM templates after they are interpolated - they will provide Maven coordinates for artifacts and their dependencies, nothing more. And the process to produce them can be carried out as part of generating Maven artifacts. I don't think they will need to be checked in, but they certainly could be.
Its this sort of middle ground that I'm unsure about. We basically have maven support today but not really. Some people understand how it works, but I've certainly seen users who are dabbling in some development and are confused by our maven support. I cant think of another project that generates its poms (even ant itself has the poms well-formed inside its codebase) so suddenly our support has become more unusual.
What I'm hoping is that we can actually add quality support for maven in such a way that users wouldn't be surprised, developers like you and I can get the benefits we know maven has, and that also simplifies what Robert has to do.
If all we can do at the moment is bolt bits of maven onto ant, I'm unsure its worth while.
> Auto-generate POM templates from Ant builds
> -------------------------------------------
>
> Key: LUCENE-2657
> URL: https://issues.apache.org/jira/browse/LUCENE-2657
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Build
> Reporter: Steven Rowe
> Priority: Minor
> Fix For: 3.1, 4.0
>
>
> Lucene and Solr modules' POM templates are manually maintained, and so are not always in sync with the dependencies used by the Ant build.
> It should be possible to auto-generate POM templates using build tools extending Ant's [BuildListener|http://api.dpml.net/ant/1.6.5/org/apache/tools/ant/BuildListener.html] interface, similarly to how the [ant2ide|http://gleamynode.net/articles/2234/] project generates eclipse project files.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Issue Comment Edited: (LUCENE-2657) Auto-generate POM
templates from Ant builds
Posted by "Robert Muir (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12912487#action_12912487 ]
Robert Muir edited comment on LUCENE-2657 at 9/20/10 10:59 AM:
---------------------------------------------------------------
How would the BuildListener interface know about dependencies? Does it have some magic way to know this?
As an example, lets take modules/analysis/icu which has 3 dependencies:
* Lucene core itself (implicit from contrib-build.xml)
* external dependency: ICU
* internal dependency: modules/analysis/common
take a look at modules/analysis/icu's pom.xml which has:
{noformat}
<dependencies>
<dependency>
<groupId>com.ibm.icu</groupId>
<artifactId>icu4j</artifactId>
<version>${icu-version}</version>
</dependency>
</dependencies>
{noformat}
However, our ant builds (that depend on common-build/contrib-build) declare their dependencies in a semi-standard way:
* External dependencies:
{noformat}
<path id="additional.dependencies">
<fileset dir="lib" includes="icu4j-*.jar"/>
</path>
{noformat}
* Internal dependencies:
{noformat}
<module-uptodate name="analysis/common"
jarfile="../build/common/lucene-analyzers-common-${version}.jar"
property="analyzers-common.uptodate" classpath.property="analyzers-common.jar"/>
{noformat}
The contrib-build.xml already has a 'dist-maven' target, that is called recursively. Perhaps an alternative would be to improve contrib-build.xml for it to have a 'generate-maven' target, also called recursively.
I've already prototyped/proposed in SOLR-2002 that we migrate the solr build to extend the lucene build, so everywhere would use it.
Furthermore, couldnt we also make a recursive 'test-maven' target, that generates a maven project to 'download' or whatever it needs, then tries to run all the tests?
If somehow the maven is broken, the tests simply won't pass.
I realize that running all of a modules tests again redundantly via 'maven' might not be the most elegant solution, but it seems like it would test that everything is working.
was (Author: rcmuir):
How would the BuildListener interface know about dependencies? Does it have some magic way to know this?
As an example, lets take modules/analysis/icu which has 3 dependencies:
* Lucene core itself (implicit from contrib-build.xml)
* external dependency: ICU
* internal dependency: modules/analysis/common
take a look at modules/analysis/icu's pom.xml which has:
{noformat}
<dependencies>
<dependency>
<groupId>com.ibm.icu</groupId>
<artifactId>icu4j</artifactId>
<version>${icu-version}</version>
</dependency>
</dependencies>
{noformat}
However, our ant builds (that depend on common-build/contrib-build) declare their dependencies in a semi-standard way:
* External dependencies:
{noformat}
<path id="additional.dependencies">
<fileset dir="lib" includes="icu4j-*.jar"/>
</path>
* Internal dependencies:
{noformat}
<module-uptodate name="analysis/common" jarfile="../build/common/lucene-analyzers-common-${version}.jar"
property="analyzers-common.uptodate" classpath.property="analyzers-common.jar"/>
{noformat}
The contrib-build.xml already has a 'dist-maven' target, that is called recursively. Perhaps an alternative would be to improve contrib-build.xml for it to have a 'generate-maven' target, also called recursively.
I've already prototyped/proposed in SOLR-2002 that we migrate the solr build to extend the lucene build, so everywhere would use it.
Furthermore, couldnt we also make a recursive 'test-maven' target, that generates a maven project to 'download' or whatever it needs, then tries to run all the tests?
If somehow the maven is broken, the tests simply won't pass.
I realize that running all of a modules tests again redundantly via 'maven' might not be the most elegant solution, but it seems like it would test that everything is working.
> Auto-generate POM templates from Ant builds
> -------------------------------------------
>
> Key: LUCENE-2657
> URL: https://issues.apache.org/jira/browse/LUCENE-2657
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Build
> Reporter: Steven Rowe
> Priority: Minor
> Fix For: 3.1, 4.0
>
>
> Lucene and Solr modules' POM templates are manually maintained, and so are not always in sync with the dependencies used by the Ant build.
> It should be possible to auto-generate POM templates using build tools extending Ant's [BuildListener|http://api.dpml.net/ant/1.6.5/org/apache/tools/ant/BuildListener.html] interface, similarly to how the [ant2ide|http://gleamynode.net/articles/2234/] project generates eclipse project files.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Commented: (LUCENE-2657) Auto-generate POM templates from
Ant builds
Posted by "Chris Male (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12913450#action_12913450 ]
Chris Male commented on LUCENE-2657:
------------------------------------
I should add that I'm currently looking at how ant supports maven releases since it does have a contrib-like folder of poms, well formed.
> Auto-generate POM templates from Ant builds
> -------------------------------------------
>
> Key: LUCENE-2657
> URL: https://issues.apache.org/jira/browse/LUCENE-2657
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Build
> Reporter: Steven Rowe
> Priority: Minor
> Fix For: 3.1, 4.0
>
>
> Lucene and Solr modules' POM templates are manually maintained, and so are not always in sync with the dependencies used by the Ant build.
> It should be possible to auto-generate POM templates using build tools extending Ant's [BuildListener|http://api.dpml.net/ant/1.6.5/org/apache/tools/ant/BuildListener.html] interface, similarly to how the [ant2ide|http://gleamynode.net/articles/2234/] project generates eclipse project files.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Commented: (LUCENE-2657) Replace Maven POM templates with
full POMs, and change documentation accordingly
Posted by "Yonik Seeley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12914104#action_12914104 ]
Yonik Seeley commented on LUCENE-2657:
--------------------------------------
bq. Full Maven POMs will include the information necessary to run a multi-module Maven build
That sort of sounds like a parallel build process (i.e. you would be able to build lucene/solr itself with maven). Is it?
We've avoided that type of thing in the past.
> Replace Maven POM templates with full POMs, and change documentation accordingly
> --------------------------------------------------------------------------------
>
> Key: LUCENE-2657
> URL: https://issues.apache.org/jira/browse/LUCENE-2657
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Build
> Reporter: Steven Rowe
> Assignee: Steven Rowe
> Fix For: 3.1, 4.0
>
>
> The current Maven POM templates only contain dependency information, the bare bones necessary for uploading artifacts to the Maven repository.
> Full Maven POMs will include the information necessary to run a multi-module Maven build, in addition to serving the same purpose as the current POM templates.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Commented: (LUCENE-2657) Auto-generate POM templates from
Ant builds
Posted by "Chris Male (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12913373#action_12913373 ]
Chris Male commented on LUCENE-2657:
------------------------------------
bq. If nothing else, it is abundantly clear that there are more than a couple of Lucene/Solr developers that don't want to put in any effort for Maven's sake.
If developers are willing to add the dependency jar and lines to the ant file but not add the 5 lines to the pom, then I'm not sure any of these attempts are really going to succeed. Sure many developers don't like maven, but I haven't seen any straight out refuse to use it yet (unless I misread the large email thread), they are just demanding ease of use.
My concern here is this process is going in the opposite direction by making it vastly more complicated. What your proposing sounds like a great app independent of Lucene - a way to convert from an ant build system to a maven one, but I'm unsure its something that should be part of a build process instead of a once off thing. Certainly when we are trying to simplify said build process.
> Auto-generate POM templates from Ant builds
> -------------------------------------------
>
> Key: LUCENE-2657
> URL: https://issues.apache.org/jira/browse/LUCENE-2657
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Build
> Reporter: Steven Rowe
> Priority: Minor
> Fix For: 3.1, 4.0
>
>
> Lucene and Solr modules' POM templates are manually maintained, and so are not always in sync with the dependencies used by the Ant build.
> It should be possible to auto-generate POM templates using build tools extending Ant's [BuildListener|http://api.dpml.net/ant/1.6.5/org/apache/tools/ant/BuildListener.html] interface, similarly to how the [ant2ide|http://gleamynode.net/articles/2234/] project generates eclipse project files.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Updated: (LUCENE-2657) Replace Maven POM templates with full
POMs, and change documentation accordingly
Posted by "Steven Rowe (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Steven Rowe updated LUCENE-2657:
--------------------------------
Attachment: LUCENE-2657.patch
This version of the patch puts the third-party non-mavenized .jar installation stuff into a profile in the top-level POM.
So instead of running the above-listed script, instead just run the following Maven invocation:
{code}
mvn -N -Pbootstrap install
{code}
{{-N}} disables recursive building, {{-Pbootstrap}} activates the "bootstrap" profile (defined in the POM) and the local repo artifact installation stuff is bound to the "install" phase in the "bootstrap" profile.
I discovered that the Tika artifact snapshots (tika-core and tika-parsers) are hosted at the Apache snapshots repository, and that was causing these dependencies not to work properly, so I switched their coordinates to org.apache.solr:solr-tika-*:4.0-SNAPSHOT, so that they will remain stable.
Also, the previous patch had some weirdness with the Solr DIH Extras POM - it wasn't being treated as a new file, but rather a diff from some other file (the POM template, I think) - anyway, this patch fixes the problem.
Everything still compiles, and all tests except the two previously failing tests still pass.
> Replace Maven POM templates with full POMs, and change documentation accordingly
> --------------------------------------------------------------------------------
>
> Key: LUCENE-2657
> URL: https://issues.apache.org/jira/browse/LUCENE-2657
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Build
> Affects Versions: 3.1, 4.0
> Reporter: Steven Rowe
> Assignee: Steven Rowe
> Fix For: 3.1, 4.0
>
> Attachments: LUCENE-2657.patch, LUCENE-2657.patch
>
>
> The current Maven POM templates only contain dependency information, the bare bones necessary for uploading artifacts to the Maven repository.
> Full Maven POMs will include the information necessary to run a multi-module Maven build, in addition to serving the same purpose as the current POM templates.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Updated: (LUCENE-2657) Replace Maven POM templates with full
POMs, and change documentation accordingly
Posted by "Steven Rowe (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Steven Rowe updated LUCENE-2657:
--------------------------------
Attachment: LUCENE-2657.patch
Synch'd to trunk (ICU4J and Xerces-J dependency upgrades). Use {{mvn -N -Pbootstrap install}} from the top-level directory in order to install the new dependencies.
> Replace Maven POM templates with full POMs, and change documentation accordingly
> --------------------------------------------------------------------------------
>
> Key: LUCENE-2657
> URL: https://issues.apache.org/jira/browse/LUCENE-2657
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Build
> Affects Versions: 3.1, 4.0
> Reporter: Steven Rowe
> Assignee: Steven Rowe
> Fix For: 3.1, 4.0
>
> Attachments: LUCENE-2657.patch, LUCENE-2657.patch, LUCENE-2657.patch, LUCENE-2657.patch
>
>
> The current Maven POM templates only contain dependency information, the bare bones necessary for uploading artifacts to the Maven repository.
> Full Maven POMs will include the information necessary to run a multi-module Maven build, in addition to serving the same purpose as the current POM templates.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Commented: (LUCENE-2657) Replace Maven POM templates with
full POMs, and change documentation accordingly
Posted by "Steven Rowe (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12919252#action_12919252 ]
Steven Rowe commented on LUCENE-2657:
-------------------------------------
bq. icu4j was missing from solr/contrib/extraction/pom.xml
See SOLR-1813 for details.
> Replace Maven POM templates with full POMs, and change documentation accordingly
> --------------------------------------------------------------------------------
>
> Key: LUCENE-2657
> URL: https://issues.apache.org/jira/browse/LUCENE-2657
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Build
> Affects Versions: 3.1, 4.0
> Reporter: Steven Rowe
> Assignee: Steven Rowe
> Fix For: 3.1, 4.0
>
> Attachments: LUCENE-2657.patch, LUCENE-2657.patch, LUCENE-2657.patch
>
>
> The current Maven POM templates only contain dependency information, the bare bones necessary for uploading artifacts to the Maven repository.
> Full Maven POMs will include the information necessary to run a multi-module Maven build, in addition to serving the same purpose as the current POM templates.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Commented: (LUCENE-2657) Auto-generate POM templates from
Ant builds
Posted by "Steven Rowe (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12913391#action_12913391 ]
Steven Rowe commented on LUCENE-2657:
-------------------------------------
{quote}
bq. It is not the build process that requires simplification (is anybody really calling for that?), but rather developers' interaction with it?
I am trying to simplify the build, with the prototype in SOLR-2002, whereas most things are in a single common-build and we invoke them recursively (things like rat, dist, test, javadocs).
{quote}
I stand corrected. Build simplification is certainly a goal I support; without the flexibility afforded by a coherent build system, issues like this one would definitely be more difficult to implement.
> Auto-generate POM templates from Ant builds
> -------------------------------------------
>
> Key: LUCENE-2657
> URL: https://issues.apache.org/jira/browse/LUCENE-2657
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Build
> Reporter: Steven Rowe
> Priority: Minor
> Fix For: 3.1, 4.0
>
>
> Lucene and Solr modules' POM templates are manually maintained, and so are not always in sync with the dependencies used by the Ant build.
> It should be possible to auto-generate POM templates using build tools extending Ant's [BuildListener|http://api.dpml.net/ant/1.6.5/org/apache/tools/ant/BuildListener.html] interface, similarly to how the [ant2ide|http://gleamynode.net/articles/2234/] project generates eclipse project files.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Commented: (LUCENE-2657) Replace Maven POM templates with
full POMs, and change documentation accordingly
Posted by "Steven Rowe (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12917750#action_12917750 ]
Steven Rowe commented on LUCENE-2657:
-------------------------------------
bq. What failure are you seeing?
Two apparently unrelated errors:
# {{JettyWebappTest.testJSP()}} returns an HTTP 500 error as soon as it attempts to connect to the Jetty {{Server}} instantiated in {{setUp()}}, on line #88. I haven't figured out how to set Jetty Logging up so that I have visibility on what's happening yet. I see a call setting the {{solr.solr.home}} property in setUp() on line #49, so maybe that value is not in sync with the current directory I transferred from the Ant configuration.
# {{solr/contrib/extraction}}'s {{ExtractingRequestHandlerTest.testArabicPDF()}} test fails because no result is found containing the extracted Arabic term (line #353). I have traced content extraction up through plain text extraction from the PDF, and that seems to be working, so I don't think it's a classpath issue with Tika or PDFBox. I need to keep tracing to see happens after the text extraction.
> Replace Maven POM templates with full POMs, and change documentation accordingly
> --------------------------------------------------------------------------------
>
> Key: LUCENE-2657
> URL: https://issues.apache.org/jira/browse/LUCENE-2657
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Build
> Affects Versions: 3.1, 4.0
> Reporter: Steven Rowe
> Assignee: Steven Rowe
> Fix For: 3.1, 4.0
>
> Attachments: LUCENE-2657.patch, LUCENE-2657.patch
>
>
> The current Maven POM templates only contain dependency information, the bare bones necessary for uploading artifacts to the Maven repository.
> Full Maven POMs will include the information necessary to run a multi-module Maven build, in addition to serving the same purpose as the current POM templates.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Commented: (LUCENE-2657) Replace Maven POM templates with
full POMs, and change documentation accordingly
Posted by "Steven Rowe (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12917636#action_12917636 ]
Steven Rowe commented on LUCENE-2657:
-------------------------------------
{quote}
bq. Except for two Solr tests, all tests pass. The two tests that fail for me (on Win Vista 64, Sun JDK 1.6.0_13) are: JettyWebAppTest and Solr Cell's ExtractingRequestHandlerTest.testArabicPDF().
maybe it is an icu classpath issue. modules/analysis and solr/contrib/extraction both use different, out of date versions.
i opened LUCENE-2683
{quote}
I dropped in the ICU4J 4.4.2 jar, and everything compiled and all tests succeeded under Maven except for the same two tests, which continued to fail.
> Replace Maven POM templates with full POMs, and change documentation accordingly
> --------------------------------------------------------------------------------
>
> Key: LUCENE-2657
> URL: https://issues.apache.org/jira/browse/LUCENE-2657
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Build
> Affects Versions: 3.1, 4.0
> Reporter: Steven Rowe
> Assignee: Steven Rowe
> Fix For: 3.1, 4.0
>
> Attachments: LUCENE-2657.patch, LUCENE-2657.patch
>
>
> The current Maven POM templates only contain dependency information, the bare bones necessary for uploading artifacts to the Maven repository.
> Full Maven POMs will include the information necessary to run a multi-module Maven build, in addition to serving the same purpose as the current POM templates.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Commented: (LUCENE-2657) Replace Maven POM templates with
full POMs, and change documentation accordingly
Posted by "Steven Rowe (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12917448#action_12917448 ]
Steven Rowe commented on LUCENE-2657:
-------------------------------------
bq. Would system scope dependencies be an alternative way?
I thought of that, and it probably makes sense for most of them.
But a jar like tika-parsers-0.8-SNAPSHOT.jar, with 10 or 15 dependencies, would be even messier to deal with as a system dependency than as a locally installed artifact. We would have to go look at the POM for it (contained conveniently within the jar, as it happens), compare the dependencies against the versions of the local dependencies (which would have previously had to be transferred to the both the Solr Cell and the DIH Extras POMs) and make changes in both places.
> Replace Maven POM templates with full POMs, and change documentation accordingly
> --------------------------------------------------------------------------------
>
> Key: LUCENE-2657
> URL: https://issues.apache.org/jira/browse/LUCENE-2657
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Build
> Affects Versions: 3.1, 4.0
> Reporter: Steven Rowe
> Assignee: Steven Rowe
> Fix For: 3.1, 4.0
>
> Attachments: LUCENE-2657.patch
>
>
> The current Maven POM templates only contain dependency information, the bare bones necessary for uploading artifacts to the Maven repository.
> Full Maven POMs will include the information necessary to run a multi-module Maven build, in addition to serving the same purpose as the current POM templates.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Commented: (LUCENE-2657) Auto-generate POM templates from
Ant builds
Posted by "Robert Muir (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12912548#action_12912548 ]
Robert Muir commented on LUCENE-2657:
-------------------------------------
{quote}
BuildListener has hooks for build task onset and completion events (inter alia). ant2ide listens for Javac task completion, and captures from it the source and target directories, as well as the build classpath. You have to invoke compilation from an Ant build in order for this to work.
Seems kinda magical to me
{quote}
OK, i guess this is similar to when I open the project up in eclipse and say 'from existing ant build.xml file' and it sets up my eclipse project.
> Auto-generate POM templates from Ant builds
> -------------------------------------------
>
> Key: LUCENE-2657
> URL: https://issues.apache.org/jira/browse/LUCENE-2657
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Build
> Reporter: Steven Rowe
> Priority: Minor
> Fix For: 3.1, 4.0
>
>
> Lucene and Solr modules' POM templates are manually maintained, and so are not always in sync with the dependencies used by the Ant build.
> It should be possible to auto-generate POM templates using build tools extending Ant's [BuildListener|http://api.dpml.net/ant/1.6.5/org/apache/tools/ant/BuildListener.html] interface, similarly to how the [ant2ide|http://gleamynode.net/articles/2234/] project generates eclipse project files.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Commented: (LUCENE-2657) Replace Maven POM templates with
full POMs, and change documentation accordingly
Posted by "Chris Male (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12917438#action_12917438 ]
Chris Male commented on LUCENE-2657:
------------------------------------
Hi Steven,
bq. The following script, which installs non-mavenized artifacts into the user's local Maven repository
Would system scope dependencies be an alternative way? Then we can specify the systemPath to where the jars are in the lib folders.
Reason I suggest this is that if someone replaces one of those jar (which will happen I'm sure) or simply adds another, then we cannot
simply update the appropriate pom. We'd also need to install the jars locally (possibly again).
> Replace Maven POM templates with full POMs, and change documentation accordingly
> --------------------------------------------------------------------------------
>
> Key: LUCENE-2657
> URL: https://issues.apache.org/jira/browse/LUCENE-2657
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Build
> Affects Versions: 3.1, 4.0
> Reporter: Steven Rowe
> Assignee: Steven Rowe
> Fix For: 3.1, 4.0
>
> Attachments: LUCENE-2657.patch
>
>
> The current Maven POM templates only contain dependency information, the bare bones necessary for uploading artifacts to the Maven repository.
> Full Maven POMs will include the information necessary to run a multi-module Maven build, in addition to serving the same purpose as the current POM templates.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Commented: (LUCENE-2657) Auto-generate POM templates from
Ant builds
Posted by "Steven Rowe (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12913379#action_12913379 ]
Steven Rowe commented on LUCENE-2657:
-------------------------------------
bq. My concern here is this process is going in the opposite direction by making it vastly more complicated. What your proposing sounds like a great app independent of Lucene - a way to convert from an ant build system to a maven one, but I'm unsure its something that should be part of a build process instead of a once off thing. Certainly when we are trying to simplify said build process.
It is not the build process that requires simplification (is anybody really calling for that?), but rather developers' interaction with it?
> Auto-generate POM templates from Ant builds
> -------------------------------------------
>
> Key: LUCENE-2657
> URL: https://issues.apache.org/jira/browse/LUCENE-2657
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Build
> Reporter: Steven Rowe
> Priority: Minor
> Fix For: 3.1, 4.0
>
>
> Lucene and Solr modules' POM templates are manually maintained, and so are not always in sync with the dependencies used by the Ant build.
> It should be possible to auto-generate POM templates using build tools extending Ant's [BuildListener|http://api.dpml.net/ant/1.6.5/org/apache/tools/ant/BuildListener.html] interface, similarly to how the [ant2ide|http://gleamynode.net/articles/2234/] project generates eclipse project files.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Updated: (LUCENE-2657) Replace Maven POM templates with full
POMs, and change documentation accordingly
Posted by "Steven Rowe (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Steven Rowe updated LUCENE-2657:
--------------------------------
Summary: Replace Maven POM templates with full POMs, and change documentation accordingly (was: Auto-generate POM templates from Ant builds)
Priority: Major (was: Minor)
Description:
The current Maven POM templates only contain dependency information, the bare bones necessary for uploading artifacts to the Maven repository.
Full Maven POMs will include the information necessary to run a multi-module Maven build, in addition to serving the same purpose as the current POM templates.
was:
Lucene and Solr modules' POM templates are manually maintained, and so are not always in sync with the dependencies used by the Ant build.
It should be possible to auto-generate POM templates using build tools extending Ant's [BuildListener|http://api.dpml.net/ant/1.6.5/org/apache/tools/ant/BuildListener.html] interface, similarly to how the [ant2ide|http://gleamynode.net/articles/2234/] project generates eclipse project files.
I have changed the issue summary and description to manual generation of authoritative POMs, rather than automatic synching of POM templates with Ant dependencies, which can be pursued later.
> Replace Maven POM templates with full POMs, and change documentation accordingly
> --------------------------------------------------------------------------------
>
> Key: LUCENE-2657
> URL: https://issues.apache.org/jira/browse/LUCENE-2657
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Build
> Reporter: Steven Rowe
> Fix For: 3.1, 4.0
>
>
> The current Maven POM templates only contain dependency information, the bare bones necessary for uploading artifacts to the Maven repository.
> Full Maven POMs will include the information necessary to run a multi-module Maven build, in addition to serving the same purpose as the current POM templates.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Commented: (LUCENE-2657) Auto-generate POM templates from
Ant builds
Posted by "Chris Male (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12913350#action_12913350 ]
Chris Male commented on LUCENE-2657:
------------------------------------
I'm wondering whether generating poms is just going to make them more mysterious. While they are templates I can at least see them and they can be validated by XML validators (including IDEs). Suddenly they are created through a more complex process which we also have to maintain.
Assuming we have well-structured poms in the first place (huge assumption, different issue), are developers unprepared to add the appropriate dependency to the necessary pom? It doesn't seem like it'll happen very often, most of our artifacts have very few dependencies and we resist increasing them. We can document the process, make it very clear exactly what needs to be added and where.
Also, I'm not sure all jars are going to have the appropriate manifest. If the pom is generated (like Artifactory can do btw), I'm not sure it is packaged in the jar?
I think Robert was onto the right idea with this original proposal: If we consider our modules section and if we add a pom at the modules level and then one for each module, running a build from the modules level will recursively build each module. If there is something wrong with the poms themselves, maven will fail. If a dependency is omitted meaning the code cannot be built, maven will fail. If any of these 2 issues occur, then the build would fail and an issue can be created for those more comfortable with maven to fix.
> Auto-generate POM templates from Ant builds
> -------------------------------------------
>
> Key: LUCENE-2657
> URL: https://issues.apache.org/jira/browse/LUCENE-2657
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Build
> Reporter: Steven Rowe
> Priority: Minor
> Fix For: 3.1, 4.0
>
>
> Lucene and Solr modules' POM templates are manually maintained, and so are not always in sync with the dependencies used by the Ant build.
> It should be possible to auto-generate POM templates using build tools extending Ant's [BuildListener|http://api.dpml.net/ant/1.6.5/org/apache/tools/ant/BuildListener.html] interface, similarly to how the [ant2ide|http://gleamynode.net/articles/2234/] project generates eclipse project files.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Commented: (LUCENE-2657) Replace Maven POM templates with
full POMs, and change documentation accordingly
Posted by "Robert Muir (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12917720#action_12917720 ]
Robert Muir commented on LUCENE-2657:
-------------------------------------
bq. I dropped in the ICU4J 4.4.2 jar, and everything compiled and all tests succeeded under Maven except for the same two tests, which continued to fail.
Well it wasn't the icu issue then, but we needed to upgrade anyway for the bugfixes, no loss.
What failure are you seeing? When prototyping the the ant system refactor (SOLR-2002 / SOLR-2002_merged.patch), i had a few issues
with some tests but I don't remember the details. I remember it was really important to get the CWD of the junit run correct, e.g. in that patch its:
{noformat}
<property name="tests.userdir" location="src/test/resources"/>
and later invoked as junit's dir="${tests.userdir}" attribute
{noformat}
> Replace Maven POM templates with full POMs, and change documentation accordingly
> --------------------------------------------------------------------------------
>
> Key: LUCENE-2657
> URL: https://issues.apache.org/jira/browse/LUCENE-2657
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Build
> Affects Versions: 3.1, 4.0
> Reporter: Steven Rowe
> Assignee: Steven Rowe
> Fix For: 3.1, 4.0
>
> Attachments: LUCENE-2657.patch, LUCENE-2657.patch
>
>
> The current Maven POM templates only contain dependency information, the bare bones necessary for uploading artifacts to the Maven repository.
> Full Maven POMs will include the information necessary to run a multi-module Maven build, in addition to serving the same purpose as the current POM templates.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Commented: (LUCENE-2657) Auto-generate POM templates from
Ant builds
Posted by "Robert Muir (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12913393#action_12913393 ]
Robert Muir commented on LUCENE-2657:
-------------------------------------
bq. I stand corrected. Build simplification is certainly a goal I support; without the flexibility afforded by a coherent build system, issues like this one would definitely be more difficult to implement.
Exactly, because if you were to write a patch for this issue today, you have to implement it in 4 or 5 different places at the moment (or at least employ dirty ant hacks)
The same problem prevents us from really improving our build in other ways: for example we should probably support findbugs/pmd integration and things like that, and it should be simple.
> Auto-generate POM templates from Ant builds
> -------------------------------------------
>
> Key: LUCENE-2657
> URL: https://issues.apache.org/jira/browse/LUCENE-2657
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Build
> Reporter: Steven Rowe
> Priority: Minor
> Fix For: 3.1, 4.0
>
>
> Lucene and Solr modules' POM templates are manually maintained, and so are not always in sync with the dependencies used by the Ant build.
> It should be possible to auto-generate POM templates using build tools extending Ant's [BuildListener|http://api.dpml.net/ant/1.6.5/org/apache/tools/ant/BuildListener.html] interface, similarly to how the [ant2ide|http://gleamynode.net/articles/2234/] project generates eclipse project files.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Updated: (LUCENE-2657) Replace Maven POM templates with full
POMs, and change documentation accordingly
Posted by "Steven Rowe (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Steven Rowe updated LUCENE-2657:
--------------------------------
Affects Version/s: 3.1
4.0
Lucene Fields: [New, Patch Available] (was: [New])
> Replace Maven POM templates with full POMs, and change documentation accordingly
> --------------------------------------------------------------------------------
>
> Key: LUCENE-2657
> URL: https://issues.apache.org/jira/browse/LUCENE-2657
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Build
> Affects Versions: 3.1, 4.0
> Reporter: Steven Rowe
> Assignee: Steven Rowe
> Fix For: 3.1, 4.0
>
> Attachments: LUCENE-2657.patch
>
>
> The current Maven POM templates only contain dependency information, the bare bones necessary for uploading artifacts to the Maven repository.
> Full Maven POMs will include the information necessary to run a multi-module Maven build, in addition to serving the same purpose as the current POM templates.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Commented: (LUCENE-2657) Replace Maven POM templates with
full POMs, and change documentation accordingly
Posted by "Robert Muir (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12917368#action_12917368 ]
Robert Muir commented on LUCENE-2657:
-------------------------------------
bq. Except for two Solr tests, all tests pass. The two tests that fail for me (on Win Vista 64, Sun JDK 1.6.0_13) are: JettyWebAppTest and Solr Cell's ExtractingRequestHandlerTest.testArabicPDF().
maybe it is an icu classpath issue. modules/analysis and solr/contrib/extraction both use different, out of date versions.
i opened LUCENE-2683
> Replace Maven POM templates with full POMs, and change documentation accordingly
> --------------------------------------------------------------------------------
>
> Key: LUCENE-2657
> URL: https://issues.apache.org/jira/browse/LUCENE-2657
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Build
> Reporter: Steven Rowe
> Assignee: Steven Rowe
> Fix For: 3.1, 4.0
>
> Attachments: LUCENE-2657.patch
>
>
> The current Maven POM templates only contain dependency information, the bare bones necessary for uploading artifacts to the Maven repository.
> Full Maven POMs will include the information necessary to run a multi-module Maven build, in addition to serving the same purpose as the current POM templates.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Commented: (LUCENE-2657) Replace Maven POM templates with
full POMs, and change documentation accordingly
Posted by "Steven Rowe (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12914112#action_12914112 ]
Steven Rowe commented on LUCENE-2657:
-------------------------------------
{quote}
bq. Full Maven POMs will include the information necessary to run a multi-module Maven build
That sort of sounds like a parallel build process (i.e. you would be able to build lucene/solr itself with maven). Is it?
We've avoided that type of thing in the past.
{quote}
Yes, it constitutes a parallel build process, but the Ant build process would remain the official build. For example, the artifacts produced by the Maven build will not be exactly the same as those produced by the Ant build process, and so cannot be used as release artifacts.
It has been noted elsewhere that a Maven build has never been produced for Lucene. I hope in this issue to provide that, so that the discussion about whether or not to include it in the source tree has a concrete reference point.
> Replace Maven POM templates with full POMs, and change documentation accordingly
> --------------------------------------------------------------------------------
>
> Key: LUCENE-2657
> URL: https://issues.apache.org/jira/browse/LUCENE-2657
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Build
> Reporter: Steven Rowe
> Assignee: Steven Rowe
> Fix For: 3.1, 4.0
>
>
> The current Maven POM templates only contain dependency information, the bare bones necessary for uploading artifacts to the Maven repository.
> Full Maven POMs will include the information necessary to run a multi-module Maven build, in addition to serving the same purpose as the current POM templates.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Commented: (LUCENE-2657) Auto-generate POM templates from
Ant builds
Posted by "Robert Muir (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12913387#action_12913387 ]
Robert Muir commented on LUCENE-2657:
-------------------------------------
bq. It is not the build process that requires simplification (is anybody really calling for that?), but rather developers' interaction with it?
I am trying to simplify the build, with the prototype in SOLR-2002, whereas most things are in a single common-build and we invoke them recursively (things like rat, dist, test, javadocs).
Right now it is complicated mostly because there are actually many build setups that are essentially standalone, and then have been linked to depend on each other.
> Auto-generate POM templates from Ant builds
> -------------------------------------------
>
> Key: LUCENE-2657
> URL: https://issues.apache.org/jira/browse/LUCENE-2657
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Build
> Reporter: Steven Rowe
> Priority: Minor
> Fix For: 3.1, 4.0
>
>
> Lucene and Solr modules' POM templates are manually maintained, and so are not always in sync with the dependencies used by the Ant build.
> It should be possible to auto-generate POM templates using build tools extending Ant's [BuildListener|http://api.dpml.net/ant/1.6.5/org/apache/tools/ant/BuildListener.html] interface, similarly to how the [ant2ide|http://gleamynode.net/articles/2234/] project generates eclipse project files.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Commented: (LUCENE-2657) Auto-generate POM templates from
Ant builds
Posted by "Steven Rowe (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12913361#action_12913361 ]
Steven Rowe commented on LUCENE-2657:
-------------------------------------
bq. Assuming we have well-structured poms in the first place (huge assumption, different issue), are developers unprepared to add the appropriate dependency to the necessary pom?
If nothing else, it is abundantly clear that there are more than a couple of Lucene/Solr developers that don't want to put in *any* effort for Maven's sake.
bq. Also, I'm not sure all jars are going to have the appropriate manifest. If the pom is generated (like Artifactory can do btw), I'm not sure it is packaged in the jar?
I've looked into 10-15 jars, and very few have the required info. Some even have the wrong information in the manifest...
I think we can use .jar digest (SHA-1) as identifiers to look up the appropriate Maven info to automatically set up POM dependencies. I wrote a script to crawl the central Maven repo for SHA-1 keyed Maven coordinates (groupId:artifactId:version), and it seems to work, but at the rate it's progressing, it will take 4 days to finish. Anyway, it seems like overkill to gather info on all of the ~250k artifacts in the Maven central repo, just to handle the ~80 .jar files in the Lucene/Solr source tree.
I found a free service that provides Maven central repo search, including by artifact digest: http://www.jarvana.com/jarvana/search - it uses Lucene to power the search :). I plan on using this as the basis for automatic Maven coordinate lookup for arbitrary .jars.
> Auto-generate POM templates from Ant builds
> -------------------------------------------
>
> Key: LUCENE-2657
> URL: https://issues.apache.org/jira/browse/LUCENE-2657
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Build
> Reporter: Steven Rowe
> Priority: Minor
> Fix For: 3.1, 4.0
>
>
> Lucene and Solr modules' POM templates are manually maintained, and so are not always in sync with the dependencies used by the Ant build.
> It should be possible to auto-generate POM templates using build tools extending Ant's [BuildListener|http://api.dpml.net/ant/1.6.5/org/apache/tools/ant/BuildListener.html] interface, similarly to how the [ant2ide|http://gleamynode.net/articles/2234/] project generates eclipse project files.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Commented: (LUCENE-2657) Auto-generate POM templates from
Ant builds
Posted by "Steven Rowe (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12913386#action_12913386 ]
Steven Rowe commented on LUCENE-2657:
-------------------------------------
I don't see this issue as providing a complete Ant->Maven conversion system. The poms I envision will look just like the POM templates after they are interpolated - they will provide Maven coordinates for artifacts and their dependencies, nothing more. And the process to produce them can be carried out as part of generating Maven artifacts. I don't think they will need to be checked in, but they certainly could be.
In other words, I don't plan on building a complete Maven build configuration here. Just the bare-bones POMs required to upload artifacts, like the manually synched system we have now. Only without the manual synching part.
Vive la simplification!
> Auto-generate POM templates from Ant builds
> -------------------------------------------
>
> Key: LUCENE-2657
> URL: https://issues.apache.org/jira/browse/LUCENE-2657
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Build
> Reporter: Steven Rowe
> Priority: Minor
> Fix For: 3.1, 4.0
>
>
> Lucene and Solr modules' POM templates are manually maintained, and so are not always in sync with the dependencies used by the Ant build.
> It should be possible to auto-generate POM templates using build tools extending Ant's [BuildListener|http://api.dpml.net/ant/1.6.5/org/apache/tools/ant/BuildListener.html] interface, similarly to how the [ant2ide|http://gleamynode.net/articles/2234/] project generates eclipse project files.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Commented: (LUCENE-2657) Auto-generate POM templates from
Ant builds
Posted by "Steven Rowe (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12912495#action_12912495 ]
Steven Rowe commented on LUCENE-2657:
-------------------------------------
bq. How would the BuildListener interface know about dependencies? Does it have some magic way to know this?
BuildListener has hooks for build task onset and completion events (inter alia). ant2ide listens for Javac task completion, and captures from it the source and target directories, as well as the build classpath. You have to invoke compilation from an Ant build in order for this to work.
Seems kinda magical to me :)
The missing part here is figuring out the maven groupId/artifactId/version, and I *think* this can be dealt with by looking at the manifest in the jar. Maven-produced jars also contain their POMs, and pulling from there would be even simpler.
> Auto-generate POM templates from Ant builds
> -------------------------------------------
>
> Key: LUCENE-2657
> URL: https://issues.apache.org/jira/browse/LUCENE-2657
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Build
> Reporter: Steven Rowe
> Priority: Minor
> Fix For: 3.1, 4.0
>
>
> Lucene and Solr modules' POM templates are manually maintained, and so are not always in sync with the dependencies used by the Ant build.
> It should be possible to auto-generate POM templates using build tools extending Ant's [BuildListener|http://api.dpml.net/ant/1.6.5/org/apache/tools/ant/BuildListener.html] interface, similarly to how the [ant2ide|http://gleamynode.net/articles/2234/] project generates eclipse project files.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Assigned: (LUCENE-2657) Replace Maven POM templates with
full POMs, and change documentation accordingly
Posted by "Steven Rowe (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Steven Rowe reassigned LUCENE-2657:
-----------------------------------
Assignee: Steven Rowe
> Replace Maven POM templates with full POMs, and change documentation accordingly
> --------------------------------------------------------------------------------
>
> Key: LUCENE-2657
> URL: https://issues.apache.org/jira/browse/LUCENE-2657
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Build
> Reporter: Steven Rowe
> Assignee: Steven Rowe
> Fix For: 3.1, 4.0
>
>
> The current Maven POM templates only contain dependency information, the bare bones necessary for uploading artifacts to the Maven repository.
> Full Maven POMs will include the information necessary to run a multi-module Maven build, in addition to serving the same purpose as the current POM templates.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Updated: (LUCENE-2657) Replace Maven POM templates with full
POMs, and change documentation accordingly
Posted by "Steven Rowe (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Steven Rowe updated LUCENE-2657:
--------------------------------
Attachment: LUCENE-2657.patch
Both of the failing tests were caused by missing dependencies: jetty jsp-2.1 (scope=provided) was missing from {{solr/src/pom.xml}}, and icu4j was missing from {{solr/contrib/extraction/pom.xml}}.
With this version of the patch, all tests now pass.
IntelliJ IDEA project import from the Maven configuration mostly works, with the exception that because the Solr core and Solrj modules each include two source directories, and IntelliJ doesn't grok this fact from the POMs (specified via {{build-helper-maven-plugin}} configuration), builds including these two modules fail, since they aren't seeing all of their source code.
http://youtrack.jetbrains.net/issue/IDEA-27176 has a work-around for this issue: click on "update folders for all projects" button in the maven projects pane - this works for Solr core ({{webapp/src/}} is added a source root), but not for Solrj, likely because the second source root {{solr/src/common/}} is a sibling of the module root {{solr/src/solrj/}}, rather than a child.
> Replace Maven POM templates with full POMs, and change documentation accordingly
> --------------------------------------------------------------------------------
>
> Key: LUCENE-2657
> URL: https://issues.apache.org/jira/browse/LUCENE-2657
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Build
> Affects Versions: 3.1, 4.0
> Reporter: Steven Rowe
> Assignee: Steven Rowe
> Fix For: 3.1, 4.0
>
> Attachments: LUCENE-2657.patch, LUCENE-2657.patch, LUCENE-2657.patch
>
>
> The current Maven POM templates only contain dependency information, the bare bones necessary for uploading artifacts to the Maven repository.
> Full Maven POMs will include the information necessary to run a multi-module Maven build, in addition to serving the same purpose as the current POM templates.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Commented: (LUCENE-2657) Auto-generate POM templates from
Ant builds
Posted by "Robert Muir (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12912487#action_12912487 ]
Robert Muir commented on LUCENE-2657:
-------------------------------------
How would the BuildListener interface know about dependencies? Does it have some magic way to know this?
As an example, lets take modules/analysis/icu which has 3 dependencies:
* Lucene core itself (implicit from contrib-build.xml)
* external dependency: ICU
* internal dependency: modules/analysis/common
take a look at modules/analysis/icu's pom.xml which has:
{noformat}
<dependencies>
<dependency>
<groupId>com.ibm.icu</groupId>
<artifactId>icu4j</artifactId>
<version>${icu-version}</version>
</dependency>
</dependencies>
{noformat}
However, our ant builds (that depend on common-build/contrib-build) declare their dependencies in a semi-standard way:
* External dependencies:
{noformat}
<path id="additional.dependencies">
<fileset dir="lib" includes="icu4j-*.jar"/>
</path>
* Internal dependencies:
{noformat}
<module-uptodate name="analysis/common" jarfile="../build/common/lucene-analyzers-common-${version}.jar"
property="analyzers-common.uptodate" classpath.property="analyzers-common.jar"/>
{noformat}
The contrib-build.xml already has a 'dist-maven' target, that is called recursively. Perhaps an alternative would be to improve contrib-build.xml for it to have a 'generate-maven' target, also called recursively.
I've already prototyped/proposed in SOLR-2002 that we migrate the solr build to extend the lucene build, so everywhere would use it.
Furthermore, couldnt we also make a recursive 'test-maven' target, that generates a maven project to 'download' or whatever it needs, then tries to run all the tests?
If somehow the maven is broken, the tests simply won't pass.
I realize that running all of a modules tests again redundantly via 'maven' might not be the most elegant solution, but it seems like it would test that everything is working.
> Auto-generate POM templates from Ant builds
> -------------------------------------------
>
> Key: LUCENE-2657
> URL: https://issues.apache.org/jira/browse/LUCENE-2657
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Build
> Reporter: Steven Rowe
> Priority: Minor
> Fix For: 3.1, 4.0
>
>
> Lucene and Solr modules' POM templates are manually maintained, and so are not always in sync with the dependencies used by the Ant build.
> It should be possible to auto-generate POM templates using build tools extending Ant's [BuildListener|http://api.dpml.net/ant/1.6.5/org/apache/tools/ant/BuildListener.html] interface, similarly to how the [ant2ide|http://gleamynode.net/articles/2234/] project generates eclipse project files.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org