[jira] [Comment Edited] (LUCENE-9836) Fix 8.x Maven to work with Maven Central again

["Uwe Schindler (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/LUCENE-9836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17300815#comment-17300815 ] 

Uwe Schindler edited comment on LUCENE-9836 at 3/13/21, 12:05 PM:
------------------------------------------------------------------

Hi,
I digged into that. Actually there's no way to fix this with maven-ant-tasks, as it is a bug. It happens in the following cases, and also some entries on the web clarify the same:

What we do is correct, we just add another "remote" mirror or Maven Central (which is hardcoded in maven-ant-tasks to HTTP). For normal resolving this works fine, you can see this in log outputs. For the special case of BOMs (Bill of Materials), which are POM-only artifacts without anything referenced normally only inside dependencyManagement sections of other POMs, due to a bug in maven-ant-tasks, the list of remote reposititories is not used at all, although the error message implies it.

This causes that e.g., protobuf-bom (and various others) cannot download the POM and fails, but you can reproduce it with all BOMs somewehere in the dependencies.

In short: It's unfixable (at least with completely outdated and unmaintained maven-ant-tasks).

BUT: We can migrate the 8.x builds to the new "maven-resolver-ant-tasks" (https://maven.apache.org/resolver-ant-tasks/), which I already tried. The newest version is based on Maven 3.6.3 and was released in December 2020. The problem: We have to rewrite all the Maven Ant tasks to go away from {{artifact:}} namespace to {{resolver:}} namespace.

I started with that already and I ported over the buggy validation task (which is uncritical, as it does not deploy artifacts produced by our build). It works, but there's a small problem with our POM files:
Maven-Ant-Tasks was not even trying to handle the "test-jar" artifacts that we have at two places (Maven 2 does not havy any idea about those). In our Ant build we create no test-jar artifacts and we also not deploy them. When I comment those dependencies in lucene/spatial-extras and solr/analysis-extras, the whole thing works. They are only needed if you actually try to build Lucene/Solr with Maven (which does not work, too).

Here is the PR, if you comment outthe problematic dependencies in the pom.templates it works great - without configuring any extra repositories).

My proposal: Kill the pure-Maven build completely (it does not even work, it fails at various places) and just leave the minimal POM files (like Gradle does). So remove all "tasks" so they are only to describe the structure, but it will be officially impossible to build using "mvn install". 

[~sarowe]: What do you think? The Maven build is unmaintained and does not work. So let's remove it from branch_8x and leave only the cleaned up POMs without any build logic (maybe print a error message when you try to execute a maven build on them with some enforcer rule). Then clean up dependencies and only leave the productive ones.

If you think the PR looks OK, I will look into porting over the remaining artifact: logic to resolver: logic. It supports the same features, but is based on Maven 3.6.3. The only thing that is different:
- It is only "resolver" (eclipse aether,...), no Maven logic. So it can't execute Maven POMs. But that's not an issue, I removed that anyways (see above commit)
- It does NOT fully support the local settings.xml (as it misses Maven Core), everything like repositories to push artifacts too need to be configured with credentials in the build. I like this!

Not sure how to test this, especially for pushing to Apache Snapshots and Apache Releases. I can try the best I can to figure it out. Maybe I need some help by [~dweiss] and [~sarowe].

P.S.: I planned move to Maven-Resolver-Ant-Tasks with forbiddenapis. So I looked into that already last year. If this works fine here, I will also proceed there.


was (Author: thetaphi):
Hi,
I digged into that. Actually there's no way to fix this with maven-ant-tasks, as it is a bug. It happens in the following cases, and also some entries on the web clarify the same:

What we do is correct, we just add another "remote" mirror or Maven Central (which is hardcoded in maven-ant-tasks to HTTP). For normal resolving this works fine, you can see this in log outputs. For the special case of BOMs (Bill of Materials), which are POM-only artifacts without anything referenced normally only inside dependencyManagement sections of other POMs, due to a bug in maven-ant-tasks, the list of remote reposititories is not used at all, although the error message implies it.

This causes that e.g., protobuf-bom doe snot work, but you can reproduce it with all BOMs somewehere in the dependencies.

In short: It's unfixable (at least with completely outdated and unmaintained maven-ant-tasks).

BUT: We can migrate the 8.x builds to the new "maven-resolver-ant-tasks" (https://maven.apache.org/resolver-ant-tasks/), which I already tried. The newest version is based on Maven 3.6.3 and was released in December 2020. The problem: We have to rewrite all the Maven Ant tasks to go away from {{artifact:}} namespace to {{resolver:}} namespace.

I started with that already and I ported over the buggy validation task (which is uncritical, as it does not deploy artifacts produced by our build). It works, but there's a small problem with our POM files:
Maven-Ant-Tasks was not even trying to handle the "test-jar" artifacts that we have at two places (Maven 2 does not havy any idea about those). In our Ant build we create no test-jar artifacts and we also not deploy them. When I comment those dependencies in lucene/spatial-extras and solr/analysis-extras, the whole thing works. They are only needed if you actually try to build Lucene/Solr with Maven (which does not work, too).

Here is the PR, if you comment outthe problematic dependencies in the pom.templates it works great - without configuring any extra repositories).

My proposal: Kill the pure-Maven build completely (it does not even work, it fails at various places) and just leave the minimal POM files (like Gradle does). So remove all "tasks" so they are only to describe the structure, but it will be officially impossible to build using "mvn install". 

[~sarowe]: What do you think? The Maven build is unmaintained and does not work. So let's remove it from branch_8x and leave only the cleaned up POMs without any build logic (maybe print a error message when you try to execute a maven build on them with some enforcer rule). Then clean up dependencies and only leave the productive ones.

If you think the PR looks OK, I will look into porting over the remaining artifact: logic to resolver: logic. It supports the same features, but is based on Maven 3.6.3. The only thing that is different:
- It is only "resolver" (eclipse aether,...), no Maven logic. So it can't execute Maven POMs. But that's not an issue, I removed that anyways (see above commit)
- It does NOT fully support the local settings.xml (as it misses Maven Core), everything like repositories to push artifacts too need to be configured with credentials in the build. I like this!

Not sure how to test this, especially for pushing to Apache Snapshots and Apache Releases. I can try the best I can to figure it out. Maybe I need some help by [~dweiss] and [~sarowe].

P.S.: I planned move to Maven-Resolver-Ant-Tasks with forbiddenapis. So I looked into that already last year. If this works fine here, I will also proceed there.

> Fix 8.x Maven to work with Maven Central again
> ----------------------------------------------
>
>                 Key: LUCENE-9836
>                 URL: https://issues.apache.org/jira/browse/LUCENE-9836
>             Project: Lucene - Core
>          Issue Type: Improvement
>            Reporter: Uwe Schindler
>            Priority: Major
>          Time Spent: 10m
>  Remaining Estimate: 0h
>
> Currenty the Maven related stuff in 8.x completely fails, because Maven-Ant-Tasks is so outdated, that it has hardcoded Maven Central without HTTPS. This makes downloading fail.
> You can mostly fix this with an additional remote repository, so it can fallback to that one.
> I'd like to do the following on 8.x:
> - Remove the Ant-Support for Maven: {{ant run-maven-build}} (this no longer bootsraps, because Maven Ant Tasks can't download Maven, as here is no way to override hardcoded repo; I have a workaround in forbiddenapis, but that's too complicated, so I will simply remoe that task)
> - Fix the dependency checker: This works, but unfortunately there are some artifacts which itsself have "http:" in their POM file, those fail to download. Newer Maven versions have an hardcoded "fixer" in it, but Maven Ant Tasks again is missing this. I have no idea how to handle that.
> I already tried some heavy committing, but the only way to solve this is to replace maven-ant-tasks with the followup ant task. I am not sure if this worth the trouble!
> What do others think? Should we maybe simply disable the Maven Dependency checker?



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org