You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@maven.apache.org by Matthieu BROUILLARD <ma...@brouillard.fr> on 2020/12/31 14:19:09 UTC

maven 4.0.0 new XML stuff

Hello all,

regarding the active work occurring for maven 4.0.0 I noticed the
introduction of a lot of new stuff around SAX parsing & filtering.
I am wondering if that means that it was decided that the input format of
maven projects will be XML forever meaning probably, among others, the end
of polyglot extensions.
Could you explain such a move (or point to rationals/documents) and why you
did not leverage working on the in memory object model allowing
extensions/plugins to contribute/hook in the chain of building the BuildPOM
& ConsumePOM? In the past I really thought that this move to 'Build vs
Consumer' POM would make clear separations between the input format of
descriptors and the core system but I perhaps misunderstood things.

Also, are there plans regarding the future of core extensions?
With core extensions it was possible to hook into the POM model loading and
do transformations to do dynamic changes but by working on the XML directly
I see a shift (if not red stop) in this contribution/delegation mechanism.

Thanks for your time & answers.

Matthieu Brouillard

Re: maven 4.0.0 new XML stuff

Posted by Matthieu Brouillard <ma...@brouillard.fr>.
> Created https://issues.apache.org/jira/browse/MNG-7067 to start, now we
have an umbrella let's refine the description if needed.
Excellent and thanks.

> regarding jgitver ... Do you have a stacktrace for this?
there is no error/problem because it dumps the model manually to a pom.xml
file and reattached it to projects.
When above created issue https://issues.apache.org/jira/browse/MNG-7067
will be solved there should be no need for this specific stuff anymore.

Re: maven 4.0.0 new XML stuff

Posted by Romain Manni-Bucau <rm...@gmail.com>.
Created https://issues.apache.org/jira/browse/MNG-7067 to start, now we
have an umbrella let's refine the description if needed.

Romain Manni-Bucau
@rmannibucau <https://twitter.com/rmannibucau> |  Blog
<https://rmannibucau.metawerx.net/> | Old Blog
<http://rmannibucau.wordpress.com> | Github <https://github.com/rmannibucau> |
LinkedIn <https://www.linkedin.com/in/rmannibucau> | Book
<https://www.packtpub.com/application-development/java-ee-8-high-performance>


Le mar. 5 janv. 2021 à 13:52, Robert Scholte <rf...@apache.org> a
écrit :

> Thank you, now we have something to work on.
>
> polyglot should still work, but it looks like it is getting the wrong set
> of options. So that needs to be fixed. Please create an issue[1]
> regarding jgitver and don't see an issue at first glance. It is using
> MavenXpp3Reader, which is low level, no inheritence.
> Do you have a stacktrace for this?
>
> thanks,
> Robert
> [1] https://issues.apache.org/jira/browse/MNG
> On 5-1-2021 13:17:21, Matthieu Brouillard <ma...@brouillard.fr> wrote:
> > You both keep talking about extensions, but I haven't touched that part.
> > I suggest to test current Maven 4 first and to find cases where those
> extensions don't work anymore.
> I do not know the exact internals of polyglot but for jgitver, in order to
> produce a pom-4.0.0.xml compliant file, either:
> - delegates to flatten-maven-plugin
> - uses an internal mojo
> for the production/attachment of a modified pom.xml file dump from the POM.
> So in both cases, there is somewhere in the code, some ugly code like:
>
> File dumpedModel = dumpFromModel(model);
> project.setPomFile(dumpedModel)
>
> In jgitver it is called from a Mojo(1) which in the end dump and update the
> model (2)
>
> I really thought that with 4.0 and the consumer pom this would be obsolete.
>
> For polyglot, I tried but it does not work with maven-4.0.0, see #224 (3)
>
> 1:
>
> https://github.com/jgitver/jgitver-maven-plugin/blob/master/src/main/java/fr/brouillard/oss/jgitver/mojos/JGitverAttachModifiedPomsMojo.java
> 2:
>
> https://github.com/jgitver/jgitver-maven-plugin/blob/master/src/main/java/fr/brouillard/oss/jgitver/JGitverUtils.java#L182
> 3: https://github.com/takari/polyglot-maven/issues/224
>
>
> On Tue, Jan 5, 2021 at 12:52 PM Robert Scholte wrote:
>
> > You both keep talking about extensions, but I haven't touched that part.
> > I suggest to test current Maven 4 first and to find cases where those
> > extensions don't work anymore.
> > Next we can see if this makes sense or not and how to solve it.
> >
> > Robert
> >
> > On 5-1-2021 12:28:59, Matthieu Brouillard wrote:
> > > Currently the model is mutable and this causes issues.
> > For me it is not the root cause, it does not help for sure but IMO the
> > problem is more that there are currently several sources of trust : the
> POM
> > & the pom.xml.
> >
> > Personally (only my opinion) I see no other way for the future of maven
> > than to go to a single source of trust: the POM (as in memory model).
> > During the build process this POM could (and has to) become immutable for
> > sure so that plugins cannot do ugly things anymore but there should be
> also
> > clear loading and extension mechanisms/hooks before it becomes immutable.
> >
> > > or do you push to drop extensions support?
> > I hope this is not the case.
> >
> > On Tue, Jan 5, 2021 at 11:58 AM Romain Manni-Bucau
> > wrote:
> >
> > > Hmm, extensions define this kind of lifecycle, at least the
> > afterModelRead
> > > which is a hook before it could be immutable no?
> > > So not sure how it changes the issue, or do you push to drop extensions
> > > support?
> > >
> > > Romain Manni-Bucau
> > > @rmannibucau | Blog
> > > | Old Blog
> > > | Github
> > > https://github.com/rmannibucau> |
> > > LinkedIn | Book
> > >
> > >
> >
> https://www.packtpub.com/application-development/java-ee-8-high-performance
> > > >
> > >
> > >
> > > Le mar. 5 janv. 2021 à 11:54, Robert Scholte a
> > > écrit :
> > >
> > > > Currently the model is mutable and this causes issues.
> > > > Instead I would like to see that once the BuildPlan is finished, the
> > > model
> > > > becomes immutable.
> > > >
> > > >
> > > > To give a concrete example: It must be possible for code generating
> > goals
> > > > to add dependencies.
> > > > Now, when using modello readers/writers you often need to add a
> > required
> > > > dependency by hand.
> > > > Ideally there will be a hook where a plugin can register additional
> > > > dependencies (e.g. dom4j).
> > > >
> > > > This will make any postprocessing of the pom during build obsolete
> > > >
> > > > Robert
> > > > On 5-1-2021 09:09:11, Matthieu Brouillard
> > > wrote:
> > > > > Can you give an example?
> > > > Like Romain has shown: dynamically added dependencies or a version
> > > > computation.
> > > >
> > > > > Most important is the support for CI-friendly versions
> > > > But if extensions dynamically compute and set the versions in the POM
> > > using
> > > > the API, the changes will not be reflected.
> > > > That's why today one has to use flatten-maven-plugin or any other
> > > > plugin/task modifying/dumping the POM model to a pom.xml file and
> > > > setting/attaching it the POM.
> > > >
> > > > If the produced artifacts (consumer pom) were computed from the POM
> > > (object
> > > > model), every change done by any extension would be part of it.
> > > >
> > > > > but this happens AFTER using the pom
> > > > Not always from the pom.xml. I thought extensions were allowed to
> > provide
> > > > ModelLocator & ModelReader to both decide which file to use for a
> > project
> > > > and how to build the in memory POM model.
> > > > So the truth should not be considered to be in the pom.xml but in the
> > > POM.
> > > >
> > > >
> > > > On Mon, Jan 4, 2021 at 10:00 PM Robert Scholte wrote:
> > > >
> > > > > answers are below.
> > > > >
> > > > > Robert
> > > > >
> > > > > On 4-1-2021 16:52:23, Matthieu Brouillard wrote:
> > > > > @Robert nothing is broken atm, the changes for consumer/build are
> > > > currently
> > > > > behind your feature flag.
> > > > > Robert Scholte:
> > > > > It is active by default, so it is actually not hidden.
> > > > >
> > > > >
> > > > >
> > > > > But as I feared previously, and as Romain pointed it, by working at
> > XML
> > > > > level (and not at POM level) the produced consumer pom does not
> > reflect
> > > > > changes from extensions.
> > > > > Robert Scholte:
> > > > > Can you give an example?
> > > > >
> > > > >
> > > > > I really thought that all the "consumer/build" stuff would make the
> > > > > maven-flatten-plugin useless but it looks like it will not be the
> > case
> > > if
> > > > > working at XML level.
> > > > > Robert Scholte:
> > > > > Like with most questions: it depends. Most important is the support
> > for
> > > > > CI-friendly versions. In this case you won't need the
> > > > flatten-maven-plugin
> > > > > anymore.
> > > > > However, the plugin can rewrite much more, but this happens AFTER
> > using
> > > > > the pom.
> > > > > That's something I don't like, because this POM was not used to
> build
> > > the
> > > > > project, but it was reassembled afterwards.
> > > > > My idea is to provide hooks to parts of the pom that might be
> > adjusted,
> > > > > but this is something we can work on during the 4.x releases
> > > > >
> > > > > Did Romain and I miss the whole point of the "consumer/build"
> > > > enhancements
> > > > > or is it "just" because current implementation has not yet reached
> > the
> > > > > targets/outputs?
> > > > >
> > > > > On Mon, Jan 4, 2021 at 2:56 PM Romain Manni-Bucau
> > > > > wrote:
> > > > >
> > > > > > Hmm, I don't get a few things of this IT:
> > > > > >
> > > > > > 1. the formatting seems not expected even if valid (the comments
> > are
> > > > > > finishing with the first tag for example "-->
> > > > > > public class MyListener extends
> AbstractMavenLifecycleParticipant {
> > > > > >
> > > > > > @Override
> > > > > > public void afterProjectsRead(final MavenSession session) throws
> > > > > > MavenExecutionException {
> > > > > > if (session.getCurrentProject() == null) {
> > > > > > return;
> > > > > > }
> > > > > >
> > > > > > session.getProjects().forEach(p -> {
> > > > > > final Dependency dependency = new Dependency();
> > > > > > dependency.setGroupId("junit");
> > > > > > dependency.setArtifactId("junit");
> > > > > > dependency.setVersion("3.8.1");
> > > > > > p.getDependencies().add(dependency);
> > > > > > });
> > > > > > }
> > > > > > }
> > > > > >
> > > > > >
> > > > > > 2. If you run mvn (4 snapshot) dependency:tree you get this kind
> of
> > > > > output:
> > > > > >
> > > > > > [INFO] -------------
> > > > > > >-------------
> > > > > > [INFO] Building Multi Chapter Simple Web Application Project
> > > > > > 0.9-MNG6957-SNAPSHOT [6/6]
> > > > > > [INFO] --------------------------------[ jar
> > > > > > ]---------------------------------
> > > > > > [INFO]
> > > > > > [INFO] --- maven-dependency-plugin:3.1.2:tree (default-cli) @
> > > > > simple-webapp
> > > > > > ---
> > > > > > [INFO]
> > > > > org.sonatype.mavenbook.multi:simple-webapp:jar:0.9-MNG6957-SNAPSHOT
> > > > > > [INFO] +-
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> org.sonatype.mavenbook.multi:simple-weather:jar:0.9-MNG6957-SNAPSHOT:compile
> > > > > > [INFO] \- junit:junit:jar:3.8.1:compile
> > > > > >
> > > > > > [INFO]
> > > > > >
> > > >
> > ------------------------------------------------------------------------
> > > > > >
> > > > > > 3. run the build to have produced pom and cat the simple-webapp
> > one:
> > > > > >
> > > > > >
> > > > > > 4.0.0
> > > > > >
> > > > > > org.sonatype.mavenbook.multi
> > > > > > simple-parent
> > > > > > 0.9-MNG6957-SNAPSHOT
> > > > > >
> > > > > >
> > > > > > simple-webapp
> > > > > > Multi Chapter Simple Web Application Project
> > > > > >
> > > > > >
> > > > > > org.sonatype.mavenbook.multi
> > > > > > simple-weather
> > > > > > 0.9-MNG6957-SNAPSHOT
> > > > > >
> > > > > >
> > > > > >
> > > > > > simple-webapp
> > > > > >
> > > > > >
> > > > > >
> > > > > > org.apache.maven.plugins
> > > > > > maven-war-plugin
> > > > > > 2.6
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > As you see the dependency is not there. I guess the expected
> outout
> > > is:
> > > > > >
> > > > > >
> > > > > >
> > > > > > 4.0.0
> > > > > > simple-webapp
> > > > > > Multi Chapter Simple Web Application Project
> > > > > > [description, scm, ..., all central requires sections but not
> build
> > > > ones]
> > > > > >
> > > > > >
> > > > > > org.sonatype.mavenbook.multi
> > > > > > simple-weather
> > > > > > 0.9-MNG6957-SNAPSHOT
> > > > > >
> > > > > >
> > > > > > junit
> > > > > > junit
> > > > > > 3.8.1
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > Am I missing something?
> > > > > >
> > > > > > Romain Manni-Bucau
> > > > > > @rmannibucau | Blog
> > > > > > | Old Blog
> > > > > > | Github
> > > > > > https://github.com/rmannibucau> |
> > > > > > LinkedIn | Book
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://www.packtpub.com/application-development/java-ee-8-high-performance
> > > > > > >
> > > > > >
> > > > > >
> > > > > > Le lun. 4 janv. 2021 à 13:41, Robert Scholte a
> > > > > > écrit :
> > > > > >
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://github.com/apache/maven-integration-testing/tree/master/core-it-suite/src/test/resources/mng-6957-buildconsumer
> > > > > > > is the most complete IT
> > > > > > >
> > > > > > > On 4-1-2021 12:59:51, Romain Manni-Bucau wrote:
> > > > > > > Le lun. 4 janv. 2021 à 12:36, Robert Scholte a
> > > > > > > écrit :
> > > > > > >
> > > > > > > > There's just one thing I want to say:
> > > > > > > > I'm having trouble with the term "broken".
> > > > > > > >
> > > > > > >
> > > > > > > Well, literally meant broken as decorelated from the user
> intent
> > > and
> > > > > > > extension model.
> > > > > > > Anyway, didn't intend to blame but more identify the blockers
> > for a
> > > > GA
> > > > > so
> > > > > > > point was really that it seem that on the two sides only the
> > > > producing
> > > > > > one
> > > > > > > is not yet ready since it keeps does not sanitize the model
> > > > completely
> > > > > > and
> > > > > > > keeps build only data like comments, right? Also not yet clear
> > for
> > > me
> > > > > if
> > > > > > we
> > > > > > > loose the extension enrichments there.
> > > > > > >
> > > > > > >
> > > > > > > > If a Maven project could be built with M3.6.3, it can still
> be
> > > > built
> > > > > > with
> > > > > > > > M4.
> > > > > > > > If not, it is either regression (MNG-6957, MNG-7063) which
> must
> > > be
> > > > > > fixed,
> > > > > > > > or it requires changes to a plugin for understandable reasons
> > > > > > > > (maven-pgp-plugin)
> > > > > > > > AFAIK an interesting extension like the maven-tiles has been
> > > tested
> > > > > and
> > > > > > > > still works.
> > > > > > > >
> > > > > > >
> > > > > > > Do you have this handy, is it in our test suite? I'd like to
> > check
> > > > the
> > > > > > > produced pom matches the enriched model but happy to start from
> > > > > something
> > > > > > > already there.
> > > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > Robert
> > > > > > > >
> > > > > > > > On 3-1-2021 19:35:25, Romain Manni-Bucau wrote:
> > > > > > > > Le dim. 3 janv. 2021 à 19:04, Robert Scholte a
> > > > > > > > écrit :
> > > > > > > >
> > > > > > > > > I don't remember all those details anymore, because I hit
> > those
> > > > in
> > > > > > the
> > > > > > > > > beginning.
> > > > > > > > > Trying things over and over again I decided that this is
> > > probably
> > > > > the
> > > > > > > > most
> > > > > > > > > successful approach.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > What of the goals was to keep the pom.xml as is as much as
> > > > > possible.
> > > > > > > > > We can only decide for the specific Maven elements how to
> > > handle
> > > > > > them,
> > > > > > > we
> > > > > > > > > should not decide about comments and licenses.
> > > > > > > > > BTW, the license issue was hard to solve. You cannot use it
> > > from
> > > > > the
> > > > > > > > pom's
> > > > > > > > > , because there might be multiple licenses.
> > > > > > > > >
> > > > > > > >
> > > > > > > > I disagree, it is saner IMO to evolve to support that than
> > doing
> > > > > > > > anything else.
> > > > > > > > Once again you keep things which don't make sense in a
> consumed
> > > pom
> > > > > in
> > > > > > > > current impl so i'd say the sucess in a few cases breaks as
> > much
> > > > > cases
> > > > > > so
> > > > > > > > we need to revisit anyway IMHO.
> > > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > The current implementation is a solid way to ensure we're
> not
> > > > > > breaking
> > > > > > > > too
> > > > > > > > > much, because Maven controls the XML filters.
> > > > > > > > >
> > > > > > > >
> > > > > > > > Hmm, breaking extensions seems to break too much (I'm not
> > > speaking
> > > > of
> > > > > > > other
> > > > > > > > parts which breaks the ecosystem there but just that is
> > > sufficient
> > > > > IMHO
> > > > > > > to
> > > > > > > > say we must check back our solution).
> > > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > Also keep in mind, that I only want Maven to decide which
> > > > > > modifications
> > > > > > > > > are done.
> > > > > > > > >
> > > > > > > >
> > > > > > > > For the consumed pom I agree but it is consistent with
> keeping
> > > > > > everything
> > > > > > > > working instead of breaking too, no?
> > > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > Current polyglot projects should still work, but they
> cannot
> > > > > benefit
> > > > > > > from
> > > > > > > > > the build/consumer functions yet.
> > > > > > > > >
> > > > > > > >
> > > > > > > > So pom -> build model is kept, build model -> produced pom is
> > > > broken?
> > > > > > Is
> > > > > > > it
> > > > > > > > the complete status?
> > > > > > > > Sounds ok for a 4.0 and a 4.1 can fix it if so.
> > > > > > > > Just want to ensure first part is not broken at all.
> > > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > Robert
> > > > > > > > > On 3-1-2021 16:38:38, Romain Manni-Bucau wrote:
> > > > > > > > > Le dim. 3 janv. 2021 à 16:18, Robert Scholte a
> > > > > > > > > écrit :
> > > > > > > > >
> > > > > > > > > > > So what I was expecting was: raw xml model -> converted
> > to
> > > > > > unified
> > > > > > > > > > consumed model -> extensions -> model processing.
> > > > > > > > > >
> > > > > > > > > > This is only the build pom part. You're missing the
> consume
> > > > part,
> > > > > > > where
> > > > > > > > > > the xml is distributed.
> > > > > > > > > >
> > > > > > > > >
> > > > > > > > > Yes but with previous chain the consume part is
> > > "clean/normalize
> > > > ->
> > > > > > > dump"
> > > > > > > > > since we are using consumed model - only standard model -
> in
> > > > memory
> > > > > > > > > already.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > > Build is raw + enrich, consumer is raw + enrich + reduce
> > > > > (removing
> > > > > > > > > > relativePath and modules are the first examples, but much
> > > more
> > > > is
> > > > > > > > > possible)
> > > > > > > > > >
> > > > > > > > > > Going for the in memory was also my first thought, but I
> > > would
> > > > > > loose
> > > > > > > > > > information, hence I came up with the current
> > implementation.
> > > > > > > > > >
> > > > > > > > >
> > > > > > > > > I don't see what you loose ot be honest.
> > > > > > > > > You mentionned license but this one is in the pom so not a
> > big
> > > > > deal,
> > > > > > > > > comments which are undesired IMHO as mentionned and element
> > > order
> > > > > > which
> > > > > > > > can
> > > > > > > > > really be discussed since we can desire to enforce an order
> > to
> > > > > > > normalize
> > > > > > > > > consumption + it shouldn't be important since from the
> > project
> > > > > point
> > > > > > of
> > > > > > > > > view your pom is already "broken"/lost (as all your
> > > intelligence
> > > > is
> > > > > > > lost
> > > > > > > > by
> > > > > > > > > this "not passthrough" process).
> > > > > > > > > So overall I don't see what you would loose from the
> consumer
> > > > side
> > > > > > but
> > > > > > > I
> > > > > > > > > see what you lost from maven ecosystem side.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > > Again, we're at a point where we can have counter
> > solutions,
> > > > but
> > > > > > > don't
> > > > > > > > > > expect me to implement it.
> > > > > > > > > >
> > > > > > > > >
> > > > > > > > > For now I'm just trying to ensure we agree we don't want to
> > > break
> > > > > > > > existing
> > > > > > > > > extensions and the nice ecosystem we built after years.
> > > > > > > > > This was really a move forward and it sounds like we broke
> it
> > > at
> > > > > > maven
> > > > > > > 4
> > > > > > > > > without any user gain which sounds terrible.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > thanks,
> > > > > > > > > > Robert
> > > > > > > > > > On 3-1-2021 15:25:21, Romain Manni-Bucau wrote:
> > > > > > > > > > Hi all,
> > > > > > > > > >
> > > > > > > > > > I kind of join Matthieu thoughts there, there is no point
> > to
> > > > work
> > > > > > at
> > > > > > > > xml
> > > > > > > > > > level to create the consumed pom - comments is not a
> point
> > > > since
> > > > > it
> > > > > > > can
> > > > > > > > > > commonly/easily refer to a dropped part of the pom so
> they
> > > > should
> > > > > > be
> > > > > > > > > > stripped.
> > > > > > > > > > Current extension model got proven adapted and adopted,
> > > using a
> > > > > > lower
> > > > > > > > > level
> > > > > > > > > > extension API will not since XML is, even if still
> > > mainstream,
> > > > > > often
> > > > > > > > > > replaced by alternative configurations and to have done
> the
> > > > work
> > > > > to
> > > > > > > > > inject
> > > > > > > > > > XML configuration programmatically compred to current
> > option,
> > > > it
> > > > > > is a
> > > > > > > > > pain.
> > > > > > > > > > The in memory model should stick to consumed model IMHO -
> > > being
> > > > > > > > > > programmatic there is no point to make it easier, worse
> > case
> > > we
> > > > > can
> > > > > > > add
> > > > > > > > > > helper beans (injectable) but in terms of model it will
> not
> > > > help.
> > > > > > > > > >
> > > > > > > > > > So what I was expecting was:
> > > > > > > > > >
> > > > > > > > > > raw xml model -> converted to unified consumed model ->
> > > > > extensions
> > > > > > ->
> > > > > > > > > model
> > > > > > > > > > processing.
> > > > > > > > > >
> > > > > > > > > > Indeed, real chain adds a small processing over the first
> > > arrow
> > > > > > > (inject
> > > > > > > > > > versions for example) but nothing crazy and breaking this
> > > > overall
> > > > > > > flow
> > > > > > > > > > which stays user friendly.
> > > > > > > > > >
> > > > > > > > > > Strictly speaking the new model is just a built-in
> > extension
> > > > for
> > > > > me
> > > > > > > > which
> > > > > > > > > > is particular because it will enforce IDE to integrate a
> > new
> > > > > > format -
> > > > > > > > > > wheres polyglot extensions or others don't require static
> > > > > analyzis
> > > > > > by
> > > > > > > > > > themself not being "standard".
> > > > > > > > > >
> > > > > > > > > > That said, there is nothing crazy with current
> > > implementation,
> > > > it
> > > > > > > just
> > > > > > > > > > require to be updated to be able to take extension
> changes
> > > into
> > > > > > > > account.
> > > > > > > > > > This can be done by making the extension model 'spyable'
> > (ie
> > > > if a
> > > > > > > > > > dependency/plugin is added it will be reflected in the
> > final
> > > > > > written
> > > > > > > > > > pom.xml).
> > > > > > > > > > This sounds - instrumenting the extension model API or
> > doing
> > > a
> > > > > diff
> > > > > > > > after
> > > > > > > > > > extension phase - like a compromise and let people gets
> the
> > > > best
> > > > > of
> > > > > > > > both
> > > > > > > > > > worlds to me.
> > > > > > > > > >
> > > > > > > > > > Wdyt?
> > > > > > > > > >
> > > > > > > > > > Romain Manni-Bucau
> > > > > > > > > > @rmannibucau | Blog
> > > > > > > > > > | Old Blog
> > > > > > > > > > | Github |
> > > > > > > > > > LinkedIn | Book
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > Le dim. 3 janv. 2021 à 14:46, Robert Scholte a
> > > > > > > > > > écrit :
> > > > > > > > > >
> > > > > > > > > > > Hi Matthieu,
> > > > > > > > > > >
> > > > > > > > > > > As you understand, something had to be changed to move
> > > Maven
> > > > > > > forward.
> > > > > > > > > > > I've decided to pick up that challenge and came up with
> > the
> > > > > > current
> > > > > > > > > > > solution.
> > > > > > > > > > >
> > > > > > > > > > > My main concerns was that I wanted to keep the
> fileModel
> > as
> > > > > much
> > > > > > as
> > > > > > > > is.
> > > > > > > > > > > That includes the license, comments and element order.
> > > > > > > > > > > This information if not available in the memory model,
> > so I
> > > > > > needed
> > > > > > > > the
> > > > > > > > > > > original pom file.
> > > > > > > > > > > With that in mind, the usage of XMLFilters looks like
> the
> > > > most
> > > > > > > > > > appropriate
> > > > > > > > > > > solution.
> > > > > > > > > > >
> > > > > > > > > > > I am certain that XML is still the most used format, so
> > if
> > > we
> > > > > can
> > > > > > > > have
> > > > > > > > > > > improvements for those users, I'm already very happy.
> > > > > > > > > > >
> > > > > > > > > > > And yes, there are plugins that needs to be updated,
> but
> > > > doing
> > > > > > > > nothing
> > > > > > > > > is
> > > > > > > > > > > not an option anymore.
> > > > > > > > > > >
> > > > > > > > > > > There are more people that share their concerns, but it
> > > took
> > > > me
> > > > > > > > several
> > > > > > > > > > > years to reach this point.
> > > > > > > > > > > We now have something that seems to work, anybody who
> can
> > > > > improve
> > > > > > > or
> > > > > > > > > can
> > > > > > > > > > > come up with an alternative implementation can do so.
> > > > > > > > > > >
> > > > > > > > > > > thanks,
> > > > > > > > > > > Robert
> > > > > > > > > > > On 3-1-2021 12:55:41, Matthieu Brouillard wrote:
> > > > > > > > > > > Thanks Robert for the video link.
> > > > > > > > > > >
> > > > > > > > > > > I fully understand the rationales behind the separation
> > of
> > > > > > > > > > > build/consumer pom and the video provides some insights
> > on
> > > it
> > > > > and
> > > > > > > you
> > > > > > > > > > > explain the actual implementation to introduce this
> > change.
> > > > > > > > > > > Still I do not fully understand why it was decided to
> > work
> > > on
> > > > > top
> > > > > > > of
> > > > > > > > > XML
> > > > > > > > > > by
> > > > > > > > > > > filtering/enhancing it instead of working at the POM
> (in
> > > > > > > > > > > memory datamodel) level.
> > > > > > > > > > > With the current understanding I have, by doing this
> > choice
> > > > of
> > > > > > > > working
> > > > > > > > > at
> > > > > > > > > > > XML level, it looks like it was decided to bypass (if
> not
> > > > kill)
> > > > > > > core
> > > > > > > > > > > extensions that enhance the POM itself and not the
> > pom.xml
> > > ;
> > > > > > here I
> > > > > > > > can
> > > > > > > > > > > think of (but probably not limited to):
> > > > > > > > > > > - polyglot-maven: do not use XML but other format to
> > > describe
> > > > > the
> > > > > > > POM
> > > > > > > > > > > (yaml, json, kotlin, java, other XML formats, ...)
> > > > > > > > > > > - jgitver-maven-plugin (or forks like
> > > > > > > > maven-git-versioning-extension):
> > > > > > > > > > > dynamic computation of projects version based on git
> > > history
> > > > > > > > > > >
> > > > > > > > > > > With the introduction of core extensions, I thought it
> > was
> > > a
> > > > > move
> > > > > > > to
> > > > > > > > > open
> > > > > > > > > > > the internals and let externals contribute to the
> > > > capabilities
> > > > > of
> > > > > > > > > maven.
> > > > > > > > > > > With the move to a XML handling chain, I see it as a
> > > > > > > > > restriction/regress
> > > > > > > > > > in
> > > > > > > > > > > favor of core closed functionalities. An example of
> that
> > is
> > > > > what
> > > > > > is
> > > > > > > > > > > provided as CIFriendly stuff, IMO it could/should have
> > been
> > > > > > > provided
> > > > > > > > > by a
> > > > > > > > > > > plugin/extension but instead it is hard written in
> maven
> > > core
> > > > > and
> > > > > > > is
> > > > > > > > > not
> > > > > > > > > > > opened for external contribution (plugin/extension I
> > mean).
> > > > > > > > > > >
> > > > > > > > > > > Perhaps I am totally wrong but I think that maven core
> > > should
> > > > > > > define
> > > > > > > > > all
> > > > > > > > > > > its expectations at an API level so that
> > extensions/plugins
> > > > > could
> > > > > > > > hook
> > > > > > > > > at
> > > > > > > > > > > this API level. The default packaging of maven
> > could/should
> > > > > > provide
> > > > > > > > > > default
> > > > > > > > > > > implementations of those expectations (for example
> > reading
> > > a
> > > > > > > pom.xml
> > > > > > > > > file
> > > > > > > > > > > to a POM model, dumping a POM to a pom-4.0.0.xml,
> > > > > > > > > transforming/reducing a
> > > > > > > > > > > POM to POM-consumer, dumping POM-consumer to
> > > > > > > pom-consumer-5.0.0.xml,
> > > > > > > > > ...)
> > > > > > > > > > > and let extensions/plugins/default implementations work
> > > along
> > > > > the
> > > > > > > > build
> > > > > > > > > > > process with the API & POMs to provide different
> features
> > > and
> > > > > > > > > > capabilities.
> > > > > > > > > > >
> > > > > > > > > > > Matthieu
> > > > > > > > > > >
> > > > > > > > > > > On Thu, Dec 31, 2020 at 7:01 PM Robert Scholte wrote:
> > > > > > > > > > >
> > > > > > > > > > > > I've made a recording[1] about it, which hopefully
> > > answers
> > > > > most
> > > > > > > > > > > questions.
> > > > > > > > > > > >
> > > > > > > > > > > > Robert
> > > > > > > > > > > >
> > > > > > > > > > > > [1] https://youtu.be/KDAmlNKZJto
> > > > > > > > > > > >
> > > > > > > > > > > > On 31-12-2020 16:18:57, Matthieu Brouillard
> > > > > > > > > > > > wrote:
> > > > > > > > > > > > > Not exactly sure what work you mean
> > > > > > > > > > > > everything related to maven-xml:
> > > > > Build/ConsumerPomXMLFilterxxx,
> > > > > > > > > > > > Build/ConsumerModelSourcexxxx and the transformer
> > stuff.
> > > > > > > > > > > > Especially, when looking at classes like
> > > > > CiFriendlyXMLFilter, I
> > > > > > > > would
> > > > > > > > > > > have
> > > > > > > > > > > > thought that such things could have been done
> > elsewhere,
> > > > > > working
> > > > > > > on
> > > > > > > > > the
> > > > > > > > > > > > object model (not on the XML stuff) especially for
> the
> > > > > BuildPom
> > > > > > > > part.
> > > > > > > > > > > >
> > > > > > > > > > > > > however specifically the consumer POM integrates
> with
> > > so
> > > > > many
> > > > > > > > > > external
> > > > > > > > > > > > ecosystems
> > > > > > > > > > > > We're aligned here, this has to be stable and well
> > > defined
> > > > > by a
> > > > > > > > > schema.
> > > > > > > > > > > >
> > > > > > > > > > > > Matthieu
> > > > > > > > > > > >
> > > > > > > > > > > > On Thu, Dec 31, 2020 at 3:59 PM Bernd Eckenfels
> > > > > > > > > > > > wrote:
> > > > > > > > > > > >
> > > > > > > > > > > > > Hello,
> > > > > > > > > > > > >
> > > > > > > > > > > > > Not exactly sure what work you mean and I fully
> agree
> > > > that
> > > > > > > using
> > > > > > > > a
> > > > > > > > > > core
> > > > > > > > > > > > > model should still be the API for plugins and
> > > extensions
> > > > to
> > > > > > > work
> > > > > > > > > > with,
> > > > > > > > > > > > > however specifically the consumer POM integrates
> with
> > > so
> > > > > many
> > > > > > > > > > external
> > > > > > > > > > > > > ecosystems, I would expect it to be defined in
> terms
> > of
> > > > XML
> > > > > > > > Schema
> > > > > > > > > > with
> > > > > > > > > > > > > explicite semantic (and the inherent compatibility
> > with
> > > > > > exiting
> > > > > > > > > > POMs).
> > > > > > > > > > > > >
> > > > > > > > > > > > > Gruss
> > > > > > > > > > > > > Bernd
> > > > > > > > > > > > > --
> > > > > > > > > > > > > http://bernd.eckenfels.net
> > > > > > > > > > > > > ________________________________
> > > > > > > > > > > > > Von: Matthieu BROUILLARD
> > > > > > > > > > > > > Gesendet: Thursday, December 31, 2020 3:19:09 PM
> > > > > > > > > > > > > An: dev@maven.apache.org
> > > > > > > > > > > > > Betreff: maven 4.0.0 new XML stuff
> > > > > > > > > > > > >
> > > > > > > > > > > > > Hello all,
> > > > > > > > > > > > >
> > > > > > > > > > > > > regarding the active work occurring for maven
> 4.0.0 I
> > > > > noticed
> > > > > > > the
> > > > > > > > > > > > > introduction of a lot of new stuff around SAX
> > parsing &
> > > > > > > > filtering.
> > > > > > > > > > > > > I am wondering if that means that it was decided
> that
> > > the
> > > > > > input
> > > > > > > > > > format
> > > > > > > > > > > of
> > > > > > > > > > > > > maven projects will be XML forever meaning
> probably,
> > > > among
> > > > > > > > others,
> > > > > > > > > > the
> > > > > > > > > > > > end
> > > > > > > > > > > > > of polyglot extensions.
> > > > > > > > > > > > > Could you explain such a move (or point to
> > > > > > rationals/documents)
> > > > > > > > and
> > > > > > > > > > why
> > > > > > > > > > > > you
> > > > > > > > > > > > > did not leverage working on the in memory object
> > model
> > > > > > allowing
> > > > > > > > > > > > > extensions/plugins to contribute/hook in the chain
> of
> > > > > > building
> > > > > > > > the
> > > > > > > > > > > > BuildPOM
> > > > > > > > > > > > > & ConsumePOM? In the past I really thought that
> this
> > > move
> > > > > to
> > > > > > > > 'Build
> > > > > > > > > > vs
> > > > > > > > > > > > > Consumer' POM would make clear separations between
> > the
> > > > > input
> > > > > > > > format
> > > > > > > > > > of
> > > > > > > > > > > > > descriptors and the core system but I perhaps
> > > > misunderstood
> > > > > > > > things.
> > > > > > > > > > > > >
> > > > > > > > > > > > > Also, are there plans regarding the future of core
> > > > > > extensions?
> > > > > > > > > > > > > With core extensions it was possible to hook into
> the
> > > POM
> > > > > > model
> > > > > > > > > > loading
> > > > > > > > > > > > and
> > > > > > > > > > > > > do transformations to do dynamic changes but by
> > working
> > > > on
> > > > > > the
> > > > > > > > XML
> > > > > > > > > > > > directly
> > > > > > > > > > > > > I see a shift (if not red stop) in this
> > > > > > contribution/delegation
> > > > > > > > > > > > mechanism.
> > > > > > > > > > > > >
> > > > > > > > > > > > > Thanks for your time & answers.
> > > > > > > > > > > > >
> > > > > > > > > > > > > Matthieu Brouillard
> > > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: maven 4.0.0 new XML stuff

Posted by Robert Scholte <rf...@apache.org>.
Thank you, now we have something to work on.

polyglot should still work, but it looks like it is getting the wrong set of options. So that needs to be fixed. Please create an issue[1]
regarding jgitver and don't see an issue at first glance. It is using MavenXpp3Reader, which is low level, no inheritence.
Do you have a stacktrace for this?

thanks,
Robert
[1] https://issues.apache.org/jira/browse/MNG
On 5-1-2021 13:17:21, Matthieu Brouillard <ma...@brouillard.fr> wrote:
> You both keep talking about extensions, but I haven't touched that part.
> I suggest to test current Maven 4 first and to find cases where those
extensions don't work anymore.
I do not know the exact internals of polyglot but for jgitver, in order to
produce a pom-4.0.0.xml compliant file, either:
- delegates to flatten-maven-plugin
- uses an internal mojo
for the production/attachment of a modified pom.xml file dump from the POM.
So in both cases, there is somewhere in the code, some ugly code like:

File dumpedModel = dumpFromModel(model);
project.setPomFile(dumpedModel)

In jgitver it is called from a Mojo(1) which in the end dump and update the
model (2)

I really thought that with 4.0 and the consumer pom this would be obsolete.

For polyglot, I tried but it does not work with maven-4.0.0, see #224 (3)

1:
https://github.com/jgitver/jgitver-maven-plugin/blob/master/src/main/java/fr/brouillard/oss/jgitver/mojos/JGitverAttachModifiedPomsMojo.java
2:
https://github.com/jgitver/jgitver-maven-plugin/blob/master/src/main/java/fr/brouillard/oss/jgitver/JGitverUtils.java#L182
3: https://github.com/takari/polyglot-maven/issues/224


On Tue, Jan 5, 2021 at 12:52 PM Robert Scholte wrote:

> You both keep talking about extensions, but I haven't touched that part.
> I suggest to test current Maven 4 first and to find cases where those
> extensions don't work anymore.
> Next we can see if this makes sense or not and how to solve it.
>
> Robert
>
> On 5-1-2021 12:28:59, Matthieu Brouillard wrote:
> > Currently the model is mutable and this causes issues.
> For me it is not the root cause, it does not help for sure but IMO the
> problem is more that there are currently several sources of trust : the POM
> & the pom.xml.
>
> Personally (only my opinion) I see no other way for the future of maven
> than to go to a single source of trust: the POM (as in memory model).
> During the build process this POM could (and has to) become immutable for
> sure so that plugins cannot do ugly things anymore but there should be also
> clear loading and extension mechanisms/hooks before it becomes immutable.
>
> > or do you push to drop extensions support?
> I hope this is not the case.
>
> On Tue, Jan 5, 2021 at 11:58 AM Romain Manni-Bucau
> wrote:
>
> > Hmm, extensions define this kind of lifecycle, at least the
> afterModelRead
> > which is a hook before it could be immutable no?
> > So not sure how it changes the issue, or do you push to drop extensions
> > support?
> >
> > Romain Manni-Bucau
> > @rmannibucau | Blog
> > | Old Blog
> > | Github
> > https://github.com/rmannibucau> |
> > LinkedIn | Book
> >
> >
> https://www.packtpub.com/application-development/java-ee-8-high-performance
> > >
> >
> >
> > Le mar. 5 janv. 2021 à 11:54, Robert Scholte a
> > écrit :
> >
> > > Currently the model is mutable and this causes issues.
> > > Instead I would like to see that once the BuildPlan is finished, the
> > model
> > > becomes immutable.
> > >
> > >
> > > To give a concrete example: It must be possible for code generating
> goals
> > > to add dependencies.
> > > Now, when using modello readers/writers you often need to add a
> required
> > > dependency by hand.
> > > Ideally there will be a hook where a plugin can register additional
> > > dependencies (e.g. dom4j).
> > >
> > > This will make any postprocessing of the pom during build obsolete
> > >
> > > Robert
> > > On 5-1-2021 09:09:11, Matthieu Brouillard
> > wrote:
> > > > Can you give an example?
> > > Like Romain has shown: dynamically added dependencies or a version
> > > computation.
> > >
> > > > Most important is the support for CI-friendly versions
> > > But if extensions dynamically compute and set the versions in the POM
> > using
> > > the API, the changes will not be reflected.
> > > That's why today one has to use flatten-maven-plugin or any other
> > > plugin/task modifying/dumping the POM model to a pom.xml file and
> > > setting/attaching it the POM.
> > >
> > > If the produced artifacts (consumer pom) were computed from the POM
> > (object
> > > model), every change done by any extension would be part of it.
> > >
> > > > but this happens AFTER using the pom
> > > Not always from the pom.xml. I thought extensions were allowed to
> provide
> > > ModelLocator & ModelReader to both decide which file to use for a
> project
> > > and how to build the in memory POM model.
> > > So the truth should not be considered to be in the pom.xml but in the
> > POM.
> > >
> > >
> > > On Mon, Jan 4, 2021 at 10:00 PM Robert Scholte wrote:
> > >
> > > > answers are below.
> > > >
> > > > Robert
> > > >
> > > > On 4-1-2021 16:52:23, Matthieu Brouillard wrote:
> > > > @Robert nothing is broken atm, the changes for consumer/build are
> > > currently
> > > > behind your feature flag.
> > > > Robert Scholte:
> > > > It is active by default, so it is actually not hidden.
> > > >
> > > >
> > > >
> > > > But as I feared previously, and as Romain pointed it, by working at
> XML
> > > > level (and not at POM level) the produced consumer pom does not
> reflect
> > > > changes from extensions.
> > > > Robert Scholte:
> > > > Can you give an example?
> > > >
> > > >
> > > > I really thought that all the "consumer/build" stuff would make the
> > > > maven-flatten-plugin useless but it looks like it will not be the
> case
> > if
> > > > working at XML level.
> > > > Robert Scholte:
> > > > Like with most questions: it depends. Most important is the support
> for
> > > > CI-friendly versions. In this case you won't need the
> > > flatten-maven-plugin
> > > > anymore.
> > > > However, the plugin can rewrite much more, but this happens AFTER
> using
> > > > the pom.
> > > > That's something I don't like, because this POM was not used to build
> > the
> > > > project, but it was reassembled afterwards.
> > > > My idea is to provide hooks to parts of the pom that might be
> adjusted,
> > > > but this is something we can work on during the 4.x releases
> > > >
> > > > Did Romain and I miss the whole point of the "consumer/build"
> > > enhancements
> > > > or is it "just" because current implementation has not yet reached
> the
> > > > targets/outputs?
> > > >
> > > > On Mon, Jan 4, 2021 at 2:56 PM Romain Manni-Bucau
> > > > wrote:
> > > >
> > > > > Hmm, I don't get a few things of this IT:
> > > > >
> > > > > 1. the formatting seems not expected even if valid (the comments
> are
> > > > > finishing with the first tag for example "-->
> > > > > public class MyListener extends AbstractMavenLifecycleParticipant {
> > > > >
> > > > > @Override
> > > > > public void afterProjectsRead(final MavenSession session) throws
> > > > > MavenExecutionException {
> > > > > if (session.getCurrentProject() == null) {
> > > > > return;
> > > > > }
> > > > >
> > > > > session.getProjects().forEach(p -> {
> > > > > final Dependency dependency = new Dependency();
> > > > > dependency.setGroupId("junit");
> > > > > dependency.setArtifactId("junit");
> > > > > dependency.setVersion("3.8.1");
> > > > > p.getDependencies().add(dependency);
> > > > > });
> > > > > }
> > > > > }
> > > > >
> > > > >
> > > > > 2. If you run mvn (4 snapshot) dependency:tree you get this kind of
> > > > output:
> > > > >
> > > > > [INFO] -------------
> > > > > >-------------
> > > > > [INFO] Building Multi Chapter Simple Web Application Project
> > > > > 0.9-MNG6957-SNAPSHOT [6/6]
> > > > > [INFO] --------------------------------[ jar
> > > > > ]---------------------------------
> > > > > [INFO]
> > > > > [INFO] --- maven-dependency-plugin:3.1.2:tree (default-cli) @
> > > > simple-webapp
> > > > > ---
> > > > > [INFO]
> > > > org.sonatype.mavenbook.multi:simple-webapp:jar:0.9-MNG6957-SNAPSHOT
> > > > > [INFO] +-
> > > > >
> > > > >
> > > >
> > >
> >
> org.sonatype.mavenbook.multi:simple-weather:jar:0.9-MNG6957-SNAPSHOT:compile
> > > > > [INFO] \- junit:junit:jar:3.8.1:compile
> > > > >
> > > > > [INFO]
> > > > >
> > >
> ------------------------------------------------------------------------
> > > > >
> > > > > 3. run the build to have produced pom and cat the simple-webapp
> one:
> > > > >
> > > > >
> > > > > 4.0.0
> > > > >
> > > > > org.sonatype.mavenbook.multi
> > > > > simple-parent
> > > > > 0.9-MNG6957-SNAPSHOT
> > > > >
> > > > >
> > > > > simple-webapp
> > > > > Multi Chapter Simple Web Application Project
> > > > >
> > > > >
> > > > > org.sonatype.mavenbook.multi
> > > > > simple-weather
> > > > > 0.9-MNG6957-SNAPSHOT
> > > > >
> > > > >
> > > > >
> > > > > simple-webapp
> > > > >
> > > > >
> > > > >
> > > > > org.apache.maven.plugins
> > > > > maven-war-plugin
> > > > > 2.6
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > As you see the dependency is not there. I guess the expected outout
> > is:
> > > > >
> > > > >
> > > > >
> > > > > 4.0.0
> > > > > simple-webapp
> > > > > Multi Chapter Simple Web Application Project
> > > > > [description, scm, ..., all central requires sections but not build
> > > ones]
> > > > >
> > > > >
> > > > > org.sonatype.mavenbook.multi
> > > > > simple-weather
> > > > > 0.9-MNG6957-SNAPSHOT
> > > > >
> > > > >
> > > > > junit
> > > > > junit
> > > > > 3.8.1
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > Am I missing something?
> > > > >
> > > > > Romain Manni-Bucau
> > > > > @rmannibucau | Blog
> > > > > | Old Blog
> > > > > | Github
> > > > > https://github.com/rmannibucau> |
> > > > > LinkedIn | Book
> > > > >
> > > > >
> > > >
> > >
> >
> https://www.packtpub.com/application-development/java-ee-8-high-performance
> > > > > >
> > > > >
> > > > >
> > > > > Le lun. 4 janv. 2021 à 13:41, Robert Scholte a
> > > > > écrit :
> > > > >
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://github.com/apache/maven-integration-testing/tree/master/core-it-suite/src/test/resources/mng-6957-buildconsumer
> > > > > > is the most complete IT
> > > > > >
> > > > > > On 4-1-2021 12:59:51, Romain Manni-Bucau wrote:
> > > > > > Le lun. 4 janv. 2021 à 12:36, Robert Scholte a
> > > > > > écrit :
> > > > > >
> > > > > > > There's just one thing I want to say:
> > > > > > > I'm having trouble with the term "broken".
> > > > > > >
> > > > > >
> > > > > > Well, literally meant broken as decorelated from the user intent
> > and
> > > > > > extension model.
> > > > > > Anyway, didn't intend to blame but more identify the blockers
> for a
> > > GA
> > > > so
> > > > > > point was really that it seem that on the two sides only the
> > > producing
> > > > > one
> > > > > > is not yet ready since it keeps does not sanitize the model
> > > completely
> > > > > and
> > > > > > keeps build only data like comments, right? Also not yet clear
> for
> > me
> > > > if
> > > > > we
> > > > > > loose the extension enrichments there.
> > > > > >
> > > > > >
> > > > > > > If a Maven project could be built with M3.6.3, it can still be
> > > built
> > > > > with
> > > > > > > M4.
> > > > > > > If not, it is either regression (MNG-6957, MNG-7063) which must
> > be
> > > > > fixed,
> > > > > > > or it requires changes to a plugin for understandable reasons
> > > > > > > (maven-pgp-plugin)
> > > > > > > AFAIK an interesting extension like the maven-tiles has been
> > tested
> > > > and
> > > > > > > still works.
> > > > > > >
> > > > > >
> > > > > > Do you have this handy, is it in our test suite? I'd like to
> check
> > > the
> > > > > > produced pom matches the enriched model but happy to start from
> > > > something
> > > > > > already there.
> > > > > >
> > > > > >
> > > > > > >
> > > > > > > Robert
> > > > > > >
> > > > > > > On 3-1-2021 19:35:25, Romain Manni-Bucau wrote:
> > > > > > > Le dim. 3 janv. 2021 à 19:04, Robert Scholte a
> > > > > > > écrit :
> > > > > > >
> > > > > > > > I don't remember all those details anymore, because I hit
> those
> > > in
> > > > > the
> > > > > > > > beginning.
> > > > > > > > Trying things over and over again I decided that this is
> > probably
> > > > the
> > > > > > > most
> > > > > > > > successful approach.
> > > > > > > >
> > > > > > > >
> > > > > > > > What of the goals was to keep the pom.xml as is as much as
> > > > possible.
> > > > > > > > We can only decide for the specific Maven elements how to
> > handle
> > > > > them,
> > > > > > we
> > > > > > > > should not decide about comments and licenses.
> > > > > > > > BTW, the license issue was hard to solve. You cannot use it
> > from
> > > > the
> > > > > > > pom's
> > > > > > > > , because there might be multiple licenses.
> > > > > > > >
> > > > > > >
> > > > > > > I disagree, it is saner IMO to evolve to support that than
> doing
> > > > > > > anything else.
> > > > > > > Once again you keep things which don't make sense in a consumed
> > pom
> > > > in
> > > > > > > current impl so i'd say the sucess in a few cases breaks as
> much
> > > > cases
> > > > > so
> > > > > > > we need to revisit anyway IMHO.
> > > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > The current implementation is a solid way to ensure we're not
> > > > > breaking
> > > > > > > too
> > > > > > > > much, because Maven controls the XML filters.
> > > > > > > >
> > > > > > >
> > > > > > > Hmm, breaking extensions seems to break too much (I'm not
> > speaking
> > > of
> > > > > > other
> > > > > > > parts which breaks the ecosystem there but just that is
> > sufficient
> > > > IMHO
> > > > > > to
> > > > > > > say we must check back our solution).
> > > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > Also keep in mind, that I only want Maven to decide which
> > > > > modifications
> > > > > > > > are done.
> > > > > > > >
> > > > > > >
> > > > > > > For the consumed pom I agree but it is consistent with keeping
> > > > > everything
> > > > > > > working instead of breaking too, no?
> > > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > Current polyglot projects should still work, but they cannot
> > > > benefit
> > > > > > from
> > > > > > > > the build/consumer functions yet.
> > > > > > > >
> > > > > > >
> > > > > > > So pom -> build model is kept, build model -> produced pom is
> > > broken?
> > > > > Is
> > > > > > it
> > > > > > > the complete status?
> > > > > > > Sounds ok for a 4.0 and a 4.1 can fix it if so.
> > > > > > > Just want to ensure first part is not broken at all.
> > > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > Robert
> > > > > > > > On 3-1-2021 16:38:38, Romain Manni-Bucau wrote:
> > > > > > > > Le dim. 3 janv. 2021 à 16:18, Robert Scholte a
> > > > > > > > écrit :
> > > > > > > >
> > > > > > > > > > So what I was expecting was: raw xml model -> converted
> to
> > > > > unified
> > > > > > > > > consumed model -> extensions -> model processing.
> > > > > > > > >
> > > > > > > > > This is only the build pom part. You're missing the consume
> > > part,
> > > > > > where
> > > > > > > > > the xml is distributed.
> > > > > > > > >
> > > > > > > >
> > > > > > > > Yes but with previous chain the consume part is
> > "clean/normalize
> > > ->
> > > > > > dump"
> > > > > > > > since we are using consumed model - only standard model - in
> > > memory
> > > > > > > > already.
> > > > > > > >
> > > > > > > >
> > > > > > > > > Build is raw + enrich, consumer is raw + enrich + reduce
> > > > (removing
> > > > > > > > > relativePath and modules are the first examples, but much
> > more
> > > is
> > > > > > > > possible)
> > > > > > > > >
> > > > > > > > > Going for the in memory was also my first thought, but I
> > would
> > > > > loose
> > > > > > > > > information, hence I came up with the current
> implementation.
> > > > > > > > >
> > > > > > > >
> > > > > > > > I don't see what you loose ot be honest.
> > > > > > > > You mentionned license but this one is in the pom so not a
> big
> > > > deal,
> > > > > > > > comments which are undesired IMHO as mentionned and element
> > order
> > > > > which
> > > > > > > can
> > > > > > > > really be discussed since we can desire to enforce an order
> to
> > > > > > normalize
> > > > > > > > consumption + it shouldn't be important since from the
> project
> > > > point
> > > > > of
> > > > > > > > view your pom is already "broken"/lost (as all your
> > intelligence
> > > is
> > > > > > lost
> > > > > > > by
> > > > > > > > this "not passthrough" process).
> > > > > > > > So overall I don't see what you would loose from the consumer
> > > side
> > > > > but
> > > > > > I
> > > > > > > > see what you lost from maven ecosystem side.
> > > > > > > >
> > > > > > > >
> > > > > > > > > Again, we're at a point where we can have counter
> solutions,
> > > but
> > > > > > don't
> > > > > > > > > expect me to implement it.
> > > > > > > > >
> > > > > > > >
> > > > > > > > For now I'm just trying to ensure we agree we don't want to
> > break
> > > > > > > existing
> > > > > > > > extensions and the nice ecosystem we built after years.
> > > > > > > > This was really a move forward and it sounds like we broke it
> > at
> > > > > maven
> > > > > > 4
> > > > > > > > without any user gain which sounds terrible.
> > > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > thanks,
> > > > > > > > > Robert
> > > > > > > > > On 3-1-2021 15:25:21, Romain Manni-Bucau wrote:
> > > > > > > > > Hi all,
> > > > > > > > >
> > > > > > > > > I kind of join Matthieu thoughts there, there is no point
> to
> > > work
> > > > > at
> > > > > > > xml
> > > > > > > > > level to create the consumed pom - comments is not a point
> > > since
> > > > it
> > > > > > can
> > > > > > > > > commonly/easily refer to a dropped part of the pom so they
> > > should
> > > > > be
> > > > > > > > > stripped.
> > > > > > > > > Current extension model got proven adapted and adopted,
> > using a
> > > > > lower
> > > > > > > > level
> > > > > > > > > extension API will not since XML is, even if still
> > mainstream,
> > > > > often
> > > > > > > > > replaced by alternative configurations and to have done the
> > > work
> > > > to
> > > > > > > > inject
> > > > > > > > > XML configuration programmatically compred to current
> option,
> > > it
> > > > > is a
> > > > > > > > pain.
> > > > > > > > > The in memory model should stick to consumed model IMHO -
> > being
> > > > > > > > > programmatic there is no point to make it easier, worse
> case
> > we
> > > > can
> > > > > > add
> > > > > > > > > helper beans (injectable) but in terms of model it will not
> > > help.
> > > > > > > > >
> > > > > > > > > So what I was expecting was:
> > > > > > > > >
> > > > > > > > > raw xml model -> converted to unified consumed model ->
> > > > extensions
> > > > > ->
> > > > > > > > model
> > > > > > > > > processing.
> > > > > > > > >
> > > > > > > > > Indeed, real chain adds a small processing over the first
> > arrow
> > > > > > (inject
> > > > > > > > > versions for example) but nothing crazy and breaking this
> > > overall
> > > > > > flow
> > > > > > > > > which stays user friendly.
> > > > > > > > >
> > > > > > > > > Strictly speaking the new model is just a built-in
> extension
> > > for
> > > > me
> > > > > > > which
> > > > > > > > > is particular because it will enforce IDE to integrate a
> new
> > > > > format -
> > > > > > > > > wheres polyglot extensions or others don't require static
> > > > analyzis
> > > > > by
> > > > > > > > > themself not being "standard".
> > > > > > > > >
> > > > > > > > > That said, there is nothing crazy with current
> > implementation,
> > > it
> > > > > > just
> > > > > > > > > require to be updated to be able to take extension changes
> > into
> > > > > > > account.
> > > > > > > > > This can be done by making the extension model 'spyable'
> (ie
> > > if a
> > > > > > > > > dependency/plugin is added it will be reflected in the
> final
> > > > > written
> > > > > > > > > pom.xml).
> > > > > > > > > This sounds - instrumenting the extension model API or
> doing
> > a
> > > > diff
> > > > > > > after
> > > > > > > > > extension phase - like a compromise and let people gets the
> > > best
> > > > of
> > > > > > > both
> > > > > > > > > worlds to me.
> > > > > > > > >
> > > > > > > > > Wdyt?
> > > > > > > > >
> > > > > > > > > Romain Manni-Bucau
> > > > > > > > > @rmannibucau | Blog
> > > > > > > > > | Old Blog
> > > > > > > > > | Github |
> > > > > > > > > LinkedIn | Book
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > Le dim. 3 janv. 2021 à 14:46, Robert Scholte a
> > > > > > > > > écrit :
> > > > > > > > >
> > > > > > > > > > Hi Matthieu,
> > > > > > > > > >
> > > > > > > > > > As you understand, something had to be changed to move
> > Maven
> > > > > > forward.
> > > > > > > > > > I've decided to pick up that challenge and came up with
> the
> > > > > current
> > > > > > > > > > solution.
> > > > > > > > > >
> > > > > > > > > > My main concerns was that I wanted to keep the fileModel
> as
> > > > much
> > > > > as
> > > > > > > is.
> > > > > > > > > > That includes the license, comments and element order.
> > > > > > > > > > This information if not available in the memory model,
> so I
> > > > > needed
> > > > > > > the
> > > > > > > > > > original pom file.
> > > > > > > > > > With that in mind, the usage of XMLFilters looks like the
> > > most
> > > > > > > > > appropriate
> > > > > > > > > > solution.
> > > > > > > > > >
> > > > > > > > > > I am certain that XML is still the most used format, so
> if
> > we
> > > > can
> > > > > > > have
> > > > > > > > > > improvements for those users, I'm already very happy.
> > > > > > > > > >
> > > > > > > > > > And yes, there are plugins that needs to be updated, but
> > > doing
> > > > > > > nothing
> > > > > > > > is
> > > > > > > > > > not an option anymore.
> > > > > > > > > >
> > > > > > > > > > There are more people that share their concerns, but it
> > took
> > > me
> > > > > > > several
> > > > > > > > > > years to reach this point.
> > > > > > > > > > We now have something that seems to work, anybody who can
> > > > improve
> > > > > > or
> > > > > > > > can
> > > > > > > > > > come up with an alternative implementation can do so.
> > > > > > > > > >
> > > > > > > > > > thanks,
> > > > > > > > > > Robert
> > > > > > > > > > On 3-1-2021 12:55:41, Matthieu Brouillard wrote:
> > > > > > > > > > Thanks Robert for the video link.
> > > > > > > > > >
> > > > > > > > > > I fully understand the rationales behind the separation
> of
> > > > > > > > > > build/consumer pom and the video provides some insights
> on
> > it
> > > > and
> > > > > > you
> > > > > > > > > > explain the actual implementation to introduce this
> change.
> > > > > > > > > > Still I do not fully understand why it was decided to
> work
> > on
> > > > top
> > > > > > of
> > > > > > > > XML
> > > > > > > > > by
> > > > > > > > > > filtering/enhancing it instead of working at the POM (in
> > > > > > > > > > memory datamodel) level.
> > > > > > > > > > With the current understanding I have, by doing this
> choice
> > > of
> > > > > > > working
> > > > > > > > at
> > > > > > > > > > XML level, it looks like it was decided to bypass (if not
> > > kill)
> > > > > > core
> > > > > > > > > > extensions that enhance the POM itself and not the
> pom.xml
> > ;
> > > > > here I
> > > > > > > can
> > > > > > > > > > think of (but probably not limited to):
> > > > > > > > > > - polyglot-maven: do not use XML but other format to
> > describe
> > > > the
> > > > > > POM
> > > > > > > > > > (yaml, json, kotlin, java, other XML formats, ...)
> > > > > > > > > > - jgitver-maven-plugin (or forks like
> > > > > > > maven-git-versioning-extension):
> > > > > > > > > > dynamic computation of projects version based on git
> > history
> > > > > > > > > >
> > > > > > > > > > With the introduction of core extensions, I thought it
> was
> > a
> > > > move
> > > > > > to
> > > > > > > > open
> > > > > > > > > > the internals and let externals contribute to the
> > > capabilities
> > > > of
> > > > > > > > maven.
> > > > > > > > > > With the move to a XML handling chain, I see it as a
> > > > > > > > restriction/regress
> > > > > > > > > in
> > > > > > > > > > favor of core closed functionalities. An example of that
> is
> > > > what
> > > > > is
> > > > > > > > > > provided as CIFriendly stuff, IMO it could/should have
> been
> > > > > > provided
> > > > > > > > by a
> > > > > > > > > > plugin/extension but instead it is hard written in maven
> > core
> > > > and
> > > > > > is
> > > > > > > > not
> > > > > > > > > > opened for external contribution (plugin/extension I
> mean).
> > > > > > > > > >
> > > > > > > > > > Perhaps I am totally wrong but I think that maven core
> > should
> > > > > > define
> > > > > > > > all
> > > > > > > > > > its expectations at an API level so that
> extensions/plugins
> > > > could
> > > > > > > hook
> > > > > > > > at
> > > > > > > > > > this API level. The default packaging of maven
> could/should
> > > > > provide
> > > > > > > > > default
> > > > > > > > > > implementations of those expectations (for example
> reading
> > a
> > > > > > pom.xml
> > > > > > > > file
> > > > > > > > > > to a POM model, dumping a POM to a pom-4.0.0.xml,
> > > > > > > > transforming/reducing a
> > > > > > > > > > POM to POM-consumer, dumping POM-consumer to
> > > > > > pom-consumer-5.0.0.xml,
> > > > > > > > ...)
> > > > > > > > > > and let extensions/plugins/default implementations work
> > along
> > > > the
> > > > > > > build
> > > > > > > > > > process with the API & POMs to provide different features
> > and
> > > > > > > > > capabilities.
> > > > > > > > > >
> > > > > > > > > > Matthieu
> > > > > > > > > >
> > > > > > > > > > On Thu, Dec 31, 2020 at 7:01 PM Robert Scholte wrote:
> > > > > > > > > >
> > > > > > > > > > > I've made a recording[1] about it, which hopefully
> > answers
> > > > most
> > > > > > > > > > questions.
> > > > > > > > > > >
> > > > > > > > > > > Robert
> > > > > > > > > > >
> > > > > > > > > > > [1] https://youtu.be/KDAmlNKZJto
> > > > > > > > > > >
> > > > > > > > > > > On 31-12-2020 16:18:57, Matthieu Brouillard
> > > > > > > > > > > wrote:
> > > > > > > > > > > > Not exactly sure what work you mean
> > > > > > > > > > > everything related to maven-xml:
> > > > Build/ConsumerPomXMLFilterxxx,
> > > > > > > > > > > Build/ConsumerModelSourcexxxx and the transformer
> stuff.
> > > > > > > > > > > Especially, when looking at classes like
> > > > CiFriendlyXMLFilter, I
> > > > > > > would
> > > > > > > > > > have
> > > > > > > > > > > thought that such things could have been done
> elsewhere,
> > > > > working
> > > > > > on
> > > > > > > > the
> > > > > > > > > > > object model (not on the XML stuff) especially for the
> > > > BuildPom
> > > > > > > part.
> > > > > > > > > > >
> > > > > > > > > > > > however specifically the consumer POM integrates with
> > so
> > > > many
> > > > > > > > > external
> > > > > > > > > > > ecosystems
> > > > > > > > > > > We're aligned here, this has to be stable and well
> > defined
> > > > by a
> > > > > > > > schema.
> > > > > > > > > > >
> > > > > > > > > > > Matthieu
> > > > > > > > > > >
> > > > > > > > > > > On Thu, Dec 31, 2020 at 3:59 PM Bernd Eckenfels
> > > > > > > > > > > wrote:
> > > > > > > > > > >
> > > > > > > > > > > > Hello,
> > > > > > > > > > > >
> > > > > > > > > > > > Not exactly sure what work you mean and I fully agree
> > > that
> > > > > > using
> > > > > > > a
> > > > > > > > > core
> > > > > > > > > > > > model should still be the API for plugins and
> > extensions
> > > to
> > > > > > work
> > > > > > > > > with,
> > > > > > > > > > > > however specifically the consumer POM integrates with
> > so
> > > > many
> > > > > > > > > external
> > > > > > > > > > > > ecosystems, I would expect it to be defined in terms
> of
> > > XML
> > > > > > > Schema
> > > > > > > > > with
> > > > > > > > > > > > explicite semantic (and the inherent compatibility
> with
> > > > > exiting
> > > > > > > > > POMs).
> > > > > > > > > > > >
> > > > > > > > > > > > Gruss
> > > > > > > > > > > > Bernd
> > > > > > > > > > > > --
> > > > > > > > > > > > http://bernd.eckenfels.net
> > > > > > > > > > > > ________________________________
> > > > > > > > > > > > Von: Matthieu BROUILLARD
> > > > > > > > > > > > Gesendet: Thursday, December 31, 2020 3:19:09 PM
> > > > > > > > > > > > An: dev@maven.apache.org
> > > > > > > > > > > > Betreff: maven 4.0.0 new XML stuff
> > > > > > > > > > > >
> > > > > > > > > > > > Hello all,
> > > > > > > > > > > >
> > > > > > > > > > > > regarding the active work occurring for maven 4.0.0 I
> > > > noticed
> > > > > > the
> > > > > > > > > > > > introduction of a lot of new stuff around SAX
> parsing &
> > > > > > > filtering.
> > > > > > > > > > > > I am wondering if that means that it was decided that
> > the
> > > > > input
> > > > > > > > > format
> > > > > > > > > > of
> > > > > > > > > > > > maven projects will be XML forever meaning probably,
> > > among
> > > > > > > others,
> > > > > > > > > the
> > > > > > > > > > > end
> > > > > > > > > > > > of polyglot extensions.
> > > > > > > > > > > > Could you explain such a move (or point to
> > > > > rationals/documents)
> > > > > > > and
> > > > > > > > > why
> > > > > > > > > > > you
> > > > > > > > > > > > did not leverage working on the in memory object
> model
> > > > > allowing
> > > > > > > > > > > > extensions/plugins to contribute/hook in the chain of
> > > > > building
> > > > > > > the
> > > > > > > > > > > BuildPOM
> > > > > > > > > > > > & ConsumePOM? In the past I really thought that this
> > move
> > > > to
> > > > > > > 'Build
> > > > > > > > > vs
> > > > > > > > > > > > Consumer' POM would make clear separations between
> the
> > > > input
> > > > > > > format
> > > > > > > > > of
> > > > > > > > > > > > descriptors and the core system but I perhaps
> > > misunderstood
> > > > > > > things.
> > > > > > > > > > > >
> > > > > > > > > > > > Also, are there plans regarding the future of core
> > > > > extensions?
> > > > > > > > > > > > With core extensions it was possible to hook into the
> > POM
> > > > > model
> > > > > > > > > loading
> > > > > > > > > > > and
> > > > > > > > > > > > do transformations to do dynamic changes but by
> working
> > > on
> > > > > the
> > > > > > > XML
> > > > > > > > > > > directly
> > > > > > > > > > > > I see a shift (if not red stop) in this
> > > > > contribution/delegation
> > > > > > > > > > > mechanism.
> > > > > > > > > > > >
> > > > > > > > > > > > Thanks for your time & answers.
> > > > > > > > > > > >
> > > > > > > > > > > > Matthieu Brouillard
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: maven 4.0.0 new XML stuff

Posted by Romain Manni-Bucau <rm...@gmail.com>.
Robert,
Point is that without using in memory pom model, produced pom will always
require rework to be properly consumed.
We must avoid it I think and do what you said: read pom -> extend it ->
make it immutable (optional for me) -> dump *this* model.
This is what makes a consumable feature otherwise it is just a static pom
rewriting without advantages of a producer/consumer pattern (nobody cares
to put versions to be concrete).


Le mar. 5 janv. 2021 à 13:17, Matthieu Brouillard <ma...@brouillard.fr>
a écrit :

> > You both keep talking about extensions, but I haven't touched that part.
> > I suggest to test current Maven 4 first and to find cases where those
> extensions don't work anymore.
> I do not know the exact internals of polyglot but for jgitver, in order to
> produce a pom-4.0.0.xml compliant file, either:
> - delegates to flatten-maven-plugin
> - uses an internal mojo
> for the production/attachment of a modified pom.xml file dump from the POM.
> So in both cases, there is somewhere in the code, some ugly code like:
>
> File dumpedModel = dumpFromModel(model);
> project.setPomFile(dumpedModel)
>
> In jgitver it is called from a Mojo(1) which in the end dump and update the
> model (2)
>
> I really thought that with 4.0 and the consumer pom this would be obsolete.
>
> For polyglot, I tried but it does not work with maven-4.0.0, see #224 (3)
>
> 1:
>
> https://github.com/jgitver/jgitver-maven-plugin/blob/master/src/main/java/fr/brouillard/oss/jgitver/mojos/JGitverAttachModifiedPomsMojo.java
> 2:
>
> https://github.com/jgitver/jgitver-maven-plugin/blob/master/src/main/java/fr/brouillard/oss/jgitver/JGitverUtils.java#L182
> 3: https://github.com/takari/polyglot-maven/issues/224
>
>
> On Tue, Jan 5, 2021 at 12:52 PM Robert Scholte <rf...@apache.org>
> wrote:
>
> > You both keep talking about extensions, but I haven't touched that part.
> > I suggest to test current Maven 4 first and to find cases where those
> > extensions don't work anymore.
> > Next we can see if this makes sense or not and how to solve it.
> >
> > Robert
> >
> > On 5-1-2021 12:28:59, Matthieu Brouillard <ma...@brouillard.fr>
> wrote:
> > > Currently the model is mutable and this causes issues.
> > For me it is not the root cause, it does not help for sure but IMO the
> > problem is more that there are currently several sources of trust : the
> POM
> > & the pom.xml.
> >
> > Personally (only my opinion) I see no other way for the future of maven
> > than to go to a single source of trust: the POM (as in memory model).
> > During the build process this POM could (and has to) become immutable for
> > sure so that plugins cannot do ugly things anymore but there should be
> also
> > clear loading and extension mechanisms/hooks before it becomes immutable.
> >
> > > or do you push to drop extensions support?
> > I hope this is not the case.
> >
> > On Tue, Jan 5, 2021 at 11:58 AM Romain Manni-Bucau
> > wrote:
> >
> > > Hmm, extensions define this kind of lifecycle, at least the
> > afterModelRead
> > > which is a hook before it could be immutable no?
> > > So not sure how it changes the issue, or do you push to drop extensions
> > > support?
> > >
> > > Romain Manni-Bucau
> > > @rmannibucau | Blog
> > > | Old Blog
> > > | Github <
> > > https://github.com/rmannibucau> |
> > > LinkedIn | Book
> > > <
> > >
> >
> https://www.packtpub.com/application-development/java-ee-8-high-performance
> > > >
> > >
> > >
> > > Le mar. 5 janv. 2021 à 11:54, Robert Scholte a
> > > écrit :
> > >
> > > > Currently the model is mutable and this causes issues.
> > > > Instead I would like to see that once the BuildPlan is finished, the
> > > model
> > > > becomes immutable.
> > > >
> > > >
> > > > To give a concrete example: It must be possible for code generating
> > goals
> > > > to add dependencies.
> > > > Now, when using modello readers/writers you often need to add a
> > required
> > > > dependency by hand.
> > > > Ideally there will be a hook where a plugin can register additional
> > > > dependencies (e.g. dom4j).
> > > >
> > > > This will make any postprocessing of the pom during build obsolete
> > > >
> > > > Robert
> > > > On 5-1-2021 09:09:11, Matthieu Brouillard
> > > wrote:
> > > > > Can you give an example?
> > > > Like Romain has shown: dynamically added dependencies or a version
> > > > computation.
> > > >
> > > > > Most important is the support for CI-friendly versions
> > > > But if extensions dynamically compute and set the versions in the POM
> > > using
> > > > the API, the changes will not be reflected.
> > > > That's why today one has to use flatten-maven-plugin or any other
> > > > plugin/task modifying/dumping the POM model to a pom.xml file and
> > > > setting/attaching it the POM.
> > > >
> > > > If the produced artifacts (consumer pom) were computed from the POM
> > > (object
> > > > model), every change done by any extension would be part of it.
> > > >
> > > > > but this happens AFTER using the pom
> > > > Not always from the pom.xml. I thought extensions were allowed to
> > provide
> > > > ModelLocator & ModelReader to both decide which file to use for a
> > project
> > > > and how to build the in memory POM model.
> > > > So the truth should not be considered to be in the pom.xml but in the
> > > POM.
> > > >
> > > >
> > > > On Mon, Jan 4, 2021 at 10:00 PM Robert Scholte wrote:
> > > >
> > > > > answers are below.
> > > > >
> > > > > Robert
> > > > >
> > > > > On 4-1-2021 16:52:23, Matthieu Brouillard wrote:
> > > > > @Robert nothing is broken atm, the changes for consumer/build are
> > > > currently
> > > > > behind your feature flag.
> > > > > Robert Scholte:
> > > > > It is active by default, so it is actually not hidden.
> > > > >
> > > > >
> > > > >
> > > > > But as I feared previously, and as Romain pointed it, by working at
> > XML
> > > > > level (and not at POM level) the produced consumer pom does not
> > reflect
> > > > > changes from extensions.
> > > > > Robert Scholte:
> > > > > Can you give an example?
> > > > >
> > > > >
> > > > > I really thought that all the "consumer/build" stuff would make the
> > > > > maven-flatten-plugin useless but it looks like it will not be the
> > case
> > > if
> > > > > working at XML level.
> > > > > Robert Scholte:
> > > > > Like with most questions: it depends. Most important is the support
> > for
> > > > > CI-friendly versions. In this case you won't need the
> > > > flatten-maven-plugin
> > > > > anymore.
> > > > > However, the plugin can rewrite much more, but this happens AFTER
> > using
> > > > > the pom.
> > > > > That's something I don't like, because this POM was not used to
> build
> > > the
> > > > > project, but it was reassembled afterwards.
> > > > > My idea is to provide hooks to parts of the pom that might be
> > adjusted,
> > > > > but this is something we can work on during the 4.x releases
> > > > >
> > > > > Did Romain and I miss the whole point of the "consumer/build"
> > > > enhancements
> > > > > or is it "just" because current implementation has not yet reached
> > the
> > > > > targets/outputs?
> > > > >
> > > > > On Mon, Jan 4, 2021 at 2:56 PM Romain Manni-Bucau
> > > > > wrote:
> > > > >
> > > > > > Hmm, I don't get a few things of this IT:
> > > > > >
> > > > > > 1. the formatting seems not expected even if valid (the comments
> > are
> > > > > > finishing with the first tag for example "-->
> > > > > > public class MyListener extends
> AbstractMavenLifecycleParticipant {
> > > > > >
> > > > > > @Override
> > > > > > public void afterProjectsRead(final MavenSession session) throws
> > > > > > MavenExecutionException {
> > > > > > if (session.getCurrentProject() == null) {
> > > > > > return;
> > > > > > }
> > > > > >
> > > > > > session.getProjects().forEach(p -> {
> > > > > > final Dependency dependency = new Dependency();
> > > > > > dependency.setGroupId("junit");
> > > > > > dependency.setArtifactId("junit");
> > > > > > dependency.setVersion("3.8.1");
> > > > > > p.getDependencies().add(dependency);
> > > > > > });
> > > > > > }
> > > > > > }
> > > > > >
> > > > > >
> > > > > > 2. If you run mvn (4 snapshot) dependency:tree you get this kind
> of
> > > > > output:
> > > > > >
> > > > > > [INFO] -------------
> > > > > > >-------------
> > > > > > [INFO] Building Multi Chapter Simple Web Application Project
> > > > > > 0.9-MNG6957-SNAPSHOT [6/6]
> > > > > > [INFO] --------------------------------[ jar
> > > > > > ]---------------------------------
> > > > > > [INFO]
> > > > > > [INFO] --- maven-dependency-plugin:3.1.2:tree (default-cli) @
> > > > > simple-webapp
> > > > > > ---
> > > > > > [INFO]
> > > > > org.sonatype.mavenbook.multi:simple-webapp:jar:0.9-MNG6957-SNAPSHOT
> > > > > > [INFO] +-
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> org.sonatype.mavenbook.multi:simple-weather:jar:0.9-MNG6957-SNAPSHOT:compile
> > > > > > [INFO] \- junit:junit:jar:3.8.1:compile
> > > > > >
> > > > > > [INFO]
> > > > > >
> > > >
> > ------------------------------------------------------------------------
> > > > > >
> > > > > > 3. run the build to have produced pom and cat the simple-webapp
> > one:
> > > > > >
> > > > > >
> > > > > > 4.0.0
> > > > > >
> > > > > > org.sonatype.mavenbook.multi
> > > > > > simple-parent
> > > > > > 0.9-MNG6957-SNAPSHOT
> > > > > >
> > > > > >
> > > > > > simple-webapp
> > > > > > Multi Chapter Simple Web Application Project
> > > > > >
> > > > > >
> > > > > > org.sonatype.mavenbook.multi
> > > > > > simple-weather
> > > > > > 0.9-MNG6957-SNAPSHOT
> > > > > >
> > > > > >
> > > > > >
> > > > > > simple-webapp
> > > > > >
> > > > > >
> > > > > >
> > > > > > org.apache.maven.plugins
> > > > > > maven-war-plugin
> > > > > > 2.6
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > As you see the dependency is not there. I guess the expected
> outout
> > > is:
> > > > > >
> > > > > >
> > > > > >
> > > > > > 4.0.0
> > > > > > simple-webapp
> > > > > > Multi Chapter Simple Web Application Project
> > > > > > [description, scm, ..., all central requires sections but not
> build
> > > > ones]
> > > > > >
> > > > > >
> > > > > > org.sonatype.mavenbook.multi
> > > > > > simple-weather
> > > > > > 0.9-MNG6957-SNAPSHOT
> > > > > >
> > > > > >
> > > > > > junit
> > > > > > junit
> > > > > > 3.8.1
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > Am I missing something?
> > > > > >
> > > > > > Romain Manni-Bucau
> > > > > > @rmannibucau | Blog
> > > > > > | Old Blog
> > > > > > | Github
> > > > > > https://github.com/rmannibucau> |
> > > > > > LinkedIn | Book
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://www.packtpub.com/application-development/java-ee-8-high-performance
> > > > > > >
> > > > > >
> > > > > >
> > > > > > Le lun. 4 janv. 2021 à 13:41, Robert Scholte a
> > > > > > écrit :
> > > > > >
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://github.com/apache/maven-integration-testing/tree/master/core-it-suite/src/test/resources/mng-6957-buildconsumer
> > > > > > > is the most complete IT
> > > > > > >
> > > > > > > On 4-1-2021 12:59:51, Romain Manni-Bucau wrote:
> > > > > > > Le lun. 4 janv. 2021 à 12:36, Robert Scholte a
> > > > > > > écrit :
> > > > > > >
> > > > > > > > There's just one thing I want to say:
> > > > > > > > I'm having trouble with the term "broken".
> > > > > > > >
> > > > > > >
> > > > > > > Well, literally meant broken as decorelated from the user
> intent
> > > and
> > > > > > > extension model.
> > > > > > > Anyway, didn't intend to blame but more identify the blockers
> > for a
> > > > GA
> > > > > so
> > > > > > > point was really that it seem that on the two sides only the
> > > > producing
> > > > > > one
> > > > > > > is not yet ready since it keeps does not sanitize the model
> > > > completely
> > > > > > and
> > > > > > > keeps build only data like comments, right? Also not yet clear
> > for
> > > me
> > > > > if
> > > > > > we
> > > > > > > loose the extension enrichments there.
> > > > > > >
> > > > > > >
> > > > > > > > If a Maven project could be built with M3.6.3, it can still
> be
> > > > built
> > > > > > with
> > > > > > > > M4.
> > > > > > > > If not, it is either regression (MNG-6957, MNG-7063) which
> must
> > > be
> > > > > > fixed,
> > > > > > > > or it requires changes to a plugin for understandable reasons
> > > > > > > > (maven-pgp-plugin)
> > > > > > > > AFAIK an interesting extension like the maven-tiles has been
> > > tested
> > > > > and
> > > > > > > > still works.
> > > > > > > >
> > > > > > >
> > > > > > > Do you have this handy, is it in our test suite? I'd like to
> > check
> > > > the
> > > > > > > produced pom matches the enriched model but happy to start from
> > > > > something
> > > > > > > already there.
> > > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > Robert
> > > > > > > >
> > > > > > > > On 3-1-2021 19:35:25, Romain Manni-Bucau wrote:
> > > > > > > > Le dim. 3 janv. 2021 à 19:04, Robert Scholte a
> > > > > > > > écrit :
> > > > > > > >
> > > > > > > > > I don't remember all those details anymore, because I hit
> > those
> > > > in
> > > > > > the
> > > > > > > > > beginning.
> > > > > > > > > Trying things over and over again I decided that this is
> > > probably
> > > > > the
> > > > > > > > most
> > > > > > > > > successful approach.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > What of the goals was to keep the pom.xml as is as much as
> > > > > possible.
> > > > > > > > > We can only decide for the specific Maven elements how to
> > > handle
> > > > > > them,
> > > > > > > we
> > > > > > > > > should not decide about comments and licenses.
> > > > > > > > > BTW, the license issue was hard to solve. You cannot use it
> > > from
> > > > > the
> > > > > > > > pom's
> > > > > > > > > , because there might be multiple licenses.
> > > > > > > > >
> > > > > > > >
> > > > > > > > I disagree, it is saner IMO to evolve to support that than
> > doing
> > > > > > > > anything else.
> > > > > > > > Once again you keep things which don't make sense in a
> consumed
> > > pom
> > > > > in
> > > > > > > > current impl so i'd say the sucess in a few cases breaks as
> > much
> > > > > cases
> > > > > > so
> > > > > > > > we need to revisit anyway IMHO.
> > > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > The current implementation is a solid way to ensure we're
> not
> > > > > > breaking
> > > > > > > > too
> > > > > > > > > much, because Maven controls the XML filters.
> > > > > > > > >
> > > > > > > >
> > > > > > > > Hmm, breaking extensions seems to break too much (I'm not
> > > speaking
> > > > of
> > > > > > > other
> > > > > > > > parts which breaks the ecosystem there but just that is
> > > sufficient
> > > > > IMHO
> > > > > > > to
> > > > > > > > say we must check back our solution).
> > > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > Also keep in mind, that I only want Maven to decide which
> > > > > > modifications
> > > > > > > > > are done.
> > > > > > > > >
> > > > > > > >
> > > > > > > > For the consumed pom I agree but it is consistent with
> keeping
> > > > > > everything
> > > > > > > > working instead of breaking too, no?
> > > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > Current polyglot projects should still work, but they
> cannot
> > > > > benefit
> > > > > > > from
> > > > > > > > > the build/consumer functions yet.
> > > > > > > > >
> > > > > > > >
> > > > > > > > So pom -> build model is kept, build model -> produced pom is
> > > > broken?
> > > > > > Is
> > > > > > > it
> > > > > > > > the complete status?
> > > > > > > > Sounds ok for a 4.0 and a 4.1 can fix it if so.
> > > > > > > > Just want to ensure first part is not broken at all.
> > > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > Robert
> > > > > > > > > On 3-1-2021 16:38:38, Romain Manni-Bucau wrote:
> > > > > > > > > Le dim. 3 janv. 2021 à 16:18, Robert Scholte a
> > > > > > > > > écrit :
> > > > > > > > >
> > > > > > > > > > > So what I was expecting was: raw xml model -> converted
> > to
> > > > > > unified
> > > > > > > > > > consumed model -> extensions -> model processing.
> > > > > > > > > >
> > > > > > > > > > This is only the build pom part. You're missing the
> consume
> > > > part,
> > > > > > > where
> > > > > > > > > > the xml is distributed.
> > > > > > > > > >
> > > > > > > > >
> > > > > > > > > Yes but with previous chain the consume part is
> > > "clean/normalize
> > > > ->
> > > > > > > dump"
> > > > > > > > > since we are using consumed model - only standard model -
> in
> > > > memory
> > > > > > > > > already.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > > Build is raw + enrich, consumer is raw + enrich + reduce
> > > > > (removing
> > > > > > > > > > relativePath and modules are the first examples, but much
> > > more
> > > > is
> > > > > > > > > possible)
> > > > > > > > > >
> > > > > > > > > > Going for the in memory was also my first thought, but I
> > > would
> > > > > > loose
> > > > > > > > > > information, hence I came up with the current
> > implementation.
> > > > > > > > > >
> > > > > > > > >
> > > > > > > > > I don't see what you loose ot be honest.
> > > > > > > > > You mentionned license but this one is in the pom so not a
> > big
> > > > > deal,
> > > > > > > > > comments which are undesired IMHO as mentionned and element
> > > order
> > > > > > which
> > > > > > > > can
> > > > > > > > > really be discussed since we can desire to enforce an order
> > to
> > > > > > > normalize
> > > > > > > > > consumption + it shouldn't be important since from the
> > project
> > > > > point
> > > > > > of
> > > > > > > > > view your pom is already "broken"/lost (as all your
> > > intelligence
> > > > is
> > > > > > > lost
> > > > > > > > by
> > > > > > > > > this "not passthrough" process).
> > > > > > > > > So overall I don't see what you would loose from the
> consumer
> > > > side
> > > > > > but
> > > > > > > I
> > > > > > > > > see what you lost from maven ecosystem side.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > > Again, we're at a point where we can have counter
> > solutions,
> > > > but
> > > > > > > don't
> > > > > > > > > > expect me to implement it.
> > > > > > > > > >
> > > > > > > > >
> > > > > > > > > For now I'm just trying to ensure we agree we don't want to
> > > break
> > > > > > > > existing
> > > > > > > > > extensions and the nice ecosystem we built after years.
> > > > > > > > > This was really a move forward and it sounds like we broke
> it
> > > at
> > > > > > maven
> > > > > > > 4
> > > > > > > > > without any user gain which sounds terrible.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > thanks,
> > > > > > > > > > Robert
> > > > > > > > > > On 3-1-2021 15:25:21, Romain Manni-Bucau wrote:
> > > > > > > > > > Hi all,
> > > > > > > > > >
> > > > > > > > > > I kind of join Matthieu thoughts there, there is no point
> > to
> > > > work
> > > > > > at
> > > > > > > > xml
> > > > > > > > > > level to create the consumed pom - comments is not a
> point
> > > > since
> > > > > it
> > > > > > > can
> > > > > > > > > > commonly/easily refer to a dropped part of the pom so
> they
> > > > should
> > > > > > be
> > > > > > > > > > stripped.
> > > > > > > > > > Current extension model got proven adapted and adopted,
> > > using a
> > > > > > lower
> > > > > > > > > level
> > > > > > > > > > extension API will not since XML is, even if still
> > > mainstream,
> > > > > > often
> > > > > > > > > > replaced by alternative configurations and to have done
> the
> > > > work
> > > > > to
> > > > > > > > > inject
> > > > > > > > > > XML configuration programmatically compred to current
> > option,
> > > > it
> > > > > > is a
> > > > > > > > > pain.
> > > > > > > > > > The in memory model should stick to consumed model IMHO -
> > > being
> > > > > > > > > > programmatic there is no point to make it easier, worse
> > case
> > > we
> > > > > can
> > > > > > > add
> > > > > > > > > > helper beans (injectable) but in terms of model it will
> not
> > > > help.
> > > > > > > > > >
> > > > > > > > > > So what I was expecting was:
> > > > > > > > > >
> > > > > > > > > > raw xml model -> converted to unified consumed model ->
> > > > > extensions
> > > > > > ->
> > > > > > > > > model
> > > > > > > > > > processing.
> > > > > > > > > >
> > > > > > > > > > Indeed, real chain adds a small processing over the first
> > > arrow
> > > > > > > (inject
> > > > > > > > > > versions for example) but nothing crazy and breaking this
> > > > overall
> > > > > > > flow
> > > > > > > > > > which stays user friendly.
> > > > > > > > > >
> > > > > > > > > > Strictly speaking the new model is just a built-in
> > extension
> > > > for
> > > > > me
> > > > > > > > which
> > > > > > > > > > is particular because it will enforce IDE to integrate a
> > new
> > > > > > format -
> > > > > > > > > > wheres polyglot extensions or others don't require static
> > > > > analyzis
> > > > > > by
> > > > > > > > > > themself not being "standard".
> > > > > > > > > >
> > > > > > > > > > That said, there is nothing crazy with current
> > > implementation,
> > > > it
> > > > > > > just
> > > > > > > > > > require to be updated to be able to take extension
> changes
> > > into
> > > > > > > > account.
> > > > > > > > > > This can be done by making the extension model 'spyable'
> > (ie
> > > > if a
> > > > > > > > > > dependency/plugin is added it will be reflected in the
> > final
> > > > > > written
> > > > > > > > > > pom.xml).
> > > > > > > > > > This sounds - instrumenting the extension model API or
> > doing
> > > a
> > > > > diff
> > > > > > > > after
> > > > > > > > > > extension phase - like a compromise and let people gets
> the
> > > > best
> > > > > of
> > > > > > > > both
> > > > > > > > > > worlds to me.
> > > > > > > > > >
> > > > > > > > > > Wdyt?
> > > > > > > > > >
> > > > > > > > > > Romain Manni-Bucau
> > > > > > > > > > @rmannibucau | Blog
> > > > > > > > > > | Old Blog
> > > > > > > > > > | Github |
> > > > > > > > > > LinkedIn | Book
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > Le dim. 3 janv. 2021 à 14:46, Robert Scholte a
> > > > > > > > > > écrit :
> > > > > > > > > >
> > > > > > > > > > > Hi Matthieu,
> > > > > > > > > > >
> > > > > > > > > > > As you understand, something had to be changed to move
> > > Maven
> > > > > > > forward.
> > > > > > > > > > > I've decided to pick up that challenge and came up with
> > the
> > > > > > current
> > > > > > > > > > > solution.
> > > > > > > > > > >
> > > > > > > > > > > My main concerns was that I wanted to keep the
> fileModel
> > as
> > > > > much
> > > > > > as
> > > > > > > > is.
> > > > > > > > > > > That includes the license, comments and element order.
> > > > > > > > > > > This information if not available in the memory model,
> > so I
> > > > > > needed
> > > > > > > > the
> > > > > > > > > > > original pom file.
> > > > > > > > > > > With that in mind, the usage of XMLFilters looks like
> the
> > > > most
> > > > > > > > > > appropriate
> > > > > > > > > > > solution.
> > > > > > > > > > >
> > > > > > > > > > > I am certain that XML is still the most used format, so
> > if
> > > we
> > > > > can
> > > > > > > > have
> > > > > > > > > > > improvements for those users, I'm already very happy.
> > > > > > > > > > >
> > > > > > > > > > > And yes, there are plugins that needs to be updated,
> but
> > > > doing
> > > > > > > > nothing
> > > > > > > > > is
> > > > > > > > > > > not an option anymore.
> > > > > > > > > > >
> > > > > > > > > > > There are more people that share their concerns, but it
> > > took
> > > > me
> > > > > > > > several
> > > > > > > > > > > years to reach this point.
> > > > > > > > > > > We now have something that seems to work, anybody who
> can
> > > > > improve
> > > > > > > or
> > > > > > > > > can
> > > > > > > > > > > come up with an alternative implementation can do so.
> > > > > > > > > > >
> > > > > > > > > > > thanks,
> > > > > > > > > > > Robert
> > > > > > > > > > > On 3-1-2021 12:55:41, Matthieu Brouillard wrote:
> > > > > > > > > > > Thanks Robert for the video link.
> > > > > > > > > > >
> > > > > > > > > > > I fully understand the rationales behind the separation
> > of
> > > > > > > > > > > build/consumer pom and the video provides some insights
> > on
> > > it
> > > > > and
> > > > > > > you
> > > > > > > > > > > explain the actual implementation to introduce this
> > change.
> > > > > > > > > > > Still I do not fully understand why it was decided to
> > work
> > > on
> > > > > top
> > > > > > > of
> > > > > > > > > XML
> > > > > > > > > > by
> > > > > > > > > > > filtering/enhancing it instead of working at the POM
> (in
> > > > > > > > > > > memory datamodel) level.
> > > > > > > > > > > With the current understanding I have, by doing this
> > choice
> > > > of
> > > > > > > > working
> > > > > > > > > at
> > > > > > > > > > > XML level, it looks like it was decided to bypass (if
> not
> > > > kill)
> > > > > > > core
> > > > > > > > > > > extensions that enhance the POM itself and not the
> > pom.xml
> > > ;
> > > > > > here I
> > > > > > > > can
> > > > > > > > > > > think of (but probably not limited to):
> > > > > > > > > > > - polyglot-maven: do not use XML but other format to
> > > describe
> > > > > the
> > > > > > > POM
> > > > > > > > > > > (yaml, json, kotlin, java, other XML formats, ...)
> > > > > > > > > > > - jgitver-maven-plugin (or forks like
> > > > > > > > maven-git-versioning-extension):
> > > > > > > > > > > dynamic computation of projects version based on git
> > > history
> > > > > > > > > > >
> > > > > > > > > > > With the introduction of core extensions, I thought it
> > was
> > > a
> > > > > move
> > > > > > > to
> > > > > > > > > open
> > > > > > > > > > > the internals and let externals contribute to the
> > > > capabilities
> > > > > of
> > > > > > > > > maven.
> > > > > > > > > > > With the move to a XML handling chain, I see it as a
> > > > > > > > > restriction/regress
> > > > > > > > > > in
> > > > > > > > > > > favor of core closed functionalities. An example of
> that
> > is
> > > > > what
> > > > > > is
> > > > > > > > > > > provided as CIFriendly stuff, IMO it could/should have
> > been
> > > > > > > provided
> > > > > > > > > by a
> > > > > > > > > > > plugin/extension but instead it is hard written in
> maven
> > > core
> > > > > and
> > > > > > > is
> > > > > > > > > not
> > > > > > > > > > > opened for external contribution (plugin/extension I
> > mean).
> > > > > > > > > > >
> > > > > > > > > > > Perhaps I am totally wrong but I think that maven core
> > > should
> > > > > > > define
> > > > > > > > > all
> > > > > > > > > > > its expectations at an API level so that
> > extensions/plugins
> > > > > could
> > > > > > > > hook
> > > > > > > > > at
> > > > > > > > > > > this API level. The default packaging of maven
> > could/should
> > > > > > provide
> > > > > > > > > > default
> > > > > > > > > > > implementations of those expectations (for example
> > reading
> > > a
> > > > > > > pom.xml
> > > > > > > > > file
> > > > > > > > > > > to a POM model, dumping a POM to a pom-4.0.0.xml,
> > > > > > > > > transforming/reducing a
> > > > > > > > > > > POM to POM-consumer, dumping POM-consumer to
> > > > > > > pom-consumer-5.0.0.xml,
> > > > > > > > > ...)
> > > > > > > > > > > and let extensions/plugins/default implementations work
> > > along
> > > > > the
> > > > > > > > build
> > > > > > > > > > > process with the API & POMs to provide different
> features
> > > and
> > > > > > > > > > capabilities.
> > > > > > > > > > >
> > > > > > > > > > > Matthieu
> > > > > > > > > > >
> > > > > > > > > > > On Thu, Dec 31, 2020 at 7:01 PM Robert Scholte wrote:
> > > > > > > > > > >
> > > > > > > > > > > > I've made a recording[1] about it, which hopefully
> > > answers
> > > > > most
> > > > > > > > > > > questions.
> > > > > > > > > > > >
> > > > > > > > > > > > Robert
> > > > > > > > > > > >
> > > > > > > > > > > > [1] https://youtu.be/KDAmlNKZJto
> > > > > > > > > > > >
> > > > > > > > > > > > On 31-12-2020 16:18:57, Matthieu Brouillard
> > > > > > > > > > > > wrote:
> > > > > > > > > > > > > Not exactly sure what work you mean
> > > > > > > > > > > > everything related to maven-xml:
> > > > > Build/ConsumerPomXMLFilterxxx,
> > > > > > > > > > > > Build/ConsumerModelSourcexxxx and the transformer
> > stuff.
> > > > > > > > > > > > Especially, when looking at classes like
> > > > > CiFriendlyXMLFilter, I
> > > > > > > > would
> > > > > > > > > > > have
> > > > > > > > > > > > thought that such things could have been done
> > elsewhere,
> > > > > > working
> > > > > > > on
> > > > > > > > > the
> > > > > > > > > > > > object model (not on the XML stuff) especially for
> the
> > > > > BuildPom
> > > > > > > > part.
> > > > > > > > > > > >
> > > > > > > > > > > > > however specifically the consumer POM integrates
> with
> > > so
> > > > > many
> > > > > > > > > > external
> > > > > > > > > > > > ecosystems
> > > > > > > > > > > > We're aligned here, this has to be stable and well
> > > defined
> > > > > by a
> > > > > > > > > schema.
> > > > > > > > > > > >
> > > > > > > > > > > > Matthieu
> > > > > > > > > > > >
> > > > > > > > > > > > On Thu, Dec 31, 2020 at 3:59 PM Bernd Eckenfels
> > > > > > > > > > > > wrote:
> > > > > > > > > > > >
> > > > > > > > > > > > > Hello,
> > > > > > > > > > > > >
> > > > > > > > > > > > > Not exactly sure what work you mean and I fully
> agree
> > > > that
> > > > > > > using
> > > > > > > > a
> > > > > > > > > > core
> > > > > > > > > > > > > model should still be the API for plugins and
> > > extensions
> > > > to
> > > > > > > work
> > > > > > > > > > with,
> > > > > > > > > > > > > however specifically the consumer POM integrates
> with
> > > so
> > > > > many
> > > > > > > > > > external
> > > > > > > > > > > > > ecosystems, I would expect it to be defined in
> terms
> > of
> > > > XML
> > > > > > > > Schema
> > > > > > > > > > with
> > > > > > > > > > > > > explicite semantic (and the inherent compatibility
> > with
> > > > > > exiting
> > > > > > > > > > POMs).
> > > > > > > > > > > > >
> > > > > > > > > > > > > Gruss
> > > > > > > > > > > > > Bernd
> > > > > > > > > > > > > --
> > > > > > > > > > > > > http://bernd.eckenfels.net
> > > > > > > > > > > > > ________________________________
> > > > > > > > > > > > > Von: Matthieu BROUILLARD
> > > > > > > > > > > > > Gesendet: Thursday, December 31, 2020 3:19:09 PM
> > > > > > > > > > > > > An: dev@maven.apache.org
> > > > > > > > > > > > > Betreff: maven 4.0.0 new XML stuff
> > > > > > > > > > > > >
> > > > > > > > > > > > > Hello all,
> > > > > > > > > > > > >
> > > > > > > > > > > > > regarding the active work occurring for maven
> 4.0.0 I
> > > > > noticed
> > > > > > > the
> > > > > > > > > > > > > introduction of a lot of new stuff around SAX
> > parsing &
> > > > > > > > filtering.
> > > > > > > > > > > > > I am wondering if that means that it was decided
> that
> > > the
> > > > > > input
> > > > > > > > > > format
> > > > > > > > > > > of
> > > > > > > > > > > > > maven projects will be XML forever meaning
> probably,
> > > > among
> > > > > > > > others,
> > > > > > > > > > the
> > > > > > > > > > > > end
> > > > > > > > > > > > > of polyglot extensions.
> > > > > > > > > > > > > Could you explain such a move (or point to
> > > > > > rationals/documents)
> > > > > > > > and
> > > > > > > > > > why
> > > > > > > > > > > > you
> > > > > > > > > > > > > did not leverage working on the in memory object
> > model
> > > > > > allowing
> > > > > > > > > > > > > extensions/plugins to contribute/hook in the chain
> of
> > > > > > building
> > > > > > > > the
> > > > > > > > > > > > BuildPOM
> > > > > > > > > > > > > & ConsumePOM? In the past I really thought that
> this
> > > move
> > > > > to
> > > > > > > > 'Build
> > > > > > > > > > vs
> > > > > > > > > > > > > Consumer' POM would make clear separations between
> > the
> > > > > input
> > > > > > > > format
> > > > > > > > > > of
> > > > > > > > > > > > > descriptors and the core system but I perhaps
> > > > misunderstood
> > > > > > > > things.
> > > > > > > > > > > > >
> > > > > > > > > > > > > Also, are there plans regarding the future of core
> > > > > > extensions?
> > > > > > > > > > > > > With core extensions it was possible to hook into
> the
> > > POM
> > > > > > model
> > > > > > > > > > loading
> > > > > > > > > > > > and
> > > > > > > > > > > > > do transformations to do dynamic changes but by
> > working
> > > > on
> > > > > > the
> > > > > > > > XML
> > > > > > > > > > > > directly
> > > > > > > > > > > > > I see a shift (if not red stop) in this
> > > > > > contribution/delegation
> > > > > > > > > > > > mechanism.
> > > > > > > > > > > > >
> > > > > > > > > > > > > Thanks for your time & answers.
> > > > > > > > > > > > >
> > > > > > > > > > > > > Matthieu Brouillard
> > > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: maven 4.0.0 new XML stuff

Posted by Matthieu Brouillard <ma...@brouillard.fr>.
> You both keep talking about extensions, but I haven't touched that part.
> I suggest to test current Maven 4 first and to find cases where those
extensions don't work anymore.
I do not know the exact internals of polyglot but for jgitver, in order to
produce a pom-4.0.0.xml compliant file, either:
- delegates to flatten-maven-plugin
- uses an internal mojo
for the production/attachment of a modified pom.xml file dump from the POM.
So in both cases, there is somewhere in the code, some ugly code like:

File dumpedModel = dumpFromModel(model);
project.setPomFile(dumpedModel)

In jgitver it is called from a Mojo(1) which in the end dump and update the
model (2)

I really thought that with 4.0 and the consumer pom this would be obsolete.

For polyglot, I tried but it does not work with maven-4.0.0, see #224 (3)

1:
https://github.com/jgitver/jgitver-maven-plugin/blob/master/src/main/java/fr/brouillard/oss/jgitver/mojos/JGitverAttachModifiedPomsMojo.java
2:
https://github.com/jgitver/jgitver-maven-plugin/blob/master/src/main/java/fr/brouillard/oss/jgitver/JGitverUtils.java#L182
3: https://github.com/takari/polyglot-maven/issues/224


On Tue, Jan 5, 2021 at 12:52 PM Robert Scholte <rf...@apache.org> wrote:

> You both keep talking about extensions, but I haven't touched that part.
> I suggest to test current Maven 4 first and to find cases where those
> extensions don't work anymore.
> Next we can see if this makes sense or not and how to solve it.
>
> Robert
>
> On 5-1-2021 12:28:59, Matthieu Brouillard <ma...@brouillard.fr> wrote:
> > Currently the model is mutable and this causes issues.
> For me it is not the root cause, it does not help for sure but IMO the
> problem is more that there are currently several sources of trust : the POM
> & the pom.xml.
>
> Personally (only my opinion) I see no other way for the future of maven
> than to go to a single source of trust: the POM (as in memory model).
> During the build process this POM could (and has to) become immutable for
> sure so that plugins cannot do ugly things anymore but there should be also
> clear loading and extension mechanisms/hooks before it becomes immutable.
>
> > or do you push to drop extensions support?
> I hope this is not the case.
>
> On Tue, Jan 5, 2021 at 11:58 AM Romain Manni-Bucau
> wrote:
>
> > Hmm, extensions define this kind of lifecycle, at least the
> afterModelRead
> > which is a hook before it could be immutable no?
> > So not sure how it changes the issue, or do you push to drop extensions
> > support?
> >
> > Romain Manni-Bucau
> > @rmannibucau | Blog
> > | Old Blog
> > | Github <
> > https://github.com/rmannibucau> |
> > LinkedIn | Book
> > <
> >
> https://www.packtpub.com/application-development/java-ee-8-high-performance
> > >
> >
> >
> > Le mar. 5 janv. 2021 à 11:54, Robert Scholte a
> > écrit :
> >
> > > Currently the model is mutable and this causes issues.
> > > Instead I would like to see that once the BuildPlan is finished, the
> > model
> > > becomes immutable.
> > >
> > >
> > > To give a concrete example: It must be possible for code generating
> goals
> > > to add dependencies.
> > > Now, when using modello readers/writers you often need to add a
> required
> > > dependency by hand.
> > > Ideally there will be a hook where a plugin can register additional
> > > dependencies (e.g. dom4j).
> > >
> > > This will make any postprocessing of the pom during build obsolete
> > >
> > > Robert
> > > On 5-1-2021 09:09:11, Matthieu Brouillard
> > wrote:
> > > > Can you give an example?
> > > Like Romain has shown: dynamically added dependencies or a version
> > > computation.
> > >
> > > > Most important is the support for CI-friendly versions
> > > But if extensions dynamically compute and set the versions in the POM
> > using
> > > the API, the changes will not be reflected.
> > > That's why today one has to use flatten-maven-plugin or any other
> > > plugin/task modifying/dumping the POM model to a pom.xml file and
> > > setting/attaching it the POM.
> > >
> > > If the produced artifacts (consumer pom) were computed from the POM
> > (object
> > > model), every change done by any extension would be part of it.
> > >
> > > > but this happens AFTER using the pom
> > > Not always from the pom.xml. I thought extensions were allowed to
> provide
> > > ModelLocator & ModelReader to both decide which file to use for a
> project
> > > and how to build the in memory POM model.
> > > So the truth should not be considered to be in the pom.xml but in the
> > POM.
> > >
> > >
> > > On Mon, Jan 4, 2021 at 10:00 PM Robert Scholte wrote:
> > >
> > > > answers are below.
> > > >
> > > > Robert
> > > >
> > > > On 4-1-2021 16:52:23, Matthieu Brouillard wrote:
> > > > @Robert nothing is broken atm, the changes for consumer/build are
> > > currently
> > > > behind your feature flag.
> > > > Robert Scholte:
> > > > It is active by default, so it is actually not hidden.
> > > >
> > > >
> > > >
> > > > But as I feared previously, and as Romain pointed it, by working at
> XML
> > > > level (and not at POM level) the produced consumer pom does not
> reflect
> > > > changes from extensions.
> > > > Robert Scholte:
> > > > Can you give an example?
> > > >
> > > >
> > > > I really thought that all the "consumer/build" stuff would make the
> > > > maven-flatten-plugin useless but it looks like it will not be the
> case
> > if
> > > > working at XML level.
> > > > Robert Scholte:
> > > > Like with most questions: it depends. Most important is the support
> for
> > > > CI-friendly versions. In this case you won't need the
> > > flatten-maven-plugin
> > > > anymore.
> > > > However, the plugin can rewrite much more, but this happens AFTER
> using
> > > > the pom.
> > > > That's something I don't like, because this POM was not used to build
> > the
> > > > project, but it was reassembled afterwards.
> > > > My idea is to provide hooks to parts of the pom that might be
> adjusted,
> > > > but this is something we can work on during the 4.x releases
> > > >
> > > > Did Romain and I miss the whole point of the "consumer/build"
> > > enhancements
> > > > or is it "just" because current implementation has not yet reached
> the
> > > > targets/outputs?
> > > >
> > > > On Mon, Jan 4, 2021 at 2:56 PM Romain Manni-Bucau
> > > > wrote:
> > > >
> > > > > Hmm, I don't get a few things of this IT:
> > > > >
> > > > > 1. the formatting seems not expected even if valid (the comments
> are
> > > > > finishing with the first tag for example "-->
> > > > > public class MyListener extends AbstractMavenLifecycleParticipant {
> > > > >
> > > > > @Override
> > > > > public void afterProjectsRead(final MavenSession session) throws
> > > > > MavenExecutionException {
> > > > > if (session.getCurrentProject() == null) {
> > > > > return;
> > > > > }
> > > > >
> > > > > session.getProjects().forEach(p -> {
> > > > > final Dependency dependency = new Dependency();
> > > > > dependency.setGroupId("junit");
> > > > > dependency.setArtifactId("junit");
> > > > > dependency.setVersion("3.8.1");
> > > > > p.getDependencies().add(dependency);
> > > > > });
> > > > > }
> > > > > }
> > > > >
> > > > >
> > > > > 2. If you run mvn (4 snapshot) dependency:tree you get this kind of
> > > > output:
> > > > >
> > > > > [INFO] -------------
> > > > > >-------------
> > > > > [INFO] Building Multi Chapter Simple Web Application Project
> > > > > 0.9-MNG6957-SNAPSHOT [6/6]
> > > > > [INFO] --------------------------------[ jar
> > > > > ]---------------------------------
> > > > > [INFO]
> > > > > [INFO] --- maven-dependency-plugin:3.1.2:tree (default-cli) @
> > > > simple-webapp
> > > > > ---
> > > > > [INFO]
> > > > org.sonatype.mavenbook.multi:simple-webapp:jar:0.9-MNG6957-SNAPSHOT
> > > > > [INFO] +-
> > > > >
> > > > >
> > > >
> > >
> >
> org.sonatype.mavenbook.multi:simple-weather:jar:0.9-MNG6957-SNAPSHOT:compile
> > > > > [INFO] \- junit:junit:jar:3.8.1:compile
> > > > >
> > > > > [INFO]
> > > > >
> > >
> ------------------------------------------------------------------------
> > > > >
> > > > > 3. run the build to have produced pom and cat the simple-webapp
> one:
> > > > >
> > > > >
> > > > > 4.0.0
> > > > >
> > > > > org.sonatype.mavenbook.multi
> > > > > simple-parent
> > > > > 0.9-MNG6957-SNAPSHOT
> > > > >
> > > > >
> > > > > simple-webapp
> > > > > Multi Chapter Simple Web Application Project
> > > > >
> > > > >
> > > > > org.sonatype.mavenbook.multi
> > > > > simple-weather
> > > > > 0.9-MNG6957-SNAPSHOT
> > > > >
> > > > >
> > > > >
> > > > > simple-webapp
> > > > >
> > > > >
> > > > >
> > > > > org.apache.maven.plugins
> > > > > maven-war-plugin
> > > > > 2.6
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > As you see the dependency is not there. I guess the expected outout
> > is:
> > > > >
> > > > >
> > > > >
> > > > > 4.0.0
> > > > > simple-webapp
> > > > > Multi Chapter Simple Web Application Project
> > > > > [description, scm, ..., all central requires sections but not build
> > > ones]
> > > > >
> > > > >
> > > > > org.sonatype.mavenbook.multi
> > > > > simple-weather
> > > > > 0.9-MNG6957-SNAPSHOT
> > > > >
> > > > >
> > > > > junit
> > > > > junit
> > > > > 3.8.1
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > Am I missing something?
> > > > >
> > > > > Romain Manni-Bucau
> > > > > @rmannibucau | Blog
> > > > > | Old Blog
> > > > > | Github
> > > > > https://github.com/rmannibucau> |
> > > > > LinkedIn | Book
> > > > >
> > > > >
> > > >
> > >
> >
> https://www.packtpub.com/application-development/java-ee-8-high-performance
> > > > > >
> > > > >
> > > > >
> > > > > Le lun. 4 janv. 2021 à 13:41, Robert Scholte a
> > > > > écrit :
> > > > >
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://github.com/apache/maven-integration-testing/tree/master/core-it-suite/src/test/resources/mng-6957-buildconsumer
> > > > > > is the most complete IT
> > > > > >
> > > > > > On 4-1-2021 12:59:51, Romain Manni-Bucau wrote:
> > > > > > Le lun. 4 janv. 2021 à 12:36, Robert Scholte a
> > > > > > écrit :
> > > > > >
> > > > > > > There's just one thing I want to say:
> > > > > > > I'm having trouble with the term "broken".
> > > > > > >
> > > > > >
> > > > > > Well, literally meant broken as decorelated from the user intent
> > and
> > > > > > extension model.
> > > > > > Anyway, didn't intend to blame but more identify the blockers
> for a
> > > GA
> > > > so
> > > > > > point was really that it seem that on the two sides only the
> > > producing
> > > > > one
> > > > > > is not yet ready since it keeps does not sanitize the model
> > > completely
> > > > > and
> > > > > > keeps build only data like comments, right? Also not yet clear
> for
> > me
> > > > if
> > > > > we
> > > > > > loose the extension enrichments there.
> > > > > >
> > > > > >
> > > > > > > If a Maven project could be built with M3.6.3, it can still be
> > > built
> > > > > with
> > > > > > > M4.
> > > > > > > If not, it is either regression (MNG-6957, MNG-7063) which must
> > be
> > > > > fixed,
> > > > > > > or it requires changes to a plugin for understandable reasons
> > > > > > > (maven-pgp-plugin)
> > > > > > > AFAIK an interesting extension like the maven-tiles has been
> > tested
> > > > and
> > > > > > > still works.
> > > > > > >
> > > > > >
> > > > > > Do you have this handy, is it in our test suite? I'd like to
> check
> > > the
> > > > > > produced pom matches the enriched model but happy to start from
> > > > something
> > > > > > already there.
> > > > > >
> > > > > >
> > > > > > >
> > > > > > > Robert
> > > > > > >
> > > > > > > On 3-1-2021 19:35:25, Romain Manni-Bucau wrote:
> > > > > > > Le dim. 3 janv. 2021 à 19:04, Robert Scholte a
> > > > > > > écrit :
> > > > > > >
> > > > > > > > I don't remember all those details anymore, because I hit
> those
> > > in
> > > > > the
> > > > > > > > beginning.
> > > > > > > > Trying things over and over again I decided that this is
> > probably
> > > > the
> > > > > > > most
> > > > > > > > successful approach.
> > > > > > > >
> > > > > > > >
> > > > > > > > What of the goals was to keep the pom.xml as is as much as
> > > > possible.
> > > > > > > > We can only decide for the specific Maven elements how to
> > handle
> > > > > them,
> > > > > > we
> > > > > > > > should not decide about comments and licenses.
> > > > > > > > BTW, the license issue was hard to solve. You cannot use it
> > from
> > > > the
> > > > > > > pom's
> > > > > > > > , because there might be multiple licenses.
> > > > > > > >
> > > > > > >
> > > > > > > I disagree, it is saner IMO to evolve to support that than
> doing
> > > > > > > anything else.
> > > > > > > Once again you keep things which don't make sense in a consumed
> > pom
> > > > in
> > > > > > > current impl so i'd say the sucess in a few cases breaks as
> much
> > > > cases
> > > > > so
> > > > > > > we need to revisit anyway IMHO.
> > > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > The current implementation is a solid way to ensure we're not
> > > > > breaking
> > > > > > > too
> > > > > > > > much, because Maven controls the XML filters.
> > > > > > > >
> > > > > > >
> > > > > > > Hmm, breaking extensions seems to break too much (I'm not
> > speaking
> > > of
> > > > > > other
> > > > > > > parts which breaks the ecosystem there but just that is
> > sufficient
> > > > IMHO
> > > > > > to
> > > > > > > say we must check back our solution).
> > > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > Also keep in mind, that I only want Maven to decide which
> > > > > modifications
> > > > > > > > are done.
> > > > > > > >
> > > > > > >
> > > > > > > For the consumed pom I agree but it is consistent with keeping
> > > > > everything
> > > > > > > working instead of breaking too, no?
> > > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > Current polyglot projects should still work, but they cannot
> > > > benefit
> > > > > > from
> > > > > > > > the build/consumer functions yet.
> > > > > > > >
> > > > > > >
> > > > > > > So pom -> build model is kept, build model -> produced pom is
> > > broken?
> > > > > Is
> > > > > > it
> > > > > > > the complete status?
> > > > > > > Sounds ok for a 4.0 and a 4.1 can fix it if so.
> > > > > > > Just want to ensure first part is not broken at all.
> > > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > Robert
> > > > > > > > On 3-1-2021 16:38:38, Romain Manni-Bucau wrote:
> > > > > > > > Le dim. 3 janv. 2021 à 16:18, Robert Scholte a
> > > > > > > > écrit :
> > > > > > > >
> > > > > > > > > > So what I was expecting was: raw xml model -> converted
> to
> > > > > unified
> > > > > > > > > consumed model -> extensions -> model processing.
> > > > > > > > >
> > > > > > > > > This is only the build pom part. You're missing the consume
> > > part,
> > > > > > where
> > > > > > > > > the xml is distributed.
> > > > > > > > >
> > > > > > > >
> > > > > > > > Yes but with previous chain the consume part is
> > "clean/normalize
> > > ->
> > > > > > dump"
> > > > > > > > since we are using consumed model - only standard model - in
> > > memory
> > > > > > > > already.
> > > > > > > >
> > > > > > > >
> > > > > > > > > Build is raw + enrich, consumer is raw + enrich + reduce
> > > > (removing
> > > > > > > > > relativePath and modules are the first examples, but much
> > more
> > > is
> > > > > > > > possible)
> > > > > > > > >
> > > > > > > > > Going for the in memory was also my first thought, but I
> > would
> > > > > loose
> > > > > > > > > information, hence I came up with the current
> implementation.
> > > > > > > > >
> > > > > > > >
> > > > > > > > I don't see what you loose ot be honest.
> > > > > > > > You mentionned license but this one is in the pom so not a
> big
> > > > deal,
> > > > > > > > comments which are undesired IMHO as mentionned and element
> > order
> > > > > which
> > > > > > > can
> > > > > > > > really be discussed since we can desire to enforce an order
> to
> > > > > > normalize
> > > > > > > > consumption + it shouldn't be important since from the
> project
> > > > point
> > > > > of
> > > > > > > > view your pom is already "broken"/lost (as all your
> > intelligence
> > > is
> > > > > > lost
> > > > > > > by
> > > > > > > > this "not passthrough" process).
> > > > > > > > So overall I don't see what you would loose from the consumer
> > > side
> > > > > but
> > > > > > I
> > > > > > > > see what you lost from maven ecosystem side.
> > > > > > > >
> > > > > > > >
> > > > > > > > > Again, we're at a point where we can have counter
> solutions,
> > > but
> > > > > > don't
> > > > > > > > > expect me to implement it.
> > > > > > > > >
> > > > > > > >
> > > > > > > > For now I'm just trying to ensure we agree we don't want to
> > break
> > > > > > > existing
> > > > > > > > extensions and the nice ecosystem we built after years.
> > > > > > > > This was really a move forward and it sounds like we broke it
> > at
> > > > > maven
> > > > > > 4
> > > > > > > > without any user gain which sounds terrible.
> > > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > thanks,
> > > > > > > > > Robert
> > > > > > > > > On 3-1-2021 15:25:21, Romain Manni-Bucau wrote:
> > > > > > > > > Hi all,
> > > > > > > > >
> > > > > > > > > I kind of join Matthieu thoughts there, there is no point
> to
> > > work
> > > > > at
> > > > > > > xml
> > > > > > > > > level to create the consumed pom - comments is not a point
> > > since
> > > > it
> > > > > > can
> > > > > > > > > commonly/easily refer to a dropped part of the pom so they
> > > should
> > > > > be
> > > > > > > > > stripped.
> > > > > > > > > Current extension model got proven adapted and adopted,
> > using a
> > > > > lower
> > > > > > > > level
> > > > > > > > > extension API will not since XML is, even if still
> > mainstream,
> > > > > often
> > > > > > > > > replaced by alternative configurations and to have done the
> > > work
> > > > to
> > > > > > > > inject
> > > > > > > > > XML configuration programmatically compred to current
> option,
> > > it
> > > > > is a
> > > > > > > > pain.
> > > > > > > > > The in memory model should stick to consumed model IMHO -
> > being
> > > > > > > > > programmatic there is no point to make it easier, worse
> case
> > we
> > > > can
> > > > > > add
> > > > > > > > > helper beans (injectable) but in terms of model it will not
> > > help.
> > > > > > > > >
> > > > > > > > > So what I was expecting was:
> > > > > > > > >
> > > > > > > > > raw xml model -> converted to unified consumed model ->
> > > > extensions
> > > > > ->
> > > > > > > > model
> > > > > > > > > processing.
> > > > > > > > >
> > > > > > > > > Indeed, real chain adds a small processing over the first
> > arrow
> > > > > > (inject
> > > > > > > > > versions for example) but nothing crazy and breaking this
> > > overall
> > > > > > flow
> > > > > > > > > which stays user friendly.
> > > > > > > > >
> > > > > > > > > Strictly speaking the new model is just a built-in
> extension
> > > for
> > > > me
> > > > > > > which
> > > > > > > > > is particular because it will enforce IDE to integrate a
> new
> > > > > format -
> > > > > > > > > wheres polyglot extensions or others don't require static
> > > > analyzis
> > > > > by
> > > > > > > > > themself not being "standard".
> > > > > > > > >
> > > > > > > > > That said, there is nothing crazy with current
> > implementation,
> > > it
> > > > > > just
> > > > > > > > > require to be updated to be able to take extension changes
> > into
> > > > > > > account.
> > > > > > > > > This can be done by making the extension model 'spyable'
> (ie
> > > if a
> > > > > > > > > dependency/plugin is added it will be reflected in the
> final
> > > > > written
> > > > > > > > > pom.xml).
> > > > > > > > > This sounds - instrumenting the extension model API or
> doing
> > a
> > > > diff
> > > > > > > after
> > > > > > > > > extension phase - like a compromise and let people gets the
> > > best
> > > > of
> > > > > > > both
> > > > > > > > > worlds to me.
> > > > > > > > >
> > > > > > > > > Wdyt?
> > > > > > > > >
> > > > > > > > > Romain Manni-Bucau
> > > > > > > > > @rmannibucau | Blog
> > > > > > > > > | Old Blog
> > > > > > > > > | Github |
> > > > > > > > > LinkedIn | Book
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > Le dim. 3 janv. 2021 à 14:46, Robert Scholte a
> > > > > > > > > écrit :
> > > > > > > > >
> > > > > > > > > > Hi Matthieu,
> > > > > > > > > >
> > > > > > > > > > As you understand, something had to be changed to move
> > Maven
> > > > > > forward.
> > > > > > > > > > I've decided to pick up that challenge and came up with
> the
> > > > > current
> > > > > > > > > > solution.
> > > > > > > > > >
> > > > > > > > > > My main concerns was that I wanted to keep the fileModel
> as
> > > > much
> > > > > as
> > > > > > > is.
> > > > > > > > > > That includes the license, comments and element order.
> > > > > > > > > > This information if not available in the memory model,
> so I
> > > > > needed
> > > > > > > the
> > > > > > > > > > original pom file.
> > > > > > > > > > With that in mind, the usage of XMLFilters looks like the
> > > most
> > > > > > > > > appropriate
> > > > > > > > > > solution.
> > > > > > > > > >
> > > > > > > > > > I am certain that XML is still the most used format, so
> if
> > we
> > > > can
> > > > > > > have
> > > > > > > > > > improvements for those users, I'm already very happy.
> > > > > > > > > >
> > > > > > > > > > And yes, there are plugins that needs to be updated, but
> > > doing
> > > > > > > nothing
> > > > > > > > is
> > > > > > > > > > not an option anymore.
> > > > > > > > > >
> > > > > > > > > > There are more people that share their concerns, but it
> > took
> > > me
> > > > > > > several
> > > > > > > > > > years to reach this point.
> > > > > > > > > > We now have something that seems to work, anybody who can
> > > > improve
> > > > > > or
> > > > > > > > can
> > > > > > > > > > come up with an alternative implementation can do so.
> > > > > > > > > >
> > > > > > > > > > thanks,
> > > > > > > > > > Robert
> > > > > > > > > > On 3-1-2021 12:55:41, Matthieu Brouillard wrote:
> > > > > > > > > > Thanks Robert for the video link.
> > > > > > > > > >
> > > > > > > > > > I fully understand the rationales behind the separation
> of
> > > > > > > > > > build/consumer pom and the video provides some insights
> on
> > it
> > > > and
> > > > > > you
> > > > > > > > > > explain the actual implementation to introduce this
> change.
> > > > > > > > > > Still I do not fully understand why it was decided to
> work
> > on
> > > > top
> > > > > > of
> > > > > > > > XML
> > > > > > > > > by
> > > > > > > > > > filtering/enhancing it instead of working at the POM (in
> > > > > > > > > > memory datamodel) level.
> > > > > > > > > > With the current understanding I have, by doing this
> choice
> > > of
> > > > > > > working
> > > > > > > > at
> > > > > > > > > > XML level, it looks like it was decided to bypass (if not
> > > kill)
> > > > > > core
> > > > > > > > > > extensions that enhance the POM itself and not the
> pom.xml
> > ;
> > > > > here I
> > > > > > > can
> > > > > > > > > > think of (but probably not limited to):
> > > > > > > > > > - polyglot-maven: do not use XML but other format to
> > describe
> > > > the
> > > > > > POM
> > > > > > > > > > (yaml, json, kotlin, java, other XML formats, ...)
> > > > > > > > > > - jgitver-maven-plugin (or forks like
> > > > > > > maven-git-versioning-extension):
> > > > > > > > > > dynamic computation of projects version based on git
> > history
> > > > > > > > > >
> > > > > > > > > > With the introduction of core extensions, I thought it
> was
> > a
> > > > move
> > > > > > to
> > > > > > > > open
> > > > > > > > > > the internals and let externals contribute to the
> > > capabilities
> > > > of
> > > > > > > > maven.
> > > > > > > > > > With the move to a XML handling chain, I see it as a
> > > > > > > > restriction/regress
> > > > > > > > > in
> > > > > > > > > > favor of core closed functionalities. An example of that
> is
> > > > what
> > > > > is
> > > > > > > > > > provided as CIFriendly stuff, IMO it could/should have
> been
> > > > > > provided
> > > > > > > > by a
> > > > > > > > > > plugin/extension but instead it is hard written in maven
> > core
> > > > and
> > > > > > is
> > > > > > > > not
> > > > > > > > > > opened for external contribution (plugin/extension I
> mean).
> > > > > > > > > >
> > > > > > > > > > Perhaps I am totally wrong but I think that maven core
> > should
> > > > > > define
> > > > > > > > all
> > > > > > > > > > its expectations at an API level so that
> extensions/plugins
> > > > could
> > > > > > > hook
> > > > > > > > at
> > > > > > > > > > this API level. The default packaging of maven
> could/should
> > > > > provide
> > > > > > > > > default
> > > > > > > > > > implementations of those expectations (for example
> reading
> > a
> > > > > > pom.xml
> > > > > > > > file
> > > > > > > > > > to a POM model, dumping a POM to a pom-4.0.0.xml,
> > > > > > > > transforming/reducing a
> > > > > > > > > > POM to POM-consumer, dumping POM-consumer to
> > > > > > pom-consumer-5.0.0.xml,
> > > > > > > > ...)
> > > > > > > > > > and let extensions/plugins/default implementations work
> > along
> > > > the
> > > > > > > build
> > > > > > > > > > process with the API & POMs to provide different features
> > and
> > > > > > > > > capabilities.
> > > > > > > > > >
> > > > > > > > > > Matthieu
> > > > > > > > > >
> > > > > > > > > > On Thu, Dec 31, 2020 at 7:01 PM Robert Scholte wrote:
> > > > > > > > > >
> > > > > > > > > > > I've made a recording[1] about it, which hopefully
> > answers
> > > > most
> > > > > > > > > > questions.
> > > > > > > > > > >
> > > > > > > > > > > Robert
> > > > > > > > > > >
> > > > > > > > > > > [1] https://youtu.be/KDAmlNKZJto
> > > > > > > > > > >
> > > > > > > > > > > On 31-12-2020 16:18:57, Matthieu Brouillard
> > > > > > > > > > > wrote:
> > > > > > > > > > > > Not exactly sure what work you mean
> > > > > > > > > > > everything related to maven-xml:
> > > > Build/ConsumerPomXMLFilterxxx,
> > > > > > > > > > > Build/ConsumerModelSourcexxxx and the transformer
> stuff.
> > > > > > > > > > > Especially, when looking at classes like
> > > > CiFriendlyXMLFilter, I
> > > > > > > would
> > > > > > > > > > have
> > > > > > > > > > > thought that such things could have been done
> elsewhere,
> > > > > working
> > > > > > on
> > > > > > > > the
> > > > > > > > > > > object model (not on the XML stuff) especially for the
> > > > BuildPom
> > > > > > > part.
> > > > > > > > > > >
> > > > > > > > > > > > however specifically the consumer POM integrates with
> > so
> > > > many
> > > > > > > > > external
> > > > > > > > > > > ecosystems
> > > > > > > > > > > We're aligned here, this has to be stable and well
> > defined
> > > > by a
> > > > > > > > schema.
> > > > > > > > > > >
> > > > > > > > > > > Matthieu
> > > > > > > > > > >
> > > > > > > > > > > On Thu, Dec 31, 2020 at 3:59 PM Bernd Eckenfels
> > > > > > > > > > > wrote:
> > > > > > > > > > >
> > > > > > > > > > > > Hello,
> > > > > > > > > > > >
> > > > > > > > > > > > Not exactly sure what work you mean and I fully agree
> > > that
> > > > > > using
> > > > > > > a
> > > > > > > > > core
> > > > > > > > > > > > model should still be the API for plugins and
> > extensions
> > > to
> > > > > > work
> > > > > > > > > with,
> > > > > > > > > > > > however specifically the consumer POM integrates with
> > so
> > > > many
> > > > > > > > > external
> > > > > > > > > > > > ecosystems, I would expect it to be defined in terms
> of
> > > XML
> > > > > > > Schema
> > > > > > > > > with
> > > > > > > > > > > > explicite semantic (and the inherent compatibility
> with
> > > > > exiting
> > > > > > > > > POMs).
> > > > > > > > > > > >
> > > > > > > > > > > > Gruss
> > > > > > > > > > > > Bernd
> > > > > > > > > > > > --
> > > > > > > > > > > > http://bernd.eckenfels.net
> > > > > > > > > > > > ________________________________
> > > > > > > > > > > > Von: Matthieu BROUILLARD
> > > > > > > > > > > > Gesendet: Thursday, December 31, 2020 3:19:09 PM
> > > > > > > > > > > > An: dev@maven.apache.org
> > > > > > > > > > > > Betreff: maven 4.0.0 new XML stuff
> > > > > > > > > > > >
> > > > > > > > > > > > Hello all,
> > > > > > > > > > > >
> > > > > > > > > > > > regarding the active work occurring for maven 4.0.0 I
> > > > noticed
> > > > > > the
> > > > > > > > > > > > introduction of a lot of new stuff around SAX
> parsing &
> > > > > > > filtering.
> > > > > > > > > > > > I am wondering if that means that it was decided that
> > the
> > > > > input
> > > > > > > > > format
> > > > > > > > > > of
> > > > > > > > > > > > maven projects will be XML forever meaning probably,
> > > among
> > > > > > > others,
> > > > > > > > > the
> > > > > > > > > > > end
> > > > > > > > > > > > of polyglot extensions.
> > > > > > > > > > > > Could you explain such a move (or point to
> > > > > rationals/documents)
> > > > > > > and
> > > > > > > > > why
> > > > > > > > > > > you
> > > > > > > > > > > > did not leverage working on the in memory object
> model
> > > > > allowing
> > > > > > > > > > > > extensions/plugins to contribute/hook in the chain of
> > > > > building
> > > > > > > the
> > > > > > > > > > > BuildPOM
> > > > > > > > > > > > & ConsumePOM? In the past I really thought that this
> > move
> > > > to
> > > > > > > 'Build
> > > > > > > > > vs
> > > > > > > > > > > > Consumer' POM would make clear separations between
> the
> > > > input
> > > > > > > format
> > > > > > > > > of
> > > > > > > > > > > > descriptors and the core system but I perhaps
> > > misunderstood
> > > > > > > things.
> > > > > > > > > > > >
> > > > > > > > > > > > Also, are there plans regarding the future of core
> > > > > extensions?
> > > > > > > > > > > > With core extensions it was possible to hook into the
> > POM
> > > > > model
> > > > > > > > > loading
> > > > > > > > > > > and
> > > > > > > > > > > > do transformations to do dynamic changes but by
> working
> > > on
> > > > > the
> > > > > > > XML
> > > > > > > > > > > directly
> > > > > > > > > > > > I see a shift (if not red stop) in this
> > > > > contribution/delegation
> > > > > > > > > > > mechanism.
> > > > > > > > > > > >
> > > > > > > > > > > > Thanks for your time & answers.
> > > > > > > > > > > >
> > > > > > > > > > > > Matthieu Brouillard
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: maven 4.0.0 new XML stuff

Posted by Robert Scholte <rf...@apache.org>.
You both keep talking about extensions, but I haven't touched that part.
I suggest to test current Maven 4 first and to find cases where those extensions don't work anymore.
Next we can see if this makes sense or not and how to solve it.

Robert

On 5-1-2021 12:28:59, Matthieu Brouillard <ma...@brouillard.fr> wrote:
> Currently the model is mutable and this causes issues.
For me it is not the root cause, it does not help for sure but IMO the
problem is more that there are currently several sources of trust : the POM
& the pom.xml.

Personally (only my opinion) I see no other way for the future of maven
than to go to a single source of trust: the POM (as in memory model).
During the build process this POM could (and has to) become immutable for
sure so that plugins cannot do ugly things anymore but there should be also
clear loading and extension mechanisms/hooks before it becomes immutable.

> or do you push to drop extensions support?
I hope this is not the case.

On Tue, Jan 5, 2021 at 11:58 AM Romain Manni-Bucau
wrote:

> Hmm, extensions define this kind of lifecycle, at least the afterModelRead
> which is a hook before it could be immutable no?
> So not sure how it changes the issue, or do you push to drop extensions
> support?
>
> Romain Manni-Bucau
> @rmannibucau | Blog
> | Old Blog
> | Github <
> https://github.com/rmannibucau> |
> LinkedIn | Book
> <
> https://www.packtpub.com/application-development/java-ee-8-high-performance
> >
>
>
> Le mar. 5 janv. 2021 à 11:54, Robert Scholte a
> écrit :
>
> > Currently the model is mutable and this causes issues.
> > Instead I would like to see that once the BuildPlan is finished, the
> model
> > becomes immutable.
> >
> >
> > To give a concrete example: It must be possible for code generating goals
> > to add dependencies.
> > Now, when using modello readers/writers you often need to add a required
> > dependency by hand.
> > Ideally there will be a hook where a plugin can register additional
> > dependencies (e.g. dom4j).
> >
> > This will make any postprocessing of the pom during build obsolete
> >
> > Robert
> > On 5-1-2021 09:09:11, Matthieu Brouillard
> wrote:
> > > Can you give an example?
> > Like Romain has shown: dynamically added dependencies or a version
> > computation.
> >
> > > Most important is the support for CI-friendly versions
> > But if extensions dynamically compute and set the versions in the POM
> using
> > the API, the changes will not be reflected.
> > That's why today one has to use flatten-maven-plugin or any other
> > plugin/task modifying/dumping the POM model to a pom.xml file and
> > setting/attaching it the POM.
> >
> > If the produced artifacts (consumer pom) were computed from the POM
> (object
> > model), every change done by any extension would be part of it.
> >
> > > but this happens AFTER using the pom
> > Not always from the pom.xml. I thought extensions were allowed to provide
> > ModelLocator & ModelReader to both decide which file to use for a project
> > and how to build the in memory POM model.
> > So the truth should not be considered to be in the pom.xml but in the
> POM.
> >
> >
> > On Mon, Jan 4, 2021 at 10:00 PM Robert Scholte wrote:
> >
> > > answers are below.
> > >
> > > Robert
> > >
> > > On 4-1-2021 16:52:23, Matthieu Brouillard wrote:
> > > @Robert nothing is broken atm, the changes for consumer/build are
> > currently
> > > behind your feature flag.
> > > Robert Scholte:
> > > It is active by default, so it is actually not hidden.
> > >
> > >
> > >
> > > But as I feared previously, and as Romain pointed it, by working at XML
> > > level (and not at POM level) the produced consumer pom does not reflect
> > > changes from extensions.
> > > Robert Scholte:
> > > Can you give an example?
> > >
> > >
> > > I really thought that all the "consumer/build" stuff would make the
> > > maven-flatten-plugin useless but it looks like it will not be the case
> if
> > > working at XML level.
> > > Robert Scholte:
> > > Like with most questions: it depends. Most important is the support for
> > > CI-friendly versions. In this case you won't need the
> > flatten-maven-plugin
> > > anymore.
> > > However, the plugin can rewrite much more, but this happens AFTER using
> > > the pom.
> > > That's something I don't like, because this POM was not used to build
> the
> > > project, but it was reassembled afterwards.
> > > My idea is to provide hooks to parts of the pom that might be adjusted,
> > > but this is something we can work on during the 4.x releases
> > >
> > > Did Romain and I miss the whole point of the "consumer/build"
> > enhancements
> > > or is it "just" because current implementation has not yet reached the
> > > targets/outputs?
> > >
> > > On Mon, Jan 4, 2021 at 2:56 PM Romain Manni-Bucau
> > > wrote:
> > >
> > > > Hmm, I don't get a few things of this IT:
> > > >
> > > > 1. the formatting seems not expected even if valid (the comments are
> > > > finishing with the first tag for example "-->
> > > > public class MyListener extends AbstractMavenLifecycleParticipant {
> > > >
> > > > @Override
> > > > public void afterProjectsRead(final MavenSession session) throws
> > > > MavenExecutionException {
> > > > if (session.getCurrentProject() == null) {
> > > > return;
> > > > }
> > > >
> > > > session.getProjects().forEach(p -> {
> > > > final Dependency dependency = new Dependency();
> > > > dependency.setGroupId("junit");
> > > > dependency.setArtifactId("junit");
> > > > dependency.setVersion("3.8.1");
> > > > p.getDependencies().add(dependency);
> > > > });
> > > > }
> > > > }
> > > >
> > > >
> > > > 2. If you run mvn (4 snapshot) dependency:tree you get this kind of
> > > output:
> > > >
> > > > [INFO] -------------
> > > > >-------------
> > > > [INFO] Building Multi Chapter Simple Web Application Project
> > > > 0.9-MNG6957-SNAPSHOT [6/6]
> > > > [INFO] --------------------------------[ jar
> > > > ]---------------------------------
> > > > [INFO]
> > > > [INFO] --- maven-dependency-plugin:3.1.2:tree (default-cli) @
> > > simple-webapp
> > > > ---
> > > > [INFO]
> > > org.sonatype.mavenbook.multi:simple-webapp:jar:0.9-MNG6957-SNAPSHOT
> > > > [INFO] +-
> > > >
> > > >
> > >
> >
> org.sonatype.mavenbook.multi:simple-weather:jar:0.9-MNG6957-SNAPSHOT:compile
> > > > [INFO] \- junit:junit:jar:3.8.1:compile
> > > >
> > > > [INFO]
> > > >
> > ------------------------------------------------------------------------
> > > >
> > > > 3. run the build to have produced pom and cat the simple-webapp one:
> > > >
> > > >
> > > > 4.0.0
> > > >
> > > > org.sonatype.mavenbook.multi
> > > > simple-parent
> > > > 0.9-MNG6957-SNAPSHOT
> > > >
> > > >
> > > > simple-webapp
> > > > Multi Chapter Simple Web Application Project
> > > >
> > > >
> > > > org.sonatype.mavenbook.multi
> > > > simple-weather
> > > > 0.9-MNG6957-SNAPSHOT
> > > >
> > > >
> > > >
> > > > simple-webapp
> > > >
> > > >
> > > >
> > > > org.apache.maven.plugins
> > > > maven-war-plugin
> > > > 2.6
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > As you see the dependency is not there. I guess the expected outout
> is:
> > > >
> > > >
> > > >
> > > > 4.0.0
> > > > simple-webapp
> > > > Multi Chapter Simple Web Application Project
> > > > [description, scm, ..., all central requires sections but not build
> > ones]
> > > >
> > > >
> > > > org.sonatype.mavenbook.multi
> > > > simple-weather
> > > > 0.9-MNG6957-SNAPSHOT
> > > >
> > > >
> > > > junit
> > > > junit
> > > > 3.8.1
> > > >
> > > >
> > > >
> > > >
> > > > Am I missing something?
> > > >
> > > > Romain Manni-Bucau
> > > > @rmannibucau | Blog
> > > > | Old Blog
> > > > | Github
> > > > https://github.com/rmannibucau> |
> > > > LinkedIn | Book
> > > >
> > > >
> > >
> >
> https://www.packtpub.com/application-development/java-ee-8-high-performance
> > > > >
> > > >
> > > >
> > > > Le lun. 4 janv. 2021 à 13:41, Robert Scholte a
> > > > écrit :
> > > >
> > > > >
> > > > >
> > > >
> > >
> >
> https://github.com/apache/maven-integration-testing/tree/master/core-it-suite/src/test/resources/mng-6957-buildconsumer
> > > > > is the most complete IT
> > > > >
> > > > > On 4-1-2021 12:59:51, Romain Manni-Bucau wrote:
> > > > > Le lun. 4 janv. 2021 à 12:36, Robert Scholte a
> > > > > écrit :
> > > > >
> > > > > > There's just one thing I want to say:
> > > > > > I'm having trouble with the term "broken".
> > > > > >
> > > > >
> > > > > Well, literally meant broken as decorelated from the user intent
> and
> > > > > extension model.
> > > > > Anyway, didn't intend to blame but more identify the blockers for a
> > GA
> > > so
> > > > > point was really that it seem that on the two sides only the
> > producing
> > > > one
> > > > > is not yet ready since it keeps does not sanitize the model
> > completely
> > > > and
> > > > > keeps build only data like comments, right? Also not yet clear for
> me
> > > if
> > > > we
> > > > > loose the extension enrichments there.
> > > > >
> > > > >
> > > > > > If a Maven project could be built with M3.6.3, it can still be
> > built
> > > > with
> > > > > > M4.
> > > > > > If not, it is either regression (MNG-6957, MNG-7063) which must
> be
> > > > fixed,
> > > > > > or it requires changes to a plugin for understandable reasons
> > > > > > (maven-pgp-plugin)
> > > > > > AFAIK an interesting extension like the maven-tiles has been
> tested
> > > and
> > > > > > still works.
> > > > > >
> > > > >
> > > > > Do you have this handy, is it in our test suite? I'd like to check
> > the
> > > > > produced pom matches the enriched model but happy to start from
> > > something
> > > > > already there.
> > > > >
> > > > >
> > > > > >
> > > > > > Robert
> > > > > >
> > > > > > On 3-1-2021 19:35:25, Romain Manni-Bucau wrote:
> > > > > > Le dim. 3 janv. 2021 à 19:04, Robert Scholte a
> > > > > > écrit :
> > > > > >
> > > > > > > I don't remember all those details anymore, because I hit those
> > in
> > > > the
> > > > > > > beginning.
> > > > > > > Trying things over and over again I decided that this is
> probably
> > > the
> > > > > > most
> > > > > > > successful approach.
> > > > > > >
> > > > > > >
> > > > > > > What of the goals was to keep the pom.xml as is as much as
> > > possible.
> > > > > > > We can only decide for the specific Maven elements how to
> handle
> > > > them,
> > > > > we
> > > > > > > should not decide about comments and licenses.
> > > > > > > BTW, the license issue was hard to solve. You cannot use it
> from
> > > the
> > > > > > pom's
> > > > > > > , because there might be multiple licenses.
> > > > > > >
> > > > > >
> > > > > > I disagree, it is saner IMO to evolve to support that than doing
> > > > > > anything else.
> > > > > > Once again you keep things which don't make sense in a consumed
> pom
> > > in
> > > > > > current impl so i'd say the sucess in a few cases breaks as much
> > > cases
> > > > so
> > > > > > we need to revisit anyway IMHO.
> > > > > >
> > > > > >
> > > > > > >
> > > > > > > The current implementation is a solid way to ensure we're not
> > > > breaking
> > > > > > too
> > > > > > > much, because Maven controls the XML filters.
> > > > > > >
> > > > > >
> > > > > > Hmm, breaking extensions seems to break too much (I'm not
> speaking
> > of
> > > > > other
> > > > > > parts which breaks the ecosystem there but just that is
> sufficient
> > > IMHO
> > > > > to
> > > > > > say we must check back our solution).
> > > > > >
> > > > > >
> > > > > > >
> > > > > > > Also keep in mind, that I only want Maven to decide which
> > > > modifications
> > > > > > > are done.
> > > > > > >
> > > > > >
> > > > > > For the consumed pom I agree but it is consistent with keeping
> > > > everything
> > > > > > working instead of breaking too, no?
> > > > > >
> > > > > >
> > > > > > >
> > > > > > > Current polyglot projects should still work, but they cannot
> > > benefit
> > > > > from
> > > > > > > the build/consumer functions yet.
> > > > > > >
> > > > > >
> > > > > > So pom -> build model is kept, build model -> produced pom is
> > broken?
> > > > Is
> > > > > it
> > > > > > the complete status?
> > > > > > Sounds ok for a 4.0 and a 4.1 can fix it if so.
> > > > > > Just want to ensure first part is not broken at all.
> > > > > >
> > > > > >
> > > > > > >
> > > > > > > Robert
> > > > > > > On 3-1-2021 16:38:38, Romain Manni-Bucau wrote:
> > > > > > > Le dim. 3 janv. 2021 à 16:18, Robert Scholte a
> > > > > > > écrit :
> > > > > > >
> > > > > > > > > So what I was expecting was: raw xml model -> converted to
> > > > unified
> > > > > > > > consumed model -> extensions -> model processing.
> > > > > > > >
> > > > > > > > This is only the build pom part. You're missing the consume
> > part,
> > > > > where
> > > > > > > > the xml is distributed.
> > > > > > > >
> > > > > > >
> > > > > > > Yes but with previous chain the consume part is
> "clean/normalize
> > ->
> > > > > dump"
> > > > > > > since we are using consumed model - only standard model - in
> > memory
> > > > > > > already.
> > > > > > >
> > > > > > >
> > > > > > > > Build is raw + enrich, consumer is raw + enrich + reduce
> > > (removing
> > > > > > > > relativePath and modules are the first examples, but much
> more
> > is
> > > > > > > possible)
> > > > > > > >
> > > > > > > > Going for the in memory was also my first thought, but I
> would
> > > > loose
> > > > > > > > information, hence I came up with the current implementation.
> > > > > > > >
> > > > > > >
> > > > > > > I don't see what you loose ot be honest.
> > > > > > > You mentionned license but this one is in the pom so not a big
> > > deal,
> > > > > > > comments which are undesired IMHO as mentionned and element
> order
> > > > which
> > > > > > can
> > > > > > > really be discussed since we can desire to enforce an order to
> > > > > normalize
> > > > > > > consumption + it shouldn't be important since from the project
> > > point
> > > > of
> > > > > > > view your pom is already "broken"/lost (as all your
> intelligence
> > is
> > > > > lost
> > > > > > by
> > > > > > > this "not passthrough" process).
> > > > > > > So overall I don't see what you would loose from the consumer
> > side
> > > > but
> > > > > I
> > > > > > > see what you lost from maven ecosystem side.
> > > > > > >
> > > > > > >
> > > > > > > > Again, we're at a point where we can have counter solutions,
> > but
> > > > > don't
> > > > > > > > expect me to implement it.
> > > > > > > >
> > > > > > >
> > > > > > > For now I'm just trying to ensure we agree we don't want to
> break
> > > > > > existing
> > > > > > > extensions and the nice ecosystem we built after years.
> > > > > > > This was really a move forward and it sounds like we broke it
> at
> > > > maven
> > > > > 4
> > > > > > > without any user gain which sounds terrible.
> > > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > thanks,
> > > > > > > > Robert
> > > > > > > > On 3-1-2021 15:25:21, Romain Manni-Bucau wrote:
> > > > > > > > Hi all,
> > > > > > > >
> > > > > > > > I kind of join Matthieu thoughts there, there is no point to
> > work
> > > > at
> > > > > > xml
> > > > > > > > level to create the consumed pom - comments is not a point
> > since
> > > it
> > > > > can
> > > > > > > > commonly/easily refer to a dropped part of the pom so they
> > should
> > > > be
> > > > > > > > stripped.
> > > > > > > > Current extension model got proven adapted and adopted,
> using a
> > > > lower
> > > > > > > level
> > > > > > > > extension API will not since XML is, even if still
> mainstream,
> > > > often
> > > > > > > > replaced by alternative configurations and to have done the
> > work
> > > to
> > > > > > > inject
> > > > > > > > XML configuration programmatically compred to current option,
> > it
> > > > is a
> > > > > > > pain.
> > > > > > > > The in memory model should stick to consumed model IMHO -
> being
> > > > > > > > programmatic there is no point to make it easier, worse case
> we
> > > can
> > > > > add
> > > > > > > > helper beans (injectable) but in terms of model it will not
> > help.
> > > > > > > >
> > > > > > > > So what I was expecting was:
> > > > > > > >
> > > > > > > > raw xml model -> converted to unified consumed model ->
> > > extensions
> > > > ->
> > > > > > > model
> > > > > > > > processing.
> > > > > > > >
> > > > > > > > Indeed, real chain adds a small processing over the first
> arrow
> > > > > (inject
> > > > > > > > versions for example) but nothing crazy and breaking this
> > overall
> > > > > flow
> > > > > > > > which stays user friendly.
> > > > > > > >
> > > > > > > > Strictly speaking the new model is just a built-in extension
> > for
> > > me
> > > > > > which
> > > > > > > > is particular because it will enforce IDE to integrate a new
> > > > format -
> > > > > > > > wheres polyglot extensions or others don't require static
> > > analyzis
> > > > by
> > > > > > > > themself not being "standard".
> > > > > > > >
> > > > > > > > That said, there is nothing crazy with current
> implementation,
> > it
> > > > > just
> > > > > > > > require to be updated to be able to take extension changes
> into
> > > > > > account.
> > > > > > > > This can be done by making the extension model 'spyable' (ie
> > if a
> > > > > > > > dependency/plugin is added it will be reflected in the final
> > > > written
> > > > > > > > pom.xml).
> > > > > > > > This sounds - instrumenting the extension model API or doing
> a
> > > diff
> > > > > > after
> > > > > > > > extension phase - like a compromise and let people gets the
> > best
> > > of
> > > > > > both
> > > > > > > > worlds to me.
> > > > > > > >
> > > > > > > > Wdyt?
> > > > > > > >
> > > > > > > > Romain Manni-Bucau
> > > > > > > > @rmannibucau | Blog
> > > > > > > > | Old Blog
> > > > > > > > | Github |
> > > > > > > > LinkedIn | Book
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > Le dim. 3 janv. 2021 à 14:46, Robert Scholte a
> > > > > > > > écrit :
> > > > > > > >
> > > > > > > > > Hi Matthieu,
> > > > > > > > >
> > > > > > > > > As you understand, something had to be changed to move
> Maven
> > > > > forward.
> > > > > > > > > I've decided to pick up that challenge and came up with the
> > > > current
> > > > > > > > > solution.
> > > > > > > > >
> > > > > > > > > My main concerns was that I wanted to keep the fileModel as
> > > much
> > > > as
> > > > > > is.
> > > > > > > > > That includes the license, comments and element order.
> > > > > > > > > This information if not available in the memory model, so I
> > > > needed
> > > > > > the
> > > > > > > > > original pom file.
> > > > > > > > > With that in mind, the usage of XMLFilters looks like the
> > most
> > > > > > > > appropriate
> > > > > > > > > solution.
> > > > > > > > >
> > > > > > > > > I am certain that XML is still the most used format, so if
> we
> > > can
> > > > > > have
> > > > > > > > > improvements for those users, I'm already very happy.
> > > > > > > > >
> > > > > > > > > And yes, there are plugins that needs to be updated, but
> > doing
> > > > > > nothing
> > > > > > > is
> > > > > > > > > not an option anymore.
> > > > > > > > >
> > > > > > > > > There are more people that share their concerns, but it
> took
> > me
> > > > > > several
> > > > > > > > > years to reach this point.
> > > > > > > > > We now have something that seems to work, anybody who can
> > > improve
> > > > > or
> > > > > > > can
> > > > > > > > > come up with an alternative implementation can do so.
> > > > > > > > >
> > > > > > > > > thanks,
> > > > > > > > > Robert
> > > > > > > > > On 3-1-2021 12:55:41, Matthieu Brouillard wrote:
> > > > > > > > > Thanks Robert for the video link.
> > > > > > > > >
> > > > > > > > > I fully understand the rationales behind the separation of
> > > > > > > > > build/consumer pom and the video provides some insights on
> it
> > > and
> > > > > you
> > > > > > > > > explain the actual implementation to introduce this change.
> > > > > > > > > Still I do not fully understand why it was decided to work
> on
> > > top
> > > > > of
> > > > > > > XML
> > > > > > > > by
> > > > > > > > > filtering/enhancing it instead of working at the POM (in
> > > > > > > > > memory datamodel) level.
> > > > > > > > > With the current understanding I have, by doing this choice
> > of
> > > > > > working
> > > > > > > at
> > > > > > > > > XML level, it looks like it was decided to bypass (if not
> > kill)
> > > > > core
> > > > > > > > > extensions that enhance the POM itself and not the pom.xml
> ;
> > > > here I
> > > > > > can
> > > > > > > > > think of (but probably not limited to):
> > > > > > > > > - polyglot-maven: do not use XML but other format to
> describe
> > > the
> > > > > POM
> > > > > > > > > (yaml, json, kotlin, java, other XML formats, ...)
> > > > > > > > > - jgitver-maven-plugin (or forks like
> > > > > > maven-git-versioning-extension):
> > > > > > > > > dynamic computation of projects version based on git
> history
> > > > > > > > >
> > > > > > > > > With the introduction of core extensions, I thought it was
> a
> > > move
> > > > > to
> > > > > > > open
> > > > > > > > > the internals and let externals contribute to the
> > capabilities
> > > of
> > > > > > > maven.
> > > > > > > > > With the move to a XML handling chain, I see it as a
> > > > > > > restriction/regress
> > > > > > > > in
> > > > > > > > > favor of core closed functionalities. An example of that is
> > > what
> > > > is
> > > > > > > > > provided as CIFriendly stuff, IMO it could/should have been
> > > > > provided
> > > > > > > by a
> > > > > > > > > plugin/extension but instead it is hard written in maven
> core
> > > and
> > > > > is
> > > > > > > not
> > > > > > > > > opened for external contribution (plugin/extension I mean).
> > > > > > > > >
> > > > > > > > > Perhaps I am totally wrong but I think that maven core
> should
> > > > > define
> > > > > > > all
> > > > > > > > > its expectations at an API level so that extensions/plugins
> > > could
> > > > > > hook
> > > > > > > at
> > > > > > > > > this API level. The default packaging of maven could/should
> > > > provide
> > > > > > > > default
> > > > > > > > > implementations of those expectations (for example reading
> a
> > > > > pom.xml
> > > > > > > file
> > > > > > > > > to a POM model, dumping a POM to a pom-4.0.0.xml,
> > > > > > > transforming/reducing a
> > > > > > > > > POM to POM-consumer, dumping POM-consumer to
> > > > > pom-consumer-5.0.0.xml,
> > > > > > > ...)
> > > > > > > > > and let extensions/plugins/default implementations work
> along
> > > the
> > > > > > build
> > > > > > > > > process with the API & POMs to provide different features
> and
> > > > > > > > capabilities.
> > > > > > > > >
> > > > > > > > > Matthieu
> > > > > > > > >
> > > > > > > > > On Thu, Dec 31, 2020 at 7:01 PM Robert Scholte wrote:
> > > > > > > > >
> > > > > > > > > > I've made a recording[1] about it, which hopefully
> answers
> > > most
> > > > > > > > > questions.
> > > > > > > > > >
> > > > > > > > > > Robert
> > > > > > > > > >
> > > > > > > > > > [1] https://youtu.be/KDAmlNKZJto
> > > > > > > > > >
> > > > > > > > > > On 31-12-2020 16:18:57, Matthieu Brouillard
> > > > > > > > > > wrote:
> > > > > > > > > > > Not exactly sure what work you mean
> > > > > > > > > > everything related to maven-xml:
> > > Build/ConsumerPomXMLFilterxxx,
> > > > > > > > > > Build/ConsumerModelSourcexxxx and the transformer stuff.
> > > > > > > > > > Especially, when looking at classes like
> > > CiFriendlyXMLFilter, I
> > > > > > would
> > > > > > > > > have
> > > > > > > > > > thought that such things could have been done elsewhere,
> > > > working
> > > > > on
> > > > > > > the
> > > > > > > > > > object model (not on the XML stuff) especially for the
> > > BuildPom
> > > > > > part.
> > > > > > > > > >
> > > > > > > > > > > however specifically the consumer POM integrates with
> so
> > > many
> > > > > > > > external
> > > > > > > > > > ecosystems
> > > > > > > > > > We're aligned here, this has to be stable and well
> defined
> > > by a
> > > > > > > schema.
> > > > > > > > > >
> > > > > > > > > > Matthieu
> > > > > > > > > >
> > > > > > > > > > On Thu, Dec 31, 2020 at 3:59 PM Bernd Eckenfels
> > > > > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > > Hello,
> > > > > > > > > > >
> > > > > > > > > > > Not exactly sure what work you mean and I fully agree
> > that
> > > > > using
> > > > > > a
> > > > > > > > core
> > > > > > > > > > > model should still be the API for plugins and
> extensions
> > to
> > > > > work
> > > > > > > > with,
> > > > > > > > > > > however specifically the consumer POM integrates with
> so
> > > many
> > > > > > > > external
> > > > > > > > > > > ecosystems, I would expect it to be defined in terms of
> > XML
> > > > > > Schema
> > > > > > > > with
> > > > > > > > > > > explicite semantic (and the inherent compatibility with
> > > > exiting
> > > > > > > > POMs).
> > > > > > > > > > >
> > > > > > > > > > > Gruss
> > > > > > > > > > > Bernd
> > > > > > > > > > > --
> > > > > > > > > > > http://bernd.eckenfels.net
> > > > > > > > > > > ________________________________
> > > > > > > > > > > Von: Matthieu BROUILLARD
> > > > > > > > > > > Gesendet: Thursday, December 31, 2020 3:19:09 PM
> > > > > > > > > > > An: dev@maven.apache.org
> > > > > > > > > > > Betreff: maven 4.0.0 new XML stuff
> > > > > > > > > > >
> > > > > > > > > > > Hello all,
> > > > > > > > > > >
> > > > > > > > > > > regarding the active work occurring for maven 4.0.0 I
> > > noticed
> > > > > the
> > > > > > > > > > > introduction of a lot of new stuff around SAX parsing &
> > > > > > filtering.
> > > > > > > > > > > I am wondering if that means that it was decided that
> the
> > > > input
> > > > > > > > format
> > > > > > > > > of
> > > > > > > > > > > maven projects will be XML forever meaning probably,
> > among
> > > > > > others,
> > > > > > > > the
> > > > > > > > > > end
> > > > > > > > > > > of polyglot extensions.
> > > > > > > > > > > Could you explain such a move (or point to
> > > > rationals/documents)
> > > > > > and
> > > > > > > > why
> > > > > > > > > > you
> > > > > > > > > > > did not leverage working on the in memory object model
> > > > allowing
> > > > > > > > > > > extensions/plugins to contribute/hook in the chain of
> > > > building
> > > > > > the
> > > > > > > > > > BuildPOM
> > > > > > > > > > > & ConsumePOM? In the past I really thought that this
> move
> > > to
> > > > > > 'Build
> > > > > > > > vs
> > > > > > > > > > > Consumer' POM would make clear separations between the
> > > input
> > > > > > format
> > > > > > > > of
> > > > > > > > > > > descriptors and the core system but I perhaps
> > misunderstood
> > > > > > things.
> > > > > > > > > > >
> > > > > > > > > > > Also, are there plans regarding the future of core
> > > > extensions?
> > > > > > > > > > > With core extensions it was possible to hook into the
> POM
> > > > model
> > > > > > > > loading
> > > > > > > > > > and
> > > > > > > > > > > do transformations to do dynamic changes but by working
> > on
> > > > the
> > > > > > XML
> > > > > > > > > > directly
> > > > > > > > > > > I see a shift (if not red stop) in this
> > > > contribution/delegation
> > > > > > > > > > mechanism.
> > > > > > > > > > >
> > > > > > > > > > > Thanks for your time & answers.
> > > > > > > > > > >
> > > > > > > > > > > Matthieu Brouillard
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: maven 4.0.0 new XML stuff

Posted by Matthieu Brouillard <ma...@brouillard.fr>.
 > Currently the model is mutable and this causes issues.
For me it is not the root cause, it does not help for sure but IMO the
problem is more that there are currently several sources of trust : the POM
& the pom.xml.

Personally (only my opinion) I see no other way for the future of maven
than to go to a single source of trust: the POM (as in memory model).
During the build process this POM could (and has to) become immutable for
sure so that plugins cannot do ugly things anymore but there should be also
clear loading and extension mechanisms/hooks before it becomes immutable.

> or do you push to drop extensions support?
I hope this is not the case.

On Tue, Jan 5, 2021 at 11:58 AM Romain Manni-Bucau <rm...@gmail.com>
wrote:

> Hmm, extensions define this kind of lifecycle, at least the afterModelRead
> which is a hook before it could be immutable no?
> So not sure how it changes the issue, or do you push to drop extensions
> support?
>
> Romain Manni-Bucau
> @rmannibucau <https://twitter.com/rmannibucau> |  Blog
> <https://rmannibucau.metawerx.net/> | Old Blog
> <http://rmannibucau.wordpress.com> | Github <
> https://github.com/rmannibucau> |
> LinkedIn <https://www.linkedin.com/in/rmannibucau> | Book
> <
> https://www.packtpub.com/application-development/java-ee-8-high-performance
> >
>
>
> Le mar. 5 janv. 2021 à 11:54, Robert Scholte <rf...@apache.org> a
> écrit :
>
> > Currently the model is mutable and this causes issues.
> > Instead I would like to see that once the BuildPlan is finished, the
> model
> > becomes immutable.
> >
> >
> > To give a concrete example: It must be possible for code generating goals
> > to add dependencies.
> > Now, when using modello readers/writers you often need to add a required
> > dependency by hand.
> > Ideally there will be a hook where a plugin can register additional
> > dependencies (e.g. dom4j).
> >
> > This will make any postprocessing of the pom during build obsolete
> >
> > Robert
> > On 5-1-2021 09:09:11, Matthieu Brouillard <ma...@brouillard.fr>
> wrote:
> > > Can you give an example?
> > Like Romain has shown: dynamically added dependencies or a version
> > computation.
> >
> > > Most important is the support for CI-friendly versions
> > But if extensions dynamically compute and set the versions in the POM
> using
> > the API, the changes will not be reflected.
> > That's why today one has to use flatten-maven-plugin or any other
> > plugin/task modifying/dumping the POM model to a pom.xml file and
> > setting/attaching it the POM.
> >
> > If the produced artifacts (consumer pom) were computed from the POM
> (object
> > model), every change done by any extension would be part of it.
> >
> > > but this happens AFTER using the pom
> > Not always from the pom.xml. I thought extensions were allowed to provide
> > ModelLocator & ModelReader to both decide which file to use for a project
> > and how to build the in memory POM model.
> > So the truth should not be considered to be in the pom.xml but in the
> POM.
> >
> >
> > On Mon, Jan 4, 2021 at 10:00 PM Robert Scholte wrote:
> >
> > > answers are below.
> > >
> > > Robert
> > >
> > > On 4-1-2021 16:52:23, Matthieu Brouillard wrote:
> > > @Robert nothing is broken atm, the changes for consumer/build are
> > currently
> > > behind your feature flag.
> > > Robert Scholte:
> > > It is active by default, so it is actually not hidden.
> > >
> > >
> > >
> > > But as I feared previously, and as Romain pointed it, by working at XML
> > > level (and not at POM level) the produced consumer pom does not reflect
> > > changes from extensions.
> > > Robert Scholte:
> > > Can you give an example?
> > >
> > >
> > > I really thought that all the "consumer/build" stuff would make the
> > > maven-flatten-plugin useless but it looks like it will not be the case
> if
> > > working at XML level.
> > > Robert Scholte:
> > > Like with most questions: it depends. Most important is the support for
> > > CI-friendly versions. In this case you won't need the
> > flatten-maven-plugin
> > > anymore.
> > > However, the plugin can rewrite much more, but this happens AFTER using
> > > the pom.
> > > That's something I don't like, because this POM was not used to build
> the
> > > project, but it was reassembled afterwards.
> > > My idea is to provide hooks to parts of the pom that might be adjusted,
> > > but this is something we can work on during the 4.x releases
> > >
> > > Did Romain and I miss the whole point of the "consumer/build"
> > enhancements
> > > or is it "just" because current implementation has not yet reached the
> > > targets/outputs?
> > >
> > > On Mon, Jan 4, 2021 at 2:56 PM Romain Manni-Bucau
> > > wrote:
> > >
> > > > Hmm, I don't get a few things of this IT:
> > > >
> > > > 1. the formatting seems not expected even if valid (the comments are
> > > > finishing with the first tag for example "-->
> > > > public class MyListener extends AbstractMavenLifecycleParticipant {
> > > >
> > > > @Override
> > > > public void afterProjectsRead(final MavenSession session) throws
> > > > MavenExecutionException {
> > > > if (session.getCurrentProject() == null) {
> > > > return;
> > > > }
> > > >
> > > > session.getProjects().forEach(p -> {
> > > > final Dependency dependency = new Dependency();
> > > > dependency.setGroupId("junit");
> > > > dependency.setArtifactId("junit");
> > > > dependency.setVersion("3.8.1");
> > > > p.getDependencies().add(dependency);
> > > > });
> > > > }
> > > > }
> > > >
> > > >
> > > > 2. If you run mvn (4 snapshot) dependency:tree you get this kind of
> > > output:
> > > >
> > > > [INFO] -------------
> > > > >-------------
> > > > [INFO] Building Multi Chapter Simple Web Application Project
> > > > 0.9-MNG6957-SNAPSHOT [6/6]
> > > > [INFO] --------------------------------[ jar
> > > > ]---------------------------------
> > > > [INFO]
> > > > [INFO] --- maven-dependency-plugin:3.1.2:tree (default-cli) @
> > > simple-webapp
> > > > ---
> > > > [INFO]
> > > org.sonatype.mavenbook.multi:simple-webapp:jar:0.9-MNG6957-SNAPSHOT
> > > > [INFO] +-
> > > >
> > > >
> > >
> >
> org.sonatype.mavenbook.multi:simple-weather:jar:0.9-MNG6957-SNAPSHOT:compile
> > > > [INFO] \- junit:junit:jar:3.8.1:compile
> > > >
> > > > [INFO]
> > > >
> > ------------------------------------------------------------------------
> > > >
> > > > 3. run the build to have produced pom and cat the simple-webapp one:
> > > >
> > > >
> > > > 4.0.0
> > > >
> > > > org.sonatype.mavenbook.multi
> > > > simple-parent
> > > > 0.9-MNG6957-SNAPSHOT
> > > >
> > > >
> > > > simple-webapp
> > > > Multi Chapter Simple Web Application Project
> > > >
> > > >
> > > > org.sonatype.mavenbook.multi
> > > > simple-weather
> > > > 0.9-MNG6957-SNAPSHOT
> > > >
> > > >
> > > >
> > > > simple-webapp
> > > >
> > > >
> > > >
> > > > org.apache.maven.plugins
> > > > maven-war-plugin
> > > > 2.6
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > As you see the dependency is not there. I guess the expected outout
> is:
> > > >
> > > >
> > > >
> > > > 4.0.0
> > > > simple-webapp
> > > > Multi Chapter Simple Web Application Project
> > > > [description, scm, ..., all central requires sections but not build
> > ones]
> > > >
> > > >
> > > > org.sonatype.mavenbook.multi
> > > > simple-weather
> > > > 0.9-MNG6957-SNAPSHOT
> > > >
> > > >
> > > > junit
> > > > junit
> > > > 3.8.1
> > > >
> > > >
> > > >
> > > >
> > > > Am I missing something?
> > > >
> > > > Romain Manni-Bucau
> > > > @rmannibucau | Blog
> > > > | Old Blog
> > > > | Github
> > > > https://github.com/rmannibucau> |
> > > > LinkedIn | Book
> > > >
> > > >
> > >
> >
> https://www.packtpub.com/application-development/java-ee-8-high-performance
> > > > >
> > > >
> > > >
> > > > Le lun. 4 janv. 2021 à 13:41, Robert Scholte a
> > > > écrit :
> > > >
> > > > >
> > > > >
> > > >
> > >
> >
> https://github.com/apache/maven-integration-testing/tree/master/core-it-suite/src/test/resources/mng-6957-buildconsumer
> > > > > is the most complete IT
> > > > >
> > > > > On 4-1-2021 12:59:51, Romain Manni-Bucau wrote:
> > > > > Le lun. 4 janv. 2021 à 12:36, Robert Scholte a
> > > > > écrit :
> > > > >
> > > > > > There's just one thing I want to say:
> > > > > > I'm having trouble with the term "broken".
> > > > > >
> > > > >
> > > > > Well, literally meant broken as decorelated from the user intent
> and
> > > > > extension model.
> > > > > Anyway, didn't intend to blame but more identify the blockers for a
> > GA
> > > so
> > > > > point was really that it seem that on the two sides only the
> > producing
> > > > one
> > > > > is not yet ready since it keeps does not sanitize the model
> > completely
> > > > and
> > > > > keeps build only data like comments, right? Also not yet clear for
> me
> > > if
> > > > we
> > > > > loose the extension enrichments there.
> > > > >
> > > > >
> > > > > > If a Maven project could be built with M3.6.3, it can still be
> > built
> > > > with
> > > > > > M4.
> > > > > > If not, it is either regression (MNG-6957, MNG-7063) which must
> be
> > > > fixed,
> > > > > > or it requires changes to a plugin for understandable reasons
> > > > > > (maven-pgp-plugin)
> > > > > > AFAIK an interesting extension like the maven-tiles has been
> tested
> > > and
> > > > > > still works.
> > > > > >
> > > > >
> > > > > Do you have this handy, is it in our test suite? I'd like to check
> > the
> > > > > produced pom matches the enriched model but happy to start from
> > > something
> > > > > already there.
> > > > >
> > > > >
> > > > > >
> > > > > > Robert
> > > > > >
> > > > > > On 3-1-2021 19:35:25, Romain Manni-Bucau wrote:
> > > > > > Le dim. 3 janv. 2021 à 19:04, Robert Scholte a
> > > > > > écrit :
> > > > > >
> > > > > > > I don't remember all those details anymore, because I hit those
> > in
> > > > the
> > > > > > > beginning.
> > > > > > > Trying things over and over again I decided that this is
> probably
> > > the
> > > > > > most
> > > > > > > successful approach.
> > > > > > >
> > > > > > >
> > > > > > > What of the goals was to keep the pom.xml as is as much as
> > > possible.
> > > > > > > We can only decide for the specific Maven elements how to
> handle
> > > > them,
> > > > > we
> > > > > > > should not decide about comments and licenses.
> > > > > > > BTW, the license issue was hard to solve. You cannot use it
> from
> > > the
> > > > > > pom's
> > > > > > > , because there might be multiple licenses.
> > > > > > >
> > > > > >
> > > > > > I disagree, it is saner IMO to evolve to support that than doing
> > > > > > anything else.
> > > > > > Once again you keep things which don't make sense in a consumed
> pom
> > > in
> > > > > > current impl so i'd say the sucess in a few cases breaks as much
> > > cases
> > > > so
> > > > > > we need to revisit anyway IMHO.
> > > > > >
> > > > > >
> > > > > > >
> > > > > > > The current implementation is a solid way to ensure we're not
> > > > breaking
> > > > > > too
> > > > > > > much, because Maven controls the XML filters.
> > > > > > >
> > > > > >
> > > > > > Hmm, breaking extensions seems to break too much (I'm not
> speaking
> > of
> > > > > other
> > > > > > parts which breaks the ecosystem there but just that is
> sufficient
> > > IMHO
> > > > > to
> > > > > > say we must check back our solution).
> > > > > >
> > > > > >
> > > > > > >
> > > > > > > Also keep in mind, that I only want Maven to decide which
> > > > modifications
> > > > > > > are done.
> > > > > > >
> > > > > >
> > > > > > For the consumed pom I agree but it is consistent with keeping
> > > > everything
> > > > > > working instead of breaking too, no?
> > > > > >
> > > > > >
> > > > > > >
> > > > > > > Current polyglot projects should still work, but they cannot
> > > benefit
> > > > > from
> > > > > > > the build/consumer functions yet.
> > > > > > >
> > > > > >
> > > > > > So pom -> build model is kept, build model -> produced pom is
> > broken?
> > > > Is
> > > > > it
> > > > > > the complete status?
> > > > > > Sounds ok for a 4.0 and a 4.1 can fix it if so.
> > > > > > Just want to ensure first part is not broken at all.
> > > > > >
> > > > > >
> > > > > > >
> > > > > > > Robert
> > > > > > > On 3-1-2021 16:38:38, Romain Manni-Bucau wrote:
> > > > > > > Le dim. 3 janv. 2021 à 16:18, Robert Scholte a
> > > > > > > écrit :
> > > > > > >
> > > > > > > > > So what I was expecting was: raw xml model -> converted to
> > > > unified
> > > > > > > > consumed model -> extensions -> model processing.
> > > > > > > >
> > > > > > > > This is only the build pom part. You're missing the consume
> > part,
> > > > > where
> > > > > > > > the xml is distributed.
> > > > > > > >
> > > > > > >
> > > > > > > Yes but with previous chain the consume part is
> "clean/normalize
> > ->
> > > > > dump"
> > > > > > > since we are using consumed model - only standard model - in
> > memory
> > > > > > > already.
> > > > > > >
> > > > > > >
> > > > > > > > Build is raw + enrich, consumer is raw + enrich + reduce
> > > (removing
> > > > > > > > relativePath and modules are the first examples, but much
> more
> > is
> > > > > > > possible)
> > > > > > > >
> > > > > > > > Going for the in memory was also my first thought, but I
> would
> > > > loose
> > > > > > > > information, hence I came up with the current implementation.
> > > > > > > >
> > > > > > >
> > > > > > > I don't see what you loose ot be honest.
> > > > > > > You mentionned license but this one is in the pom so not a big
> > > deal,
> > > > > > > comments which are undesired IMHO as mentionned and element
> order
> > > > which
> > > > > > can
> > > > > > > really be discussed since we can desire to enforce an order to
> > > > > normalize
> > > > > > > consumption + it shouldn't be important since from the project
> > > point
> > > > of
> > > > > > > view your pom is already "broken"/lost (as all your
> intelligence
> > is
> > > > > lost
> > > > > > by
> > > > > > > this "not passthrough" process).
> > > > > > > So overall I don't see what you would loose from the consumer
> > side
> > > > but
> > > > > I
> > > > > > > see what you lost from maven ecosystem side.
> > > > > > >
> > > > > > >
> > > > > > > > Again, we're at a point where we can have counter solutions,
> > but
> > > > > don't
> > > > > > > > expect me to implement it.
> > > > > > > >
> > > > > > >
> > > > > > > For now I'm just trying to ensure we agree we don't want to
> break
> > > > > > existing
> > > > > > > extensions and the nice ecosystem we built after years.
> > > > > > > This was really a move forward and it sounds like we broke it
> at
> > > > maven
> > > > > 4
> > > > > > > without any user gain which sounds terrible.
> > > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > thanks,
> > > > > > > > Robert
> > > > > > > > On 3-1-2021 15:25:21, Romain Manni-Bucau wrote:
> > > > > > > > Hi all,
> > > > > > > >
> > > > > > > > I kind of join Matthieu thoughts there, there is no point to
> > work
> > > > at
> > > > > > xml
> > > > > > > > level to create the consumed pom - comments is not a point
> > since
> > > it
> > > > > can
> > > > > > > > commonly/easily refer to a dropped part of the pom so they
> > should
> > > > be
> > > > > > > > stripped.
> > > > > > > > Current extension model got proven adapted and adopted,
> using a
> > > > lower
> > > > > > > level
> > > > > > > > extension API will not since XML is, even if still
> mainstream,
> > > > often
> > > > > > > > replaced by alternative configurations and to have done the
> > work
> > > to
> > > > > > > inject
> > > > > > > > XML configuration programmatically compred to current option,
> > it
> > > > is a
> > > > > > > pain.
> > > > > > > > The in memory model should stick to consumed model IMHO -
> being
> > > > > > > > programmatic there is no point to make it easier, worse case
> we
> > > can
> > > > > add
> > > > > > > > helper beans (injectable) but in terms of model it will not
> > help.
> > > > > > > >
> > > > > > > > So what I was expecting was:
> > > > > > > >
> > > > > > > > raw xml model -> converted to unified consumed model ->
> > > extensions
> > > > ->
> > > > > > > model
> > > > > > > > processing.
> > > > > > > >
> > > > > > > > Indeed, real chain adds a small processing over the first
> arrow
> > > > > (inject
> > > > > > > > versions for example) but nothing crazy and breaking this
> > overall
> > > > > flow
> > > > > > > > which stays user friendly.
> > > > > > > >
> > > > > > > > Strictly speaking the new model is just a built-in extension
> > for
> > > me
> > > > > > which
> > > > > > > > is particular because it will enforce IDE to integrate a new
> > > > format -
> > > > > > > > wheres polyglot extensions or others don't require static
> > > analyzis
> > > > by
> > > > > > > > themself not being "standard".
> > > > > > > >
> > > > > > > > That said, there is nothing crazy with current
> implementation,
> > it
> > > > > just
> > > > > > > > require to be updated to be able to take extension changes
> into
> > > > > > account.
> > > > > > > > This can be done by making the extension model 'spyable' (ie
> > if a
> > > > > > > > dependency/plugin is added it will be reflected in the final
> > > > written
> > > > > > > > pom.xml).
> > > > > > > > This sounds - instrumenting the extension model API or doing
> a
> > > diff
> > > > > > after
> > > > > > > > extension phase - like a compromise and let people gets the
> > best
> > > of
> > > > > > both
> > > > > > > > worlds to me.
> > > > > > > >
> > > > > > > > Wdyt?
> > > > > > > >
> > > > > > > > Romain Manni-Bucau
> > > > > > > > @rmannibucau | Blog
> > > > > > > > | Old Blog
> > > > > > > > | Github |
> > > > > > > > LinkedIn | Book
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > Le dim. 3 janv. 2021 à 14:46, Robert Scholte a
> > > > > > > > écrit :
> > > > > > > >
> > > > > > > > > Hi Matthieu,
> > > > > > > > >
> > > > > > > > > As you understand, something had to be changed to move
> Maven
> > > > > forward.
> > > > > > > > > I've decided to pick up that challenge and came up with the
> > > > current
> > > > > > > > > solution.
> > > > > > > > >
> > > > > > > > > My main concerns was that I wanted to keep the fileModel as
> > > much
> > > > as
> > > > > > is.
> > > > > > > > > That includes the license, comments and element order.
> > > > > > > > > This information if not available in the memory model, so I
> > > > needed
> > > > > > the
> > > > > > > > > original pom file.
> > > > > > > > > With that in mind, the usage of XMLFilters looks like the
> > most
> > > > > > > > appropriate
> > > > > > > > > solution.
> > > > > > > > >
> > > > > > > > > I am certain that XML is still the most used format, so if
> we
> > > can
> > > > > > have
> > > > > > > > > improvements for those users, I'm already very happy.
> > > > > > > > >
> > > > > > > > > And yes, there are plugins that needs to be updated, but
> > doing
> > > > > > nothing
> > > > > > > is
> > > > > > > > > not an option anymore.
> > > > > > > > >
> > > > > > > > > There are more people that share their concerns, but it
> took
> > me
> > > > > > several
> > > > > > > > > years to reach this point.
> > > > > > > > > We now have something that seems to work, anybody who can
> > > improve
> > > > > or
> > > > > > > can
> > > > > > > > > come up with an alternative implementation can do so.
> > > > > > > > >
> > > > > > > > > thanks,
> > > > > > > > > Robert
> > > > > > > > > On 3-1-2021 12:55:41, Matthieu Brouillard wrote:
> > > > > > > > > Thanks Robert for the video link.
> > > > > > > > >
> > > > > > > > > I fully understand the rationales behind the separation of
> > > > > > > > > build/consumer pom and the video provides some insights on
> it
> > > and
> > > > > you
> > > > > > > > > explain the actual implementation to introduce this change.
> > > > > > > > > Still I do not fully understand why it was decided to work
> on
> > > top
> > > > > of
> > > > > > > XML
> > > > > > > > by
> > > > > > > > > filtering/enhancing it instead of working at the POM (in
> > > > > > > > > memory datamodel) level.
> > > > > > > > > With the current understanding I have, by doing this choice
> > of
> > > > > > working
> > > > > > > at
> > > > > > > > > XML level, it looks like it was decided to bypass (if not
> > kill)
> > > > > core
> > > > > > > > > extensions that enhance the POM itself and not the pom.xml
> ;
> > > > here I
> > > > > > can
> > > > > > > > > think of (but probably not limited to):
> > > > > > > > > - polyglot-maven: do not use XML but other format to
> describe
> > > the
> > > > > POM
> > > > > > > > > (yaml, json, kotlin, java, other XML formats, ...)
> > > > > > > > > - jgitver-maven-plugin (or forks like
> > > > > > maven-git-versioning-extension):
> > > > > > > > > dynamic computation of projects version based on git
> history
> > > > > > > > >
> > > > > > > > > With the introduction of core extensions, I thought it was
> a
> > > move
> > > > > to
> > > > > > > open
> > > > > > > > > the internals and let externals contribute to the
> > capabilities
> > > of
> > > > > > > maven.
> > > > > > > > > With the move to a XML handling chain, I see it as a
> > > > > > > restriction/regress
> > > > > > > > in
> > > > > > > > > favor of core closed functionalities. An example of that is
> > > what
> > > > is
> > > > > > > > > provided as CIFriendly stuff, IMO it could/should have been
> > > > > provided
> > > > > > > by a
> > > > > > > > > plugin/extension but instead it is hard written in maven
> core
> > > and
> > > > > is
> > > > > > > not
> > > > > > > > > opened for external contribution (plugin/extension I mean).
> > > > > > > > >
> > > > > > > > > Perhaps I am totally wrong but I think that maven core
> should
> > > > > define
> > > > > > > all
> > > > > > > > > its expectations at an API level so that extensions/plugins
> > > could
> > > > > > hook
> > > > > > > at
> > > > > > > > > this API level. The default packaging of maven could/should
> > > > provide
> > > > > > > > default
> > > > > > > > > implementations of those expectations (for example reading
> a
> > > > > pom.xml
> > > > > > > file
> > > > > > > > > to a POM model, dumping a POM to a pom-4.0.0.xml,
> > > > > > > transforming/reducing a
> > > > > > > > > POM to POM-consumer, dumping POM-consumer to
> > > > > pom-consumer-5.0.0.xml,
> > > > > > > ...)
> > > > > > > > > and let extensions/plugins/default implementations work
> along
> > > the
> > > > > > build
> > > > > > > > > process with the API & POMs to provide different features
> and
> > > > > > > > capabilities.
> > > > > > > > >
> > > > > > > > > Matthieu
> > > > > > > > >
> > > > > > > > > On Thu, Dec 31, 2020 at 7:01 PM Robert Scholte wrote:
> > > > > > > > >
> > > > > > > > > > I've made a recording[1] about it, which hopefully
> answers
> > > most
> > > > > > > > > questions.
> > > > > > > > > >
> > > > > > > > > > Robert
> > > > > > > > > >
> > > > > > > > > > [1] https://youtu.be/KDAmlNKZJto
> > > > > > > > > >
> > > > > > > > > > On 31-12-2020 16:18:57, Matthieu Brouillard
> > > > > > > > > > wrote:
> > > > > > > > > > > Not exactly sure what work you mean
> > > > > > > > > > everything related to maven-xml:
> > > Build/ConsumerPomXMLFilterxxx,
> > > > > > > > > > Build/ConsumerModelSourcexxxx and the transformer stuff.
> > > > > > > > > > Especially, when looking at classes like
> > > CiFriendlyXMLFilter, I
> > > > > > would
> > > > > > > > > have
> > > > > > > > > > thought that such things could have been done elsewhere,
> > > > working
> > > > > on
> > > > > > > the
> > > > > > > > > > object model (not on the XML stuff) especially for the
> > > BuildPom
> > > > > > part.
> > > > > > > > > >
> > > > > > > > > > > however specifically the consumer POM integrates with
> so
> > > many
> > > > > > > > external
> > > > > > > > > > ecosystems
> > > > > > > > > > We're aligned here, this has to be stable and well
> defined
> > > by a
> > > > > > > schema.
> > > > > > > > > >
> > > > > > > > > > Matthieu
> > > > > > > > > >
> > > > > > > > > > On Thu, Dec 31, 2020 at 3:59 PM Bernd Eckenfels
> > > > > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > > Hello,
> > > > > > > > > > >
> > > > > > > > > > > Not exactly sure what work you mean and I fully agree
> > that
> > > > > using
> > > > > > a
> > > > > > > > core
> > > > > > > > > > > model should still be the API for plugins and
> extensions
> > to
> > > > > work
> > > > > > > > with,
> > > > > > > > > > > however specifically the consumer POM integrates with
> so
> > > many
> > > > > > > > external
> > > > > > > > > > > ecosystems, I would expect it to be defined in terms of
> > XML
> > > > > > Schema
> > > > > > > > with
> > > > > > > > > > > explicite semantic (and the inherent compatibility with
> > > > exiting
> > > > > > > > POMs).
> > > > > > > > > > >
> > > > > > > > > > > Gruss
> > > > > > > > > > > Bernd
> > > > > > > > > > > --
> > > > > > > > > > > http://bernd.eckenfels.net
> > > > > > > > > > > ________________________________
> > > > > > > > > > > Von: Matthieu BROUILLARD
> > > > > > > > > > > Gesendet: Thursday, December 31, 2020 3:19:09 PM
> > > > > > > > > > > An: dev@maven.apache.org
> > > > > > > > > > > Betreff: maven 4.0.0 new XML stuff
> > > > > > > > > > >
> > > > > > > > > > > Hello all,
> > > > > > > > > > >
> > > > > > > > > > > regarding the active work occurring for maven 4.0.0 I
> > > noticed
> > > > > the
> > > > > > > > > > > introduction of a lot of new stuff around SAX parsing &
> > > > > > filtering.
> > > > > > > > > > > I am wondering if that means that it was decided that
> the
> > > > input
> > > > > > > > format
> > > > > > > > > of
> > > > > > > > > > > maven projects will be XML forever meaning probably,
> > among
> > > > > > others,
> > > > > > > > the
> > > > > > > > > > end
> > > > > > > > > > > of polyglot extensions.
> > > > > > > > > > > Could you explain such a move (or point to
> > > > rationals/documents)
> > > > > > and
> > > > > > > > why
> > > > > > > > > > you
> > > > > > > > > > > did not leverage working on the in memory object model
> > > > allowing
> > > > > > > > > > > extensions/plugins to contribute/hook in the chain of
> > > > building
> > > > > > the
> > > > > > > > > > BuildPOM
> > > > > > > > > > > & ConsumePOM? In the past I really thought that this
> move
> > > to
> > > > > > 'Build
> > > > > > > > vs
> > > > > > > > > > > Consumer' POM would make clear separations between the
> > > input
> > > > > > format
> > > > > > > > of
> > > > > > > > > > > descriptors and the core system but I perhaps
> > misunderstood
> > > > > > things.
> > > > > > > > > > >
> > > > > > > > > > > Also, are there plans regarding the future of core
> > > > extensions?
> > > > > > > > > > > With core extensions it was possible to hook into the
> POM
> > > > model
> > > > > > > > loading
> > > > > > > > > > and
> > > > > > > > > > > do transformations to do dynamic changes but by working
> > on
> > > > the
> > > > > > XML
> > > > > > > > > > directly
> > > > > > > > > > > I see a shift (if not red stop) in this
> > > > contribution/delegation
> > > > > > > > > > mechanism.
> > > > > > > > > > >
> > > > > > > > > > > Thanks for your time & answers.
> > > > > > > > > > >
> > > > > > > > > > > Matthieu Brouillard
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: maven 4.0.0 new XML stuff

Posted by Romain Manni-Bucau <rm...@gmail.com>.
Hmm, extensions define this kind of lifecycle, at least the afterModelRead
which is a hook before it could be immutable no?
So not sure how it changes the issue, or do you push to drop extensions
support?

Romain Manni-Bucau
@rmannibucau <https://twitter.com/rmannibucau> |  Blog
<https://rmannibucau.metawerx.net/> | Old Blog
<http://rmannibucau.wordpress.com> | Github <https://github.com/rmannibucau> |
LinkedIn <https://www.linkedin.com/in/rmannibucau> | Book
<https://www.packtpub.com/application-development/java-ee-8-high-performance>


Le mar. 5 janv. 2021 à 11:54, Robert Scholte <rf...@apache.org> a
écrit :

> Currently the model is mutable and this causes issues.
> Instead I would like to see that once the BuildPlan is finished, the model
> becomes immutable.
>
>
> To give a concrete example: It must be possible for code generating goals
> to add dependencies.
> Now, when using modello readers/writers you often need to add a required
> dependency by hand.
> Ideally there will be a hook where a plugin can register additional
> dependencies (e.g. dom4j).
>
> This will make any postprocessing of the pom during build obsolete
>
> Robert
> On 5-1-2021 09:09:11, Matthieu Brouillard <ma...@brouillard.fr> wrote:
> > Can you give an example?
> Like Romain has shown: dynamically added dependencies or a version
> computation.
>
> > Most important is the support for CI-friendly versions
> But if extensions dynamically compute and set the versions in the POM using
> the API, the changes will not be reflected.
> That's why today one has to use flatten-maven-plugin or any other
> plugin/task modifying/dumping the POM model to a pom.xml file and
> setting/attaching it the POM.
>
> If the produced artifacts (consumer pom) were computed from the POM (object
> model), every change done by any extension would be part of it.
>
> > but this happens AFTER using the pom
> Not always from the pom.xml. I thought extensions were allowed to provide
> ModelLocator & ModelReader to both decide which file to use for a project
> and how to build the in memory POM model.
> So the truth should not be considered to be in the pom.xml but in the POM.
>
>
> On Mon, Jan 4, 2021 at 10:00 PM Robert Scholte wrote:
>
> > answers are below.
> >
> > Robert
> >
> > On 4-1-2021 16:52:23, Matthieu Brouillard wrote:
> > @Robert nothing is broken atm, the changes for consumer/build are
> currently
> > behind your feature flag.
> > Robert Scholte:
> > It is active by default, so it is actually not hidden.
> >
> >
> >
> > But as I feared previously, and as Romain pointed it, by working at XML
> > level (and not at POM level) the produced consumer pom does not reflect
> > changes from extensions.
> > Robert Scholte:
> > Can you give an example?
> >
> >
> > I really thought that all the "consumer/build" stuff would make the
> > maven-flatten-plugin useless but it looks like it will not be the case if
> > working at XML level.
> > Robert Scholte:
> > Like with most questions: it depends. Most important is the support for
> > CI-friendly versions. In this case you won't need the
> flatten-maven-plugin
> > anymore.
> > However, the plugin can rewrite much more, but this happens AFTER using
> > the pom.
> > That's something I don't like, because this POM was not used to build the
> > project, but it was reassembled afterwards.
> > My idea is to provide hooks to parts of the pom that might be adjusted,
> > but this is something we can work on during the 4.x releases
> >
> > Did Romain and I miss the whole point of the "consumer/build"
> enhancements
> > or is it "just" because current implementation has not yet reached the
> > targets/outputs?
> >
> > On Mon, Jan 4, 2021 at 2:56 PM Romain Manni-Bucau
> > wrote:
> >
> > > Hmm, I don't get a few things of this IT:
> > >
> > > 1. the formatting seems not expected even if valid (the comments are
> > > finishing with the first tag for example "-->
> > > public class MyListener extends AbstractMavenLifecycleParticipant {
> > >
> > > @Override
> > > public void afterProjectsRead(final MavenSession session) throws
> > > MavenExecutionException {
> > > if (session.getCurrentProject() == null) {
> > > return;
> > > }
> > >
> > > session.getProjects().forEach(p -> {
> > > final Dependency dependency = new Dependency();
> > > dependency.setGroupId("junit");
> > > dependency.setArtifactId("junit");
> > > dependency.setVersion("3.8.1");
> > > p.getDependencies().add(dependency);
> > > });
> > > }
> > > }
> > >
> > >
> > > 2. If you run mvn (4 snapshot) dependency:tree you get this kind of
> > output:
> > >
> > > [INFO] -------------
> > > >-------------
> > > [INFO] Building Multi Chapter Simple Web Application Project
> > > 0.9-MNG6957-SNAPSHOT [6/6]
> > > [INFO] --------------------------------[ jar
> > > ]---------------------------------
> > > [INFO]
> > > [INFO] --- maven-dependency-plugin:3.1.2:tree (default-cli) @
> > simple-webapp
> > > ---
> > > [INFO]
> > org.sonatype.mavenbook.multi:simple-webapp:jar:0.9-MNG6957-SNAPSHOT
> > > [INFO] +-
> > >
> > >
> >
> org.sonatype.mavenbook.multi:simple-weather:jar:0.9-MNG6957-SNAPSHOT:compile
> > > [INFO] \- junit:junit:jar:3.8.1:compile
> > >
> > > [INFO]
> > >
> ------------------------------------------------------------------------
> > >
> > > 3. run the build to have produced pom and cat the simple-webapp one:
> > >
> > >
> > > 4.0.0
> > >
> > > org.sonatype.mavenbook.multi
> > > simple-parent
> > > 0.9-MNG6957-SNAPSHOT
> > >
> > >
> > > simple-webapp
> > > Multi Chapter Simple Web Application Project
> > >
> > >
> > > org.sonatype.mavenbook.multi
> > > simple-weather
> > > 0.9-MNG6957-SNAPSHOT
> > >
> > >
> > >
> > > simple-webapp
> > >
> > >
> > >
> > > org.apache.maven.plugins
> > > maven-war-plugin
> > > 2.6
> > >
> > >
> > >
> > >
> > >
> > >
> > > As you see the dependency is not there. I guess the expected outout is:
> > >
> > >
> > >
> > > 4.0.0
> > > simple-webapp
> > > Multi Chapter Simple Web Application Project
> > > [description, scm, ..., all central requires sections but not build
> ones]
> > >
> > >
> > > org.sonatype.mavenbook.multi
> > > simple-weather
> > > 0.9-MNG6957-SNAPSHOT
> > >
> > >
> > > junit
> > > junit
> > > 3.8.1
> > >
> > >
> > >
> > >
> > > Am I missing something?
> > >
> > > Romain Manni-Bucau
> > > @rmannibucau | Blog
> > > | Old Blog
> > > | Github
> > > https://github.com/rmannibucau> |
> > > LinkedIn | Book
> > >
> > >
> >
> https://www.packtpub.com/application-development/java-ee-8-high-performance
> > > >
> > >
> > >
> > > Le lun. 4 janv. 2021 à 13:41, Robert Scholte a
> > > écrit :
> > >
> > > >
> > > >
> > >
> >
> https://github.com/apache/maven-integration-testing/tree/master/core-it-suite/src/test/resources/mng-6957-buildconsumer
> > > > is the most complete IT
> > > >
> > > > On 4-1-2021 12:59:51, Romain Manni-Bucau wrote:
> > > > Le lun. 4 janv. 2021 à 12:36, Robert Scholte a
> > > > écrit :
> > > >
> > > > > There's just one thing I want to say:
> > > > > I'm having trouble with the term "broken".
> > > > >
> > > >
> > > > Well, literally meant broken as decorelated from the user intent and
> > > > extension model.
> > > > Anyway, didn't intend to blame but more identify the blockers for a
> GA
> > so
> > > > point was really that it seem that on the two sides only the
> producing
> > > one
> > > > is not yet ready since it keeps does not sanitize the model
> completely
> > > and
> > > > keeps build only data like comments, right? Also not yet clear for me
> > if
> > > we
> > > > loose the extension enrichments there.
> > > >
> > > >
> > > > > If a Maven project could be built with M3.6.3, it can still be
> built
> > > with
> > > > > M4.
> > > > > If not, it is either regression (MNG-6957, MNG-7063) which must be
> > > fixed,
> > > > > or it requires changes to a plugin for understandable reasons
> > > > > (maven-pgp-plugin)
> > > > > AFAIK an interesting extension like the maven-tiles has been tested
> > and
> > > > > still works.
> > > > >
> > > >
> > > > Do you have this handy, is it in our test suite? I'd like to check
> the
> > > > produced pom matches the enriched model but happy to start from
> > something
> > > > already there.
> > > >
> > > >
> > > > >
> > > > > Robert
> > > > >
> > > > > On 3-1-2021 19:35:25, Romain Manni-Bucau wrote:
> > > > > Le dim. 3 janv. 2021 à 19:04, Robert Scholte a
> > > > > écrit :
> > > > >
> > > > > > I don't remember all those details anymore, because I hit those
> in
> > > the
> > > > > > beginning.
> > > > > > Trying things over and over again I decided that this is probably
> > the
> > > > > most
> > > > > > successful approach.
> > > > > >
> > > > > >
> > > > > > What of the goals was to keep the pom.xml as is as much as
> > possible.
> > > > > > We can only decide for the specific Maven elements how to handle
> > > them,
> > > > we
> > > > > > should not decide about comments and licenses.
> > > > > > BTW, the license issue was hard to solve. You cannot use it from
> > the
> > > > > pom's
> > > > > > , because there might be multiple licenses.
> > > > > >
> > > > >
> > > > > I disagree, it is saner IMO to evolve to support that than doing
> > > > > anything else.
> > > > > Once again you keep things which don't make sense in a consumed pom
> > in
> > > > > current impl so i'd say the sucess in a few cases breaks as much
> > cases
> > > so
> > > > > we need to revisit anyway IMHO.
> > > > >
> > > > >
> > > > > >
> > > > > > The current implementation is a solid way to ensure we're not
> > > breaking
> > > > > too
> > > > > > much, because Maven controls the XML filters.
> > > > > >
> > > > >
> > > > > Hmm, breaking extensions seems to break too much (I'm not speaking
> of
> > > > other
> > > > > parts which breaks the ecosystem there but just that is sufficient
> > IMHO
> > > > to
> > > > > say we must check back our solution).
> > > > >
> > > > >
> > > > > >
> > > > > > Also keep in mind, that I only want Maven to decide which
> > > modifications
> > > > > > are done.
> > > > > >
> > > > >
> > > > > For the consumed pom I agree but it is consistent with keeping
> > > everything
> > > > > working instead of breaking too, no?
> > > > >
> > > > >
> > > > > >
> > > > > > Current polyglot projects should still work, but they cannot
> > benefit
> > > > from
> > > > > > the build/consumer functions yet.
> > > > > >
> > > > >
> > > > > So pom -> build model is kept, build model -> produced pom is
> broken?
> > > Is
> > > > it
> > > > > the complete status?
> > > > > Sounds ok for a 4.0 and a 4.1 can fix it if so.
> > > > > Just want to ensure first part is not broken at all.
> > > > >
> > > > >
> > > > > >
> > > > > > Robert
> > > > > > On 3-1-2021 16:38:38, Romain Manni-Bucau wrote:
> > > > > > Le dim. 3 janv. 2021 à 16:18, Robert Scholte a
> > > > > > écrit :
> > > > > >
> > > > > > > > So what I was expecting was: raw xml model -> converted to
> > > unified
> > > > > > > consumed model -> extensions -> model processing.
> > > > > > >
> > > > > > > This is only the build pom part. You're missing the consume
> part,
> > > > where
> > > > > > > the xml is distributed.
> > > > > > >
> > > > > >
> > > > > > Yes but with previous chain the consume part is "clean/normalize
> ->
> > > > dump"
> > > > > > since we are using consumed model - only standard model - in
> memory
> > > > > > already.
> > > > > >
> > > > > >
> > > > > > > Build is raw + enrich, consumer is raw + enrich + reduce
> > (removing
> > > > > > > relativePath and modules are the first examples, but much more
> is
> > > > > > possible)
> > > > > > >
> > > > > > > Going for the in memory was also my first thought, but I would
> > > loose
> > > > > > > information, hence I came up with the current implementation.
> > > > > > >
> > > > > >
> > > > > > I don't see what you loose ot be honest.
> > > > > > You mentionned license but this one is in the pom so not a big
> > deal,
> > > > > > comments which are undesired IMHO as mentionned and element order
> > > which
> > > > > can
> > > > > > really be discussed since we can desire to enforce an order to
> > > > normalize
> > > > > > consumption + it shouldn't be important since from the project
> > point
> > > of
> > > > > > view your pom is already "broken"/lost (as all your intelligence
> is
> > > > lost
> > > > > by
> > > > > > this "not passthrough" process).
> > > > > > So overall I don't see what you would loose from the consumer
> side
> > > but
> > > > I
> > > > > > see what you lost from maven ecosystem side.
> > > > > >
> > > > > >
> > > > > > > Again, we're at a point where we can have counter solutions,
> but
> > > > don't
> > > > > > > expect me to implement it.
> > > > > > >
> > > > > >
> > > > > > For now I'm just trying to ensure we agree we don't want to break
> > > > > existing
> > > > > > extensions and the nice ecosystem we built after years.
> > > > > > This was really a move forward and it sounds like we broke it at
> > > maven
> > > > 4
> > > > > > without any user gain which sounds terrible.
> > > > > >
> > > > > >
> > > > > > >
> > > > > > > thanks,
> > > > > > > Robert
> > > > > > > On 3-1-2021 15:25:21, Romain Manni-Bucau wrote:
> > > > > > > Hi all,
> > > > > > >
> > > > > > > I kind of join Matthieu thoughts there, there is no point to
> work
> > > at
> > > > > xml
> > > > > > > level to create the consumed pom - comments is not a point
> since
> > it
> > > > can
> > > > > > > commonly/easily refer to a dropped part of the pom so they
> should
> > > be
> > > > > > > stripped.
> > > > > > > Current extension model got proven adapted and adopted, using a
> > > lower
> > > > > > level
> > > > > > > extension API will not since XML is, even if still mainstream,
> > > often
> > > > > > > replaced by alternative configurations and to have done the
> work
> > to
> > > > > > inject
> > > > > > > XML configuration programmatically compred to current option,
> it
> > > is a
> > > > > > pain.
> > > > > > > The in memory model should stick to consumed model IMHO - being
> > > > > > > programmatic there is no point to make it easier, worse case we
> > can
> > > > add
> > > > > > > helper beans (injectable) but in terms of model it will not
> help.
> > > > > > >
> > > > > > > So what I was expecting was:
> > > > > > >
> > > > > > > raw xml model -> converted to unified consumed model ->
> > extensions
> > > ->
> > > > > > model
> > > > > > > processing.
> > > > > > >
> > > > > > > Indeed, real chain adds a small processing over the first arrow
> > > > (inject
> > > > > > > versions for example) but nothing crazy and breaking this
> overall
> > > > flow
> > > > > > > which stays user friendly.
> > > > > > >
> > > > > > > Strictly speaking the new model is just a built-in extension
> for
> > me
> > > > > which
> > > > > > > is particular because it will enforce IDE to integrate a new
> > > format -
> > > > > > > wheres polyglot extensions or others don't require static
> > analyzis
> > > by
> > > > > > > themself not being "standard".
> > > > > > >
> > > > > > > That said, there is nothing crazy with current implementation,
> it
> > > > just
> > > > > > > require to be updated to be able to take extension changes into
> > > > > account.
> > > > > > > This can be done by making the extension model 'spyable' (ie
> if a
> > > > > > > dependency/plugin is added it will be reflected in the final
> > > written
> > > > > > > pom.xml).
> > > > > > > This sounds - instrumenting the extension model API or doing a
> > diff
> > > > > after
> > > > > > > extension phase - like a compromise and let people gets the
> best
> > of
> > > > > both
> > > > > > > worlds to me.
> > > > > > >
> > > > > > > Wdyt?
> > > > > > >
> > > > > > > Romain Manni-Bucau
> > > > > > > @rmannibucau | Blog
> > > > > > > | Old Blog
> > > > > > > | Github |
> > > > > > > LinkedIn | Book
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > Le dim. 3 janv. 2021 à 14:46, Robert Scholte a
> > > > > > > écrit :
> > > > > > >
> > > > > > > > Hi Matthieu,
> > > > > > > >
> > > > > > > > As you understand, something had to be changed to move Maven
> > > > forward.
> > > > > > > > I've decided to pick up that challenge and came up with the
> > > current
> > > > > > > > solution.
> > > > > > > >
> > > > > > > > My main concerns was that I wanted to keep the fileModel as
> > much
> > > as
> > > > > is.
> > > > > > > > That includes the license, comments and element order.
> > > > > > > > This information if not available in the memory model, so I
> > > needed
> > > > > the
> > > > > > > > original pom file.
> > > > > > > > With that in mind, the usage of XMLFilters looks like the
> most
> > > > > > > appropriate
> > > > > > > > solution.
> > > > > > > >
> > > > > > > > I am certain that XML is still the most used format, so if we
> > can
> > > > > have
> > > > > > > > improvements for those users, I'm already very happy.
> > > > > > > >
> > > > > > > > And yes, there are plugins that needs to be updated, but
> doing
> > > > > nothing
> > > > > > is
> > > > > > > > not an option anymore.
> > > > > > > >
> > > > > > > > There are more people that share their concerns, but it took
> me
> > > > > several
> > > > > > > > years to reach this point.
> > > > > > > > We now have something that seems to work, anybody who can
> > improve
> > > > or
> > > > > > can
> > > > > > > > come up with an alternative implementation can do so.
> > > > > > > >
> > > > > > > > thanks,
> > > > > > > > Robert
> > > > > > > > On 3-1-2021 12:55:41, Matthieu Brouillard wrote:
> > > > > > > > Thanks Robert for the video link.
> > > > > > > >
> > > > > > > > I fully understand the rationales behind the separation of
> > > > > > > > build/consumer pom and the video provides some insights on it
> > and
> > > > you
> > > > > > > > explain the actual implementation to introduce this change.
> > > > > > > > Still I do not fully understand why it was decided to work on
> > top
> > > > of
> > > > > > XML
> > > > > > > by
> > > > > > > > filtering/enhancing it instead of working at the POM (in
> > > > > > > > memory datamodel) level.
> > > > > > > > With the current understanding I have, by doing this choice
> of
> > > > > working
> > > > > > at
> > > > > > > > XML level, it looks like it was decided to bypass (if not
> kill)
> > > > core
> > > > > > > > extensions that enhance the POM itself and not the pom.xml ;
> > > here I
> > > > > can
> > > > > > > > think of (but probably not limited to):
> > > > > > > > - polyglot-maven: do not use XML but other format to describe
> > the
> > > > POM
> > > > > > > > (yaml, json, kotlin, java, other XML formats, ...)
> > > > > > > > - jgitver-maven-plugin (or forks like
> > > > > maven-git-versioning-extension):
> > > > > > > > dynamic computation of projects version based on git history
> > > > > > > >
> > > > > > > > With the introduction of core extensions, I thought it was a
> > move
> > > > to
> > > > > > open
> > > > > > > > the internals and let externals contribute to the
> capabilities
> > of
> > > > > > maven.
> > > > > > > > With the move to a XML handling chain, I see it as a
> > > > > > restriction/regress
> > > > > > > in
> > > > > > > > favor of core closed functionalities. An example of that is
> > what
> > > is
> > > > > > > > provided as CIFriendly stuff, IMO it could/should have been
> > > > provided
> > > > > > by a
> > > > > > > > plugin/extension but instead it is hard written in maven core
> > and
> > > > is
> > > > > > not
> > > > > > > > opened for external contribution (plugin/extension I mean).
> > > > > > > >
> > > > > > > > Perhaps I am totally wrong but I think that maven core should
> > > > define
> > > > > > all
> > > > > > > > its expectations at an API level so that extensions/plugins
> > could
> > > > > hook
> > > > > > at
> > > > > > > > this API level. The default packaging of maven could/should
> > > provide
> > > > > > > default
> > > > > > > > implementations of those expectations (for example reading a
> > > > pom.xml
> > > > > > file
> > > > > > > > to a POM model, dumping a POM to a pom-4.0.0.xml,
> > > > > > transforming/reducing a
> > > > > > > > POM to POM-consumer, dumping POM-consumer to
> > > > pom-consumer-5.0.0.xml,
> > > > > > ...)
> > > > > > > > and let extensions/plugins/default implementations work along
> > the
> > > > > build
> > > > > > > > process with the API & POMs to provide different features and
> > > > > > > capabilities.
> > > > > > > >
> > > > > > > > Matthieu
> > > > > > > >
> > > > > > > > On Thu, Dec 31, 2020 at 7:01 PM Robert Scholte wrote:
> > > > > > > >
> > > > > > > > > I've made a recording[1] about it, which hopefully answers
> > most
> > > > > > > > questions.
> > > > > > > > >
> > > > > > > > > Robert
> > > > > > > > >
> > > > > > > > > [1] https://youtu.be/KDAmlNKZJto
> > > > > > > > >
> > > > > > > > > On 31-12-2020 16:18:57, Matthieu Brouillard
> > > > > > > > > wrote:
> > > > > > > > > > Not exactly sure what work you mean
> > > > > > > > > everything related to maven-xml:
> > Build/ConsumerPomXMLFilterxxx,
> > > > > > > > > Build/ConsumerModelSourcexxxx and the transformer stuff.
> > > > > > > > > Especially, when looking at classes like
> > CiFriendlyXMLFilter, I
> > > > > would
> > > > > > > > have
> > > > > > > > > thought that such things could have been done elsewhere,
> > > working
> > > > on
> > > > > > the
> > > > > > > > > object model (not on the XML stuff) especially for the
> > BuildPom
> > > > > part.
> > > > > > > > >
> > > > > > > > > > however specifically the consumer POM integrates with so
> > many
> > > > > > > external
> > > > > > > > > ecosystems
> > > > > > > > > We're aligned here, this has to be stable and well defined
> > by a
> > > > > > schema.
> > > > > > > > >
> > > > > > > > > Matthieu
> > > > > > > > >
> > > > > > > > > On Thu, Dec 31, 2020 at 3:59 PM Bernd Eckenfels
> > > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > > Hello,
> > > > > > > > > >
> > > > > > > > > > Not exactly sure what work you mean and I fully agree
> that
> > > > using
> > > > > a
> > > > > > > core
> > > > > > > > > > model should still be the API for plugins and extensions
> to
> > > > work
> > > > > > > with,
> > > > > > > > > > however specifically the consumer POM integrates with so
> > many
> > > > > > > external
> > > > > > > > > > ecosystems, I would expect it to be defined in terms of
> XML
> > > > > Schema
> > > > > > > with
> > > > > > > > > > explicite semantic (and the inherent compatibility with
> > > exiting
> > > > > > > POMs).
> > > > > > > > > >
> > > > > > > > > > Gruss
> > > > > > > > > > Bernd
> > > > > > > > > > --
> > > > > > > > > > http://bernd.eckenfels.net
> > > > > > > > > > ________________________________
> > > > > > > > > > Von: Matthieu BROUILLARD
> > > > > > > > > > Gesendet: Thursday, December 31, 2020 3:19:09 PM
> > > > > > > > > > An: dev@maven.apache.org
> > > > > > > > > > Betreff: maven 4.0.0 new XML stuff
> > > > > > > > > >
> > > > > > > > > > Hello all,
> > > > > > > > > >
> > > > > > > > > > regarding the active work occurring for maven 4.0.0 I
> > noticed
> > > > the
> > > > > > > > > > introduction of a lot of new stuff around SAX parsing &
> > > > > filtering.
> > > > > > > > > > I am wondering if that means that it was decided that the
> > > input
> > > > > > > format
> > > > > > > > of
> > > > > > > > > > maven projects will be XML forever meaning probably,
> among
> > > > > others,
> > > > > > > the
> > > > > > > > > end
> > > > > > > > > > of polyglot extensions.
> > > > > > > > > > Could you explain such a move (or point to
> > > rationals/documents)
> > > > > and
> > > > > > > why
> > > > > > > > > you
> > > > > > > > > > did not leverage working on the in memory object model
> > > allowing
> > > > > > > > > > extensions/plugins to contribute/hook in the chain of
> > > building
> > > > > the
> > > > > > > > > BuildPOM
> > > > > > > > > > & ConsumePOM? In the past I really thought that this move
> > to
> > > > > 'Build
> > > > > > > vs
> > > > > > > > > > Consumer' POM would make clear separations between the
> > input
> > > > > format
> > > > > > > of
> > > > > > > > > > descriptors and the core system but I perhaps
> misunderstood
> > > > > things.
> > > > > > > > > >
> > > > > > > > > > Also, are there plans regarding the future of core
> > > extensions?
> > > > > > > > > > With core extensions it was possible to hook into the POM
> > > model
> > > > > > > loading
> > > > > > > > > and
> > > > > > > > > > do transformations to do dynamic changes but by working
> on
> > > the
> > > > > XML
> > > > > > > > > directly
> > > > > > > > > > I see a shift (if not red stop) in this
> > > contribution/delegation
> > > > > > > > > mechanism.
> > > > > > > > > >
> > > > > > > > > > Thanks for your time & answers.
> > > > > > > > > >
> > > > > > > > > > Matthieu Brouillard
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: maven 4.0.0 new XML stuff

Posted by Robert Scholte <rf...@apache.org>.
Currently the model is mutable and this causes issues.
Instead I would like to see that once the BuildPlan is finished, the model becomes immutable.


To give a concrete example: It must be possible for code generating goals to add dependencies. 
Now, when using modello readers/writers you often need to add a required dependency by hand.
Ideally there will be a hook where a plugin can register additional dependencies (e.g. dom4j).

This will make any postprocessing of the pom during build obsolete

Robert
On 5-1-2021 09:09:11, Matthieu Brouillard <ma...@brouillard.fr> wrote:
> Can you give an example?
Like Romain has shown: dynamically added dependencies or a version
computation.

> Most important is the support for CI-friendly versions
But if extensions dynamically compute and set the versions in the POM using
the API, the changes will not be reflected.
That's why today one has to use flatten-maven-plugin or any other
plugin/task modifying/dumping the POM model to a pom.xml file and
setting/attaching it the POM.

If the produced artifacts (consumer pom) were computed from the POM (object
model), every change done by any extension would be part of it.

> but this happens AFTER using the pom
Not always from the pom.xml. I thought extensions were allowed to provide
ModelLocator & ModelReader to both decide which file to use for a project
and how to build the in memory POM model.
So the truth should not be considered to be in the pom.xml but in the POM.


On Mon, Jan 4, 2021 at 10:00 PM Robert Scholte wrote:

> answers are below.
>
> Robert
>
> On 4-1-2021 16:52:23, Matthieu Brouillard wrote:
> @Robert nothing is broken atm, the changes for consumer/build are currently
> behind your feature flag.
> Robert Scholte:
> It is active by default, so it is actually not hidden.
>
>
>
> But as I feared previously, and as Romain pointed it, by working at XML
> level (and not at POM level) the produced consumer pom does not reflect
> changes from extensions.
> Robert Scholte:
> Can you give an example?
>
>
> I really thought that all the "consumer/build" stuff would make the
> maven-flatten-plugin useless but it looks like it will not be the case if
> working at XML level.
> Robert Scholte:
> Like with most questions: it depends. Most important is the support for
> CI-friendly versions. In this case you won't need the flatten-maven-plugin
> anymore.
> However, the plugin can rewrite much more, but this happens AFTER using
> the pom.
> That's something I don't like, because this POM was not used to build the
> project, but it was reassembled afterwards.
> My idea is to provide hooks to parts of the pom that might be adjusted,
> but this is something we can work on during the 4.x releases
>
> Did Romain and I miss the whole point of the "consumer/build" enhancements
> or is it "just" because current implementation has not yet reached the
> targets/outputs?
>
> On Mon, Jan 4, 2021 at 2:56 PM Romain Manni-Bucau
> wrote:
>
> > Hmm, I don't get a few things of this IT:
> >
> > 1. the formatting seems not expected even if valid (the comments are
> > finishing with the first tag for example "-->
> > public class MyListener extends AbstractMavenLifecycleParticipant {
> >
> > @Override
> > public void afterProjectsRead(final MavenSession session) throws
> > MavenExecutionException {
> > if (session.getCurrentProject() == null) {
> > return;
> > }
> >
> > session.getProjects().forEach(p -> {
> > final Dependency dependency = new Dependency();
> > dependency.setGroupId("junit");
> > dependency.setArtifactId("junit");
> > dependency.setVersion("3.8.1");
> > p.getDependencies().add(dependency);
> > });
> > }
> > }
> >
> >
> > 2. If you run mvn (4 snapshot) dependency:tree you get this kind of
> output:
> >
> > [INFO] -------------
> > >-------------
> > [INFO] Building Multi Chapter Simple Web Application Project
> > 0.9-MNG6957-SNAPSHOT [6/6]
> > [INFO] --------------------------------[ jar
> > ]---------------------------------
> > [INFO]
> > [INFO] --- maven-dependency-plugin:3.1.2:tree (default-cli) @
> simple-webapp
> > ---
> > [INFO]
> org.sonatype.mavenbook.multi:simple-webapp:jar:0.9-MNG6957-SNAPSHOT
> > [INFO] +-
> >
> >
> org.sonatype.mavenbook.multi:simple-weather:jar:0.9-MNG6957-SNAPSHOT:compile
> > [INFO] \- junit:junit:jar:3.8.1:compile
> >
> > [INFO]
> > ------------------------------------------------------------------------
> >
> > 3. run the build to have produced pom and cat the simple-webapp one:
> >
> >
> > 4.0.0
> >
> > org.sonatype.mavenbook.multi
> > simple-parent
> > 0.9-MNG6957-SNAPSHOT
> >
> >
> > simple-webapp
> > Multi Chapter Simple Web Application Project
> >
> >
> > org.sonatype.mavenbook.multi
> > simple-weather
> > 0.9-MNG6957-SNAPSHOT
> >
> >
> >
> > simple-webapp
> >
> >
> >
> > org.apache.maven.plugins
> > maven-war-plugin
> > 2.6
> >
> >
> >
> >
> >
> >
> > As you see the dependency is not there. I guess the expected outout is:
> >
> >
> >
> > 4.0.0
> > simple-webapp
> > Multi Chapter Simple Web Application Project
> > [description, scm, ..., all central requires sections but not build ones]
> >
> >
> > org.sonatype.mavenbook.multi
> > simple-weather
> > 0.9-MNG6957-SNAPSHOT
> >
> >
> > junit
> > junit
> > 3.8.1
> >
> >
> >
> >
> > Am I missing something?
> >
> > Romain Manni-Bucau
> > @rmannibucau | Blog
> > | Old Blog
> > | Github
> > https://github.com/rmannibucau> |
> > LinkedIn | Book
> >
> >
> https://www.packtpub.com/application-development/java-ee-8-high-performance
> > >
> >
> >
> > Le lun. 4 janv. 2021 à 13:41, Robert Scholte a
> > écrit :
> >
> > >
> > >
> >
> https://github.com/apache/maven-integration-testing/tree/master/core-it-suite/src/test/resources/mng-6957-buildconsumer
> > > is the most complete IT
> > >
> > > On 4-1-2021 12:59:51, Romain Manni-Bucau wrote:
> > > Le lun. 4 janv. 2021 à 12:36, Robert Scholte a
> > > écrit :
> > >
> > > > There's just one thing I want to say:
> > > > I'm having trouble with the term "broken".
> > > >
> > >
> > > Well, literally meant broken as decorelated from the user intent and
> > > extension model.
> > > Anyway, didn't intend to blame but more identify the blockers for a GA
> so
> > > point was really that it seem that on the two sides only the producing
> > one
> > > is not yet ready since it keeps does not sanitize the model completely
> > and
> > > keeps build only data like comments, right? Also not yet clear for me
> if
> > we
> > > loose the extension enrichments there.
> > >
> > >
> > > > If a Maven project could be built with M3.6.3, it can still be built
> > with
> > > > M4.
> > > > If not, it is either regression (MNG-6957, MNG-7063) which must be
> > fixed,
> > > > or it requires changes to a plugin for understandable reasons
> > > > (maven-pgp-plugin)
> > > > AFAIK an interesting extension like the maven-tiles has been tested
> and
> > > > still works.
> > > >
> > >
> > > Do you have this handy, is it in our test suite? I'd like to check the
> > > produced pom matches the enriched model but happy to start from
> something
> > > already there.
> > >
> > >
> > > >
> > > > Robert
> > > >
> > > > On 3-1-2021 19:35:25, Romain Manni-Bucau wrote:
> > > > Le dim. 3 janv. 2021 à 19:04, Robert Scholte a
> > > > écrit :
> > > >
> > > > > I don't remember all those details anymore, because I hit those in
> > the
> > > > > beginning.
> > > > > Trying things over and over again I decided that this is probably
> the
> > > > most
> > > > > successful approach.
> > > > >
> > > > >
> > > > > What of the goals was to keep the pom.xml as is as much as
> possible.
> > > > > We can only decide for the specific Maven elements how to handle
> > them,
> > > we
> > > > > should not decide about comments and licenses.
> > > > > BTW, the license issue was hard to solve. You cannot use it from
> the
> > > > pom's
> > > > > , because there might be multiple licenses.
> > > > >
> > > >
> > > > I disagree, it is saner IMO to evolve to support that than doing
> > > > anything else.
> > > > Once again you keep things which don't make sense in a consumed pom
> in
> > > > current impl so i'd say the sucess in a few cases breaks as much
> cases
> > so
> > > > we need to revisit anyway IMHO.
> > > >
> > > >
> > > > >
> > > > > The current implementation is a solid way to ensure we're not
> > breaking
> > > > too
> > > > > much, because Maven controls the XML filters.
> > > > >
> > > >
> > > > Hmm, breaking extensions seems to break too much (I'm not speaking of
> > > other
> > > > parts which breaks the ecosystem there but just that is sufficient
> IMHO
> > > to
> > > > say we must check back our solution).
> > > >
> > > >
> > > > >
> > > > > Also keep in mind, that I only want Maven to decide which
> > modifications
> > > > > are done.
> > > > >
> > > >
> > > > For the consumed pom I agree but it is consistent with keeping
> > everything
> > > > working instead of breaking too, no?
> > > >
> > > >
> > > > >
> > > > > Current polyglot projects should still work, but they cannot
> benefit
> > > from
> > > > > the build/consumer functions yet.
> > > > >
> > > >
> > > > So pom -> build model is kept, build model -> produced pom is broken?
> > Is
> > > it
> > > > the complete status?
> > > > Sounds ok for a 4.0 and a 4.1 can fix it if so.
> > > > Just want to ensure first part is not broken at all.
> > > >
> > > >
> > > > >
> > > > > Robert
> > > > > On 3-1-2021 16:38:38, Romain Manni-Bucau wrote:
> > > > > Le dim. 3 janv. 2021 à 16:18, Robert Scholte a
> > > > > écrit :
> > > > >
> > > > > > > So what I was expecting was: raw xml model -> converted to
> > unified
> > > > > > consumed model -> extensions -> model processing.
> > > > > >
> > > > > > This is only the build pom part. You're missing the consume part,
> > > where
> > > > > > the xml is distributed.
> > > > > >
> > > > >
> > > > > Yes but with previous chain the consume part is "clean/normalize ->
> > > dump"
> > > > > since we are using consumed model - only standard model - in memory
> > > > > already.
> > > > >
> > > > >
> > > > > > Build is raw + enrich, consumer is raw + enrich + reduce
> (removing
> > > > > > relativePath and modules are the first examples, but much more is
> > > > > possible)
> > > > > >
> > > > > > Going for the in memory was also my first thought, but I would
> > loose
> > > > > > information, hence I came up with the current implementation.
> > > > > >
> > > > >
> > > > > I don't see what you loose ot be honest.
> > > > > You mentionned license but this one is in the pom so not a big
> deal,
> > > > > comments which are undesired IMHO as mentionned and element order
> > which
> > > > can
> > > > > really be discussed since we can desire to enforce an order to
> > > normalize
> > > > > consumption + it shouldn't be important since from the project
> point
> > of
> > > > > view your pom is already "broken"/lost (as all your intelligence is
> > > lost
> > > > by
> > > > > this "not passthrough" process).
> > > > > So overall I don't see what you would loose from the consumer side
> > but
> > > I
> > > > > see what you lost from maven ecosystem side.
> > > > >
> > > > >
> > > > > > Again, we're at a point where we can have counter solutions, but
> > > don't
> > > > > > expect me to implement it.
> > > > > >
> > > > >
> > > > > For now I'm just trying to ensure we agree we don't want to break
> > > > existing
> > > > > extensions and the nice ecosystem we built after years.
> > > > > This was really a move forward and it sounds like we broke it at
> > maven
> > > 4
> > > > > without any user gain which sounds terrible.
> > > > >
> > > > >
> > > > > >
> > > > > > thanks,
> > > > > > Robert
> > > > > > On 3-1-2021 15:25:21, Romain Manni-Bucau wrote:
> > > > > > Hi all,
> > > > > >
> > > > > > I kind of join Matthieu thoughts there, there is no point to work
> > at
> > > > xml
> > > > > > level to create the consumed pom - comments is not a point since
> it
> > > can
> > > > > > commonly/easily refer to a dropped part of the pom so they should
> > be
> > > > > > stripped.
> > > > > > Current extension model got proven adapted and adopted, using a
> > lower
> > > > > level
> > > > > > extension API will not since XML is, even if still mainstream,
> > often
> > > > > > replaced by alternative configurations and to have done the work
> to
> > > > > inject
> > > > > > XML configuration programmatically compred to current option, it
> > is a
> > > > > pain.
> > > > > > The in memory model should stick to consumed model IMHO - being
> > > > > > programmatic there is no point to make it easier, worse case we
> can
> > > add
> > > > > > helper beans (injectable) but in terms of model it will not help.
> > > > > >
> > > > > > So what I was expecting was:
> > > > > >
> > > > > > raw xml model -> converted to unified consumed model ->
> extensions
> > ->
> > > > > model
> > > > > > processing.
> > > > > >
> > > > > > Indeed, real chain adds a small processing over the first arrow
> > > (inject
> > > > > > versions for example) but nothing crazy and breaking this overall
> > > flow
> > > > > > which stays user friendly.
> > > > > >
> > > > > > Strictly speaking the new model is just a built-in extension for
> me
> > > > which
> > > > > > is particular because it will enforce IDE to integrate a new
> > format -
> > > > > > wheres polyglot extensions or others don't require static
> analyzis
> > by
> > > > > > themself not being "standard".
> > > > > >
> > > > > > That said, there is nothing crazy with current implementation, it
> > > just
> > > > > > require to be updated to be able to take extension changes into
> > > > account.
> > > > > > This can be done by making the extension model 'spyable' (ie if a
> > > > > > dependency/plugin is added it will be reflected in the final
> > written
> > > > > > pom.xml).
> > > > > > This sounds - instrumenting the extension model API or doing a
> diff
> > > > after
> > > > > > extension phase - like a compromise and let people gets the best
> of
> > > > both
> > > > > > worlds to me.
> > > > > >
> > > > > > Wdyt?
> > > > > >
> > > > > > Romain Manni-Bucau
> > > > > > @rmannibucau | Blog
> > > > > > | Old Blog
> > > > > > | Github |
> > > > > > LinkedIn | Book
> > > > > >
> > > > > >
> > > > > >
> > > > > > Le dim. 3 janv. 2021 à 14:46, Robert Scholte a
> > > > > > écrit :
> > > > > >
> > > > > > > Hi Matthieu,
> > > > > > >
> > > > > > > As you understand, something had to be changed to move Maven
> > > forward.
> > > > > > > I've decided to pick up that challenge and came up with the
> > current
> > > > > > > solution.
> > > > > > >
> > > > > > > My main concerns was that I wanted to keep the fileModel as
> much
> > as
> > > > is.
> > > > > > > That includes the license, comments and element order.
> > > > > > > This information if not available in the memory model, so I
> > needed
> > > > the
> > > > > > > original pom file.
> > > > > > > With that in mind, the usage of XMLFilters looks like the most
> > > > > > appropriate
> > > > > > > solution.
> > > > > > >
> > > > > > > I am certain that XML is still the most used format, so if we
> can
> > > > have
> > > > > > > improvements for those users, I'm already very happy.
> > > > > > >
> > > > > > > And yes, there are plugins that needs to be updated, but doing
> > > > nothing
> > > > > is
> > > > > > > not an option anymore.
> > > > > > >
> > > > > > > There are more people that share their concerns, but it took me
> > > > several
> > > > > > > years to reach this point.
> > > > > > > We now have something that seems to work, anybody who can
> improve
> > > or
> > > > > can
> > > > > > > come up with an alternative implementation can do so.
> > > > > > >
> > > > > > > thanks,
> > > > > > > Robert
> > > > > > > On 3-1-2021 12:55:41, Matthieu Brouillard wrote:
> > > > > > > Thanks Robert for the video link.
> > > > > > >
> > > > > > > I fully understand the rationales behind the separation of
> > > > > > > build/consumer pom and the video provides some insights on it
> and
> > > you
> > > > > > > explain the actual implementation to introduce this change.
> > > > > > > Still I do not fully understand why it was decided to work on
> top
> > > of
> > > > > XML
> > > > > > by
> > > > > > > filtering/enhancing it instead of working at the POM (in
> > > > > > > memory datamodel) level.
> > > > > > > With the current understanding I have, by doing this choice of
> > > > working
> > > > > at
> > > > > > > XML level, it looks like it was decided to bypass (if not kill)
> > > core
> > > > > > > extensions that enhance the POM itself and not the pom.xml ;
> > here I
> > > > can
> > > > > > > think of (but probably not limited to):
> > > > > > > - polyglot-maven: do not use XML but other format to describe
> the
> > > POM
> > > > > > > (yaml, json, kotlin, java, other XML formats, ...)
> > > > > > > - jgitver-maven-plugin (or forks like
> > > > maven-git-versioning-extension):
> > > > > > > dynamic computation of projects version based on git history
> > > > > > >
> > > > > > > With the introduction of core extensions, I thought it was a
> move
> > > to
> > > > > open
> > > > > > > the internals and let externals contribute to the capabilities
> of
> > > > > maven.
> > > > > > > With the move to a XML handling chain, I see it as a
> > > > > restriction/regress
> > > > > > in
> > > > > > > favor of core closed functionalities. An example of that is
> what
> > is
> > > > > > > provided as CIFriendly stuff, IMO it could/should have been
> > > provided
> > > > > by a
> > > > > > > plugin/extension but instead it is hard written in maven core
> and
> > > is
> > > > > not
> > > > > > > opened for external contribution (plugin/extension I mean).
> > > > > > >
> > > > > > > Perhaps I am totally wrong but I think that maven core should
> > > define
> > > > > all
> > > > > > > its expectations at an API level so that extensions/plugins
> could
> > > > hook
> > > > > at
> > > > > > > this API level. The default packaging of maven could/should
> > provide
> > > > > > default
> > > > > > > implementations of those expectations (for example reading a
> > > pom.xml
> > > > > file
> > > > > > > to a POM model, dumping a POM to a pom-4.0.0.xml,
> > > > > transforming/reducing a
> > > > > > > POM to POM-consumer, dumping POM-consumer to
> > > pom-consumer-5.0.0.xml,
> > > > > ...)
> > > > > > > and let extensions/plugins/default implementations work along
> the
> > > > build
> > > > > > > process with the API & POMs to provide different features and
> > > > > > capabilities.
> > > > > > >
> > > > > > > Matthieu
> > > > > > >
> > > > > > > On Thu, Dec 31, 2020 at 7:01 PM Robert Scholte wrote:
> > > > > > >
> > > > > > > > I've made a recording[1] about it, which hopefully answers
> most
> > > > > > > questions.
> > > > > > > >
> > > > > > > > Robert
> > > > > > > >
> > > > > > > > [1] https://youtu.be/KDAmlNKZJto
> > > > > > > >
> > > > > > > > On 31-12-2020 16:18:57, Matthieu Brouillard
> > > > > > > > wrote:
> > > > > > > > > Not exactly sure what work you mean
> > > > > > > > everything related to maven-xml:
> Build/ConsumerPomXMLFilterxxx,
> > > > > > > > Build/ConsumerModelSourcexxxx and the transformer stuff.
> > > > > > > > Especially, when looking at classes like
> CiFriendlyXMLFilter, I
> > > > would
> > > > > > > have
> > > > > > > > thought that such things could have been done elsewhere,
> > working
> > > on
> > > > > the
> > > > > > > > object model (not on the XML stuff) especially for the
> BuildPom
> > > > part.
> > > > > > > >
> > > > > > > > > however specifically the consumer POM integrates with so
> many
> > > > > > external
> > > > > > > > ecosystems
> > > > > > > > We're aligned here, this has to be stable and well defined
> by a
> > > > > schema.
> > > > > > > >
> > > > > > > > Matthieu
> > > > > > > >
> > > > > > > > On Thu, Dec 31, 2020 at 3:59 PM Bernd Eckenfels
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > > > Hello,
> > > > > > > > >
> > > > > > > > > Not exactly sure what work you mean and I fully agree that
> > > using
> > > > a
> > > > > > core
> > > > > > > > > model should still be the API for plugins and extensions to
> > > work
> > > > > > with,
> > > > > > > > > however specifically the consumer POM integrates with so
> many
> > > > > > external
> > > > > > > > > ecosystems, I would expect it to be defined in terms of XML
> > > > Schema
> > > > > > with
> > > > > > > > > explicite semantic (and the inherent compatibility with
> > exiting
> > > > > > POMs).
> > > > > > > > >
> > > > > > > > > Gruss
> > > > > > > > > Bernd
> > > > > > > > > --
> > > > > > > > > http://bernd.eckenfels.net
> > > > > > > > > ________________________________
> > > > > > > > > Von: Matthieu BROUILLARD
> > > > > > > > > Gesendet: Thursday, December 31, 2020 3:19:09 PM
> > > > > > > > > An: dev@maven.apache.org
> > > > > > > > > Betreff: maven 4.0.0 new XML stuff
> > > > > > > > >
> > > > > > > > > Hello all,
> > > > > > > > >
> > > > > > > > > regarding the active work occurring for maven 4.0.0 I
> noticed
> > > the
> > > > > > > > > introduction of a lot of new stuff around SAX parsing &
> > > > filtering.
> > > > > > > > > I am wondering if that means that it was decided that the
> > input
> > > > > > format
> > > > > > > of
> > > > > > > > > maven projects will be XML forever meaning probably, among
> > > > others,
> > > > > > the
> > > > > > > > end
> > > > > > > > > of polyglot extensions.
> > > > > > > > > Could you explain such a move (or point to
> > rationals/documents)
> > > > and
> > > > > > why
> > > > > > > > you
> > > > > > > > > did not leverage working on the in memory object model
> > allowing
> > > > > > > > > extensions/plugins to contribute/hook in the chain of
> > building
> > > > the
> > > > > > > > BuildPOM
> > > > > > > > > & ConsumePOM? In the past I really thought that this move
> to
> > > > 'Build
> > > > > > vs
> > > > > > > > > Consumer' POM would make clear separations between the
> input
> > > > format
> > > > > > of
> > > > > > > > > descriptors and the core system but I perhaps misunderstood
> > > > things.
> > > > > > > > >
> > > > > > > > > Also, are there plans regarding the future of core
> > extensions?
> > > > > > > > > With core extensions it was possible to hook into the POM
> > model
> > > > > > loading
> > > > > > > > and
> > > > > > > > > do transformations to do dynamic changes but by working on
> > the
> > > > XML
> > > > > > > > directly
> > > > > > > > > I see a shift (if not red stop) in this
> > contribution/delegation
> > > > > > > > mechanism.
> > > > > > > > >
> > > > > > > > > Thanks for your time & answers.
> > > > > > > > >
> > > > > > > > > Matthieu Brouillard
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: maven 4.0.0 new XML stuff

Posted by Romain Manni-Bucau <rm...@gmail.com>.
So it seems we must enhance that read/write logic with these elements to
make them usable:

1. Make it extension/Maven-IoC (even without extensions you can have the
same issue with just guice components) friendly (code showing the bug is in
a previous response)
2. Decide what is the produced pom, either the exact same pom as the source
one without the model > 4 changes (= just a model N -> 4 converter) or a
pom optimized to be consumed (flattened, without build section etc).
Another point here is about comments, it should likely be a config in the
pom to know if they must be stripped or not. If I take all projects I
worked on, comments are build comments - except the license - so must not
end up in the consummed pom if designed to be used by user (even if it does
not hurt). Last pom is that it should also be configurable if some part of
the pom are kept or not (maybe using plexus attributes) since some tool
tend to parse poms at runtime to extract some metadata from there, guess it
will slowly move to other descriptors but pom were nice because 75% of the
time there so we should make the migration smooth IMHO by enabling to keep
these elements. All that can be seen differently, if you take it from the
outside, like from a gradle or sbt pespective, you kind of script the
mapping memory descriptor -> consumable pom. Maven is not different as soon
as it embraces this model so it must be user controlled - no way we do it
right without user inputs. Guess a trivial way is to just let the user give
a xslt in a maven pom producer plugin which would replace all this core
code. If the plugin is not define we can use the default behavior adding it
in the deploy phase out of the box.

So overall I to join Matthieu on the target since it is what makes sense
IMHO and seems to be widely used by people.

Side nice note for us: It also opens the door to way faster resolutions if
we add a property in these poms (maven.consumable.pom=true? or so) to cut
the recursive dependency resolution if already done at deploy time. Can
make it hundreds of time faster - used that in some previous work projects.

Romain Manni-Bucau
@rmannibucau <https://twitter.com/rmannibucau> |  Blog
<https://rmannibucau.metawerx.net/> | Old Blog
<http://rmannibucau.wordpress.com> | Github <https://github.com/rmannibucau> |
LinkedIn <https://www.linkedin.com/in/rmannibucau> | Book
<https://www.packtpub.com/application-development/java-ee-8-high-performance>


Le mar. 5 janv. 2021 à 09:09, Matthieu Brouillard <ma...@brouillard.fr>
a écrit :

> > Can you give an example?
> Like Romain has shown: dynamically added dependencies or a version
> computation.
>
> > Most important is the support for CI-friendly versions
> But if extensions dynamically compute and set the versions in the POM using
> the API, the changes will not be reflected.
> That's why today one has to use flatten-maven-plugin or any other
> plugin/task modifying/dumping the POM model to a pom.xml file and
> setting/attaching it the POM.
>
> If the produced artifacts (consumer pom) were computed from the POM (object
> model), every change done by any extension would be part of it.
>
> > but this happens AFTER using the pom
> Not always from the pom.xml. I thought extensions were allowed to provide
> ModelLocator & ModelReader to both decide which file to use for a project
> and how to build the in memory POM model.
> So the truth should not be considered to be in the pom.xml but in the POM.
>
>
> On Mon, Jan 4, 2021 at 10:00 PM Robert Scholte <rf...@apache.org>
> wrote:
>
> > answers are below.
> >
> > Robert
> >
> > On 4-1-2021 16:52:23, Matthieu Brouillard <ma...@brouillard.fr>
> wrote:
> > @Robert nothing is broken atm, the changes for consumer/build are
> currently
> > behind your feature flag.
> > Robert Scholte:
> > It is active by default, so it is actually not hidden.
> >
> >
> >
> > But as I feared previously, and as Romain pointed it, by working at XML
> > level (and not at POM level) the produced consumer pom does not reflect
> > changes from extensions.
> > Robert Scholte:
> > Can you give an example?
> >
> >
> > I really thought that all the "consumer/build" stuff would make the
> > maven-flatten-plugin useless but it looks like it will not be the case if
> > working at XML level.
> > Robert Scholte:
> > Like with most questions: it depends. Most important is the support for
> > CI-friendly versions. In this case you won't need the
> flatten-maven-plugin
> > anymore.
> > However, the plugin can rewrite much more, but this happens AFTER using
> > the pom.
> > That's something I don't like, because this POM was not used to build the
> > project, but it was reassembled afterwards.
> > My idea is to provide hooks to parts of the pom that might be adjusted,
> > but this is something we can work on during the 4.x releases
> >
> > Did Romain and I miss the whole point of the "consumer/build"
> enhancements
> > or is it "just" because current implementation has not yet reached the
> > targets/outputs?
> >
> > On Mon, Jan 4, 2021 at 2:56 PM Romain Manni-Bucau
> > wrote:
> >
> > > Hmm, I don't get a few things of this IT:
> > >
> > > 1. the formatting seems not expected even if valid (the comments are
> > > finishing with the first tag for example "-->
> > > public class MyListener extends AbstractMavenLifecycleParticipant {
> > >
> > > @Override
> > > public void afterProjectsRead(final MavenSession session) throws
> > > MavenExecutionException {
> > > if (session.getCurrentProject() == null) {
> > > return;
> > > }
> > >
> > > session.getProjects().forEach(p -> {
> > > final Dependency dependency = new Dependency();
> > > dependency.setGroupId("junit");
> > > dependency.setArtifactId("junit");
> > > dependency.setVersion("3.8.1");
> > > p.getDependencies().add(dependency);
> > > });
> > > }
> > > }
> > >
> > >
> > > 2. If you run mvn (4 snapshot) dependency:tree you get this kind of
> > output:
> > >
> > > [INFO] -------------< org.sonatype.mavenbook.multi:simple-webapp
> > > >-------------
> > > [INFO] Building Multi Chapter Simple Web Application Project
> > > 0.9-MNG6957-SNAPSHOT [6/6]
> > > [INFO] --------------------------------[ jar
> > > ]---------------------------------
> > > [INFO]
> > > [INFO] --- maven-dependency-plugin:3.1.2:tree (default-cli) @
> > simple-webapp
> > > ---
> > > [INFO]
> > org.sonatype.mavenbook.multi:simple-webapp:jar:0.9-MNG6957-SNAPSHOT
> > > [INFO] +-
> > >
> > >
> >
> org.sonatype.mavenbook.multi:simple-weather:jar:0.9-MNG6957-SNAPSHOT:compile
> > > [INFO] \- junit:junit:jar:3.8.1:compile
> > > <-- THIS IS WHAT WE WANT TO SEE
> > > [INFO]
> > >
> ------------------------------------------------------------------------
> > >
> > > 3. run the build to have produced pom and cat the simple-webapp one:
> > >
> > >
> > > 4.0.0
> > >
> > > org.sonatype.mavenbook.multi
> > > simple-parent
> > > 0.9-MNG6957-SNAPSHOT
> > >
> > >
> > > simple-webapp
> > > Multi Chapter Simple Web Application Project
> > >
> > >
> > > org.sonatype.mavenbook.multi
> > > simple-weather
> > > 0.9-MNG6957-SNAPSHOT
> > >
> > >
> > >
> > > simple-webapp
> > >
> > >
> > >
> > > org.apache.maven.plugins
> > > maven-war-plugin
> > > 2.6
> > >
> > >
> > >
> > >
> > >
> > >
> > > As you see the dependency is not there. I guess the expected outout is:
> > >
> > >
> > >
> > > 4.0.0
> > > simple-webapp
> > > Multi Chapter Simple Web Application Project
> > > [description, scm, ..., all central requires sections but not build
> ones]
> > >
> > >
> > > org.sonatype.mavenbook.multi
> > > simple-weather
> > > 0.9-MNG6957-SNAPSHOT
> > >
> > >
> > > junit
> > > junit
> > > 3.8.1
> > >
> > >
> > >
> > >
> > > Am I missing something?
> > >
> > > Romain Manni-Bucau
> > > @rmannibucau | Blog
> > > | Old Blog
> > > | Github <
> > > https://github.com/rmannibucau> |
> > > LinkedIn | Book
> > > <
> > >
> >
> https://www.packtpub.com/application-development/java-ee-8-high-performance
> > > >
> > >
> > >
> > > Le lun. 4 janv. 2021 à 13:41, Robert Scholte a
> > > écrit :
> > >
> > > >
> > > >
> > >
> >
> https://github.com/apache/maven-integration-testing/tree/master/core-it-suite/src/test/resources/mng-6957-buildconsumer
> > > > is the most complete IT
> > > >
> > > > On 4-1-2021 12:59:51, Romain Manni-Bucau wrote:
> > > > Le lun. 4 janv. 2021 à 12:36, Robert Scholte a
> > > > écrit :
> > > >
> > > > > There's just one thing I want to say:
> > > > > I'm having trouble with the term "broken".
> > > > >
> > > >
> > > > Well, literally meant broken as decorelated from the user intent and
> > > > extension model.
> > > > Anyway, didn't intend to blame but more identify the blockers for a
> GA
> > so
> > > > point was really that it seem that on the two sides only the
> producing
> > > one
> > > > is not yet ready since it keeps does not sanitize the model
> completely
> > > and
> > > > keeps build only data like comments, right? Also not yet clear for me
> > if
> > > we
> > > > loose the extension enrichments there.
> > > >
> > > >
> > > > > If a Maven project could be built with M3.6.3, it can still be
> built
> > > with
> > > > > M4.
> > > > > If not, it is either regression (MNG-6957, MNG-7063) which must be
> > > fixed,
> > > > > or it requires changes to a plugin for understandable reasons
> > > > > (maven-pgp-plugin)
> > > > > AFAIK an interesting extension like the maven-tiles has been tested
> > and
> > > > > still works.
> > > > >
> > > >
> > > > Do you have this handy, is it in our test suite? I'd like to check
> the
> > > > produced pom matches the enriched model but happy to start from
> > something
> > > > already there.
> > > >
> > > >
> > > > >
> > > > > Robert
> > > > >
> > > > > On 3-1-2021 19:35:25, Romain Manni-Bucau wrote:
> > > > > Le dim. 3 janv. 2021 à 19:04, Robert Scholte a
> > > > > écrit :
> > > > >
> > > > > > I don't remember all those details anymore, because I hit those
> in
> > > the
> > > > > > beginning.
> > > > > > Trying things over and over again I decided that this is probably
> > the
> > > > > most
> > > > > > successful approach.
> > > > > >
> > > > > >
> > > > > > What of the goals was to keep the pom.xml as is as much as
> > possible.
> > > > > > We can only decide for the specific Maven elements how to handle
> > > them,
> > > > we
> > > > > > should not decide about comments and licenses.
> > > > > > BTW, the license issue was hard to solve. You cannot use it from
> > the
> > > > > pom's
> > > > > > , because there might be multiple licenses.
> > > > > >
> > > > >
> > > > > I disagree, it is saner IMO to evolve to support that than doing
> > > > > anything else.
> > > > > Once again you keep things which don't make sense in a consumed pom
> > in
> > > > > current impl so i'd say the sucess in a few cases breaks as much
> > cases
> > > so
> > > > > we need to revisit anyway IMHO.
> > > > >
> > > > >
> > > > > >
> > > > > > The current implementation is a solid way to ensure we're not
> > > breaking
> > > > > too
> > > > > > much, because Maven controls the XML filters.
> > > > > >
> > > > >
> > > > > Hmm, breaking extensions seems to break too much (I'm not speaking
> of
> > > > other
> > > > > parts which breaks the ecosystem there but just that is sufficient
> > IMHO
> > > > to
> > > > > say we must check back our solution).
> > > > >
> > > > >
> > > > > >
> > > > > > Also keep in mind, that I only want Maven to decide which
> > > modifications
> > > > > > are done.
> > > > > >
> > > > >
> > > > > For the consumed pom I agree but it is consistent with keeping
> > > everything
> > > > > working instead of breaking too, no?
> > > > >
> > > > >
> > > > > >
> > > > > > Current polyglot projects should still work, but they cannot
> > benefit
> > > > from
> > > > > > the build/consumer functions yet.
> > > > > >
> > > > >
> > > > > So pom -> build model is kept, build model -> produced pom is
> broken?
> > > Is
> > > > it
> > > > > the complete status?
> > > > > Sounds ok for a 4.0 and a 4.1 can fix it if so.
> > > > > Just want to ensure first part is not broken at all.
> > > > >
> > > > >
> > > > > >
> > > > > > Robert
> > > > > > On 3-1-2021 16:38:38, Romain Manni-Bucau wrote:
> > > > > > Le dim. 3 janv. 2021 à 16:18, Robert Scholte a
> > > > > > écrit :
> > > > > >
> > > > > > > > So what I was expecting was: raw xml model -> converted to
> > > unified
> > > > > > > consumed model -> extensions -> model processing.
> > > > > > >
> > > > > > > This is only the build pom part. You're missing the consume
> part,
> > > > where
> > > > > > > the xml is distributed.
> > > > > > >
> > > > > >
> > > > > > Yes but with previous chain the consume part is "clean/normalize
> ->
> > > > dump"
> > > > > > since we are using consumed model - only standard model - in
> memory
> > > > > > already.
> > > > > >
> > > > > >
> > > > > > > Build is raw + enrich, consumer is raw + enrich + reduce
> > (removing
> > > > > > > relativePath and modules are the first examples, but much more
> is
> > > > > > possible)
> > > > > > >
> > > > > > > Going for the in memory was also my first thought, but I would
> > > loose
> > > > > > > information, hence I came up with the current implementation.
> > > > > > >
> > > > > >
> > > > > > I don't see what you loose ot be honest.
> > > > > > You mentionned license but this one is in the pom so not a big
> > deal,
> > > > > > comments which are undesired IMHO as mentionned and element order
> > > which
> > > > > can
> > > > > > really be discussed since we can desire to enforce an order to
> > > > normalize
> > > > > > consumption + it shouldn't be important since from the project
> > point
> > > of
> > > > > > view your pom is already "broken"/lost (as all your intelligence
> is
> > > > lost
> > > > > by
> > > > > > this "not passthrough" process).
> > > > > > So overall I don't see what you would loose from the consumer
> side
> > > but
> > > > I
> > > > > > see what you lost from maven ecosystem side.
> > > > > >
> > > > > >
> > > > > > > Again, we're at a point where we can have counter solutions,
> but
> > > > don't
> > > > > > > expect me to implement it.
> > > > > > >
> > > > > >
> > > > > > For now I'm just trying to ensure we agree we don't want to break
> > > > > existing
> > > > > > extensions and the nice ecosystem we built after years.
> > > > > > This was really a move forward and it sounds like we broke it at
> > > maven
> > > > 4
> > > > > > without any user gain which sounds terrible.
> > > > > >
> > > > > >
> > > > > > >
> > > > > > > thanks,
> > > > > > > Robert
> > > > > > > On 3-1-2021 15:25:21, Romain Manni-Bucau wrote:
> > > > > > > Hi all,
> > > > > > >
> > > > > > > I kind of join Matthieu thoughts there, there is no point to
> work
> > > at
> > > > > xml
> > > > > > > level to create the consumed pom - comments is not a point
> since
> > it
> > > > can
> > > > > > > commonly/easily refer to a dropped part of the pom so they
> should
> > > be
> > > > > > > stripped.
> > > > > > > Current extension model got proven adapted and adopted, using a
> > > lower
> > > > > > level
> > > > > > > extension API will not since XML is, even if still mainstream,
> > > often
> > > > > > > replaced by alternative configurations and to have done the
> work
> > to
> > > > > > inject
> > > > > > > XML configuration programmatically compred to current option,
> it
> > > is a
> > > > > > pain.
> > > > > > > The in memory model should stick to consumed model IMHO - being
> > > > > > > programmatic there is no point to make it easier, worse case we
> > can
> > > > add
> > > > > > > helper beans (injectable) but in terms of model it will not
> help.
> > > > > > >
> > > > > > > So what I was expecting was:
> > > > > > >
> > > > > > > raw xml model -> converted to unified consumed model ->
> > extensions
> > > ->
> > > > > > model
> > > > > > > processing.
> > > > > > >
> > > > > > > Indeed, real chain adds a small processing over the first arrow
> > > > (inject
> > > > > > > versions for example) but nothing crazy and breaking this
> overall
> > > > flow
> > > > > > > which stays user friendly.
> > > > > > >
> > > > > > > Strictly speaking the new model is just a built-in extension
> for
> > me
> > > > > which
> > > > > > > is particular because it will enforce IDE to integrate a new
> > > format -
> > > > > > > wheres polyglot extensions or others don't require static
> > analyzis
> > > by
> > > > > > > themself not being "standard".
> > > > > > >
> > > > > > > That said, there is nothing crazy with current implementation,
> it
> > > > just
> > > > > > > require to be updated to be able to take extension changes into
> > > > > account.
> > > > > > > This can be done by making the extension model 'spyable' (ie
> if a
> > > > > > > dependency/plugin is added it will be reflected in the final
> > > written
> > > > > > > pom.xml).
> > > > > > > This sounds - instrumenting the extension model API or doing a
> > diff
> > > > > after
> > > > > > > extension phase - like a compromise and let people gets the
> best
> > of
> > > > > both
> > > > > > > worlds to me.
> > > > > > >
> > > > > > > Wdyt?
> > > > > > >
> > > > > > > Romain Manni-Bucau
> > > > > > > @rmannibucau | Blog
> > > > > > > | Old Blog
> > > > > > > | Github |
> > > > > > > LinkedIn | Book
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > Le dim. 3 janv. 2021 à 14:46, Robert Scholte a
> > > > > > > écrit :
> > > > > > >
> > > > > > > > Hi Matthieu,
> > > > > > > >
> > > > > > > > As you understand, something had to be changed to move Maven
> > > > forward.
> > > > > > > > I've decided to pick up that challenge and came up with the
> > > current
> > > > > > > > solution.
> > > > > > > >
> > > > > > > > My main concerns was that I wanted to keep the fileModel as
> > much
> > > as
> > > > > is.
> > > > > > > > That includes the license, comments and element order.
> > > > > > > > This information if not available in the memory model, so I
> > > needed
> > > > > the
> > > > > > > > original pom file.
> > > > > > > > With that in mind, the usage of XMLFilters looks like the
> most
> > > > > > > appropriate
> > > > > > > > solution.
> > > > > > > >
> > > > > > > > I am certain that XML is still the most used format, so if we
> > can
> > > > > have
> > > > > > > > improvements for those users, I'm already very happy.
> > > > > > > >
> > > > > > > > And yes, there are plugins that needs to be updated, but
> doing
> > > > > nothing
> > > > > > is
> > > > > > > > not an option anymore.
> > > > > > > >
> > > > > > > > There are more people that share their concerns, but it took
> me
> > > > > several
> > > > > > > > years to reach this point.
> > > > > > > > We now have something that seems to work, anybody who can
> > improve
> > > > or
> > > > > > can
> > > > > > > > come up with an alternative implementation can do so.
> > > > > > > >
> > > > > > > > thanks,
> > > > > > > > Robert
> > > > > > > > On 3-1-2021 12:55:41, Matthieu Brouillard wrote:
> > > > > > > > Thanks Robert for the video link.
> > > > > > > >
> > > > > > > > I fully understand the rationales behind the separation of
> > > > > > > > build/consumer pom and the video provides some insights on it
> > and
> > > > you
> > > > > > > > explain the actual implementation to introduce this change.
> > > > > > > > Still I do not fully understand why it was decided to work on
> > top
> > > > of
> > > > > > XML
> > > > > > > by
> > > > > > > > filtering/enhancing it instead of working at the POM (in
> > > > > > > > memory datamodel) level.
> > > > > > > > With the current understanding I have, by doing this choice
> of
> > > > > working
> > > > > > at
> > > > > > > > XML level, it looks like it was decided to bypass (if not
> kill)
> > > > core
> > > > > > > > extensions that enhance the POM itself and not the pom.xml ;
> > > here I
> > > > > can
> > > > > > > > think of (but probably not limited to):
> > > > > > > > - polyglot-maven: do not use XML but other format to describe
> > the
> > > > POM
> > > > > > > > (yaml, json, kotlin, java, other XML formats, ...)
> > > > > > > > - jgitver-maven-plugin (or forks like
> > > > > maven-git-versioning-extension):
> > > > > > > > dynamic computation of projects version based on git history
> > > > > > > >
> > > > > > > > With the introduction of core extensions, I thought it was a
> > move
> > > > to
> > > > > > open
> > > > > > > > the internals and let externals contribute to the
> capabilities
> > of
> > > > > > maven.
> > > > > > > > With the move to a XML handling chain, I see it as a
> > > > > > restriction/regress
> > > > > > > in
> > > > > > > > favor of core closed functionalities. An example of that is
> > what
> > > is
> > > > > > > > provided as CIFriendly stuff, IMO it could/should have been
> > > > provided
> > > > > > by a
> > > > > > > > plugin/extension but instead it is hard written in maven core
> > and
> > > > is
> > > > > > not
> > > > > > > > opened for external contribution (plugin/extension I mean).
> > > > > > > >
> > > > > > > > Perhaps I am totally wrong but I think that maven core should
> > > > define
> > > > > > all
> > > > > > > > its expectations at an API level so that extensions/plugins
> > could
> > > > > hook
> > > > > > at
> > > > > > > > this API level. The default packaging of maven could/should
> > > provide
> > > > > > > default
> > > > > > > > implementations of those expectations (for example reading a
> > > > pom.xml
> > > > > > file
> > > > > > > > to a POM model, dumping a POM to a pom-4.0.0.xml,
> > > > > > transforming/reducing a
> > > > > > > > POM to POM-consumer, dumping POM-consumer to
> > > > pom-consumer-5.0.0.xml,
> > > > > > ...)
> > > > > > > > and let extensions/plugins/default implementations work along
> > the
> > > > > build
> > > > > > > > process with the API & POMs to provide different features and
> > > > > > > capabilities.
> > > > > > > >
> > > > > > > > Matthieu
> > > > > > > >
> > > > > > > > On Thu, Dec 31, 2020 at 7:01 PM Robert Scholte wrote:
> > > > > > > >
> > > > > > > > > I've made a recording[1] about it, which hopefully answers
> > most
> > > > > > > > questions.
> > > > > > > > >
> > > > > > > > > Robert
> > > > > > > > >
> > > > > > > > > [1] https://youtu.be/KDAmlNKZJto
> > > > > > > > >
> > > > > > > > > On 31-12-2020 16:18:57, Matthieu Brouillard
> > > > > > > > > wrote:
> > > > > > > > > > Not exactly sure what work you mean
> > > > > > > > > everything related to maven-xml:
> > Build/ConsumerPomXMLFilterxxx,
> > > > > > > > > Build/ConsumerModelSourcexxxx and the transformer stuff.
> > > > > > > > > Especially, when looking at classes like
> > CiFriendlyXMLFilter, I
> > > > > would
> > > > > > > > have
> > > > > > > > > thought that such things could have been done elsewhere,
> > > working
> > > > on
> > > > > > the
> > > > > > > > > object model (not on the XML stuff) especially for the
> > BuildPom
> > > > > part.
> > > > > > > > >
> > > > > > > > > > however specifically the consumer POM integrates with so
> > many
> > > > > > > external
> > > > > > > > > ecosystems
> > > > > > > > > We're aligned here, this has to be stable and well defined
> > by a
> > > > > > schema.
> > > > > > > > >
> > > > > > > > > Matthieu
> > > > > > > > >
> > > > > > > > > On Thu, Dec 31, 2020 at 3:59 PM Bernd Eckenfels
> > > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > > Hello,
> > > > > > > > > >
> > > > > > > > > > Not exactly sure what work you mean and I fully agree
> that
> > > > using
> > > > > a
> > > > > > > core
> > > > > > > > > > model should still be the API for plugins and extensions
> to
> > > > work
> > > > > > > with,
> > > > > > > > > > however specifically the consumer POM integrates with so
> > many
> > > > > > > external
> > > > > > > > > > ecosystems, I would expect it to be defined in terms of
> XML
> > > > > Schema
> > > > > > > with
> > > > > > > > > > explicite semantic (and the inherent compatibility with
> > > exiting
> > > > > > > POMs).
> > > > > > > > > >
> > > > > > > > > > Gruss
> > > > > > > > > > Bernd
> > > > > > > > > > --
> > > > > > > > > > http://bernd.eckenfels.net
> > > > > > > > > > ________________________________
> > > > > > > > > > Von: Matthieu BROUILLARD
> > > > > > > > > > Gesendet: Thursday, December 31, 2020 3:19:09 PM
> > > > > > > > > > An: dev@maven.apache.org
> > > > > > > > > > Betreff: maven 4.0.0 new XML stuff
> > > > > > > > > >
> > > > > > > > > > Hello all,
> > > > > > > > > >
> > > > > > > > > > regarding the active work occurring for maven 4.0.0 I
> > noticed
> > > > the
> > > > > > > > > > introduction of a lot of new stuff around SAX parsing &
> > > > > filtering.
> > > > > > > > > > I am wondering if that means that it was decided that the
> > > input
> > > > > > > format
> > > > > > > > of
> > > > > > > > > > maven projects will be XML forever meaning probably,
> among
> > > > > others,
> > > > > > > the
> > > > > > > > > end
> > > > > > > > > > of polyglot extensions.
> > > > > > > > > > Could you explain such a move (or point to
> > > rationals/documents)
> > > > > and
> > > > > > > why
> > > > > > > > > you
> > > > > > > > > > did not leverage working on the in memory object model
> > > allowing
> > > > > > > > > > extensions/plugins to contribute/hook in the chain of
> > > building
> > > > > the
> > > > > > > > > BuildPOM
> > > > > > > > > > & ConsumePOM? In the past I really thought that this move
> > to
> > > > > 'Build
> > > > > > > vs
> > > > > > > > > > Consumer' POM would make clear separations between the
> > input
> > > > > format
> > > > > > > of
> > > > > > > > > > descriptors and the core system but I perhaps
> misunderstood
> > > > > things.
> > > > > > > > > >
> > > > > > > > > > Also, are there plans regarding the future of core
> > > extensions?
> > > > > > > > > > With core extensions it was possible to hook into the POM
> > > model
> > > > > > > loading
> > > > > > > > > and
> > > > > > > > > > do transformations to do dynamic changes but by working
> on
> > > the
> > > > > XML
> > > > > > > > > directly
> > > > > > > > > > I see a shift (if not red stop) in this
> > > contribution/delegation
> > > > > > > > > mechanism.
> > > > > > > > > >
> > > > > > > > > > Thanks for your time & answers.
> > > > > > > > > >
> > > > > > > > > > Matthieu Brouillard
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: maven 4.0.0 new XML stuff

Posted by Matthieu Brouillard <ma...@brouillard.fr>.
> Can you give an example?
Like Romain has shown: dynamically added dependencies or a version
computation.

> Most important is the support for CI-friendly versions
But if extensions dynamically compute and set the versions in the POM using
the API, the changes will not be reflected.
That's why today one has to use flatten-maven-plugin or any other
plugin/task modifying/dumping the POM model to a pom.xml file and
setting/attaching it the POM.

If the produced artifacts (consumer pom) were computed from the POM (object
model), every change done by any extension would be part of it.

> but this happens AFTER using the pom
Not always from the pom.xml. I thought extensions were allowed to provide
ModelLocator & ModelReader to both decide which file to use for a project
and how to build the in memory POM model.
So the truth should not be considered to be in the pom.xml but in the POM.


On Mon, Jan 4, 2021 at 10:00 PM Robert Scholte <rf...@apache.org> wrote:

> answers are below.
>
> Robert
>
> On 4-1-2021 16:52:23, Matthieu Brouillard <ma...@brouillard.fr> wrote:
> @Robert nothing is broken atm, the changes for consumer/build are currently
> behind your feature flag.
> Robert Scholte:
> It is active by default, so it is actually not hidden.
>
>
>
> But as I feared previously, and as Romain pointed it, by working at XML
> level (and not at POM level) the produced consumer pom does not reflect
> changes from extensions.
> Robert Scholte:
> Can you give an example?
>
>
> I really thought that all the "consumer/build" stuff would make the
> maven-flatten-plugin useless but it looks like it will not be the case if
> working at XML level.
> Robert Scholte:
> Like with most questions: it depends. Most important is the support for
> CI-friendly versions. In this case you won't need the flatten-maven-plugin
> anymore.
> However, the plugin can rewrite much more, but this happens AFTER using
> the pom.
> That's something I don't like, because this POM was not used to build the
> project, but it was reassembled afterwards.
> My idea is to provide hooks to parts of the pom that might be adjusted,
> but this is something we can work on during the 4.x releases
>
> Did Romain and I miss the whole point of the "consumer/build" enhancements
> or is it "just" because current implementation has not yet reached the
> targets/outputs?
>
> On Mon, Jan 4, 2021 at 2:56 PM Romain Manni-Bucau
> wrote:
>
> > Hmm, I don't get a few things of this IT:
> >
> > 1. the formatting seems not expected even if valid (the comments are
> > finishing with the first tag for example "-->
> > public class MyListener extends AbstractMavenLifecycleParticipant {
> >
> > @Override
> > public void afterProjectsRead(final MavenSession session) throws
> > MavenExecutionException {
> > if (session.getCurrentProject() == null) {
> > return;
> > }
> >
> > session.getProjects().forEach(p -> {
> > final Dependency dependency = new Dependency();
> > dependency.setGroupId("junit");
> > dependency.setArtifactId("junit");
> > dependency.setVersion("3.8.1");
> > p.getDependencies().add(dependency);
> > });
> > }
> > }
> >
> >
> > 2. If you run mvn (4 snapshot) dependency:tree you get this kind of
> output:
> >
> > [INFO] -------------< org.sonatype.mavenbook.multi:simple-webapp
> > >-------------
> > [INFO] Building Multi Chapter Simple Web Application Project
> > 0.9-MNG6957-SNAPSHOT [6/6]
> > [INFO] --------------------------------[ jar
> > ]---------------------------------
> > [INFO]
> > [INFO] --- maven-dependency-plugin:3.1.2:tree (default-cli) @
> simple-webapp
> > ---
> > [INFO]
> org.sonatype.mavenbook.multi:simple-webapp:jar:0.9-MNG6957-SNAPSHOT
> > [INFO] +-
> >
> >
> org.sonatype.mavenbook.multi:simple-weather:jar:0.9-MNG6957-SNAPSHOT:compile
> > [INFO] \- junit:junit:jar:3.8.1:compile
> > <-- THIS IS WHAT WE WANT TO SEE
> > [INFO]
> > ------------------------------------------------------------------------
> >
> > 3. run the build to have produced pom and cat the simple-webapp one:
> >
> >
> > 4.0.0
> >
> > org.sonatype.mavenbook.multi
> > simple-parent
> > 0.9-MNG6957-SNAPSHOT
> >
> >
> > simple-webapp
> > Multi Chapter Simple Web Application Project
> >
> >
> > org.sonatype.mavenbook.multi
> > simple-weather
> > 0.9-MNG6957-SNAPSHOT
> >
> >
> >
> > simple-webapp
> >
> >
> >
> > org.apache.maven.plugins
> > maven-war-plugin
> > 2.6
> >
> >
> >
> >
> >
> >
> > As you see the dependency is not there. I guess the expected outout is:
> >
> >
> >
> > 4.0.0
> > simple-webapp
> > Multi Chapter Simple Web Application Project
> > [description, scm, ..., all central requires sections but not build ones]
> >
> >
> > org.sonatype.mavenbook.multi
> > simple-weather
> > 0.9-MNG6957-SNAPSHOT
> >
> >
> > junit
> > junit
> > 3.8.1
> >
> >
> >
> >
> > Am I missing something?
> >
> > Romain Manni-Bucau
> > @rmannibucau | Blog
> > | Old Blog
> > | Github <
> > https://github.com/rmannibucau> |
> > LinkedIn | Book
> > <
> >
> https://www.packtpub.com/application-development/java-ee-8-high-performance
> > >
> >
> >
> > Le lun. 4 janv. 2021 à 13:41, Robert Scholte a
> > écrit :
> >
> > >
> > >
> >
> https://github.com/apache/maven-integration-testing/tree/master/core-it-suite/src/test/resources/mng-6957-buildconsumer
> > > is the most complete IT
> > >
> > > On 4-1-2021 12:59:51, Romain Manni-Bucau wrote:
> > > Le lun. 4 janv. 2021 à 12:36, Robert Scholte a
> > > écrit :
> > >
> > > > There's just one thing I want to say:
> > > > I'm having trouble with the term "broken".
> > > >
> > >
> > > Well, literally meant broken as decorelated from the user intent and
> > > extension model.
> > > Anyway, didn't intend to blame but more identify the blockers for a GA
> so
> > > point was really that it seem that on the two sides only the producing
> > one
> > > is not yet ready since it keeps does not sanitize the model completely
> > and
> > > keeps build only data like comments, right? Also not yet clear for me
> if
> > we
> > > loose the extension enrichments there.
> > >
> > >
> > > > If a Maven project could be built with M3.6.3, it can still be built
> > with
> > > > M4.
> > > > If not, it is either regression (MNG-6957, MNG-7063) which must be
> > fixed,
> > > > or it requires changes to a plugin for understandable reasons
> > > > (maven-pgp-plugin)
> > > > AFAIK an interesting extension like the maven-tiles has been tested
> and
> > > > still works.
> > > >
> > >
> > > Do you have this handy, is it in our test suite? I'd like to check the
> > > produced pom matches the enriched model but happy to start from
> something
> > > already there.
> > >
> > >
> > > >
> > > > Robert
> > > >
> > > > On 3-1-2021 19:35:25, Romain Manni-Bucau wrote:
> > > > Le dim. 3 janv. 2021 à 19:04, Robert Scholte a
> > > > écrit :
> > > >
> > > > > I don't remember all those details anymore, because I hit those in
> > the
> > > > > beginning.
> > > > > Trying things over and over again I decided that this is probably
> the
> > > > most
> > > > > successful approach.
> > > > >
> > > > >
> > > > > What of the goals was to keep the pom.xml as is as much as
> possible.
> > > > > We can only decide for the specific Maven elements how to handle
> > them,
> > > we
> > > > > should not decide about comments and licenses.
> > > > > BTW, the license issue was hard to solve. You cannot use it from
> the
> > > > pom's
> > > > > , because there might be multiple licenses.
> > > > >
> > > >
> > > > I disagree, it is saner IMO to evolve to support that than doing
> > > > anything else.
> > > > Once again you keep things which don't make sense in a consumed pom
> in
> > > > current impl so i'd say the sucess in a few cases breaks as much
> cases
> > so
> > > > we need to revisit anyway IMHO.
> > > >
> > > >
> > > > >
> > > > > The current implementation is a solid way to ensure we're not
> > breaking
> > > > too
> > > > > much, because Maven controls the XML filters.
> > > > >
> > > >
> > > > Hmm, breaking extensions seems to break too much (I'm not speaking of
> > > other
> > > > parts which breaks the ecosystem there but just that is sufficient
> IMHO
> > > to
> > > > say we must check back our solution).
> > > >
> > > >
> > > > >
> > > > > Also keep in mind, that I only want Maven to decide which
> > modifications
> > > > > are done.
> > > > >
> > > >
> > > > For the consumed pom I agree but it is consistent with keeping
> > everything
> > > > working instead of breaking too, no?
> > > >
> > > >
> > > > >
> > > > > Current polyglot projects should still work, but they cannot
> benefit
> > > from
> > > > > the build/consumer functions yet.
> > > > >
> > > >
> > > > So pom -> build model is kept, build model -> produced pom is broken?
> > Is
> > > it
> > > > the complete status?
> > > > Sounds ok for a 4.0 and a 4.1 can fix it if so.
> > > > Just want to ensure first part is not broken at all.
> > > >
> > > >
> > > > >
> > > > > Robert
> > > > > On 3-1-2021 16:38:38, Romain Manni-Bucau wrote:
> > > > > Le dim. 3 janv. 2021 à 16:18, Robert Scholte a
> > > > > écrit :
> > > > >
> > > > > > > So what I was expecting was: raw xml model -> converted to
> > unified
> > > > > > consumed model -> extensions -> model processing.
> > > > > >
> > > > > > This is only the build pom part. You're missing the consume part,
> > > where
> > > > > > the xml is distributed.
> > > > > >
> > > > >
> > > > > Yes but with previous chain the consume part is "clean/normalize ->
> > > dump"
> > > > > since we are using consumed model - only standard model - in memory
> > > > > already.
> > > > >
> > > > >
> > > > > > Build is raw + enrich, consumer is raw + enrich + reduce
> (removing
> > > > > > relativePath and modules are the first examples, but much more is
> > > > > possible)
> > > > > >
> > > > > > Going for the in memory was also my first thought, but I would
> > loose
> > > > > > information, hence I came up with the current implementation.
> > > > > >
> > > > >
> > > > > I don't see what you loose ot be honest.
> > > > > You mentionned license but this one is in the pom so not a big
> deal,
> > > > > comments which are undesired IMHO as mentionned and element order
> > which
> > > > can
> > > > > really be discussed since we can desire to enforce an order to
> > > normalize
> > > > > consumption + it shouldn't be important since from the project
> point
> > of
> > > > > view your pom is already "broken"/lost (as all your intelligence is
> > > lost
> > > > by
> > > > > this "not passthrough" process).
> > > > > So overall I don't see what you would loose from the consumer side
> > but
> > > I
> > > > > see what you lost from maven ecosystem side.
> > > > >
> > > > >
> > > > > > Again, we're at a point where we can have counter solutions, but
> > > don't
> > > > > > expect me to implement it.
> > > > > >
> > > > >
> > > > > For now I'm just trying to ensure we agree we don't want to break
> > > > existing
> > > > > extensions and the nice ecosystem we built after years.
> > > > > This was really a move forward and it sounds like we broke it at
> > maven
> > > 4
> > > > > without any user gain which sounds terrible.
> > > > >
> > > > >
> > > > > >
> > > > > > thanks,
> > > > > > Robert
> > > > > > On 3-1-2021 15:25:21, Romain Manni-Bucau wrote:
> > > > > > Hi all,
> > > > > >
> > > > > > I kind of join Matthieu thoughts there, there is no point to work
> > at
> > > > xml
> > > > > > level to create the consumed pom - comments is not a point since
> it
> > > can
> > > > > > commonly/easily refer to a dropped part of the pom so they should
> > be
> > > > > > stripped.
> > > > > > Current extension model got proven adapted and adopted, using a
> > lower
> > > > > level
> > > > > > extension API will not since XML is, even if still mainstream,
> > often
> > > > > > replaced by alternative configurations and to have done the work
> to
> > > > > inject
> > > > > > XML configuration programmatically compred to current option, it
> > is a
> > > > > pain.
> > > > > > The in memory model should stick to consumed model IMHO - being
> > > > > > programmatic there is no point to make it easier, worse case we
> can
> > > add
> > > > > > helper beans (injectable) but in terms of model it will not help.
> > > > > >
> > > > > > So what I was expecting was:
> > > > > >
> > > > > > raw xml model -> converted to unified consumed model ->
> extensions
> > ->
> > > > > model
> > > > > > processing.
> > > > > >
> > > > > > Indeed, real chain adds a small processing over the first arrow
> > > (inject
> > > > > > versions for example) but nothing crazy and breaking this overall
> > > flow
> > > > > > which stays user friendly.
> > > > > >
> > > > > > Strictly speaking the new model is just a built-in extension for
> me
> > > > which
> > > > > > is particular because it will enforce IDE to integrate a new
> > format -
> > > > > > wheres polyglot extensions or others don't require static
> analyzis
> > by
> > > > > > themself not being "standard".
> > > > > >
> > > > > > That said, there is nothing crazy with current implementation, it
> > > just
> > > > > > require to be updated to be able to take extension changes into
> > > > account.
> > > > > > This can be done by making the extension model 'spyable' (ie if a
> > > > > > dependency/plugin is added it will be reflected in the final
> > written
> > > > > > pom.xml).
> > > > > > This sounds - instrumenting the extension model API or doing a
> diff
> > > > after
> > > > > > extension phase - like a compromise and let people gets the best
> of
> > > > both
> > > > > > worlds to me.
> > > > > >
> > > > > > Wdyt?
> > > > > >
> > > > > > Romain Manni-Bucau
> > > > > > @rmannibucau | Blog
> > > > > > | Old Blog
> > > > > > | Github |
> > > > > > LinkedIn | Book
> > > > > >
> > > > > >
> > > > > >
> > > > > > Le dim. 3 janv. 2021 à 14:46, Robert Scholte a
> > > > > > écrit :
> > > > > >
> > > > > > > Hi Matthieu,
> > > > > > >
> > > > > > > As you understand, something had to be changed to move Maven
> > > forward.
> > > > > > > I've decided to pick up that challenge and came up with the
> > current
> > > > > > > solution.
> > > > > > >
> > > > > > > My main concerns was that I wanted to keep the fileModel as
> much
> > as
> > > > is.
> > > > > > > That includes the license, comments and element order.
> > > > > > > This information if not available in the memory model, so I
> > needed
> > > > the
> > > > > > > original pom file.
> > > > > > > With that in mind, the usage of XMLFilters looks like the most
> > > > > > appropriate
> > > > > > > solution.
> > > > > > >
> > > > > > > I am certain that XML is still the most used format, so if we
> can
> > > > have
> > > > > > > improvements for those users, I'm already very happy.
> > > > > > >
> > > > > > > And yes, there are plugins that needs to be updated, but doing
> > > > nothing
> > > > > is
> > > > > > > not an option anymore.
> > > > > > >
> > > > > > > There are more people that share their concerns, but it took me
> > > > several
> > > > > > > years to reach this point.
> > > > > > > We now have something that seems to work, anybody who can
> improve
> > > or
> > > > > can
> > > > > > > come up with an alternative implementation can do so.
> > > > > > >
> > > > > > > thanks,
> > > > > > > Robert
> > > > > > > On 3-1-2021 12:55:41, Matthieu Brouillard wrote:
> > > > > > > Thanks Robert for the video link.
> > > > > > >
> > > > > > > I fully understand the rationales behind the separation of
> > > > > > > build/consumer pom and the video provides some insights on it
> and
> > > you
> > > > > > > explain the actual implementation to introduce this change.
> > > > > > > Still I do not fully understand why it was decided to work on
> top
> > > of
> > > > > XML
> > > > > > by
> > > > > > > filtering/enhancing it instead of working at the POM (in
> > > > > > > memory datamodel) level.
> > > > > > > With the current understanding I have, by doing this choice of
> > > > working
> > > > > at
> > > > > > > XML level, it looks like it was decided to bypass (if not kill)
> > > core
> > > > > > > extensions that enhance the POM itself and not the pom.xml ;
> > here I
> > > > can
> > > > > > > think of (but probably not limited to):
> > > > > > > - polyglot-maven: do not use XML but other format to describe
> the
> > > POM
> > > > > > > (yaml, json, kotlin, java, other XML formats, ...)
> > > > > > > - jgitver-maven-plugin (or forks like
> > > > maven-git-versioning-extension):
> > > > > > > dynamic computation of projects version based on git history
> > > > > > >
> > > > > > > With the introduction of core extensions, I thought it was a
> move
> > > to
> > > > > open
> > > > > > > the internals and let externals contribute to the capabilities
> of
> > > > > maven.
> > > > > > > With the move to a XML handling chain, I see it as a
> > > > > restriction/regress
> > > > > > in
> > > > > > > favor of core closed functionalities. An example of that is
> what
> > is
> > > > > > > provided as CIFriendly stuff, IMO it could/should have been
> > > provided
> > > > > by a
> > > > > > > plugin/extension but instead it is hard written in maven core
> and
> > > is
> > > > > not
> > > > > > > opened for external contribution (plugin/extension I mean).
> > > > > > >
> > > > > > > Perhaps I am totally wrong but I think that maven core should
> > > define
> > > > > all
> > > > > > > its expectations at an API level so that extensions/plugins
> could
> > > > hook
> > > > > at
> > > > > > > this API level. The default packaging of maven could/should
> > provide
> > > > > > default
> > > > > > > implementations of those expectations (for example reading a
> > > pom.xml
> > > > > file
> > > > > > > to a POM model, dumping a POM to a pom-4.0.0.xml,
> > > > > transforming/reducing a
> > > > > > > POM to POM-consumer, dumping POM-consumer to
> > > pom-consumer-5.0.0.xml,
> > > > > ...)
> > > > > > > and let extensions/plugins/default implementations work along
> the
> > > > build
> > > > > > > process with the API & POMs to provide different features and
> > > > > > capabilities.
> > > > > > >
> > > > > > > Matthieu
> > > > > > >
> > > > > > > On Thu, Dec 31, 2020 at 7:01 PM Robert Scholte wrote:
> > > > > > >
> > > > > > > > I've made a recording[1] about it, which hopefully answers
> most
> > > > > > > questions.
> > > > > > > >
> > > > > > > > Robert
> > > > > > > >
> > > > > > > > [1] https://youtu.be/KDAmlNKZJto
> > > > > > > >
> > > > > > > > On 31-12-2020 16:18:57, Matthieu Brouillard
> > > > > > > > wrote:
> > > > > > > > > Not exactly sure what work you mean
> > > > > > > > everything related to maven-xml:
> Build/ConsumerPomXMLFilterxxx,
> > > > > > > > Build/ConsumerModelSourcexxxx and the transformer stuff.
> > > > > > > > Especially, when looking at classes like
> CiFriendlyXMLFilter, I
> > > > would
> > > > > > > have
> > > > > > > > thought that such things could have been done elsewhere,
> > working
> > > on
> > > > > the
> > > > > > > > object model (not on the XML stuff) especially for the
> BuildPom
> > > > part.
> > > > > > > >
> > > > > > > > > however specifically the consumer POM integrates with so
> many
> > > > > > external
> > > > > > > > ecosystems
> > > > > > > > We're aligned here, this has to be stable and well defined
> by a
> > > > > schema.
> > > > > > > >
> > > > > > > > Matthieu
> > > > > > > >
> > > > > > > > On Thu, Dec 31, 2020 at 3:59 PM Bernd Eckenfels
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > > > Hello,
> > > > > > > > >
> > > > > > > > > Not exactly sure what work you mean and I fully agree that
> > > using
> > > > a
> > > > > > core
> > > > > > > > > model should still be the API for plugins and extensions to
> > > work
> > > > > > with,
> > > > > > > > > however specifically the consumer POM integrates with so
> many
> > > > > > external
> > > > > > > > > ecosystems, I would expect it to be defined in terms of XML
> > > > Schema
> > > > > > with
> > > > > > > > > explicite semantic (and the inherent compatibility with
> > exiting
> > > > > > POMs).
> > > > > > > > >
> > > > > > > > > Gruss
> > > > > > > > > Bernd
> > > > > > > > > --
> > > > > > > > > http://bernd.eckenfels.net
> > > > > > > > > ________________________________
> > > > > > > > > Von: Matthieu BROUILLARD
> > > > > > > > > Gesendet: Thursday, December 31, 2020 3:19:09 PM
> > > > > > > > > An: dev@maven.apache.org
> > > > > > > > > Betreff: maven 4.0.0 new XML stuff
> > > > > > > > >
> > > > > > > > > Hello all,
> > > > > > > > >
> > > > > > > > > regarding the active work occurring for maven 4.0.0 I
> noticed
> > > the
> > > > > > > > > introduction of a lot of new stuff around SAX parsing &
> > > > filtering.
> > > > > > > > > I am wondering if that means that it was decided that the
> > input
> > > > > > format
> > > > > > > of
> > > > > > > > > maven projects will be XML forever meaning probably, among
> > > > others,
> > > > > > the
> > > > > > > > end
> > > > > > > > > of polyglot extensions.
> > > > > > > > > Could you explain such a move (or point to
> > rationals/documents)
> > > > and
> > > > > > why
> > > > > > > > you
> > > > > > > > > did not leverage working on the in memory object model
> > allowing
> > > > > > > > > extensions/plugins to contribute/hook in the chain of
> > building
> > > > the
> > > > > > > > BuildPOM
> > > > > > > > > & ConsumePOM? In the past I really thought that this move
> to
> > > > 'Build
> > > > > > vs
> > > > > > > > > Consumer' POM would make clear separations between the
> input
> > > > format
> > > > > > of
> > > > > > > > > descriptors and the core system but I perhaps misunderstood
> > > > things.
> > > > > > > > >
> > > > > > > > > Also, are there plans regarding the future of core
> > extensions?
> > > > > > > > > With core extensions it was possible to hook into the POM
> > model
> > > > > > loading
> > > > > > > > and
> > > > > > > > > do transformations to do dynamic changes but by working on
> > the
> > > > XML
> > > > > > > > directly
> > > > > > > > > I see a shift (if not red stop) in this
> > contribution/delegation
> > > > > > > > mechanism.
> > > > > > > > >
> > > > > > > > > Thanks for your time & answers.
> > > > > > > > >
> > > > > > > > > Matthieu Brouillard
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: maven 4.0.0 new XML stuff

Posted by Robert Scholte <rf...@apache.org>.
answers are below.

Robert

On 4-1-2021 16:52:23, Matthieu Brouillard <ma...@brouillard.fr> wrote:
@Robert nothing is broken atm, the changes for consumer/build are currently
behind your feature flag.
Robert Scholte: 
It is active by default, so it is actually not hidden.



But as I feared previously, and as Romain pointed it, by working at XML
level (and not at POM level) the produced consumer pom does not reflect
changes from extensions.
Robert Scholte: 
Can you give an example?


I really thought that all the "consumer/build" stuff would make the
maven-flatten-plugin useless but it looks like it will not be the case if
working at XML level.
Robert Scholte: 
Like with most questions: it depends. Most important is the support for CI-friendly versions. In this case you won't need the flatten-maven-plugin anymore.
However, the plugin can rewrite much more, but this happens AFTER using the pom.
That's something I don't like, because this POM was not used to build the project, but it was reassembled afterwards.
My idea is to provide hooks to parts of the pom that might be adjusted, but this is something we can work on during the 4.x releases

Did Romain and I miss the whole point of the "consumer/build" enhancements
or is it "just" because current implementation has not yet reached the
targets/outputs?

On Mon, Jan 4, 2021 at 2:56 PM Romain Manni-Bucau
wrote:

> Hmm, I don't get a few things of this IT:
>
> 1. the formatting seems not expected even if valid (the comments are
> finishing with the first tag for example "-->
> public class MyListener extends AbstractMavenLifecycleParticipant {
>
> @Override
> public void afterProjectsRead(final MavenSession session) throws
> MavenExecutionException {
> if (session.getCurrentProject() == null) {
> return;
> }
>
> session.getProjects().forEach(p -> {
> final Dependency dependency = new Dependency();
> dependency.setGroupId("junit");
> dependency.setArtifactId("junit");
> dependency.setVersion("3.8.1");
> p.getDependencies().add(dependency);
> });
> }
> }
>
>
> 2. If you run mvn (4 snapshot) dependency:tree you get this kind of output:
>
> [INFO] -------------< org.sonatype.mavenbook.multi:simple-webapp
> >-------------
> [INFO] Building Multi Chapter Simple Web Application Project
> 0.9-MNG6957-SNAPSHOT [6/6]
> [INFO] --------------------------------[ jar
> ]---------------------------------
> [INFO]
> [INFO] --- maven-dependency-plugin:3.1.2:tree (default-cli) @ simple-webapp
> ---
> [INFO] org.sonatype.mavenbook.multi:simple-webapp:jar:0.9-MNG6957-SNAPSHOT
> [INFO] +-
>
> org.sonatype.mavenbook.multi:simple-weather:jar:0.9-MNG6957-SNAPSHOT:compile
> [INFO] \- junit:junit:jar:3.8.1:compile
> <-- THIS IS WHAT WE WANT TO SEE
> [INFO]
> ------------------------------------------------------------------------
>
> 3. run the build to have produced pom and cat the simple-webapp one:
>
>
> 4.0.0
>
> org.sonatype.mavenbook.multi
> simple-parent
> 0.9-MNG6957-SNAPSHOT
>
>
> simple-webapp
> Multi Chapter Simple Web Application Project
>
>
> org.sonatype.mavenbook.multi
> simple-weather
> 0.9-MNG6957-SNAPSHOT
>
>
>
> simple-webapp
>
>
>
> org.apache.maven.plugins
> maven-war-plugin
> 2.6
>
>
>
>
>
>
> As you see the dependency is not there. I guess the expected outout is:
>
>
>
> 4.0.0
> simple-webapp
> Multi Chapter Simple Web Application Project
> [description, scm, ..., all central requires sections but not build ones]
>
>
> org.sonatype.mavenbook.multi
> simple-weather
> 0.9-MNG6957-SNAPSHOT
>
>
> junit
> junit
> 3.8.1
>
>
>
>
> Am I missing something?
>
> Romain Manni-Bucau
> @rmannibucau | Blog
> | Old Blog
> | Github <
> https://github.com/rmannibucau> |
> LinkedIn | Book
> <
> https://www.packtpub.com/application-development/java-ee-8-high-performance
> >
>
>
> Le lun. 4 janv. 2021 à 13:41, Robert Scholte a
> écrit :
>
> >
> >
> https://github.com/apache/maven-integration-testing/tree/master/core-it-suite/src/test/resources/mng-6957-buildconsumer
> > is the most complete IT
> >
> > On 4-1-2021 12:59:51, Romain Manni-Bucau wrote:
> > Le lun. 4 janv. 2021 à 12:36, Robert Scholte a
> > écrit :
> >
> > > There's just one thing I want to say:
> > > I'm having trouble with the term "broken".
> > >
> >
> > Well, literally meant broken as decorelated from the user intent and
> > extension model.
> > Anyway, didn't intend to blame but more identify the blockers for a GA so
> > point was really that it seem that on the two sides only the producing
> one
> > is not yet ready since it keeps does not sanitize the model completely
> and
> > keeps build only data like comments, right? Also not yet clear for me if
> we
> > loose the extension enrichments there.
> >
> >
> > > If a Maven project could be built with M3.6.3, it can still be built
> with
> > > M4.
> > > If not, it is either regression (MNG-6957, MNG-7063) which must be
> fixed,
> > > or it requires changes to a plugin for understandable reasons
> > > (maven-pgp-plugin)
> > > AFAIK an interesting extension like the maven-tiles has been tested and
> > > still works.
> > >
> >
> > Do you have this handy, is it in our test suite? I'd like to check the
> > produced pom matches the enriched model but happy to start from something
> > already there.
> >
> >
> > >
> > > Robert
> > >
> > > On 3-1-2021 19:35:25, Romain Manni-Bucau wrote:
> > > Le dim. 3 janv. 2021 à 19:04, Robert Scholte a
> > > écrit :
> > >
> > > > I don't remember all those details anymore, because I hit those in
> the
> > > > beginning.
> > > > Trying things over and over again I decided that this is probably the
> > > most
> > > > successful approach.
> > > >
> > > >
> > > > What of the goals was to keep the pom.xml as is as much as possible.
> > > > We can only decide for the specific Maven elements how to handle
> them,
> > we
> > > > should not decide about comments and licenses.
> > > > BTW, the license issue was hard to solve. You cannot use it from the
> > > pom's
> > > > , because there might be multiple licenses.
> > > >
> > >
> > > I disagree, it is saner IMO to evolve to support that than doing
> > > anything else.
> > > Once again you keep things which don't make sense in a consumed pom in
> > > current impl so i'd say the sucess in a few cases breaks as much cases
> so
> > > we need to revisit anyway IMHO.
> > >
> > >
> > > >
> > > > The current implementation is a solid way to ensure we're not
> breaking
> > > too
> > > > much, because Maven controls the XML filters.
> > > >
> > >
> > > Hmm, breaking extensions seems to break too much (I'm not speaking of
> > other
> > > parts which breaks the ecosystem there but just that is sufficient IMHO
> > to
> > > say we must check back our solution).
> > >
> > >
> > > >
> > > > Also keep in mind, that I only want Maven to decide which
> modifications
> > > > are done.
> > > >
> > >
> > > For the consumed pom I agree but it is consistent with keeping
> everything
> > > working instead of breaking too, no?
> > >
> > >
> > > >
> > > > Current polyglot projects should still work, but they cannot benefit
> > from
> > > > the build/consumer functions yet.
> > > >
> > >
> > > So pom -> build model is kept, build model -> produced pom is broken?
> Is
> > it
> > > the complete status?
> > > Sounds ok for a 4.0 and a 4.1 can fix it if so.
> > > Just want to ensure first part is not broken at all.
> > >
> > >
> > > >
> > > > Robert
> > > > On 3-1-2021 16:38:38, Romain Manni-Bucau wrote:
> > > > Le dim. 3 janv. 2021 à 16:18, Robert Scholte a
> > > > écrit :
> > > >
> > > > > > So what I was expecting was: raw xml model -> converted to
> unified
> > > > > consumed model -> extensions -> model processing.
> > > > >
> > > > > This is only the build pom part. You're missing the consume part,
> > where
> > > > > the xml is distributed.
> > > > >
> > > >
> > > > Yes but with previous chain the consume part is "clean/normalize ->
> > dump"
> > > > since we are using consumed model - only standard model - in memory
> > > > already.
> > > >
> > > >
> > > > > Build is raw + enrich, consumer is raw + enrich + reduce (removing
> > > > > relativePath and modules are the first examples, but much more is
> > > > possible)
> > > > >
> > > > > Going for the in memory was also my first thought, but I would
> loose
> > > > > information, hence I came up with the current implementation.
> > > > >
> > > >
> > > > I don't see what you loose ot be honest.
> > > > You mentionned license but this one is in the pom so not a big deal,
> > > > comments which are undesired IMHO as mentionned and element order
> which
> > > can
> > > > really be discussed since we can desire to enforce an order to
> > normalize
> > > > consumption + it shouldn't be important since from the project point
> of
> > > > view your pom is already "broken"/lost (as all your intelligence is
> > lost
> > > by
> > > > this "not passthrough" process).
> > > > So overall I don't see what you would loose from the consumer side
> but
> > I
> > > > see what you lost from maven ecosystem side.
> > > >
> > > >
> > > > > Again, we're at a point where we can have counter solutions, but
> > don't
> > > > > expect me to implement it.
> > > > >
> > > >
> > > > For now I'm just trying to ensure we agree we don't want to break
> > > existing
> > > > extensions and the nice ecosystem we built after years.
> > > > This was really a move forward and it sounds like we broke it at
> maven
> > 4
> > > > without any user gain which sounds terrible.
> > > >
> > > >
> > > > >
> > > > > thanks,
> > > > > Robert
> > > > > On 3-1-2021 15:25:21, Romain Manni-Bucau wrote:
> > > > > Hi all,
> > > > >
> > > > > I kind of join Matthieu thoughts there, there is no point to work
> at
> > > xml
> > > > > level to create the consumed pom - comments is not a point since it
> > can
> > > > > commonly/easily refer to a dropped part of the pom so they should
> be
> > > > > stripped.
> > > > > Current extension model got proven adapted and adopted, using a
> lower
> > > > level
> > > > > extension API will not since XML is, even if still mainstream,
> often
> > > > > replaced by alternative configurations and to have done the work to
> > > > inject
> > > > > XML configuration programmatically compred to current option, it
> is a
> > > > pain.
> > > > > The in memory model should stick to consumed model IMHO - being
> > > > > programmatic there is no point to make it easier, worse case we can
> > add
> > > > > helper beans (injectable) but in terms of model it will not help.
> > > > >
> > > > > So what I was expecting was:
> > > > >
> > > > > raw xml model -> converted to unified consumed model -> extensions
> ->
> > > > model
> > > > > processing.
> > > > >
> > > > > Indeed, real chain adds a small processing over the first arrow
> > (inject
> > > > > versions for example) but nothing crazy and breaking this overall
> > flow
> > > > > which stays user friendly.
> > > > >
> > > > > Strictly speaking the new model is just a built-in extension for me
> > > which
> > > > > is particular because it will enforce IDE to integrate a new
> format -
> > > > > wheres polyglot extensions or others don't require static analyzis
> by
> > > > > themself not being "standard".
> > > > >
> > > > > That said, there is nothing crazy with current implementation, it
> > just
> > > > > require to be updated to be able to take extension changes into
> > > account.
> > > > > This can be done by making the extension model 'spyable' (ie if a
> > > > > dependency/plugin is added it will be reflected in the final
> written
> > > > > pom.xml).
> > > > > This sounds - instrumenting the extension model API or doing a diff
> > > after
> > > > > extension phase - like a compromise and let people gets the best of
> > > both
> > > > > worlds to me.
> > > > >
> > > > > Wdyt?
> > > > >
> > > > > Romain Manni-Bucau
> > > > > @rmannibucau | Blog
> > > > > | Old Blog
> > > > > | Github |
> > > > > LinkedIn | Book
> > > > >
> > > > >
> > > > >
> > > > > Le dim. 3 janv. 2021 à 14:46, Robert Scholte a
> > > > > écrit :
> > > > >
> > > > > > Hi Matthieu,
> > > > > >
> > > > > > As you understand, something had to be changed to move Maven
> > forward.
> > > > > > I've decided to pick up that challenge and came up with the
> current
> > > > > > solution.
> > > > > >
> > > > > > My main concerns was that I wanted to keep the fileModel as much
> as
> > > is.
> > > > > > That includes the license, comments and element order.
> > > > > > This information if not available in the memory model, so I
> needed
> > > the
> > > > > > original pom file.
> > > > > > With that in mind, the usage of XMLFilters looks like the most
> > > > > appropriate
> > > > > > solution.
> > > > > >
> > > > > > I am certain that XML is still the most used format, so if we can
> > > have
> > > > > > improvements for those users, I'm already very happy.
> > > > > >
> > > > > > And yes, there are plugins that needs to be updated, but doing
> > > nothing
> > > > is
> > > > > > not an option anymore.
> > > > > >
> > > > > > There are more people that share their concerns, but it took me
> > > several
> > > > > > years to reach this point.
> > > > > > We now have something that seems to work, anybody who can improve
> > or
> > > > can
> > > > > > come up with an alternative implementation can do so.
> > > > > >
> > > > > > thanks,
> > > > > > Robert
> > > > > > On 3-1-2021 12:55:41, Matthieu Brouillard wrote:
> > > > > > Thanks Robert for the video link.
> > > > > >
> > > > > > I fully understand the rationales behind the separation of
> > > > > > build/consumer pom and the video provides some insights on it and
> > you
> > > > > > explain the actual implementation to introduce this change.
> > > > > > Still I do not fully understand why it was decided to work on top
> > of
> > > > XML
> > > > > by
> > > > > > filtering/enhancing it instead of working at the POM (in
> > > > > > memory datamodel) level.
> > > > > > With the current understanding I have, by doing this choice of
> > > working
> > > > at
> > > > > > XML level, it looks like it was decided to bypass (if not kill)
> > core
> > > > > > extensions that enhance the POM itself and not the pom.xml ;
> here I
> > > can
> > > > > > think of (but probably not limited to):
> > > > > > - polyglot-maven: do not use XML but other format to describe the
> > POM
> > > > > > (yaml, json, kotlin, java, other XML formats, ...)
> > > > > > - jgitver-maven-plugin (or forks like
> > > maven-git-versioning-extension):
> > > > > > dynamic computation of projects version based on git history
> > > > > >
> > > > > > With the introduction of core extensions, I thought it was a move
> > to
> > > > open
> > > > > > the internals and let externals contribute to the capabilities of
> > > > maven.
> > > > > > With the move to a XML handling chain, I see it as a
> > > > restriction/regress
> > > > > in
> > > > > > favor of core closed functionalities. An example of that is what
> is
> > > > > > provided as CIFriendly stuff, IMO it could/should have been
> > provided
> > > > by a
> > > > > > plugin/extension but instead it is hard written in maven core and
> > is
> > > > not
> > > > > > opened for external contribution (plugin/extension I mean).
> > > > > >
> > > > > > Perhaps I am totally wrong but I think that maven core should
> > define
> > > > all
> > > > > > its expectations at an API level so that extensions/plugins could
> > > hook
> > > > at
> > > > > > this API level. The default packaging of maven could/should
> provide
> > > > > default
> > > > > > implementations of those expectations (for example reading a
> > pom.xml
> > > > file
> > > > > > to a POM model, dumping a POM to a pom-4.0.0.xml,
> > > > transforming/reducing a
> > > > > > POM to POM-consumer, dumping POM-consumer to
> > pom-consumer-5.0.0.xml,
> > > > ...)
> > > > > > and let extensions/plugins/default implementations work along the
> > > build
> > > > > > process with the API & POMs to provide different features and
> > > > > capabilities.
> > > > > >
> > > > > > Matthieu
> > > > > >
> > > > > > On Thu, Dec 31, 2020 at 7:01 PM Robert Scholte wrote:
> > > > > >
> > > > > > > I've made a recording[1] about it, which hopefully answers most
> > > > > > questions.
> > > > > > >
> > > > > > > Robert
> > > > > > >
> > > > > > > [1] https://youtu.be/KDAmlNKZJto
> > > > > > >
> > > > > > > On 31-12-2020 16:18:57, Matthieu Brouillard
> > > > > > > wrote:
> > > > > > > > Not exactly sure what work you mean
> > > > > > > everything related to maven-xml: Build/ConsumerPomXMLFilterxxx,
> > > > > > > Build/ConsumerModelSourcexxxx and the transformer stuff.
> > > > > > > Especially, when looking at classes like CiFriendlyXMLFilter, I
> > > would
> > > > > > have
> > > > > > > thought that such things could have been done elsewhere,
> working
> > on
> > > > the
> > > > > > > object model (not on the XML stuff) especially for the BuildPom
> > > part.
> > > > > > >
> > > > > > > > however specifically the consumer POM integrates with so many
> > > > > external
> > > > > > > ecosystems
> > > > > > > We're aligned here, this has to be stable and well defined by a
> > > > schema.
> > > > > > >
> > > > > > > Matthieu
> > > > > > >
> > > > > > > On Thu, Dec 31, 2020 at 3:59 PM Bernd Eckenfels
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Hello,
> > > > > > > >
> > > > > > > > Not exactly sure what work you mean and I fully agree that
> > using
> > > a
> > > > > core
> > > > > > > > model should still be the API for plugins and extensions to
> > work
> > > > > with,
> > > > > > > > however specifically the consumer POM integrates with so many
> > > > > external
> > > > > > > > ecosystems, I would expect it to be defined in terms of XML
> > > Schema
> > > > > with
> > > > > > > > explicite semantic (and the inherent compatibility with
> exiting
> > > > > POMs).
> > > > > > > >
> > > > > > > > Gruss
> > > > > > > > Bernd
> > > > > > > > --
> > > > > > > > http://bernd.eckenfels.net
> > > > > > > > ________________________________
> > > > > > > > Von: Matthieu BROUILLARD
> > > > > > > > Gesendet: Thursday, December 31, 2020 3:19:09 PM
> > > > > > > > An: dev@maven.apache.org
> > > > > > > > Betreff: maven 4.0.0 new XML stuff
> > > > > > > >
> > > > > > > > Hello all,
> > > > > > > >
> > > > > > > > regarding the active work occurring for maven 4.0.0 I noticed
> > the
> > > > > > > > introduction of a lot of new stuff around SAX parsing &
> > > filtering.
> > > > > > > > I am wondering if that means that it was decided that the
> input
> > > > > format
> > > > > > of
> > > > > > > > maven projects will be XML forever meaning probably, among
> > > others,
> > > > > the
> > > > > > > end
> > > > > > > > of polyglot extensions.
> > > > > > > > Could you explain such a move (or point to
> rationals/documents)
> > > and
> > > > > why
> > > > > > > you
> > > > > > > > did not leverage working on the in memory object model
> allowing
> > > > > > > > extensions/plugins to contribute/hook in the chain of
> building
> > > the
> > > > > > > BuildPOM
> > > > > > > > & ConsumePOM? In the past I really thought that this move to
> > > 'Build
> > > > > vs
> > > > > > > > Consumer' POM would make clear separations between the input
> > > format
> > > > > of
> > > > > > > > descriptors and the core system but I perhaps misunderstood
> > > things.
> > > > > > > >
> > > > > > > > Also, are there plans regarding the future of core
> extensions?
> > > > > > > > With core extensions it was possible to hook into the POM
> model
> > > > > loading
> > > > > > > and
> > > > > > > > do transformations to do dynamic changes but by working on
> the
> > > XML
> > > > > > > directly
> > > > > > > > I see a shift (if not red stop) in this
> contribution/delegation
> > > > > > > mechanism.
> > > > > > > >
> > > > > > > > Thanks for your time & answers.
> > > > > > > >
> > > > > > > > Matthieu Brouillard
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: maven 4.0.0 new XML stuff

Posted by Matthieu Brouillard <ma...@brouillard.fr>.
@Robert nothing is broken atm, the changes for consumer/build are currently
behind your feature flag.

But as I feared previously, and as Romain pointed it, by working at XML
level (and not at POM level) the produced consumer pom does not reflect
changes from extensions.
I really thought that all the "consumer/build" stuff would make the
maven-flatten-plugin useless but it looks like it will not be the case if
working at XML level.
Did Romain and I miss the whole point of the "consumer/build" enhancements
or is it "just" because current implementation has not yet reached the
targets/outputs?

On Mon, Jan 4, 2021 at 2:56 PM Romain Manni-Bucau <rm...@gmail.com>
wrote:

> Hmm, I don't get a few things of this IT:
>
> 1. the formatting seems not expected even if valid (the comments are
> finishing with the first tag for example "--><project xmlns=")
> 2. produced poms keep the parent, build sections (including management one
> and extensions for ex) so it seems it just resolves the versions for the
> parent and dependencies
> 3. until I did something wrong the extension API is really broken (as there
> is a regression which must be fixed) - was why i jumped into this thread,
> for me this is not good at all for our community
>
> About 3, let me explain what i did to see if I missed something:
>
> 1. I created a custom extension with this listener:
>
> @Component(role = AbstractMavenLifecycleParticipant.class, hint =
> "rmannibucau-test")
> public class MyListener extends AbstractMavenLifecycleParticipant {
>
>     @Override
>     public void afterProjectsRead(final MavenSession session) throws
> MavenExecutionException {
>         if (session.getCurrentProject() == null) {
>             return;
>         }
>
>         session.getProjects().forEach(p -> {
>             final Dependency dependency = new Dependency();
>             dependency.setGroupId("junit");
>             dependency.setArtifactId("junit");
>             dependency.setVersion("3.8.1");
>             p.getDependencies().add(dependency);
>         });
>     }
> }
>
>
> 2. If you run mvn (4 snapshot) dependency:tree you get this kind of output:
>
> [INFO] -------------< org.sonatype.mavenbook.multi:simple-webapp
> >-------------
> [INFO] Building Multi Chapter Simple Web Application Project
> 0.9-MNG6957-SNAPSHOT [6/6]
> [INFO] --------------------------------[ jar
> ]---------------------------------
> [INFO]
> [INFO] --- maven-dependency-plugin:3.1.2:tree (default-cli) @ simple-webapp
> ---
> [INFO] org.sonatype.mavenbook.multi:simple-webapp:jar:0.9-MNG6957-SNAPSHOT
> [INFO] +-
>
> org.sonatype.mavenbook.multi:simple-weather:jar:0.9-MNG6957-SNAPSHOT:compile
> [INFO] \- junit:junit:jar:3.8.1:compile
>                                         <-- THIS IS WHAT WE WANT TO SEE
> [INFO]
> ------------------------------------------------------------------------
>
> 3. run the build to have produced pom and cat the simple-webapp one:
>
> <?xml version="1.0" encoding="UTF-8"?><!--
> Licensed to the Apache Software Foundation (ASF) under one
> or more contributor license agreements.  See the NOTICE file
> distributed with this work for additional information
> regarding copyright ownership.  The ASF licenses this file
> to you under the Apache License, Version 2.0 (the
> "License"); you may not use this file except in compliance
> with the License.  You may obtain a copy of the License at
>
>   http://www.apache.org/licenses/LICENSE-2.0
>
> Unless required by applicable law or agreed to in writing,
> software distributed under the License is distributed on an
> "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
> KIND, either express or implied.  See the License for the
> specific language governing permissions and limitations
> under the License.
> --><project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="
> http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="
> http://maven.apache.org/POM/4.0.0
> https://maven.apache.org/xsd/maven-4.0.0.xsd">
>   <modelVersion>4.0.0</modelVersion>
>   <parent>
>     <groupId>org.sonatype.mavenbook.multi</groupId>
>     <artifactId>simple-parent</artifactId>
>     <version>0.9-MNG6957-SNAPSHOT</version>
>   </parent>
>
>   <artifactId>simple-webapp</artifactId>
>   <name>Multi Chapter Simple Web Application Project</name>
>   <dependencies>
>     <dependency>
>       <groupId>org.sonatype.mavenbook.multi</groupId>
>       <artifactId>simple-weather</artifactId>
>       <version>0.9-MNG6957-SNAPSHOT</version>
>     </dependency>
>   </dependencies>
>   <build>
>     <finalName>simple-webapp</finalName>
>     <pluginManagement>
>       <plugins>
>         <plugin>
>           <groupId>org.apache.maven.plugins</groupId>
>           <artifactId>maven-war-plugin</artifactId>
>           <version>2.6</version>
>         </plugin>
>       </plugins>
>     </pluginManagement>
>   </build>
> </project>
>
> As you see the dependency is not there. I guess the expected outout is:
>
> <?xml version="1.0" encoding="UTF-8"?>
> <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="
> http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="
> http://maven.apache.org/POM/4.0.0
> https://maven.apache.org/xsd/maven-4.0.0.xsd">
>   <modelVersion>4.0.0</modelVersion>
>   <artifactId>simple-webapp</artifactId>
>   <name>Multi Chapter Simple Web Application Project</name>
>   [description, scm, ..., all central requires sections but not build ones]
>   <dependencies>
>     <dependency>
>       <groupId>org.sonatype.mavenbook.multi</groupId>
>       <artifactId>simple-weather</artifactId>
>       <version>0.9-MNG6957-SNAPSHOT</version>
>     </dependency>
>     <dependency>
>       <groupId>junit</groupId>
>       <artifactId>junit</artifactId>
>       <version>3.8.1</version>
>     </dependency>
>   </dependencies>
> </project>
>
> Am I missing something?
>
> Romain Manni-Bucau
> @rmannibucau <https://twitter.com/rmannibucau> |  Blog
> <https://rmannibucau.metawerx.net/> | Old Blog
> <http://rmannibucau.wordpress.com> | Github <
> https://github.com/rmannibucau> |
> LinkedIn <https://www.linkedin.com/in/rmannibucau> | Book
> <
> https://www.packtpub.com/application-development/java-ee-8-high-performance
> >
>
>
> Le lun. 4 janv. 2021 à 13:41, Robert Scholte <rf...@apache.org> a
> écrit :
>
> >
> >
> https://github.com/apache/maven-integration-testing/tree/master/core-it-suite/src/test/resources/mng-6957-buildconsumer
> > is the most complete IT
> >
> > On 4-1-2021 12:59:51, Romain Manni-Bucau <rm...@gmail.com> wrote:
> > Le lun. 4 janv. 2021 à 12:36, Robert Scholte a
> > écrit :
> >
> > > There's just one thing I want to say:
> > > I'm having trouble with the term "broken".
> > >
> >
> > Well, literally meant broken as decorelated from the user intent and
> > extension model.
> > Anyway, didn't intend to blame but more identify the blockers for a GA so
> > point was really that it seem that on the two sides only the producing
> one
> > is not yet ready since it keeps does not sanitize the model completely
> and
> > keeps build only data like comments, right? Also not yet clear for me if
> we
> > loose the extension enrichments there.
> >
> >
> > > If a Maven project could be built with M3.6.3, it can still be built
> with
> > > M4.
> > > If not, it is either regression (MNG-6957, MNG-7063) which must be
> fixed,
> > > or it requires changes to a plugin for understandable reasons
> > > (maven-pgp-plugin)
> > > AFAIK an interesting extension like the maven-tiles has been tested and
> > > still works.
> > >
> >
> > Do you have this handy, is it in our test suite? I'd like to check the
> > produced pom matches the enriched model but happy to start from something
> > already there.
> >
> >
> > >
> > > Robert
> > >
> > > On 3-1-2021 19:35:25, Romain Manni-Bucau wrote:
> > > Le dim. 3 janv. 2021 à 19:04, Robert Scholte a
> > > écrit :
> > >
> > > > I don't remember all those details anymore, because I hit those in
> the
> > > > beginning.
> > > > Trying things over and over again I decided that this is probably the
> > > most
> > > > successful approach.
> > > >
> > > >
> > > > What of the goals was to keep the pom.xml as is as much as possible.
> > > > We can only decide for the specific Maven elements how to handle
> them,
> > we
> > > > should not decide about comments and licenses.
> > > > BTW, the license issue was hard to solve. You cannot use it from the
> > > pom's
> > > > , because there might be multiple licenses.
> > > >
> > >
> > > I disagree, it is saner IMO to evolve to support that than doing
> > > anything else.
> > > Once again you keep things which don't make sense in a consumed pom in
> > > current impl so i'd say the sucess in a few cases breaks as much cases
> so
> > > we need to revisit anyway IMHO.
> > >
> > >
> > > >
> > > > The current implementation is a solid way to ensure we're not
> breaking
> > > too
> > > > much, because Maven controls the XML filters.
> > > >
> > >
> > > Hmm, breaking extensions seems to break too much (I'm not speaking of
> > other
> > > parts which breaks the ecosystem there but just that is sufficient IMHO
> > to
> > > say we must check back our solution).
> > >
> > >
> > > >
> > > > Also keep in mind, that I only want Maven to decide which
> modifications
> > > > are done.
> > > >
> > >
> > > For the consumed pom I agree but it is consistent with keeping
> everything
> > > working instead of breaking too, no?
> > >
> > >
> > > >
> > > > Current polyglot projects should still work, but they cannot benefit
> > from
> > > > the build/consumer functions yet.
> > > >
> > >
> > > So pom -> build model is kept, build model -> produced pom is broken?
> Is
> > it
> > > the complete status?
> > > Sounds ok for a 4.0 and a 4.1 can fix it if so.
> > > Just want to ensure first part is not broken at all.
> > >
> > >
> > > >
> > > > Robert
> > > > On 3-1-2021 16:38:38, Romain Manni-Bucau wrote:
> > > > Le dim. 3 janv. 2021 à 16:18, Robert Scholte a
> > > > écrit :
> > > >
> > > > > > So what I was expecting was: raw xml model -> converted to
> unified
> > > > > consumed model -> extensions -> model processing.
> > > > >
> > > > > This is only the build pom part. You're missing the consume part,
> > where
> > > > > the xml is distributed.
> > > > >
> > > >
> > > > Yes but with previous chain the consume part is "clean/normalize ->
> > dump"
> > > > since we are using consumed model - only standard model - in memory
> > > > already.
> > > >
> > > >
> > > > > Build is raw + enrich, consumer is raw + enrich + reduce (removing
> > > > > relativePath and modules are the first examples, but much more is
> > > > possible)
> > > > >
> > > > > Going for the in memory was also my first thought, but I would
> loose
> > > > > information, hence I came up with the current implementation.
> > > > >
> > > >
> > > > I don't see what you loose ot be honest.
> > > > You mentionned license but this one is in the pom so not a big deal,
> > > > comments which are undesired IMHO as mentionned and element order
> which
> > > can
> > > > really be discussed since we can desire to enforce an order to
> > normalize
> > > > consumption + it shouldn't be important since from the project point
> of
> > > > view your pom is already "broken"/lost (as all your intelligence is
> > lost
> > > by
> > > > this "not passthrough" process).
> > > > So overall I don't see what you would loose from the consumer side
> but
> > I
> > > > see what you lost from maven ecosystem side.
> > > >
> > > >
> > > > > Again, we're at a point where we can have counter solutions, but
> > don't
> > > > > expect me to implement it.
> > > > >
> > > >
> > > > For now I'm just trying to ensure we agree we don't want to break
> > > existing
> > > > extensions and the nice ecosystem we built after years.
> > > > This was really a move forward and it sounds like we broke it at
> maven
> > 4
> > > > without any user gain which sounds terrible.
> > > >
> > > >
> > > > >
> > > > > thanks,
> > > > > Robert
> > > > > On 3-1-2021 15:25:21, Romain Manni-Bucau wrote:
> > > > > Hi all,
> > > > >
> > > > > I kind of join Matthieu thoughts there, there is no point to work
> at
> > > xml
> > > > > level to create the consumed pom - comments is not a point since it
> > can
> > > > > commonly/easily refer to a dropped part of the pom so they should
> be
> > > > > stripped.
> > > > > Current extension model got proven adapted and adopted, using a
> lower
> > > > level
> > > > > extension API will not since XML is, even if still mainstream,
> often
> > > > > replaced by alternative configurations and to have done the work to
> > > > inject
> > > > > XML configuration programmatically compred to current option, it
> is a
> > > > pain.
> > > > > The in memory model should stick to consumed model IMHO - being
> > > > > programmatic there is no point to make it easier, worse case we can
> > add
> > > > > helper beans (injectable) but in terms of model it will not help.
> > > > >
> > > > > So what I was expecting was:
> > > > >
> > > > > raw xml model -> converted to unified consumed model -> extensions
> ->
> > > > model
> > > > > processing.
> > > > >
> > > > > Indeed, real chain adds a small processing over the first arrow
> > (inject
> > > > > versions for example) but nothing crazy and breaking this overall
> > flow
> > > > > which stays user friendly.
> > > > >
> > > > > Strictly speaking the new model is just a built-in extension for me
> > > which
> > > > > is particular because it will enforce IDE to integrate a new
> format -
> > > > > wheres polyglot extensions or others don't require static analyzis
> by
> > > > > themself not being "standard".
> > > > >
> > > > > That said, there is nothing crazy with current implementation, it
> > just
> > > > > require to be updated to be able to take extension changes into
> > > account.
> > > > > This can be done by making the extension model 'spyable' (ie if a
> > > > > dependency/plugin is added it will be reflected in the final
> written
> > > > > pom.xml).
> > > > > This sounds - instrumenting the extension model API or doing a diff
> > > after
> > > > > extension phase - like a compromise and let people gets the best of
> > > both
> > > > > worlds to me.
> > > > >
> > > > > Wdyt?
> > > > >
> > > > > Romain Manni-Bucau
> > > > > @rmannibucau | Blog
> > > > > | Old Blog
> > > > > | Github |
> > > > > LinkedIn | Book
> > > > >
> > > > >
> > > > >
> > > > > Le dim. 3 janv. 2021 à 14:46, Robert Scholte a
> > > > > écrit :
> > > > >
> > > > > > Hi Matthieu,
> > > > > >
> > > > > > As you understand, something had to be changed to move Maven
> > forward.
> > > > > > I've decided to pick up that challenge and came up with the
> current
> > > > > > solution.
> > > > > >
> > > > > > My main concerns was that I wanted to keep the fileModel as much
> as
> > > is.
> > > > > > That includes the license, comments and element order.
> > > > > > This information if not available in the memory model, so I
> needed
> > > the
> > > > > > original pom file.
> > > > > > With that in mind, the usage of XMLFilters looks like the most
> > > > > appropriate
> > > > > > solution.
> > > > > >
> > > > > > I am certain that XML is still the most used format, so if we can
> > > have
> > > > > > improvements for those users, I'm already very happy.
> > > > > >
> > > > > > And yes, there are plugins that needs to be updated, but doing
> > > nothing
> > > > is
> > > > > > not an option anymore.
> > > > > >
> > > > > > There are more people that share their concerns, but it took me
> > > several
> > > > > > years to reach this point.
> > > > > > We now have something that seems to work, anybody who can improve
> > or
> > > > can
> > > > > > come up with an alternative implementation can do so.
> > > > > >
> > > > > > thanks,
> > > > > > Robert
> > > > > > On 3-1-2021 12:55:41, Matthieu Brouillard wrote:
> > > > > > Thanks Robert for the video link.
> > > > > >
> > > > > > I fully understand the rationales behind the separation of
> > > > > > build/consumer pom and the video provides some insights on it and
> > you
> > > > > > explain the actual implementation to introduce this change.
> > > > > > Still I do not fully understand why it was decided to work on top
> > of
> > > > XML
> > > > > by
> > > > > > filtering/enhancing it instead of working at the POM (in
> > > > > > memory datamodel) level.
> > > > > > With the current understanding I have, by doing this choice of
> > > working
> > > > at
> > > > > > XML level, it looks like it was decided to bypass (if not kill)
> > core
> > > > > > extensions that enhance the POM itself and not the pom.xml ;
> here I
> > > can
> > > > > > think of (but probably not limited to):
> > > > > > - polyglot-maven: do not use XML but other format to describe the
> > POM
> > > > > > (yaml, json, kotlin, java, other XML formats, ...)
> > > > > > - jgitver-maven-plugin (or forks like
> > > maven-git-versioning-extension):
> > > > > > dynamic computation of projects version based on git history
> > > > > >
> > > > > > With the introduction of core extensions, I thought it was a move
> > to
> > > > open
> > > > > > the internals and let externals contribute to the capabilities of
> > > > maven.
> > > > > > With the move to a XML handling chain, I see it as a
> > > > restriction/regress
> > > > > in
> > > > > > favor of core closed functionalities. An example of that is what
> is
> > > > > > provided as CIFriendly stuff, IMO it could/should have been
> > provided
> > > > by a
> > > > > > plugin/extension but instead it is hard written in maven core and
> > is
> > > > not
> > > > > > opened for external contribution (plugin/extension I mean).
> > > > > >
> > > > > > Perhaps I am totally wrong but I think that maven core should
> > define
> > > > all
> > > > > > its expectations at an API level so that extensions/plugins could
> > > hook
> > > > at
> > > > > > this API level. The default packaging of maven could/should
> provide
> > > > > default
> > > > > > implementations of those expectations (for example reading a
> > pom.xml
> > > > file
> > > > > > to a POM model, dumping a POM to a pom-4.0.0.xml,
> > > > transforming/reducing a
> > > > > > POM to POM-consumer, dumping POM-consumer to
> > pom-consumer-5.0.0.xml,
> > > > ...)
> > > > > > and let extensions/plugins/default implementations work along the
> > > build
> > > > > > process with the API & POMs to provide different features and
> > > > > capabilities.
> > > > > >
> > > > > > Matthieu
> > > > > >
> > > > > > On Thu, Dec 31, 2020 at 7:01 PM Robert Scholte wrote:
> > > > > >
> > > > > > > I've made a recording[1] about it, which hopefully answers most
> > > > > > questions.
> > > > > > >
> > > > > > > Robert
> > > > > > >
> > > > > > > [1] https://youtu.be/KDAmlNKZJto
> > > > > > >
> > > > > > > On 31-12-2020 16:18:57, Matthieu Brouillard
> > > > > > > wrote:
> > > > > > > > Not exactly sure what work you mean
> > > > > > > everything related to maven-xml: Build/ConsumerPomXMLFilterxxx,
> > > > > > > Build/ConsumerModelSourcexxxx and the transformer stuff.
> > > > > > > Especially, when looking at classes like CiFriendlyXMLFilter, I
> > > would
> > > > > > have
> > > > > > > thought that such things could have been done elsewhere,
> working
> > on
> > > > the
> > > > > > > object model (not on the XML stuff) especially for the BuildPom
> > > part.
> > > > > > >
> > > > > > > > however specifically the consumer POM integrates with so many
> > > > > external
> > > > > > > ecosystems
> > > > > > > We're aligned here, this has to be stable and well defined by a
> > > > schema.
> > > > > > >
> > > > > > > Matthieu
> > > > > > >
> > > > > > > On Thu, Dec 31, 2020 at 3:59 PM Bernd Eckenfels
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Hello,
> > > > > > > >
> > > > > > > > Not exactly sure what work you mean and I fully agree that
> > using
> > > a
> > > > > core
> > > > > > > > model should still be the API for plugins and extensions to
> > work
> > > > > with,
> > > > > > > > however specifically the consumer POM integrates with so many
> > > > > external
> > > > > > > > ecosystems, I would expect it to be defined in terms of XML
> > > Schema
> > > > > with
> > > > > > > > explicite semantic (and the inherent compatibility with
> exiting
> > > > > POMs).
> > > > > > > >
> > > > > > > > Gruss
> > > > > > > > Bernd
> > > > > > > > --
> > > > > > > > http://bernd.eckenfels.net
> > > > > > > > ________________________________
> > > > > > > > Von: Matthieu BROUILLARD
> > > > > > > > Gesendet: Thursday, December 31, 2020 3:19:09 PM
> > > > > > > > An: dev@maven.apache.org
> > > > > > > > Betreff: maven 4.0.0 new XML stuff
> > > > > > > >
> > > > > > > > Hello all,
> > > > > > > >
> > > > > > > > regarding the active work occurring for maven 4.0.0 I noticed
> > the
> > > > > > > > introduction of a lot of new stuff around SAX parsing &
> > > filtering.
> > > > > > > > I am wondering if that means that it was decided that the
> input
> > > > > format
> > > > > > of
> > > > > > > > maven projects will be XML forever meaning probably, among
> > > others,
> > > > > the
> > > > > > > end
> > > > > > > > of polyglot extensions.
> > > > > > > > Could you explain such a move (or point to
> rationals/documents)
> > > and
> > > > > why
> > > > > > > you
> > > > > > > > did not leverage working on the in memory object model
> allowing
> > > > > > > > extensions/plugins to contribute/hook in the chain of
> building
> > > the
> > > > > > > BuildPOM
> > > > > > > > & ConsumePOM? In the past I really thought that this move to
> > > 'Build
> > > > > vs
> > > > > > > > Consumer' POM would make clear separations between the input
> > > format
> > > > > of
> > > > > > > > descriptors and the core system but I perhaps misunderstood
> > > things.
> > > > > > > >
> > > > > > > > Also, are there plans regarding the future of core
> extensions?
> > > > > > > > With core extensions it was possible to hook into the POM
> model
> > > > > loading
> > > > > > > and
> > > > > > > > do transformations to do dynamic changes but by working on
> the
> > > XML
> > > > > > > directly
> > > > > > > > I see a shift (if not red stop) in this
> contribution/delegation
> > > > > > > mechanism.
> > > > > > > >
> > > > > > > > Thanks for your time & answers.
> > > > > > > >
> > > > > > > > Matthieu Brouillard
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: maven 4.0.0 new XML stuff

Posted by Romain Manni-Bucau <rm...@gmail.com>.
Hmm, I don't get a few things of this IT:

1. the formatting seems not expected even if valid (the comments are
finishing with the first tag for example "--><project xmlns=")
2. produced poms keep the parent, build sections (including management one
and extensions for ex) so it seems it just resolves the versions for the
parent and dependencies
3. until I did something wrong the extension API is really broken (as there
is a regression which must be fixed) - was why i jumped into this thread,
for me this is not good at all for our community

About 3, let me explain what i did to see if I missed something:

1. I created a custom extension with this listener:

@Component(role = AbstractMavenLifecycleParticipant.class, hint =
"rmannibucau-test")
public class MyListener extends AbstractMavenLifecycleParticipant {

    @Override
    public void afterProjectsRead(final MavenSession session) throws
MavenExecutionException {
        if (session.getCurrentProject() == null) {
            return;
        }

        session.getProjects().forEach(p -> {
            final Dependency dependency = new Dependency();
            dependency.setGroupId("junit");
            dependency.setArtifactId("junit");
            dependency.setVersion("3.8.1");
            p.getDependencies().add(dependency);
        });
    }
}


2. If you run mvn (4 snapshot) dependency:tree you get this kind of output:

[INFO] -------------< org.sonatype.mavenbook.multi:simple-webapp
>-------------
[INFO] Building Multi Chapter Simple Web Application Project
0.9-MNG6957-SNAPSHOT [6/6]
[INFO] --------------------------------[ jar
]---------------------------------
[INFO]
[INFO] --- maven-dependency-plugin:3.1.2:tree (default-cli) @ simple-webapp
---
[INFO] org.sonatype.mavenbook.multi:simple-webapp:jar:0.9-MNG6957-SNAPSHOT
[INFO] +-
org.sonatype.mavenbook.multi:simple-weather:jar:0.9-MNG6957-SNAPSHOT:compile
[INFO] \- junit:junit:jar:3.8.1:compile
                                        <-- THIS IS WHAT WE WANT TO SEE
[INFO]
------------------------------------------------------------------------

3. run the build to have produced pom and cat the simple-webapp one:

<?xml version="1.0" encoding="UTF-8"?><!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements.  See the NOTICE file
distributed with this work for additional information
regarding copyright ownership.  The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License.  You may obtain a copy of the License at

  http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied.  See the License for the
specific language governing permissions and limitations
under the License.
--><project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="
http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="
http://maven.apache.org/POM/4.0.0
https://maven.apache.org/xsd/maven-4.0.0.xsd">
  <modelVersion>4.0.0</modelVersion>
  <parent>
    <groupId>org.sonatype.mavenbook.multi</groupId>
    <artifactId>simple-parent</artifactId>
    <version>0.9-MNG6957-SNAPSHOT</version>
  </parent>

  <artifactId>simple-webapp</artifactId>
  <name>Multi Chapter Simple Web Application Project</name>
  <dependencies>
    <dependency>
      <groupId>org.sonatype.mavenbook.multi</groupId>
      <artifactId>simple-weather</artifactId>
      <version>0.9-MNG6957-SNAPSHOT</version>
    </dependency>
  </dependencies>
  <build>
    <finalName>simple-webapp</finalName>
    <pluginManagement>
      <plugins>
        <plugin>
          <groupId>org.apache.maven.plugins</groupId>
          <artifactId>maven-war-plugin</artifactId>
          <version>2.6</version>
        </plugin>
      </plugins>
    </pluginManagement>
  </build>
</project>

As you see the dependency is not there. I guess the expected outout is:

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="
http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="
http://maven.apache.org/POM/4.0.0
https://maven.apache.org/xsd/maven-4.0.0.xsd">
  <modelVersion>4.0.0</modelVersion>
  <artifactId>simple-webapp</artifactId>
  <name>Multi Chapter Simple Web Application Project</name>
  [description, scm, ..., all central requires sections but not build ones]
  <dependencies>
    <dependency>
      <groupId>org.sonatype.mavenbook.multi</groupId>
      <artifactId>simple-weather</artifactId>
      <version>0.9-MNG6957-SNAPSHOT</version>
    </dependency>
    <dependency>
      <groupId>junit</groupId>
      <artifactId>junit</artifactId>
      <version>3.8.1</version>
    </dependency>
  </dependencies>
</project>

Am I missing something?

Romain Manni-Bucau
@rmannibucau <https://twitter.com/rmannibucau> |  Blog
<https://rmannibucau.metawerx.net/> | Old Blog
<http://rmannibucau.wordpress.com> | Github <https://github.com/rmannibucau> |
LinkedIn <https://www.linkedin.com/in/rmannibucau> | Book
<https://www.packtpub.com/application-development/java-ee-8-high-performance>


Le lun. 4 janv. 2021 à 13:41, Robert Scholte <rf...@apache.org> a
écrit :

>
> https://github.com/apache/maven-integration-testing/tree/master/core-it-suite/src/test/resources/mng-6957-buildconsumer
> is the most complete IT
>
> On 4-1-2021 12:59:51, Romain Manni-Bucau <rm...@gmail.com> wrote:
> Le lun. 4 janv. 2021 à 12:36, Robert Scholte a
> écrit :
>
> > There's just one thing I want to say:
> > I'm having trouble with the term "broken".
> >
>
> Well, literally meant broken as decorelated from the user intent and
> extension model.
> Anyway, didn't intend to blame but more identify the blockers for a GA so
> point was really that it seem that on the two sides only the producing one
> is not yet ready since it keeps does not sanitize the model completely and
> keeps build only data like comments, right? Also not yet clear for me if we
> loose the extension enrichments there.
>
>
> > If a Maven project could be built with M3.6.3, it can still be built with
> > M4.
> > If not, it is either regression (MNG-6957, MNG-7063) which must be fixed,
> > or it requires changes to a plugin for understandable reasons
> > (maven-pgp-plugin)
> > AFAIK an interesting extension like the maven-tiles has been tested and
> > still works.
> >
>
> Do you have this handy, is it in our test suite? I'd like to check the
> produced pom matches the enriched model but happy to start from something
> already there.
>
>
> >
> > Robert
> >
> > On 3-1-2021 19:35:25, Romain Manni-Bucau wrote:
> > Le dim. 3 janv. 2021 à 19:04, Robert Scholte a
> > écrit :
> >
> > > I don't remember all those details anymore, because I hit those in the
> > > beginning.
> > > Trying things over and over again I decided that this is probably the
> > most
> > > successful approach.
> > >
> > >
> > > What of the goals was to keep the pom.xml as is as much as possible.
> > > We can only decide for the specific Maven elements how to handle them,
> we
> > > should not decide about comments and licenses.
> > > BTW, the license issue was hard to solve. You cannot use it from the
> > pom's
> > > , because there might be multiple licenses.
> > >
> >
> > I disagree, it is saner IMO to evolve to support that than doing
> > anything else.
> > Once again you keep things which don't make sense in a consumed pom in
> > current impl so i'd say the sucess in a few cases breaks as much cases so
> > we need to revisit anyway IMHO.
> >
> >
> > >
> > > The current implementation is a solid way to ensure we're not breaking
> > too
> > > much, because Maven controls the XML filters.
> > >
> >
> > Hmm, breaking extensions seems to break too much (I'm not speaking of
> other
> > parts which breaks the ecosystem there but just that is sufficient IMHO
> to
> > say we must check back our solution).
> >
> >
> > >
> > > Also keep in mind, that I only want Maven to decide which modifications
> > > are done.
> > >
> >
> > For the consumed pom I agree but it is consistent with keeping everything
> > working instead of breaking too, no?
> >
> >
> > >
> > > Current polyglot projects should still work, but they cannot benefit
> from
> > > the build/consumer functions yet.
> > >
> >
> > So pom -> build model is kept, build model -> produced pom is broken? Is
> it
> > the complete status?
> > Sounds ok for a 4.0 and a 4.1 can fix it if so.
> > Just want to ensure first part is not broken at all.
> >
> >
> > >
> > > Robert
> > > On 3-1-2021 16:38:38, Romain Manni-Bucau wrote:
> > > Le dim. 3 janv. 2021 à 16:18, Robert Scholte a
> > > écrit :
> > >
> > > > > So what I was expecting was: raw xml model -> converted to unified
> > > > consumed model -> extensions -> model processing.
> > > >
> > > > This is only the build pom part. You're missing the consume part,
> where
> > > > the xml is distributed.
> > > >
> > >
> > > Yes but with previous chain the consume part is "clean/normalize ->
> dump"
> > > since we are using consumed model - only standard model - in memory
> > > already.
> > >
> > >
> > > > Build is raw + enrich, consumer is raw + enrich + reduce (removing
> > > > relativePath and modules are the first examples, but much more is
> > > possible)
> > > >
> > > > Going for the in memory was also my first thought, but I would loose
> > > > information, hence I came up with the current implementation.
> > > >
> > >
> > > I don't see what you loose ot be honest.
> > > You mentionned license but this one is in the pom so not a big deal,
> > > comments which are undesired IMHO as mentionned and element order which
> > can
> > > really be discussed since we can desire to enforce an order to
> normalize
> > > consumption + it shouldn't be important since from the project point of
> > > view your pom is already "broken"/lost (as all your intelligence is
> lost
> > by
> > > this "not passthrough" process).
> > > So overall I don't see what you would loose from the consumer side but
> I
> > > see what you lost from maven ecosystem side.
> > >
> > >
> > > > Again, we're at a point where we can have counter solutions, but
> don't
> > > > expect me to implement it.
> > > >
> > >
> > > For now I'm just trying to ensure we agree we don't want to break
> > existing
> > > extensions and the nice ecosystem we built after years.
> > > This was really a move forward and it sounds like we broke it at maven
> 4
> > > without any user gain which sounds terrible.
> > >
> > >
> > > >
> > > > thanks,
> > > > Robert
> > > > On 3-1-2021 15:25:21, Romain Manni-Bucau wrote:
> > > > Hi all,
> > > >
> > > > I kind of join Matthieu thoughts there, there is no point to work at
> > xml
> > > > level to create the consumed pom - comments is not a point since it
> can
> > > > commonly/easily refer to a dropped part of the pom so they should be
> > > > stripped.
> > > > Current extension model got proven adapted and adopted, using a lower
> > > level
> > > > extension API will not since XML is, even if still mainstream, often
> > > > replaced by alternative configurations and to have done the work to
> > > inject
> > > > XML configuration programmatically compred to current option, it is a
> > > pain.
> > > > The in memory model should stick to consumed model IMHO - being
> > > > programmatic there is no point to make it easier, worse case we can
> add
> > > > helper beans (injectable) but in terms of model it will not help.
> > > >
> > > > So what I was expecting was:
> > > >
> > > > raw xml model -> converted to unified consumed model -> extensions ->
> > > model
> > > > processing.
> > > >
> > > > Indeed, real chain adds a small processing over the first arrow
> (inject
> > > > versions for example) but nothing crazy and breaking this overall
> flow
> > > > which stays user friendly.
> > > >
> > > > Strictly speaking the new model is just a built-in extension for me
> > which
> > > > is particular because it will enforce IDE to integrate a new format -
> > > > wheres polyglot extensions or others don't require static analyzis by
> > > > themself not being "standard".
> > > >
> > > > That said, there is nothing crazy with current implementation, it
> just
> > > > require to be updated to be able to take extension changes into
> > account.
> > > > This can be done by making the extension model 'spyable' (ie if a
> > > > dependency/plugin is added it will be reflected in the final written
> > > > pom.xml).
> > > > This sounds - instrumenting the extension model API or doing a diff
> > after
> > > > extension phase - like a compromise and let people gets the best of
> > both
> > > > worlds to me.
> > > >
> > > > Wdyt?
> > > >
> > > > Romain Manni-Bucau
> > > > @rmannibucau | Blog
> > > > | Old Blog
> > > > | Github |
> > > > LinkedIn | Book
> > > >
> > > >
> > > >
> > > > Le dim. 3 janv. 2021 à 14:46, Robert Scholte a
> > > > écrit :
> > > >
> > > > > Hi Matthieu,
> > > > >
> > > > > As you understand, something had to be changed to move Maven
> forward.
> > > > > I've decided to pick up that challenge and came up with the current
> > > > > solution.
> > > > >
> > > > > My main concerns was that I wanted to keep the fileModel as much as
> > is.
> > > > > That includes the license, comments and element order.
> > > > > This information if not available in the memory model, so I needed
> > the
> > > > > original pom file.
> > > > > With that in mind, the usage of XMLFilters looks like the most
> > > > appropriate
> > > > > solution.
> > > > >
> > > > > I am certain that XML is still the most used format, so if we can
> > have
> > > > > improvements for those users, I'm already very happy.
> > > > >
> > > > > And yes, there are plugins that needs to be updated, but doing
> > nothing
> > > is
> > > > > not an option anymore.
> > > > >
> > > > > There are more people that share their concerns, but it took me
> > several
> > > > > years to reach this point.
> > > > > We now have something that seems to work, anybody who can improve
> or
> > > can
> > > > > come up with an alternative implementation can do so.
> > > > >
> > > > > thanks,
> > > > > Robert
> > > > > On 3-1-2021 12:55:41, Matthieu Brouillard wrote:
> > > > > Thanks Robert for the video link.
> > > > >
> > > > > I fully understand the rationales behind the separation of
> > > > > build/consumer pom and the video provides some insights on it and
> you
> > > > > explain the actual implementation to introduce this change.
> > > > > Still I do not fully understand why it was decided to work on top
> of
> > > XML
> > > > by
> > > > > filtering/enhancing it instead of working at the POM (in
> > > > > memory datamodel) level.
> > > > > With the current understanding I have, by doing this choice of
> > working
> > > at
> > > > > XML level, it looks like it was decided to bypass (if not kill)
> core
> > > > > extensions that enhance the POM itself and not the pom.xml ; here I
> > can
> > > > > think of (but probably not limited to):
> > > > > - polyglot-maven: do not use XML but other format to describe the
> POM
> > > > > (yaml, json, kotlin, java, other XML formats, ...)
> > > > > - jgitver-maven-plugin (or forks like
> > maven-git-versioning-extension):
> > > > > dynamic computation of projects version based on git history
> > > > >
> > > > > With the introduction of core extensions, I thought it was a move
> to
> > > open
> > > > > the internals and let externals contribute to the capabilities of
> > > maven.
> > > > > With the move to a XML handling chain, I see it as a
> > > restriction/regress
> > > > in
> > > > > favor of core closed functionalities. An example of that is what is
> > > > > provided as CIFriendly stuff, IMO it could/should have been
> provided
> > > by a
> > > > > plugin/extension but instead it is hard written in maven core and
> is
> > > not
> > > > > opened for external contribution (plugin/extension I mean).
> > > > >
> > > > > Perhaps I am totally wrong but I think that maven core should
> define
> > > all
> > > > > its expectations at an API level so that extensions/plugins could
> > hook
> > > at
> > > > > this API level. The default packaging of maven could/should provide
> > > > default
> > > > > implementations of those expectations (for example reading a
> pom.xml
> > > file
> > > > > to a POM model, dumping a POM to a pom-4.0.0.xml,
> > > transforming/reducing a
> > > > > POM to POM-consumer, dumping POM-consumer to
> pom-consumer-5.0.0.xml,
> > > ...)
> > > > > and let extensions/plugins/default implementations work along the
> > build
> > > > > process with the API & POMs to provide different features and
> > > > capabilities.
> > > > >
> > > > > Matthieu
> > > > >
> > > > > On Thu, Dec 31, 2020 at 7:01 PM Robert Scholte wrote:
> > > > >
> > > > > > I've made a recording[1] about it, which hopefully answers most
> > > > > questions.
> > > > > >
> > > > > > Robert
> > > > > >
> > > > > > [1] https://youtu.be/KDAmlNKZJto
> > > > > >
> > > > > > On 31-12-2020 16:18:57, Matthieu Brouillard
> > > > > > wrote:
> > > > > > > Not exactly sure what work you mean
> > > > > > everything related to maven-xml: Build/ConsumerPomXMLFilterxxx,
> > > > > > Build/ConsumerModelSourcexxxx and the transformer stuff.
> > > > > > Especially, when looking at classes like CiFriendlyXMLFilter, I
> > would
> > > > > have
> > > > > > thought that such things could have been done elsewhere, working
> on
> > > the
> > > > > > object model (not on the XML stuff) especially for the BuildPom
> > part.
> > > > > >
> > > > > > > however specifically the consumer POM integrates with so many
> > > > external
> > > > > > ecosystems
> > > > > > We're aligned here, this has to be stable and well defined by a
> > > schema.
> > > > > >
> > > > > > Matthieu
> > > > > >
> > > > > > On Thu, Dec 31, 2020 at 3:59 PM Bernd Eckenfels
> > > > > > wrote:
> > > > > >
> > > > > > > Hello,
> > > > > > >
> > > > > > > Not exactly sure what work you mean and I fully agree that
> using
> > a
> > > > core
> > > > > > > model should still be the API for plugins and extensions to
> work
> > > > with,
> > > > > > > however specifically the consumer POM integrates with so many
> > > > external
> > > > > > > ecosystems, I would expect it to be defined in terms of XML
> > Schema
> > > > with
> > > > > > > explicite semantic (and the inherent compatibility with exiting
> > > > POMs).
> > > > > > >
> > > > > > > Gruss
> > > > > > > Bernd
> > > > > > > --
> > > > > > > http://bernd.eckenfels.net
> > > > > > > ________________________________
> > > > > > > Von: Matthieu BROUILLARD
> > > > > > > Gesendet: Thursday, December 31, 2020 3:19:09 PM
> > > > > > > An: dev@maven.apache.org
> > > > > > > Betreff: maven 4.0.0 new XML stuff
> > > > > > >
> > > > > > > Hello all,
> > > > > > >
> > > > > > > regarding the active work occurring for maven 4.0.0 I noticed
> the
> > > > > > > introduction of a lot of new stuff around SAX parsing &
> > filtering.
> > > > > > > I am wondering if that means that it was decided that the input
> > > > format
> > > > > of
> > > > > > > maven projects will be XML forever meaning probably, among
> > others,
> > > > the
> > > > > > end
> > > > > > > of polyglot extensions.
> > > > > > > Could you explain such a move (or point to rationals/documents)
> > and
> > > > why
> > > > > > you
> > > > > > > did not leverage working on the in memory object model allowing
> > > > > > > extensions/plugins to contribute/hook in the chain of building
> > the
> > > > > > BuildPOM
> > > > > > > & ConsumePOM? In the past I really thought that this move to
> > 'Build
> > > > vs
> > > > > > > Consumer' POM would make clear separations between the input
> > format
> > > > of
> > > > > > > descriptors and the core system but I perhaps misunderstood
> > things.
> > > > > > >
> > > > > > > Also, are there plans regarding the future of core extensions?
> > > > > > > With core extensions it was possible to hook into the POM model
> > > > loading
> > > > > > and
> > > > > > > do transformations to do dynamic changes but by working on the
> > XML
> > > > > > directly
> > > > > > > I see a shift (if not red stop) in this contribution/delegation
> > > > > > mechanism.
> > > > > > >
> > > > > > > Thanks for your time & answers.
> > > > > > >
> > > > > > > Matthieu Brouillard
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: maven 4.0.0 new XML stuff

Posted by Robert Scholte <rf...@apache.org>.
https://github.com/apache/maven-integration-testing/tree/master/core-it-suite/src/test/resources/mng-6957-buildconsumer is the most complete IT

On 4-1-2021 12:59:51, Romain Manni-Bucau <rm...@gmail.com> wrote:
Le lun. 4 janv. 2021 à 12:36, Robert Scholte a
écrit :

> There's just one thing I want to say:
> I'm having trouble with the term "broken".
>

Well, literally meant broken as decorelated from the user intent and
extension model.
Anyway, didn't intend to blame but more identify the blockers for a GA so
point was really that it seem that on the two sides only the producing one
is not yet ready since it keeps does not sanitize the model completely and
keeps build only data like comments, right? Also not yet clear for me if we
loose the extension enrichments there.


> If a Maven project could be built with M3.6.3, it can still be built with
> M4.
> If not, it is either regression (MNG-6957, MNG-7063) which must be fixed,
> or it requires changes to a plugin for understandable reasons
> (maven-pgp-plugin)
> AFAIK an interesting extension like the maven-tiles has been tested and
> still works.
>

Do you have this handy, is it in our test suite? I'd like to check the
produced pom matches the enriched model but happy to start from something
already there.


>
> Robert
>
> On 3-1-2021 19:35:25, Romain Manni-Bucau wrote:
> Le dim. 3 janv. 2021 à 19:04, Robert Scholte a
> écrit :
>
> > I don't remember all those details anymore, because I hit those in the
> > beginning.
> > Trying things over and over again I decided that this is probably the
> most
> > successful approach.
> >
> >
> > What of the goals was to keep the pom.xml as is as much as possible.
> > We can only decide for the specific Maven elements how to handle them, we
> > should not decide about comments and licenses.
> > BTW, the license issue was hard to solve. You cannot use it from the
> pom's
> > , because there might be multiple licenses.
> >
>
> I disagree, it is saner IMO to evolve to support that than doing
> anything else.
> Once again you keep things which don't make sense in a consumed pom in
> current impl so i'd say the sucess in a few cases breaks as much cases so
> we need to revisit anyway IMHO.
>
>
> >
> > The current implementation is a solid way to ensure we're not breaking
> too
> > much, because Maven controls the XML filters.
> >
>
> Hmm, breaking extensions seems to break too much (I'm not speaking of other
> parts which breaks the ecosystem there but just that is sufficient IMHO to
> say we must check back our solution).
>
>
> >
> > Also keep in mind, that I only want Maven to decide which modifications
> > are done.
> >
>
> For the consumed pom I agree but it is consistent with keeping everything
> working instead of breaking too, no?
>
>
> >
> > Current polyglot projects should still work, but they cannot benefit from
> > the build/consumer functions yet.
> >
>
> So pom -> build model is kept, build model -> produced pom is broken? Is it
> the complete status?
> Sounds ok for a 4.0 and a 4.1 can fix it if so.
> Just want to ensure first part is not broken at all.
>
>
> >
> > Robert
> > On 3-1-2021 16:38:38, Romain Manni-Bucau wrote:
> > Le dim. 3 janv. 2021 à 16:18, Robert Scholte a
> > écrit :
> >
> > > > So what I was expecting was: raw xml model -> converted to unified
> > > consumed model -> extensions -> model processing.
> > >
> > > This is only the build pom part. You're missing the consume part, where
> > > the xml is distributed.
> > >
> >
> > Yes but with previous chain the consume part is "clean/normalize -> dump"
> > since we are using consumed model - only standard model - in memory
> > already.
> >
> >
> > > Build is raw + enrich, consumer is raw + enrich + reduce (removing
> > > relativePath and modules are the first examples, but much more is
> > possible)
> > >
> > > Going for the in memory was also my first thought, but I would loose
> > > information, hence I came up with the current implementation.
> > >
> >
> > I don't see what you loose ot be honest.
> > You mentionned license but this one is in the pom so not a big deal,
> > comments which are undesired IMHO as mentionned and element order which
> can
> > really be discussed since we can desire to enforce an order to normalize
> > consumption + it shouldn't be important since from the project point of
> > view your pom is already "broken"/lost (as all your intelligence is lost
> by
> > this "not passthrough" process).
> > So overall I don't see what you would loose from the consumer side but I
> > see what you lost from maven ecosystem side.
> >
> >
> > > Again, we're at a point where we can have counter solutions, but don't
> > > expect me to implement it.
> > >
> >
> > For now I'm just trying to ensure we agree we don't want to break
> existing
> > extensions and the nice ecosystem we built after years.
> > This was really a move forward and it sounds like we broke it at maven 4
> > without any user gain which sounds terrible.
> >
> >
> > >
> > > thanks,
> > > Robert
> > > On 3-1-2021 15:25:21, Romain Manni-Bucau wrote:
> > > Hi all,
> > >
> > > I kind of join Matthieu thoughts there, there is no point to work at
> xml
> > > level to create the consumed pom - comments is not a point since it can
> > > commonly/easily refer to a dropped part of the pom so they should be
> > > stripped.
> > > Current extension model got proven adapted and adopted, using a lower
> > level
> > > extension API will not since XML is, even if still mainstream, often
> > > replaced by alternative configurations and to have done the work to
> > inject
> > > XML configuration programmatically compred to current option, it is a
> > pain.
> > > The in memory model should stick to consumed model IMHO - being
> > > programmatic there is no point to make it easier, worse case we can add
> > > helper beans (injectable) but in terms of model it will not help.
> > >
> > > So what I was expecting was:
> > >
> > > raw xml model -> converted to unified consumed model -> extensions ->
> > model
> > > processing.
> > >
> > > Indeed, real chain adds a small processing over the first arrow (inject
> > > versions for example) but nothing crazy and breaking this overall flow
> > > which stays user friendly.
> > >
> > > Strictly speaking the new model is just a built-in extension for me
> which
> > > is particular because it will enforce IDE to integrate a new format -
> > > wheres polyglot extensions or others don't require static analyzis by
> > > themself not being "standard".
> > >
> > > That said, there is nothing crazy with current implementation, it just
> > > require to be updated to be able to take extension changes into
> account.
> > > This can be done by making the extension model 'spyable' (ie if a
> > > dependency/plugin is added it will be reflected in the final written
> > > pom.xml).
> > > This sounds - instrumenting the extension model API or doing a diff
> after
> > > extension phase - like a compromise and let people gets the best of
> both
> > > worlds to me.
> > >
> > > Wdyt?
> > >
> > > Romain Manni-Bucau
> > > @rmannibucau | Blog
> > > | Old Blog
> > > | Github |
> > > LinkedIn | Book
> > >
> > >
> > >
> > > Le dim. 3 janv. 2021 à 14:46, Robert Scholte a
> > > écrit :
> > >
> > > > Hi Matthieu,
> > > >
> > > > As you understand, something had to be changed to move Maven forward.
> > > > I've decided to pick up that challenge and came up with the current
> > > > solution.
> > > >
> > > > My main concerns was that I wanted to keep the fileModel as much as
> is.
> > > > That includes the license, comments and element order.
> > > > This information if not available in the memory model, so I needed
> the
> > > > original pom file.
> > > > With that in mind, the usage of XMLFilters looks like the most
> > > appropriate
> > > > solution.
> > > >
> > > > I am certain that XML is still the most used format, so if we can
> have
> > > > improvements for those users, I'm already very happy.
> > > >
> > > > And yes, there are plugins that needs to be updated, but doing
> nothing
> > is
> > > > not an option anymore.
> > > >
> > > > There are more people that share their concerns, but it took me
> several
> > > > years to reach this point.
> > > > We now have something that seems to work, anybody who can improve or
> > can
> > > > come up with an alternative implementation can do so.
> > > >
> > > > thanks,
> > > > Robert
> > > > On 3-1-2021 12:55:41, Matthieu Brouillard wrote:
> > > > Thanks Robert for the video link.
> > > >
> > > > I fully understand the rationales behind the separation of
> > > > build/consumer pom and the video provides some insights on it and you
> > > > explain the actual implementation to introduce this change.
> > > > Still I do not fully understand why it was decided to work on top of
> > XML
> > > by
> > > > filtering/enhancing it instead of working at the POM (in
> > > > memory datamodel) level.
> > > > With the current understanding I have, by doing this choice of
> working
> > at
> > > > XML level, it looks like it was decided to bypass (if not kill) core
> > > > extensions that enhance the POM itself and not the pom.xml ; here I
> can
> > > > think of (but probably not limited to):
> > > > - polyglot-maven: do not use XML but other format to describe the POM
> > > > (yaml, json, kotlin, java, other XML formats, ...)
> > > > - jgitver-maven-plugin (or forks like
> maven-git-versioning-extension):
> > > > dynamic computation of projects version based on git history
> > > >
> > > > With the introduction of core extensions, I thought it was a move to
> > open
> > > > the internals and let externals contribute to the capabilities of
> > maven.
> > > > With the move to a XML handling chain, I see it as a
> > restriction/regress
> > > in
> > > > favor of core closed functionalities. An example of that is what is
> > > > provided as CIFriendly stuff, IMO it could/should have been provided
> > by a
> > > > plugin/extension but instead it is hard written in maven core and is
> > not
> > > > opened for external contribution (plugin/extension I mean).
> > > >
> > > > Perhaps I am totally wrong but I think that maven core should define
> > all
> > > > its expectations at an API level so that extensions/plugins could
> hook
> > at
> > > > this API level. The default packaging of maven could/should provide
> > > default
> > > > implementations of those expectations (for example reading a pom.xml
> > file
> > > > to a POM model, dumping a POM to a pom-4.0.0.xml,
> > transforming/reducing a
> > > > POM to POM-consumer, dumping POM-consumer to pom-consumer-5.0.0.xml,
> > ...)
> > > > and let extensions/plugins/default implementations work along the
> build
> > > > process with the API & POMs to provide different features and
> > > capabilities.
> > > >
> > > > Matthieu
> > > >
> > > > On Thu, Dec 31, 2020 at 7:01 PM Robert Scholte wrote:
> > > >
> > > > > I've made a recording[1] about it, which hopefully answers most
> > > > questions.
> > > > >
> > > > > Robert
> > > > >
> > > > > [1] https://youtu.be/KDAmlNKZJto
> > > > >
> > > > > On 31-12-2020 16:18:57, Matthieu Brouillard
> > > > > wrote:
> > > > > > Not exactly sure what work you mean
> > > > > everything related to maven-xml: Build/ConsumerPomXMLFilterxxx,
> > > > > Build/ConsumerModelSourcexxxx and the transformer stuff.
> > > > > Especially, when looking at classes like CiFriendlyXMLFilter, I
> would
> > > > have
> > > > > thought that such things could have been done elsewhere, working on
> > the
> > > > > object model (not on the XML stuff) especially for the BuildPom
> part.
> > > > >
> > > > > > however specifically the consumer POM integrates with so many
> > > external
> > > > > ecosystems
> > > > > We're aligned here, this has to be stable and well defined by a
> > schema.
> > > > >
> > > > > Matthieu
> > > > >
> > > > > On Thu, Dec 31, 2020 at 3:59 PM Bernd Eckenfels
> > > > > wrote:
> > > > >
> > > > > > Hello,
> > > > > >
> > > > > > Not exactly sure what work you mean and I fully agree that using
> a
> > > core
> > > > > > model should still be the API for plugins and extensions to work
> > > with,
> > > > > > however specifically the consumer POM integrates with so many
> > > external
> > > > > > ecosystems, I would expect it to be defined in terms of XML
> Schema
> > > with
> > > > > > explicite semantic (and the inherent compatibility with exiting
> > > POMs).
> > > > > >
> > > > > > Gruss
> > > > > > Bernd
> > > > > > --
> > > > > > http://bernd.eckenfels.net
> > > > > > ________________________________
> > > > > > Von: Matthieu BROUILLARD
> > > > > > Gesendet: Thursday, December 31, 2020 3:19:09 PM
> > > > > > An: dev@maven.apache.org
> > > > > > Betreff: maven 4.0.0 new XML stuff
> > > > > >
> > > > > > Hello all,
> > > > > >
> > > > > > regarding the active work occurring for maven 4.0.0 I noticed the
> > > > > > introduction of a lot of new stuff around SAX parsing &
> filtering.
> > > > > > I am wondering if that means that it was decided that the input
> > > format
> > > > of
> > > > > > maven projects will be XML forever meaning probably, among
> others,
> > > the
> > > > > end
> > > > > > of polyglot extensions.
> > > > > > Could you explain such a move (or point to rationals/documents)
> and
> > > why
> > > > > you
> > > > > > did not leverage working on the in memory object model allowing
> > > > > > extensions/plugins to contribute/hook in the chain of building
> the
> > > > > BuildPOM
> > > > > > & ConsumePOM? In the past I really thought that this move to
> 'Build
> > > vs
> > > > > > Consumer' POM would make clear separations between the input
> format
> > > of
> > > > > > descriptors and the core system but I perhaps misunderstood
> things.
> > > > > >
> > > > > > Also, are there plans regarding the future of core extensions?
> > > > > > With core extensions it was possible to hook into the POM model
> > > loading
> > > > > and
> > > > > > do transformations to do dynamic changes but by working on the
> XML
> > > > > directly
> > > > > > I see a shift (if not red stop) in this contribution/delegation
> > > > > mechanism.
> > > > > >
> > > > > > Thanks for your time & answers.
> > > > > >
> > > > > > Matthieu Brouillard
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: maven 4.0.0 new XML stuff

Posted by Romain Manni-Bucau <rm...@gmail.com>.
Le lun. 4 janv. 2021 à 12:36, Robert Scholte <rf...@apache.org> a
écrit :

> There's just one thing I want to say:
> I'm having trouble with the term "broken".
>

Well, literally meant broken as decorelated from the user intent and
extension model.
Anyway, didn't intend to blame but more identify the blockers for a GA so
point was really that it seem that on the two sides only the producing one
is not yet ready since it keeps does not sanitize the model completely and
keeps build only data like comments, right? Also not yet clear for me if we
loose the extension enrichments there.


> If a Maven project could be built with M3.6.3, it can still be built with
> M4.
> If not, it is either regression (MNG-6957, MNG-7063) which must be fixed,
> or it requires changes to a plugin for understandable reasons
> (maven-pgp-plugin)
> AFAIK an interesting extension like the maven-tiles has been tested and
> still works.
>

Do you have this handy, is it in our test suite? I'd like to check the
produced pom matches the enriched model but happy to start from something
already there.


>
> Robert
>
> On 3-1-2021 19:35:25, Romain Manni-Bucau <rm...@gmail.com> wrote:
> Le dim. 3 janv. 2021 à 19:04, Robert Scholte a
> écrit :
>
> > I don't remember all those details anymore, because I hit those in the
> > beginning.
> > Trying things over and over again I decided that this is probably the
> most
> > successful approach.
> >
> >
> > What of the goals was to keep the pom.xml as is as much as possible.
> > We can only decide for the specific Maven elements how to handle them, we
> > should not decide about comments and licenses.
> > BTW, the license issue was hard to solve. You cannot use it from the
> pom's
> > , because there might be multiple licenses.
> >
>
> I disagree, it is saner IMO to evolve to support that than doing
> anything else.
> Once again you keep things which don't make sense in a consumed pom in
> current impl so i'd say the sucess in a few cases breaks as much cases so
> we need to revisit anyway IMHO.
>
>
> >
> > The current implementation is a solid way to ensure we're not breaking
> too
> > much, because Maven controls the XML filters.
> >
>
> Hmm, breaking extensions seems to break too much (I'm not speaking of other
> parts which breaks the ecosystem there but just that is sufficient IMHO to
> say we must check back our solution).
>
>
> >
> > Also keep in mind, that I only want Maven to decide which modifications
> > are done.
> >
>
> For the consumed pom I agree but it is consistent with keeping everything
> working instead of breaking too, no?
>
>
> >
> > Current polyglot projects should still work, but they cannot benefit from
> > the build/consumer functions yet.
> >
>
> So pom -> build model is kept, build model -> produced pom is broken? Is it
> the complete status?
> Sounds ok for a 4.0 and a 4.1 can fix it if so.
> Just want to ensure first part is not broken at all.
>
>
> >
> > Robert
> > On 3-1-2021 16:38:38, Romain Manni-Bucau wrote:
> > Le dim. 3 janv. 2021 à 16:18, Robert Scholte a
> > écrit :
> >
> > > > So what I was expecting was: raw xml model -> converted to unified
> > > consumed model -> extensions -> model processing.
> > >
> > > This is only the build pom part. You're missing the consume part, where
> > > the xml is distributed.
> > >
> >
> > Yes but with previous chain the consume part is "clean/normalize -> dump"
> > since we are using consumed model - only standard model - in memory
> > already.
> >
> >
> > > Build is raw + enrich, consumer is raw + enrich + reduce (removing
> > > relativePath and modules are the first examples, but much more is
> > possible)
> > >
> > > Going for the in memory was also my first thought, but I would loose
> > > information, hence I came up with the current implementation.
> > >
> >
> > I don't see what you loose ot be honest.
> > You mentionned license but this one is in the pom so not a big deal,
> > comments which are undesired IMHO as mentionned and element order which
> can
> > really be discussed since we can desire to enforce an order to normalize
> > consumption + it shouldn't be important since from the project point of
> > view your pom is already "broken"/lost (as all your intelligence is lost
> by
> > this "not passthrough" process).
> > So overall I don't see what you would loose from the consumer side but I
> > see what you lost from maven ecosystem side.
> >
> >
> > > Again, we're at a point where we can have counter solutions, but don't
> > > expect me to implement it.
> > >
> >
> > For now I'm just trying to ensure we agree we don't want to break
> existing
> > extensions and the nice ecosystem we built after years.
> > This was really a move forward and it sounds like we broke it at maven 4
> > without any user gain which sounds terrible.
> >
> >
> > >
> > > thanks,
> > > Robert
> > > On 3-1-2021 15:25:21, Romain Manni-Bucau wrote:
> > > Hi all,
> > >
> > > I kind of join Matthieu thoughts there, there is no point to work at
> xml
> > > level to create the consumed pom - comments is not a point since it can
> > > commonly/easily refer to a dropped part of the pom so they should be
> > > stripped.
> > > Current extension model got proven adapted and adopted, using a lower
> > level
> > > extension API will not since XML is, even if still mainstream, often
> > > replaced by alternative configurations and to have done the work to
> > inject
> > > XML configuration programmatically compred to current option, it is a
> > pain.
> > > The in memory model should stick to consumed model IMHO - being
> > > programmatic there is no point to make it easier, worse case we can add
> > > helper beans (injectable) but in terms of model it will not help.
> > >
> > > So what I was expecting was:
> > >
> > > raw xml model -> converted to unified consumed model -> extensions ->
> > model
> > > processing.
> > >
> > > Indeed, real chain adds a small processing over the first arrow (inject
> > > versions for example) but nothing crazy and breaking this overall flow
> > > which stays user friendly.
> > >
> > > Strictly speaking the new model is just a built-in extension for me
> which
> > > is particular because it will enforce IDE to integrate a new format -
> > > wheres polyglot extensions or others don't require static analyzis by
> > > themself not being "standard".
> > >
> > > That said, there is nothing crazy with current implementation, it just
> > > require to be updated to be able to take extension changes into
> account.
> > > This can be done by making the extension model 'spyable' (ie if a
> > > dependency/plugin is added it will be reflected in the final written
> > > pom.xml).
> > > This sounds - instrumenting the extension model API or doing a diff
> after
> > > extension phase - like a compromise and let people gets the best of
> both
> > > worlds to me.
> > >
> > > Wdyt?
> > >
> > > Romain Manni-Bucau
> > > @rmannibucau | Blog
> > > | Old Blog
> > > | Github |
> > > LinkedIn | Book
> > >
> > >
> > >
> > > Le dim. 3 janv. 2021 à 14:46, Robert Scholte a
> > > écrit :
> > >
> > > > Hi Matthieu,
> > > >
> > > > As you understand, something had to be changed to move Maven forward.
> > > > I've decided to pick up that challenge and came up with the current
> > > > solution.
> > > >
> > > > My main concerns was that I wanted to keep the fileModel as much as
> is.
> > > > That includes the license, comments and element order.
> > > > This information if not available in the memory model, so I needed
> the
> > > > original pom file.
> > > > With that in mind, the usage of XMLFilters looks like the most
> > > appropriate
> > > > solution.
> > > >
> > > > I am certain that XML is still the most used format, so if we can
> have
> > > > improvements for those users, I'm already very happy.
> > > >
> > > > And yes, there are plugins that needs to be updated, but doing
> nothing
> > is
> > > > not an option anymore.
> > > >
> > > > There are more people that share their concerns, but it took me
> several
> > > > years to reach this point.
> > > > We now have something that seems to work, anybody who can improve or
> > can
> > > > come up with an alternative implementation can do so.
> > > >
> > > > thanks,
> > > > Robert
> > > > On 3-1-2021 12:55:41, Matthieu Brouillard wrote:
> > > > Thanks Robert for the video link.
> > > >
> > > > I fully understand the rationales behind the separation of
> > > > build/consumer pom and the video provides some insights on it and you
> > > > explain the actual implementation to introduce this change.
> > > > Still I do not fully understand why it was decided to work on top of
> > XML
> > > by
> > > > filtering/enhancing it instead of working at the POM (in
> > > > memory datamodel) level.
> > > > With the current understanding I have, by doing this choice of
> working
> > at
> > > > XML level, it looks like it was decided to bypass (if not kill) core
> > > > extensions that enhance the POM itself and not the pom.xml ; here I
> can
> > > > think of (but probably not limited to):
> > > > - polyglot-maven: do not use XML but other format to describe the POM
> > > > (yaml, json, kotlin, java, other XML formats, ...)
> > > > - jgitver-maven-plugin (or forks like
> maven-git-versioning-extension):
> > > > dynamic computation of projects version based on git history
> > > >
> > > > With the introduction of core extensions, I thought it was a move to
> > open
> > > > the internals and let externals contribute to the capabilities of
> > maven.
> > > > With the move to a XML handling chain, I see it as a
> > restriction/regress
> > > in
> > > > favor of core closed functionalities. An example of that is what is
> > > > provided as CIFriendly stuff, IMO it could/should have been provided
> > by a
> > > > plugin/extension but instead it is hard written in maven core and is
> > not
> > > > opened for external contribution (plugin/extension I mean).
> > > >
> > > > Perhaps I am totally wrong but I think that maven core should define
> > all
> > > > its expectations at an API level so that extensions/plugins could
> hook
> > at
> > > > this API level. The default packaging of maven could/should provide
> > > default
> > > > implementations of those expectations (for example reading a pom.xml
> > file
> > > > to a POM model, dumping a POM to a pom-4.0.0.xml,
> > transforming/reducing a
> > > > POM to POM-consumer, dumping POM-consumer to pom-consumer-5.0.0.xml,
> > ...)
> > > > and let extensions/plugins/default implementations work along the
> build
> > > > process with the API & POMs to provide different features and
> > > capabilities.
> > > >
> > > > Matthieu
> > > >
> > > > On Thu, Dec 31, 2020 at 7:01 PM Robert Scholte wrote:
> > > >
> > > > > I've made a recording[1] about it, which hopefully answers most
> > > > questions.
> > > > >
> > > > > Robert
> > > > >
> > > > > [1] https://youtu.be/KDAmlNKZJto
> > > > >
> > > > > On 31-12-2020 16:18:57, Matthieu Brouillard
> > > > > wrote:
> > > > > > Not exactly sure what work you mean
> > > > > everything related to maven-xml: Build/ConsumerPomXMLFilterxxx,
> > > > > Build/ConsumerModelSourcexxxx and the transformer stuff.
> > > > > Especially, when looking at classes like CiFriendlyXMLFilter, I
> would
> > > > have
> > > > > thought that such things could have been done elsewhere, working on
> > the
> > > > > object model (not on the XML stuff) especially for the BuildPom
> part.
> > > > >
> > > > > > however specifically the consumer POM integrates with so many
> > > external
> > > > > ecosystems
> > > > > We're aligned here, this has to be stable and well defined by a
> > schema.
> > > > >
> > > > > Matthieu
> > > > >
> > > > > On Thu, Dec 31, 2020 at 3:59 PM Bernd Eckenfels
> > > > > wrote:
> > > > >
> > > > > > Hello,
> > > > > >
> > > > > > Not exactly sure what work you mean and I fully agree that using
> a
> > > core
> > > > > > model should still be the API for plugins and extensions to work
> > > with,
> > > > > > however specifically the consumer POM integrates with so many
> > > external
> > > > > > ecosystems, I would expect it to be defined in terms of XML
> Schema
> > > with
> > > > > > explicite semantic (and the inherent compatibility with exiting
> > > POMs).
> > > > > >
> > > > > > Gruss
> > > > > > Bernd
> > > > > > --
> > > > > > http://bernd.eckenfels.net
> > > > > > ________________________________
> > > > > > Von: Matthieu BROUILLARD
> > > > > > Gesendet: Thursday, December 31, 2020 3:19:09 PM
> > > > > > An: dev@maven.apache.org
> > > > > > Betreff: maven 4.0.0 new XML stuff
> > > > > >
> > > > > > Hello all,
> > > > > >
> > > > > > regarding the active work occurring for maven 4.0.0 I noticed the
> > > > > > introduction of a lot of new stuff around SAX parsing &
> filtering.
> > > > > > I am wondering if that means that it was decided that the input
> > > format
> > > > of
> > > > > > maven projects will be XML forever meaning probably, among
> others,
> > > the
> > > > > end
> > > > > > of polyglot extensions.
> > > > > > Could you explain such a move (or point to rationals/documents)
> and
> > > why
> > > > > you
> > > > > > did not leverage working on the in memory object model allowing
> > > > > > extensions/plugins to contribute/hook in the chain of building
> the
> > > > > BuildPOM
> > > > > > & ConsumePOM? In the past I really thought that this move to
> 'Build
> > > vs
> > > > > > Consumer' POM would make clear separations between the input
> format
> > > of
> > > > > > descriptors and the core system but I perhaps misunderstood
> things.
> > > > > >
> > > > > > Also, are there plans regarding the future of core extensions?
> > > > > > With core extensions it was possible to hook into the POM model
> > > loading
> > > > > and
> > > > > > do transformations to do dynamic changes but by working on the
> XML
> > > > > directly
> > > > > > I see a shift (if not red stop) in this contribution/delegation
> > > > > mechanism.
> > > > > >
> > > > > > Thanks for your time & answers.
> > > > > >
> > > > > > Matthieu Brouillard
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: maven 4.0.0 new XML stuff

Posted by Robert Scholte <rf...@apache.org>.
There's just one thing I want to say:
I'm having trouble with the term "broken".
If a Maven project could be built with M3.6.3, it can still be built with M4.
If not, it is either regression (MNG-6957, MNG-7063) which must be fixed, or it requires changes to a plugin for understandable reasons (maven-pgp-plugin)
AFAIK an interesting extension like the maven-tiles has been tested and still works.

Robert

On 3-1-2021 19:35:25, Romain Manni-Bucau <rm...@gmail.com> wrote:
Le dim. 3 janv. 2021 à 19:04, Robert Scholte a
écrit :

> I don't remember all those details anymore, because I hit those in the
> beginning.
> Trying things over and over again I decided that this is probably the most
> successful approach.
>
>
> What of the goals was to keep the pom.xml as is as much as possible.
> We can only decide for the specific Maven elements how to handle them, we
> should not decide about comments and licenses.
> BTW, the license issue was hard to solve. You cannot use it from the pom's
> , because there might be multiple licenses.
>

I disagree, it is saner IMO to evolve to support that than doing
anything else.
Once again you keep things which don't make sense in a consumed pom in
current impl so i'd say the sucess in a few cases breaks as much cases so
we need to revisit anyway IMHO.


>
> The current implementation is a solid way to ensure we're not breaking too
> much, because Maven controls the XML filters.
>

Hmm, breaking extensions seems to break too much (I'm not speaking of other
parts which breaks the ecosystem there but just that is sufficient IMHO to
say we must check back our solution).


>
> Also keep in mind, that I only want Maven to decide which modifications
> are done.
>

For the consumed pom I agree but it is consistent with keeping everything
working instead of breaking too, no?


>
> Current polyglot projects should still work, but they cannot benefit from
> the build/consumer functions yet.
>

So pom -> build model is kept, build model -> produced pom is broken? Is it
the complete status?
Sounds ok for a 4.0 and a 4.1 can fix it if so.
Just want to ensure first part is not broken at all.


>
> Robert
> On 3-1-2021 16:38:38, Romain Manni-Bucau wrote:
> Le dim. 3 janv. 2021 à 16:18, Robert Scholte a
> écrit :
>
> > > So what I was expecting was: raw xml model -> converted to unified
> > consumed model -> extensions -> model processing.
> >
> > This is only the build pom part. You're missing the consume part, where
> > the xml is distributed.
> >
>
> Yes but with previous chain the consume part is "clean/normalize -> dump"
> since we are using consumed model - only standard model - in memory
> already.
>
>
> > Build is raw + enrich, consumer is raw + enrich + reduce (removing
> > relativePath and modules are the first examples, but much more is
> possible)
> >
> > Going for the in memory was also my first thought, but I would loose
> > information, hence I came up with the current implementation.
> >
>
> I don't see what you loose ot be honest.
> You mentionned license but this one is in the pom so not a big deal,
> comments which are undesired IMHO as mentionned and element order which can
> really be discussed since we can desire to enforce an order to normalize
> consumption + it shouldn't be important since from the project point of
> view your pom is already "broken"/lost (as all your intelligence is lost by
> this "not passthrough" process).
> So overall I don't see what you would loose from the consumer side but I
> see what you lost from maven ecosystem side.
>
>
> > Again, we're at a point where we can have counter solutions, but don't
> > expect me to implement it.
> >
>
> For now I'm just trying to ensure we agree we don't want to break existing
> extensions and the nice ecosystem we built after years.
> This was really a move forward and it sounds like we broke it at maven 4
> without any user gain which sounds terrible.
>
>
> >
> > thanks,
> > Robert
> > On 3-1-2021 15:25:21, Romain Manni-Bucau wrote:
> > Hi all,
> >
> > I kind of join Matthieu thoughts there, there is no point to work at xml
> > level to create the consumed pom - comments is not a point since it can
> > commonly/easily refer to a dropped part of the pom so they should be
> > stripped.
> > Current extension model got proven adapted and adopted, using a lower
> level
> > extension API will not since XML is, even if still mainstream, often
> > replaced by alternative configurations and to have done the work to
> inject
> > XML configuration programmatically compred to current option, it is a
> pain.
> > The in memory model should stick to consumed model IMHO - being
> > programmatic there is no point to make it easier, worse case we can add
> > helper beans (injectable) but in terms of model it will not help.
> >
> > So what I was expecting was:
> >
> > raw xml model -> converted to unified consumed model -> extensions ->
> model
> > processing.
> >
> > Indeed, real chain adds a small processing over the first arrow (inject
> > versions for example) but nothing crazy and breaking this overall flow
> > which stays user friendly.
> >
> > Strictly speaking the new model is just a built-in extension for me which
> > is particular because it will enforce IDE to integrate a new format -
> > wheres polyglot extensions or others don't require static analyzis by
> > themself not being "standard".
> >
> > That said, there is nothing crazy with current implementation, it just
> > require to be updated to be able to take extension changes into account.
> > This can be done by making the extension model 'spyable' (ie if a
> > dependency/plugin is added it will be reflected in the final written
> > pom.xml).
> > This sounds - instrumenting the extension model API or doing a diff after
> > extension phase - like a compromise and let people gets the best of both
> > worlds to me.
> >
> > Wdyt?
> >
> > Romain Manni-Bucau
> > @rmannibucau | Blog
> > | Old Blog
> > | Github |
> > LinkedIn | Book
> >
> >
> >
> > Le dim. 3 janv. 2021 à 14:46, Robert Scholte a
> > écrit :
> >
> > > Hi Matthieu,
> > >
> > > As you understand, something had to be changed to move Maven forward.
> > > I've decided to pick up that challenge and came up with the current
> > > solution.
> > >
> > > My main concerns was that I wanted to keep the fileModel as much as is.
> > > That includes the license, comments and element order.
> > > This information if not available in the memory model, so I needed the
> > > original pom file.
> > > With that in mind, the usage of XMLFilters looks like the most
> > appropriate
> > > solution.
> > >
> > > I am certain that XML is still the most used format, so if we can have
> > > improvements for those users, I'm already very happy.
> > >
> > > And yes, there are plugins that needs to be updated, but doing nothing
> is
> > > not an option anymore.
> > >
> > > There are more people that share their concerns, but it took me several
> > > years to reach this point.
> > > We now have something that seems to work, anybody who can improve or
> can
> > > come up with an alternative implementation can do so.
> > >
> > > thanks,
> > > Robert
> > > On 3-1-2021 12:55:41, Matthieu Brouillard wrote:
> > > Thanks Robert for the video link.
> > >
> > > I fully understand the rationales behind the separation of
> > > build/consumer pom and the video provides some insights on it and you
> > > explain the actual implementation to introduce this change.
> > > Still I do not fully understand why it was decided to work on top of
> XML
> > by
> > > filtering/enhancing it instead of working at the POM (in
> > > memory datamodel) level.
> > > With the current understanding I have, by doing this choice of working
> at
> > > XML level, it looks like it was decided to bypass (if not kill) core
> > > extensions that enhance the POM itself and not the pom.xml ; here I can
> > > think of (but probably not limited to):
> > > - polyglot-maven: do not use XML but other format to describe the POM
> > > (yaml, json, kotlin, java, other XML formats, ...)
> > > - jgitver-maven-plugin (or forks like maven-git-versioning-extension):
> > > dynamic computation of projects version based on git history
> > >
> > > With the introduction of core extensions, I thought it was a move to
> open
> > > the internals and let externals contribute to the capabilities of
> maven.
> > > With the move to a XML handling chain, I see it as a
> restriction/regress
> > in
> > > favor of core closed functionalities. An example of that is what is
> > > provided as CIFriendly stuff, IMO it could/should have been provided
> by a
> > > plugin/extension but instead it is hard written in maven core and is
> not
> > > opened for external contribution (plugin/extension I mean).
> > >
> > > Perhaps I am totally wrong but I think that maven core should define
> all
> > > its expectations at an API level so that extensions/plugins could hook
> at
> > > this API level. The default packaging of maven could/should provide
> > default
> > > implementations of those expectations (for example reading a pom.xml
> file
> > > to a POM model, dumping a POM to a pom-4.0.0.xml,
> transforming/reducing a
> > > POM to POM-consumer, dumping POM-consumer to pom-consumer-5.0.0.xml,
> ...)
> > > and let extensions/plugins/default implementations work along the build
> > > process with the API & POMs to provide different features and
> > capabilities.
> > >
> > > Matthieu
> > >
> > > On Thu, Dec 31, 2020 at 7:01 PM Robert Scholte wrote:
> > >
> > > > I've made a recording[1] about it, which hopefully answers most
> > > questions.
> > > >
> > > > Robert
> > > >
> > > > [1] https://youtu.be/KDAmlNKZJto
> > > >
> > > > On 31-12-2020 16:18:57, Matthieu Brouillard
> > > > wrote:
> > > > > Not exactly sure what work you mean
> > > > everything related to maven-xml: Build/ConsumerPomXMLFilterxxx,
> > > > Build/ConsumerModelSourcexxxx and the transformer stuff.
> > > > Especially, when looking at classes like CiFriendlyXMLFilter, I would
> > > have
> > > > thought that such things could have been done elsewhere, working on
> the
> > > > object model (not on the XML stuff) especially for the BuildPom part.
> > > >
> > > > > however specifically the consumer POM integrates with so many
> > external
> > > > ecosystems
> > > > We're aligned here, this has to be stable and well defined by a
> schema.
> > > >
> > > > Matthieu
> > > >
> > > > On Thu, Dec 31, 2020 at 3:59 PM Bernd Eckenfels
> > > > wrote:
> > > >
> > > > > Hello,
> > > > >
> > > > > Not exactly sure what work you mean and I fully agree that using a
> > core
> > > > > model should still be the API for plugins and extensions to work
> > with,
> > > > > however specifically the consumer POM integrates with so many
> > external
> > > > > ecosystems, I would expect it to be defined in terms of XML Schema
> > with
> > > > > explicite semantic (and the inherent compatibility with exiting
> > POMs).
> > > > >
> > > > > Gruss
> > > > > Bernd
> > > > > --
> > > > > http://bernd.eckenfels.net
> > > > > ________________________________
> > > > > Von: Matthieu BROUILLARD
> > > > > Gesendet: Thursday, December 31, 2020 3:19:09 PM
> > > > > An: dev@maven.apache.org
> > > > > Betreff: maven 4.0.0 new XML stuff
> > > > >
> > > > > Hello all,
> > > > >
> > > > > regarding the active work occurring for maven 4.0.0 I noticed the
> > > > > introduction of a lot of new stuff around SAX parsing & filtering.
> > > > > I am wondering if that means that it was decided that the input
> > format
> > > of
> > > > > maven projects will be XML forever meaning probably, among others,
> > the
> > > > end
> > > > > of polyglot extensions.
> > > > > Could you explain such a move (or point to rationals/documents) and
> > why
> > > > you
> > > > > did not leverage working on the in memory object model allowing
> > > > > extensions/plugins to contribute/hook in the chain of building the
> > > > BuildPOM
> > > > > & ConsumePOM? In the past I really thought that this move to 'Build
> > vs
> > > > > Consumer' POM would make clear separations between the input format
> > of
> > > > > descriptors and the core system but I perhaps misunderstood things.
> > > > >
> > > > > Also, are there plans regarding the future of core extensions?
> > > > > With core extensions it was possible to hook into the POM model
> > loading
> > > > and
> > > > > do transformations to do dynamic changes but by working on the XML
> > > > directly
> > > > > I see a shift (if not red stop) in this contribution/delegation
> > > > mechanism.
> > > > >
> > > > > Thanks for your time & answers.
> > > > >
> > > > > Matthieu Brouillard
> > > > >
> > > >
> > >
> >
>

Re: maven 4.0.0 new XML stuff

Posted by Romain Manni-Bucau <rm...@gmail.com>.
Le dim. 3 janv. 2021 à 19:04, Robert Scholte <rf...@apache.org> a
écrit :

> I don't remember all those details anymore, because I hit those in the
> beginning.
> Trying things over and over again I decided that this is probably the most
> successful approach.
>
>
> What of the goals was to keep the pom.xml as is as much as possible.
> We can only decide for the specific Maven elements how to handle them, we
> should not decide about comments and licenses.
> BTW, the license issue was hard to solve. You cannot use it from the pom's
> <licenses/>, because there might be multiple licenses.
>

I disagree, it is saner IMO to evolve <license> to support that than doing
anything else.
Once again you keep things which don't make sense in a consumed pom in
current impl so i'd say the sucess in a few cases breaks as much cases so
we need to revisit anyway IMHO.


>
> The current implementation is a solid way to ensure we're not breaking too
> much, because Maven controls the XML filters.
>

Hmm, breaking extensions seems to break too much (I'm not speaking of other
parts which breaks the ecosystem there but just that is sufficient IMHO to
say we must check back our solution).


>
> Also keep in mind, that I only want Maven to decide which modifications
> are done.
>

For the consumed pom I agree but it is consistent with keeping everything
working instead of breaking too, no?


>
> Current polyglot projects should still work, but they cannot benefit from
> the build/consumer functions yet.
>

So pom -> build model is kept, build model -> produced pom is broken? Is it
the complete status?
Sounds ok for a 4.0 and a 4.1 can fix it if so.
Just want to ensure first part is not broken at all.


>
> Robert
> On 3-1-2021 16:38:38, Romain Manni-Bucau <rm...@gmail.com> wrote:
> Le dim. 3 janv. 2021 à 16:18, Robert Scholte a
> écrit :
>
> > > So what I was expecting was: raw xml model -> converted to unified
> > consumed model -> extensions -> model processing.
> >
> > This is only the build pom part. You're missing the consume part, where
> > the xml is distributed.
> >
>
> Yes but with previous chain the consume part is "clean/normalize -> dump"
> since we are using consumed model - only standard model - in memory
> already.
>
>
> > Build is raw + enrich, consumer is raw + enrich + reduce (removing
> > relativePath and modules are the first examples, but much more is
> possible)
> >
> > Going for the in memory was also my first thought, but I would loose
> > information, hence I came up with the current implementation.
> >
>
> I don't see what you loose ot be honest.
> You mentionned license but this one is in the pom so not a big deal,
> comments which are undesired IMHO as mentionned and element order which can
> really be discussed since we can desire to enforce an order to normalize
> consumption + it shouldn't be important since from the project point of
> view your pom is already "broken"/lost (as all your intelligence is lost by
> this "not passthrough" process).
> So overall I don't see what you would loose from the consumer side but I
> see what you lost from maven ecosystem side.
>
>
> > Again, we're at a point where we can have counter solutions, but don't
> > expect me to implement it.
> >
>
> For now I'm just trying to ensure we agree we don't want to break existing
> extensions and the nice ecosystem we built after years.
> This was really a move forward and it sounds like we broke it at maven 4
> without any user gain which sounds terrible.
>
>
> >
> > thanks,
> > Robert
> > On 3-1-2021 15:25:21, Romain Manni-Bucau wrote:
> > Hi all,
> >
> > I kind of join Matthieu thoughts there, there is no point to work at xml
> > level to create the consumed pom - comments is not a point since it can
> > commonly/easily refer to a dropped part of the pom so they should be
> > stripped.
> > Current extension model got proven adapted and adopted, using a lower
> level
> > extension API will not since XML is, even if still mainstream, often
> > replaced by alternative configurations and to have done the work to
> inject
> > XML configuration programmatically compred to current option, it is a
> pain.
> > The in memory model should stick to consumed model IMHO - being
> > programmatic there is no point to make it easier, worse case we can add
> > helper beans (injectable) but in terms of model it will not help.
> >
> > So what I was expecting was:
> >
> > raw xml model -> converted to unified consumed model -> extensions ->
> model
> > processing.
> >
> > Indeed, real chain adds a small processing over the first arrow (inject
> > versions for example) but nothing crazy and breaking this overall flow
> > which stays user friendly.
> >
> > Strictly speaking the new model is just a built-in extension for me which
> > is particular because it will enforce IDE to integrate a new format -
> > wheres polyglot extensions or others don't require static analyzis by
> > themself not being "standard".
> >
> > That said, there is nothing crazy with current implementation, it just
> > require to be updated to be able to take extension changes into account.
> > This can be done by making the extension model 'spyable' (ie if a
> > dependency/plugin is added it will be reflected in the final written
> > pom.xml).
> > This sounds - instrumenting the extension model API or doing a diff after
> > extension phase - like a compromise and let people gets the best of both
> > worlds to me.
> >
> > Wdyt?
> >
> > Romain Manni-Bucau
> > @rmannibucau | Blog
> > | Old Blog
> > | Github |
> > LinkedIn | Book
> >
> >
> >
> > Le dim. 3 janv. 2021 à 14:46, Robert Scholte a
> > écrit :
> >
> > > Hi Matthieu,
> > >
> > > As you understand, something had to be changed to move Maven forward.
> > > I've decided to pick up that challenge and came up with the current
> > > solution.
> > >
> > > My main concerns was that I wanted to keep the fileModel as much as is.
> > > That includes the license, comments and element order.
> > > This information if not available in the memory model, so I needed the
> > > original pom file.
> > > With that in mind, the usage of XMLFilters looks like the most
> > appropriate
> > > solution.
> > >
> > > I am certain that XML is still the most used format, so if we can have
> > > improvements for those users, I'm already very happy.
> > >
> > > And yes, there are plugins that needs to be updated, but doing nothing
> is
> > > not an option anymore.
> > >
> > > There are more people that share their concerns, but it took me several
> > > years to reach this point.
> > > We now have something that seems to work, anybody who can improve or
> can
> > > come up with an alternative implementation can do so.
> > >
> > > thanks,
> > > Robert
> > > On 3-1-2021 12:55:41, Matthieu Brouillard wrote:
> > > Thanks Robert for the video link.
> > >
> > > I fully understand the rationales behind the separation of
> > > build/consumer pom and the video provides some insights on it and you
> > > explain the actual implementation to introduce this change.
> > > Still I do not fully understand why it was decided to work on top of
> XML
> > by
> > > filtering/enhancing it instead of working at the POM (in
> > > memory datamodel) level.
> > > With the current understanding I have, by doing this choice of working
> at
> > > XML level, it looks like it was decided to bypass (if not kill) core
> > > extensions that enhance the POM itself and not the pom.xml ; here I can
> > > think of (but probably not limited to):
> > > - polyglot-maven: do not use XML but other format to describe the POM
> > > (yaml, json, kotlin, java, other XML formats, ...)
> > > - jgitver-maven-plugin (or forks like maven-git-versioning-extension):
> > > dynamic computation of projects version based on git history
> > >
> > > With the introduction of core extensions, I thought it was a move to
> open
> > > the internals and let externals contribute to the capabilities of
> maven.
> > > With the move to a XML handling chain, I see it as a
> restriction/regress
> > in
> > > favor of core closed functionalities. An example of that is what is
> > > provided as CIFriendly stuff, IMO it could/should have been provided
> by a
> > > plugin/extension but instead it is hard written in maven core and is
> not
> > > opened for external contribution (plugin/extension I mean).
> > >
> > > Perhaps I am totally wrong but I think that maven core should define
> all
> > > its expectations at an API level so that extensions/plugins could hook
> at
> > > this API level. The default packaging of maven could/should provide
> > default
> > > implementations of those expectations (for example reading a pom.xml
> file
> > > to a POM model, dumping a POM to a pom-4.0.0.xml,
> transforming/reducing a
> > > POM to POM-consumer, dumping POM-consumer to pom-consumer-5.0.0.xml,
> ...)
> > > and let extensions/plugins/default implementations work along the build
> > > process with the API & POMs to provide different features and
> > capabilities.
> > >
> > > Matthieu
> > >
> > > On Thu, Dec 31, 2020 at 7:01 PM Robert Scholte wrote:
> > >
> > > > I've made a recording[1] about it, which hopefully answers most
> > > questions.
> > > >
> > > > Robert
> > > >
> > > > [1] https://youtu.be/KDAmlNKZJto
> > > >
> > > > On 31-12-2020 16:18:57, Matthieu Brouillard
> > > > wrote:
> > > > > Not exactly sure what work you mean
> > > > everything related to maven-xml: Build/ConsumerPomXMLFilterxxx,
> > > > Build/ConsumerModelSourcexxxx and the transformer stuff.
> > > > Especially, when looking at classes like CiFriendlyXMLFilter, I would
> > > have
> > > > thought that such things could have been done elsewhere, working on
> the
> > > > object model (not on the XML stuff) especially for the BuildPom part.
> > > >
> > > > > however specifically the consumer POM integrates with so many
> > external
> > > > ecosystems
> > > > We're aligned here, this has to be stable and well defined by a
> schema.
> > > >
> > > > Matthieu
> > > >
> > > > On Thu, Dec 31, 2020 at 3:59 PM Bernd Eckenfels
> > > > wrote:
> > > >
> > > > > Hello,
> > > > >
> > > > > Not exactly sure what work you mean and I fully agree that using a
> > core
> > > > > model should still be the API for plugins and extensions to work
> > with,
> > > > > however specifically the consumer POM integrates with so many
> > external
> > > > > ecosystems, I would expect it to be defined in terms of XML Schema
> > with
> > > > > explicite semantic (and the inherent compatibility with exiting
> > POMs).
> > > > >
> > > > > Gruss
> > > > > Bernd
> > > > > --
> > > > > http://bernd.eckenfels.net
> > > > > ________________________________
> > > > > Von: Matthieu BROUILLARD
> > > > > Gesendet: Thursday, December 31, 2020 3:19:09 PM
> > > > > An: dev@maven.apache.org
> > > > > Betreff: maven 4.0.0 new XML stuff
> > > > >
> > > > > Hello all,
> > > > >
> > > > > regarding the active work occurring for maven 4.0.0 I noticed the
> > > > > introduction of a lot of new stuff around SAX parsing & filtering.
> > > > > I am wondering if that means that it was decided that the input
> > format
> > > of
> > > > > maven projects will be XML forever meaning probably, among others,
> > the
> > > > end
> > > > > of polyglot extensions.
> > > > > Could you explain such a move (or point to rationals/documents) and
> > why
> > > > you
> > > > > did not leverage working on the in memory object model allowing
> > > > > extensions/plugins to contribute/hook in the chain of building the
> > > > BuildPOM
> > > > > & ConsumePOM? In the past I really thought that this move to 'Build
> > vs
> > > > > Consumer' POM would make clear separations between the input format
> > of
> > > > > descriptors and the core system but I perhaps misunderstood things.
> > > > >
> > > > > Also, are there plans regarding the future of core extensions?
> > > > > With core extensions it was possible to hook into the POM model
> > loading
> > > > and
> > > > > do transformations to do dynamic changes but by working on the XML
> > > > directly
> > > > > I see a shift (if not red stop) in this contribution/delegation
> > > > mechanism.
> > > > >
> > > > > Thanks for your time & answers.
> > > > >
> > > > > Matthieu Brouillard
> > > > >
> > > >
> > >
> >
>

Re: maven 4.0.0 new XML stuff

Posted by Robert Scholte <rf...@apache.org>.
I don't remember all those details anymore, because I hit those in the beginning.
Trying things over and over again I decided that this is probably the most successful approach.


What of the goals was to keep the pom.xml as is as much as possible.
We can only decide for the specific Maven elements how to handle them, we should not decide about comments and licenses.
BTW, the license issue was hard to solve. You cannot use it from the pom's <licenses/>, because there might be multiple licenses. 

The current implementation is a solid way to ensure we're not breaking too much, because Maven controls the XML filters.

Also keep in mind, that I only want Maven to decide which modifications are done.

Current polyglot projects should still work, but they cannot benefit from the build/consumer functions yet.

Robert
On 3-1-2021 16:38:38, Romain Manni-Bucau <rm...@gmail.com> wrote:
Le dim. 3 janv. 2021 à 16:18, Robert Scholte a
écrit :

> > So what I was expecting was: raw xml model -> converted to unified
> consumed model -> extensions -> model processing.
>
> This is only the build pom part. You're missing the consume part, where
> the xml is distributed.
>

Yes but with previous chain the consume part is "clean/normalize -> dump"
since we are using consumed model - only standard model - in memory already.


> Build is raw + enrich, consumer is raw + enrich + reduce (removing
> relativePath and modules are the first examples, but much more is possible)
>
> Going for the in memory was also my first thought, but I would loose
> information, hence I came up with the current implementation.
>

I don't see what you loose ot be honest.
You mentionned license but this one is in the pom so not a big deal,
comments which are undesired IMHO as mentionned and element order which can
really be discussed since we can desire to enforce an order to normalize
consumption + it shouldn't be important since from the project point of
view your pom is already "broken"/lost (as all your intelligence is lost by
this "not passthrough" process).
So overall I don't see what you would loose from the consumer side but I
see what you lost from maven ecosystem side.


> Again, we're at a point where we can have counter solutions, but don't
> expect me to implement it.
>

For now I'm just trying to ensure we agree we don't want to break existing
extensions and the nice ecosystem we built after years.
This was really a move forward and it sounds like we broke it at maven 4
without any user gain which sounds terrible.


>
> thanks,
> Robert
> On 3-1-2021 15:25:21, Romain Manni-Bucau wrote:
> Hi all,
>
> I kind of join Matthieu thoughts there, there is no point to work at xml
> level to create the consumed pom - comments is not a point since it can
> commonly/easily refer to a dropped part of the pom so they should be
> stripped.
> Current extension model got proven adapted and adopted, using a lower level
> extension API will not since XML is, even if still mainstream, often
> replaced by alternative configurations and to have done the work to inject
> XML configuration programmatically compred to current option, it is a pain.
> The in memory model should stick to consumed model IMHO - being
> programmatic there is no point to make it easier, worse case we can add
> helper beans (injectable) but in terms of model it will not help.
>
> So what I was expecting was:
>
> raw xml model -> converted to unified consumed model -> extensions -> model
> processing.
>
> Indeed, real chain adds a small processing over the first arrow (inject
> versions for example) but nothing crazy and breaking this overall flow
> which stays user friendly.
>
> Strictly speaking the new model is just a built-in extension for me which
> is particular because it will enforce IDE to integrate a new format -
> wheres polyglot extensions or others don't require static analyzis by
> themself not being "standard".
>
> That said, there is nothing crazy with current implementation, it just
> require to be updated to be able to take extension changes into account.
> This can be done by making the extension model 'spyable' (ie if a
> dependency/plugin is added it will be reflected in the final written
> pom.xml).
> This sounds - instrumenting the extension model API or doing a diff after
> extension phase - like a compromise and let people gets the best of both
> worlds to me.
>
> Wdyt?
>
> Romain Manni-Bucau
> @rmannibucau | Blog
> | Old Blog
> | Github |
> LinkedIn | Book
>
>
>
> Le dim. 3 janv. 2021 à 14:46, Robert Scholte a
> écrit :
>
> > Hi Matthieu,
> >
> > As you understand, something had to be changed to move Maven forward.
> > I've decided to pick up that challenge and came up with the current
> > solution.
> >
> > My main concerns was that I wanted to keep the fileModel as much as is.
> > That includes the license, comments and element order.
> > This information if not available in the memory model, so I needed the
> > original pom file.
> > With that in mind, the usage of XMLFilters looks like the most
> appropriate
> > solution.
> >
> > I am certain that XML is still the most used format, so if we can have
> > improvements for those users, I'm already very happy.
> >
> > And yes, there are plugins that needs to be updated, but doing nothing is
> > not an option anymore.
> >
> > There are more people that share their concerns, but it took me several
> > years to reach this point.
> > We now have something that seems to work, anybody who can improve or can
> > come up with an alternative implementation can do so.
> >
> > thanks,
> > Robert
> > On 3-1-2021 12:55:41, Matthieu Brouillard wrote:
> > Thanks Robert for the video link.
> >
> > I fully understand the rationales behind the separation of
> > build/consumer pom and the video provides some insights on it and you
> > explain the actual implementation to introduce this change.
> > Still I do not fully understand why it was decided to work on top of XML
> by
> > filtering/enhancing it instead of working at the POM (in
> > memory datamodel) level.
> > With the current understanding I have, by doing this choice of working at
> > XML level, it looks like it was decided to bypass (if not kill) core
> > extensions that enhance the POM itself and not the pom.xml ; here I can
> > think of (but probably not limited to):
> > - polyglot-maven: do not use XML but other format to describe the POM
> > (yaml, json, kotlin, java, other XML formats, ...)
> > - jgitver-maven-plugin (or forks like maven-git-versioning-extension):
> > dynamic computation of projects version based on git history
> >
> > With the introduction of core extensions, I thought it was a move to open
> > the internals and let externals contribute to the capabilities of maven.
> > With the move to a XML handling chain, I see it as a restriction/regress
> in
> > favor of core closed functionalities. An example of that is what is
> > provided as CIFriendly stuff, IMO it could/should have been provided by a
> > plugin/extension but instead it is hard written in maven core and is not
> > opened for external contribution (plugin/extension I mean).
> >
> > Perhaps I am totally wrong but I think that maven core should define all
> > its expectations at an API level so that extensions/plugins could hook at
> > this API level. The default packaging of maven could/should provide
> default
> > implementations of those expectations (for example reading a pom.xml file
> > to a POM model, dumping a POM to a pom-4.0.0.xml, transforming/reducing a
> > POM to POM-consumer, dumping POM-consumer to pom-consumer-5.0.0.xml, ...)
> > and let extensions/plugins/default implementations work along the build
> > process with the API & POMs to provide different features and
> capabilities.
> >
> > Matthieu
> >
> > On Thu, Dec 31, 2020 at 7:01 PM Robert Scholte wrote:
> >
> > > I've made a recording[1] about it, which hopefully answers most
> > questions.
> > >
> > > Robert
> > >
> > > [1] https://youtu.be/KDAmlNKZJto
> > >
> > > On 31-12-2020 16:18:57, Matthieu Brouillard
> > > wrote:
> > > > Not exactly sure what work you mean
> > > everything related to maven-xml: Build/ConsumerPomXMLFilterxxx,
> > > Build/ConsumerModelSourcexxxx and the transformer stuff.
> > > Especially, when looking at classes like CiFriendlyXMLFilter, I would
> > have
> > > thought that such things could have been done elsewhere, working on the
> > > object model (not on the XML stuff) especially for the BuildPom part.
> > >
> > > > however specifically the consumer POM integrates with so many
> external
> > > ecosystems
> > > We're aligned here, this has to be stable and well defined by a schema.
> > >
> > > Matthieu
> > >
> > > On Thu, Dec 31, 2020 at 3:59 PM Bernd Eckenfels
> > > wrote:
> > >
> > > > Hello,
> > > >
> > > > Not exactly sure what work you mean and I fully agree that using a
> core
> > > > model should still be the API for plugins and extensions to work
> with,
> > > > however specifically the consumer POM integrates with so many
> external
> > > > ecosystems, I would expect it to be defined in terms of XML Schema
> with
> > > > explicite semantic (and the inherent compatibility with exiting
> POMs).
> > > >
> > > > Gruss
> > > > Bernd
> > > > --
> > > > http://bernd.eckenfels.net
> > > > ________________________________
> > > > Von: Matthieu BROUILLARD
> > > > Gesendet: Thursday, December 31, 2020 3:19:09 PM
> > > > An: dev@maven.apache.org
> > > > Betreff: maven 4.0.0 new XML stuff
> > > >
> > > > Hello all,
> > > >
> > > > regarding the active work occurring for maven 4.0.0 I noticed the
> > > > introduction of a lot of new stuff around SAX parsing & filtering.
> > > > I am wondering if that means that it was decided that the input
> format
> > of
> > > > maven projects will be XML forever meaning probably, among others,
> the
> > > end
> > > > of polyglot extensions.
> > > > Could you explain such a move (or point to rationals/documents) and
> why
> > > you
> > > > did not leverage working on the in memory object model allowing
> > > > extensions/plugins to contribute/hook in the chain of building the
> > > BuildPOM
> > > > & ConsumePOM? In the past I really thought that this move to 'Build
> vs
> > > > Consumer' POM would make clear separations between the input format
> of
> > > > descriptors and the core system but I perhaps misunderstood things.
> > > >
> > > > Also, are there plans regarding the future of core extensions?
> > > > With core extensions it was possible to hook into the POM model
> loading
> > > and
> > > > do transformations to do dynamic changes but by working on the XML
> > > directly
> > > > I see a shift (if not red stop) in this contribution/delegation
> > > mechanism.
> > > >
> > > > Thanks for your time & answers.
> > > >
> > > > Matthieu Brouillard
> > > >
> > >
> >
>

Re: maven 4.0.0 new XML stuff

Posted by Romain Manni-Bucau <rm...@gmail.com>.
Le dim. 3 janv. 2021 à 16:18, Robert Scholte <rf...@apache.org> a
écrit :

> > So what I was expecting was: raw xml model -> converted to unified
> consumed model -> extensions -> model processing.
>
> This is only the build pom part. You're missing the consume part, where
> the xml is distributed.
>

Yes but with previous chain the consume part is "clean/normalize -> dump"
since we are using consumed model - only standard model - in memory already.


> Build is raw + enrich, consumer is raw + enrich + reduce (removing
> relativePath and modules are the first examples, but much more is possible)
>
> Going for the in memory was also my first thought, but I would loose
> information, hence I came up with the current implementation.
>

I don't see what you loose ot be honest.
You mentionned license but this one is in the pom so not a big deal,
comments which are undesired IMHO as mentionned and element order which can
really be discussed since we can desire to enforce an order to normalize
consumption + it shouldn't be important since from the project point of
view your pom is already "broken"/lost (as all your intelligence is lost by
this "not passthrough" process).
So overall I don't see what you would loose from the consumer side but I
see what you lost from maven ecosystem side.


> Again, we're at a point where we can have counter solutions, but don't
> expect me to implement it.
>

For now I'm just trying to ensure we agree we don't want to break existing
extensions and the nice ecosystem we built after years.
This was really a move forward and it sounds like we broke it at maven 4
without any user gain which sounds terrible.


>
> thanks,
> Robert
> On 3-1-2021 15:25:21, Romain Manni-Bucau <rm...@gmail.com> wrote:
> Hi all,
>
> I kind of join Matthieu thoughts there, there is no point to work at xml
> level to create the consumed pom - comments is not a point since it can
> commonly/easily refer to a dropped part of the pom so they should be
> stripped.
> Current extension model got proven adapted and adopted, using a lower level
> extension API will not since XML is, even if still mainstream, often
> replaced by alternative configurations and to have done the work to inject
> XML configuration programmatically compred to current option, it is a pain.
> The in memory model should stick to consumed model IMHO - being
> programmatic there is no point to make it easier, worse case we can add
> helper beans (injectable) but in terms of model it will not help.
>
> So what I was expecting was:
>
> raw xml model -> converted to unified consumed model -> extensions -> model
> processing.
>
> Indeed, real chain adds a small processing over the first arrow (inject
> versions for example) but nothing crazy and breaking this overall flow
> which stays user friendly.
>
> Strictly speaking the new model is just a built-in extension for me which
> is particular because it will enforce IDE to integrate a new format -
> wheres polyglot extensions or others don't require static analyzis by
> themself not being "standard".
>
> That said, there is nothing crazy with current implementation, it just
> require to be updated to be able to take extension changes into account.
> This can be done by making the extension model 'spyable' (ie if a
> dependency/plugin is added it will be reflected in the final written
> pom.xml).
> This sounds - instrumenting the extension model API or doing a diff after
> extension phase - like a compromise and let people gets the best of both
> worlds to me.
>
> Wdyt?
>
> Romain Manni-Bucau
> @rmannibucau | Blog
> | Old Blog
> | Github |
> LinkedIn | Book
>
>
>
> Le dim. 3 janv. 2021 à 14:46, Robert Scholte a
> écrit :
>
> > Hi Matthieu,
> >
> > As you understand, something had to be changed to move Maven forward.
> > I've decided to pick up that challenge and came up with the current
> > solution.
> >
> > My main concerns was that I wanted to keep the fileModel as much as is.
> > That includes the license, comments and element order.
> > This information if not available in the memory model, so I needed the
> > original pom file.
> > With that in mind, the usage of XMLFilters looks like the most
> appropriate
> > solution.
> >
> > I am certain that XML is still the most used format, so if we can have
> > improvements for those users, I'm already very happy.
> >
> > And yes, there are plugins that needs to be updated, but doing nothing is
> > not an option anymore.
> >
> > There are more people that share their concerns, but it took me several
> > years to reach this point.
> > We now have something that seems to work, anybody who can improve or can
> > come up with an alternative implementation can do so.
> >
> > thanks,
> > Robert
> > On 3-1-2021 12:55:41, Matthieu Brouillard wrote:
> > Thanks Robert for the video link.
> >
> > I fully understand the rationales behind the separation of
> > build/consumer pom and the video provides some insights on it and you
> > explain the actual implementation to introduce this change.
> > Still I do not fully understand why it was decided to work on top of XML
> by
> > filtering/enhancing it instead of working at the POM (in
> > memory datamodel) level.
> > With the current understanding I have, by doing this choice of working at
> > XML level, it looks like it was decided to bypass (if not kill) core
> > extensions that enhance the POM itself and not the pom.xml ; here I can
> > think of (but probably not limited to):
> > - polyglot-maven: do not use XML but other format to describe the POM
> > (yaml, json, kotlin, java, other XML formats, ...)
> > - jgitver-maven-plugin (or forks like maven-git-versioning-extension):
> > dynamic computation of projects version based on git history
> >
> > With the introduction of core extensions, I thought it was a move to open
> > the internals and let externals contribute to the capabilities of maven.
> > With the move to a XML handling chain, I see it as a restriction/regress
> in
> > favor of core closed functionalities. An example of that is what is
> > provided as CIFriendly stuff, IMO it could/should have been provided by a
> > plugin/extension but instead it is hard written in maven core and is not
> > opened for external contribution (plugin/extension I mean).
> >
> > Perhaps I am totally wrong but I think that maven core should define all
> > its expectations at an API level so that extensions/plugins could hook at
> > this API level. The default packaging of maven could/should provide
> default
> > implementations of those expectations (for example reading a pom.xml file
> > to a POM model, dumping a POM to a pom-4.0.0.xml, transforming/reducing a
> > POM to POM-consumer, dumping POM-consumer to pom-consumer-5.0.0.xml, ...)
> > and let extensions/plugins/default implementations work along the build
> > process with the API & POMs to provide different features and
> capabilities.
> >
> > Matthieu
> >
> > On Thu, Dec 31, 2020 at 7:01 PM Robert Scholte wrote:
> >
> > > I've made a recording[1] about it, which hopefully answers most
> > questions.
> > >
> > > Robert
> > >
> > > [1] https://youtu.be/KDAmlNKZJto
> > >
> > > On 31-12-2020 16:18:57, Matthieu Brouillard
> > > wrote:
> > > > Not exactly sure what work you mean
> > > everything related to maven-xml: Build/ConsumerPomXMLFilterxxx,
> > > Build/ConsumerModelSourcexxxx and the transformer stuff.
> > > Especially, when looking at classes like CiFriendlyXMLFilter, I would
> > have
> > > thought that such things could have been done elsewhere, working on the
> > > object model (not on the XML stuff) especially for the BuildPom part.
> > >
> > > > however specifically the consumer POM integrates with so many
> external
> > > ecosystems
> > > We're aligned here, this has to be stable and well defined by a schema.
> > >
> > > Matthieu
> > >
> > > On Thu, Dec 31, 2020 at 3:59 PM Bernd Eckenfels
> > > wrote:
> > >
> > > > Hello,
> > > >
> > > > Not exactly sure what work you mean and I fully agree that using a
> core
> > > > model should still be the API for plugins and extensions to work
> with,
> > > > however specifically the consumer POM integrates with so many
> external
> > > > ecosystems, I would expect it to be defined in terms of XML Schema
> with
> > > > explicite semantic (and the inherent compatibility with exiting
> POMs).
> > > >
> > > > Gruss
> > > > Bernd
> > > > --
> > > > http://bernd.eckenfels.net
> > > > ________________________________
> > > > Von: Matthieu BROUILLARD
> > > > Gesendet: Thursday, December 31, 2020 3:19:09 PM
> > > > An: dev@maven.apache.org
> > > > Betreff: maven 4.0.0 new XML stuff
> > > >
> > > > Hello all,
> > > >
> > > > regarding the active work occurring for maven 4.0.0 I noticed the
> > > > introduction of a lot of new stuff around SAX parsing & filtering.
> > > > I am wondering if that means that it was decided that the input
> format
> > of
> > > > maven projects will be XML forever meaning probably, among others,
> the
> > > end
> > > > of polyglot extensions.
> > > > Could you explain such a move (or point to rationals/documents) and
> why
> > > you
> > > > did not leverage working on the in memory object model allowing
> > > > extensions/plugins to contribute/hook in the chain of building the
> > > BuildPOM
> > > > & ConsumePOM? In the past I really thought that this move to 'Build
> vs
> > > > Consumer' POM would make clear separations between the input format
> of
> > > > descriptors and the core system but I perhaps misunderstood things.
> > > >
> > > > Also, are there plans regarding the future of core extensions?
> > > > With core extensions it was possible to hook into the POM model
> loading
> > > and
> > > > do transformations to do dynamic changes but by working on the XML
> > > directly
> > > > I see a shift (if not red stop) in this contribution/delegation
> > > mechanism.
> > > >
> > > > Thanks for your time & answers.
> > > >
> > > > Matthieu Brouillard
> > > >
> > >
> >
>

Re: maven 4.0.0 new XML stuff

Posted by Robert Scholte <rf...@apache.org>.
> So what I was expecting was: raw xml model -> converted to unified consumed model -> extensions -> model processing.

This is only the build pom part. You're missing the consume part, where the xml is distributed.
Build is raw + enrich, consumer is raw + enrich + reduce (removing relativePath and modules are the first examples, but much more is possible)

Going for the in memory was also my first thought, but I would loose information, hence I came up with the current implementation.

Again, we're at a point where we can have counter solutions, but don't expect me to implement it.

thanks,
Robert
On 3-1-2021 15:25:21, Romain Manni-Bucau <rm...@gmail.com> wrote:
Hi all,

I kind of join Matthieu thoughts there, there is no point to work at xml
level to create the consumed pom - comments is not a point since it can
commonly/easily refer to a dropped part of the pom so they should be
stripped.
Current extension model got proven adapted and adopted, using a lower level
extension API will not since XML is, even if still mainstream, often
replaced by alternative configurations and to have done the work to inject
XML configuration programmatically compred to current option, it is a pain.
The in memory model should stick to consumed model IMHO - being
programmatic there is no point to make it easier, worse case we can add
helper beans (injectable) but in terms of model it will not help.

So what I was expecting was:

raw xml model -> converted to unified consumed model -> extensions -> model
processing.

Indeed, real chain adds a small processing over the first arrow (inject
versions for example) but nothing crazy and breaking this overall flow
which stays user friendly.

Strictly speaking the new model is just a built-in extension for me which
is particular because it will enforce IDE to integrate a new format -
wheres polyglot extensions or others don't require static analyzis by
themself not being "standard".

That said, there is nothing crazy with current implementation, it just
require to be updated to be able to take extension changes into account.
This can be done by making the extension model 'spyable' (ie if a
dependency/plugin is added it will be reflected in the final written
pom.xml).
This sounds - instrumenting the extension model API or doing a diff after
extension phase - like a compromise and let people gets the best of both
worlds to me.

Wdyt?

Romain Manni-Bucau
@rmannibucau | Blog
| Old Blog
| Github |
LinkedIn | Book



Le dim. 3 janv. 2021 à 14:46, Robert Scholte a
écrit :

> Hi Matthieu,
>
> As you understand, something had to be changed to move Maven forward.
> I've decided to pick up that challenge and came up with the current
> solution.
>
> My main concerns was that I wanted to keep the fileModel as much as is.
> That includes the license, comments and element order.
> This information if not available in the memory model, so I needed the
> original pom file.
> With that in mind, the usage of XMLFilters looks like the most appropriate
> solution.
>
> I am certain that XML is still the most used format, so if we can have
> improvements for those users, I'm already very happy.
>
> And yes, there are plugins that needs to be updated, but doing nothing is
> not an option anymore.
>
> There are more people that share their concerns, but it took me several
> years to reach this point.
> We now have something that seems to work, anybody who can improve or can
> come up with an alternative implementation can do so.
>
> thanks,
> Robert
> On 3-1-2021 12:55:41, Matthieu Brouillard wrote:
> Thanks Robert for the video link.
>
> I fully understand the rationales behind the separation of
> build/consumer pom and the video provides some insights on it and you
> explain the actual implementation to introduce this change.
> Still I do not fully understand why it was decided to work on top of XML by
> filtering/enhancing it instead of working at the POM (in
> memory datamodel) level.
> With the current understanding I have, by doing this choice of working at
> XML level, it looks like it was decided to bypass (if not kill) core
> extensions that enhance the POM itself and not the pom.xml ; here I can
> think of (but probably not limited to):
> - polyglot-maven: do not use XML but other format to describe the POM
> (yaml, json, kotlin, java, other XML formats, ...)
> - jgitver-maven-plugin (or forks like maven-git-versioning-extension):
> dynamic computation of projects version based on git history
>
> With the introduction of core extensions, I thought it was a move to open
> the internals and let externals contribute to the capabilities of maven.
> With the move to a XML handling chain, I see it as a restriction/regress in
> favor of core closed functionalities. An example of that is what is
> provided as CIFriendly stuff, IMO it could/should have been provided by a
> plugin/extension but instead it is hard written in maven core and is not
> opened for external contribution (plugin/extension I mean).
>
> Perhaps I am totally wrong but I think that maven core should define all
> its expectations at an API level so that extensions/plugins could hook at
> this API level. The default packaging of maven could/should provide default
> implementations of those expectations (for example reading a pom.xml file
> to a POM model, dumping a POM to a pom-4.0.0.xml, transforming/reducing a
> POM to POM-consumer, dumping POM-consumer to pom-consumer-5.0.0.xml, ...)
> and let extensions/plugins/default implementations work along the build
> process with the API & POMs to provide different features and capabilities.
>
> Matthieu
>
> On Thu, Dec 31, 2020 at 7:01 PM Robert Scholte wrote:
>
> > I've made a recording[1] about it, which hopefully answers most
> questions.
> >
> > Robert
> >
> > [1] https://youtu.be/KDAmlNKZJto
> >
> > On 31-12-2020 16:18:57, Matthieu Brouillard
> > wrote:
> > > Not exactly sure what work you mean
> > everything related to maven-xml: Build/ConsumerPomXMLFilterxxx,
> > Build/ConsumerModelSourcexxxx and the transformer stuff.
> > Especially, when looking at classes like CiFriendlyXMLFilter, I would
> have
> > thought that such things could have been done elsewhere, working on the
> > object model (not on the XML stuff) especially for the BuildPom part.
> >
> > > however specifically the consumer POM integrates with so many external
> > ecosystems
> > We're aligned here, this has to be stable and well defined by a schema.
> >
> > Matthieu
> >
> > On Thu, Dec 31, 2020 at 3:59 PM Bernd Eckenfels
> > wrote:
> >
> > > Hello,
> > >
> > > Not exactly sure what work you mean and I fully agree that using a core
> > > model should still be the API for plugins and extensions to work with,
> > > however specifically the consumer POM integrates with so many external
> > > ecosystems, I would expect it to be defined in terms of XML Schema with
> > > explicite semantic (and the inherent compatibility with exiting POMs).
> > >
> > > Gruss
> > > Bernd
> > > --
> > > http://bernd.eckenfels.net
> > > ________________________________
> > > Von: Matthieu BROUILLARD
> > > Gesendet: Thursday, December 31, 2020 3:19:09 PM
> > > An: dev@maven.apache.org
> > > Betreff: maven 4.0.0 new XML stuff
> > >
> > > Hello all,
> > >
> > > regarding the active work occurring for maven 4.0.0 I noticed the
> > > introduction of a lot of new stuff around SAX parsing & filtering.
> > > I am wondering if that means that it was decided that the input format
> of
> > > maven projects will be XML forever meaning probably, among others, the
> > end
> > > of polyglot extensions.
> > > Could you explain such a move (or point to rationals/documents) and why
> > you
> > > did not leverage working on the in memory object model allowing
> > > extensions/plugins to contribute/hook in the chain of building the
> > BuildPOM
> > > & ConsumePOM? In the past I really thought that this move to 'Build vs
> > > Consumer' POM would make clear separations between the input format of
> > > descriptors and the core system but I perhaps misunderstood things.
> > >
> > > Also, are there plans regarding the future of core extensions?
> > > With core extensions it was possible to hook into the POM model loading
> > and
> > > do transformations to do dynamic changes but by working on the XML
> > directly
> > > I see a shift (if not red stop) in this contribution/delegation
> > mechanism.
> > >
> > > Thanks for your time & answers.
> > >
> > > Matthieu Brouillard
> > >
> >
>

Re: maven 4.0.0 new XML stuff

Posted by Romain Manni-Bucau <rm...@gmail.com>.
Hi all,

I kind of join Matthieu thoughts there, there is no point to work at xml
level to create the consumed pom - comments is not a point since it can
commonly/easily refer to a dropped part of the pom so they should be
stripped.
Current extension model got proven adapted and adopted, using a lower level
extension API will not since XML is, even if still mainstream, often
replaced by alternative configurations and to have done the work to inject
XML configuration programmatically compred to current option, it is a pain.
The in memory model should stick to consumed model IMHO - being
programmatic there is no point to make it easier, worse case we can add
helper beans (injectable) but in terms of model it will not help.

So what I was expecting was:

raw xml model -> converted to unified consumed model -> extensions -> model
processing.

Indeed, real chain adds a small processing over the first arrow (inject
versions for example) but nothing crazy and breaking this overall flow
which stays user friendly.

Strictly speaking the new model is just a built-in extension for me which
is particular because it will enforce IDE to integrate a new format -
wheres polyglot extensions or others don't require static analyzis by
themself not being "standard".

That said, there is nothing crazy with current implementation, it just
require to be updated to be able to take extension changes into account.
This can be done by making the extension model 'spyable' (ie if a
dependency/plugin is added it will be reflected in the final written
pom.xml).
This sounds - instrumenting the extension model API or doing a diff after
extension phase - like a compromise and let people gets the best of both
worlds to me.

Wdyt?

Romain Manni-Bucau
@rmannibucau <https://twitter.com/rmannibucau> |  Blog
<https://rmannibucau.metawerx.net/> | Old Blog
<http://rmannibucau.wordpress.com> | Github <https://github.com/rmannibucau> |
LinkedIn <https://www.linkedin.com/in/rmannibucau> | Book
<https://www.packtpub.com/application-development/java-ee-8-high-performance>


Le dim. 3 janv. 2021 à 14:46, Robert Scholte <rf...@apache.org> a
écrit :

> Hi Matthieu,
>
> As you understand, something had to be changed to move Maven forward.
> I've decided to pick up that challenge and came up with the current
> solution.
>
> My main concerns was that I wanted to keep the fileModel as much as is.
> That includes the license, comments and element order.
> This information if not available in the memory model, so I needed the
> original pom file.
> With that in mind, the usage of XMLFilters looks like the most appropriate
> solution.
>
> I am certain that XML is still the most used format, so if we can have
> improvements for those users, I'm already very happy.
>
> And yes, there are plugins that needs to be updated, but doing nothing is
> not an option anymore.
>
> There are more people that share their concerns, but it took me several
> years to reach this point.
> We now have something that seems to work, anybody who can improve or can
> come up with an alternative implementation can do so.
>
> thanks,
> Robert
> On 3-1-2021 12:55:41, Matthieu Brouillard <ma...@brouillard.fr> wrote:
> Thanks Robert for the video link.
>
> I fully understand the rationales behind the separation of
> build/consumer pom and the video provides some insights on it and you
> explain the actual implementation to introduce this change.
> Still I do not fully understand why it was decided to work on top of XML by
> filtering/enhancing it instead of working at the POM (in
> memory datamodel) level.
> With the current understanding I have, by doing this choice of working at
> XML level, it looks like it was decided to bypass (if not kill) core
> extensions that enhance the POM itself and not the pom.xml ; here I can
> think of (but probably not limited to):
> - polyglot-maven: do not use XML but other format to describe the POM
> (yaml, json, kotlin, java, other XML formats, ...)
> - jgitver-maven-plugin (or forks like maven-git-versioning-extension):
> dynamic computation of projects version based on git history
>
> With the introduction of core extensions, I thought it was a move to open
> the internals and let externals contribute to the capabilities of maven.
> With the move to a XML handling chain, I see it as a restriction/regress in
> favor of core closed functionalities. An example of that is what is
> provided as CIFriendly stuff, IMO it could/should have been provided by a
> plugin/extension but instead it is hard written in maven core and is not
> opened for external contribution (plugin/extension I mean).
>
> Perhaps I am totally wrong but I think that maven core should define all
> its expectations at an API level so that extensions/plugins could hook at
> this API level. The default packaging of maven could/should provide default
> implementations of those expectations (for example reading a pom.xml file
> to a POM model, dumping a POM to a pom-4.0.0.xml, transforming/reducing a
> POM to POM-consumer, dumping POM-consumer to pom-consumer-5.0.0.xml, ...)
> and let extensions/plugins/default implementations work along the build
> process with the API & POMs to provide different features and capabilities.
>
> Matthieu
>
> On Thu, Dec 31, 2020 at 7:01 PM Robert Scholte wrote:
>
> > I've made a recording[1] about it, which hopefully answers most
> questions.
> >
> > Robert
> >
> > [1] https://youtu.be/KDAmlNKZJto
> >
> > On 31-12-2020 16:18:57, Matthieu Brouillard
> > wrote:
> > > Not exactly sure what work you mean
> > everything related to maven-xml: Build/ConsumerPomXMLFilterxxx,
> > Build/ConsumerModelSourcexxxx and the transformer stuff.
> > Especially, when looking at classes like CiFriendlyXMLFilter, I would
> have
> > thought that such things could have been done elsewhere, working on the
> > object model (not on the XML stuff) especially for the BuildPom part.
> >
> > > however specifically the consumer POM integrates with so many external
> > ecosystems
> > We're aligned here, this has to be stable and well defined by a schema.
> >
> > Matthieu
> >
> > On Thu, Dec 31, 2020 at 3:59 PM Bernd Eckenfels
> > wrote:
> >
> > > Hello,
> > >
> > > Not exactly sure what work you mean and I fully agree that using a core
> > > model should still be the API for plugins and extensions to work with,
> > > however specifically the consumer POM integrates with so many external
> > > ecosystems, I would expect it to be defined in terms of XML Schema with
> > > explicite semantic (and the inherent compatibility with exiting POMs).
> > >
> > > Gruss
> > > Bernd
> > > --
> > > http://bernd.eckenfels.net
> > > ________________________________
> > > Von: Matthieu BROUILLARD
> > > Gesendet: Thursday, December 31, 2020 3:19:09 PM
> > > An: dev@maven.apache.org
> > > Betreff: maven 4.0.0 new XML stuff
> > >
> > > Hello all,
> > >
> > > regarding the active work occurring for maven 4.0.0 I noticed the
> > > introduction of a lot of new stuff around SAX parsing & filtering.
> > > I am wondering if that means that it was decided that the input format
> of
> > > maven projects will be XML forever meaning probably, among others, the
> > end
> > > of polyglot extensions.
> > > Could you explain such a move (or point to rationals/documents) and why
> > you
> > > did not leverage working on the in memory object model allowing
> > > extensions/plugins to contribute/hook in the chain of building the
> > BuildPOM
> > > & ConsumePOM? In the past I really thought that this move to 'Build vs
> > > Consumer' POM would make clear separations between the input format of
> > > descriptors and the core system but I perhaps misunderstood things.
> > >
> > > Also, are there plans regarding the future of core extensions?
> > > With core extensions it was possible to hook into the POM model loading
> > and
> > > do transformations to do dynamic changes but by working on the XML
> > directly
> > > I see a shift (if not red stop) in this contribution/delegation
> > mechanism.
> > >
> > > Thanks for your time & answers.
> > >
> > > Matthieu Brouillard
> > >
> >
>

Re: maven 4.0.0 new XML stuff

Posted by Robert Scholte <rf...@apache.org>.
Hi Matthieu,

As you understand, something had to be changed to move Maven forward.
I've decided to pick up that challenge and came up with the current solution.

My main concerns was that I wanted to keep the fileModel as much as is.
That includes the license, comments and element order. 
This information if not available in the memory model, so I needed the original pom file.
With that in mind, the usage of XMLFilters looks like the most appropriate solution.

I am certain that XML is still the most used format, so if we can have improvements for those users, I'm already very happy.

And yes, there are plugins that needs to be updated, but doing nothing is not an option anymore.

There are more people that share their concerns, but it took me several years to reach this point.
We now have something that seems to work, anybody who can improve or can come up with an alternative implementation can do so.

thanks,
Robert
On 3-1-2021 12:55:41, Matthieu Brouillard <ma...@brouillard.fr> wrote:
Thanks Robert for the video link.

I fully understand the rationales behind the separation of
build/consumer pom and the video provides some insights on it and you
explain the actual implementation to introduce this change.
Still I do not fully understand why it was decided to work on top of XML by
filtering/enhancing it instead of working at the POM (in
memory datamodel) level.
With the current understanding I have, by doing this choice of working at
XML level, it looks like it was decided to bypass (if not kill) core
extensions that enhance the POM itself and not the pom.xml ; here I can
think of (but probably not limited to):
- polyglot-maven: do not use XML but other format to describe the POM
(yaml, json, kotlin, java, other XML formats, ...)
- jgitver-maven-plugin (or forks like maven-git-versioning-extension):
dynamic computation of projects version based on git history

With the introduction of core extensions, I thought it was a move to open
the internals and let externals contribute to the capabilities of maven.
With the move to a XML handling chain, I see it as a restriction/regress in
favor of core closed functionalities. An example of that is what is
provided as CIFriendly stuff, IMO it could/should have been provided by a
plugin/extension but instead it is hard written in maven core and is not
opened for external contribution (plugin/extension I mean).

Perhaps I am totally wrong but I think that maven core should define all
its expectations at an API level so that extensions/plugins could hook at
this API level. The default packaging of maven could/should provide default
implementations of those expectations (for example reading a pom.xml file
to a POM model, dumping a POM to a pom-4.0.0.xml, transforming/reducing a
POM to POM-consumer, dumping POM-consumer to pom-consumer-5.0.0.xml, ...)
and let extensions/plugins/default implementations work along the build
process with the API & POMs to provide different features and capabilities.

Matthieu

On Thu, Dec 31, 2020 at 7:01 PM Robert Scholte wrote:

> I've made a recording[1] about it, which hopefully answers most questions.
>
> Robert
>
> [1] https://youtu.be/KDAmlNKZJto
>
> On 31-12-2020 16:18:57, Matthieu Brouillard
> wrote:
> > Not exactly sure what work you mean
> everything related to maven-xml: Build/ConsumerPomXMLFilterxxx,
> Build/ConsumerModelSourcexxxx and the transformer stuff.
> Especially, when looking at classes like CiFriendlyXMLFilter, I would have
> thought that such things could have been done elsewhere, working on the
> object model (not on the XML stuff) especially for the BuildPom part.
>
> > however specifically the consumer POM integrates with so many external
> ecosystems
> We're aligned here, this has to be stable and well defined by a schema.
>
> Matthieu
>
> On Thu, Dec 31, 2020 at 3:59 PM Bernd Eckenfels
> wrote:
>
> > Hello,
> >
> > Not exactly sure what work you mean and I fully agree that using a core
> > model should still be the API for plugins and extensions to work with,
> > however specifically the consumer POM integrates with so many external
> > ecosystems, I would expect it to be defined in terms of XML Schema with
> > explicite semantic (and the inherent compatibility with exiting POMs).
> >
> > Gruss
> > Bernd
> > --
> > http://bernd.eckenfels.net
> > ________________________________
> > Von: Matthieu BROUILLARD
> > Gesendet: Thursday, December 31, 2020 3:19:09 PM
> > An: dev@maven.apache.org
> > Betreff: maven 4.0.0 new XML stuff
> >
> > Hello all,
> >
> > regarding the active work occurring for maven 4.0.0 I noticed the
> > introduction of a lot of new stuff around SAX parsing & filtering.
> > I am wondering if that means that it was decided that the input format of
> > maven projects will be XML forever meaning probably, among others, the
> end
> > of polyglot extensions.
> > Could you explain such a move (or point to rationals/documents) and why
> you
> > did not leverage working on the in memory object model allowing
> > extensions/plugins to contribute/hook in the chain of building the
> BuildPOM
> > & ConsumePOM? In the past I really thought that this move to 'Build vs
> > Consumer' POM would make clear separations between the input format of
> > descriptors and the core system but I perhaps misunderstood things.
> >
> > Also, are there plans regarding the future of core extensions?
> > With core extensions it was possible to hook into the POM model loading
> and
> > do transformations to do dynamic changes but by working on the XML
> directly
> > I see a shift (if not red stop) in this contribution/delegation
> mechanism.
> >
> > Thanks for your time & answers.
> >
> > Matthieu Brouillard
> >
>

Re: maven 4.0.0 new XML stuff

Posted by Matthieu Brouillard <ma...@brouillard.fr>.
Thanks Robert for the video link.

I fully understand the rationales behind the separation of
build/consumer pom and the video provides some insights on it and you
explain the actual implementation to introduce this change.
Still I do not fully understand why it was decided to work on top of XML by
filtering/enhancing it instead of working at the POM (in
memory datamodel) level.
With the current understanding I have, by doing this choice of working at
XML level, it looks like it was decided to bypass (if not kill) core
extensions that enhance the POM itself and not the pom.xml ; here I can
think of (but probably not limited to):
- polyglot-maven: do not use XML but other format to describe the POM
(yaml, json, kotlin, java, other XML formats, ...)
- jgitver-maven-plugin (or forks like maven-git-versioning-extension):
dynamic computation of projects version based on git history

With the introduction of core extensions, I thought it was a move to open
the internals and let externals contribute to the capabilities of maven.
With the move to a XML handling chain, I see it as a restriction/regress in
favor of core closed functionalities. An example of that is what is
provided as CIFriendly stuff, IMO it could/should have been provided by a
plugin/extension but instead it is hard written in maven core and is not
opened for external contribution (plugin/extension I mean).

Perhaps I am totally wrong but I think that maven core should define all
its expectations at an API level so that extensions/plugins could hook at
this API level. The default packaging of maven could/should provide default
implementations of those expectations (for example reading a pom.xml file
to a POM model, dumping a POM to a pom-4.0.0.xml, transforming/reducing a
POM to POM-consumer, dumping POM-consumer to pom-consumer-5.0.0.xml, ...)
and let extensions/plugins/default implementations work along the build
process with the API & POMs to provide different features and capabilities.

Matthieu

On Thu, Dec 31, 2020 at 7:01 PM Robert Scholte <rf...@apache.org> wrote:

> I've made a recording[1] about it, which hopefully answers most questions.
>
> Robert
>
> [1] https://youtu.be/KDAmlNKZJto
>
> On 31-12-2020 16:18:57, Matthieu Brouillard <ma...@brouillard.fr>
> wrote:
> > Not exactly sure what work you mean
> everything related to maven-xml: Build/ConsumerPomXMLFilterxxx,
> Build/ConsumerModelSourcexxxx and the transformer stuff.
> Especially, when looking at classes like CiFriendlyXMLFilter, I would have
> thought that such things could have been done elsewhere, working on the
> object model (not on the XML stuff) especially for the BuildPom part.
>
> > however specifically the consumer POM integrates with so many external
> ecosystems
> We're aligned here, this has to be stable and well defined by a schema.
>
> Matthieu
>
> On Thu, Dec 31, 2020 at 3:59 PM Bernd Eckenfels
> wrote:
>
> > Hello,
> >
> > Not exactly sure what work you mean and I fully agree that using a core
> > model should still be the API for plugins and extensions to work with,
> > however specifically the consumer POM integrates with so many external
> > ecosystems, I would expect it to be defined in terms of XML Schema with
> > explicite semantic (and the inherent compatibility with exiting POMs).
> >
> > Gruss
> > Bernd
> > --
> > http://bernd.eckenfels.net
> > ________________________________
> > Von: Matthieu BROUILLARD
> > Gesendet: Thursday, December 31, 2020 3:19:09 PM
> > An: dev@maven.apache.org
> > Betreff: maven 4.0.0 new XML stuff
> >
> > Hello all,
> >
> > regarding the active work occurring for maven 4.0.0 I noticed the
> > introduction of a lot of new stuff around SAX parsing & filtering.
> > I am wondering if that means that it was decided that the input format of
> > maven projects will be XML forever meaning probably, among others, the
> end
> > of polyglot extensions.
> > Could you explain such a move (or point to rationals/documents) and why
> you
> > did not leverage working on the in memory object model allowing
> > extensions/plugins to contribute/hook in the chain of building the
> BuildPOM
> > & ConsumePOM? In the past I really thought that this move to 'Build vs
> > Consumer' POM would make clear separations between the input format of
> > descriptors and the core system but I perhaps misunderstood things.
> >
> > Also, are there plans regarding the future of core extensions?
> > With core extensions it was possible to hook into the POM model loading
> and
> > do transformations to do dynamic changes but by working on the XML
> directly
> > I see a shift (if not red stop) in this contribution/delegation
> mechanism.
> >
> > Thanks for your time & answers.
> >
> > Matthieu Brouillard
> >
>

Re: maven 4.0.0 new XML stuff

Posted by Robert Scholte <rf...@apache.org>.
Yes, I think so.
Per artifact, if it includes the POM, we need to decide which one to include.
It should never be the raw one (e.g. it might miss the version to its parent), most of the time the consumer POM, but sometimes the build POM.

Robert
On 1-1-2021 15:14:52, Gary Gregory <ga...@gmail.com> wrote:
Great idea to record this video, TY!

I would guess that the javadoc plugin would change in the same way the
source plugin needs to change, right?

Gary

On Thu, Dec 31, 2020, 13:01 Robert Scholte wrote:

> I've made a recording[1] about it, which hopefully answers most questions.
>
> Robert
>
> [1] https://youtu.be/KDAmlNKZJto
>
> On 31-12-2020 16:18:57, Matthieu Brouillard
> wrote:
> > Not exactly sure what work you mean
> everything related to maven-xml: Build/ConsumerPomXMLFilterxxx,
> Build/ConsumerModelSourcexxxx and the transformer stuff.
> Especially, when looking at classes like CiFriendlyXMLFilter, I would have
> thought that such things could have been done elsewhere, working on the
> object model (not on the XML stuff) especially for the BuildPom part.
>
> > however specifically the consumer POM integrates with so many external
> ecosystems
> We're aligned here, this has to be stable and well defined by a schema.
>
> Matthieu
>
> On Thu, Dec 31, 2020 at 3:59 PM Bernd Eckenfels
> wrote:
>
> > Hello,
> >
> > Not exactly sure what work you mean and I fully agree that using a core
> > model should still be the API for plugins and extensions to work with,
> > however specifically the consumer POM integrates with so many external
> > ecosystems, I would expect it to be defined in terms of XML Schema with
> > explicite semantic (and the inherent compatibility with exiting POMs).
> >
> > Gruss
> > Bernd
> > --
> > http://bernd.eckenfels.net
> > ________________________________
> > Von: Matthieu BROUILLARD
> > Gesendet: Thursday, December 31, 2020 3:19:09 PM
> > An: dev@maven.apache.org
> > Betreff: maven 4.0.0 new XML stuff
> >
> > Hello all,
> >
> > regarding the active work occurring for maven 4.0.0 I noticed the
> > introduction of a lot of new stuff around SAX parsing & filtering.
> > I am wondering if that means that it was decided that the input format of
> > maven projects will be XML forever meaning probably, among others, the
> end
> > of polyglot extensions.
> > Could you explain such a move (or point to rationals/documents) and why
> you
> > did not leverage working on the in memory object model allowing
> > extensions/plugins to contribute/hook in the chain of building the
> BuildPOM
> > & ConsumePOM? In the past I really thought that this move to 'Build vs
> > Consumer' POM would make clear separations between the input format of
> > descriptors and the core system but I perhaps misunderstood things.
> >
> > Also, are there plans regarding the future of core extensions?
> > With core extensions it was possible to hook into the POM model loading
> and
> > do transformations to do dynamic changes but by working on the XML
> directly
> > I see a shift (if not red stop) in this contribution/delegation
> mechanism.
> >
> > Thanks for your time & answers.
> >
> > Matthieu Brouillard
> >
>

Re: maven 4.0.0 new XML stuff

Posted by Gary Gregory <ga...@gmail.com>.
Great idea to record this video, TY!

I would guess that the javadoc plugin would change in the same way the
source plugin needs to change, right?

Gary

On Thu, Dec 31, 2020, 13:01 Robert Scholte <rf...@apache.org> wrote:

> I've made a recording[1] about it, which hopefully answers most questions.
>
> Robert
>
> [1] https://youtu.be/KDAmlNKZJto
>
> On 31-12-2020 16:18:57, Matthieu Brouillard <ma...@brouillard.fr>
> wrote:
> > Not exactly sure what work you mean
> everything related to maven-xml: Build/ConsumerPomXMLFilterxxx,
> Build/ConsumerModelSourcexxxx and the transformer stuff.
> Especially, when looking at classes like CiFriendlyXMLFilter, I would have
> thought that such things could have been done elsewhere, working on the
> object model (not on the XML stuff) especially for the BuildPom part.
>
> > however specifically the consumer POM integrates with so many external
> ecosystems
> We're aligned here, this has to be stable and well defined by a schema.
>
> Matthieu
>
> On Thu, Dec 31, 2020 at 3:59 PM Bernd Eckenfels
> wrote:
>
> > Hello,
> >
> > Not exactly sure what work you mean and I fully agree that using a core
> > model should still be the API for plugins and extensions to work with,
> > however specifically the consumer POM integrates with so many external
> > ecosystems, I would expect it to be defined in terms of XML Schema with
> > explicite semantic (and the inherent compatibility with exiting POMs).
> >
> > Gruss
> > Bernd
> > --
> > http://bernd.eckenfels.net
> > ________________________________
> > Von: Matthieu BROUILLARD
> > Gesendet: Thursday, December 31, 2020 3:19:09 PM
> > An: dev@maven.apache.org
> > Betreff: maven 4.0.0 new XML stuff
> >
> > Hello all,
> >
> > regarding the active work occurring for maven 4.0.0 I noticed the
> > introduction of a lot of new stuff around SAX parsing & filtering.
> > I am wondering if that means that it was decided that the input format of
> > maven projects will be XML forever meaning probably, among others, the
> end
> > of polyglot extensions.
> > Could you explain such a move (or point to rationals/documents) and why
> you
> > did not leverage working on the in memory object model allowing
> > extensions/plugins to contribute/hook in the chain of building the
> BuildPOM
> > & ConsumePOM? In the past I really thought that this move to 'Build vs
> > Consumer' POM would make clear separations between the input format of
> > descriptors and the core system but I perhaps misunderstood things.
> >
> > Also, are there plans regarding the future of core extensions?
> > With core extensions it was possible to hook into the POM model loading
> and
> > do transformations to do dynamic changes but by working on the XML
> directly
> > I see a shift (if not red stop) in this contribution/delegation
> mechanism.
> >
> > Thanks for your time & answers.
> >
> > Matthieu Brouillard
> >
>

Re: maven 4.0.0 new XML stuff

Posted by Robert Scholte <rf...@apache.org>.
I've made a recording[1] about it, which hopefully answers most questions.

Robert

[1] https://youtu.be/KDAmlNKZJto

On 31-12-2020 16:18:57, Matthieu Brouillard <ma...@brouillard.fr> wrote:
> Not exactly sure what work you mean
everything related to maven-xml: Build/ConsumerPomXMLFilterxxx,
Build/ConsumerModelSourcexxxx and the transformer stuff.
Especially, when looking at classes like CiFriendlyXMLFilter, I would have
thought that such things could have been done elsewhere, working on the
object model (not on the XML stuff) especially for the BuildPom part.

> however specifically the consumer POM integrates with so many external
ecosystems
We're aligned here, this has to be stable and well defined by a schema.

Matthieu

On Thu, Dec 31, 2020 at 3:59 PM Bernd Eckenfels
wrote:

> Hello,
>
> Not exactly sure what work you mean and I fully agree that using a core
> model should still be the API for plugins and extensions to work with,
> however specifically the consumer POM integrates with so many external
> ecosystems, I would expect it to be defined in terms of XML Schema with
> explicite semantic (and the inherent compatibility with exiting POMs).
>
> Gruss
> Bernd
> --
> http://bernd.eckenfels.net
> ________________________________
> Von: Matthieu BROUILLARD
> Gesendet: Thursday, December 31, 2020 3:19:09 PM
> An: dev@maven.apache.org
> Betreff: maven 4.0.0 new XML stuff
>
> Hello all,
>
> regarding the active work occurring for maven 4.0.0 I noticed the
> introduction of a lot of new stuff around SAX parsing & filtering.
> I am wondering if that means that it was decided that the input format of
> maven projects will be XML forever meaning probably, among others, the end
> of polyglot extensions.
> Could you explain such a move (or point to rationals/documents) and why you
> did not leverage working on the in memory object model allowing
> extensions/plugins to contribute/hook in the chain of building the BuildPOM
> & ConsumePOM? In the past I really thought that this move to 'Build vs
> Consumer' POM would make clear separations between the input format of
> descriptors and the core system but I perhaps misunderstood things.
>
> Also, are there plans regarding the future of core extensions?
> With core extensions it was possible to hook into the POM model loading and
> do transformations to do dynamic changes but by working on the XML directly
> I see a shift (if not red stop) in this contribution/delegation mechanism.
>
> Thanks for your time & answers.
>
> Matthieu Brouillard
>

Re: maven 4.0.0 new XML stuff

Posted by Matthieu Brouillard <ma...@brouillard.fr>.
> Not exactly sure what work you mean
everything related to maven-xml: Build/ConsumerPomXMLFilterxxx,
Build/ConsumerModelSourcexxxx and the transformer stuff.
Especially, when looking at classes like CiFriendlyXMLFilter, I would have
thought that such things could have been done elsewhere, working on the
object model (not on the XML stuff) especially for the BuildPom part.

> however specifically the consumer POM integrates with so many external
ecosystems
We're aligned here, this has to be stable and well defined by a schema.

Matthieu

On Thu, Dec 31, 2020 at 3:59 PM Bernd Eckenfels <ec...@zusammenkunft.net>
wrote:

> Hello,
>
> Not exactly sure what work you mean and I fully agree that using a core
> model should still be the API for plugins and extensions to work with,
> however specifically the consumer POM integrates with so many external
> ecosystems, I would expect it to be defined in terms of XML Schema with
> explicite semantic (and the inherent compatibility with exiting POMs).
>
> Gruss
> Bernd
> --
> http://bernd.eckenfels.net
> ________________________________
> Von: Matthieu BROUILLARD <ma...@brouillard.fr>
> Gesendet: Thursday, December 31, 2020 3:19:09 PM
> An: dev@maven.apache.org <de...@maven.apache.org>
> Betreff: maven 4.0.0 new XML stuff
>
> Hello all,
>
> regarding the active work occurring for maven 4.0.0 I noticed the
> introduction of a lot of new stuff around SAX parsing & filtering.
> I am wondering if that means that it was decided that the input format of
> maven projects will be XML forever meaning probably, among others, the end
> of polyglot extensions.
> Could you explain such a move (or point to rationals/documents) and why you
> did not leverage working on the in memory object model allowing
> extensions/plugins to contribute/hook in the chain of building the BuildPOM
> & ConsumePOM? In the past I really thought that this move to 'Build vs
> Consumer' POM would make clear separations between the input format of
> descriptors and the core system but I perhaps misunderstood things.
>
> Also, are there plans regarding the future of core extensions?
> With core extensions it was possible to hook into the POM model loading and
> do transformations to do dynamic changes but by working on the XML directly
> I see a shift (if not red stop) in this contribution/delegation mechanism.
>
> Thanks for your time & answers.
>
> Matthieu Brouillard
>

Re: maven 4.0.0 new XML stuff

Posted by Bernd Eckenfels <ec...@zusammenkunft.net>.
Hello,

Not exactly sure what work you mean and I fully agree that using a core model should still be the API for plugins and extensions to work with, however specifically the consumer POM integrates with so many external ecosystems, I would expect it to be defined in terms of XML Schema with explicite semantic (and the inherent compatibility with exiting POMs).

Gruss
Bernd
--
http://bernd.eckenfels.net
________________________________
Von: Matthieu BROUILLARD <ma...@brouillard.fr>
Gesendet: Thursday, December 31, 2020 3:19:09 PM
An: dev@maven.apache.org <de...@maven.apache.org>
Betreff: maven 4.0.0 new XML stuff

Hello all,

regarding the active work occurring for maven 4.0.0 I noticed the
introduction of a lot of new stuff around SAX parsing & filtering.
I am wondering if that means that it was decided that the input format of
maven projects will be XML forever meaning probably, among others, the end
of polyglot extensions.
Could you explain such a move (or point to rationals/documents) and why you
did not leverage working on the in memory object model allowing
extensions/plugins to contribute/hook in the chain of building the BuildPOM
& ConsumePOM? In the past I really thought that this move to 'Build vs
Consumer' POM would make clear separations between the input format of
descriptors and the core system but I perhaps misunderstood things.

Also, are there plans regarding the future of core extensions?
With core extensions it was possible to hook into the POM model loading and
do transformations to do dynamic changes but by working on the XML directly
I see a shift (if not red stop) in this contribution/delegation mechanism.

Thanks for your time & answers.

Matthieu Brouillard