You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@commons.apache.org by Gilles Sadowski <gi...@gmail.com> on 2022/01/18 00:48:32 UTC

[Math][All] "dist-archive" module

Hi.

Is the "dist-archive" module/directory still in use.
I noticed that the command
  $ cd dist-archive && mvn assembly::single
(as documented e.g. in [1]) generates "bin" archives (".tar.gz"
and ".zip" files) that only contain the NOTICE and LICENCE
files).  The "src" seem OK.
However, I also see that ".tar.gz" and ".zip" archive files[2] are
generated under each module's "target" directory that indeed
contain the expected JAR files.
I thus wonder whether the "dist-archive" is still in use.

Thanks,
Gilles

[1] https://gitbox.apache.org/repos/asf?p=commons-numbers.git;a=blob;f=doc/release/release.howto.txt;h=284edebded17a6047b1cf8b6753d2003233ed87d;hb=HEAD#l124
[2] Those archive files also include unexpected "apidocs" directories.

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org

Re: [Math][All] "dist-archive" module

Posted by Gilles Sadowski <gi...@gmail.com>.
Hello Alex.

Thanks a lot for the fixes.

> [...]
>
> So there are 2 issues:
>
> 1. The release dry run cannot complete as it fails on detach-distributions.
> 2. The site build of sofm module overrwrites some pom files

I'll try to find what is different here from similar sub-modules in [RNG].

Regards,
Gilles

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org

Re: [Math][All] "dist-archive" module

Posted by Gilles Sadowski <gi...@gmail.com>.
Le mer. 19 janv. 2022 à 01:16, Alex Herbert <al...@gmail.com> a écrit :
>
> On Tue, 18 Jan 2022 at 23:02, Gilles Sadowski <gi...@gmail.com> wrote:
> >
> > Hi.
> >
> > Commit d81b5e921e17278510b024b83258b7bc1c773d99 actually causes
> > the build to fail on Jenkins...[1]
> >
> > Regards,
> > Gilles
> >
> > [1] https://ci-builds.apache.org/job/Commons/job/commons-math/213/
>
> Jenkins runs using the release profile using:
>
> mvn -e -V clean deploy --batch-mode -Dgpg.skip -Prelease

I've changed the arg list to
  -e -V clean deploy --batch-mode -Dgpg.skip

I don't recall why it used the "release" profile.

>
> So there is definitely an issue with the dist-archive module in the
> newly introduced release profile. I've pushed a change to the assembly
> files in the dist-archive module. The configuration should now mirror
> that in RNG.

As it should be, at least for all math-related components (IMO).

> If this does not work then the release profile in the
> parent pom should be disabled until this can be fixed.

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org

Re: [Math][All] "dist-archive" module

Posted by Alex Herbert <al...@gmail.com>.
On Tue, 18 Jan 2022 at 23:02, Gilles Sadowski <gi...@gmail.com> wrote:
>
> Hi.
>
> Commit d81b5e921e17278510b024b83258b7bc1c773d99 actually causes
> the build to fail on Jenkins...[1]
>
> Regards,
> Gilles
>
> [1] https://ci-builds.apache.org/job/Commons/job/commons-math/213/

Jenkins runs using the release profile using:

mvn -e -V clean deploy --batch-mode -Dgpg.skip -Prelease

So there is definitely an issue with the dist-archive module in the
newly introduced release profile. I've pushed a change to the assembly
files in the dist-archive module. The configuration should now mirror
that in RNG. If this does not work then the release profile in the
parent pom should be disabled until this can be fixed.

Alex


>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org

Re: [Math][All] "dist-archive" module

Posted by Gilles Sadowski <gi...@gmail.com>.
Hi.

Commit d81b5e921e17278510b024b83258b7bc1c773d99 actually causes
the build to fail on Jenkins...[1]

Regards,
Gilles

[1] https://ci-builds.apache.org/job/Commons/job/commons-math/213/

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org

Re: [Math][All] "dist-archive" module

Posted by Alex Herbert <al...@gmail.com>.
On Tue, 18 Jan 2022 at 15:47, Gilles Sadowski <gi...@gmail.com> wrote:

<!-- SNIP -->

> Build is fine in [RNG].
>
> >
> > So this is a documentation issue. On the next release of a
> > multi-module project (e.g. Numbers) the section can be altered to
> > state what is expected to be performed by the release plugin. The
> > manual steps to do the same thing may not work and can be removed if
> > they are invalid. However they may work if 'mvn package' has been run
> > before the 'assembly:single' goal (I've not checked).
>
> The maven configuration in [Math] is somehow broken.
>
> When running
>   $ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64/ mvn
> -Duser.name="erans" -Dcommons.release.dryRun=true -Ptest-deploy
> -Prelease clean package site deploy
> the build is successful (when all "randomized" unit tests have passed).
> [But the "Prelease" switch did not seem to trigger anything
> related to "dist-archive".]

There is no release profile in the [Math] or [Numbers] pom. There is
in [RNG] and [Geometry]. Look for <id>release</id>.

In geometry the release profile includes the examples module only. In
RNG it includes the dist-archive module. So the command you listed
would not work for any project apart from RNG. I've not released any
of the other multi-module projects. Matt performed the last release of
Numbers and Geometry so maybe there is something missing from the
documented steps. Somehow he got it to work with a parent pom that
does not include the dist-archive module.

I think these multi-module projects now build the site separately
(because it includes the examples) and then stage them manually using
svn commands. So an actual release can just run the package goal from
the top level directory and then this from the dist-archive directory:

mvn -Duser.name="erans" -Dcommons.release.dryRun=true -Ptest-deploy
-Prelease deploy

i.e. only the deploy goal. This should then pick up all the packages
from the other modules and copy them somewhere local in a dry run of
deploying them to nexus and the commons dev dist area (for the binary
distributions).

I did update the parent pom to have a release profile. I then found
that the dist-archive module pom did not have the correct artifact
Ids. They should have math4 in the name:

commons-math-legacy-exception -> commons-math4-legacy-exception

I've corrected this.

> However, the above command unexpectedly created/modified files

Not sure how that happened. I tried to reproduce on my machine but did
not have spurious files when I build without the site and skipped the
tests:

mvn -Duser.name=aherbert -Dcommons.release.dryRun=true -Ptest-deploy
-Prelease clean package deploy -DskipTests

Note: I have JAVA_HOME set in my environment. When all the modules
have built their packages I can then run the following to get the
distribution archives:

cd dist-archive && mvn assembly:single

However I still get an issue with the full dry run when I added the
dist-archive to a release profile. It fails with this message in the
dist-archive module:

[ERROR] Failed to execute goal
org.apache.commons:commons-release-plugin:1.7:detach-distributions
(detatch-distributions) on project
commons-math: Execution detatch-distributions of goal
org.apache.commons:commons-release-plugin:1.7:detach-distributions
failed.: UnsupportedOperationException -> [Help 1]

I am not sure what is happening here. Running with -X does not provide
any useful error information. I do not know what the
detach-distributions goal is trying to do.

I tried again adding 'site site:stage' to check if this is required
and got the same error. However this time I had the modified pom files
as you described. So something in the site target is running for the
sofm module that overwrites the pom files as you previously listed.

When building the site I noted that the pom requires the fix to not do
the svn checkout of the site. So I added this too.

So there are 2 issues:

1. The release dry run cannot complete as it fails on detach-distributions.
2. The site build of sofm module overrwrites some pom files

I have not verified if issue 2 only occurs during a release dry run.

Alex

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org

Re: [Math][All] "dist-archive" module

Posted by Gilles Sadowski <gi...@gmail.com>.
Hello.

Le mar. 18 janv. 2022 à 12:12, Alex Herbert <al...@gmail.com> a écrit :
>
> On Tue, 18 Jan 2022 at 00:48, Gilles Sadowski <gi...@gmail.com> wrote:
> >
> > Hi.
> >
> > Is the "dist-archive" module/directory still in use.
> > I noticed that the command
> >   $ cd dist-archive && mvn assembly::single
> > (as documented e.g. in [1]) generates "bin" archives (".tar.gz"
> > and ".zip" files) that only contain the NOTICE and LICENCE
> > files).  The "src" seem OK.
> > However, I also see that ".tar.gz" and ".zip" archive files[2] are
> > generated under each module's "target" directory that indeed
> > contain the expected JAR files.
> > I thus wonder whether the "dist-archive" is still in use.
>
> The dist-archive module is used in the release profile. It creates the
> correct assembly archives after the package phase has been run as part
> of the release phase. This is used by the Commons release plugin.
>
> The assembly:single goal is not explicitly used any more. In the
> release.howto.txt document for RNG this is all under section 12. It
> states that this section is now all performed by the Commons release
> plugin. The latest RNG guide has the commands that I used for release
> 1.4 (Sept 2021) to verify the release plugin did correctly perform the
> tasks in section 12.

Build is fine in [RNG].

>
> So this is a documentation issue. On the next release of a
> multi-module project (e.g. Numbers) the section can be altered to
> state what is expected to be performed by the release plugin. The
> manual steps to do the same thing may not work and can be removed if
> they are invalid. However they may work if 'mvn package' has been run
> before the 'assembly:single' goal (I've not checked).

The maven configuration in [Math] is somehow broken.

When running
  $ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64/ mvn
-Duser.name="erans" -Dcommons.release.dryRun=true -Ptest-deploy
-Prelease clean package site deploy
the build is successful (when all "randomized" unit tests have passed).
[But the "Prelease" switch did not seem to trigger anything
related to "dist-archive".]

However, the above command unexpectedly created/modified files:

$ git status .
On branch test-release
Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git checkout -- <file>..." to discard changes in working directory)

        modified:   commons-math-examples/examples-sofm/pom.xml
        modified:   commons-math-examples/pom.xml

Untracked files:
  (use "git add <file>..." to include in what will be committed)

        commons-math-examples/examples-sofm/dependency-reduced-pom.xml

Strangely, POM files were overwritten with JAR contents:
$ file commons-math-examples/examples-sofm/pom.xml
commons-math-examples/pom.xml
commons-math-examples/examples-sofm/dependency-reduced-pom.xml
commons-math-examples/examples-sofm/pom.xml:                    Java
archive data (JAR)
commons-math-examples/pom.xml:                                  Java
archive data (JAR)
commons-math-examples/examples-sofm/dependency-reduced-pom.xml: XML
1.0 document, ASCII text, with CRLF line terminators

I don't know enough of maven in order to understand why the POM
of each component has to be so complex.
Could a higher-level POM be defined for all components that agree
on a project's layout and release process?
If not, shall I just delete all [Math] POMs and copy/paste from [RNG]?

Thanks for your help,
Gilles

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org

Re: [Math][All] "dist-archive" module

Posted by Alex Herbert <al...@gmail.com>.
On Tue, 18 Jan 2022 at 00:48, Gilles Sadowski <gi...@gmail.com> wrote:
>
> Hi.
>
> Is the "dist-archive" module/directory still in use.
> I noticed that the command
>   $ cd dist-archive && mvn assembly::single
> (as documented e.g. in [1]) generates "bin" archives (".tar.gz"
> and ".zip" files) that only contain the NOTICE and LICENCE
> files).  The "src" seem OK.
> However, I also see that ".tar.gz" and ".zip" archive files[2] are
> generated under each module's "target" directory that indeed
> contain the expected JAR files.
> I thus wonder whether the "dist-archive" is still in use.

The dist-archive module is used in the release profile. It creates the
correct assembly archives after the package phase has been run as part
of the release phase. This is used by the Commons release plugin.

The assembly:single goal is not explicitly used any more. In the
release.howto.txt document for RNG this is all under section 12. It
states that this section is now all performed by the Commons release
plugin. The latest RNG guide has the commands that I used for release
1.4 (Sept 2021) to verify the release plugin did correctly perform the
tasks in section 12.

So this is a documentation issue. On the next release of a
multi-module project (e.g. Numbers) the section can be altered to
state what is expected to be performed by the release plugin. The
manual steps to do the same thing may not work and can be removed if
they are invalid. However they may work if 'mvn package' has been run
before the 'assembly:single' goal (I've not checked).

Alex

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org