You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@maven.apache.org by Gary Gregory <ga...@gmail.com> on 2020/07/10 14:44:26 UTC

Reproducible builds, jars, bundles

Hi All,

Over at Apache Commons, we are looking at creating reproducible builds via:

https://github.com/apache/commons-parent/pull/9

As discussed in that thread, a bundle's "Include-Resource" manifest header
has a value with file system elements in different orders depending on file
system used to generate the jar, IOW, not bit by bit reproducible.

Folks in that discussion and in another here
https://github.com/apache/commons-lang/pull/578, suggested that the issue
might reside in Maven relying on the file system order and not in
maven-bundle-plugin or the bnd library as initially considered.

May you please provide any guidance?

Thank you,
Gary

Re: Reproducible builds, jars, bundles

Posted by Hervé BOUTEMY <he...@free.fr>.
yes, it has to be sorted at plugin level
And even more precisely at each goal level: it seems only some goal produce 
non-reproducible content, or even only some options of some goals

I Felix could produce a doc on how to configure for Reproducible Builds, that 
would be awesome, since this plugin is one of the most complex to configure: 
I'd happily update to link to that documentation in the table from 
https://maven.apache.org/guides/mini/guide-reproducible-builds.html

Regards,

Hervé

Le vendredi 10 juillet 2020, 22:20:39 CEST Romain Manni-Bucau a écrit :
> So maybe ask on felix@ instead of maven@ - that said pretty sure you can
> sort it, one fun hack to test can be to use maven shade with a custom
> transformer to do it.
> 
> Le ven. 10 juil. 2020 à 21:29, Gary Gregory <ga...@gmail.com> a
> 
> écrit :
> > Romain,
> > 
> > Yeah, I found the same code fragment but it is not clear to me if keeping
> > that set sorted would have unintended consequences.
> > 
> > I pointed folks in the PRs to this thread so I am hopping a plugin
> > maintainer will opine here.
> > 
> > Gary
> > 
> > On Fri, Jul 10, 2020, 10:53 Romain Manni-Bucau <rm...@gmail.com>
> > 
> > wrote:
> > > Hi Gary,
> > > 
> > > Maybe I misunderstood something but isn't it that felix bundle plugin
> > 
> > does
> > 
> > > not sort the maven resources ([1]?
> > > 
> > > [1]
> > 
> > https://github.com/apache/felix-dev/blob/7f4d31b384d9d83c772680a8627df18ff
> > 078eaa4/tools/maven-bundle-plugin/src/main/java/org/apache/felix/bundleplu
> > gin/BundlePlugin.java#L2036> 
> > > Romain Manni-Bucau
> > > @rmannibucau <https://twitter.com/rmannibucau> |  Blog
> > > <https://rmannibucau.metawerx.net/> | Old Blog
> > > <http://rmannibucau.wordpress.com> | Github <
> > > https://github.com/rmannibucau> |
> > > LinkedIn <https://www.linkedin.com/in/rmannibucau> | Book
> > > <
> > 
> > https://www.packtpub.com/application-development/java-ee-8-high-performanc
> > e
> > 
> > > Le ven. 10 juil. 2020 à 16:44, Gary Gregory <ga...@gmail.com> a
> > > 
> > > écrit :
> > > > Hi All,
> > > > 
> > > > Over at Apache Commons, we are looking at creating reproducible builds
> > > 
> > > via:
> > > > https://github.com/apache/commons-parent/pull/9
> > > > 
> > > > As discussed in that thread, a bundle's "Include-Resource" manifest
> > > 
> > > header
> > > 
> > > > has a value with file system elements in different orders depending on
> > > 
> > > file
> > > 
> > > > system used to generate the jar, IOW, not bit by bit reproducible.
> > > > 
> > > > Folks in that discussion and in another here
> > > > https://github.com/apache/commons-lang/pull/578, suggested that the
> > > 
> > > issue
> > > 
> > > > might reside in Maven relying on the file system order and not in
> > > > maven-bundle-plugin or the bnd library as initially considered.
> > > > 
> > > > May you please provide any guidance?
> > > > 
> > > > Thank you,
> > > > Gary





---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org


Re: Reproducible builds, jars, bundles

Posted by Romain Manni-Bucau <rm...@gmail.com>.
So maybe ask on felix@ instead of maven@ - that said pretty sure you can
sort it, one fun hack to test can be to use maven shade with a custom
transformer to do it.

Le ven. 10 juil. 2020 à 21:29, Gary Gregory <ga...@gmail.com> a
écrit :

> Romain,
>
> Yeah, I found the same code fragment but it is not clear to me if keeping
> that set sorted would have unintended consequences.
>
> I pointed folks in the PRs to this thread so I am hopping a plugin
> maintainer will opine here.
>
> Gary
>
> On Fri, Jul 10, 2020, 10:53 Romain Manni-Bucau <rm...@gmail.com>
> wrote:
>
> > Hi Gary,
> >
> > Maybe I misunderstood something but isn't it that felix bundle plugin
> does
> > not sort the maven resources ([1]?
> >
> > [1]
> >
> >
> https://github.com/apache/felix-dev/blob/7f4d31b384d9d83c772680a8627df18ff078eaa4/tools/maven-bundle-plugin/src/main/java/org/apache/felix/bundleplugin/BundlePlugin.java#L2036
> >
> > Romain Manni-Bucau
> > @rmannibucau <https://twitter.com/rmannibucau> |  Blog
> > <https://rmannibucau.metawerx.net/> | Old Blog
> > <http://rmannibucau.wordpress.com> | Github <
> > https://github.com/rmannibucau> |
> > LinkedIn <https://www.linkedin.com/in/rmannibucau> | Book
> > <
> >
> https://www.packtpub.com/application-development/java-ee-8-high-performance
> > >
> >
> >
> > Le ven. 10 juil. 2020 à 16:44, Gary Gregory <ga...@gmail.com> a
> > écrit :
> >
> > > Hi All,
> > >
> > > Over at Apache Commons, we are looking at creating reproducible builds
> > via:
> > >
> > > https://github.com/apache/commons-parent/pull/9
> > >
> > > As discussed in that thread, a bundle's "Include-Resource" manifest
> > header
> > > has a value with file system elements in different orders depending on
> > file
> > > system used to generate the jar, IOW, not bit by bit reproducible.
> > >
> > > Folks in that discussion and in another here
> > > https://github.com/apache/commons-lang/pull/578, suggested that the
> > issue
> > > might reside in Maven relying on the file system order and not in
> > > maven-bundle-plugin or the bnd library as initially considered.
> > >
> > > May you please provide any guidance?
> > >
> > > Thank you,
> > > Gary
> > >
> >
>

Re: Reproducible builds, jars, bundles

Posted by Gary Gregory <ga...@gmail.com>.
Romain,

Yeah, I found the same code fragment but it is not clear to me if keeping
that set sorted would have unintended consequences.

I pointed folks in the PRs to this thread so I am hopping a plugin
maintainer will opine here.

Gary

On Fri, Jul 10, 2020, 10:53 Romain Manni-Bucau <rm...@gmail.com>
wrote:

> Hi Gary,
>
> Maybe I misunderstood something but isn't it that felix bundle plugin does
> not sort the maven resources ([1]?
>
> [1]
>
> https://github.com/apache/felix-dev/blob/7f4d31b384d9d83c772680a8627df18ff078eaa4/tools/maven-bundle-plugin/src/main/java/org/apache/felix/bundleplugin/BundlePlugin.java#L2036
>
> Romain Manni-Bucau
> @rmannibucau <https://twitter.com/rmannibucau> |  Blog
> <https://rmannibucau.metawerx.net/> | Old Blog
> <http://rmannibucau.wordpress.com> | Github <
> https://github.com/rmannibucau> |
> LinkedIn <https://www.linkedin.com/in/rmannibucau> | Book
> <
> https://www.packtpub.com/application-development/java-ee-8-high-performance
> >
>
>
> Le ven. 10 juil. 2020 à 16:44, Gary Gregory <ga...@gmail.com> a
> écrit :
>
> > Hi All,
> >
> > Over at Apache Commons, we are looking at creating reproducible builds
> via:
> >
> > https://github.com/apache/commons-parent/pull/9
> >
> > As discussed in that thread, a bundle's "Include-Resource" manifest
> header
> > has a value with file system elements in different orders depending on
> file
> > system used to generate the jar, IOW, not bit by bit reproducible.
> >
> > Folks in that discussion and in another here
> > https://github.com/apache/commons-lang/pull/578, suggested that the
> issue
> > might reside in Maven relying on the file system order and not in
> > maven-bundle-plugin or the bnd library as initially considered.
> >
> > May you please provide any guidance?
> >
> > Thank you,
> > Gary
> >
>

Re: Reproducible builds, jars, bundles

Posted by Romain Manni-Bucau <rm...@gmail.com>.
Hi Gary,

Maybe I misunderstood something but isn't it that felix bundle plugin does
not sort the maven resources ([1]?

[1]
https://github.com/apache/felix-dev/blob/7f4d31b384d9d83c772680a8627df18ff078eaa4/tools/maven-bundle-plugin/src/main/java/org/apache/felix/bundleplugin/BundlePlugin.java#L2036

Romain Manni-Bucau
@rmannibucau <https://twitter.com/rmannibucau> |  Blog
<https://rmannibucau.metawerx.net/> | Old Blog
<http://rmannibucau.wordpress.com> | Github <https://github.com/rmannibucau> |
LinkedIn <https://www.linkedin.com/in/rmannibucau> | Book
<https://www.packtpub.com/application-development/java-ee-8-high-performance>


Le ven. 10 juil. 2020 à 16:44, Gary Gregory <ga...@gmail.com> a
écrit :

> Hi All,
>
> Over at Apache Commons, we are looking at creating reproducible builds via:
>
> https://github.com/apache/commons-parent/pull/9
>
> As discussed in that thread, a bundle's "Include-Resource" manifest header
> has a value with file system elements in different orders depending on file
> system used to generate the jar, IOW, not bit by bit reproducible.
>
> Folks in that discussion and in another here
> https://github.com/apache/commons-lang/pull/578, suggested that the issue
> might reside in Maven relying on the file system order and not in
> maven-bundle-plugin or the bnd library as initially considered.
>
> May you please provide any guidance?
>
> Thank you,
> Gary
>