You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@maven.apache.org by Hervé BOUTEMY <he...@free.fr> on 2019/09/28 15:55:24 UTC
[DISCUSS] configuration for Reproducible Builds
Achieving Reproducible Builds require only one parameter: plugins that create
zip or tar archives require a fixed timestamp for entries
Putting that parameter as a pom property with a well known name and value
format permits to share the configuration between every packaging plugin.
This also has the advantage that child poms will inherit from parent value,
and eventually override.
The question is: *what property name and what value format should we keep?*
For the PoC, I chose to extrapolate from a convention from Reproducible Builds
project, which is very Linux-oriented: SOURCE_DATE_EPOCH environment variable,
that I transformed into source-date-epoch property name, keeping the "date +
%s" value
https://reproducible-builds.org/docs/source-date-epoch/
But I feel we can do a more user-readable solution by choosing another name
and format, like "reproducible-build-timestamp" with an ISO-8601 combined date
and time representation
WDYT? Any other idea?
Regards,
Hervé
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: [DISCUSS] configuration for Reproducible Builds
Posted by Hervé BOUTEMY <he...@free.fr>.
very good question: this was a key question to me, that lead me to the PoC to
test and see
this PoC showed a basic fact: a POM will inherit the value from his parent
Then once a parent POM has a "reproducibility" timestamp, child POMs inherit
reproducible configuration, and can eventually override value
I imagine that we'll do a maven-release-plugin feature to update the value to
match release timestamp
But we can start by updating the value by hand if we want to have a value that
has a better meaning that the inherited one
Regards,
Hervé
Le samedi 28 septembre 2019, 22:52:04 CEST Enrico Olivelli a écrit :
> Hervé
> When will you set this value? During release:prepare and modify the pom?
>
> Enrico
>
> Il sab 28 set 2019, 17:55 Hervé BOUTEMY <he...@free.fr> ha scritto:
> > Achieving Reproducible Builds require only one parameter: plugins that
> > create
> > zip or tar archives require a fixed timestamp for entries
> >
> > Putting that parameter as a pom property with a well known name and value
> > format permits to share the configuration between every packaging plugin.
> > This also has the advantage that child poms will inherit from parent
> > value,
> > and eventually override.
> >
> > The question is: *what property name and what value format should we
> > keep?*
> >
> > For the PoC, I chose to extrapolate from a convention from Reproducible
> > Builds
> > project, which is very Linux-oriented: SOURCE_DATE_EPOCH environment
> > variable,
> > that I transformed into source-date-epoch property name, keeping the "date
> > +
> > %s" value
> > https://reproducible-builds.org/docs/source-date-epoch/
> >
> >
> > But I feel we can do a more user-readable solution by choosing another
> > name
> > and format, like "reproducible-build-timestamp" with an ISO-8601 combined
> > date
> > and time representation
> >
> >
> > WDYT? Any other idea?
> >
> > Regards,
> >
> > Hervé
> >
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> > For additional commands, e-mail: dev-help@maven.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: [DISCUSS] configuration for Reproducible Builds
Posted by Enrico Olivelli <eo...@gmail.com>.
Hervé
When will you set this value? During release:prepare and modify the pom?
Enrico
Il sab 28 set 2019, 17:55 Hervé BOUTEMY <he...@free.fr> ha scritto:
> Achieving Reproducible Builds require only one parameter: plugins that
> create
> zip or tar archives require a fixed timestamp for entries
>
> Putting that parameter as a pom property with a well known name and value
> format permits to share the configuration between every packaging plugin.
> This also has the advantage that child poms will inherit from parent
> value,
> and eventually override.
>
> The question is: *what property name and what value format should we keep?*
>
> For the PoC, I chose to extrapolate from a convention from Reproducible
> Builds
> project, which is very Linux-oriented: SOURCE_DATE_EPOCH environment
> variable,
> that I transformed into source-date-epoch property name, keeping the "date
> +
> %s" value
> https://reproducible-builds.org/docs/source-date-epoch/
>
>
> But I feel we can do a more user-readable solution by choosing another
> name
> and format, like "reproducible-build-timestamp" with an ISO-8601 combined
> date
> and time representation
>
>
> WDYT? Any other idea?
>
> Regards,
>
> Hervé
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> For additional commands, e-mail: dev-help@maven.apache.org
>
>
Re: [DISCUSS] configuration for Reproducible Builds
Posted by Hervé BOUTEMY <he...@free.fr>.
Le dimanche 29 septembre 2019, 16:54:04 CEST Emmanuel Bourg a écrit :
> Le 28/09/2019 à 17:55, Hervé BOUTEMY a écrit :
> > Putting that parameter as a pom property with a well known name and value
> > format permits to share the configuration between every packaging plugin.
> > This also has the advantage that child poms will inherit from parent
> > value,
> > and eventually override.
>
> It seems a bit odd to me to tie the build timestamp to the pom. The fact
> that it could be inherited is disturbing, i.e. if you forget to override
> it in a subproject your build time will be wrong.
what does "wrong" or "right" mean?
I know some solution for reproducible builds that blindly put zip entries
timestamp to 1970-01-01
>
> > WDYT? Any other idea?
>
> I thought the timestamp would rather go to a separate file deployed
> along the pom and capturing the build environment. What Maven needs then
> is a command line parameter to force a specific build time (and/or
> support for the SOURCE_DATE_EPOCH environment variable).
a separate file? oh no!!!
please try Maven core branch reproducible and you'll see Maven native
Reproducible Build in action
https://github.com/apache/maven/tree/reproducible
The timestamp was defined 4 days ago, and will remain defined for as long as
nobody change it in git
Regards,
Hervé
>
> Emmanuel Bourg
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> For additional commands, e-mail: dev-help@maven.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: [DISCUSS] configuration for Reproducible Builds
Posted by Emmanuel Bourg <eb...@apache.org>.
Le 28/09/2019 à 17:55, Hervé BOUTEMY a écrit :
> Putting that parameter as a pom property with a well known name and value
> format permits to share the configuration between every packaging plugin.
> This also has the advantage that child poms will inherit from parent value,
> and eventually override.
It seems a bit odd to me to tie the build timestamp to the pom. The fact
that it could be inherited is disturbing, i.e. if you forget to override
it in a subproject your build time will be wrong.
> WDYT? Any other idea?
I thought the timestamp would rather go to a separate file deployed
along the pom and capturing the build environment. What Maven needs then
is a command line parameter to force a specific build time (and/or
support for the SOURCE_DATE_EPOCH environment variable).
Emmanuel Bourg
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: [DISCUSS] configuration for Reproducible Builds
Posted by Hervé BOUTEMY <he...@free.fr>.
Le dimanche 29 septembre 2019, 12:41:17 CEST Enrico Olivelli a écrit :
> Il dom 29 set 2019, 12:16 Robert Scholte <rf...@apache.org> ha scritto:
> > I would think that all project.* properties represent the pom.xml and are
> > immutable. To be more precise: the same pom.xml should effectively stay
> > the same with every build.
> > Instead this seems more related the (maven)session, right?
>
> If I understand correctly the value f this property is to be committed in
> the source code.
> Some thoughts:
> - it should default to 'now' sampled at the beginning of the session
which session? since there will be new sessions every now and then, when
someone builds
> - it can be overridden with a value writen in the pom or on a source file
> - it must be sampled and commited as final value during a 'release process'
IMHO, not *must* but *can* or *should*: it it stays at a past timestamp, there
won't be any big harm.
But if it can be the timestamp of the release process, it can be better for
people who look at zip entries timestamp (if anybody looks at it: do you look
at it?)
Regards,
Hervé
>
>
> Enrico
>
> > Robert
> >
> > On Sun, 29 Sep 2019 11:19:45 +0200, Hervé BOUTEMY <he...@free.fr>
> >
> > wrote:
> > > regarding the property name, I had an idea:
> > >
> > > why not do like we already did for ${project.build.sourceEncoding},
> >
> > ie.
> >
> > > mimic
> > > a future element in pom.xml, in build?
> > >
> > > could be project.build.timestamp?
> > >
> > > Le samedi 28 septembre 2019, 17:55:24 CEST Hervé BOUTEMY a écrit :
> > >> Achieving Reproducible Builds require only one parameter: plugins that
> > >> create zip or tar archives require a fixed timestamp for entries
> > >>
> > >> Putting that parameter as a pom property with a well known name and
> > >> value
> > >> format permits to share the configuration between every packaging
> > >> plugin.
> > >> This also has the advantage that child poms will inherit from parent
> > >> value,
> > >> and eventually override.
> > >>
> > >> The question is: *what property name and what value format should we
> > >> keep?*
> > >>
> > >> For the PoC, I chose to extrapolate from a convention from Reproducible
> > >> Builds project, which is very Linux-oriented: SOURCE_DATE_EPOCH
> > >> environment
> > >> variable, that I transformed into source-date-epoch property name,
> > >> keeping
> > >> the "date + %s" value
> > >> https://reproducible-builds.org/docs/source-date-epoch/
> > >>
> > >>
> > >> But I feel we can do a more user-readable solution by choosing another
> > >> name
> > >> and format, like "reproducible-build-timestamp" with an ISO-8601
> > >> combined
> > >> date and time representation
> > >>
> > >>
> > >> WDYT? Any other idea?
> > >>
> > >> Regards,
> > >>
> > >> Hervé
> > >>
> > >>
> > >>
> > >> ---------------------------------------------------------------------
> > >> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> > >> For additional commands, e-mail: dev-help@maven.apache.org
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> > > For additional commands, e-mail: dev-help@maven.apache.org
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> > For additional commands, e-mail: dev-help@maven.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: [DISCUSS] configuration for Reproducible Builds
Posted by Hervé BOUTEMY <he...@free.fr>.
Le dimanche 29 septembre 2019, 16:31:35 CEST Robert Scholte a écrit :
> On Sun, 29 Sep 2019 12:41:17 +0200, Enrico Olivelli <eo...@gmail.com>
>
> wrote:
> > Il dom 29 set 2019, 12:16 Robert Scholte <rf...@apache.org> ha
> >
> > scritto:
> >> I would think that all project.* properties represent the pom.xml and
> >> are
> >> immutable. To be more precise: the same pom.xml should effectively stay
> >> the same with every build.
> >> Instead this seems more related the (maven)session, right?
> >
> > If I understand correctly the value f this property is to be committed in
> > the source code.
> > Some thoughts:
> > - it should default to 'now' sampled at the beginning of the session
> > - it can be overridden with a value writen in the pom or on a source file
> > - it must be sampled and commited as final value during a 'release
> > process'
>
> There is already a property with a similar name: maven.build.timestamp
> We need to keep in mind when thinking of a good name: which one is used
> for what?
>
> Reading the mail again, it looks like it is not about the timestamp of the
> build. It is more about setting the lastModified value (and maybe also
> created + opened?) of files inside the archive.
> I would prefer a name that clearly reflects that.
+1 you're right, need to find a better name
reproducibleTimestamp?
outputTimestamp?
>
> I don't like the idea of being able to set a property that starts with
> `project.`.
> Based on stack overflow people find it sometimes hard to understand, so we
> should be very clear about this:
> `project.` represents project-roottag of the pom.xml, and with this you
> can read any value, but you cannot SET a value.
yes, this is the objective of the choice: in the future, when we have Build vs
Consumer POM as discussed in parallel, my intent is to add an element to the
POM
= exactly like we'll do for project.build.sourceEncoding and
project.reporting.outputEncoding
>
> Is this value important for snapshots too?
the fact that there is a value is important, not the precise value itself.
And if we do care about the value, for SNAPSHOTs, any timestamp from the
previous release to the next one may make sense: I imagine the timestamp from
the last release would be a perfect value for the whole duration of a SNASPHOT
lifecycle
> If not, it shouldn't be too
> hard to adjust the maven-release-plugin for setting an explicit value.
then during the next release, having maven-release-plugin update the value
would make sense
Regards,
Hervé
>
> Robert
>
> [1]
> https://maven.apache.org/guides/introduction/introduction-to-the-pom.html#Av
> ailable_Variables
> > Enrico
> >
> >> Robert
> >>
> >> On Sun, 29 Sep 2019 11:19:45 +0200, Hervé BOUTEMY
> >> <he...@free.fr>
> >>
> >> wrote:
> >> > regarding the property name, I had an idea:
> >> >
> >> > why not do like we already did for ${project.build.sourceEncoding},
> >>
> >> ie.
> >>
> >> > mimic
> >> > a future element in pom.xml, in build?
> >> >
> >> > could be project.build.timestamp?
> >> >
> >> > Le samedi 28 septembre 2019, 17:55:24 CEST Hervé BOUTEMY a écrit :
> >> >> Achieving Reproducible Builds require only one parameter: plugins
> >>
> >> that
> >>
> >> >> create zip or tar archives require a fixed timestamp for entries
> >> >>
> >> >> Putting that parameter as a pom property with a well known name and
> >> >> value
> >> >> format permits to share the configuration between every packaging
> >> >> plugin.
> >> >> This also has the advantage that child poms will inherit from parent
> >> >> value,
> >> >> and eventually override.
> >> >>
> >> >> The question is: *what property name and what value format should we
> >> >> keep?*
> >> >>
> >> >> For the PoC, I chose to extrapolate from a convention from
> >>
> >> Reproducible
> >>
> >> >> Builds project, which is very Linux-oriented: SOURCE_DATE_EPOCH
> >> >> environment
> >> >> variable, that I transformed into source-date-epoch property name,
> >> >> keeping
> >> >> the "date + %s" value
> >> >> https://reproducible-builds.org/docs/source-date-epoch/
> >> >>
> >> >>
> >> >> But I feel we can do a more user-readable solution by choosing
> >>
> >> another
> >>
> >> >> name
> >> >> and format, like "reproducible-build-timestamp" with an ISO-8601
> >> >> combined
> >> >> date and time representation
> >> >>
> >> >>
> >> >> WDYT? Any other idea?
> >> >>
> >> >> Regards,
> >> >>
> >> >> Hervé
> >> >>
> >> >>
> >> >>
> >> >> ---------------------------------------------------------------------
> >> >> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> >> >> For additional commands, e-mail: dev-help@maven.apache.org
> >> >
> >> > ---------------------------------------------------------------------
> >> > To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> >> > For additional commands, e-mail: dev-help@maven.apache.org
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> >> For additional commands, e-mail: dev-help@maven.apache.org
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> For additional commands, e-mail: dev-help@maven.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: [DISCUSS] configuration for Reproducible Builds
Posted by Robert Scholte <rf...@apache.org>.
On Sun, 29 Sep 2019 12:41:17 +0200, Enrico Olivelli <eo...@gmail.com>
wrote:
> Il dom 29 set 2019, 12:16 Robert Scholte <rf...@apache.org> ha
> scritto:
>
>> I would think that all project.* properties represent the pom.xml and
>> are
>> immutable. To be more precise: the same pom.xml should effectively stay
>> the same with every build.
>> Instead this seems more related the (maven)session, right?
>>
>
> If I understand correctly the value f this property is to be committed in
> the source code.
> Some thoughts:
> - it should default to 'now' sampled at the beginning of the session
> - it can be overridden with a value writen in the pom or on a source file
> - it must be sampled and commited as final value during a 'release
> process'
>
There is already a property with a similar name: maven.build.timestamp
We need to keep in mind when thinking of a good name: which one is used
for what?
Reading the mail again, it looks like it is not about the timestamp of the
build. It is more about setting the lastModified value (and maybe also
created + opened?) of files inside the archive.
I would prefer a name that clearly reflects that.
I don't like the idea of being able to set a property that starts with
`project.`.
Based on stack overflow people find it sometimes hard to understand, so we
should be very clear about this:
`project.` represents project-roottag of the pom.xml, and with this you
can read any value, but you cannot SET a value.
Is this value important for snapshots too? If not, it shouldn't be too
hard to adjust the maven-release-plugin for setting an explicit value.
Robert
[1]
https://maven.apache.org/guides/introduction/introduction-to-the-pom.html#Available_Variables
>
> Enrico
>
>
>
>> Robert
>>
>> On Sun, 29 Sep 2019 11:19:45 +0200, Hervé BOUTEMY
>> <he...@free.fr>
>>
>> wrote:
>>
>> > regarding the property name, I had an idea:
>> >
>> > why not do like we already did for ${project.build.sourceEncoding},
>> ie.
>> > mimic
>> > a future element in pom.xml, in build?
>> >
>> > could be project.build.timestamp?
>> >
>> > Le samedi 28 septembre 2019, 17:55:24 CEST Hervé BOUTEMY a écrit :
>> >> Achieving Reproducible Builds require only one parameter: plugins
>> that
>> >> create zip or tar archives require a fixed timestamp for entries
>> >>
>> >> Putting that parameter as a pom property with a well known name and
>> >> value
>> >> format permits to share the configuration between every packaging
>> >> plugin.
>> >> This also has the advantage that child poms will inherit from parent
>> >> value,
>> >> and eventually override.
>> >>
>> >> The question is: *what property name and what value format should we
>> >> keep?*
>> >>
>> >> For the PoC, I chose to extrapolate from a convention from
>> Reproducible
>> >> Builds project, which is very Linux-oriented: SOURCE_DATE_EPOCH
>> >> environment
>> >> variable, that I transformed into source-date-epoch property name,
>> >> keeping
>> >> the "date + %s" value
>> >> https://reproducible-builds.org/docs/source-date-epoch/
>> >>
>> >>
>> >> But I feel we can do a more user-readable solution by choosing
>> another
>> >> name
>> >> and format, like "reproducible-build-timestamp" with an ISO-8601
>> >> combined
>> >> date and time representation
>> >>
>> >>
>> >> WDYT? Any other idea?
>> >>
>> >> Regards,
>> >>
>> >> Hervé
>> >>
>> >>
>> >>
>> >> ---------------------------------------------------------------------
>> >> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
>> >> For additional commands, e-mail: dev-help@maven.apache.org
>> >
>> >
>> >
>> >
>> >
>> > ---------------------------------------------------------------------
>> > To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
>> > For additional commands, e-mail: dev-help@maven.apache.org
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
>> For additional commands, e-mail: dev-help@maven.apache.org
>>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: [DISCUSS] configuration for Reproducible Builds
Posted by Enrico Olivelli <eo...@gmail.com>.
Il dom 29 set 2019, 12:16 Robert Scholte <rf...@apache.org> ha scritto:
> I would think that all project.* properties represent the pom.xml and are
> immutable. To be more precise: the same pom.xml should effectively stay
> the same with every build.
> Instead this seems more related the (maven)session, right?
>
If I understand correctly the value f this property is to be committed in
the source code.
Some thoughts:
- it should default to 'now' sampled at the beginning of the session
- it can be overridden with a value writen in the pom or on a source file
- it must be sampled and commited as final value during a 'release process'
Enrico
> Robert
>
> On Sun, 29 Sep 2019 11:19:45 +0200, Hervé BOUTEMY <he...@free.fr>
>
> wrote:
>
> > regarding the property name, I had an idea:
> >
> > why not do like we already did for ${project.build.sourceEncoding},
> ie.
> > mimic
> > a future element in pom.xml, in build?
> >
> > could be project.build.timestamp?
> >
> > Le samedi 28 septembre 2019, 17:55:24 CEST Hervé BOUTEMY a écrit :
> >> Achieving Reproducible Builds require only one parameter: plugins that
> >> create zip or tar archives require a fixed timestamp for entries
> >>
> >> Putting that parameter as a pom property with a well known name and
> >> value
> >> format permits to share the configuration between every packaging
> >> plugin.
> >> This also has the advantage that child poms will inherit from parent
> >> value,
> >> and eventually override.
> >>
> >> The question is: *what property name and what value format should we
> >> keep?*
> >>
> >> For the PoC, I chose to extrapolate from a convention from Reproducible
> >> Builds project, which is very Linux-oriented: SOURCE_DATE_EPOCH
> >> environment
> >> variable, that I transformed into source-date-epoch property name,
> >> keeping
> >> the "date + %s" value
> >> https://reproducible-builds.org/docs/source-date-epoch/
> >>
> >>
> >> But I feel we can do a more user-readable solution by choosing another
> >> name
> >> and format, like "reproducible-build-timestamp" with an ISO-8601
> >> combined
> >> date and time representation
> >>
> >>
> >> WDYT? Any other idea?
> >>
> >> Regards,
> >>
> >> Hervé
> >>
> >>
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> >> For additional commands, e-mail: dev-help@maven.apache.org
> >
> >
> >
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> > For additional commands, e-mail: dev-help@maven.apache.org
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> For additional commands, e-mail: dev-help@maven.apache.org
>
>
Re: [DISCUSS] configuration for Reproducible Builds
Posted by Hervé BOUTEMY <he...@free.fr>.
Le dimanche 29 septembre 2019, 12:16:48 CEST Robert Scholte a écrit :
> I would think that all project.* properties represent the pom.xml and are
> immutable. To be more precise: the same pom.xml should effectively stay
> the same with every build.
+1
> Instead this seems more related the (maven)session, right?
no
It has to be reproducible from build to build = that's the objective.
That's why a value has to be recorded, whatever the exact value is.
A consequence is that the value won't be the effective build time, since it has
to be the same value next month, next year, or next century.
It has to be recorded one day, then stay for posterity.
Ideally, it will be recorded during a release process: that's why I imagine
that we'll have maven-release-plugin do an automated update.
But it can stay to a point in time: who cares of the exact value? As I already
said, to avoid a debate, some Reproducible Builds solutions just use
1970-01-01T00:00:00Z as timestamp: basic, but it works
Regards,
Hervé
>
> Robert
>
> On Sun, 29 Sep 2019 11:19:45 +0200, Hervé BOUTEMY <he...@free.fr>
>
> wrote:
> > regarding the property name, I had an idea:
> >
> > why not do like we already did for ${project.build.sourceEncoding}, ie.
> > mimic
> > a future element in pom.xml, in build?
> >
> > could be project.build.timestamp?
> >
> > Le samedi 28 septembre 2019, 17:55:24 CEST Hervé BOUTEMY a écrit :
> >> Achieving Reproducible Builds require only one parameter: plugins that
> >> create zip or tar archives require a fixed timestamp for entries
> >>
> >> Putting that parameter as a pom property with a well known name and
> >> value
> >> format permits to share the configuration between every packaging
> >> plugin.
> >> This also has the advantage that child poms will inherit from parent
> >> value,
> >> and eventually override.
> >>
> >> The question is: *what property name and what value format should we
> >> keep?*
> >>
> >> For the PoC, I chose to extrapolate from a convention from Reproducible
> >> Builds project, which is very Linux-oriented: SOURCE_DATE_EPOCH
> >> environment
> >> variable, that I transformed into source-date-epoch property name,
> >> keeping
> >> the "date + %s" value
> >> https://reproducible-builds.org/docs/source-date-epoch/
> >>
> >>
> >> But I feel we can do a more user-readable solution by choosing another
> >> name
> >> and format, like "reproducible-build-timestamp" with an ISO-8601
> >> combined
> >> date and time representation
> >>
> >>
> >> WDYT? Any other idea?
> >>
> >> Regards,
> >>
> >> Hervé
> >>
> >>
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> >> For additional commands, e-mail: dev-help@maven.apache.org
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> > For additional commands, e-mail: dev-help@maven.apache.org
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> For additional commands, e-mail: dev-help@maven.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: [DISCUSS] configuration for Reproducible Builds
Posted by Robert Scholte <rf...@apache.org>.
I would think that all project.* properties represent the pom.xml and are
immutable. To be more precise: the same pom.xml should effectively stay
the same with every build.
Instead this seems more related the (maven)session, right?
Robert
On Sun, 29 Sep 2019 11:19:45 +0200, Hervé BOUTEMY <he...@free.fr>
wrote:
> regarding the property name, I had an idea:
>
> why not do like we already did for ${project.build.sourceEncoding}, ie.
> mimic
> a future element in pom.xml, in build?
>
> could be project.build.timestamp?
>
> Le samedi 28 septembre 2019, 17:55:24 CEST Hervé BOUTEMY a écrit :
>> Achieving Reproducible Builds require only one parameter: plugins that
>> create zip or tar archives require a fixed timestamp for entries
>>
>> Putting that parameter as a pom property with a well known name and
>> value
>> format permits to share the configuration between every packaging
>> plugin.
>> This also has the advantage that child poms will inherit from parent
>> value,
>> and eventually override.
>>
>> The question is: *what property name and what value format should we
>> keep?*
>>
>> For the PoC, I chose to extrapolate from a convention from Reproducible
>> Builds project, which is very Linux-oriented: SOURCE_DATE_EPOCH
>> environment
>> variable, that I transformed into source-date-epoch property name,
>> keeping
>> the "date + %s" value
>> https://reproducible-builds.org/docs/source-date-epoch/
>>
>>
>> But I feel we can do a more user-readable solution by choosing another
>> name
>> and format, like "reproducible-build-timestamp" with an ISO-8601
>> combined
>> date and time representation
>>
>>
>> WDYT? Any other idea?
>>
>> Regards,
>>
>> Hervé
>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
>> For additional commands, e-mail: dev-help@maven.apache.org
>
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> For additional commands, e-mail: dev-help@maven.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: [DISCUSS] configuration for Reproducible Builds
Posted by Tibor Digaňa <ti...@googlemail.com.INVALID>.
Let's not talk about technologies but talk about the use case because the
use case will uncover the purpose what target is in the background of this
request.
Romain, an therefore i was not firstly talking about concret SCM, but
Harald, and it means the SCM can be one of the implementations -
strategies. Then we will be closer to incremental build...
The problem is that "buildTimeStamp" means one purpose for you and
something else for other e.g. build-helper:timestamp-property.
Therefore it's better to list all usecases and we will see if we really
need build timestamp or another type of timestamp and what about
guarantees. Or this solution is only a glue code between POM and the User?
Just a question. The usecase would uncover all practicies with this
property, even if it is realistic and reliable.
T
On Sun, Sep 29, 2019 at 8:07 PM Romain Manni-Bucau <rm...@gmail.com>
wrote:
> scm does not work cause one common use case is to rebuild from source the
> same artifacts (debian rebuild from source AFAIK, even java apps)
> since scm can be "proxied", copied etc then the source can differ and
> therefore commits can be differents but the content can be the same
> this is why jib uses epoch+1s to enforce reproducibility.
> that said once you have the timestamp the code is the same so let's not
> block on that, worse case we would enable to plug a value resolver with a
> few default strategy.
> this is not the central part of the feature IMHO.
>
> Romain Manni-Bucau
> @rmannibucau <https://twitter.com/rmannibucau> | Blog
> <https://rmannibucau.metawerx.net/> | Old Blog
> <http://rmannibucau.wordpress.com> | Github <
> https://github.com/rmannibucau> |
> LinkedIn <https://www.linkedin.com/in/rmannibucau> | Book
> <
> https://www.packtpub.com/application-development/java-ee-8-high-performance
> >
>
>
> Le dim. 29 sept. 2019 à 19:58, Tibor Digana <ti...@apache.org> a
> écrit :
>
> > yes Hunter, exactly this was one possibility.
> > The names of the property can be just like in the HTTP Headers:
> > Last-Modified
> > If-Modified-Since -> maybe here can be also the commit hash, not only
> time
> > in millis/UTC
> > ETag
> >
> > and every module may have different value ;-) then. and then the SCM...
> has
> > to resolve " If-Modified-Since " to the TIME.
> > The missing "..." is some layer useful in the incremental build too.
> >
>
--
Cheers
Tibor
Re: [DISCUSS] configuration for Reproducible Builds
Posted by Romain Manni-Bucau <rm...@gmail.com>.
scm does not work cause one common use case is to rebuild from source the
same artifacts (debian rebuild from source AFAIK, even java apps)
since scm can be "proxied", copied etc then the source can differ and
therefore commits can be differents but the content can be the same
this is why jib uses epoch+1s to enforce reproducibility.
that said once you have the timestamp the code is the same so let's not
block on that, worse case we would enable to plug a value resolver with a
few default strategy.
this is not the central part of the feature IMHO.
Romain Manni-Bucau
@rmannibucau <https://twitter.com/rmannibucau> | Blog
<https://rmannibucau.metawerx.net/> | Old Blog
<http://rmannibucau.wordpress.com> | Github <https://github.com/rmannibucau> |
LinkedIn <https://www.linkedin.com/in/rmannibucau> | Book
<https://www.packtpub.com/application-development/java-ee-8-high-performance>
Le dim. 29 sept. 2019 à 19:58, Tibor Digana <ti...@apache.org> a
écrit :
> yes Hunter, exactly this was one possibility.
> The names of the property can be just like in the HTTP Headers:
> Last-Modified
> If-Modified-Since -> maybe here can be also the commit hash, not only time
> in millis/UTC
> ETag
>
> and every module may have different value ;-) then. and then the SCM... has
> to resolve " If-Modified-Since " to the TIME.
> The missing "..." is some layer useful in the incremental build too.
>
Re: [DISCUSS] configuration for Reproducible Builds
Posted by Tibor Digana <ti...@apache.org>.
yes Hunter, exactly this was one possibility.
The names of the property can be just like in the HTTP Headers:
Last-Modified
If-Modified-Since -> maybe here can be also the commit hash, not only time
in millis/UTC
ETag
and every module may have different value ;-) then. and then the SCM... has
to resolve " If-Modified-Since " to the TIME.
The missing "..." is some layer useful in the incremental build too.
Re: [DISCUSS] configuration for Reproducible Builds
Posted by Paul Hammant <pa...@hammant.org>.
Hunter wrote:
What if that timestamp was based upon the scm's last commit timestamp
> instead of the time of the build?
>
Folks: remember that Subversion works differently to Git. With subversion
there's no hash, just an integer that goes up constantly for everything in
the larger repo. Your trunk/master/release-branch may not have changed in
10 mins, but the repo might have done for an unrelated project/dir.
You're best to make a merkle tree for all files in the checkout (ignoring
target/). You can do that with Subversion too of course even though it does
not itself maintain a Merkle tree of the current set of files nor the
history (as Git goes). Full disclosure - I am obsessed with Merkle trees
(but not blockchain).
Re: [DISCUSS] configuration for Reproducible Builds
Posted by Hunter C Payne <hu...@yahoo.com.INVALID>.
What if that timestamp was based upon the scm's last commit timestamp instead of the time of the build?
Hunter
On Sunday, September 29, 2019, 10:25:41 AM PDT, Tibor Digana <ti...@apache.org> wrote:
Can somebody explain a realistic USE CASE when you trigger two consequent
builds (with no changes in sources) and that you expect identical MD5 of
the build artifacts (JAR)?
This can be achieved only when the "timeStamp" in properties is fixed
unmodified in POM.
Does it make sense to call it timeStamp if it is not related to build time
nothing but some kind of , well I don't know what time because the time is
incremental and does not stop.
Even not very pragmatic to control it because you as USER have to correlate
MD5 with the algorith which produces the timeStamp and its changes.
If no MD5 has changed, the timeStamp would not change either. And here is
the problem: what if this rule is broken? And another problem: how you want
to ensure that the timeStamp is changed if and only if the changes have
been done. Somehow connected to the previous discussion with the
Incremental Build.
On Sun, Sep 29, 2019 at 5:55 PM Romain Manni-Bucau <rm...@gmail.com>
wrote:
> Hi all,
>
> Wonder if it can't "just" (this is not a small task but in terms of design
> it is small ;)) be a flag on higher level archiver plugins
> (maven-jar-plugin being the first one we'll all have in mind).
> I take as a reference jib here which takes into account a creation time for
> that case (
>
> https://github.com/GoogleContainerTools/jib/blob/master/jib-maven-plugin/src/main/java/com/google/cloud/tools/jib/maven/JibPluginConfiguration.java#L190
> ).
> Long story short, they have ~3 modes:
>
> 1. set epoch + 1s (there are issue setting epoch directly)
> 2. set a constant time configured by the user
> 3. respect file time (not reproducible but enable to disable it)
>
> At the end it means we don't need a project.build.* property but just to
> enrich plugins (maybe let's start with jar one) to handle that.
>
> I also wonder if I'm too biased on the topic but if I would have to work on
> it now with our current ecosystem, I would "just" (again ;)) use
> maven-shade-plugin and a set of transformers to handle all files which can
> have not deterministic changes.
> This enables to get the feature immediately without anything specific in
> maven core and handle even external plugin generated files through external
> transformers - a real reproducible build feature would need this extension
> anyway, think about frontend resources included in META-INF/resources for
> example.
> Only missing piece in shade plugin is the Jar reproducibility handling but
> this is likely very doable since we already have JarOutputStream impl at
> apache which can host it IMHO.
>
> Romain Manni-Bucau
> @rmannibucau <https://twitter.com/rmannibucau> | Blog
> <https://rmannibucau.metawerx.net/> | Old Blog
> <http://rmannibucau.wordpress.com> | Github <
> https://github.com/rmannibucau> |
> LinkedIn <https://www.linkedin.com/in/rmannibucau> | Book
> <
> https://www.packtpub.com/application-development/java-ee-8-high-performance
> >
>
>
> Le dim. 29 sept. 2019 à 17:45, Hervé BOUTEMY <he...@free.fr> a
> écrit :
>
> > Le dimanche 29 septembre 2019, 12:29:47 CEST Karl Heinz Marbaise a écrit
> :
> > > Hi Hervé,
> > >
> > > On 29.09.19 11:19, Hervé BOUTEMY wrote:
> > > > regarding the property name, I had an idea:
> > > >
> > > > why not do like we already did for ${project.build.sourceEncoding},
> > ie.
> > > > mimic a future element in pom.xml, in build?
> > > >
> > > > could be project.build.timestamp?
> > >
> > > This sounds like the best idea...
> > >
> > >
> > > This would mean to define something like this:
> > >
> > > <properties>
> > > <project.build.timestamp>..</project.build.timestamp>
> > > </properties>
> > >
> > > But now there are coming up some questions:
> > >
> > > * Is that the real value to be used?
> > > * Or should it activate the mechanism ? (boolean?)
> > we can define both a boolean and a timestamp
> > but the timestamp de-facto means also a boolean: defined means true,
> > undefined
> > means false
> >
> > >
> > > <properties>
> > > <project.build.timestamp.usage>true</project.build.timestamp.usage>
> > > </properties>
> > >
> > >
> > > * Or should we use it by default and giving the user the opportunity
> > > to overrite the current timestamp by fixed timestamp for building ?
> > > This means we would define only the real time to be used during
> > > building. No need for a kind of activation etc.
> > > So you could call Maven via:
> > >
> > > mvn -Dproject.build.timestamp=... package
> > >
> > >
> > > * Or do we need a combination of the above
> > >
> > > First activate, define the format and the timestamp to be used.
> > >
> > >
> > > Furthermore do we need to define a format either which could look like
> > this:
> > >
> > > <properties>
> > > <project.build.timestamp.usage>true</project.build.timestamp.usage>
> > > <project.build.timestamp>..</project.build.timestamp>
> > >
> > <project.build.timestamp.format>ISO-8601</project.build.timestamp.format>
> > > </properties>
> > letting the format as a third parameter is of course feasible, but adds
> > complexity: is it really necessary? Isn't ISO-8601 sufficient to you?
> >
> > >
> > >
> > > Kind regards
> > > Karl Heinz Marbaise
> > >
> > > > Le samedi 28 septembre 2019, 17:55:24 CEST Hervé BOUTEMY a écrit :
> > > >> Achieving Reproducible Builds require only one parameter: plugins
> that
> > > >> create zip or tar archives require a fixed timestamp for entries
> > > >>
> > > >> Putting that parameter as a pom property with a well known name and
> > value
> > > >> format permits to share the configuration between every packaging
> > plugin.
> > > >> This also has the advantage that child poms will inherit from parent
> > > >> value,
> > > >> and eventually override.
> > > >>
> > > >> The question is: *what property name and what value format should we
> > > >> keep?*
> > > >>
> > > >> For the PoC, I chose to extrapolate from a convention from
> > Reproducible
> > > >> Builds project, which is very Linux-oriented: SOURCE_DATE_EPOCH
> > > >> environment
> > > >> variable, that I transformed into source-date-epoch property name,
> > > >> keeping
> > > >> the "date + %s" value
> > > >> https://reproducible-builds.org/docs/source-date-epoch/
> > > >>
> > > >>
> > > >> But I feel we can do a more user-readable solution by choosing
> another
> > > >> name
> > > >> and format, like "reproducible-build-timestamp" with an ISO-8601
> > combined
> > > >> date and time representation
> > > >>
> > > >>
> > > >> WDYT? Any other idea?
> > > >>
> > > >> Regards,
> > > >>
> > > >> Hervé
> >
> >
> >
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> > For additional commands, e-mail: dev-help@maven.apache.org
> >
> >
>
Re: [DISCUSS] configuration for Reproducible Builds
Posted by Tibor Digana <ti...@apache.org>.
Can somebody explain a realistic USE CASE when you trigger two consequent
builds (with no changes in sources) and that you expect identical MD5 of
the build artifacts (JAR)?
This can be achieved only when the "timeStamp" in properties is fixed
unmodified in POM.
Does it make sense to call it timeStamp if it is not related to build time
nothing but some kind of , well I don't know what time because the time is
incremental and does not stop.
Even not very pragmatic to control it because you as USER have to correlate
MD5 with the algorith which produces the timeStamp and its changes.
If no MD5 has changed, the timeStamp would not change either. And here is
the problem: what if this rule is broken? And another problem: how you want
to ensure that the timeStamp is changed if and only if the changes have
been done. Somehow connected to the previous discussion with the
Incremental Build.
On Sun, Sep 29, 2019 at 5:55 PM Romain Manni-Bucau <rm...@gmail.com>
wrote:
> Hi all,
>
> Wonder if it can't "just" (this is not a small task but in terms of design
> it is small ;)) be a flag on higher level archiver plugins
> (maven-jar-plugin being the first one we'll all have in mind).
> I take as a reference jib here which takes into account a creation time for
> that case (
>
> https://github.com/GoogleContainerTools/jib/blob/master/jib-maven-plugin/src/main/java/com/google/cloud/tools/jib/maven/JibPluginConfiguration.java#L190
> ).
> Long story short, they have ~3 modes:
>
> 1. set epoch + 1s (there are issue setting epoch directly)
> 2. set a constant time configured by the user
> 3. respect file time (not reproducible but enable to disable it)
>
> At the end it means we don't need a project.build.* property but just to
> enrich plugins (maybe let's start with jar one) to handle that.
>
> I also wonder if I'm too biased on the topic but if I would have to work on
> it now with our current ecosystem, I would "just" (again ;)) use
> maven-shade-plugin and a set of transformers to handle all files which can
> have not deterministic changes.
> This enables to get the feature immediately without anything specific in
> maven core and handle even external plugin generated files through external
> transformers - a real reproducible build feature would need this extension
> anyway, think about frontend resources included in META-INF/resources for
> example.
> Only missing piece in shade plugin is the Jar reproducibility handling but
> this is likely very doable since we already have JarOutputStream impl at
> apache which can host it IMHO.
>
> Romain Manni-Bucau
> @rmannibucau <https://twitter.com/rmannibucau> | Blog
> <https://rmannibucau.metawerx.net/> | Old Blog
> <http://rmannibucau.wordpress.com> | Github <
> https://github.com/rmannibucau> |
> LinkedIn <https://www.linkedin.com/in/rmannibucau> | Book
> <
> https://www.packtpub.com/application-development/java-ee-8-high-performance
> >
>
>
> Le dim. 29 sept. 2019 à 17:45, Hervé BOUTEMY <he...@free.fr> a
> écrit :
>
> > Le dimanche 29 septembre 2019, 12:29:47 CEST Karl Heinz Marbaise a écrit
> :
> > > Hi Hervé,
> > >
> > > On 29.09.19 11:19, Hervé BOUTEMY wrote:
> > > > regarding the property name, I had an idea:
> > > >
> > > > why not do like we already did for ${project.build.sourceEncoding},
> > ie.
> > > > mimic a future element in pom.xml, in build?
> > > >
> > > > could be project.build.timestamp?
> > >
> > > This sounds like the best idea...
> > >
> > >
> > > This would mean to define something like this:
> > >
> > > <properties>
> > > <project.build.timestamp>..</project.build.timestamp>
> > > </properties>
> > >
> > > But now there are coming up some questions:
> > >
> > > * Is that the real value to be used?
> > > * Or should it activate the mechanism ? (boolean?)
> > we can define both a boolean and a timestamp
> > but the timestamp de-facto means also a boolean: defined means true,
> > undefined
> > means false
> >
> > >
> > > <properties>
> > > <project.build.timestamp.usage>true</project.build.timestamp.usage>
> > > </properties>
> > >
> > >
> > > * Or should we use it by default and giving the user the opportunity
> > > to overrite the current timestamp by fixed timestamp for building ?
> > > This means we would define only the real time to be used during
> > > building. No need for a kind of activation etc.
> > > So you could call Maven via:
> > >
> > > mvn -Dproject.build.timestamp=... package
> > >
> > >
> > > * Or do we need a combination of the above
> > >
> > > First activate, define the format and the timestamp to be used.
> > >
> > >
> > > Furthermore do we need to define a format either which could look like
> > this:
> > >
> > > <properties>
> > > <project.build.timestamp.usage>true</project.build.timestamp.usage>
> > > <project.build.timestamp>..</project.build.timestamp>
> > >
> > <project.build.timestamp.format>ISO-8601</project.build.timestamp.format>
> > > </properties>
> > letting the format as a third parameter is of course feasible, but adds
> > complexity: is it really necessary? Isn't ISO-8601 sufficient to you?
> >
> > >
> > >
> > > Kind regards
> > > Karl Heinz Marbaise
> > >
> > > > Le samedi 28 septembre 2019, 17:55:24 CEST Hervé BOUTEMY a écrit :
> > > >> Achieving Reproducible Builds require only one parameter: plugins
> that
> > > >> create zip or tar archives require a fixed timestamp for entries
> > > >>
> > > >> Putting that parameter as a pom property with a well known name and
> > value
> > > >> format permits to share the configuration between every packaging
> > plugin.
> > > >> This also has the advantage that child poms will inherit from parent
> > > >> value,
> > > >> and eventually override.
> > > >>
> > > >> The question is: *what property name and what value format should we
> > > >> keep?*
> > > >>
> > > >> For the PoC, I chose to extrapolate from a convention from
> > Reproducible
> > > >> Builds project, which is very Linux-oriented: SOURCE_DATE_EPOCH
> > > >> environment
> > > >> variable, that I transformed into source-date-epoch property name,
> > > >> keeping
> > > >> the "date + %s" value
> > > >> https://reproducible-builds.org/docs/source-date-epoch/
> > > >>
> > > >>
> > > >> But I feel we can do a more user-readable solution by choosing
> another
> > > >> name
> > > >> and format, like "reproducible-build-timestamp" with an ISO-8601
> > combined
> > > >> date and time representation
> > > >>
> > > >>
> > > >> WDYT? Any other idea?
> > > >>
> > > >> Regards,
> > > >>
> > > >> Hervé
> >
> >
> >
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> > For additional commands, e-mail: dev-help@maven.apache.org
> >
> >
>
Re: [DISCUSS] configuration for Reproducible Builds
Posted by Romain Manni-Bucau <rm...@gmail.com>.
Hi all,
Wonder if it can't "just" (this is not a small task but in terms of design
it is small ;)) be a flag on higher level archiver plugins
(maven-jar-plugin being the first one we'll all have in mind).
I take as a reference jib here which takes into account a creation time for
that case (
https://github.com/GoogleContainerTools/jib/blob/master/jib-maven-plugin/src/main/java/com/google/cloud/tools/jib/maven/JibPluginConfiguration.java#L190
).
Long story short, they have ~3 modes:
1. set epoch + 1s (there are issue setting epoch directly)
2. set a constant time configured by the user
3. respect file time (not reproducible but enable to disable it)
At the end it means we don't need a project.build.* property but just to
enrich plugins (maybe let's start with jar one) to handle that.
I also wonder if I'm too biased on the topic but if I would have to work on
it now with our current ecosystem, I would "just" (again ;)) use
maven-shade-plugin and a set of transformers to handle all files which can
have not deterministic changes.
This enables to get the feature immediately without anything specific in
maven core and handle even external plugin generated files through external
transformers - a real reproducible build feature would need this extension
anyway, think about frontend resources included in META-INF/resources for
example.
Only missing piece in shade plugin is the Jar reproducibility handling but
this is likely very doable since we already have JarOutputStream impl at
apache which can host it IMHO.
Romain Manni-Bucau
@rmannibucau <https://twitter.com/rmannibucau> | Blog
<https://rmannibucau.metawerx.net/> | Old Blog
<http://rmannibucau.wordpress.com> | Github <https://github.com/rmannibucau> |
LinkedIn <https://www.linkedin.com/in/rmannibucau> | Book
<https://www.packtpub.com/application-development/java-ee-8-high-performance>
Le dim. 29 sept. 2019 à 17:45, Hervé BOUTEMY <he...@free.fr> a
écrit :
> Le dimanche 29 septembre 2019, 12:29:47 CEST Karl Heinz Marbaise a écrit :
> > Hi Hervé,
> >
> > On 29.09.19 11:19, Hervé BOUTEMY wrote:
> > > regarding the property name, I had an idea:
> > >
> > > why not do like we already did for ${project.build.sourceEncoding},
> ie.
> > > mimic a future element in pom.xml, in build?
> > >
> > > could be project.build.timestamp?
> >
> > This sounds like the best idea...
> >
> >
> > This would mean to define something like this:
> >
> > <properties>
> > <project.build.timestamp>..</project.build.timestamp>
> > </properties>
> >
> > But now there are coming up some questions:
> >
> > * Is that the real value to be used?
> > * Or should it activate the mechanism ? (boolean?)
> we can define both a boolean and a timestamp
> but the timestamp de-facto means also a boolean: defined means true,
> undefined
> means false
>
> >
> > <properties>
> > <project.build.timestamp.usage>true</project.build.timestamp.usage>
> > </properties>
> >
> >
> > * Or should we use it by default and giving the user the opportunity
> > to overrite the current timestamp by fixed timestamp for building ?
> > This means we would define only the real time to be used during
> > building. No need for a kind of activation etc.
> > So you could call Maven via:
> >
> > mvn -Dproject.build.timestamp=... package
> >
> >
> > * Or do we need a combination of the above
> >
> > First activate, define the format and the timestamp to be used.
> >
> >
> > Furthermore do we need to define a format either which could look like
> this:
> >
> > <properties>
> > <project.build.timestamp.usage>true</project.build.timestamp.usage>
> > <project.build.timestamp>..</project.build.timestamp>
> >
> <project.build.timestamp.format>ISO-8601</project.build.timestamp.format>
> > </properties>
> letting the format as a third parameter is of course feasible, but adds
> complexity: is it really necessary? Isn't ISO-8601 sufficient to you?
>
> >
> >
> > Kind regards
> > Karl Heinz Marbaise
> >
> > > Le samedi 28 septembre 2019, 17:55:24 CEST Hervé BOUTEMY a écrit :
> > >> Achieving Reproducible Builds require only one parameter: plugins that
> > >> create zip or tar archives require a fixed timestamp for entries
> > >>
> > >> Putting that parameter as a pom property with a well known name and
> value
> > >> format permits to share the configuration between every packaging
> plugin.
> > >> This also has the advantage that child poms will inherit from parent
> > >> value,
> > >> and eventually override.
> > >>
> > >> The question is: *what property name and what value format should we
> > >> keep?*
> > >>
> > >> For the PoC, I chose to extrapolate from a convention from
> Reproducible
> > >> Builds project, which is very Linux-oriented: SOURCE_DATE_EPOCH
> > >> environment
> > >> variable, that I transformed into source-date-epoch property name,
> > >> keeping
> > >> the "date + %s" value
> > >> https://reproducible-builds.org/docs/source-date-epoch/
> > >>
> > >>
> > >> But I feel we can do a more user-readable solution by choosing another
> > >> name
> > >> and format, like "reproducible-build-timestamp" with an ISO-8601
> combined
> > >> date and time representation
> > >>
> > >>
> > >> WDYT? Any other idea?
> > >>
> > >> Regards,
> > >>
> > >> Hervé
>
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> For additional commands, e-mail: dev-help@maven.apache.org
>
>
Re: [DISCUSS] configuration for Reproducible Builds
Posted by Hervé BOUTEMY <he...@free.fr>.
Le dimanche 29 septembre 2019, 12:29:47 CEST Karl Heinz Marbaise a écrit :
> Hi Hervé,
>
> On 29.09.19 11:19, Hervé BOUTEMY wrote:
> > regarding the property name, I had an idea:
> >
> > why not do like we already did for ${project.build.sourceEncoding}, ie.
> > mimic a future element in pom.xml, in build?
> >
> > could be project.build.timestamp?
>
> This sounds like the best idea...
>
>
> This would mean to define something like this:
>
> <properties>
> <project.build.timestamp>..</project.build.timestamp>
> </properties>
>
> But now there are coming up some questions:
>
> * Is that the real value to be used?
> * Or should it activate the mechanism ? (boolean?)
we can define both a boolean and a timestamp
but the timestamp de-facto means also a boolean: defined means true, undefined
means false
>
> <properties>
> <project.build.timestamp.usage>true</project.build.timestamp.usage>
> </properties>
>
>
> * Or should we use it by default and giving the user the opportunity
> to overrite the current timestamp by fixed timestamp for building ?
> This means we would define only the real time to be used during
> building. No need for a kind of activation etc.
> So you could call Maven via:
>
> mvn -Dproject.build.timestamp=... package
>
>
> * Or do we need a combination of the above
>
> First activate, define the format and the timestamp to be used.
>
>
> Furthermore do we need to define a format either which could look like this:
>
> <properties>
> <project.build.timestamp.usage>true</project.build.timestamp.usage>
> <project.build.timestamp>..</project.build.timestamp>
> <project.build.timestamp.format>ISO-8601</project.build.timestamp.format>
> </properties>
letting the format as a third parameter is of course feasible, but adds
complexity: is it really necessary? Isn't ISO-8601 sufficient to you?
>
>
> Kind regards
> Karl Heinz Marbaise
>
> > Le samedi 28 septembre 2019, 17:55:24 CEST Hervé BOUTEMY a écrit :
> >> Achieving Reproducible Builds require only one parameter: plugins that
> >> create zip or tar archives require a fixed timestamp for entries
> >>
> >> Putting that parameter as a pom property with a well known name and value
> >> format permits to share the configuration between every packaging plugin.
> >> This also has the advantage that child poms will inherit from parent
> >> value,
> >> and eventually override.
> >>
> >> The question is: *what property name and what value format should we
> >> keep?*
> >>
> >> For the PoC, I chose to extrapolate from a convention from Reproducible
> >> Builds project, which is very Linux-oriented: SOURCE_DATE_EPOCH
> >> environment
> >> variable, that I transformed into source-date-epoch property name,
> >> keeping
> >> the "date + %s" value
> >> https://reproducible-builds.org/docs/source-date-epoch/
> >>
> >>
> >> But I feel we can do a more user-readable solution by choosing another
> >> name
> >> and format, like "reproducible-build-timestamp" with an ISO-8601 combined
> >> date and time representation
> >>
> >>
> >> WDYT? Any other idea?
> >>
> >> Regards,
> >>
> >> Hervé
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: [DISCUSS] configuration for Reproducible Builds
Posted by Karl Heinz Marbaise <kh...@gmx.de>.
Hi Hervé,
On 29.09.19 11:19, Hervé BOUTEMY wrote:
> regarding the property name, I had an idea:
>
> why not do like we already did for ${project.build.sourceEncoding}, ie. mimic
> a future element in pom.xml, in build?
>
> could be project.build.timestamp?
This sounds like the best idea...
This would mean to define something like this:
<properties>
<project.build.timestamp>..</project.build.timestamp>
</properties>
But now there are coming up some questions:
* Is that the real value to be used?
* Or should it activate the mechanism ? (boolean?)
<properties>
<project.build.timestamp.usage>true</project.build.timestamp.usage>
</properties>
* Or should we use it by default and giving the user the opportunity
to overrite the current timestamp by fixed timestamp for building ?
This means we would define only the real time to be used during
building. No need for a kind of activation etc.
So you could call Maven via:
mvn -Dproject.build.timestamp=... package
* Or do we need a combination of the above
First activate, define the format and the timestamp to be used.
Furthermore do we need to define a format either which could look like this:
<properties>
<project.build.timestamp.usage>true</project.build.timestamp.usage>
<project.build.timestamp>..</project.build.timestamp>
<project.build.timestamp.format>ISO-8601</project.build.timestamp.format>
</properties>
Kind regards
Karl Heinz Marbaise
>
> Le samedi 28 septembre 2019, 17:55:24 CEST Hervé BOUTEMY a écrit :
>> Achieving Reproducible Builds require only one parameter: plugins that
>> create zip or tar archives require a fixed timestamp for entries
>>
>> Putting that parameter as a pom property with a well known name and value
>> format permits to share the configuration between every packaging plugin.
>> This also has the advantage that child poms will inherit from parent value,
>> and eventually override.
>>
>> The question is: *what property name and what value format should we keep?*
>>
>> For the PoC, I chose to extrapolate from a convention from Reproducible
>> Builds project, which is very Linux-oriented: SOURCE_DATE_EPOCH environment
>> variable, that I transformed into source-date-epoch property name, keeping
>> the "date + %s" value
>> https://reproducible-builds.org/docs/source-date-epoch/
>>
>>
>> But I feel we can do a more user-readable solution by choosing another name
>> and format, like "reproducible-build-timestamp" with an ISO-8601 combined
>> date and time representation
>>
>>
>> WDYT? Any other idea?
>>
>> Regards,
>>
>> Hervé
>>
>>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: [DISCUSS] configuration for Reproducible Builds
Posted by Hervé BOUTEMY <he...@free.fr>.
regarding the property name, I had an idea:
why not do like we already did for ${project.build.sourceEncoding}, ie. mimic
a future element in pom.xml, in build?
could be project.build.timestamp?
Le samedi 28 septembre 2019, 17:55:24 CEST Hervé BOUTEMY a écrit :
> Achieving Reproducible Builds require only one parameter: plugins that
> create zip or tar archives require a fixed timestamp for entries
>
> Putting that parameter as a pom property with a well known name and value
> format permits to share the configuration between every packaging plugin.
> This also has the advantage that child poms will inherit from parent value,
> and eventually override.
>
> The question is: *what property name and what value format should we keep?*
>
> For the PoC, I chose to extrapolate from a convention from Reproducible
> Builds project, which is very Linux-oriented: SOURCE_DATE_EPOCH environment
> variable, that I transformed into source-date-epoch property name, keeping
> the "date + %s" value
> https://reproducible-builds.org/docs/source-date-epoch/
>
>
> But I feel we can do a more user-readable solution by choosing another name
> and format, like "reproducible-build-timestamp" with an ISO-8601 combined
> date and time representation
>
>
> WDYT? Any other idea?
>
> Regards,
>
> Hervé
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> For additional commands, e-mail: dev-help@maven.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: [DISCUSS] configuration for Reproducible Builds
Posted by Hervé BOUTEMY <he...@free.fr>.
Le dimanche 29 septembre 2019, 21:31:58 CEST Emmanuel Bourg a écrit :
> Le 29/09/2019 à 20:46, Hervé BOUTEMY a écrit :
> > This is exactly how I see Reproducible Builds for the future:
> > - select versions of plugins that bring reproducible output
> > - either inherit or define a local timestamp
> >
> > et voilà, it's so easy (once plugins support)...
>
> How do you plan to capture the elements of the build environment
> necessary to build identical artifacts (JDK used, command line
> parameters) ? As project properties too?
currently, JDK version is recorded by m-jar-p in MANIFEST, and it's easy to
know if it's Windows or not: then for people who know what command to launch,
it won't be hard
for wider case, where we'd like to rebuild anything that we don't precisely
know, even with other build tools than Maven, we wrote 1 year ago a proposal:
https://reproducible-builds.org/docs/jvm/
I stopped working on the proposal, since it was not really useful until we
were able to do reproducible builds: in a few weeks, once Maven can do
Reproducible Builds quite easily, we can continue working on this proposal
with a chance people will provide feedback based on real experience
Regards,
Hervé
>
> Emmanuel Bourg
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> For additional commands, e-mail: dev-help@maven.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: [DISCUSS] configuration for Reproducible Builds
Posted by Romain Manni-Bucau <rm...@gmail.com>.
While default stays not reproducible (keep in mind we enrich with dated
timestamp or build user metadata in jar quite commonly and i dont think we
want to sed/drop it) and reproducible mode gets an explicit flag in plugins
it sounds very promishing to me, in particular for containers case.
Le dim. 29 sept. 2019 à 21:32, Emmanuel Bourg <eb...@apache.org> a écrit :
> Le 29/09/2019 à 20:46, Hervé BOUTEMY a écrit :
>
> > This is exactly how I see Reproducible Builds for the future:
> > - select versions of plugins that bring reproducible output
> > - either inherit or define a local timestamp
> >
> > et voilà, it's so easy (once plugins support)...
>
> How do you plan to capture the elements of the build environment
> necessary to build identical artifacts (JDK used, command line
> parameters) ? As project properties too?
>
> Emmanuel Bourg
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> For additional commands, e-mail: dev-help@maven.apache.org
>
>
Re: [DISCUSS] configuration for Reproducible Builds
Posted by Emmanuel Bourg <eb...@apache.org>.
Le 29/09/2019 à 20:46, Hervé BOUTEMY a écrit :
> This is exactly how I see Reproducible Builds for the future:
> - select versions of plugins that bring reproducible output
> - either inherit or define a local timestamp
>
> et voilà, it's so easy (once plugins support)...
How do you plan to capture the elements of the build environment
necessary to build identical artifacts (JDK used, command line
parameters) ? As project properties too?
Emmanuel Bourg
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: [DISCUSS] configuration for Reproducible Builds
Posted by Hervé BOUTEMY <he...@free.fr>.
perhaps I should basically show how Reproducible Builds was configured in the PoC:
1. in apache-parent:
https://github.com/apache/maven-apache-parent/commit/d379a72d07173c500be65fd6e549da1fedb46b5f
It's about 3 packaging plugins that have a version supporting Reproducible Builds
and one parameter: the timestamp to put on archive entries = the value of "now" when I commited the feature
2. in maven-parent:
https://github.com/apache/maven-parent/commit/d213125b171c8a2f7e4cbb74b713e11e0fd055dd
I just updated the parent POM
And chose to define a local value for timestamp, to override the value defined in parent
But honestly, who cares if the timestamp in zip file was 2019-09-21T16:03:05Z (the value inherited from parent) instead of 2019-09-21T16:15:16Z (the overridden value)?
3. in maven:
https://github.com/apache/maven/commit/d23d3b4fab9b3951177eac5bbfa109e990e95899
same as before: changed parent, and chose to define a local value for timestamp
Every commit after that is just little improvement or fixes because a few plugins don't yet bring reproducible output
This is exactly how I see Reproducible Builds for the future:
- select versions of plugins that bring reproducible output
- either inherit or define a local timestamp
et voilà, it's so easy (once plugins support)...
Regards,
Hervé
Le samedi 28 septembre 2019, 17:55:24 CEST Hervé BOUTEMY a écrit :
> Achieving Reproducible Builds require only one parameter: plugins that
> create zip or tar archives require a fixed timestamp for entries
>
> Putting that parameter as a pom property with a well known name and value
> format permits to share the configuration between every packaging plugin.
> This also has the advantage that child poms will inherit from parent value,
> and eventually override.
>
> The question is: *what property name and what value format should we keep?*
>
> For the PoC, I chose to extrapolate from a convention from Reproducible
> Builds project, which is very Linux-oriented: SOURCE_DATE_EPOCH environment
> variable, that I transformed into source-date-epoch property name, keeping
> the "date + %s" value
> https://reproducible-builds.org/docs/source-date-epoch/
>
>
> But I feel we can do a more user-readable solution by choosing another name
> and format, like "reproducible-build-timestamp" with an ISO-8601 combined
> date and time representation
>
>
> WDYT? Any other idea?
>
> Regards,
>
> Hervé
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> For additional commands, e-mail: dev-help@maven.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org