You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@maven.apache.org by Hervé BOUTEMY <he...@free.fr> on 2019/10/05 17:52:22 UTC
last review of Reproducible Builds proposal
based on the feedback I got, I updated the proposal:
https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=74682318
The archives entries timestamp is now configured with project.build.outputTimestamp property, in ISO-8601 format
<project>
<properties>
<project.build.outputTimestamp>2019-10-02T08:04:00Z</project.build.outputTimestamp>
</properties>
</project>
The shared components, plugins, parent poms and Maven core branches have been updated to match this new proposal
If no one objects, next week-end, I'll start the (heavy) release train to bring (binary) Reproducible Builds plugins to general availability
Regards,
Hervé
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: last review of Reproducible Builds proposal
Posted by Michael Osipov <mi...@apache.org>.
Am 2019-10-06 um 20:35 schrieb Hervé BOUTEMY:
> Le dimanche 6 octobre 2019 20:29:46 CEST, vous avez écrit :
>> Am 2019-10-06 um 20:21 schrieb Hervé BOUTEMY:> Le dimanche 6 octobre
>>
>> 2019 12:24:57 CEST, vous avez écrit :
>> >> Am 2019-10-06 um 09:35 schrieb Hervé BOUTEMY:
>> >>> Le samedi 5 octobre 2019, 22:46:20 CEST Michael Osipov a écrit :
>> >>>> Am 2019-10-05 um 22:10 schrieb Hervé BOUTEMY:
>> >>>>> Le samedi 5 octobre 2019 20:41:40 CEST, vous avez écrit :
>> >>>>>> Am 2019-10-05 um 19:52 schrieb Hervé BOUTEMY:
>> >>>>>>> based on the feedback I got, I updated the proposal:
>> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=74682
>>
>> >>>>>>> 31
>> >>>>>>> 8
>> >>>>>>>
>> >>>>>>> The archives entries timestamp is now configured with
>> >>>>>>> project.build.outputTimestamp property, in ISO-8601 format
>>
>> <project>
>>
>> >>>>>>> <properties>
>>
>> <project.build.outputTimestamp>2019-10-02T08:04:00Z</project.bu
>>
>> >>>>>>> ild
>> >>>>>>> .ou
>> >>>>>>> tputTimestamp>>
>> >>>>>>>
>> >>>>>>> </properties>
>> >>>>>>>
>> >>>>>>> </project>
>> >>>>>>>
>> >>>>>>> The shared components, plugins, parent poms and Maven core branches
>> >>>>>>> have
>> >>>>>>> been updated to match this new proposal
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> If no one objects, next week-end, I'll start the (heavy)
>>
>> release train
>>
>> >>>>>>> to
>> >>>>>>> bring (binary) Reproducible Builds plugins to general availability
>> >>>>>>
>> >>>>>> Really, really nice work..
>> >>>>>
>> >>>>> thank you, it required a lot of energy for a long period of time...
>> >>>>>
>> >>>>>> I do like <project.build.outputTimestamp>. Why did you make it a
>> >>>>>> String?
>> >>>>>> Why not go directly with Instant? It gives your ISO 8601 for free
>> >>>>>
>> >>>>> I tried to explain it in
>> >>>>> https://issues.apache.org/jira/browse/MSHARED-837
>> >>>>> Plexus Date injection support is really limited: could not match the
>> >>>>> requirements
>> >>>>
>> >>>> yyyy-MM-dd'T'HH:mm:ssXXX and SimpleDateFormat will do the trick.
>>
>> It also
>>
>> >>>> will require to change the converter of course.
>> >>>>
>> >>>> Alternatively, you could try
>>
>> https://commons.apache.org/proper/commons-lang/javadocs/api-release/src-h
>>
>> >>>> tml /org/apache/commons/lang3/time/DateFormatUtils.html#line.72
>>
>> which I
>>
>> >>>> have added long time ago.
>> >>>
>> >>> the question is not about coding the date format: it's already done in
>> >>> maven-archiver
>>
>> https://github.com/apache/maven-archiver/commit/5f07f227aa89e0bb4163c125a
>>
>> >>> 46fbd4c78445301
>> >>
>> >> The commit contains some flaws. I will leave comments there.
>> >>
>> >>> the question, as you perfectly asked initially, is: do you have an
>>
>> idea on
>>
>> >>> how to have the direct Date injection, instead of having a String
>>
>> in the
>>
>> >>> plugin parameter?
>> >>
>> >> I assumed the DateConverter will do as soon as you say:
>> >> @Parameter
>> >> Date myDate;
>> >>
>> >> Won't it?
>> >
>> > yes, it will
>> > and if you look at the SimpleDateFormat value, you'll see that it
>>
>> does not
>>
>> > permit any timezone, then uses the local developper timezone, then is not
>> > reproducible
>>
>> No, it won't if you supply and enforce XXX => yyyy-MM-dd'T'HH:mm:ssXXX.
>> The input has to have the tz offset, thus you'll have it properly
>> converted. It would perform the very same conversion and your code
>> currently does in Maven Archiver. You should give it a try.
> DateConverter is part of plexus-containers and Sisu: I'm not modifying them
Why don't you say some upfront? I assumed you want to change it.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: last review of Reproducible Builds proposal
Posted by Michael Osipov <mi...@apache.org>.
Am 2019-10-06 um 20:21 schrieb Hervé BOUTEMY:> Le dimanche 6 octobre
2019 12:24:57 CEST, vous avez écrit :
>> Am 2019-10-06 um 09:35 schrieb Hervé BOUTEMY:
>>> Le samedi 5 octobre 2019, 22:46:20 CEST Michael Osipov a écrit :
>>>> Am 2019-10-05 um 22:10 schrieb Hervé BOUTEMY:
>>>>> Le samedi 5 octobre 2019 20:41:40 CEST, vous avez écrit :
>>>>>> Am 2019-10-05 um 19:52 schrieb Hervé BOUTEMY:
>>>>>>> based on the feedback I got, I updated the proposal:
>>>>>>>
https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=74682
>>>>>>> 31
>>>>>>> 8
>>>>>>>
>>>>>>> The archives entries timestamp is now configured with
>>>>>>> project.build.outputTimestamp property, in ISO-8601 format
<project>
>>>>>>>
>>>>>>> <properties>
>>>>>>>
>>>>>>>
<project.build.outputTimestamp>2019-10-02T08:04:00Z</project.bu
>>>>>>> ild
>>>>>>> .ou
>>>>>>> tputTimestamp>>
>>>>>>>
>>>>>>> </properties>
>>>>>>>
>>>>>>> </project>
>>>>>>>
>>>>>>> The shared components, plugins, parent poms and Maven core branches
>>>>>>> have
>>>>>>> been updated to match this new proposal
>>>>>>>
>>>>>>>
>>>>>>> If no one objects, next week-end, I'll start the (heavy)
release train
>>>>>>> to
>>>>>>> bring (binary) Reproducible Builds plugins to general availability
>>>>>>
>>>>>> Really, really nice work..
>>>>>
>>>>> thank you, it required a lot of energy for a long period of time...
>>>>>
>>>>>> I do like <project.build.outputTimestamp>. Why did you make it a
>>>>>> String?
>>>>>> Why not go directly with Instant? It gives your ISO 8601 for free
>>>>>
>>>>> I tried to explain it in
>>>>> https://issues.apache.org/jira/browse/MSHARED-837
>>>>> Plexus Date injection support is really limited: could not match the
>>>>> requirements
>>>>
>>>> yyyy-MM-dd'T'HH:mm:ssXXX and SimpleDateFormat will do the trick.
It also
>>>> will require to change the converter of course.
>>>>
>>>> Alternatively, you could try
>>>>
https://commons.apache.org/proper/commons-lang/javadocs/api-release/src-h
>>>> tml /org/apache/commons/lang3/time/DateFormatUtils.html#line.72
which I
>>>> have added long time ago.
>>>
>>> the question is not about coding the date format: it's already done in
>>> maven-archiver
>>>
https://github.com/apache/maven-archiver/commit/5f07f227aa89e0bb4163c125a
>>> 46fbd4c78445301
>> The commit contains some flaws. I will leave comments there.
>>
>>> the question, as you perfectly asked initially, is: do you have an
idea on
>>> how to have the direct Date injection, instead of having a String
in the
>>> plugin parameter?
>> I assumed the DateConverter will do as soon as you say:
>> @Parameter
>> Date myDate;
>>
>> Won't it?
> yes, it will
> and if you look at the SimpleDateFormat value, you'll see that it
does not
> permit any timezone, then uses the local developper timezone, then is not
> reproducible
No, it won't if you supply and enforce XXX => yyyy-MM-dd'T'HH:mm:ssXXX.
The input has to have the tz offset, thus you'll have it properly
converted. It would perform the very same conversion and your code
currently does in Maven Archiver. You should give it a try.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: last review of Reproducible Builds proposal
Posted by Vladimir Sitnikov <si...@gmail.com>.
>ISO 8601 is neither aware of zones or
>DSTs, just abstract offsets which is a good thing
Well. ISO 8601 allows timestamps without "UTC offset"
For instance, 2019-10-06T12:30:00 is ISO 8601-compatible timestamp, however
it is ambiguous.
So I suggest that the property must require Z or +HH:MM part.
Vladimir
Re: last review of Reproducible Builds proposal
Posted by Michael Osipov <mi...@apache.org>.
I don't understand that either. ISO 8601 is neither aware of zones or
DSTs, just abstract offsets which is a good thing. The format Hervé has
chosen is almost correct (comments) left. The way the value has to be
provided can *always* be canonicalized to UTC. I don't see here a big
hurdle to solve a problem. When you provide 2019-10-06T14:30:00+02:00 is
clear and equals to 2019-10-06T12:30:00Z. Same applies for every other
offset.
Please keep in mind that java.util.Date is merely a wrapper around Unix
timestamp. SimpleDateFormat will work.
Am 2019-10-06 um 14:24 schrieb Tibor Digana:
> The difference between this time and UTC is ( Zone Offset + DST offset ).
> I think here this feature does not need to describe the LOCAL time nothing
> but the timestamp and that is in UTC. It makes sense that the plugins use
> particular FORMAT, forinstance platform format of predefined format by the
> Maven. It is save to manipulate with the UTC time even if you use it a
> minute before DST or after. The UTC will be always precisely understood
> exact clock by Java and Maven too.
>
> On Sun, Oct 6, 2019 at 12:37 PM Michael Osipov <mi...@apache.org> wrote:
>
>> I still don't see and issue because the offset is there. If you subtract
>> or add the offset and you have the Zulu time.
>>
>> Can you provide this concrete example? I am quite certain that there was
>> an error on some side.
>>
>> If you case is true, the entire time logic in Java 8 woudn't be able to
>> perform conversions from Istants to LocalDateTime.
>>
>> Am 2019-10-06 um 12:34 schrieb Tibor Digana:
>>> Michael, it is the problem with summer time. Do you know what i mean?We
>> had
>>> this problem in my company therefore we strictly used Z as UTC and if
>>> somebody sent another timezone we sent back an error from REST.
>>> You cannot say that you disagree if you do not understand. Pls have it
>>> logically explained first!
>>>
>>> On Sun, Oct 6, 2019 at 12:32 PM Michael Osipov <mi...@apache.org>
>> wrote:
>>>
>>>> Am 2019-10-06 um 12:25 schrieb Tibor Digana:
>>>>> ISO format was often discussed and this was found as problematic
>> format
>>>>> because you cannot always compute it to UTC due to GMT offset. The
>> offset
>>>>> is not enough. What is required for EXACT computing to UTC is Time zome
>>>>> name but this format does not support it. It is exactly the same
>> problem
>>>> in
>>>>> XML.
>>>>
>>>> I don't understand that and do not agree. Of course, you can do
>>>> normalization. Can you please elaborate?
>>>>
>>>
>>
>>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: last review of Reproducible Builds proposal
Posted by Tibor Digana <ti...@apache.org>.
The difference between this time and UTC is ( Zone Offset + DST offset ).
I think here this feature does not need to describe the LOCAL time nothing
but the timestamp and that is in UTC. It makes sense that the plugins use
particular FORMAT, forinstance platform format of predefined format by the
Maven. It is save to manipulate with the UTC time even if you use it a
minute before DST or after. The UTC will be always precisely understood
exact clock by Java and Maven too.
On Sun, Oct 6, 2019 at 12:37 PM Michael Osipov <mi...@apache.org> wrote:
> I still don't see and issue because the offset is there. If you subtract
> or add the offset and you have the Zulu time.
>
> Can you provide this concrete example? I am quite certain that there was
> an error on some side.
>
> If you case is true, the entire time logic in Java 8 woudn't be able to
> perform conversions from Istants to LocalDateTime.
>
> Am 2019-10-06 um 12:34 schrieb Tibor Digana:
> > Michael, it is the problem with summer time. Do you know what i mean?We
> had
> > this problem in my company therefore we strictly used Z as UTC and if
> > somebody sent another timezone we sent back an error from REST.
> > You cannot say that you disagree if you do not understand. Pls have it
> > logically explained first!
> >
> > On Sun, Oct 6, 2019 at 12:32 PM Michael Osipov <mi...@apache.org>
> wrote:
> >
> >> Am 2019-10-06 um 12:25 schrieb Tibor Digana:
> >>> ISO format was often discussed and this was found as problematic
> format
> >>> because you cannot always compute it to UTC due to GMT offset. The
> offset
> >>> is not enough. What is required for EXACT computing to UTC is Time zome
> >>> name but this format does not support it. It is exactly the same
> problem
> >> in
> >>> XML.
> >>
> >> I don't understand that and do not agree. Of course, you can do
> >> normalization. Can you please elaborate?
> >>
> >
>
>
Re: last review of Reproducible Builds proposal
Posted by Michael Osipov <mi...@apache.org>.
I still don't see and issue because the offset is there. If you subtract
or add the offset and you have the Zulu time.
Can you provide this concrete example? I am quite certain that there was
an error on some side.
If you case is true, the entire time logic in Java 8 woudn't be able to
perform conversions from Istants to LocalDateTime.
Am 2019-10-06 um 12:34 schrieb Tibor Digana:
> Michael, it is the problem with summer time. Do you know what i mean?We had
> this problem in my company therefore we strictly used Z as UTC and if
> somebody sent another timezone we sent back an error from REST.
> You cannot say that you disagree if you do not understand. Pls have it
> logically explained first!
>
> On Sun, Oct 6, 2019 at 12:32 PM Michael Osipov <mi...@apache.org> wrote:
>
>> Am 2019-10-06 um 12:25 schrieb Tibor Digana:
>>> ISO format was often discussed and this was found as problematic format
>>> because you cannot always compute it to UTC due to GMT offset. The offset
>>> is not enough. What is required for EXACT computing to UTC is Time zome
>>> name but this format does not support it. It is exactly the same problem
>> in
>>> XML.
>>
>> I don't understand that and do not agree. Of course, you can do
>> normalization. Can you please elaborate?
>>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: last review of Reproducible Builds proposal
Posted by Tibor Digana <ti...@apache.org>.
Michael, it is the problem with summer time. Do you know what i mean?We had
this problem in my company therefore we strictly used Z as UTC and if
somebody sent another timezone we sent back an error from REST.
You cannot say that you disagree if you do not understand. Pls have it
logically explained first!
On Sun, Oct 6, 2019 at 12:32 PM Michael Osipov <mi...@apache.org> wrote:
> Am 2019-10-06 um 12:25 schrieb Tibor Digana:
> > ISO format was often discussed and this was found as problematic format
> > because you cannot always compute it to UTC due to GMT offset. The offset
> > is not enough. What is required for EXACT computing to UTC is Time zome
> > name but this format does not support it. It is exactly the same problem
> in
> > XML.
>
> I don't understand that and do not agree. Of course, you can do
> normalization. Can you please elaborate?
>
Re: last review of Reproducible Builds proposal
Posted by Michael Osipov <mi...@apache.org>.
Am 2019-10-06 um 12:25 schrieb Tibor Digana:
> ISO format was often discussed and this was found as problematic format
> because you cannot always compute it to UTC due to GMT offset. The offset
> is not enough. What is required for EXACT computing to UTC is Time zome
> name but this format does not support it. It is exactly the same problem in
> XML.
I don't understand that and do not agree. Of course, you can do
normalization. Can you please elaborate?
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: last review of Reproducible Builds proposal
Posted by Tibor Digana <ti...@apache.org>.
ISO format was often discussed and this was found as problematic format
because you cannot always compute it to UTC due to GMT offset. The offset
is not enough. What is required for EXACT computing to UTC is Time zome
name but this format does not support it. It is exactly the same problem in
XML.
I prefer using LONG by default because it naturaly represents the time in
UTC.
Format in second property would override but I do not recommend to use it.
Then the plugins can compute LONG/UTC to whatever format they want to or
they prefer.
On Sun, Oct 6, 2019 at 12:25 PM Michael Osipov <mi...@apache.org> wrote:
> Am 2019-10-06 um 09:35 schrieb Hervé BOUTEMY:
> > Le samedi 5 octobre 2019, 22:46:20 CEST Michael Osipov a écrit :
> >> Am 2019-10-05 um 22:10 schrieb Hervé BOUTEMY:
> >>> Le samedi 5 octobre 2019 20:41:40 CEST, vous avez écrit :
> >>>> Am 2019-10-05 um 19:52 schrieb Hervé BOUTEMY:
> >>>>> based on the feedback I got, I updated the proposal:
> >>>>>
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=7468231
> >>>>> 8
> >>>>>
> >>>>> The archives entries timestamp is now configured with
> >>>>> project.build.outputTimestamp property, in ISO-8601 format <project>
> >>>>>
> >>>>> <properties>
> >>>>>
> >>>>>
> <project.build.outputTimestamp>2019-10-02T08:04:00Z</project.build
> >>>>> .ou
> >>>>> tputTimestamp>>
> >>>>>
> >>>>> </properties>
> >>>>>
> >>>>> </project>
> >>>>>
> >>>>> The shared components, plugins, parent poms and Maven core branches
> have
> >>>>> been updated to match this new proposal
> >>>>>
> >>>>>
> >>>>> If no one objects, next week-end, I'll start the (heavy) release
> train
> >>>>> to
> >>>>> bring (binary) Reproducible Builds plugins to general availability
> >>>>
> >>>> Really, really nice work..
> >>>
> >>> thank you, it required a lot of energy for a long period of time...
> >>>
> >>>> I do like <project.build.outputTimestamp>. Why did you make it a
> String?
> >>>> Why not go directly with Instant? It gives your ISO 8601 for free
> >>>
> >>> I tried to explain it in
> https://issues.apache.org/jira/browse/MSHARED-837
> >>> Plexus Date injection support is really limited: could not match the
> >>> requirements
> >>
> >> yyyy-MM-dd'T'HH:mm:ssXXX and SimpleDateFormat will do the trick. It also
> >> will require to change the converter of course.
> >>
> >> Alternatively, you could try
> >>
> https://commons.apache.org/proper/commons-lang/javadocs/api-release/src-html
> >> /org/apache/commons/lang3/time/DateFormatUtils.html#line.72 which I have
> >> added long time ago.
> > the question is not about coding the date format: it's already done in
> maven-archiver
> >
> https://github.com/apache/maven-archiver/commit/5f07f227aa89e0bb4163c125a46fbd4c78445301
>
> The commit contains some flaws. I will leave comments there.
>
> > the question, as you perfectly asked initially, is: do you have an idea
> on how to have the direct Date injection, instead of having a String in the
> plugin parameter?
>
> I assumed the DateConverter will do as soon as you say:
> @Parameter
> Date myDate;
>
> Won't it?
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> For additional commands, e-mail: dev-help@maven.apache.org
>
>
Re: last review of Reproducible Builds proposal
Posted by Michael Osipov <mi...@apache.org>.
Am 2019-10-06 um 09:35 schrieb Hervé BOUTEMY:
> Le samedi 5 octobre 2019, 22:46:20 CEST Michael Osipov a écrit :
>> Am 2019-10-05 um 22:10 schrieb Hervé BOUTEMY:
>>> Le samedi 5 octobre 2019 20:41:40 CEST, vous avez écrit :
>>>> Am 2019-10-05 um 19:52 schrieb Hervé BOUTEMY:
>>>>> based on the feedback I got, I updated the proposal:
>>>>> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=7468231
>>>>> 8
>>>>>
>>>>> The archives entries timestamp is now configured with
>>>>> project.build.outputTimestamp property, in ISO-8601 format <project>
>>>>>
>>>>> <properties>
>>>>>
>>>>> <project.build.outputTimestamp>2019-10-02T08:04:00Z</project.build
>>>>> .ou
>>>>> tputTimestamp>>
>>>>>
>>>>> </properties>
>>>>>
>>>>> </project>
>>>>>
>>>>> The shared components, plugins, parent poms and Maven core branches have
>>>>> been updated to match this new proposal
>>>>>
>>>>>
>>>>> If no one objects, next week-end, I'll start the (heavy) release train
>>>>> to
>>>>> bring (binary) Reproducible Builds plugins to general availability
>>>>
>>>> Really, really nice work..
>>>
>>> thank you, it required a lot of energy for a long period of time...
>>>
>>>> I do like <project.build.outputTimestamp>. Why did you make it a String?
>>>> Why not go directly with Instant? It gives your ISO 8601 for free
>>>
>>> I tried to explain it in https://issues.apache.org/jira/browse/MSHARED-837
>>> Plexus Date injection support is really limited: could not match the
>>> requirements
>>
>> yyyy-MM-dd'T'HH:mm:ssXXX and SimpleDateFormat will do the trick. It also
>> will require to change the converter of course.
>>
>> Alternatively, you could try
>> https://commons.apache.org/proper/commons-lang/javadocs/api-release/src-html
>> /org/apache/commons/lang3/time/DateFormatUtils.html#line.72 which I have
>> added long time ago.
> the question is not about coding the date format: it's already done in maven-archiver
> https://github.com/apache/maven-archiver/commit/5f07f227aa89e0bb4163c125a46fbd4c78445301
The commit contains some flaws. I will leave comments there.
> the question, as you perfectly asked initially, is: do you have an idea on how to have the direct Date injection, instead of having a String in the plugin parameter?
I assumed the DateConverter will do as soon as you say:
@Parameter
Date myDate;
Won't it?
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: last review of Reproducible Builds proposal
Posted by Hervé BOUTEMY <he...@free.fr>.
Le samedi 5 octobre 2019, 22:46:20 CEST Michael Osipov a écrit :
> Am 2019-10-05 um 22:10 schrieb Hervé BOUTEMY:
> > Le samedi 5 octobre 2019 20:41:40 CEST, vous avez écrit :
> >> Am 2019-10-05 um 19:52 schrieb Hervé BOUTEMY:
> >>> based on the feedback I got, I updated the proposal:
> >>> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=7468231
> >>> 8
> >>>
> >>> The archives entries timestamp is now configured with
> >>> project.build.outputTimestamp property, in ISO-8601 format <project>
> >>>
> >>> <properties>
> >>>
> >>> <project.build.outputTimestamp>2019-10-02T08:04:00Z</project.build
> >>> .ou
> >>> tputTimestamp>>
> >>>
> >>> </properties>
> >>>
> >>> </project>
> >>>
> >>> The shared components, plugins, parent poms and Maven core branches have
> >>> been updated to match this new proposal
> >>>
> >>>
> >>> If no one objects, next week-end, I'll start the (heavy) release train
> >>> to
> >>> bring (binary) Reproducible Builds plugins to general availability
> >>
> >> Really, really nice work..
> >
> > thank you, it required a lot of energy for a long period of time...
> >
> >> I do like <project.build.outputTimestamp>. Why did you make it a String?
> >> Why not go directly with Instant? It gives your ISO 8601 for free
> >
> > I tried to explain it in https://issues.apache.org/jira/browse/MSHARED-837
> > Plexus Date injection support is really limited: could not match the
> > requirements
>
> yyyy-MM-dd'T'HH:mm:ssXXX and SimpleDateFormat will do the trick. It also
> will require to change the converter of course.
>
> Alternatively, you could try
> https://commons.apache.org/proper/commons-lang/javadocs/api-release/src-html
> /org/apache/commons/lang3/time/DateFormatUtils.html#line.72 which I have
> added long time ago.
the question is not about coding the date format: it's already done in maven-archiver
https://github.com/apache/maven-archiver/commit/5f07f227aa89e0bb4163c125a46fbd4c78445301
the question, as you perfectly asked initially, is: do you have an idea on how to have the direct Date injection, instead of having a String in the plugin parameter?
>
> Moreover, I'd like these updated plugins be part of MNG-6169.
sure, having native Reproducible Builds by default is a good feature for Maven Core.
Even if there will be a warning since people need to define the plugin version in their pom to avoid dependency on Maven core version: https://issues.apache.org/jira/browse/MNG-6562
>
> Michael
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> For additional commands, e-mail: dev-help@maven.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: last review of Reproducible Builds proposal
Posted by Michael Osipov <mi...@apache.org>.
Am 2019-10-05 um 22:10 schrieb Hervé BOUTEMY:
> Le samedi 5 octobre 2019 20:41:40 CEST, vous avez écrit :
>> Am 2019-10-05 um 19:52 schrieb Hervé BOUTEMY:
>>> based on the feedback I got, I updated the proposal:
>>> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=74682318
>>>
>>> The archives entries timestamp is now configured with
>>> project.build.outputTimestamp property, in ISO-8601 format <project>
>>>
>>> <properties>
>>>
>>> <project.build.outputTimestamp>2019-10-02T08:04:00Z</project.build.ou
>>> tputTimestamp>>
>>> </properties>
>>>
>>> </project>
>>>
>>> The shared components, plugins, parent poms and Maven core branches have
>>> been updated to match this new proposal
>>>
>>>
>>> If no one objects, next week-end, I'll start the (heavy) release train to
>>> bring (binary) Reproducible Builds plugins to general availability
>> Really, really nice work..
> thank you, it required a lot of energy for a long period of time...
>
>>
>> I do like <project.build.outputTimestamp>. Why did you make it a String?
>> Why not go directly with Instant? It gives your ISO 8601 for free
> I tried to explain it in https://issues.apache.org/jira/browse/MSHARED-837
> Plexus Date injection support is really limited: could not match the
> requirements
yyyy-MM-dd'T'HH:mm:ssXXX and SimpleDateFormat will do the trick. It also
will require to change the converter of course.
Alternatively, you could try
https://commons.apache.org/proper/commons-lang/javadocs/api-release/src-html/org/apache/commons/lang3/time/DateFormatUtils.html#line.72
which I have added long time ago.
Moreover, I'd like these updated plugins be part of MNG-6169.
Michael
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: last review of Reproducible Builds proposal
Posted by Michael Osipov <mi...@apache.org>.
Am 2019-10-05 um 19:52 schrieb Hervé BOUTEMY:
> based on the feedback I got, I updated the proposal:
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=74682318
>
> The archives entries timestamp is now configured with project.build.outputTimestamp property, in ISO-8601 format
> <project>
> <properties>
> <project.build.outputTimestamp>2019-10-02T08:04:00Z</project.build.outputTimestamp>
> </properties>
> </project>
>
> The shared components, plugins, parent poms and Maven core branches have been updated to match this new proposal
>
>
> If no one objects, next week-end, I'll start the (heavy) release train to bring (binary) Reproducible Builds plugins to general availability
Really, really nice work..
I do like <project.build.outputTimestamp>. Why did you make it a String?
Why not go directly with Instant? It gives your ISO 8601 for free
Michael
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: last review of Reproducible Builds proposal
Posted by Emmanuel Bourg <eb...@apache.org>.
Le 07/10/2019 à 14:40, Andreas Sewe a écrit :
> - Can we get something analogous to maven.build.timestamp.format? One
> could then write the following to be compatible with env.SOURCE_DATE_EPOCH:
>
> <project>
> <properties>
>
> <project.build.outputTimestamp>${env.SOURCE_DATE_EPOCH}<project.build.outputTimestamp>
>
> <project.build.outputTimestamp.format>...</project.build.outputTimestamp.format>
> </properties>
> </project>
+1, some kind of support for SOURCE_DATE_EPOCH would be nice. Either
this (but maybe with only one property supporting both formats) or by
overriding automatically the property when SOURCE_DATE_EPOCH is set.
Emmanuel Bourg
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: last review of Reproducible Builds proposal
Posted by Andreas Sewe <se...@st.informatik.tu-darmstadt.de>.
Hi,
Hervé BOUTEMY wrote:
> based on the feedback I got, I updated the proposal:
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=74682318
>
> The archives entries timestamp is now configured with project.build.outputTimestamp property, in ISO-8601 format
> <project>
> <properties>
> <project.build.outputTimestamp>2019-10-02T08:04:00Z</project.build.outputTimestamp>
> </properties>
> </project>
nice work, Hervé.
Two comments:
- We already have maven.build.timestamp [1], so I wonder whether it
should be maven.build.outputTimestap, too. Although one may argue that
this *output* timestamp is a property of the project being built, not
the Maven execution (in particular if the property becomes a POM element
in POM version 5).
- Can we get something analogous to maven.build.timestamp.format? One
could then write the following to be compatible with env.SOURCE_DATE_EPOCH:
<project>
<properties>
<project.build.outputTimestamp>${env.SOURCE_DATE_EPOCH}<project.build.outputTimestamp>
<project.build.outputTimestamp.format>...</project.build.outputTimestamp.format>
</properties>
</project>
Unfortunately, SimpleDateFormat or its java.time equivalents do not seem
to have a format string for "seconds since the epoch", so we may want to
make "seconds since the epoch" the default for
project.build.outputTimestamp's format and only parse according to
project.build.outputTimestamp.format when that property is indeed set.
WDYT?
Best wishes.
Andreas
<https://maven.apache.org/guides/introduction/introduction-to-the-pom.html#Available_Variables>
Re: last review of Reproducible Builds proposal
Posted by Hervé BOUTEMY <he...@free.fr>.
Le dimanche 6 octobre 2019, 16:41:27 CEST Emmanuel Bourg a écrit :
> Le 06/10/2019 à 09:43, Hervé BOUTEMY a écrit :
> > Notice that you can also express a timezone (as digits), as seen in the
> > unit tests.
> I know but that's not desirable, otherwise the formatted timestamp would
> depend on the timezone of the developer and that would harm the
> reproducibility of the pom (I assume the timestamp is added
> automatically to the pom during the build, is that right?).
>
> >> Would it be possible to prevent this property from being inherited?
> >
> > AFAIK no.
> > And I find that default inheritance from parent to child is a nice
> > feature.
>
> Why do you think this is a nice feature? I can only see the downside of
> child projects having a timestamp clamped in the past and leaving the
> developers clueless about the reason. This would be especially bad if a
> widely used parent pom like org.sonatype.oss:oss-parent or
> org.apache:apache had it.
choosing to add Reproducible Builds configuration (ie output timestamp) in a
parent is a strong choice, because it will inherit to every child.
Looks like the current fear of many things means we won't put the configuration
in ASF parent for now until parent learn by real experience how it works well:
next ASF parent POM release will use plugins versions that are able to support
Reproducible Builds, but won't configure a timestamp to activate the feature.
It will be an opt-in option for projects that like this idea.
Perhaps in a later version, experience will be sufficient to configure the
timestamp by default, and always keep the opt-out option when some project has
issues
>
> Emmanuel Bourg
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> For additional commands, e-mail: dev-help@maven.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: last review of Reproducible Builds proposal
Posted by Hervé BOUTEMY <he...@free.fr>.
Le dimanche 6 octobre 2019, 22:18:59 CEST Emmanuel Bourg a écrit :
> Le 06/10/2019 à 20:13, Hervé BOUTEMY a écrit :
> > no, it does not add any dependency on developer configuration:
> > 2019-10-05T18:37:42Z == 2019-10-05T20:37:42+02:00 ==
> > 2019-10-05T16:37:42-02:00
> yes but:
>
> "2019-10-05T18:37:42Z" != "2019-10-05T20:37:42+02:00" !=
> "2019-10-05T16:37:42-02:00"
>
> The point is, two developers may generate a different pom if the local
> timezone is used. A fixed timezone is necessary to ensure the pom itself
> is reproducible.
There is a misunderstanding here: the pom.xml is saved in the source control,
with the timestamp in it.
There is no question of "reproducible pom.xml"
>
> > when will this value be written in the pom.xml is independant: currently,
> > in my PoC, I wrote the values by hand. In the future, it will probably be
> > updated by maven-release-plugin, and we'll have to choose if the
> > timestamp is written in Z or if it is written in local timezone with its
> > offset: both ways of expressing the timestamp are valid and will give
> > reproducible result
> The jar generated is reproducible, but not the pom. I suspect that if
> the jar includes the pom it'll break its reproducibility too (this is
> the default for maven-jar-plugin, but I don't know if it embeds the
> original pom without the timestamp, or the generated pom with the
> timestamp).
>
> > once again, war files taken apart for web servers, who looks at timestamp
> > in zip files?
>
> archive timestamps are just the tip of the iceberg. There are more
> visible timestamps elsewhere, for example in the javadoc headers, in
> .properties files, in OSGi attributes, sometimes in the source files...
Sure, many plugins have already been modified to remove such noise in output,
and probably others will require to be updated. Because in Reproducible
Builds, timestamps to know when something has been generated is less useful:
Reproduciblity is the ultimate proof of knowledge of what has been built,
since the result will be the same now, tomorrow, in one year...
Regards,
Hervé
>
> Emmanuel Bourg
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> For additional commands, e-mail: dev-help@maven.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: last review of Reproducible Builds proposal
Posted by Romain Manni-Bucau <rm...@gmail.com>.
Small reminder: if you want to be reproducible you must fix the timestamp
so whatever zone, whatever format works. It is common to use new Date(1000)
in utc but not important at the end.
Side note: same applies for most of the env though (locale for ex.).
Le dim. 6 oct. 2019 à 22:57, Tibor Digana <ti...@apache.org> a écrit :
> Hi Emmanuel,
>
> >> The point is, two developers may generate a different pom if the local
> timezone is used.
>
> very well explained! It could not be better: utc time is the same however
> the text is different and that's important for the stream identity.
> Karl had a proposal with additional property "format" as a compromise.
> Is the geography a localization important information for the behavior of
> reproducible build? IMO, it is not!
> So why not to have Karl's format NULL by default, means the UTC.
>
> btw, there is whole bunch of problems with time with/out TZ in webapps and
> rdbms databases and cloud system and distributed HTTP clients but that's
> another story.
>
>
> >> I suspect that if the jar includes the pom it'll break its
> reproducibility too
>
> Actually, there is "pom.properties" in META-INF folder of the JAR archive
> artifact and this file containes the comment having the system time
> generated by JDK.
> There's a PR for this issue. The POM and the properties file are both
> embedded in META-INF.
>
>
> Cheers
> Tibor17
>
> On Sun, Oct 6, 2019 at 10:19 PM Emmanuel Bourg <eb...@apache.org> wrote:
>
> > Le 06/10/2019 à 20:13, Hervé BOUTEMY a écrit :
> >
> > > no, it does not add any dependency on developer configuration:
> > > 2019-10-05T18:37:42Z == 2019-10-05T20:37:42+02:00 ==
> > 2019-10-05T16:37:42-02:00
> >
> > yes but:
> >
> > "2019-10-05T18:37:42Z" != "2019-10-05T20:37:42+02:00" !=
> > "2019-10-05T16:37:42-02:00"
> >
> > The point is, two developers may generate a different pom if the local
> > timezone is used. A fixed timezone is necessary to ensure the pom itself
> > is reproducible.
> >
> >
> > > when will this value be written in the pom.xml is independant:
> > currently, in
> > > my PoC, I wrote the values by hand. In the future, it will probably be
> > updated
> > > by maven-release-plugin, and we'll have to choose if the timestamp is
> > written
> > > in Z or if it is written in local timezone with its offset: both ways
> of
> > > expressing the timestamp are valid and will give reproducible result
> >
> > The jar generated is reproducible, but not the pom. I suspect that if
> > the jar includes the pom it'll break its reproducibility too (this is
> > the default for maven-jar-plugin, but I don't know if it embeds the
> > original pom without the timestamp, or the generated pom with the
> > timestamp).
> >
> >
> > > once again, war files taken apart for web servers, who looks at
> > timestamp in
> > > zip files?
> >
> > archive timestamps are just the tip of the iceberg. There are more
> > visible timestamps elsewhere, for example in the javadoc headers, in
> > .properties files, in OSGi attributes, sometimes in the source files...
> >
> > Emmanuel Bourg
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> > For additional commands, e-mail: dev-help@maven.apache.org
> >
> >
>
Re: last review of Reproducible Builds proposal
Posted by Tibor Digana <ti...@apache.org>.
Hi Emmanuel,
>> The point is, two developers may generate a different pom if the local
timezone is used.
very well explained! It could not be better: utc time is the same however
the text is different and that's important for the stream identity.
Karl had a proposal with additional property "format" as a compromise.
Is the geography a localization important information for the behavior of
reproducible build? IMO, it is not!
So why not to have Karl's format NULL by default, means the UTC.
btw, there is whole bunch of problems with time with/out TZ in webapps and
rdbms databases and cloud system and distributed HTTP clients but that's
another story.
>> I suspect that if the jar includes the pom it'll break its
reproducibility too
Actually, there is "pom.properties" in META-INF folder of the JAR archive
artifact and this file containes the comment having the system time
generated by JDK.
There's a PR for this issue. The POM and the properties file are both
embedded in META-INF.
Cheers
Tibor17
On Sun, Oct 6, 2019 at 10:19 PM Emmanuel Bourg <eb...@apache.org> wrote:
> Le 06/10/2019 à 20:13, Hervé BOUTEMY a écrit :
>
> > no, it does not add any dependency on developer configuration:
> > 2019-10-05T18:37:42Z == 2019-10-05T20:37:42+02:00 ==
> 2019-10-05T16:37:42-02:00
>
> yes but:
>
> "2019-10-05T18:37:42Z" != "2019-10-05T20:37:42+02:00" !=
> "2019-10-05T16:37:42-02:00"
>
> The point is, two developers may generate a different pom if the local
> timezone is used. A fixed timezone is necessary to ensure the pom itself
> is reproducible.
>
>
> > when will this value be written in the pom.xml is independant:
> currently, in
> > my PoC, I wrote the values by hand. In the future, it will probably be
> updated
> > by maven-release-plugin, and we'll have to choose if the timestamp is
> written
> > in Z or if it is written in local timezone with its offset: both ways of
> > expressing the timestamp are valid and will give reproducible result
>
> The jar generated is reproducible, but not the pom. I suspect that if
> the jar includes the pom it'll break its reproducibility too (this is
> the default for maven-jar-plugin, but I don't know if it embeds the
> original pom without the timestamp, or the generated pom with the
> timestamp).
>
>
> > once again, war files taken apart for web servers, who looks at
> timestamp in
> > zip files?
>
> archive timestamps are just the tip of the iceberg. There are more
> visible timestamps elsewhere, for example in the javadoc headers, in
> .properties files, in OSGi attributes, sometimes in the source files...
>
> Emmanuel Bourg
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> For additional commands, e-mail: dev-help@maven.apache.org
>
>
Re: last review of Reproducible Builds proposal
Posted by Emmanuel Bourg <eb...@apache.org>.
Le 06/10/2019 à 20:13, Hervé BOUTEMY a écrit :
> no, it does not add any dependency on developer configuration:
> 2019-10-05T18:37:42Z == 2019-10-05T20:37:42+02:00 == 2019-10-05T16:37:42-02:00
yes but:
"2019-10-05T18:37:42Z" != "2019-10-05T20:37:42+02:00" !=
"2019-10-05T16:37:42-02:00"
The point is, two developers may generate a different pom if the local
timezone is used. A fixed timezone is necessary to ensure the pom itself
is reproducible.
> when will this value be written in the pom.xml is independant: currently, in
> my PoC, I wrote the values by hand. In the future, it will probably be updated
> by maven-release-plugin, and we'll have to choose if the timestamp is written
> in Z or if it is written in local timezone with its offset: both ways of
> expressing the timestamp are valid and will give reproducible result
The jar generated is reproducible, but not the pom. I suspect that if
the jar includes the pom it'll break its reproducibility too (this is
the default for maven-jar-plugin, but I don't know if it embeds the
original pom without the timestamp, or the generated pom with the
timestamp).
> once again, war files taken apart for web servers, who looks at timestamp in
> zip files?
archive timestamps are just the tip of the iceberg. There are more
visible timestamps elsewhere, for example in the javadoc headers, in
.properties files, in OSGi attributes, sometimes in the source files...
Emmanuel Bourg
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: last review of Reproducible Builds proposal
Posted by Hervé BOUTEMY <he...@free.fr>.
Le dimanche 6 octobre 2019, 16:41:27 CEST Emmanuel Bourg a écrit :
> Le 06/10/2019 à 09:43, Hervé BOUTEMY a écrit :
> > Notice that you can also express a timezone (as digits), as seen in the
> > unit tests.
> I know but that's not desirable, otherwise the formatted timestamp would
> depend on the timezone of the developer
no, it does not add any dependency on developer configuration:
2019-10-05T18:37:42Z == 2019-10-05T20:37:42+02:00 == 2019-10-05T16:37:42-02:00
> and that would harm the
> reproducibility of the pom (I assume the timestamp is added
> automatically to the pom during the build, is that right?).
when will this value be written in the pom.xml is independant: currently, in
my PoC, I wrote the values by hand. In the future, it will probably be updated
by maven-release-plugin, and we'll have to choose if the timestamp is written
in Z or if it is written in local timezone with its offset: both ways of
expressing the timestamp are valid and will give reproducible result
>
> >> Would it be possible to prevent this property from being inherited?
> >
> > AFAIK no.
> > And I find that default inheritance from parent to child is a nice
> > feature.
>
> Why do you think this is a nice feature? I can only see the downside of
> child projects having a timestamp clamped in the past and leaving the
> developers clueless about the reason. This would be especially bad if a
> widely used parent pom like org.sonatype.oss:oss-parent or
> org.apache:apache had it.
once again, war files taken apart for web servers, who looks at timestamp in
zip files?
>
> Emmanuel Bourg
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> For additional commands, e-mail: dev-help@maven.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: last review of Reproducible Builds proposal
Posted by Emmanuel Bourg <eb...@apache.org>.
Le 06/10/2019 à 09:43, Hervé BOUTEMY a écrit :
> Notice that you can also express a timezone (as digits), as seen in the unit tests.
I know but that's not desirable, otherwise the formatted timestamp would
depend on the timezone of the developer and that would harm the
reproducibility of the pom (I assume the timestamp is added
automatically to the pom during the build, is that right?).
>> Would it be possible to prevent this property from being inherited?
> AFAIK no.
> And I find that default inheritance from parent to child is a nice feature.
Why do you think this is a nice feature? I can only see the downside of
child projects having a timestamp clamped in the past and leaving the
developers clueless about the reason. This would be especially bad if a
widely used parent pom like org.sonatype.oss:oss-parent or
org.apache:apache had it.
Emmanuel Bourg
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: last review of Reproducible Builds proposal
Posted by Hervé BOUTEMY <he...@free.fr>.
Le samedi 5 octobre 2019, 23:15:58 CEST Emmanuel Bourg a écrit :
> Le 05/10/2019 à 19:52, Hervé BOUTEMY a écrit :
> > based on the feedback I got, I updated the proposal:
> > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=74682318
> >
> > The archives entries timestamp is now configured with
> > project.build.outputTimestamp property, in ISO-8601 format <project>
> >
> > <properties>
> >
> > <project.build.outputTimestamp>2019-10-02T08:04:00Z</project.build.out
> > putTimestamp>>
> > </properties>
> >
> > </project>
>
> You may want to specify explicitly that the timestamp must be formatted
> with the UTC timezone (as in your example).
the current documentation on the plugin parameter is "Timestamp for reproducible archive entries."
https://github.com/apache/maven-source-plugin/commit/1279fe02564d805e21d75cbfc8408178190fe12a
Don't hesitate to propose a better explanation: once ok, I'll add the result to the proposal, to be easily copy/pasted to every packaging plugin.
Notice that you can also express a timezone (as digits), as seen in the unit tests.
>
> Would it be possible to prevent this property from being inherited?
AFAIK no.
And I find that default inheritance from parent to child is a nice feature.
What I added, in specific case where inheritance could cause an issue, is that you can disable the Reproducible Builds configuration by setting a 1 character value, whatever the character is.
This is the only trick I found to have inheritance not ignore child pom's value: tested also in the unit tests
>
> Emmanuel Bourg
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> For additional commands, e-mail: dev-help@maven.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: last review of Reproducible Builds proposal
Posted by Hervé BOUTEMY <he...@free.fr>.
Le dimanche 6 octobre 2019, 12:19:35 CEST Michael Osipov a écrit :
> Am 2019-10-05 um 23:15 schrieb Emmanuel Bourg:
> > Le 05/10/2019 à 19:52, Hervé BOUTEMY a écrit :
> >> based on the feedback I got, I updated the proposal:
> >> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=74682318
> >>
> >> The archives entries timestamp is now configured with
> >> project.build.outputTimestamp property, in ISO-8601 format <project>
> >>
> >> <properties>
> >>
> >> <project.build.outputTimestamp>2019-10-02T08:04:00Z</project.build.o
> >> utputTimestamp>>>
> >> </properties>
> >>
> >> </project>
> >
> > You may want to specify explicitly that the timestamp must be formatted
> > with the UTC timezone (as in your example).
>
> No, if you have a decent format, like ISO 8601, it is regardless of the
> timezone offset because it is properly parsed to.
wrong reasoning: ISO 8601 format supports many ways of expressing a timestamp.
If you want to express a timestamp with a timezone offset, why not, it's
perfectly legitimate to do so, even if it is not your personal taste
>
> Michael
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> For additional commands, e-mail: dev-help@maven.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: last review of Reproducible Builds proposal
Posted by Michael Osipov <mi...@apache.org>.
Am 2019-10-05 um 23:15 schrieb Emmanuel Bourg:
> Le 05/10/2019 à 19:52, Hervé BOUTEMY a écrit :
>> based on the feedback I got, I updated the proposal:
>> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=74682318
>>
>> The archives entries timestamp is now configured with project.build.outputTimestamp property, in ISO-8601 format
>> <project>
>> <properties>
>> <project.build.outputTimestamp>2019-10-02T08:04:00Z</project.build.outputTimestamp>
>> </properties>
>> </project>
>
> You may want to specify explicitly that the timestamp must be formatted
> with the UTC timezone (as in your example).
No, if you have a decent format, like ISO 8601, it is regardless of the
timezone offset because it is properly parsed to.
Michael
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: last review of Reproducible Builds proposal
Posted by Emmanuel Bourg <eb...@apache.org>.
Le 05/10/2019 à 19:52, Hervé BOUTEMY a écrit :
> based on the feedback I got, I updated the proposal:
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=74682318
>
> The archives entries timestamp is now configured with project.build.outputTimestamp property, in ISO-8601 format
> <project>
> <properties>
> <project.build.outputTimestamp>2019-10-02T08:04:00Z</project.build.outputTimestamp>
> </properties>
> </project>
You may want to specify explicitly that the timestamp must be formatted
with the UTC timezone (as in your example).
Would it be possible to prevent this property from being inherited?
Emmanuel Bourg
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: last review of Reproducible Builds proposal
Posted by Michael Osipov <mi...@apache.org>.
Am 2019-10-05 um 19:52 schrieb Hervé BOUTEMY:
> based on the feedback I got, I updated the proposal:
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=74682318
>
> The archives entries timestamp is now configured with project.build.outputTimestamp property, in ISO-8601 format
> <project>
> <properties>
> <project.build.outputTimestamp>2019-10-02T08:04:00Z</project.build.outputTimestamp>
> </properties>
> </project>
>
> The shared components, plugins, parent poms and Maven core branches have been updated to match this new proposal
>
>
> If no one objects, next week-end, I'll start the (heavy) release train to bring (binary) Reproducible Builds plugins to general availability
One more thing: if there is a slight chance that more properties will be
added in the future, it'd be worth to group them into
<project.build.reproducible. ...>
Michael
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: last review of Reproducible Builds proposal
Posted by Hervé BOUTEMY <he...@free.fr>.
Yes, having the timestamp to 0 is something I wanted generally to avoid.
But this has opened another question: use what value? Can this be automated?
I knew that war files could be a specific use case.
Perhaps this plugin requires a specific way of handling reproducibility, even
more than the standard in the proposal "define a timestamp in pom.xml"?
Do you want that I create a Reproducible branch in mave-war-plugin like I did
for Jar, Source and Assembly, so we can test?
Regards,
Hervé
Le samedi 5 octobre 2019, 22:19:54 CEST Vladimir Sitnikov a écrit :
> >but
> >who really looks at the timestamp of entries in release zips/jars/tar.gz
> >honestly?
>
> Tomcat when it decides on what to send in the "Last-Modified" header.
> For instance, current Gradle does not allow to configure the timestamp, and
> for reproducible builds it always sets the timestamp to 0 or so.
> It breaks Tomcat's assumptions:
> https://github.com/gradle/gradle/issues/10917
>
> Vladimir
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: last review of Reproducible Builds proposal
Posted by Vladimir Sitnikov <si...@gmail.com>.
>but
>who really looks at the timestamp of entries in release zips/jars/tar.gz
>honestly?
Tomcat when it decides on what to send in the "Last-Modified" header.
For instance, current Gradle does not allow to configure the timestamp, and
for reproducible builds it always sets the timestamp to 0 or so.
It breaks Tomcat's assumptions:
https://github.com/gradle/gradle/issues/10917
Vladimir
Re: last review of Reproducible Builds proposal
Posted by Hervé BOUTEMY <he...@free.fr>.
Le mardi 8 octobre 2019, 23:42:55 CEST Mark Derricutt a écrit :
> On 6 Oct 2019, at 9:14, Hervé BOUTEMY wrote:
> > if anybody cares about the exact value: but
> > who really looks at the timestamp of entries in release zips/jars/tar.gz
> > honestly?
>
> I've actually done so in the past trying to find differences between two
> versions of a jar for repair reasons.
>
> VERY infrequent tho - if you're wanting to generate a delta-patch between
> two jars/zips then I guess it might also be handy (altho you might be
> better off going direct for checksumdifferences there ).
FYI, when you try to know the difference between 2 archives, the ideal tool is
diffoscope [1]: it has been done exactly for that
[1] https://diffoscope.org/
>
>
> ---
> "The ease with which a change can be implemented has no relevance at all to
> whether it is the right change for the (Java) Platform for all time."
> — Mark Reinhold.
>
> Mark Derricutt
> http://www.theoryinpractice.net
> http://www.chaliceofblood.net
> http://plus.google.com/+MarkDerricutt
> http://twitter.com/talios
> http://facebook.com/mderricutt
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: last review of Reproducible Builds proposal
Posted by Mark Derricutt <ma...@talios.com>.
On 6 Oct 2019, at 9:14, Hervé BOUTEMY wrote:
> if anybody cares about the exact value: but
> who really looks at the timestamp of entries in release zips/jars/tar.gz
> honestly?
I've actually done so in the past trying to find differences between two versions of a jar for repair reasons.
VERY infrequent tho - if you're wanting to generate a delta-patch between two jars/zips then I guess it might also be handy (altho you might be better off going direct for checksumdifferences there ).
---
"The ease with which a change can be implemented has no relevance at all to whether it is the right change for the (Java) Platform for all time." — Mark Reinhold.
Mark Derricutt
http://www.theoryinpractice.net
http://www.chaliceofblood.net
http://plus.google.com/+MarkDerricutt
http://twitter.com/talios
http://facebook.com/mderricutt
Re: last review of Reproducible Builds proposal
Posted by Hervé BOUTEMY <he...@free.fr>.
with inheritance from parent, this question disappears: once parent pom has a
timestamp value to have a reproducible release, child poms inherit the value
and the reproducible feature
changing the value in child is just a little improvement to get a timestamp
that has more meaningful value, if anybody cares about the exact value: but
who really looks at the timestamp of entries in release zips/jars/tar.gz
honestly?
Regards,
Hervé
Le samedi 5 octobre 2019, 20:42:04 CEST Tibor Digana a écrit :
> Hi Herve,
>
> I want to make sure we understand correctly. So. What you want to achieve
> with this property is to stick the property to one fixed value when the
> user has supposed the archive would have same content. And opposite, means
> that the property would be real when the content of the archive should
> change.
>
> That's technically ok that the plugins are aware of shared time stamp, but
> this looks like the problem with egg and chicken. Practically, the user
> would never be sure when is the time that the time stamp should be fixed
> value. Not sure if the user would be able to have fully working
> reproducible build.
> Perhaps the users will have exactly this question too.
>
> Cheers
> Tibor17
>
> On Sat, Oct 5, 2019 at 7:52 PM Hervé BOUTEMY <he...@free.fr> wrote:
> > based on the feedback I got, I updated the proposal:
> > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=74682318
> >
> > The archives entries timestamp is now configured with
> > project.build.outputTimestamp property, in ISO-8601 format
> > <project>
> >
> > <properties>
> >
> > <project.build.outputTimestamp>2019-10-02T08:04:00Z</project.build.outputT
> > imestamp>>
> > </properties>
> >
> > </project>
> >
> > The shared components, plugins, parent poms and Maven core branches have
> > been updated to match this new proposal
> >
> >
> > If no one objects, next week-end, I'll start the (heavy) release train to
> > bring (binary) Reproducible Builds plugins to general availability
> >
> > Regards,
> >
> > Hervé
> >
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> > For additional commands, e-mail: dev-help@maven.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: last review of Reproducible Builds proposal
Posted by Tibor Digana <ti...@apache.org>.
Hi Herve,
I want to make sure we understand correctly. So. What you want to achieve
with this property is to stick the property to one fixed value when the
user has supposed the archive would have same content. And opposite, means
that the property would be real when the content of the archive should
change.
That's technically ok that the plugins are aware of shared time stamp, but
this looks like the problem with egg and chicken. Practically, the user
would never be sure when is the time that the time stamp should be fixed
value. Not sure if the user would be able to have fully working
reproducible build.
Perhaps the users will have exactly this question too.
Cheers
Tibor17
On Sat, Oct 5, 2019 at 7:52 PM Hervé BOUTEMY <he...@free.fr> wrote:
> based on the feedback I got, I updated the proposal:
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=74682318
>
> The archives entries timestamp is now configured with
> project.build.outputTimestamp property, in ISO-8601 format
> <project>
> <properties>
>
> <project.build.outputTimestamp>2019-10-02T08:04:00Z</project.build.outputTimestamp>
> </properties>
> </project>
>
> The shared components, plugins, parent poms and Maven core branches have
> been updated to match this new proposal
>
>
> If no one objects, next week-end, I'll start the (heavy) release train to
> bring (binary) Reproducible Builds plugins to general availability
>
> Regards,
>
> Hervé
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> For additional commands, e-mail: dev-help@maven.apache.org
>
>
Re: last review of Reproducible Builds proposal
Posted by Andreas Sewe <se...@st.informatik.tu-darmstadt.de>.
Hi,
> but the format of the timestamp is completely different ... doesn't that matter?
I don't think so; at least I don't think the Bnd-LastModified header is
meant to be parsed by machines.
But I have _removeheaders>Bnd-*</_removeheaders> (yes, wildcards are
supported) in my <instructions>, so what do I know...
> I was currently going for the option number 1 with removing the header.
>
> In order to be 100% happy with this, I would prefer a setup where the normal mechanisms are used if no maven.build.outputTimestap is defined, but if it is (because a future version of the maven release plugin set it there) it uses that.
You could use a <profile> that is activated when
maven.build.outputTimestap is defined (or not defined) to implement this
behavior. I haven't tested this, however.
Best wishes,
Andreas
AW: last review of Reproducible Builds proposal
Posted by Christofer Dutz <ch...@c-ware.de>.
Hi all,
And I might want to add, that includes pom, jar, sources jar, javadoc, release-asembly and fat-jars ...
Currently applying these changes to other apache projects as an experiment.
So is anyone already working on the release plugin?
Chris
________________________________
Von: Christofer Dutz <ch...@c-ware.de>
Gesendet: Montag, 4. November 2019 18:58:10
An: Maven Developers List <de...@maven.apache.org>
Betreff: Re: last review of Reproducible Builds proposal
Hi all,
so today I made sure the LastModified and the Creator username are omitted and now the Apache PLC4X build had a 100% reproducible build (at least on my one machine) ... will be checking this out on other machines.
Chris
Am 04.11.19, 10:12 schrieb "Christofer Dutz" <ch...@c-ware.de>:
Hi Andreas,
but the format of the timestamp is completely different ... doesn't that matter?
I was currently going for the option number 1 with removing the header.
In order to be 100% happy with this, I would prefer a setup where the normal mechanisms are used if no maven.build.outputTimestap is defined, but if it is (because a future version of the maven release plugin set it there) it uses that.
Will try out your suggestions as soon as I'm able to build again (I unfortunately installed that Mac OS update ... now things aren't working as they should be)
Chris
Am 04.11.19, 09:42 schrieb "Andreas Sewe" <se...@st.informatik.tu-darmstadt.de>:
Christofer Dutz wrote:
> Well I have a new candidate:
>
> <artifactId>maven-bundle-plugin</artifactId>
>
> Creates: Bnd-LastModified in the MANIFEST.MF
>
> Gotta find out a way to either suppress that entry (Would be great if it could consume the same property)
<configuration>
<instructions>
<_removeheaders>Bnd-LastModified</_removeheaders>
</instructions>
<configuration>
Alternatively
<configuration>
<instructions>
<Bnd-LastModified>${maven.build.outputTimestap}</Bnd-LastModified>
</instructions>
<configuration
Hope that helps,
Andreas
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: last review of Reproducible Builds proposal
Posted by Christofer Dutz <ch...@c-ware.de>.
Hi all,
so today I made sure the LastModified and the Creator username are omitted and now the Apache PLC4X build had a 100% reproducible build (at least on my one machine) ... will be checking this out on other machines.
Chris
Am 04.11.19, 10:12 schrieb "Christofer Dutz" <ch...@c-ware.de>:
Hi Andreas,
but the format of the timestamp is completely different ... doesn't that matter?
I was currently going for the option number 1 with removing the header.
In order to be 100% happy with this, I would prefer a setup where the normal mechanisms are used if no maven.build.outputTimestap is defined, but if it is (because a future version of the maven release plugin set it there) it uses that.
Will try out your suggestions as soon as I'm able to build again (I unfortunately installed that Mac OS update ... now things aren't working as they should be)
Chris
Am 04.11.19, 09:42 schrieb "Andreas Sewe" <se...@st.informatik.tu-darmstadt.de>:
Christofer Dutz wrote:
> Well I have a new candidate:
>
> <artifactId>maven-bundle-plugin</artifactId>
>
> Creates: Bnd-LastModified in the MANIFEST.MF
>
> Gotta find out a way to either suppress that entry (Would be great if it could consume the same property)
<configuration>
<instructions>
<_removeheaders>Bnd-LastModified</_removeheaders>
</instructions>
<configuration>
Alternatively
<configuration>
<instructions>
<Bnd-LastModified>${maven.build.outputTimestap}</Bnd-LastModified>
</instructions>
<configuration
Hope that helps,
Andreas
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: last review of Reproducible Builds proposal
Posted by Christofer Dutz <ch...@c-ware.de>.
Hi Andreas,
but the format of the timestamp is completely different ... doesn't that matter?
I was currently going for the option number 1 with removing the header.
In order to be 100% happy with this, I would prefer a setup where the normal mechanisms are used if no maven.build.outputTimestap is defined, but if it is (because a future version of the maven release plugin set it there) it uses that.
Will try out your suggestions as soon as I'm able to build again (I unfortunately installed that Mac OS update ... now things aren't working as they should be)
Chris
Am 04.11.19, 09:42 schrieb "Andreas Sewe" <se...@st.informatik.tu-darmstadt.de>:
Christofer Dutz wrote:
> Well I have a new candidate:
>
> <artifactId>maven-bundle-plugin</artifactId>
>
> Creates: Bnd-LastModified in the MANIFEST.MF
>
> Gotta find out a way to either suppress that entry (Would be great if it could consume the same property)
<configuration>
<instructions>
<_removeheaders>Bnd-LastModified</_removeheaders>
</instructions>
<configuration>
Alternatively
<configuration>
<instructions>
<Bnd-LastModified>${maven.build.outputTimestap}</Bnd-LastModified>
</instructions>
<configuration
Hope that helps,
Andreas
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: last review of Reproducible Builds proposal
Posted by Andreas Sewe <se...@st.informatik.tu-darmstadt.de>.
Christofer Dutz wrote:
> Well I have a new candidate:
>
> <artifactId>maven-bundle-plugin</artifactId>
>
> Creates: Bnd-LastModified in the MANIFEST.MF
>
> Gotta find out a way to either suppress that entry (Would be great if it could consume the same property)
<configuration>
<instructions>
<_removeheaders>Bnd-LastModified</_removeheaders>
</instructions>
<configuration>
Alternatively
<configuration>
<instructions>
<Bnd-LastModified>${maven.build.outputTimestap}</Bnd-LastModified>
</instructions>
<configuration
Hope that helps,
Andreas
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: last review of Reproducible Builds proposal
Posted by Christofer Dutz <ch...@c-ware.de>.
Well I have a new candidate:
<artifactId>maven-bundle-plugin</artifactId>
Creates: Bnd-LastModified in the MANIFEST.MF
Gotta find out a way to either suppress that entry (Would be great if it could consume the same property)
Chris
Am 03.11.19, 20:25 schrieb "herve.boutemy@free.fr" <he...@free.fr>:
ok, Reproducible Builds are not so easy to get: each plugin that you use can cause an issue
I really recommend you diffoscope to investigate differences, it really helps a lot by easily giving you precise differences
Good luck for the investigations :)
Regards,
Hervé
----- Mail original -----
De: "Christofer Dutz" <ch...@c-ware.de>
À: "Maven Developers List" <de...@maven.apache.org>
Envoyé: Dimanche 3 Novembre 2019 20:16:42
Objet: Re: last review of Reproducible Builds proposal
Hi Herve,
unfortunately I now have implemented some tooling to compare the stuff produced by the updated reproducible builds.
And it does show quite a number of non binary equal files.
Will investigate what's the difference.
Chris
Am 03.11.19, 11:08 schrieb "Hervé BOUTEMY" <he...@free.fr>:
great feedback, thank you: this proves the initial set of only 3 plugins is a
good basis.
For checking the many output artifacts from a build, future buildinfo file [1]
should help a lot. I still have many usability concerns for Maven multi-module
builds (starting with: should we generate only the root buildinfo or one
buildinfo per Maven module?)
Let's keep the ball rolling: today, it's plugins release day!!!
Regards,
Hervé
[1] https://reproducible-builds.org/docs/jvm/
Le vendredi 1 novembre 2019, 13:34:32 CET Christofer Dutz a écrit :
> Hi,
>
> so I just updated the versions of the 3 plugins and gave the Apache PLC4X
> build a little spin ...
> https://github.com/apache/plc4x/tree/feature/reproducible-builds
>
> Did two executions of this:
> mvn -U -P
> with-boost,with-java,with-dotnet,with-cpp,with-python,with-proxies,with-san
> dbox,with
> -DaltDeploymentRepository=snapshot-repo::default::file:./local-snapshots-di
> r clean deploy
(but with a different second local repo location)
>
> Then I used "diff" and "cmp" to compare individual files and it didn't show
> differences with the artifacts I chose ...
now I guess I'll have to whip
> up some little bash script to sort of compare the artifacts from one
> directory with the other (Unfortunately the SNAPSHOT timestamps are a
> little annoying :-/
> We do have some C++, C# and Python modules ... but I wouldn't expect the C++
> and C# to be reproducible.
> Chris
>
>
> Am 01.11.19, 12:04 schrieb "Hervé BOUTEMY" <he...@free.fr>:
>
> Le vendredi 1 novembre 2019, 09:40:40 CET Christofer Dutz a écrit :
>
> > Hi Herve,
> >
> > thanks for that will try it asap and report any findings back.
> >
> > But good to know that there is a difference between JDK major versions
> > and
> > OSes ... so it would probably be best to stage releases on Linux with
> > an
> > OpenJDK of the minimum supported version?
> > Just thinking how to make it
> > possible to verify without having to buy Mac or Windows licenses ...
> > guess
> > on every machine you could whip up a Ubuntu VM for verification. Just
> > thinking about it ... perhaps it would be best to create a Docker
> > image for
doing the reproducible stuff ...
>
> just to be clear: the difference is on newline only, then Windows vs
> anything
else. You get the same result on Linux, MacOS, FreeBSD, or any
> other Unix.
> And if I want to be complete, if you get a source tarball from Windows,
>
extract it to a Linux/MacOS/... box and build with "mvn -
> Dline.separator='\r\n'", in general, you get the same result as a build
> under
Windows: I tested multiple times, it worked, but we'll see if it
> works always or just "in general"
>
>
> > Are there any plans on creating a plugin to allow verification?
> >
> > Sort of something like this:
> > "mvn package release:verify-reproduicble
> > -DstagingRepoUrl=a.b.c.de/repo/blahblahblah"
> > (Which doesn't deploy the
> > artifacts, but instead download them and do a binary comparison)
>
> yes, but for now it was completely another form: this is the "Buildinfo
> file"
proposal https://reproducible-builds.org/docs/jvm/
> that I stopped to work on 1 year ago given I had no chance to get the
> same
output: it's now a good time to restart working on it as next steps
>
>
> > Also it could be great if the release-plugin could automatically set
> > the
> > property:
> > a) if it finds the "project.build.outputTimestamp" set to some
> > placeholder value b) if some switch tells it to prepare a
> > reproducible
> > build by using some sort of "switch" parameter
> > Guess that would sort of close the loop to get the biggest benefit out
> > of
> > the reproducible builds.
>
> +1
> issue has been created
> https://issues.apache.org/jira/browse/MRELEASE-1029
But I didn't work on
> it yet, help welcome
>
>
> > I would be happy to help as I think this is one
> > of the greatest new features. (Ok ... perhaps besides the
> > sound-output-extension I learned about yesterday ;-) )
>
> +1
> the current step is important, but it's only the beginning of the story
> from
an effective usage perspective
>
> Regards,
>
> Hervé
>
>
> >
> > Chris
> >
> >
> > Am 01.11.19, 09:24 schrieb "Hervé BOUTEMY" <he...@free.fr>:
> >
> >
> > Le jeudi 31 octobre 2019, 17:26:52 CET Christofer Dutz a écrit :
> >
> >
> >
> > > Hi all,
> > >
> > > as I can see you're voting on releasing the reproducible build
> > > extended
> > > plugin versions.
> >
> >
> >
> > Is there any documentation on how to use this new
> >
> >
> >
> > > feature?
> > >
> > > I had a look at the confluence page, but that seemed like a
> > > brainstorming
> > > session.
> >
> >
> >
> > ok, the Wiki page [1] started as a brainstorming session, was
> > updated to
> >
> > a proposal (the "Output Archive Entries Timestamp" parapgraph).
>
> And now I
>
> > probably should order paragraph a little bit, and add a "Making your
> > build
> > reproducible" section for end uses to have a quick explanation.
> >
> > I'll write the explanation here as a first try before working on
> > the
> >
> > Wiki:
>
>
>
> > 1. upgrade your plugins to reproducible version, particularly
> >
> > mpaven-source-plugin, maven-jar-plugin and maven-assembly-plugin to
> > vesion
> > 3.2.0
>
> 2. add project.build.outputTimestamp property with the timestamp
>
> > value that will be used in zip/jar/tar archives: <properties>
> >
> >
> >
> > <project.build.outputTimestamp>2019-10-02T08:04:00Z</project.build.out
> > putTi
mestamp>
>
> </properties>
>
> >
> > Notice:
> > - there is no Maven version prerequisite, everything happens at
> > plugins
> >
> > level
>
> - Reproducible Builds require to have no version ranges in
>
> > dependencies, generally gives different result on Unixes vs Windows,
> > and
> > generally depend on the major version of JDK used to compile (see "Out
> > of
> > Scope" paragraph)
> >
> > You have the basis configured, the output should be reproducible
> > now.
> > If something is still not reproducible, use diffoscope to find
> > the
> >
> > unstable output, find the plugin that generated this output and check
> > if
> > there is a reproducible version available: if not, please open an
> > issue to
> > help plugin maintainers improving Reproducible Builds support at
> > every
> > plugin level.
>
>
>
> > [1]
> >
> > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=74682
> > 318
>
>
>
> >
> >
> >
> >
> >
> >
> > > I would love to add this to the PLC4X build asap.
> >
> >
> >
> > I'd love to have feedback
> > Don't hesitate to ping me.
> >
> >
> >
> > >
> > > So I would like to test the release-candidates and vote too.
> >
> >
> >
> > I would love to have many tester and votes :)
> >
> >
> >
> >
> > >
> > > Chris
> > >
> > >
> > >
> > > Am 16.10.19, 14:42 schrieb "Hervé BOUTEMY"
> > > <he...@free.fr>:
> > >
> > >
> > >
> > > Le mercredi 16 octobre 2019, 13:40:48 CEST Andreas Sewe a
> > > écrit :
> > >
> > >
> > >
> > >
> > >
> > > > Emmanuel Bourg wrote:
> > > >
> > > >
> > > >
> > > > > Le 16/10/2019 à 08:35, Hervé BOUTEMY a écrit :
> > > > >
> > > > >
> > > > >
> > > > >> last question: now that we seem to fully understand
> > > > >> each
> > > > >> other,
> > > > >> does it
> > > > >> mean that you don't need any more "seconds since the
> > > > >> epoch"
> > > > >> format
> > > > >> support for the property?
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > If Maven supports the SOURCE_DATE_EPOCH environment
> > > > > variable
> > > > > I
> > > > > don't
> > > > > think that's necessary, otherwise it would be nice to be
> > > > > able
> > > > > to
> > > > > invoke
> > > > >
> > > > > Maven with:
> > > > >
> > > > >
> > > > >
> > > > > mvn package
> > > > > -Dproject.build.outputTimestamp=$SOURCE_DATE_EPOCH
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > and this means supporting a timestamp formatted as
> > > > > seconds
> > > > > since
> > > > > the
> > > > > epoch.
> > > >
> > > >
> > > >
> > > >
> > > > +1
> > > >
> > > > The above would be a nice, simple way of bridging the gap
> > > > between
> > > > SOURCE_DATE_EPOCH and project.build.outputTimestamp.
> > >
> > >
> > >
> > >
> > >
> > > told like that, ok, why not
> > >
> > >
> > >
> > >
> > >
> > >
> > > >
> > > > If it is not too much trouble to implement the "\d+ ->
> > > > seconds
> > > > since
> > > > epoch" heuristic, them I would love to see it included.
> > >
> > >
> > >
> > >
> > >
> > > ok, I'll do and prepare the release
> > >
> > > Regards,
> > >
> > > Hervé
> > >
> > >
> > >
> > >
> > >
> > >
> > > >
> > > > Best wishes,
> > > >
> > > > Andreas
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > > ------------------------------------------------------------
> > > ------
> > > ---
> > > To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> > > For additional commands, e-mail: dev-help@maven.apache.org
> > >
> > >
> > >
> > >
> > >
> > >
> > > ----------------------------------------------------------------
> > > -----
> > > To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> > > For additional commands, e-mail: dev-help@maven.apache.org
> > >
> >
> >
> >
> >
> >
> >
> >
> >
> > ------------------------------------------------------------------
> > ---
> > To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> > For additional commands, e-mail: dev-help@maven.apache.org
> >
> >
> >
> >
>
>
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> For additional commands, e-mail: dev-help@maven.apache.org
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: last review of Reproducible Builds proposal
Posted by he...@free.fr.
ok, Reproducible Builds are not so easy to get: each plugin that you use can cause an issue
I really recommend you diffoscope to investigate differences, it really helps a lot by easily giving you precise differences
Good luck for the investigations :)
Regards,
Hervé
----- Mail original -----
De: "Christofer Dutz" <ch...@c-ware.de>
À: "Maven Developers List" <de...@maven.apache.org>
Envoyé: Dimanche 3 Novembre 2019 20:16:42
Objet: Re: last review of Reproducible Builds proposal
Hi Herve,
unfortunately I now have implemented some tooling to compare the stuff produced by the updated reproducible builds.
And it does show quite a number of non binary equal files.
Will investigate what's the difference.
Chris
Am 03.11.19, 11:08 schrieb "Hervé BOUTEMY" <he...@free.fr>:
great feedback, thank you: this proves the initial set of only 3 plugins is a
good basis.
For checking the many output artifacts from a build, future buildinfo file [1]
should help a lot. I still have many usability concerns for Maven multi-module
builds (starting with: should we generate only the root buildinfo or one
buildinfo per Maven module?)
Let's keep the ball rolling: today, it's plugins release day!!!
Regards,
Hervé
[1] https://reproducible-builds.org/docs/jvm/
Le vendredi 1 novembre 2019, 13:34:32 CET Christofer Dutz a écrit :
> Hi,
>
> so I just updated the versions of the 3 plugins and gave the Apache PLC4X
> build a little spin ...
> https://github.com/apache/plc4x/tree/feature/reproducible-builds
>
> Did two executions of this:
> mvn -U -P
> with-boost,with-java,with-dotnet,with-cpp,with-python,with-proxies,with-san
> dbox,with
> -DaltDeploymentRepository=snapshot-repo::default::file:./local-snapshots-di
> r clean deploy
(but with a different second local repo location)
>
> Then I used "diff" and "cmp" to compare individual files and it didn't show
> differences with the artifacts I chose ...
now I guess I'll have to whip
> up some little bash script to sort of compare the artifacts from one
> directory with the other (Unfortunately the SNAPSHOT timestamps are a
> little annoying :-/
> We do have some C++, C# and Python modules ... but I wouldn't expect the C++
> and C# to be reproducible.
> Chris
>
>
> Am 01.11.19, 12:04 schrieb "Hervé BOUTEMY" <he...@free.fr>:
>
> Le vendredi 1 novembre 2019, 09:40:40 CET Christofer Dutz a écrit :
>
> > Hi Herve,
> >
> > thanks for that will try it asap and report any findings back.
> >
> > But good to know that there is a difference between JDK major versions
> > and
> > OSes ... so it would probably be best to stage releases on Linux with
> > an
> > OpenJDK of the minimum supported version?
> > Just thinking how to make it
> > possible to verify without having to buy Mac or Windows licenses ...
> > guess
> > on every machine you could whip up a Ubuntu VM for verification. Just
> > thinking about it ... perhaps it would be best to create a Docker
> > image for
doing the reproducible stuff ...
>
> just to be clear: the difference is on newline only, then Windows vs
> anything
else. You get the same result on Linux, MacOS, FreeBSD, or any
> other Unix.
> And if I want to be complete, if you get a source tarball from Windows,
>
extract it to a Linux/MacOS/... box and build with "mvn -
> Dline.separator='\r\n'", in general, you get the same result as a build
> under
Windows: I tested multiple times, it worked, but we'll see if it
> works always or just "in general"
>
>
> > Are there any plans on creating a plugin to allow verification?
> >
> > Sort of something like this:
> > "mvn package release:verify-reproduicble
> > -DstagingRepoUrl=a.b.c.de/repo/blahblahblah"
> > (Which doesn't deploy the
> > artifacts, but instead download them and do a binary comparison)
>
> yes, but for now it was completely another form: this is the "Buildinfo
> file"
proposal https://reproducible-builds.org/docs/jvm/
> that I stopped to work on 1 year ago given I had no chance to get the
> same
output: it's now a good time to restart working on it as next steps
>
>
> > Also it could be great if the release-plugin could automatically set
> > the
> > property:
> > a) if it finds the "project.build.outputTimestamp" set to some
> > placeholder value b) if some switch tells it to prepare a
> > reproducible
> > build by using some sort of "switch" parameter
> > Guess that would sort of close the loop to get the biggest benefit out
> > of
> > the reproducible builds.
>
> +1
> issue has been created
> https://issues.apache.org/jira/browse/MRELEASE-1029
But I didn't work on
> it yet, help welcome
>
>
> > I would be happy to help as I think this is one
> > of the greatest new features. (Ok ... perhaps besides the
> > sound-output-extension I learned about yesterday ;-) )
>
> +1
> the current step is important, but it's only the beginning of the story
> from
an effective usage perspective
>
> Regards,
>
> Hervé
>
>
> >
> > Chris
> >
> >
> > Am 01.11.19, 09:24 schrieb "Hervé BOUTEMY" <he...@free.fr>:
> >
> >
> > Le jeudi 31 octobre 2019, 17:26:52 CET Christofer Dutz a écrit :
> >
> >
> >
> > > Hi all,
> > >
> > > as I can see you're voting on releasing the reproducible build
> > > extended
> > > plugin versions.
> >
> >
> >
> > Is there any documentation on how to use this new
> >
> >
> >
> > > feature?
> > >
> > > I had a look at the confluence page, but that seemed like a
> > > brainstorming
> > > session.
> >
> >
> >
> > ok, the Wiki page [1] started as a brainstorming session, was
> > updated to
> >
> > a proposal (the "Output Archive Entries Timestamp" parapgraph).
>
> And now I
>
> > probably should order paragraph a little bit, and add a "Making your
> > build
> > reproducible" section for end uses to have a quick explanation.
> >
> > I'll write the explanation here as a first try before working on
> > the
> >
> > Wiki:
>
>
>
> > 1. upgrade your plugins to reproducible version, particularly
> >
> > mpaven-source-plugin, maven-jar-plugin and maven-assembly-plugin to
> > vesion
> > 3.2.0
>
> 2. add project.build.outputTimestamp property with the timestamp
>
> > value that will be used in zip/jar/tar archives: <properties>
> >
> >
> >
> > <project.build.outputTimestamp>2019-10-02T08:04:00Z</project.build.out
> > putTi
mestamp>
>
> </properties>
>
> >
> > Notice:
> > - there is no Maven version prerequisite, everything happens at
> > plugins
> >
> > level
>
> - Reproducible Builds require to have no version ranges in
>
> > dependencies, generally gives different result on Unixes vs Windows,
> > and
> > generally depend on the major version of JDK used to compile (see "Out
> > of
> > Scope" paragraph)
> >
> > You have the basis configured, the output should be reproducible
> > now.
> > If something is still not reproducible, use diffoscope to find
> > the
> >
> > unstable output, find the plugin that generated this output and check
> > if
> > there is a reproducible version available: if not, please open an
> > issue to
> > help plugin maintainers improving Reproducible Builds support at
> > every
> > plugin level.
>
>
>
> > [1]
> >
> > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=74682
> > 318
>
>
>
> >
> >
> >
> >
> >
> >
> > > I would love to add this to the PLC4X build asap.
> >
> >
> >
> > I'd love to have feedback
> > Don't hesitate to ping me.
> >
> >
> >
> > >
> > > So I would like to test the release-candidates and vote too.
> >
> >
> >
> > I would love to have many tester and votes :)
> >
> >
> >
> >
> > >
> > > Chris
> > >
> > >
> > >
> > > Am 16.10.19, 14:42 schrieb "Hervé BOUTEMY"
> > > <he...@free.fr>:
> > >
> > >
> > >
> > > Le mercredi 16 octobre 2019, 13:40:48 CEST Andreas Sewe a
> > > écrit :
> > >
> > >
> > >
> > >
> > >
> > > > Emmanuel Bourg wrote:
> > > >
> > > >
> > > >
> > > > > Le 16/10/2019 à 08:35, Hervé BOUTEMY a écrit :
> > > > >
> > > > >
> > > > >
> > > > >> last question: now that we seem to fully understand
> > > > >> each
> > > > >> other,
> > > > >> does it
> > > > >> mean that you don't need any more "seconds since the
> > > > >> epoch"
> > > > >> format
> > > > >> support for the property?
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > If Maven supports the SOURCE_DATE_EPOCH environment
> > > > > variable
> > > > > I
> > > > > don't
> > > > > think that's necessary, otherwise it would be nice to be
> > > > > able
> > > > > to
> > > > > invoke
> > > > >
> > > > > Maven with:
> > > > >
> > > > >
> > > > >
> > > > > mvn package
> > > > > -Dproject.build.outputTimestamp=$SOURCE_DATE_EPOCH
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > and this means supporting a timestamp formatted as
> > > > > seconds
> > > > > since
> > > > > the
> > > > > epoch.
> > > >
> > > >
> > > >
> > > >
> > > > +1
> > > >
> > > > The above would be a nice, simple way of bridging the gap
> > > > between
> > > > SOURCE_DATE_EPOCH and project.build.outputTimestamp.
> > >
> > >
> > >
> > >
> > >
> > > told like that, ok, why not
> > >
> > >
> > >
> > >
> > >
> > >
> > > >
> > > > If it is not too much trouble to implement the "\d+ ->
> > > > seconds
> > > > since
> > > > epoch" heuristic, them I would love to see it included.
> > >
> > >
> > >
> > >
> > >
> > > ok, I'll do and prepare the release
> > >
> > > Regards,
> > >
> > > Hervé
> > >
> > >
> > >
> > >
> > >
> > >
> > > >
> > > > Best wishes,
> > > >
> > > > Andreas
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > > ------------------------------------------------------------
> > > ------
> > > ---
> > > To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> > > For additional commands, e-mail: dev-help@maven.apache.org
> > >
> > >
> > >
> > >
> > >
> > >
> > > ----------------------------------------------------------------
> > > -----
> > > To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> > > For additional commands, e-mail: dev-help@maven.apache.org
> > >
> >
> >
> >
> >
> >
> >
> >
> >
> > ------------------------------------------------------------------
> > ---
> > To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> > For additional commands, e-mail: dev-help@maven.apache.org
> >
> >
> >
> >
>
>
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> For additional commands, e-mail: dev-help@maven.apache.org
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: last review of Reproducible Builds proposal
Posted by Christofer Dutz <ch...@c-ware.de>.
Hi Herve,
unfortunately I now have implemented some tooling to compare the stuff produced by the updated reproducible builds.
And it does show quite a number of non binary equal files.
Will investigate what's the difference.
Chris
Am 03.11.19, 11:08 schrieb "Hervé BOUTEMY" <he...@free.fr>:
great feedback, thank you: this proves the initial set of only 3 plugins is a
good basis.
For checking the many output artifacts from a build, future buildinfo file [1]
should help a lot. I still have many usability concerns for Maven multi-module
builds (starting with: should we generate only the root buildinfo or one
buildinfo per Maven module?)
Let's keep the ball rolling: today, it's plugins release day!!!
Regards,
Hervé
[1] https://reproducible-builds.org/docs/jvm/
Le vendredi 1 novembre 2019, 13:34:32 CET Christofer Dutz a écrit :
> Hi,
>
> so I just updated the versions of the 3 plugins and gave the Apache PLC4X
> build a little spin ...
> https://github.com/apache/plc4x/tree/feature/reproducible-builds
>
> Did two executions of this:
> mvn -U -P
> with-boost,with-java,with-dotnet,with-cpp,with-python,with-proxies,with-san
> dbox,with
> -DaltDeploymentRepository=snapshot-repo::default::file:./local-snapshots-di
> r clean deploy
(but with a different second local repo location)
>
> Then I used "diff" and "cmp" to compare individual files and it didn't show
> differences with the artifacts I chose ...
now I guess I'll have to whip
> up some little bash script to sort of compare the artifacts from one
> directory with the other (Unfortunately the SNAPSHOT timestamps are a
> little annoying :-/
> We do have some C++, C# and Python modules ... but I wouldn't expect the C++
> and C# to be reproducible.
> Chris
>
>
> Am 01.11.19, 12:04 schrieb "Hervé BOUTEMY" <he...@free.fr>:
>
> Le vendredi 1 novembre 2019, 09:40:40 CET Christofer Dutz a écrit :
>
> > Hi Herve,
> >
> > thanks for that will try it asap and report any findings back.
> >
> > But good to know that there is a difference between JDK major versions
> > and
> > OSes ... so it would probably be best to stage releases on Linux with
> > an
> > OpenJDK of the minimum supported version?
> > Just thinking how to make it
> > possible to verify without having to buy Mac or Windows licenses ...
> > guess
> > on every machine you could whip up a Ubuntu VM for verification. Just
> > thinking about it ... perhaps it would be best to create a Docker
> > image for
doing the reproducible stuff ...
>
> just to be clear: the difference is on newline only, then Windows vs
> anything
else. You get the same result on Linux, MacOS, FreeBSD, or any
> other Unix.
> And if I want to be complete, if you get a source tarball from Windows,
>
extract it to a Linux/MacOS/... box and build with "mvn -
> Dline.separator='\r\n'", in general, you get the same result as a build
> under
Windows: I tested multiple times, it worked, but we'll see if it
> works always or just "in general"
>
>
> > Are there any plans on creating a plugin to allow verification?
> >
> > Sort of something like this:
> > "mvn package release:verify-reproduicble
> > -DstagingRepoUrl=a.b.c.de/repo/blahblahblah"
> > (Which doesn't deploy the
> > artifacts, but instead download them and do a binary comparison)
>
> yes, but for now it was completely another form: this is the "Buildinfo
> file"
proposal https://reproducible-builds.org/docs/jvm/
> that I stopped to work on 1 year ago given I had no chance to get the
> same
output: it's now a good time to restart working on it as next steps
>
>
> > Also it could be great if the release-plugin could automatically set
> > the
> > property:
> > a) if it finds the "project.build.outputTimestamp" set to some
> > placeholder value b) if some switch tells it to prepare a
> > reproducible
> > build by using some sort of "switch" parameter
> > Guess that would sort of close the loop to get the biggest benefit out
> > of
> > the reproducible builds.
>
> +1
> issue has been created
> https://issues.apache.org/jira/browse/MRELEASE-1029
But I didn't work on
> it yet, help welcome
>
>
> > I would be happy to help as I think this is one
> > of the greatest new features. (Ok ... perhaps besides the
> > sound-output-extension I learned about yesterday ;-) )
>
> +1
> the current step is important, but it's only the beginning of the story
> from
an effective usage perspective
>
> Regards,
>
> Hervé
>
>
> >
> > Chris
> >
> >
> > Am 01.11.19, 09:24 schrieb "Hervé BOUTEMY" <he...@free.fr>:
> >
> >
> > Le jeudi 31 octobre 2019, 17:26:52 CET Christofer Dutz a écrit :
> >
> >
> >
> > > Hi all,
> > >
> > > as I can see you're voting on releasing the reproducible build
> > > extended
> > > plugin versions.
> >
> >
> >
> > Is there any documentation on how to use this new
> >
> >
> >
> > > feature?
> > >
> > > I had a look at the confluence page, but that seemed like a
> > > brainstorming
> > > session.
> >
> >
> >
> > ok, the Wiki page [1] started as a brainstorming session, was
> > updated to
> >
> > a proposal (the "Output Archive Entries Timestamp" parapgraph).
>
> And now I
>
> > probably should order paragraph a little bit, and add a "Making your
> > build
> > reproducible" section for end uses to have a quick explanation.
> >
> > I'll write the explanation here as a first try before working on
> > the
> >
> > Wiki:
>
>
>
> > 1. upgrade your plugins to reproducible version, particularly
> >
> > mpaven-source-plugin, maven-jar-plugin and maven-assembly-plugin to
> > vesion
> > 3.2.0
>
> 2. add project.build.outputTimestamp property with the timestamp
>
> > value that will be used in zip/jar/tar archives: <properties>
> >
> >
> >
> > <project.build.outputTimestamp>2019-10-02T08:04:00Z</project.build.out
> > putTi
mestamp>
>
> </properties>
>
> >
> > Notice:
> > - there is no Maven version prerequisite, everything happens at
> > plugins
> >
> > level
>
> - Reproducible Builds require to have no version ranges in
>
> > dependencies, generally gives different result on Unixes vs Windows,
> > and
> > generally depend on the major version of JDK used to compile (see "Out
> > of
> > Scope" paragraph)
> >
> > You have the basis configured, the output should be reproducible
> > now.
> > If something is still not reproducible, use diffoscope to find
> > the
> >
> > unstable output, find the plugin that generated this output and check
> > if
> > there is a reproducible version available: if not, please open an
> > issue to
> > help plugin maintainers improving Reproducible Builds support at
> > every
> > plugin level.
>
>
>
> > [1]
> >
> > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=74682
> > 318
>
>
>
> >
> >
> >
> >
> >
> >
> > > I would love to add this to the PLC4X build asap.
> >
> >
> >
> > I'd love to have feedback
> > Don't hesitate to ping me.
> >
> >
> >
> > >
> > > So I would like to test the release-candidates and vote too.
> >
> >
> >
> > I would love to have many tester and votes :)
> >
> >
> >
> >
> > >
> > > Chris
> > >
> > >
> > >
> > > Am 16.10.19, 14:42 schrieb "Hervé BOUTEMY"
> > > <he...@free.fr>:
> > >
> > >
> > >
> > > Le mercredi 16 octobre 2019, 13:40:48 CEST Andreas Sewe a
> > > écrit :
> > >
> > >
> > >
> > >
> > >
> > > > Emmanuel Bourg wrote:
> > > >
> > > >
> > > >
> > > > > Le 16/10/2019 à 08:35, Hervé BOUTEMY a écrit :
> > > > >
> > > > >
> > > > >
> > > > >> last question: now that we seem to fully understand
> > > > >> each
> > > > >> other,
> > > > >> does it
> > > > >> mean that you don't need any more "seconds since the
> > > > >> epoch"
> > > > >> format
> > > > >> support for the property?
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > If Maven supports the SOURCE_DATE_EPOCH environment
> > > > > variable
> > > > > I
> > > > > don't
> > > > > think that's necessary, otherwise it would be nice to be
> > > > > able
> > > > > to
> > > > > invoke
> > > > >
> > > > > Maven with:
> > > > >
> > > > >
> > > > >
> > > > > mvn package
> > > > > -Dproject.build.outputTimestamp=$SOURCE_DATE_EPOCH
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > and this means supporting a timestamp formatted as
> > > > > seconds
> > > > > since
> > > > > the
> > > > > epoch.
> > > >
> > > >
> > > >
> > > >
> > > > +1
> > > >
> > > > The above would be a nice, simple way of bridging the gap
> > > > between
> > > > SOURCE_DATE_EPOCH and project.build.outputTimestamp.
> > >
> > >
> > >
> > >
> > >
> > > told like that, ok, why not
> > >
> > >
> > >
> > >
> > >
> > >
> > > >
> > > > If it is not too much trouble to implement the "\d+ ->
> > > > seconds
> > > > since
> > > > epoch" heuristic, them I would love to see it included.
> > >
> > >
> > >
> > >
> > >
> > > ok, I'll do and prepare the release
> > >
> > > Regards,
> > >
> > > Hervé
> > >
> > >
> > >
> > >
> > >
> > >
> > > >
> > > > Best wishes,
> > > >
> > > > Andreas
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > > ------------------------------------------------------------
> > > ------
> > > ---
> > > To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> > > For additional commands, e-mail: dev-help@maven.apache.org
> > >
> > >
> > >
> > >
> > >
> > >
> > > ----------------------------------------------------------------
> > > -----
> > > To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> > > For additional commands, e-mail: dev-help@maven.apache.org
> > >
> >
> >
> >
> >
> >
> >
> >
> >
> > ------------------------------------------------------------------
> > ---
> > To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> > For additional commands, e-mail: dev-help@maven.apache.org
> >
> >
> >
> >
>
>
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> For additional commands, e-mail: dev-help@maven.apache.org
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: last review of Reproducible Builds proposal
Posted by Hervé BOUTEMY <he...@free.fr>.
great feedback, thank you: this proves the initial set of only 3 plugins is a
good basis.
For checking the many output artifacts from a build, future buildinfo file [1]
should help a lot. I still have many usability concerns for Maven multi-module
builds (starting with: should we generate only the root buildinfo or one
buildinfo per Maven module?)
Let's keep the ball rolling: today, it's plugins release day!!!
Regards,
Hervé
[1] https://reproducible-builds.org/docs/jvm/
Le vendredi 1 novembre 2019, 13:34:32 CET Christofer Dutz a écrit :
> Hi,
>
> so I just updated the versions of the 3 plugins and gave the Apache PLC4X
> build a little spin ...
> https://github.com/apache/plc4x/tree/feature/reproducible-builds
>
> Did two executions of this:
> mvn -U -P
> with-boost,with-java,with-dotnet,with-cpp,with-python,with-proxies,with-san
> dbox,with
> -DaltDeploymentRepository=snapshot-repo::default::file:./local-snapshots-di
> r clean deploy
(but with a different second local repo location)
>
> Then I used "diff" and "cmp" to compare individual files and it didn't show
> differences with the artifacts I chose ...
now I guess I'll have to whip
> up some little bash script to sort of compare the artifacts from one
> directory with the other (Unfortunately the SNAPSHOT timestamps are a
> little annoying :-/
> We do have some C++, C# and Python modules ... but I wouldn't expect the C++
> and C# to be reproducible.
> Chris
>
>
> Am 01.11.19, 12:04 schrieb "Hervé BOUTEMY" <he...@free.fr>:
>
> Le vendredi 1 novembre 2019, 09:40:40 CET Christofer Dutz a écrit :
>
> > Hi Herve,
> >
> > thanks for that will try it asap and report any findings back.
> >
> > But good to know that there is a difference between JDK major versions
> > and
> > OSes ... so it would probably be best to stage releases on Linux with
> > an
> > OpenJDK of the minimum supported version?
> > Just thinking how to make it
> > possible to verify without having to buy Mac or Windows licenses ...
> > guess
> > on every machine you could whip up a Ubuntu VM for verification. Just
> > thinking about it ... perhaps it would be best to create a Docker
> > image for
doing the reproducible stuff ...
>
> just to be clear: the difference is on newline only, then Windows vs
> anything
else. You get the same result on Linux, MacOS, FreeBSD, or any
> other Unix.
> And if I want to be complete, if you get a source tarball from Windows,
>
extract it to a Linux/MacOS/... box and build with "mvn -
> Dline.separator='\r\n'", in general, you get the same result as a build
> under
Windows: I tested multiple times, it worked, but we'll see if it
> works always or just "in general"
>
>
> > Are there any plans on creating a plugin to allow verification?
> >
> > Sort of something like this:
> > "mvn package release:verify-reproduicble
> > -DstagingRepoUrl=a.b.c.de/repo/blahblahblah"
> > (Which doesn't deploy the
> > artifacts, but instead download them and do a binary comparison)
>
> yes, but for now it was completely another form: this is the "Buildinfo
> file"
proposal https://reproducible-builds.org/docs/jvm/
> that I stopped to work on 1 year ago given I had no chance to get the
> same
output: it's now a good time to restart working on it as next steps
>
>
> > Also it could be great if the release-plugin could automatically set
> > the
> > property:
> > a) if it finds the "project.build.outputTimestamp" set to some
> > placeholder value b) if some switch tells it to prepare a
> > reproducible
> > build by using some sort of "switch" parameter
> > Guess that would sort of close the loop to get the biggest benefit out
> > of
> > the reproducible builds.
>
> +1
> issue has been created
> https://issues.apache.org/jira/browse/MRELEASE-1029
But I didn't work on
> it yet, help welcome
>
>
> > I would be happy to help as I think this is one
> > of the greatest new features. (Ok ... perhaps besides the
> > sound-output-extension I learned about yesterday ;-) )
>
> +1
> the current step is important, but it's only the beginning of the story
> from
an effective usage perspective
>
> Regards,
>
> Hervé
>
>
> >
> > Chris
> >
> >
> > Am 01.11.19, 09:24 schrieb "Hervé BOUTEMY" <he...@free.fr>:
> >
> >
> > Le jeudi 31 octobre 2019, 17:26:52 CET Christofer Dutz a écrit :
> >
> >
> >
> > > Hi all,
> > >
> > > as I can see you're voting on releasing the reproducible build
> > > extended
> > > plugin versions.
> >
> >
> >
> > Is there any documentation on how to use this new
> >
> >
> >
> > > feature?
> > >
> > > I had a look at the confluence page, but that seemed like a
> > > brainstorming
> > > session.
> >
> >
> >
> > ok, the Wiki page [1] started as a brainstorming session, was
> > updated to
> >
> > a proposal (the "Output Archive Entries Timestamp" parapgraph).
>
> And now I
>
> > probably should order paragraph a little bit, and add a "Making your
> > build
> > reproducible" section for end uses to have a quick explanation.
> >
> > I'll write the explanation here as a first try before working on
> > the
> >
> > Wiki:
>
>
>
> > 1. upgrade your plugins to reproducible version, particularly
> >
> > mpaven-source-plugin, maven-jar-plugin and maven-assembly-plugin to
> > vesion
> > 3.2.0
>
> 2. add project.build.outputTimestamp property with the timestamp
>
> > value that will be used in zip/jar/tar archives: <properties>
> >
> >
> >
> > <project.build.outputTimestamp>2019-10-02T08:04:00Z</project.build.out
> > putTi
mestamp>
>
> </properties>
>
> >
> > Notice:
> > - there is no Maven version prerequisite, everything happens at
> > plugins
> >
> > level
>
> - Reproducible Builds require to have no version ranges in
>
> > dependencies, generally gives different result on Unixes vs Windows,
> > and
> > generally depend on the major version of JDK used to compile (see "Out
> > of
> > Scope" paragraph)
> >
> > You have the basis configured, the output should be reproducible
> > now.
> > If something is still not reproducible, use diffoscope to find
> > the
> >
> > unstable output, find the plugin that generated this output and check
> > if
> > there is a reproducible version available: if not, please open an
> > issue to
> > help plugin maintainers improving Reproducible Builds support at
> > every
> > plugin level.
>
>
>
> > [1]
> >
> > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=74682
> > 318
>
>
>
> >
> >
> >
> >
> >
> >
> > > I would love to add this to the PLC4X build asap.
> >
> >
> >
> > I'd love to have feedback
> > Don't hesitate to ping me.
> >
> >
> >
> > >
> > > So I would like to test the release-candidates and vote too.
> >
> >
> >
> > I would love to have many tester and votes :)
> >
> >
> >
> >
> > >
> > > Chris
> > >
> > >
> > >
> > > Am 16.10.19, 14:42 schrieb "Hervé BOUTEMY"
> > > <he...@free.fr>:
> > >
> > >
> > >
> > > Le mercredi 16 octobre 2019, 13:40:48 CEST Andreas Sewe a
> > > écrit :
> > >
> > >
> > >
> > >
> > >
> > > > Emmanuel Bourg wrote:
> > > >
> > > >
> > > >
> > > > > Le 16/10/2019 à 08:35, Hervé BOUTEMY a écrit :
> > > > >
> > > > >
> > > > >
> > > > >> last question: now that we seem to fully understand
> > > > >> each
> > > > >> other,
> > > > >> does it
> > > > >> mean that you don't need any more "seconds since the
> > > > >> epoch"
> > > > >> format
> > > > >> support for the property?
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > If Maven supports the SOURCE_DATE_EPOCH environment
> > > > > variable
> > > > > I
> > > > > don't
> > > > > think that's necessary, otherwise it would be nice to be
> > > > > able
> > > > > to
> > > > > invoke
> > > > >
> > > > > Maven with:
> > > > >
> > > > >
> > > > >
> > > > > mvn package
> > > > > -Dproject.build.outputTimestamp=$SOURCE_DATE_EPOCH
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > and this means supporting a timestamp formatted as
> > > > > seconds
> > > > > since
> > > > > the
> > > > > epoch.
> > > >
> > > >
> > > >
> > > >
> > > > +1
> > > >
> > > > The above would be a nice, simple way of bridging the gap
> > > > between
> > > > SOURCE_DATE_EPOCH and project.build.outputTimestamp.
> > >
> > >
> > >
> > >
> > >
> > > told like that, ok, why not
> > >
> > >
> > >
> > >
> > >
> > >
> > > >
> > > > If it is not too much trouble to implement the "\d+ ->
> > > > seconds
> > > > since
> > > > epoch" heuristic, them I would love to see it included.
> > >
> > >
> > >
> > >
> > >
> > > ok, I'll do and prepare the release
> > >
> > > Regards,
> > >
> > > Hervé
> > >
> > >
> > >
> > >
> > >
> > >
> > > >
> > > > Best wishes,
> > > >
> > > > Andreas
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > > ------------------------------------------------------------
> > > ------
> > > ---
> > > To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> > > For additional commands, e-mail: dev-help@maven.apache.org
> > >
> > >
> > >
> > >
> > >
> > >
> > > ----------------------------------------------------------------
> > > -----
> > > To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> > > For additional commands, e-mail: dev-help@maven.apache.org
> > >
> >
> >
> >
> >
> >
> >
> >
> >
> > ------------------------------------------------------------------
> > ---
> > To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> > For additional commands, e-mail: dev-help@maven.apache.org
> >
> >
> >
> >
>
>
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> For additional commands, e-mail: dev-help@maven.apache.org
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: last review of Reproducible Builds proposal
Posted by Christofer Dutz <ch...@c-ware.de>.
Hi,
so I just updated the versions of the 3 plugins and gave the Apache PLC4X build a little spin ...
https://github.com/apache/plc4x/tree/feature/reproducible-builds
Did two executions of this:
mvn -U -P with-boost,with-java,with-dotnet,with-cpp,with-python,with-proxies,with-sandbox,with -DaltDeploymentRepository=snapshot-repo::default::file:./local-snapshots-dir clean deploy
(but with a different second local repo location)
Then I used "diff" and "cmp" to compare individual files and it didn't show differences with the artifacts I chose ...
now I guess I'll have to whip up some little bash script to sort of compare the artifacts from one directory with the other (Unfortunately the SNAPSHOT timestamps are a little annoying :-/
We do have some C++, C# and Python modules ... but I wouldn't expect the C++ and C# to be reproducible.
Chris
Am 01.11.19, 12:04 schrieb "Hervé BOUTEMY" <he...@free.fr>:
Le vendredi 1 novembre 2019, 09:40:40 CET Christofer Dutz a écrit :
> Hi Herve,
>
> thanks for that will try it asap and report any findings back.
>
> But good to know that there is a difference between JDK major versions and
> OSes ... so it would probably be best to stage releases on Linux with an
> OpenJDK of the minimum supported version?
> Just thinking how to make it
> possible to verify without having to buy Mac or Windows licenses ... guess
> on every machine you could whip up a Ubuntu VM for verification. Just
> thinking about it ... perhaps it would be best to create a Docker image for
> doing the reproducible stuff ...
just to be clear: the difference is on newline only, then Windows vs anything
else. You get the same result on Linux, MacOS, FreeBSD, or any other Unix.
And if I want to be complete, if you get a source tarball from Windows,
extract it to a Linux/MacOS/... box and build with "mvn -
Dline.separator='\r\n'", in general, you get the same result as a build under
Windows: I tested multiple times, it worked, but we'll see if it works always
or just "in general"
> Are there any plans on creating a plugin to allow verification?
>
> Sort of something like this:
> "mvn package release:verify-reproduicble
> -DstagingRepoUrl=a.b.c.de/repo/blahblahblah"
> (Which doesn't deploy the
> artifacts, but instead download them and do a binary comparison)
yes, but for now it was completely another form: this is the "Buildinfo file"
proposal https://reproducible-builds.org/docs/jvm/
that I stopped to work on 1 year ago given I had no chance to get the same
output: it's now a good time to restart working on it as next steps
> Also it could be great if the release-plugin could automatically set the
> property:
> a) if it finds the "project.build.outputTimestamp" set to some
> placeholder value b) if some switch tells it to prepare a reproducible
> build by using some sort of "switch" parameter
> Guess that would sort of close the loop to get the biggest benefit out of
> the reproducible builds.
+1
issue has been created https://issues.apache.org/jira/browse/MRELEASE-1029
But I didn't work on it yet, help welcome
> I would be happy to help as I think this is one
> of the greatest new features. (Ok ... perhaps besides the
> sound-output-extension I learned about yesterday ;-) )
+1
the current step is important, but it's only the beginning of the story from
an effective usage perspective
Regards,
Hervé
>
> Chris
>
>
> Am 01.11.19, 09:24 schrieb "Hervé BOUTEMY" <he...@free.fr>:
>
> Le jeudi 31 octobre 2019, 17:26:52 CET Christofer Dutz a écrit :
>
> > Hi all,
> >
> > as I can see you're voting on releasing the reproducible build
> > extended
> > plugin versions.
>
> Is there any documentation on how to use this new
>
> > feature?
> >
> > I had a look at the confluence page, but that seemed like a
> > brainstorming
> > session.
>
> ok, the Wiki page [1] started as a brainstorming session, was updated to
> a proposal (the "Output Archive Entries Timestamp" parapgraph).
And now I
> probably should order paragraph a little bit, and add a "Making your build
> reproducible" section for end uses to have a quick explanation.
> I'll write the explanation here as a first try before working on the
> Wiki:
> 1. upgrade your plugins to reproducible version, particularly
> mpaven-source-plugin, maven-jar-plugin and maven-assembly-plugin to vesion
> 3.2.0
2. add project.build.outputTimestamp property with the timestamp
> value that will be used in zip/jar/tar archives: <properties>
>
> <project.build.outputTimestamp>2019-10-02T08:04:00Z</project.build.outputTi
> mestamp>
</properties>
>
> Notice:
> - there is no Maven version prerequisite, everything happens at plugins
> level
- Reproducible Builds require to have no version ranges in
> dependencies, generally gives different result on Unixes vs Windows, and
> generally depend on the major version of JDK used to compile (see "Out of
> Scope" paragraph)
> You have the basis configured, the output should be reproducible now.
> If something is still not reproducible, use diffoscope to find the
> unstable output, find the plugin that generated this output and check if
> there is a reproducible version available: if not, please open an issue to
> help plugin maintainers improving Reproducible Builds support at every
> plugin level.
> [1]
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=74682318
>
>
>
> > I would love to add this to the PLC4X build asap.
>
> I'd love to have feedback
> Don't hesitate to ping me.
>
> >
> > So I would like to test the release-candidates and vote too.
>
> I would love to have many tester and votes :)
>
>
> >
> > Chris
> >
> >
> >
> > Am 16.10.19, 14:42 schrieb "Hervé BOUTEMY" <he...@free.fr>:
> >
> >
> > Le mercredi 16 octobre 2019, 13:40:48 CEST Andreas Sewe a écrit :
> >
> >
> >
> > > Emmanuel Bourg wrote:
> > >
> > >
> > > > Le 16/10/2019 à 08:35, Hervé BOUTEMY a écrit :
> > > >
> > > >
> > > >> last question: now that we seem to fully understand each
> > > >> other,
> > > >> does it
> > > >> mean that you don't need any more "seconds since the epoch"
> > > >> format
> > > >> support for the property?
> > > >
> > > >
> > > >
> > > > If Maven supports the SOURCE_DATE_EPOCH environment variable
> > > > I
> > > > don't
> > > > think that's necessary, otherwise it would be nice to be able
> > > > to
> > > > invoke
> > > >
> > > > Maven with:
> > > >
> > > >
> > > > mvn package
> > > > -Dproject.build.outputTimestamp=$SOURCE_DATE_EPOCH
> > > >
> > > >
> > > >
> > > > and this means supporting a timestamp formatted as seconds
> > > > since
> > > > the
> > > > epoch.
> > >
> > >
> > >
> > > +1
> > >
> > > The above would be a nice, simple way of bridging the gap
> > > between
> > > SOURCE_DATE_EPOCH and project.build.outputTimestamp.
> >
> >
> >
> > told like that, ok, why not
> >
> >
> >
> >
> > >
> > > If it is not too much trouble to implement the "\d+ -> seconds
> > > since
> > > epoch" heuristic, them I would love to see it included.
> >
> >
> >
> > ok, I'll do and prepare the release
> >
> > Regards,
> >
> > Hervé
> >
> >
> >
> >
> > >
> > > Best wishes,
> > >
> > > Andreas
> >
> >
> >
> >
> >
> >
> >
> >
> > ------------------------------------------------------------------
> > ---
> > To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> > For additional commands, e-mail: dev-help@maven.apache.org
> >
> >
> >
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> > For additional commands, e-mail: dev-help@maven.apache.org
> >
>
>
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> For additional commands, e-mail: dev-help@maven.apache.org
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: last review of Reproducible Builds proposal
Posted by Hervé BOUTEMY <he...@free.fr>.
Le vendredi 1 novembre 2019, 09:40:40 CET Christofer Dutz a écrit :
> Hi Herve,
>
> thanks for that will try it asap and report any findings back.
>
> But good to know that there is a difference between JDK major versions and
> OSes ... so it would probably be best to stage releases on Linux with an
> OpenJDK of the minimum supported version?
> Just thinking how to make it
> possible to verify without having to buy Mac or Windows licenses ... guess
> on every machine you could whip up a Ubuntu VM for verification. Just
> thinking about it ... perhaps it would be best to create a Docker image for
> doing the reproducible stuff ...
just to be clear: the difference is on newline only, then Windows vs anything
else. You get the same result on Linux, MacOS, FreeBSD, or any other Unix.
And if I want to be complete, if you get a source tarball from Windows,
extract it to a Linux/MacOS/... box and build with "mvn -
Dline.separator='\r\n'", in general, you get the same result as a build under
Windows: I tested multiple times, it worked, but we'll see if it works always
or just "in general"
> Are there any plans on creating a plugin to allow verification?
>
> Sort of something like this:
> "mvn package release:verify-reproduicble
> -DstagingRepoUrl=a.b.c.de/repo/blahblahblah"
> (Which doesn't deploy the
> artifacts, but instead download them and do a binary comparison)
yes, but for now it was completely another form: this is the "Buildinfo file"
proposal https://reproducible-builds.org/docs/jvm/
that I stopped to work on 1 year ago given I had no chance to get the same
output: it's now a good time to restart working on it as next steps
> Also it could be great if the release-plugin could automatically set the
> property:
> a) if it finds the "project.build.outputTimestamp" set to some
> placeholder value b) if some switch tells it to prepare a reproducible
> build by using some sort of "switch" parameter
> Guess that would sort of close the loop to get the biggest benefit out of
> the reproducible builds.
+1
issue has been created https://issues.apache.org/jira/browse/MRELEASE-1029
But I didn't work on it yet, help welcome
> I would be happy to help as I think this is one
> of the greatest new features. (Ok ... perhaps besides the
> sound-output-extension I learned about yesterday ;-) )
+1
the current step is important, but it's only the beginning of the story from
an effective usage perspective
Regards,
Hervé
>
> Chris
>
>
> Am 01.11.19, 09:24 schrieb "Hervé BOUTEMY" <he...@free.fr>:
>
> Le jeudi 31 octobre 2019, 17:26:52 CET Christofer Dutz a écrit :
>
> > Hi all,
> >
> > as I can see you're voting on releasing the reproducible build
> > extended
> > plugin versions.
>
> Is there any documentation on how to use this new
>
> > feature?
> >
> > I had a look at the confluence page, but that seemed like a
> > brainstorming
> > session.
>
> ok, the Wiki page [1] started as a brainstorming session, was updated to
> a proposal (the "Output Archive Entries Timestamp" parapgraph).
And now I
> probably should order paragraph a little bit, and add a "Making your build
> reproducible" section for end uses to have a quick explanation.
> I'll write the explanation here as a first try before working on the
> Wiki:
> 1. upgrade your plugins to reproducible version, particularly
> mpaven-source-plugin, maven-jar-plugin and maven-assembly-plugin to vesion
> 3.2.0
2. add project.build.outputTimestamp property with the timestamp
> value that will be used in zip/jar/tar archives: <properties>
>
> <project.build.outputTimestamp>2019-10-02T08:04:00Z</project.build.outputTi
> mestamp>
</properties>
>
> Notice:
> - there is no Maven version prerequisite, everything happens at plugins
> level
- Reproducible Builds require to have no version ranges in
> dependencies, generally gives different result on Unixes vs Windows, and
> generally depend on the major version of JDK used to compile (see "Out of
> Scope" paragraph)
> You have the basis configured, the output should be reproducible now.
> If something is still not reproducible, use diffoscope to find the
> unstable output, find the plugin that generated this output and check if
> there is a reproducible version available: if not, please open an issue to
> help plugin maintainers improving Reproducible Builds support at every
> plugin level.
> [1]
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=74682318
>
>
>
> > I would love to add this to the PLC4X build asap.
>
> I'd love to have feedback
> Don't hesitate to ping me.
>
> >
> > So I would like to test the release-candidates and vote too.
>
> I would love to have many tester and votes :)
>
>
> >
> > Chris
> >
> >
> >
> > Am 16.10.19, 14:42 schrieb "Hervé BOUTEMY" <he...@free.fr>:
> >
> >
> > Le mercredi 16 octobre 2019, 13:40:48 CEST Andreas Sewe a écrit :
> >
> >
> >
> > > Emmanuel Bourg wrote:
> > >
> > >
> > > > Le 16/10/2019 à 08:35, Hervé BOUTEMY a écrit :
> > > >
> > > >
> > > >> last question: now that we seem to fully understand each
> > > >> other,
> > > >> does it
> > > >> mean that you don't need any more "seconds since the epoch"
> > > >> format
> > > >> support for the property?
> > > >
> > > >
> > > >
> > > > If Maven supports the SOURCE_DATE_EPOCH environment variable
> > > > I
> > > > don't
> > > > think that's necessary, otherwise it would be nice to be able
> > > > to
> > > > invoke
> > > >
> > > > Maven with:
> > > >
> > > >
> > > > mvn package
> > > > -Dproject.build.outputTimestamp=$SOURCE_DATE_EPOCH
> > > >
> > > >
> > > >
> > > > and this means supporting a timestamp formatted as seconds
> > > > since
> > > > the
> > > > epoch.
> > >
> > >
> > >
> > > +1
> > >
> > > The above would be a nice, simple way of bridging the gap
> > > between
> > > SOURCE_DATE_EPOCH and project.build.outputTimestamp.
> >
> >
> >
> > told like that, ok, why not
> >
> >
> >
> >
> > >
> > > If it is not too much trouble to implement the "\d+ -> seconds
> > > since
> > > epoch" heuristic, them I would love to see it included.
> >
> >
> >
> > ok, I'll do and prepare the release
> >
> > Regards,
> >
> > Hervé
> >
> >
> >
> >
> > >
> > > Best wishes,
> > >
> > > Andreas
> >
> >
> >
> >
> >
> >
> >
> >
> > ------------------------------------------------------------------
> > ---
> > To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> > For additional commands, e-mail: dev-help@maven.apache.org
> >
> >
> >
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> > For additional commands, e-mail: dev-help@maven.apache.org
> >
>
>
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> For additional commands, e-mail: dev-help@maven.apache.org
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: last review of Reproducible Builds proposal
Posted by Hervé BOUTEMY <he...@free.fr>.
please provide an example of a visible issue so we can work on it
Regards,
Hervé
Le vendredi 1 novembre 2019, 10:27:36 CET Vladimir Sitnikov a écrit :
> OpenJDK8 uses hashmap for manifest entries, so jar files are not really
> reproducible there.
>
> Note: there's run-to-run randomization in j.u.HashMap, so the manifest
> contents is not predictable.
>
> Vladimir
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: last review of Reproducible Builds proposal
Posted by Vladimir Sitnikov <si...@gmail.com>.
OpenJDK8 uses hashmap for manifest entries, so jar files are not really
reproducible there.
Note: there's run-to-run randomization in j.u.HashMap, so the manifest
contents is not predictable.
Vladimir
Re: last review of Reproducible Builds proposal
Posted by Christofer Dutz <ch...@c-ware.de>.
Hi Herve,
thanks for that will try it asap and report any findings back.
But good to know that there is a difference between JDK major versions and OSes ... so it would probably be best to stage releases on Linux with an OpenJDK of the minimum supported version?
Just thinking how to make it possible to verify without having to buy Mac or Windows licenses ... guess on every machine you could whip up a Ubuntu VM for verification.
Just thinking about it ... perhaps it would be best to create a Docker image for doing the reproducible stuff ...
Are there any plans on creating a plugin to allow verification?
Sort of something like this:
"mvn package release:verify-reproduicble -DstagingRepoUrl=a.b.c.de/repo/blahblahblah"
(Which doesn't deploy the artifacts, but instead download them and do a binary comparison)
Also it could be great if the release-plugin could automatically set the property:
a) if it finds the "project.build.outputTimestamp" set to some placeholder value
b) if some switch tells it to prepare a reproducible build by using some sort of "switch" parameter
Guess that would sort of close the loop to get the biggest benefit out of the reproducible builds.
I would be happy to help as I think this is one of the greatest new features.
(Ok ... perhaps besides the sound-output-extension I learned about yesterday ;-) )
Chris
Am 01.11.19, 09:24 schrieb "Hervé BOUTEMY" <he...@free.fr>:
Le jeudi 31 octobre 2019, 17:26:52 CET Christofer Dutz a écrit :
> Hi all,
>
> as I can see you're voting on releasing the reproducible build extended
> plugin versions.
Is there any documentation on how to use this new
> feature?
>
> I had a look at the confluence page, but that seemed like a brainstorming
> session.
ok, the Wiki page [1] started as a brainstorming session, was updated to a proposal (the "Output Archive Entries Timestamp" parapgraph).
And now I probably should order paragraph a little bit, and add a "Making your build reproducible" section for end uses to have a quick explanation.
I'll write the explanation here as a first try before working on the Wiki:
1. upgrade your plugins to reproducible version, particularly mpaven-source-plugin, maven-jar-plugin and maven-assembly-plugin to vesion 3.2.0
2. add project.build.outputTimestamp property with the timestamp value that will be used in zip/jar/tar archives:
<properties>
<project.build.outputTimestamp>2019-10-02T08:04:00Z</project.build.outputTimestamp>
</properties>
Notice:
- there is no Maven version prerequisite, everything happens at plugins level
- Reproducible Builds require to have no version ranges in dependencies, generally gives different result on Unixes vs Windows, and generally depend on the major version of JDK used to compile (see "Out of Scope" paragraph)
You have the basis configured, the output should be reproducible now.
If something is still not reproducible, use diffoscope to find the unstable output, find the plugin that generated this output and check if there is a reproducible version available: if not, please open an issue to help plugin maintainers improving Reproducible Builds support at every plugin level.
[1] https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=74682318
> I would love to add this to the PLC4X build asap.
I'd love to have feedback
Don't hesitate to ping me.
>
> So I would like to test the release-candidates and vote too.
I would love to have many tester and votes :)
>
> Chris
>
>
>
> Am 16.10.19, 14:42 schrieb "Hervé BOUTEMY" <he...@free.fr>:
>
> Le mercredi 16 octobre 2019, 13:40:48 CEST Andreas Sewe a écrit :
>
> > Emmanuel Bourg wrote:
> >
> > > Le 16/10/2019 à 08:35, Hervé BOUTEMY a écrit :
> > >
> > >> last question: now that we seem to fully understand each other,
> > >> does it
> > >> mean that you don't need any more "seconds since the epoch" format
> > >> support for the property?
> > >
> > >
> > > If Maven supports the SOURCE_DATE_EPOCH environment variable I
> > > don't
> > > think that's necessary, otherwise it would be nice to be able to
> > > invoke
> > >
> > > Maven with:
> > >
> > > mvn package -Dproject.build.outputTimestamp=$SOURCE_DATE_EPOCH
> > >
> > >
> > > and this means supporting a timestamp formatted as seconds since
> > > the
> > > epoch.
> >
> >
> > +1
> >
> > The above would be a nice, simple way of bridging the gap between
> > SOURCE_DATE_EPOCH and project.build.outputTimestamp.
>
> told like that, ok, why not
>
>
> >
> > If it is not too much trouble to implement the "\d+ -> seconds since
> > epoch" heuristic, them I would love to see it included.
>
> ok, I'll do and prepare the release
>
> Regards,
>
> Hervé
>
>
> >
> > Best wishes,
> >
> > Andreas
>
>
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> For additional commands, e-mail: dev-help@maven.apache.org
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> For additional commands, e-mail: dev-help@maven.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: last review of Reproducible Builds proposal
Posted by Hervé BOUTEMY <he...@free.fr>.
Le jeudi 31 octobre 2019, 17:26:52 CET Christofer Dutz a écrit :
> Hi all,
>
> as I can see you're voting on releasing the reproducible build extended
> plugin versions.
Is there any documentation on how to use this new
> feature?
>
> I had a look at the confluence page, but that seemed like a brainstorming
> session.
ok, the Wiki page [1] started as a brainstorming session, was updated to a proposal (the "Output Archive Entries Timestamp" parapgraph).
And now I probably should order paragraph a little bit, and add a "Making your build reproducible" section for end uses to have a quick explanation.
I'll write the explanation here as a first try before working on the Wiki:
1. upgrade your plugins to reproducible version, particularly mpaven-source-plugin, maven-jar-plugin and maven-assembly-plugin to vesion 3.2.0
2. add project.build.outputTimestamp property with the timestamp value that will be used in zip/jar/tar archives:
<properties>
<project.build.outputTimestamp>2019-10-02T08:04:00Z</project.build.outputTimestamp>
</properties>
Notice:
- there is no Maven version prerequisite, everything happens at plugins level
- Reproducible Builds require to have no version ranges in dependencies, generally gives different result on Unixes vs Windows, and generally depend on the major version of JDK used to compile (see "Out of Scope" paragraph)
You have the basis configured, the output should be reproducible now.
If something is still not reproducible, use diffoscope to find the unstable output, find the plugin that generated this output and check if there is a reproducible version available: if not, please open an issue to help plugin maintainers improving Reproducible Builds support at every plugin level.
[1] https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=74682318
> I would love to add this to the PLC4X build asap.
I'd love to have feedback
Don't hesitate to ping me.
>
> So I would like to test the release-candidates and vote too.
I would love to have many tester and votes :)
>
> Chris
>
>
>
> Am 16.10.19, 14:42 schrieb "Hervé BOUTEMY" <he...@free.fr>:
>
> Le mercredi 16 octobre 2019, 13:40:48 CEST Andreas Sewe a écrit :
>
> > Emmanuel Bourg wrote:
> >
> > > Le 16/10/2019 à 08:35, Hervé BOUTEMY a écrit :
> > >
> > >> last question: now that we seem to fully understand each other,
> > >> does it
> > >> mean that you don't need any more "seconds since the epoch" format
> > >> support for the property?
> > >
> > >
> > > If Maven supports the SOURCE_DATE_EPOCH environment variable I
> > > don't
> > > think that's necessary, otherwise it would be nice to be able to
> > > invoke
> > >
> > > Maven with:
> > >
> > > mvn package -Dproject.build.outputTimestamp=$SOURCE_DATE_EPOCH
> > >
> > >
> > > and this means supporting a timestamp formatted as seconds since
> > > the
> > > epoch.
> >
> >
> > +1
> >
> > The above would be a nice, simple way of bridging the gap between
> > SOURCE_DATE_EPOCH and project.build.outputTimestamp.
>
> told like that, ok, why not
>
>
> >
> > If it is not too much trouble to implement the "\d+ -> seconds since
> > epoch" heuristic, them I would love to see it included.
>
> ok, I'll do and prepare the release
>
> Regards,
>
> Hervé
>
>
> >
> > Best wishes,
> >
> > Andreas
>
>
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> For additional commands, e-mail: dev-help@maven.apache.org
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> For additional commands, e-mail: dev-help@maven.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: last review of Reproducible Builds proposal
Posted by Christofer Dutz <ch...@c-ware.de>.
Hi all,
as I can see you're voting on releasing the reproducible build extended plugin versions.
Is there any documentation on how to use this new feature?
I had a look at the confluence page, but that seemed like a brainstorming session.
I would love to add this to the PLC4X build asap.
So I would like to test the release-candidates and vote too.
Chris
Am 16.10.19, 14:42 schrieb "Hervé BOUTEMY" <he...@free.fr>:
Le mercredi 16 octobre 2019, 13:40:48 CEST Andreas Sewe a écrit :
> Emmanuel Bourg wrote:
> > Le 16/10/2019 à 08:35, Hervé BOUTEMY a écrit :
> >> last question: now that we seem to fully understand each other, does it
> >> mean that you don't need any more "seconds since the epoch" format
> >> support for the property?
> >
> > If Maven supports the SOURCE_DATE_EPOCH environment variable I don't
> > think that's necessary, otherwise it would be nice to be able to invoke
> >
> > Maven with:
> > mvn package -Dproject.build.outputTimestamp=$SOURCE_DATE_EPOCH
> >
> > and this means supporting a timestamp formatted as seconds since the
> > epoch.
>
> +1
>
> The above would be a nice, simple way of bridging the gap between
> SOURCE_DATE_EPOCH and project.build.outputTimestamp.
told like that, ok, why not
>
> If it is not too much trouble to implement the "\d+ -> seconds since
> epoch" heuristic, them I would love to see it included.
ok, I'll do and prepare the release
Regards,
Hervé
>
> Best wishes,
>
> Andreas
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: last review of Reproducible Builds proposal
Posted by Hervé BOUTEMY <he...@free.fr>.
Le mercredi 16 octobre 2019, 13:40:48 CEST Andreas Sewe a écrit :
> Emmanuel Bourg wrote:
> > Le 16/10/2019 à 08:35, Hervé BOUTEMY a écrit :
> >> last question: now that we seem to fully understand each other, does it
> >> mean that you don't need any more "seconds since the epoch" format
> >> support for the property?
> >
> > If Maven supports the SOURCE_DATE_EPOCH environment variable I don't
> > think that's necessary, otherwise it would be nice to be able to invoke
> >
> > Maven with:
> > mvn package -Dproject.build.outputTimestamp=$SOURCE_DATE_EPOCH
> >
> > and this means supporting a timestamp formatted as seconds since the
> > epoch.
>
> +1
>
> The above would be a nice, simple way of bridging the gap between
> SOURCE_DATE_EPOCH and project.build.outputTimestamp.
told like that, ok, why not
>
> If it is not too much trouble to implement the "\d+ -> seconds since
> epoch" heuristic, them I would love to see it included.
ok, I'll do and prepare the release
Regards,
Hervé
>
> Best wishes,
>
> Andreas
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: last review of Reproducible Builds proposal
Posted by Andreas Sewe <se...@st.informatik.tu-darmstadt.de>.
Emmanuel Bourg wrote:
> Le 16/10/2019 à 08:35, Hervé BOUTEMY a écrit :
>> last question: now that we seem to fully understand each other, does it mean
>> that you don't need any more "seconds since the epoch" format support for the
>> property?
>
> If Maven supports the SOURCE_DATE_EPOCH environment variable I don't
> think that's necessary, otherwise it would be nice to be able to invoke
> Maven with:
>
> mvn package -Dproject.build.outputTimestamp=$SOURCE_DATE_EPOCH
>
> and this means supporting a timestamp formatted as seconds since the epoch.
+1
The above would be a nice, simple way of bridging the gap between
SOURCE_DATE_EPOCH and project.build.outputTimestamp.
If it is not too much trouble to implement the "\d+ -> seconds since
epoch" heuristic, them I would love to see it included.
Best wishes,
Andreas
Re: last review of Reproducible Builds proposal
Posted by Emmanuel Bourg <eb...@apache.org>.
Le 16/10/2019 à 08:35, Hervé BOUTEMY a écrit :
> last question: now that we seem to fully understand each other, does it mean
> that you don't need any more "seconds since the epoch" format support for the
> property?
If Maven supports the SOURCE_DATE_EPOCH environment variable I don't
think that's necessary, otherwise it would be nice to be able to invoke
Maven with:
mvn package -Dproject.build.outputTimestamp=$SOURCE_DATE_EPOCH
and this means supporting a timestamp formatted as seconds since the epoch.
Emmanuel Bourg
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: last review of Reproducible Builds proposal
Posted by Michael Osipov <mi...@apache.org>.
Am 2019-10-16 um 08:35 schrieb Hervé BOUTEMY:
> last question: now that we seem to fully understand each other, does it mean
> that you don't need any more "seconds since the epoch" format support for the
> property?
>
> I can add it if you feel it may be useful, since it won't break any logic,
> even if I don't really see the use case given the approach of "native Maven
> binary Reproducible Builds".
>
> It's the last little decision to do before I do the maven-archiver release
> that will lead to packaging plugins releases
Hervé,
one could satisfy all with namespaces:
1. <element>seconds_since_epoch:12324...</element>
2. <element>java_ts:122344....<element>
3. <element>iso:2019-...</element>
4. <element>2019...</element>
WDYT?
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: last review of Reproducible Builds proposal
Posted by Hervé BOUTEMY <he...@free.fr>.
last question: now that we seem to fully understand each other, does it mean
that you don't need any more "seconds since the epoch" format support for the
property?
I can add it if you feel it may be useful, since it won't break any logic,
even if I don't really see the use case given the approach of "native Maven
binary Reproducible Builds".
It's the last little decision to do before I do the maven-archiver release
that will lead to packaging plugins releases
Regards,
Hervé
Le jeudi 10 octobre 2019, 23:50:34 CEST Emmanuel Bourg a écrit :
> Le 10/10/2019 à 19:28, Hervé BOUTEMY a écrit :
> > the only little mis-interpretation is that it is a pure build information,
> > then I don't see why this property would appear in a consumer POM
>
> Thank you for the clarification, that makes perfectly sense. And I now
> see the benefit of using a property that can be inherited. In a multi
> modules project it's only necessary to define the timestamp once in the
> root pom. Parent poms deployed to Maven Central will never include a
> timestamp and there is no risk of affecting other projects deriving from
> the pom.
>
> Emmanuel Bourg
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> For additional commands, e-mail: dev-help@maven.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: last review of Reproducible Builds proposal
Posted by Hervé BOUTEMY <he...@free.fr>.
Le vendredi 11 octobre 2019, 10:03:10 CEST Christofer Dutz a écrit :
> Hi Enrico,
>
> so I would definitely +1 to be able to do it this way ...
> I think reproducible builds are important for releases, but I am not that
> sure the same applies for the daily business in a project. For being able
> to do releases it would be a huge improvement, if this is automatically
> handled the same way the versions are updated.
see https://issues.apache.org/jira/browse/MRELEASE-1029
> For the plc4x project I was even thinking of building some maven tooling to
> automatically verify the built archives with the ones staged in nexus for
> binary equality.
yes, that's a next step: I see we have the same idea :)
At the moment, this is what I expect from Buildinfo file proposal we wrote
(with a sbt developer, since that proposal is globally JVM-oriented, whatever
the build tool is):
https://reproducible-builds.org/docs/jvm/
If the release manager publishes the buildinfo of his "reference" release
build (= 1 buildinfo, even if there are thousands of artifacts), it could be
easily compared to a rebuilder's buildinfo done during his local rebuild.
One year ago, I was ready to write a Maven plugin for generating such
buildinfo, when I thought that starting from this plugin would not bring real
value until we had a chance to get reproducible results. Now that native
Reproducible Builds for Maven is near, writing the plugin that will generate a
buildinfo file makes sense and should not be really complex (less complex than
MRELEASE-1029 , for example)
For those interested in Reproducible Builds, next yearly meeting is in
december https://reproducible-builds.org/events/
Last year event in Paris was key to my understanding of so many aspects we'll
need to manage in future years to get the full value of Reproducible Builds in
Java
Of course, we can also work on this during Apache Conference Europe 2019
https://aceu19.apachecon.com/
I hope to meet a lot of people and discuss a lot of good steps forward
Regards,
Hervé
> Chris
>
> Am 11.10.19, 09:05 schrieb "Enrico Olivelli" <eo...@gmail.com>:
>
> Il ven 11 ott 2019, 08:31 Christofer Dutz <ch...@c-ware.de>
> ha
scritto:
>
>
> > Just a small question. I have been following this thread with great
> > interest.
> >
> >
> >
> > I think this is going to be a big thing when it makes the changes
> > available to the main maven system.
> >
> >
> >
> > So as far as I understand the core part will be a fixed timestamp
> > which
> > will then be used throughout the build by multiple pluggins.
> >
> >
> >
> > So if I provide the same timestamp the result should be binary
> > identical.
> >
> >
> >
> > Would it be possible to have this timestamp written/updated in the pom
> > as
> > part of the release:prepare step?
> >
> >
>
>
> Yep, this was one of my questions at the beginning of this thread.
> I see value in this proposal
>
> Enrico
>
>
>
>
>
> > Sort of setting it to some constant (ie REPODUCEABLE_BUILD_TIMESTAMP)
> > simply takes the current time but if there is a concrete value, it
> > uses
> > that instead?
> >
> >
> >
> > Hope im not asking anything obvious. To me it looked as if the
> > timestamp
> > has to be manipulated manually.
> >
> >
> >
> > Chris
> >
> >
> >
> > Holen Sie sich Outlook für Android<https://aka.ms/ghei36>
> > ________________________________
> > From: Emmanuel Bourg <eb...@apache.org>
> > Sent: Thursday, October 10, 2019 11:50:34 PM
> > To: dev@maven.apache.org <de...@maven.apache.org>
> > Subject: Re: last review of Reproducible Builds proposal
> >
> >
> >
> > Le 10/10/2019 à 19:28, Hervé BOUTEMY a écrit :
> >
> >
> >
> > > the only little mis-interpretation is that it is a pure build
> >
> > information,
> >
> > > then I don't see why this property would appear in a consumer POM
> >
> >
> >
> > Thank you for the clarification, that makes perfectly sense. And I
> > now
> > see the benefit of using a property that can be inherited. In a multi
> > modules project it's only necessary to define the timestamp once in
> > the
> > root pom. Parent poms deployed to Maven Central will never include a
> > timestamp and there is no risk of affecting other projects deriving
> > from
> > the pom.
> >
> >
> >
> > Emmanuel Bourg
> >
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> > For additional commands, e-mail: dev-help@maven.apache.org
> >
> >
> >
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: last review of Reproducible Builds proposal
Posted by Christofer Dutz <ch...@c-ware.de>.
Hi Enrico,
so I would definitely +1 to be able to do it this way ...
I think reproducible builds are important for releases, but I am not that sure the same applies for the daily business in a project.
For being able to do releases it would be a huge improvement, if this is automatically handled the same way the versions are updated.
For the plc4x project I was even thinking of building some maven tooling to automatically verify the built archives with the ones staged in nexus for binary equality.
Chris
Am 11.10.19, 09:05 schrieb "Enrico Olivelli" <eo...@gmail.com>:
Il ven 11 ott 2019, 08:31 Christofer Dutz <ch...@c-ware.de> ha
scritto:
> Just a small question. I have been following this thread with great
> interest.
>
> I think this is going to be a big thing when it makes the changes
> available to the main maven system.
>
> So as far as I understand the core part will be a fixed timestamp which
> will then be used throughout the build by multiple pluggins.
>
> So if I provide the same timestamp the result should be binary identical.
>
> Would it be possible to have this timestamp written/updated in the pom as
> part of the release:prepare step?
>
Yep, this was one of my questions at the beginning of this thread.
I see value in this proposal
Enrico
> Sort of setting it to some constant (ie REPODUCEABLE_BUILD_TIMESTAMP)
> simply takes the current time but if there is a concrete value, it uses
> that instead?
>
> Hope im not asking anything obvious. To me it looked as if the timestamp
> has to be manipulated manually.
>
> Chris
>
> Holen Sie sich Outlook für Android<https://aka.ms/ghei36>
> ________________________________
> From: Emmanuel Bourg <eb...@apache.org>
> Sent: Thursday, October 10, 2019 11:50:34 PM
> To: dev@maven.apache.org <de...@maven.apache.org>
> Subject: Re: last review of Reproducible Builds proposal
>
> Le 10/10/2019 à 19:28, Hervé BOUTEMY a écrit :
>
> > the only little mis-interpretation is that it is a pure build
> information,
> > then I don't see why this property would appear in a consumer POM
>
> Thank you for the clarification, that makes perfectly sense. And I now
> see the benefit of using a property that can be inherited. In a multi
> modules project it's only necessary to define the timestamp once in the
> root pom. Parent poms deployed to Maven Central will never include a
> timestamp and there is no risk of affecting other projects deriving from
> the pom.
>
> Emmanuel Bourg
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> For additional commands, e-mail: dev-help@maven.apache.org
>
>
Re: last review of Reproducible Builds proposal
Posted by Enrico Olivelli <eo...@gmail.com>.
Il ven 11 ott 2019, 08:31 Christofer Dutz <ch...@c-ware.de> ha
scritto:
> Just a small question. I have been following this thread with great
> interest.
>
> I think this is going to be a big thing when it makes the changes
> available to the main maven system.
>
> So as far as I understand the core part will be a fixed timestamp which
> will then be used throughout the build by multiple pluggins.
>
> So if I provide the same timestamp the result should be binary identical.
>
> Would it be possible to have this timestamp written/updated in the pom as
> part of the release:prepare step?
>
Yep, this was one of my questions at the beginning of this thread.
I see value in this proposal
Enrico
> Sort of setting it to some constant (ie REPODUCEABLE_BUILD_TIMESTAMP)
> simply takes the current time but if there is a concrete value, it uses
> that instead?
>
> Hope im not asking anything obvious. To me it looked as if the timestamp
> has to be manipulated manually.
>
> Chris
>
> Holen Sie sich Outlook für Android<https://aka.ms/ghei36>
> ________________________________
> From: Emmanuel Bourg <eb...@apache.org>
> Sent: Thursday, October 10, 2019 11:50:34 PM
> To: dev@maven.apache.org <de...@maven.apache.org>
> Subject: Re: last review of Reproducible Builds proposal
>
> Le 10/10/2019 à 19:28, Hervé BOUTEMY a écrit :
>
> > the only little mis-interpretation is that it is a pure build
> information,
> > then I don't see why this property would appear in a consumer POM
>
> Thank you for the clarification, that makes perfectly sense. And I now
> see the benefit of using a property that can be inherited. In a multi
> modules project it's only necessary to define the timestamp once in the
> root pom. Parent poms deployed to Maven Central will never include a
> timestamp and there is no risk of affecting other projects deriving from
> the pom.
>
> Emmanuel Bourg
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> For additional commands, e-mail: dev-help@maven.apache.org
>
>
Re: last review of Reproducible Builds proposal
Posted by Christofer Dutz <ch...@c-ware.de>.
Just a small question. I have been following this thread with great interest.
I think this is going to be a big thing when it makes the changes available to the main maven system.
So as far as I understand the core part will be a fixed timestamp which will then be used throughout the build by multiple pluggins.
So if I provide the same timestamp the result should be binary identical.
Would it be possible to have this timestamp written/updated in the pom as part of the release:prepare step?
Sort of setting it to some constant (ie REPODUCEABLE_BUILD_TIMESTAMP) simply takes the current time but if there is a concrete value, it uses that instead?
Hope im not asking anything obvious. To me it looked as if the timestamp has to be manipulated manually.
Chris
Holen Sie sich Outlook für Android<https://aka.ms/ghei36>
________________________________
From: Emmanuel Bourg <eb...@apache.org>
Sent: Thursday, October 10, 2019 11:50:34 PM
To: dev@maven.apache.org <de...@maven.apache.org>
Subject: Re: last review of Reproducible Builds proposal
Le 10/10/2019 à 19:28, Hervé BOUTEMY a écrit :
> the only little mis-interpretation is that it is a pure build information,
> then I don't see why this property would appear in a consumer POM
Thank you for the clarification, that makes perfectly sense. And I now
see the benefit of using a property that can be inherited. In a multi
modules project it's only necessary to define the timestamp once in the
root pom. Parent poms deployed to Maven Central will never include a
timestamp and there is no risk of affecting other projects deriving from
the pom.
Emmanuel Bourg
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: last review of Reproducible Builds proposal
Posted by Emmanuel Bourg <eb...@apache.org>.
Le 10/10/2019 à 19:28, Hervé BOUTEMY a écrit :
> the only little mis-interpretation is that it is a pure build information,
> then I don't see why this property would appear in a consumer POM
Thank you for the clarification, that makes perfectly sense. And I now
see the benefit of using a property that can be inherited. In a multi
modules project it's only necessary to define the timestamp once in the
root pom. Parent poms deployed to Maven Central will never include a
timestamp and there is no risk of affecting other projects deriving from
the pom.
Emmanuel Bourg
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: last review of Reproducible Builds proposal
Posted by Hervé BOUTEMY <he...@free.fr>.
Le jeudi 10 octobre 2019 17:17:07 CEST, vous avez écrit :
> Hi Hervé,
>
> >> - We already have maven.build.timestamp [1], so I wonder whether it
> >> should be maven.build.outputTimestap, too. Although one may argue that
> >> this *output* timestamp is a property of the project being built, not
> >> the Maven execution (in particular if the property becomes a POM element
> >> in POM version 5).
> >
> > you got it: it's project.* because it will be a POM element in POM v5
>
> Makes sense. Actually, my interpretation of the maven.* prefix (as being
> for the execution itself) seems to be wrong: A lot of core
> maven-*-plugins use this prefix for their properties, e.g.,
> maven.clean.failOnError, maven.install.skip, or maven.jar.forceCreation.
>
> Anyway, project.* makes more sense.
>
> >> Unfortunately, SimpleDateFormat or its java.time equivalents do not seem
> >> to have a format string for "seconds since the epoch", so we may want to
> >> make "seconds since the epoch" the default for
> >> project.build.outputTimestamp's format and only parse according to
> >> project.build.outputTimestamp.format when that property is indeed set.
> >>
> >> WDYT?
> >
> > the parsing from String to Date is done in maven-archiver code [1], then
> > we
> > can have the algorithm we want. We can even code that if the value is only
> > digits, it's parsed as "seconds since the epoch": no need to add the
> > format
> > property, the choice on one format over the other would be based on
> > effective value
>
> Yes, all digits -> "seconds since epoch" would be a reasonable heuristic.
>
> > What I don't get is: why do you absolutely want such a format?
> > And why do you want to use ${env.SOURCE_DATE_EPOCH}?
> > The proposal is about having "native" Reproducible Builds for Maven, then
> > not something added by an external rebuilder: it will be configured by
> > the original release.
>
> OK, I think I have misunderstood your use case:
>
> In the make-style builds I know that use SOURCE_DATE_EPOCH, some build
> step typically extracts a commit timestamp from, say, git and then sets
> SOURCE_DATE_EPOCH. This variable is then read and interpreted by the
> various built steps that follow. In other words, SOURCE_DATE_EPOCH is
> fed into the build from the outside.
>
> You seem to envision project.build.outputTimestamp to be hard-coded in
> the (consumer?) POM, i.e., not added from the outside (e.g., via
> ${env.SOURCE_DATE_EPOCH}). Is this interpretation correct?
yes
the only little mis-interpretation is that it is a pure build information,
then I don't see why this property would appear in a consumer POM
you can test it by building the fully working PoC:
https://github.com/apache/maven/tree/reproducible
>
> This means that updating the value would fall, e.g., to the
> maven-release-plugin (via [1]), so only releases would have a
> project.build.outputTimestamp (matching the release's time, not the
> timestamp of the corresponding commit, I assume).
the release plugin will update the release timestamp, yes.
But SNAPSHOTs would also have a timestamp just because they kept the timestamp
of previous release or they inherited from parent
>
> I would prefer if each commit would get an output timestamp matching the
> commit timestamp. Maybe one can use the buildnumber-maven-plugin [1] to
> achieve this, though. Setting <timestampPropertyName> to
> project.build.outputTimestamp would initialize that property during the
> "initialize" phase. Would that be early enough to be picked up by rest
> of the build.
imagine the commit is about a unit test, or documentation: changing the
timestamp would change the jar that has exactly the same .class content.
I think that updating the timestamp on every commit for SNAPSHOTs can seem
logical, but finally it would be counter-productive: that would break one
benefit of Reproducible Builds we can have for SNAPSHOTS is that we can use jar
hash to know if the code has changed or not = one use case that someone was
interested in when I had discussions with people (not on mailing list)
>
> Also, would a property set this way be part of the consumer POM?
I see this data only useful at build time: I don't know why it could be useful
at consume time
>
> I hope you can see what I am getting at, even if I might not have
> explained it very well.
I understand your solutions that look reasonably feasible, but not really
their objective.
But I appreciate the concrete discussion: maybe you have a use case that was
not defined before
Regards,
Hervé
>
> Best wishes,
>
> Andreas
>
> [1] <<https://issues.apache.org/jira/browse/MRELEASE-1029>
> [2] <https://www.mojohaus.org/buildnumber-maven-plugin/create-mojo.html>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org