You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@commons.apache.org by Aman Sharma <am...@kth.se.INVALID> on 2025/01/27 17:07:55 UTC
Release commons-daemon:commons-daemon:1.4.0 is unreproducible
Hi all,
I am trying to investigate why `commons-daemon:commons-daemon:1.4.0` release is unreproducible and how to fix it. Since I did not see any related mail here<https://lists.apache.org/list?dev@commons.apache.org:lte=1M:reproducible>, I thought I should start another thread.
The diff between the artifacts are documented here<https://github.com/jvm-repo-rebuild/reproducible-central/blob/master/content/org/apache/commons/commons-daemon/commons-daemon-1.4.0.diffoscope>. It seems most of the zip and tar archives have some binaries missing. For example, `commons-daemon-1.4.0-bin-windows.zip` has three missing Windows executables. Checking the release instructions<https://github.com/apache/commons-daemon/blob/2b4fa9a14b8b073ade4c95ba4564455c06e1ae29/HOWTO-RELEASE.txt#L83-L100>, it seems that these executables require running some shell scripts and nmake command.
Hence, I tried to make the release reproducible by placing the commands under `execBefore` in the buildspec file<https://github.com/jvm-repo-rebuild/reproducible-central/blob/master/doc/BUILDSPEC.md>. It seems that it fixes artifacts "commons-daemon-1.4.0-native-src.tar.gz commons-daemon-1.4.0-native-src.zip commons-daemon-1.4.0-src.tar.gz commons-daemon-1.4.0-src.zip", however, `commons-daemon-1.4.0-bin-windows.zip` still marked as unreproducible. I saw that it builds with nmake which is on Windows. A solution make build scripts portable is to use `cmake`. I think successfully building them and place them in the target folder should achieve a successful reproducible build.
Regards,
Aman Sharma
PhD Student
KTH Royal Institute of Technology
School of Electrical Engineering and Computer Science (EECS)
Department of Theoretical Computer Science (TCS)
<http://www.kth.se><https://www.kth.se/profile/amansha><https://www.kth.se/profile/amansha>
<https://www.kth.se/profile/amansha>https://algomaster99.github.io/
Re: Release commons-daemon:commons-daemon:1.4.0 is unreproducible
Posted by sebb <se...@gmail.com>.
On Tue, 28 Jan 2025 at 07:10, Herve Boutemy <hb...@apache.org> wrote:
>
> thank you Aman,
>
> hi level view of results: https://github.com/jvm-repo-rebuild/reproducible-central/blob/master/content/org/apache/commons/commons-daemon/README.md
>
> reading release instructions https://github.com/apache/commons-daemon/blob/master/HOWTO-RELEASE.txt
>
> IIUC, a Windows machine is expected to create the Windows binaries.
> It will make it hard for Reproducible Central to automatically rebuild: we'll need to at least document this.
> Eventually mark commons-daemon-*-bin-windows.zip as ignore during comparison.
> Unlike SPDX file, executable output are really runtme output, ignoring the fact that we did not reproduce may bring a false sense of rebuilding everything... Need to think and choose.
>
> Another finding when analyzing differences between rebuild and published binaries is that "configure" script is stored in "src" archives (multiple formats): is it intentional?
> If intentional, we'll have to define if we can build it on Linux
Might be worth looking at how crypto is built, as that also has native code.
> Regards,
>
> Hervé
>
> On 2025/01/27 22:31:20 Aman Sharma wrote:
> > It has been pointed out to me that this claim is too strong. I would want to take it back and simply admit that I was not able to reproduce the artifacts from source. I will appreciate any help to build those Windows binary and package them in the jar on my Linux machine.
> >
> >
> > Regards,
> > Aman Sharma
> >
> > PhD Student
> > KTH Royal Institute of Technology
> > School of Electrical Engineering and Computer Science (EECS)
> > Department of Theoretical Computer Science (TCS)
> > <http://www.kth.se><https://www.kth.se/profile/amansha><https://www.kth.se/profile/amansha>
> > <https://www.kth.se/profile/amansha>https://algomaster99.github.io/
> > ________________________________
> > From: Aman Sharma
> > Sent: Monday, January 27, 2025 6:07:55 PM
> > To: dev@commons.apache.org
> > Cc: herve.boutemy@gmail.com
> > Subject: Release commons-daemon:commons-daemon:1.4.0 is unreproducible
> >
> >
> > Hi all,
> >
> >
> > I am trying to investigate why `commons-daemon:commons-daemon:1.4.0` release is unreproducible and how to fix it. Since I did not see any related mail here<https://lists.apache.org/list?dev@commons.apache.org:lte=1M:reproducible>, I thought I should start another thread.
> >
> >
> > The diff between the artifacts are documented here<https://github.com/jvm-repo-rebuild/reproducible-central/blob/master/content/org/apache/commons/commons-daemon/commons-daemon-1.4.0.diffoscope>. It seems most of the zip and tar archives have some binaries missing. For example, `commons-daemon-1.4.0-bin-windows.zip` has three missing Windows executables. Checking the release instructions<https://github.com/apache/commons-daemon/blob/2b4fa9a14b8b073ade4c95ba4564455c06e1ae29/HOWTO-RELEASE.txt#L83-L100>, it seems that these executables require running some shell scripts and nmake command.
> >
> >
> > Hence, I tried to make the release reproducible by placing the commands under `execBefore` in the buildspec file<https://github.com/jvm-repo-rebuild/reproducible-central/blob/master/doc/BUILDSPEC.md>. It seems that it fixes artifacts "commons-daemon-1.4.0-native-src.tar.gz commons-daemon-1.4.0-native-src.zip commons-daemon-1.4.0-src.tar.gz commons-daemon-1.4.0-src.zip", however, `commons-daemon-1.4.0-bin-windows.zip` still marked as unreproducible. I saw that it builds with nmake which is on Windows. A solution make build scripts portable is to use `cmake`. I think successfully building them and place them in the target folder should achieve a successful reproducible build.
> >
> >
> > Regards,
> > Aman Sharma
> >
> > PhD Student
> > KTH Royal Institute of Technology
> > School of Electrical Engineering and Computer Science (EECS)
> > Department of Theoretical Computer Science (TCS)
> > <http://www.kth.se><https://www.kth.se/profile/amansha><https://www.kth.se/profile/amansha>
> > <https://www.kth.se/profile/amansha>https://algomaster99.github.io/
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: Release commons-daemon:commons-daemon:1.4.0 is unreproducible
Posted by Herve Boutemy <hb...@apache.org>.
thank you Aman,
hi level view of results: https://github.com/jvm-repo-rebuild/reproducible-central/blob/master/content/org/apache/commons/commons-daemon/README.md
reading release instructions https://github.com/apache/commons-daemon/blob/master/HOWTO-RELEASE.txt
IIUC, a Windows machine is expected to create the Windows binaries.
It will make it hard for Reproducible Central to automatically rebuild: we'll need to at least document this.
Eventually mark commons-daemon-*-bin-windows.zip as ignore during comparison.
Unlike SPDX file, executable output are really runtme output, ignoring the fact that we did not reproduce may bring a false sense of rebuilding everything... Need to think and choose.
Another finding when analyzing differences between rebuild and published binaries is that "configure" script is stored in "src" archives (multiple formats): is it intentional?
If intentional, we'll have to define if we can build it on Linux
Regards,
Hervé
On 2025/01/27 22:31:20 Aman Sharma wrote:
> It has been pointed out to me that this claim is too strong. I would want to take it back and simply admit that I was not able to reproduce the artifacts from source. I will appreciate any help to build those Windows binary and package them in the jar on my Linux machine.
>
>
> Regards,
> Aman Sharma
>
> PhD Student
> KTH Royal Institute of Technology
> School of Electrical Engineering and Computer Science (EECS)
> Department of Theoretical Computer Science (TCS)
> <http://www.kth.se><https://www.kth.se/profile/amansha><https://www.kth.se/profile/amansha>
> <https://www.kth.se/profile/amansha>https://algomaster99.github.io/
> ________________________________
> From: Aman Sharma
> Sent: Monday, January 27, 2025 6:07:55 PM
> To: dev@commons.apache.org
> Cc: herve.boutemy@gmail.com
> Subject: Release commons-daemon:commons-daemon:1.4.0 is unreproducible
>
>
> Hi all,
>
>
> I am trying to investigate why `commons-daemon:commons-daemon:1.4.0` release is unreproducible and how to fix it. Since I did not see any related mail here<https://lists.apache.org/list?dev@commons.apache.org:lte=1M:reproducible>, I thought I should start another thread.
>
>
> The diff between the artifacts are documented here<https://github.com/jvm-repo-rebuild/reproducible-central/blob/master/content/org/apache/commons/commons-daemon/commons-daemon-1.4.0.diffoscope>. It seems most of the zip and tar archives have some binaries missing. For example, `commons-daemon-1.4.0-bin-windows.zip` has three missing Windows executables. Checking the release instructions<https://github.com/apache/commons-daemon/blob/2b4fa9a14b8b073ade4c95ba4564455c06e1ae29/HOWTO-RELEASE.txt#L83-L100>, it seems that these executables require running some shell scripts and nmake command.
>
>
> Hence, I tried to make the release reproducible by placing the commands under `execBefore` in the buildspec file<https://github.com/jvm-repo-rebuild/reproducible-central/blob/master/doc/BUILDSPEC.md>. It seems that it fixes artifacts "commons-daemon-1.4.0-native-src.tar.gz commons-daemon-1.4.0-native-src.zip commons-daemon-1.4.0-src.tar.gz commons-daemon-1.4.0-src.zip", however, `commons-daemon-1.4.0-bin-windows.zip` still marked as unreproducible. I saw that it builds with nmake which is on Windows. A solution make build scripts portable is to use `cmake`. I think successfully building them and place them in the target folder should achieve a successful reproducible build.
>
>
> Regards,
> Aman Sharma
>
> PhD Student
> KTH Royal Institute of Technology
> School of Electrical Engineering and Computer Science (EECS)
> Department of Theoretical Computer Science (TCS)
> <http://www.kth.se><https://www.kth.se/profile/amansha><https://www.kth.se/profile/amansha>
> <https://www.kth.se/profile/amansha>https://algomaster99.github.io/
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: Release commons-daemon:commons-daemon:1.4.0 is unreproducible
Posted by Aman Sharma <am...@kth.se.INVALID>.
It has been pointed out to me that this claim is too strong. I would want to take it back and simply admit that I was not able to reproduce the artifacts from source. I will appreciate any help to build those Windows binary and package them in the jar on my Linux machine.
Regards,
Aman Sharma
PhD Student
KTH Royal Institute of Technology
School of Electrical Engineering and Computer Science (EECS)
Department of Theoretical Computer Science (TCS)
<http://www.kth.se><https://www.kth.se/profile/amansha><https://www.kth.se/profile/amansha>
<https://www.kth.se/profile/amansha>https://algomaster99.github.io/
________________________________
From: Aman Sharma
Sent: Monday, January 27, 2025 6:07:55 PM
To: dev@commons.apache.org
Cc: herve.boutemy@gmail.com
Subject: Release commons-daemon:commons-daemon:1.4.0 is unreproducible
Hi all,
I am trying to investigate why `commons-daemon:commons-daemon:1.4.0` release is unreproducible and how to fix it. Since I did not see any related mail here<https://lists.apache.org/list?dev@commons.apache.org:lte=1M:reproducible>, I thought I should start another thread.
The diff between the artifacts are documented here<https://github.com/jvm-repo-rebuild/reproducible-central/blob/master/content/org/apache/commons/commons-daemon/commons-daemon-1.4.0.diffoscope>. It seems most of the zip and tar archives have some binaries missing. For example, `commons-daemon-1.4.0-bin-windows.zip` has three missing Windows executables. Checking the release instructions<https://github.com/apache/commons-daemon/blob/2b4fa9a14b8b073ade4c95ba4564455c06e1ae29/HOWTO-RELEASE.txt#L83-L100>, it seems that these executables require running some shell scripts and nmake command.
Hence, I tried to make the release reproducible by placing the commands under `execBefore` in the buildspec file<https://github.com/jvm-repo-rebuild/reproducible-central/blob/master/doc/BUILDSPEC.md>. It seems that it fixes artifacts "commons-daemon-1.4.0-native-src.tar.gz commons-daemon-1.4.0-native-src.zip commons-daemon-1.4.0-src.tar.gz commons-daemon-1.4.0-src.zip", however, `commons-daemon-1.4.0-bin-windows.zip` still marked as unreproducible. I saw that it builds with nmake which is on Windows. A solution make build scripts portable is to use `cmake`. I think successfully building them and place them in the target folder should achieve a successful reproducible build.
Regards,
Aman Sharma
PhD Student
KTH Royal Institute of Technology
School of Electrical Engineering and Computer Science (EECS)
Department of Theoretical Computer Science (TCS)
<http://www.kth.se><https://www.kth.se/profile/amansha><https://www.kth.se/profile/amansha>
<https://www.kth.se/profile/amansha>https://algomaster99.github.io/