You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@maven.apache.org by GitBox <gi...@apache.org> on 2022/07/04 09:51:32 UTC

[GitHub] [maven-javadoc-plugin] jorsol opened a new pull request, #152: Add Integration Test for reproducible builds

jorsol opened a new pull request, #152:
URL: https://github.com/apache/maven-javadoc-plugin/pull/152

   Signed-off-by: Jorge Solórzano <jo...@gmail.com>
   
   Following this checklist to help us incorporate your 
   contribution quickly and easily:
   
    - [X] Make sure there is a [JIRA issue](https://issues.apache.org/jira/browse/MJAVADOC) filed 
          for the change (usually before you start working on it).  Trivial changes like typos do not 
          require a JIRA issue.  Your pull request should address just this issue, without 
          pulling in other changes.
    - [X] Each commit in the pull request should have a meaningful subject line and body.
    - [X] Format the pull request title like `[MJAVADOC-XXX] - Fixes bug in ApproximateQuantiles`,
          where you replace `MJAVADOC-XXX` with the appropriate JIRA issue. Best practice
          is to use the JIRA issue title in the pull request title and in the first line of the 
          commit message.
    - [X] Write a pull request description that is detailed enough to understand what the pull request does, how, and why.
    - [X] Run `mvn clean verify -Prun-its` to make sure basic checks pass. A more thorough check will 
          be performed on your pull request automatically.
   
   If your pull request is about ~20 lines of code you don't need to sign an
   [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf) if you are unsure
   please ask on the developers list.
   
   To make clear that you license your contribution under 
   the [Apache License Version 2.0, January 2004](http://www.apache.org/licenses/LICENSE-2.0)
   you have to acknowledge this by using the following check-box.
   
    - [X] I hereby declare this contribution to be licenced under the [Apache License Version 2.0, January 2004](http://www.apache.org/licenses/LICENSE-2.0)
   
    - [X] In any other case, please file an [Apache Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@maven.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [maven-javadoc-plugin] jorsol commented on a diff in pull request #152: Add Integration Test for reproducible builds

Posted by GitBox <gi...@apache.org>.
jorsol commented on code in PR #152:
URL: https://github.com/apache/maven-javadoc-plugin/pull/152#discussion_r916139511


##########
src/it/projects/reproducible/verify.groovy:
##########
@@ -0,0 +1,65 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+import java.nio.file.attribute.FileTime;
+import java.time.Instant;
+import java.util.jar.*;
+
+def target = new File( basedir, 'target' )
+assert target.isDirectory()
+
+def apidocs = new File( target, 'apidocs' )
+assert apidocs.isDirectory()
+
+def options = new File( apidocs, 'options' )
+assert options.isFile()
+assert options.text.contains( "'Copyright &#169; 2020. All rights reserved.'" )
+
+def artifact = new File( target, 'bar-0.1.0-SNAPSHOT-javadoc.jar' )
+assert artifact.isFile()
+
+// notimestamp in html files
+apidocs.eachFileRecurse
+{
+    if ( it.name.endsWith( '.html' ) )
+    {
+        assert it.text =~ /<!-- Generated by javadoc (\(\d+\) )?-->/
+    }
+}
+
+// Normalize to UTC
+long normalizeUTC( String timestamp )
+{
+  long millis = Instant.parse( timestamp ).toEpochMilli()
+  Calendar cal = Calendar.getInstance()
+  cal.setTimeInMillis( millis )
+  return millis - ( cal.get( Calendar.ZONE_OFFSET ) + cal.get( Calendar.DST_OFFSET ) )
+}
+
+JarFile jar = new JarFile( artifact )
+
+// All entries should have the same timestamp
+FileTime expectedTimestamp = FileTime.fromMillis( normalizeUTC( "2020-02-29T23:59:58Z" ) )
+def testTimestamp =
+{
+    JarEntry entry -> assert expectedTimestamp.equals( entry.getLastModifiedTime() )
+}
+
+jar.stream().forEach( testTimestamp )
+jar.close()

Review Comment:
   Well, I tried to use try-with-resource here, but it looks like it's only supported in Groovy 3+ (with maven-invoker-plugin 3.3.0) and right now it's using Groovy 2.x.
   
   Anyway, this is actually not needed, the [close](https://docs.oracle.com/javase/8/docs/api/java/util/zip/ZipFile.html#close--) method will close all of the input streams previously returned by invocations of the [getInputStream](https://docs.oracle.com/javase/8/docs/api/java/util/zip/ZipFile.html#getInputStream-java.util.zip.ZipEntry-) method, and that method is not called in this test, and even if assert fail the jvm will exit.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@maven.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [maven-javadoc-plugin] slawekjaranowski merged pull request #152: Add Integration Test for reproducible builds

Posted by GitBox <gi...@apache.org>.
slawekjaranowski merged PR #152:
URL: https://github.com/apache/maven-javadoc-plugin/pull/152


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@maven.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [maven-javadoc-plugin] slawekjaranowski commented on a diff in pull request #152: Add Integration Test for reproducible builds

Posted by GitBox <gi...@apache.org>.
slawekjaranowski commented on code in PR #152:
URL: https://github.com/apache/maven-javadoc-plugin/pull/152#discussion_r916144329


##########
src/it/projects/reproducible/verify.groovy:
##########
@@ -0,0 +1,65 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+import java.nio.file.attribute.FileTime;
+import java.time.Instant;
+import java.util.jar.*;
+
+def target = new File( basedir, 'target' )
+assert target.isDirectory()
+
+def apidocs = new File( target, 'apidocs' )
+assert apidocs.isDirectory()
+
+def options = new File( apidocs, 'options' )
+assert options.isFile()
+assert options.text.contains( "'Copyright &#169; 2020. All rights reserved.'" )
+
+def artifact = new File( target, 'bar-0.1.0-SNAPSHOT-javadoc.jar' )
+assert artifact.isFile()
+
+// notimestamp in html files
+apidocs.eachFileRecurse
+{
+    if ( it.name.endsWith( '.html' ) )
+    {
+        assert it.text =~ /<!-- Generated by javadoc (\(\d+\) )?-->/
+    }
+}
+
+// Normalize to UTC
+long normalizeUTC( String timestamp )
+{
+  long millis = Instant.parse( timestamp ).toEpochMilli()
+  Calendar cal = Calendar.getInstance()
+  cal.setTimeInMillis( millis )
+  return millis - ( cal.get( Calendar.ZONE_OFFSET ) + cal.get( Calendar.DST_OFFSET ) )
+}
+
+JarFile jar = new JarFile( artifact )
+
+// All entries should have the same timestamp
+FileTime expectedTimestamp = FileTime.fromMillis( normalizeUTC( "2020-02-29T23:59:58Z" ) )
+def testTimestamp =
+{
+    JarEntry entry -> assert expectedTimestamp.equals( entry.getLastModifiedTime() )
+}
+
+jar.stream().forEach( testTimestamp )
+jar.close()

Review Comment:
   ok,



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@maven.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [maven-javadoc-plugin] slawekjaranowski commented on a diff in pull request #152: Add Integration Test for reproducible builds

Posted by GitBox <gi...@apache.org>.
slawekjaranowski commented on code in PR #152:
URL: https://github.com/apache/maven-javadoc-plugin/pull/152#discussion_r916110842


##########
src/it/projects/reproducible/verify.groovy:
##########
@@ -0,0 +1,65 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+import java.nio.file.attribute.FileTime;
+import java.time.Instant;
+import java.util.jar.*;
+
+def target = new File( basedir, 'target' )
+assert target.isDirectory()
+
+def apidocs = new File( target, 'apidocs' )
+assert apidocs.isDirectory()
+
+def options = new File( apidocs, 'options' )
+assert options.isFile()
+assert options.text.contains( "'Copyright &#169; 2020. All rights reserved.'" )
+
+def artifact = new File( target, 'bar-0.1.0-SNAPSHOT-javadoc.jar' )
+assert artifact.isFile()
+
+// notimestamp in html files
+apidocs.eachFileRecurse
+{
+    if ( it.name.endsWith( '.html' ) )
+    {
+        assert it.text =~ /<!-- Generated by javadoc (\(\d+\) )?-->/
+    }
+}
+
+// Normalize to UTC
+long normalizeUTC( String timestamp )
+{
+  long millis = Instant.parse( timestamp ).toEpochMilli()
+  Calendar cal = Calendar.getInstance()
+  cal.setTimeInMillis( millis )
+  return millis - ( cal.get( Calendar.ZONE_OFFSET ) + cal.get( Calendar.DST_OFFSET ) )
+}
+
+JarFile jar = new JarFile( artifact )
+
+// All entries should have the same timestamp
+FileTime expectedTimestamp = FileTime.fromMillis( normalizeUTC( "2020-02-29T23:59:58Z" ) )
+def testTimestamp =
+{
+    JarEntry entry -> assert expectedTimestamp.equals( entry.getLastModifiedTime() )
+}
+
+jar.stream().forEach( testTimestamp )
+jar.close()

Review Comment:
   Should we use some of try-with-resource here ...?
   If assert fail - should we call a close method on jar?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@maven.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [maven-javadoc-plugin] jorsol commented on pull request #152: Add Integration Test for reproducible builds

Posted by GitBox <gi...@apache.org>.
jorsol commented on PR #152:
URL: https://github.com/apache/maven-javadoc-plugin/pull/152#issuecomment-1178923698

   Skip version 9-14 and other small improvements: https://github.com/apache/maven-javadoc-plugin/pull/155


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@maven.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [maven-javadoc-plugin] jorsol commented on a diff in pull request #152: Add Integration Test for reproducible builds

Posted by GitBox <gi...@apache.org>.
jorsol commented on code in PR #152:
URL: https://github.com/apache/maven-javadoc-plugin/pull/152#discussion_r916139511


##########
src/it/projects/reproducible/verify.groovy:
##########
@@ -0,0 +1,65 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+import java.nio.file.attribute.FileTime;
+import java.time.Instant;
+import java.util.jar.*;
+
+def target = new File( basedir, 'target' )
+assert target.isDirectory()
+
+def apidocs = new File( target, 'apidocs' )
+assert apidocs.isDirectory()
+
+def options = new File( apidocs, 'options' )
+assert options.isFile()
+assert options.text.contains( "'Copyright &#169; 2020. All rights reserved.'" )
+
+def artifact = new File( target, 'bar-0.1.0-SNAPSHOT-javadoc.jar' )
+assert artifact.isFile()
+
+// notimestamp in html files
+apidocs.eachFileRecurse
+{
+    if ( it.name.endsWith( '.html' ) )
+    {
+        assert it.text =~ /<!-- Generated by javadoc (\(\d+\) )?-->/
+    }
+}
+
+// Normalize to UTC
+long normalizeUTC( String timestamp )
+{
+  long millis = Instant.parse( timestamp ).toEpochMilli()
+  Calendar cal = Calendar.getInstance()
+  cal.setTimeInMillis( millis )
+  return millis - ( cal.get( Calendar.ZONE_OFFSET ) + cal.get( Calendar.DST_OFFSET ) )
+}
+
+JarFile jar = new JarFile( artifact )
+
+// All entries should have the same timestamp
+FileTime expectedTimestamp = FileTime.fromMillis( normalizeUTC( "2020-02-29T23:59:58Z" ) )
+def testTimestamp =
+{
+    JarEntry entry -> assert expectedTimestamp.equals( entry.getLastModifiedTime() )
+}
+
+jar.stream().forEach( testTimestamp )
+jar.close()

Review Comment:
   Well, I tried to use try-with-resource here, but it looks like it's only supported in Groovy 3+ and (with maven-invoker-plugin 3.3.0) right now it's using Groovy 2.x.
   
   Anyway, this is actually not needed, the [close](https://docs.oracle.com/javase/8/docs/api/java/util/zip/ZipFile.html#close--) method will close all of the input streams previously returned by invocations of the [getInputStream](https://docs.oracle.com/javase/8/docs/api/java/util/zip/ZipFile.html#getInputStream-java.util.zip.ZipEntry-) method, and that method is not called in this test, and even if assert fail the jvm will exit.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@maven.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [maven-javadoc-plugin] jorsol commented on pull request #152: Add Integration Test for reproducible builds

Posted by GitBox <gi...@apache.org>.
jorsol commented on PR #152:
URL: https://github.com/apache/maven-javadoc-plugin/pull/152#issuecomment-1178804464

   > any hint why fail on jenkins on some nodes ...? https://ci-maven.apache.org/blue/organizations/jenkins/Maven%2Fmaven-box%2Fmaven-javadoc-plugin/detail/master/28/pipeline
   
   Oh my! those pesky bugs from javadoc.
   
   On versions of java lower than 11.0.13, the `-notimestamp` does not work on index.html. See Bug [JDK-8268771](https://bugs.openjdk.org/browse/JDK-8268771), on newer versions (11.0.13+) this was fixed, on Jenkins nodes the versions used are 11.0.12 on Linux and 11.0.11 on Windows, so this triggers the fail.
   
   But even when using newer versions of java (11.0.13+), the second bug is the worst since it uses zipped index files, and the timestamp of the zipped files can change between runs, this is fixed on Java 15+ See Bug [JDK-8237909](https://bugs.openjdk.org/browse/JDK-8237909). In other words, it's almost impossible to get reproducible javadoc jars on Java versions between 9 and 14. Also, this issue is not actually caught on the test, and that's why it passes on GH Actions (that use Java 11.0.15).
   
   There is even a closed issue from Maven: https://issues.apache.org/jira/browse/MJAVADOC-681
   
   I will open another PR that just skips versions between 9 and 14 since there is not much we can do.
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@maven.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [maven-javadoc-plugin] slawekjaranowski commented on pull request #152: Add Integration Test for reproducible builds

Posted by GitBox <gi...@apache.org>.
slawekjaranowski commented on PR #152:
URL: https://github.com/apache/maven-javadoc-plugin/pull/152#issuecomment-1178124690

   any hint why fail on jenkins on some nodes ...?
   https://ci-maven.apache.org/blue/organizations/jenkins/Maven%2Fmaven-box%2Fmaven-javadoc-plugin/detail/master/28/pipeline


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@maven.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org