You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "adoroszlai (via GitHub)" <gi...@apache.org> on 2024/12/06 20:28:16 UTC

[PR] HDDS-11872. Disable Apache snapshots repo [ozone]

adoroszlai opened a new pull request, #7536:
URL: https://github.com/apache/ozone/pull/7536

   ## What changes were proposed in this pull request?
   
   CI build sometimes times out fetching information from snapshot repository, e.g.:
   
   ```
   [INFO] --- cyclonedx-maven-plugin:2.9.1:makeAggregateBom (default) @ ozone-main ---
   [INFO] CycloneDX: Resolving Aggregated Dependencies
   [WARNING] Could not transfer metadata org.apache.ozone:hdds-interface-client:2.0.0-SNAPSHOT/maven-metadata.xml from/to apache.snapshots.https (https://repository.apache.org/content/repositories/snapshots): transfer failed for https://repository.apache.org/content/repositories/snapshots/org/apache/ozone/hdds-interface-client/2.0.0-SNAPSHOT/maven-metadata.xml
   [WARNING] org.apache.ozone:hdds-interface-client:2.0.0-SNAPSHOT/maven-metadata.xmlfailed to transfer from https://repository.apache.org/content/repositories/snapshots during a previous attempt. This failure was cached in the local repository and resolution will not be reattempted until the update interval of apache.snapshots.https has elapsed or updates are forced. Original error: Could not transfer metadata org.apache.ozone:hdds-interface-client:2.0.0-SNAPSHOT/maven-metadata.xml from/to apache.snapshots.https (https://repository.apache.org/content/repositories/snapshots): transfer failed for https://repository.apache.org/content/repositories/snapshots/org/apache/ozone/hdds-interface-client/2.0.0-SNAPSHOT/maven-metadata.xml
   [WARNING] org.apache.ozone:hdds-interface-client:2.0.0-SNAPSHOT/maven-metadata.xmlfailed to transfer from https://repository.apache.org/content/repositories/snapshots during a previous attempt. This failure was cached in the local repository and resolution will not be reattempted until the update interval of apache.snapshots.https has elapsed or updates are forced. Original error: Could not transfer metadata org.apache.ozone:hdds-interface-client:2.0.0-SNAPSHOT/maven-metadata.xml from/to apache.snapshots.https (https://repository.apache.org/content/repositories/snapshots): transfer failed for https://repository.apache.org/content/repositories/snapshots/org/apache/ozone/hdds-interface-client/2.0.0-SNAPSHOT/maven-metadata.xml
   [WARNING] Could not transfer metadata org.apache.ozone:hdds-hadoop-dependency-test:2.0.0-SNAPSHOT/maven-metadata.xml from/to apache.snapshots.https (https://repository.apache.org/content/repositories/snapshots): transfer failed for https://repository.apache.org/content/repositories/snapshots/org/apache/ozone/hdds-hadoop-dependency-test/2.0.0-SNAPSHOT/maven-metadata.xml
   ...
   ```
   
   Ozone does not use snapshot dependencies.  The only snapshot artifacts are Ozone's own modules, which are created by the build, not available from from remote Maven repositories.
   
   Snapshot repo usage was inherited from Hadoop, which does (used to?) publish snapshot artifacts (MAPREDUCE-3003).
   
   So I propose disabling snapshots.
   
   https://issues.apache.org/jira/browse/HDDS-11872
   
   ## How was this patch tested?
   
   First removed all Ozone snapshot artifacts from local Maven repo:
   
   ```
   find ~/.m2/repository/org/apache/ozone -name '*SNAPSHOT*' | xargs rm -fr
   ```
   
   Tried to build some Ozone module without its dependencies.  Maven failed as expected, but did not mention remote repositories (`apache.snapshots...`), and did not save any information about update attempts to local repo:
   
   ```
   $ mvn -pl :hdds-common -DskipTests clean package
   ...
   [ERROR] Failed to execute goal on project hdds-common: Could not resolve dependencies for project org.apache.ozone:hdds-common:jar:2.0.0-SNAPSHOT: The following artifacts could not be resolved: org.apache.ozone:hdds-hadoop-dependency-client:jar:2.0.0-SNAPSHOT, org.apache.ozone:hdds-annotation-processing:jar:2.0.0-SNAPSHOT, org.apache.ozone:hdds-config:jar:2.0.0-SNAPSHOT, org.apache.ozone:hdds-interface-client:jar:2.0.0-SNAPSHOT, org.apache.ozone:hdds-interface-admin:jar:2.0.0-SNAPSHOT, org.apache.ozone:hdds-hadoop-dependency-test:jar:2.0.0-SNAPSHOT, org.apache.ozone:hdds-test-utils:jar:2.0.0-SNAPSHOT: Could not find artifact org.apache.ozone:hdds-hadoop-dependency-client:jar:2.0.0-SNAPSHOT
   ...
   [INFO] Total time:  0.714 s
   
   $ find ~/.m2/repository/org/apache/ozone -name '*SNAPSHOT*' | wc -l
   0
   
   $ mvn -pl :ozone-manager -DskipTests clean package
   ...
   [ERROR] Failed to execute goal on project ozone-manager: Could not resolve dependencies for project org.apache.ozone:ozone-manager:jar:2.0.0-SNAPSHOT: The following artifacts could not be resolved: org.apache.ozone:hdds-annotation-processing:jar:2.0.0-SNAPSHOT, org.apache.ozone:hdds-client:jar:2.0.0-SNAPSHOT, org.apache.ozone:hdds-common:jar:2.0.0-SNAPSHOT, org.apache.ozone:hdds-config:jar:2.0.0-SNAPSHOT, org.apache.ozone:hdds-interface-server:jar:2.0.0-SNAPSHOT, org.apache.ozone:hdds-managed-rocksdb:jar:2.0.0-SNAPSHOT, org.apache.ozone:ozone-interface-client:jar:2.0.0-SNAPSHOT, org.apache.ozone:ozone-common:jar:2.0.0-SNAPSHOT, org.apache.ozone:ozone-client:jar:2.0.0-SNAPSHOT, org.apache.ozone:hdds-docs:jar:2.0.0-SNAPSHOT, org.apache.ozone:hdds-interface-client:jar:2.0.0-SNAPSHOT, org.apache.ozone:ozone-interface-storage:jar:2.0.0-SNAPSHOT, org.apache.ozone:hdds-server-framework:jar:2.0.0-SNAPSHOT, org.apache.ozone:hdds-hadoop-dependency-server:jar:2.0.0-SNAPSHOT, org.apache.ozone
 :rocksdb-checkpoint-differ:jar:2.0.0-SNAPSHOT, org.apache.ozone:hdds-rocks-native:jar:2.0.0-SNAPSHOT, org.apache.ozone:hdds-common:jar:tests:2.0.0-SNAPSHOT, org.apache.ozone:hdds-hadoop-dependency-test:jar:2.0.0-SNAPSHOT, org.apache.ozone:hdds-server-scm:jar:tests:2.0.0-SNAPSHOT, org.apache.ozone:hdds-test-utils:jar:2.0.0-SNAPSHOT: Could not find artifact org.apache.ozone:hdds-annotation-processing:jar:2.0.0-SNAPSHOT
   ...
   [INFO] Total time:  1.834 s
   
   $ find ~/.m2/repository/org/apache/ozone -name '*SNAPSHOT*' | wc -l
   0
   ```
   
   For reference, failure in same case on `master` (with "normal" download speed):
   
   ```
   $ mvn -pl :ozone-manager -DskipTests clean package
   ...
   [ERROR] ... Could not find artifact org.apache.ozone:hdds-annotation-processing:jar:2.0.0-SNAPSHOT in apache.snapshots.https (https://repository.apache.org/content/repositories/snapshots)
   ...
   [INFO] Total time:  5.697 s
   
   $ find ~/.m2/repository/org/apache/ozone -name '*SNAPSHOT*' | wc -l
   58
   ```
   
   CI:
   https://github.com/adoroszlai/ozone/actions/runs/12202358564


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org


Re: [PR] HDDS-11872. Disable Apache snapshots repo [ozone]

Posted by "adoroszlai (via GitHub)" <gi...@apache.org>.
adoroszlai commented on PR #7536:
URL: https://github.com/apache/ozone/pull/7536#issuecomment-2528252075

   Tested with:
   
   ```
   mvn --batch-mode -Pdist -Drocks_tools_native eu.maveniverse.maven.plugins:toolbox:list-repositories
   ```
   
   On `master`:
   
   ```
   $ grep 'repository.apache.org.*snapshot' HDDS-11872-master.txt | sort -u        
   [INFO]  * apache.snapshots (http://repository.apache.org/snapshots, default, snapshots)
   [INFO]  * apache.snapshots.https (https://repository.apache.org/content/repositories/snapshots, default, releases+snapshots)
   [INFO]  * apache.snapshots (https://repository.apache.org/content/repositories/snapshots/, default, snapshots)
   [INFO]  * apache.snapshots (https://repository.apache.org/snapshots, default, snapshots)
   ```
   
   With the fix, all `repository.apache.org` snaphots are disabled:
   
   ```
   $ grep 'repository.apache.org.*snapshot' HDDS-11872-patch.txt | sort -u 
   [INFO]  * apache.snapshots.https (https://repository.apache.org/content/repositories/snapshots, default, disabled)
   [INFO]  * apache.snapshots (https://repository.apache.org/snapshots, default, disabled)
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org


Re: [PR] HDDS-11872. Disable Apache snapshots repo [ozone]

Posted by "adoroszlai (via GitHub)" <gi...@apache.org>.
adoroszlai merged PR #7536:
URL: https://github.com/apache/ozone/pull/7536


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org


Re: [PR] HDDS-11872. Disable Apache snapshots repo [ozone]

Posted by "adoroszlai (via GitHub)" <gi...@apache.org>.
adoroszlai commented on PR #7536:
URL: https://github.com/apache/ozone/pull/7536#issuecomment-2525863496

   Found that `cyclonedx-maven-plugin` is downloading from repository.apache.org even if our POM has not defined and/or disabled these repos.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org


Re: [PR] HDDS-11872. Disable Apache snapshots repo [ozone]

Posted by "adoroszlai (via GitHub)" <gi...@apache.org>.
adoroszlai commented on PR #7536:
URL: https://github.com/apache/ozone/pull/7536#issuecomment-2529139698

   Thanks @szetszwo for the review.  @swamirishi please let me know if you have any comments, I can address them in a follow-up.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org


Re: [PR] HDDS-11872. Disable Apache snapshots repo [ozone]

Posted by "adoroszlai (via GitHub)" <gi...@apache.org>.
adoroszlai closed pull request #7536: HDDS-11872. Disable Apache snapshots repo
URL: https://github.com/apache/ozone/pull/7536


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org