You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@maven.apache.org by "cstamas (via GitHub)" <gi...@apache.org> on 2023/02/07 11:16:14 UTC
[GitHub] [maven-resolver] cstamas opened a new pull request, #240: [MRESOLVER-319] Parallel Deploy Support
cstamas opened a new pull request, #240:
URL: https://github.com/apache/maven-resolver/pull/240
Make basic connector be able to perform parallel PUTs in very same manner as it did so far parallel GETs.
Still, there is a "safety switch" to turn off this new feature that is ENABLED by default.
Effects:
* set threads to 1: GET and PUT are executed sequentially
* set parallelPut to false: Behave as Maven 3.8.x line was
* default: perform parallel GETs and PUTs
---
https://issues.apache.org/jira/browse/MRESOLVER-319
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@maven.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [maven-resolver] michael-o commented on a diff in pull request #240: [MRESOLVER-319] Parallel Deploy Support
Posted by "michael-o (via GitHub)" <gi...@apache.org>.
michael-o commented on code in PR #240:
URL: https://github.com/apache/maven-resolver/pull/240#discussion_r1098528694
##########
maven-resolver-connector-basic/src/main/java/org/eclipse/aether/connector/basic/BasicRepositoryConnector.java:
##########
@@ -298,9 +306,11 @@ public void put( Collection<? extends ArtifactUpload> artifactUploads,
Runnable task = new PutTaskRunner( location, transfer.getFile(), transfer.getFileTransformer(),
checksumLocations, listener );
- task.run();
+ executor.execute( errorForwarder.wrap( task ) );
}
+ errorForwarder.await(); // make sure all artifacts are PUT before we go with Metadata
+
Review Comment:
The forwarder will throw an exception if something goes wrong?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@maven.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [maven-resolver] cstamas commented on a diff in pull request #240: [MRESOLVER-319] Parallel Deploy Support
Posted by "cstamas (via GitHub)" <gi...@apache.org>.
cstamas commented on code in PR #240:
URL: https://github.com/apache/maven-resolver/pull/240#discussion_r1098533115
##########
maven-resolver-connector-basic/src/main/java/org/eclipse/aether/connector/basic/BasicRepositoryConnector.java:
##########
@@ -284,6 +289,9 @@ public void put( Collection<? extends ArtifactUpload> artifactUploads,
Collection<? extends ArtifactUpload> safeArtifactUploads = safe( artifactUploads );
Collection<? extends MetadataUpload> safeMetadataUploads = safe( metadataUploads );
+ Executor executor = getExecutor( parallelPut ? safeArtifactUploads.size() + safeMetadataUploads.size() : 1 );
Review Comment:
It is literally same as this line:
https://github.com/apache/maven-resolver/pull/240/files#diff-ac0d3c1eeeae1e33fd8674465b3cd81dee6963dd017c290a1ad21173fd7571dcR216
but with option to fall back to 1, in which Direct executor is used (sequential)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@maven.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [maven-resolver] michael-o commented on a diff in pull request #240: [MRESOLVER-319] Parallel Deploy Support
Posted by "michael-o (via GitHub)" <gi...@apache.org>.
michael-o commented on code in PR #240:
URL: https://github.com/apache/maven-resolver/pull/240#discussion_r1098593179
##########
maven-resolver-connector-basic/src/main/java/org/eclipse/aether/connector/basic/BasicRepositoryConnector.java:
##########
@@ -143,6 +147,8 @@
maxThreads = ExecutorUtils.threadCount( session, 5, CONFIG_PROP_THREADS, "maven.artifact.threads" );
smartChecksums = ConfigUtils.getBoolean( session, true, CONFIG_PROP_SMART_CHECKSUMS );
+ parallelPut = ConfigUtils.getBoolean( session, true,
+ CONFIG_PROP_PARALLEL_PUT + "." + repository.getId(), CONFIG_PROP_PARALLEL_PUT );
Review Comment:
This should be global and repo id suffix specific, like other props. WDYT?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@maven.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [maven-resolver] michael-o commented on a diff in pull request #240: [MRESOLVER-319] Parallel Deploy Support
Posted by "michael-o (via GitHub)" <gi...@apache.org>.
michael-o commented on code in PR #240:
URL: https://github.com/apache/maven-resolver/pull/240#discussion_r1098593656
##########
maven-resolver-connector-basic/src/main/java/org/eclipse/aether/connector/basic/BasicRepositoryConnector.java:
##########
@@ -284,6 +289,9 @@ public void put( Collection<? extends ArtifactUpload> artifactUploads,
Collection<? extends ArtifactUpload> safeArtifactUploads = safe( artifactUploads );
Collection<? extends MetadataUpload> safeMetadataUploads = safe( metadataUploads );
+ Executor executor = getExecutor( parallelPut ? safeArtifactUploads.size() + safeMetadataUploads.size() : 1 );
Review Comment:
Is the lnk really correct?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@maven.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [maven-resolver] cstamas commented on a diff in pull request #240: [MRESOLVER-319] Parallel Deploy Support
Posted by "cstamas (via GitHub)" <gi...@apache.org>.
cstamas commented on code in PR #240:
URL: https://github.com/apache/maven-resolver/pull/240#discussion_r1098533680
##########
maven-resolver-connector-basic/src/main/java/org/eclipse/aether/connector/basic/BasicRepositoryConnector.java:
##########
@@ -298,9 +306,11 @@ public void put( Collection<? extends ArtifactUpload> artifactUploads,
Runnable task = new PutTaskRunner( location, transfer.getFile(), transfer.getFileTransformer(),
checksumLocations, listener );
- task.run();
+ executor.execute( errorForwarder.wrap( task ) );
}
+ errorForwarder.await(); // make sure all artifacts are PUT before we go with Metadata
+
Review Comment:
yes
##########
maven-resolver-connector-basic/src/main/java/org/eclipse/aether/connector/basic/BasicRepositoryConnector.java:
##########
@@ -298,9 +306,11 @@ public void put( Collection<? extends ArtifactUpload> artifactUploads,
Runnable task = new PutTaskRunner( location, transfer.getFile(), transfer.getFileTransformer(),
checksumLocations, listener );
- task.run();
+ executor.execute( errorForwarder.wrap( task ) );
}
+ errorForwarder.await(); // make sure all artifacts are PUT before we go with Metadata
+
Review Comment:
yes, hence the name
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@maven.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [maven-resolver] michael-o commented on a diff in pull request #240: [MRESOLVER-319] Parallel Deploy Support
Posted by "michael-o (via GitHub)" <gi...@apache.org>.
michael-o commented on code in PR #240:
URL: https://github.com/apache/maven-resolver/pull/240#discussion_r1098527926
##########
maven-resolver-connector-basic/src/main/java/org/eclipse/aether/connector/basic/BasicRepositoryConnector.java:
##########
@@ -284,6 +289,9 @@ public void put( Collection<? extends ArtifactUpload> artifactUploads,
Collection<? extends ArtifactUpload> safeArtifactUploads = safe( artifactUploads );
Collection<? extends MetadataUpload> safeMetadataUploads = safe( metadataUploads );
+ Executor executor = getExecutor( parallelPut ? safeArtifactUploads.size() + safeMetadataUploads.size() : 1 );
Review Comment:
Shouldn't this ues the same `maxThreads` as download?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@maven.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [maven-resolver] cstamas commented on a diff in pull request #240: [MRESOLVER-319] Parallel Deploy Support
Posted by "cstamas (via GitHub)" <gi...@apache.org>.
cstamas commented on code in PR #240:
URL: https://github.com/apache/maven-resolver/pull/240#discussion_r1098613956
##########
maven-resolver-connector-basic/src/main/java/org/eclipse/aether/connector/basic/BasicRepositoryConnector.java:
##########
@@ -284,6 +289,9 @@ public void put( Collection<? extends ArtifactUpload> artifactUploads,
Collection<? extends ArtifactUpload> safeArtifactUploads = safe( artifactUploads );
Collection<? extends MetadataUpload> safeMetadataUploads = safe( metadataUploads );
+ Executor executor = getExecutor( parallelPut ? safeArtifactUploads.size() + safeMetadataUploads.size() : 1 );
Review Comment:
Yes, R216, but seems folding hides it
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@maven.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [maven-resolver] cstamas merged pull request #240: [MRESOLVER-319] Parallel Deploy Support
Posted by "cstamas (via GitHub)" <gi...@apache.org>.
cstamas merged PR #240:
URL: https://github.com/apache/maven-resolver/pull/240
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@maven.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [maven-resolver] cstamas commented on a diff in pull request #240: [MRESOLVER-319] Parallel Deploy Support
Posted by "cstamas (via GitHub)" <gi...@apache.org>.
cstamas commented on code in PR #240:
URL: https://github.com/apache/maven-resolver/pull/240#discussion_r1098597888
##########
maven-resolver-connector-basic/src/main/java/org/eclipse/aether/connector/basic/BasicRepositoryConnector.java:
##########
@@ -143,6 +147,8 @@
maxThreads = ExecutorUtils.threadCount( session, 5, CONFIG_PROP_THREADS, "maven.artifact.threads" );
smartChecksums = ConfigUtils.getBoolean( session, true, CONFIG_PROP_SMART_CHECKSUMS );
+ parallelPut = ConfigUtils.getBoolean( session, true,
+ CONFIG_PROP_PARALLEL_PUT + "." + repository.getId(), CONFIG_PROP_PARALLEL_PUT );
Review Comment:
This IS repo id suffix specific or global ("closest wins"), the very same as for every other props (that are marked with "yes" in last column)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@maven.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [maven-resolver] michael-o commented on a diff in pull request #240: [MRESOLVER-319] Parallel Deploy Support
Posted by "michael-o (via GitHub)" <gi...@apache.org>.
michael-o commented on code in PR #240:
URL: https://github.com/apache/maven-resolver/pull/240#discussion_r1098638046
##########
maven-resolver-connector-basic/src/main/java/org/eclipse/aether/connector/basic/BasicRepositoryConnector.java:
##########
@@ -284,6 +289,9 @@ public void put( Collection<? extends ArtifactUpload> artifactUploads,
Collection<? extends ArtifactUpload> safeArtifactUploads = safe( artifactUploads );
Collection<? extends MetadataUpload> safeMetadataUploads = safe( metadataUploads );
+ Executor executor = getExecutor( parallelPut ? safeArtifactUploads.size() + safeMetadataUploads.size() : 1 );
Review Comment:
You are right.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@maven.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org