You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@maven.apache.org by "slawekjaranowski (via GitHub)" <gi...@apache.org> on 2023/01/26 23:06:37 UTC
[GitHub] [maven-deploy-plugin] slawekjaranowski opened a new pull request, #35: [MDEPLOY-206] Support parallel deployment for deployAtEnd
slawekjaranowski opened a new pull request, #35:
URL: https://github.com/apache/maven-deploy-plugin/pull/35
Each whole module will be deployed by resolver in one thread.
We have parallel deployment on module level.
---
Following this checklist to help us incorporate your
contribution quickly and easily:
- [x] Make sure there is a [JIRA issue](https://issues.apache.org/jira/browse/MDEPLOY) filed
for the change (usually before you start working on it). Trivial changes like typos do not
require a JIRA issue. Your pull request should address just this issue, without
pulling in other changes.
- [x] Each commit in the pull request should have a meaningful subject line and body.
- [x] Format the pull request title like `[MDEPLOY-XXX] - Fixes bug in ApproximateQuantiles`,
where you replace `MDEPLOY-XXX` with the appropriate JIRA issue. Best practice
is to use the JIRA issue title in the pull request title and in the first line of the
commit message.
- [x] Write a pull request description that is detailed enough to understand what the pull request does, how, and why.
- [x] Run `mvn clean verify` to make sure basic checks pass. A more thorough check will
be performed on your pull request automatically.
- [x] You have run the integration tests successfully (`mvn -Prun-its clean verify`).
If your pull request is about ~20 lines of code you don't need to sign an
[Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf) if you are unsure
please ask on the developers list.
To make clear that you license your contribution under
the [Apache License Version 2.0, January 2004](http://www.apache.org/licenses/LICENSE-2.0)
you have to acknowledge this by using the following check-box.
- [x] I hereby declare this contribution to be licenced under the [Apache License Version 2.0, January 2004](http://www.apache.org/licenses/LICENSE-2.0)
- [x] In any other case, please file an [Apache Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@maven.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [maven-deploy-plugin] cstamas commented on pull request #35: [MDEPLOY-206] Support parallel deployment for deployAtEnd
Posted by "cstamas (via GitHub)" <gi...@apache.org>.
cstamas commented on PR #35:
URL: https://github.com/apache/maven-deploy-plugin/pull/35#issuecomment-1410838691
> > @gnodet Is my answer sufficient for you?
>
> Yes, though, due to the locking that happens inside the resolver, I wonder if it would make more sense to have this parallel upload done inside the resolver rather than in the deploy plugin. @cstamas ?
See https://issues.apache.org/jira/browse/MRESOLVER-32?focusedCommentId=17607557&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17607557
And I agree, this should be better solved in resolver (would be nicer at least). IMO this plugin should simply create 1 deploy request and lump it onto resolver to "solve it".
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@maven.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [maven-deploy-plugin] slawekjaranowski merged pull request #35: [MDEPLOY-206] Support parallel deployment for deployAtEnd
Posted by "slawekjaranowski (via GitHub)" <gi...@apache.org>.
slawekjaranowski merged PR #35:
URL: https://github.com/apache/maven-deploy-plugin/pull/35
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@maven.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [maven-deploy-plugin] gnodet commented on pull request #35: [MDEPLOY-206] Support parallel deployment for deployAtEnd
Posted by "gnodet (via GitHub)" <gi...@apache.org>.
gnodet commented on PR #35:
URL: https://github.com/apache/maven-deploy-plugin/pull/35#issuecomment-1407407360
> @gnodet Is my answer sufficient for you?
Yes, though, due to the locking that happens inside the resolver, I wonder if it would make more sense to have this parallel upload done inside the resolver rather than in the deploy plugin. @cstamas ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@maven.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [maven-deploy-plugin] slawekjaranowski commented on pull request #35: [MDEPLOY-206] Support parallel deployment for deployAtEnd
Posted by "slawekjaranowski (via GitHub)" <gi...@apache.org>.
slawekjaranowski commented on PR #35:
URL: https://github.com/apache/maven-deploy-plugin/pull/35#issuecomment-1407421512
> Yes, though, due to the locking that happens inside the resolver, I wonder if it would make more sense to have this parallel upload done inside the resolver rather than in the deploy plugin.
That is the reason that I execute deploy for whole module in separate thread - we have the same situation like executing Maven with `-T`, each module will be deployed in own thread.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@maven.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [maven-deploy-plugin] slawekjaranowski commented on pull request #35: [MDEPLOY-206] Support parallel deployment for deployAtEnd
Posted by "slawekjaranowski (via GitHub)" <gi...@apache.org>.
slawekjaranowski commented on PR #35:
URL: https://github.com/apache/maven-deploy-plugin/pull/35#issuecomment-1407126412
There are two code paths:
- number of threads <= 1 - collect all artifacts from all module and call in **one** request for Resolver
- number of threads > 1 - create thread pool and create **separate** deploy request per module
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@maven.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [maven-deploy-plugin] slawekjaranowski commented on a diff in pull request #35: [MDEPLOY-206] Support parallel deployment for deployAtEnd
Posted by "slawekjaranowski (via GitHub)" <gi...@apache.org>.
slawekjaranowski commented on code in PR #35:
URL: https://github.com/apache/maven-deploy-plugin/pull/35#discussion_r1092302289
##########
src/main/java/org/apache/maven/plugins/deploy/DeployMojo.java:
##########
@@ -71,6 +75,14 @@ public class DeployMojo extends AbstractDeployMojo {
@Parameter(defaultValue = "false", property = "deployAtEnd")
private boolean deployAtEnd;
+ /**
+ * The number of parallel threads which will be used for deployment with the {@code deployAtEnd} feature.
+ *
+ * @since 3.1.0
+ */
+ @Parameter(property = "maven.deploy.parallelThreads", defaultValue = "1")
Review Comment:
Can be ... 😄
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@maven.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [maven-deploy-plugin] gnodet commented on pull request #35: [MDEPLOY-206] Support parallel deployment for deployAtEnd
Posted by "gnodet (via GitHub)" <gi...@apache.org>.
gnodet commented on PR #35:
URL: https://github.com/apache/maven-deploy-plugin/pull/35#issuecomment-1407031604
> @slawekjaranowski the code for multi-threaded deploy should work if the number of threads is set to `1`. Should we just use a single code path for both ?
In case we really want to avoid creating a thread pool with 1 thread, we could use https://guava.dev/releases/19.0/api/docs/com/google/common/util/concurrent/MoreExecutors.html#newDirectExecutorService() ...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@maven.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [maven-deploy-plugin] gnodet commented on pull request #35: [MDEPLOY-206] Support parallel deployment for deployAtEnd
Posted by "gnodet (via GitHub)" <gi...@apache.org>.
gnodet commented on PR #35:
URL: https://github.com/apache/maven-deploy-plugin/pull/35#issuecomment-1407025183
@slawekjaranowski the code for multi-threaded deploy should work if the number of threads is set to `1`. Should we just use a single code path for both ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@maven.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [maven-deploy-plugin] slawekjaranowski commented on pull request #35: [MDEPLOY-206] Support parallel deployment for deployAtEnd
Posted by "slawekjaranowski (via GitHub)" <gi...@apache.org>.
slawekjaranowski commented on PR #35:
URL: https://github.com/apache/maven-deploy-plugin/pull/35#issuecomment-1407406737
@gnodet Is my answer sufficient for you?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@maven.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [maven-deploy-plugin] mattnelson commented on a diff in pull request #35: [MDEPLOY-206] Support parallel deployment for deployAtEnd
Posted by "mattnelson (via GitHub)" <gi...@apache.org>.
mattnelson commented on code in PR #35:
URL: https://github.com/apache/maven-deploy-plugin/pull/35#discussion_r1092292051
##########
src/main/java/org/apache/maven/plugins/deploy/DeployMojo.java:
##########
@@ -71,6 +75,14 @@ public class DeployMojo extends AbstractDeployMojo {
@Parameter(defaultValue = "false", property = "deployAtEnd")
private boolean deployAtEnd;
+ /**
+ * The number of parallel threads which will be used for deployment with the {@code deployAtEnd} feature.
+ *
+ * @since 3.1.0
+ */
+ @Parameter(property = "maven.deploy.parallelThreads", defaultValue = "1")
Review Comment:
sorry for the late review.
Would it be possible to support dynamic syntax like is done with the CLI parallelism flag `-T 1C` that way the deploy threads could scale with the hardware without being tightly coupled to config.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@maven.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org