You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@maven.apache.org by "cstamas (via GitHub)" <gi...@apache.org> on 2023/02/15 16:01:09 UTC
[GitHub] [maven-resolver] cstamas opened a new pull request, #245: [MRESOLVER-319] Parallel deploy, pt2
cstamas opened a new pull request, #245:
URL: https://github.com/apache/maven-resolver/pull/245
In the mean time turns out we have two important "implementation details" to obey, and behave correctly:
* we need ONE transport to happen (go thru pipe fully) before we can pour the rest onto transport
The HTTP transporter is equipped with "shared" caches for auth that is heavily used by HttpClient to make decisions (ie. about re-attempting request). Current code is quite complicated, but auth sharing happens when one transfer task is done, and that step "inseminates" the cache to be used by others (and skip the auth dance from that point onwards). Similar step happens by transfer by issuing OPTIONS to remote to discover does it deals with WebDAV server or not (is MKCOL dance needed or not). All these states are once task is done shared and remembered for the lifetime of session (per remote repository). Hence, to obey these rules, simple fix is applied: FIRST transfer is sent thru pipe alone fully, and once is done, the REST is pushed in parallel, as at that point, all the auth caches and DAV state of remote repositry are already discovered by first one. This simple "trick" should not introduce big performance loss for parallel vs sequential deploy.
* we must ensure proper ordering, that was not ensured by origina implementation
Clients when going for metadata are getting them as G -> A -> V (the longest chain in case of snapshot maven plugin). Hence, we must ensure we deploy the opposite order, and we start deploying next group ONLY when we are fully done with previous group. Not ensuring this would leave a window of opportunity of failed build, where for example A level md is present, but V level not yet.
---
https://issues.apache.org/jira/browse/MRESOLVER-319
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@maven.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [maven-resolver] cstamas merged pull request #245: [MRESOLVER-319] Parallel deploy, pt2
Posted by "cstamas (via GitHub)" <gi...@apache.org>.
cstamas merged PR #245:
URL: https://github.com/apache/maven-resolver/pull/245
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@maven.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [maven-resolver] cstamas commented on pull request #245: [MRESOLVER-319] Parallel deploy, pt2
Posted by "cstamas (via GitHub)" <gi...@apache.org>.
cstamas commented on PR #245:
URL: https://github.com/apache/maven-resolver/pull/245#issuecomment-1432068984
Created https://issues.apache.org/jira/browse/MRESOLVER-323 for followup.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@maven.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [maven-resolver] cstamas commented on pull request #245: [MRESOLVER-319] Parallel deploy, pt2
Posted by "cstamas (via GitHub)" <gi...@apache.org>.
cstamas commented on PR #245:
URL: https://github.com/apache/maven-resolver/pull/245#issuecomment-1431993162
Yes, this is definitely some concurrency problem of transport-http, more precisely, initial population of auth cache and shared state,
Unsure would Wagon be better, as I see it keeps AuthCache per transport instance (per HttpWagon instance that is created per-lookup, not a singleton), while transport-http during lifetime of a session... (so it does Auth dance once per session, and caches auth, Wagon does auth dance per transport instance,, ie. per module deploy in multi module project).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@maven.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [maven-resolver] michael-o commented on pull request #245: [MRESOLVER-319] Parallel deploy, pt2
Posted by "michael-o (via GitHub)" <gi...@apache.org>.
michael-o commented on PR #245:
URL: https://github.com/apache/maven-resolver/pull/245#issuecomment-1431946119
W/o having looked deeper, this looks like a band-aid for a tranport problem and not a solution. I bet that Wagon transport does the right thing and native is just broken in this regard and needs fixing.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@maven.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org