You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@maven.apache.org by "Stepan Roh (JIRA)" <ji...@codehaus.org> on 2007/01/31 10:48:44 UTC
[jira] Created: (MNG-2802) Concurrent-safe access to local Maven
repository
Concurrent-safe access to local Maven repository
------------------------------------------------
Key: MNG-2802
URL: http://jira.codehaus.org/browse/MNG-2802
Project: Maven 2
Issue Type: Improvement
Components: Artifacts and Repositories
Reporter: Stepan Roh
It seems that access to local Maven repository is not concurrent-safe that is multiple Mavens running in parallel may damage contents of local Maven repository. It would be a nice improvement, because sharing of local repository will lower the need for contacting any other repository. I know that Maven proxy can be used, but that adds another layer which may unnecessarily stress the machine it runs on.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (MNG-2802) Concurrent-safe access to local Maven
repository
Posted by "John Lonergan (JIRA)" <ji...@codehaus.org>.
[ http://jira.codehaus.org/browse/MNG-2802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=238175#action_238175 ]
John Lonergan commented on MNG-2802:
------------------------------------
Or possibly someone could advise where the piece of code that maintains the local repo lies and I'll have look.
Things that hurt seem mostly to be that the repo metadata gets corrupted.
> Concurrent-safe access to local Maven repository
> ------------------------------------------------
>
> Key: MNG-2802
> URL: http://jira.codehaus.org/browse/MNG-2802
> Project: Maven 2 & 3
> Issue Type: Improvement
> Components: Artifacts and Repositories
> Reporter: Stepan Roh
> Assignee: John Casey
> Fix For: Issues to be reviewed for 3.x
>
>
> It seems that access to local Maven repository is not concurrent-safe that is multiple Mavens running in parallel may damage contents of local Maven repository. It would be a nice improvement, because sharing of local repository will lower the need for contacting any other repository. I know that Maven proxy can be used, but that adds another layer which may unnecessarily stress the machine it runs on.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] (MNG-2802) Concurrent-safe access to local Maven repository
Posted by "Jason van Zyl (JIRA)" <ji...@codehaus.org>.
[ https://jira.codehaus.org/browse/MNG-2802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=315341#comment-315341 ]
Jason van Zyl commented on MNG-2802:
------------------------------------
You can drop this implementation into your ${M2_HOME}/lib/ext:
http://search.maven.org/#artifactdetails%7Cio.tesla%7Ctesla-concurrent-localrepo%7C0.0.2%7Cjar
I will be integrating this with the Aether version once Maven core is updated to Aether 0.9.0.M1
> Concurrent-safe access to local Maven repository
> ------------------------------------------------
>
> Key: MNG-2802
> URL: https://jira.codehaus.org/browse/MNG-2802
> Project: Maven 2 & 3
> Issue Type: Improvement
> Components: Artifacts and Repositories
> Reporter: Stepan Roh
> Assignee: John Casey
> Fix For: Issues to be reviewed for 3.x
>
>
> It seems that access to local Maven repository is not concurrent-safe that is multiple Mavens running in parallel may damage contents of local Maven repository. It would be a nice improvement, because sharing of local repository will lower the need for contacting any other repository. I know that Maven proxy can be used, but that adds another layer which may unnecessarily stress the machine it runs on.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://jira.codehaus.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (MNG-2802) Concurrent-safe access to local Maven
repository
Posted by "Brett Porter (JIRA)" <ji...@codehaus.org>.
[ http://jira.codehaus.org/browse/MNG-2802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=147569#action_147569 ]
Brett Porter commented on MNG-2802:
-----------------------------------
This is also discussed in http://docs.codehaus.org/display/MAVEN/Local+repository+separation, along with MNG-3655
> Concurrent-safe access to local Maven repository
> ------------------------------------------------
>
> Key: MNG-2802
> URL: http://jira.codehaus.org/browse/MNG-2802
> Project: Maven 2
> Issue Type: Improvement
> Components: Artifacts and Repositories
> Reporter: Stepan Roh
> Assignee: John Casey
> Fix For: 3.0
>
>
> It seems that access to local Maven repository is not concurrent-safe that is multiple Mavens running in parallel may damage contents of local Maven repository. It would be a nice improvement, because sharing of local repository will lower the need for contacting any other repository. I know that Maven proxy can be used, but that adds another layer which may unnecessarily stress the machine it runs on.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] (MNG-2802) Concurrent-safe access to local Maven repository
Posted by "luke call (JIRA)" <ji...@codehaus.org>.
[ https://jira.codehaus.org/browse/MNG-2802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=315768#comment-315768 ]
luke call commented on MNG-2802:
--------------------------------
Thanks Jason!!
> Concurrent-safe access to local Maven repository
> ------------------------------------------------
>
> Key: MNG-2802
> URL: https://jira.codehaus.org/browse/MNG-2802
> Project: Maven 2 & 3
> Issue Type: Improvement
> Components: Artifacts and Repositories
> Reporter: Stepan Roh
> Assignee: John Casey
> Fix For: Issues to be reviewed for 3.x
>
>
> It seems that access to local Maven repository is not concurrent-safe that is multiple Mavens running in parallel may damage contents of local Maven repository. It would be a nice improvement, because sharing of local repository will lower the need for contacting any other repository. I know that Maven proxy can be used, but that adds another layer which may unnecessarily stress the machine it runs on.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://jira.codehaus.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (MNG-2802) Concurrent-safe access to local Maven
repository
Posted by "Paul Benedict (JIRA)" <ji...@codehaus.org>.
[ http://jira.codehaus.org/browse/MNG-2802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=215219#action_215219 ]
Paul Benedict commented on MNG-2802:
------------------------------------
Giving Maven an exclusive lock on an artifact's local metadata should be a sure way of ensuring this.
> Concurrent-safe access to local Maven repository
> ------------------------------------------------
>
> Key: MNG-2802
> URL: http://jira.codehaus.org/browse/MNG-2802
> Project: Maven 2 & 3
> Issue Type: Improvement
> Components: Artifacts and Repositories
> Reporter: Stepan Roh
> Assignee: John Casey
> Fix For: 3.x (to be reviewed)
>
>
> It seems that access to local Maven repository is not concurrent-safe that is multiple Mavens running in parallel may damage contents of local Maven repository. It would be a nice improvement, because sharing of local repository will lower the need for contacting any other repository. I know that Maven proxy can be used, but that adds another layer which may unnecessarily stress the machine it runs on.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] (MNG-2802) Concurrent-safe access to local Maven repository
Posted by "Eric Pabst (JIRA)" <ji...@codehaus.org>.
[ https://jira.codehaus.org/browse/MNG-2802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=315337#comment-315337 ]
Eric Pabst commented on MNG-2802:
---------------------------------
It's very frustrating that there is no apparent progress on this. It was reported years ago and is still hurting productivity significantly.
> Concurrent-safe access to local Maven repository
> ------------------------------------------------
>
> Key: MNG-2802
> URL: https://jira.codehaus.org/browse/MNG-2802
> Project: Maven 2 & 3
> Issue Type: Improvement
> Components: Artifacts and Repositories
> Reporter: Stepan Roh
> Assignee: John Casey
> Fix For: Issues to be reviewed for 3.x
>
>
> It seems that access to local Maven repository is not concurrent-safe that is multiple Mavens running in parallel may damage contents of local Maven repository. It would be a nice improvement, because sharing of local repository will lower the need for contacting any other repository. I know that Maven proxy can be used, but that adds another layer which may unnecessarily stress the machine it runs on.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://jira.codehaus.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (MNG-2802) Concurrent-safe access to local Maven
repository
Posted by "Christian Gruber (JIRA)" <ji...@codehaus.org>.
[ http://jira.codehaus.org/browse/MNG-2802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=146184#action_146184 ]
Christian Gruber commented on MNG-2802:
---------------------------------------
Frankly, one could mitigate this substantially by downloading artifacts into temporary files, then once the artifact is down with all its pieces, moving the temporary files into the permanent location. There is still a race condition but it's down to the order of magnitude of milliseconds during the move operation, rather than the many seconds during downloading.
> Concurrent-safe access to local Maven repository
> ------------------------------------------------
>
> Key: MNG-2802
> URL: http://jira.codehaus.org/browse/MNG-2802
> Project: Maven 2
> Issue Type: Improvement
> Components: Artifacts and Repositories
> Reporter: Stepan Roh
> Assignee: John Casey
> Fix For: 3.0
>
>
> It seems that access to local Maven repository is not concurrent-safe that is multiple Mavens running in parallel may damage contents of local Maven repository. It would be a nice improvement, because sharing of local repository will lower the need for contacting any other repository. I know that Maven proxy can be used, but that adds another layer which may unnecessarily stress the machine it runs on.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (MNG-2802) Concurrent-safe access to local Maven
repository
Posted by "Brian Fox (JIRA)" <ji...@codehaus.org>.
[ http://jira.codehaus.org/browse/MNG-2802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Brian Fox updated MNG-2802:
---------------------------
Assignee: John Casey
Fix Version/s: (was: Reviewed Pending Version Assignment)
2.1
John, you were recently fixing this in 2.1 i believe
> Concurrent-safe access to local Maven repository
> ------------------------------------------------
>
> Key: MNG-2802
> URL: http://jira.codehaus.org/browse/MNG-2802
> Project: Maven 2
> Issue Type: Improvement
> Components: Artifacts and Repositories
> Reporter: Stepan Roh
> Assignee: John Casey
> Fix For: 2.1
>
>
> It seems that access to local Maven repository is not concurrent-safe that is multiple Mavens running in parallel may damage contents of local Maven repository. It would be a nice improvement, because sharing of local repository will lower the need for contacting any other repository. I know that Maven proxy can be used, but that adds another layer which may unnecessarily stress the machine it runs on.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (MNG-2802) Concurrent-safe access to local Maven
repository
Posted by "cforce (JIRA)" <ji...@codehaus.org>.
[ https://jira.codehaus.org/browse/MNG-2802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=278089#comment-278089 ]
cforce commented on MNG-2802:
-----------------------------
This issue hits us most on a Jenkins buidl server, where all Maven builds use same repo folder. Having many dupes of local repos will use a lot of local hard disk space. Maybe there is a Maven Jenkins Plugin for Jenkins which coordinates the Concurrent access?
> Concurrent-safe access to local Maven repository
> ------------------------------------------------
>
> Key: MNG-2802
> URL: https://jira.codehaus.org/browse/MNG-2802
> Project: Maven 2 & 3
> Issue Type: Improvement
> Components: Artifacts and Repositories
> Reporter: Stepan Roh
> Assignee: John Casey
> Fix For: Issues to be reviewed for 3.x
>
>
> It seems that access to local Maven repository is not concurrent-safe that is multiple Mavens running in parallel may damage contents of local Maven repository. It would be a nice improvement, because sharing of local repository will lower the need for contacting any other repository. I know that Maven proxy can be used, but that adds another layer which may unnecessarily stress the machine it runs on.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (MNG-2802) Concurrent-safe access to local Maven
repository
Posted by "John Lonergan (JIRA)" <ji...@codehaus.org>.
[ http://jira.codehaus.org/browse/MNG-2802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=238118#action_238118 ]
John Lonergan commented on MNG-2802:
------------------------------------
This issue really impacts our ability to run concurrent continuous integration builds making out CI take several times longer to cycle than we'd like. We have compensated by having each CI task have it's own private local repo but this massively expands the disk requirements and has the affect of slowing everything down due to all the duplicate file downloading going on. But the really painful thing is that when we have separate local repo's the various threads cannot share the snapshots via the local repo.
Please try to get a fix in place for this.
> Concurrent-safe access to local Maven repository
> ------------------------------------------------
>
> Key: MNG-2802
> URL: http://jira.codehaus.org/browse/MNG-2802
> Project: Maven 2 & 3
> Issue Type: Improvement
> Components: Artifacts and Repositories
> Reporter: Stepan Roh
> Assignee: John Casey
> Fix For: Issues to be reviewed for 3.x
>
>
> It seems that access to local Maven repository is not concurrent-safe that is multiple Mavens running in parallel may damage contents of local Maven repository. It would be a nice improvement, because sharing of local repository will lower the need for contacting any other repository. I know that Maven proxy can be used, but that adds another layer which may unnecessarily stress the machine it runs on.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (MNG-2802) Concurrent-safe access to local Maven
repository
Posted by "John Casey (JIRA)" <ji...@codehaus.org>.
[ http://jira.codehaus.org/browse/MNG-2802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_126325 ]
John Casey commented on MNG-2802:
---------------------------------
To go any further than just ensuring the locking of resolver-status files (which are a sort of proto-metadata for the resolver to use that helps track unfound artifacts and adhere to an updateInterval for those), we'll need to modify the various Wagons, I think. Since these are file-oriented, they manage opening, writing to, and closing the files internally. One thing I might be able to do is try to manage a file lock on the main artifact file instead of the temp file that the wagon is using as its destination (which may not exist, so that could complicate things even further). Yet another approach might involve creating a lockfile, and managing a lock on that...
I'll look into a couple of these strategies. I don't think it's optimal to bury the locking logic in the various wagons, as we're likely to run into concurrency problems again and again with each marginal wagon implementation.
> Concurrent-safe access to local Maven repository
> ------------------------------------------------
>
> Key: MNG-2802
> URL: http://jira.codehaus.org/browse/MNG-2802
> Project: Maven 2
> Issue Type: Improvement
> Components: Artifacts and Repositories
> Reporter: Stepan Roh
> Assignee: John Casey
> Fix For: 2.1
>
>
> It seems that access to local Maven repository is not concurrent-safe that is multiple Mavens running in parallel may damage contents of local Maven repository. It would be a nice improvement, because sharing of local repository will lower the need for contacting any other repository. I know that Maven proxy can be used, but that adds another layer which may unnecessarily stress the machine it runs on.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira