You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@archiva.apache.org by "Mike R. Haller (JIRA)" <ji...@codehaus.org> on 2011/01/17 09:36:58 UTC
[jira] Created: (MRM-1449) Remote connections for many repos
exhausts proxy limits
Remote connections for many repos exhausts proxy limits
-------------------------------------------------------
Key: MRM-1449
URL: http://jira.codehaus.org/browse/MRM-1449
Project: Archiva
Issue Type: New Feature
Components: remote proxy
Reporter: Mike R. Haller
Our Archiva installation uses a company-internal caching proxy (ISA Server) to connect to remote repositories.
When there are many remote repositories and many developers trying to look up artifacts (existing and non-existing artifacts, e.g. often -sources and -javadoc attachments), Archiva is creating many HTTP connections to the remote repositories.
This leads to a situation where the caching proxy thinks Archiva is creating too many connections. The ISA warning mail even suggests the host computer may be infected with a worm because it creates so many new connections and blocks the host completely for all outgoing HTTP requests.
The policies for the remote repositories are configured for retrieving "once", "never" or "daily", depending on whether it's releases or snapshots. Caching failures is disabled and i'm trying with enabled failure caching, but it doesn't make much difference and the problem still occurs once in a while.
I think Archiva should have a configurable way to limit the number of (new) connections made per time unit, e.g. "max 60 connections / minute" to prevent this. It's kind of a potential denial of service vulnerability.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (MRM-1449) Remote connections for many repos
exhausts proxy limits
Posted by "Brett Porter (JIRA)" <ji...@codehaus.org>.
[ http://jira.codehaus.org/browse/MRM-1449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=252303#action_252303 ]
Brett Porter commented on MRM-1449:
-----------------------------------
interesting feature. We might consider having this both on incoming connections and outgoing proxy connections.
Would you be interested in helping with a patch?
> Remote connections for many repos exhausts proxy limits
> -------------------------------------------------------
>
> Key: MRM-1449
> URL: http://jira.codehaus.org/browse/MRM-1449
> Project: Archiva
> Issue Type: New Feature
> Components: remote proxy
> Reporter: Mike R. Haller
>
> Our Archiva installation uses a company-internal caching proxy (ISA Server) to connect to remote repositories.
> When there are many remote repositories and many developers trying to look up artifacts (existing and non-existing artifacts, e.g. often -sources and -javadoc attachments), Archiva is creating many HTTP connections to the remote repositories.
> This leads to a situation where the caching proxy thinks Archiva is creating too many connections. The ISA warning mail even suggests the host computer may be infected with a worm because it creates so many new connections and blocks the host completely for all outgoing HTTP requests.
> The policies for the remote repositories are configured for retrieving "once", "never" or "daily", depending on whether it's releases or snapshots. Caching failures is disabled and i'm trying with enabled failure caching, but it doesn't make much difference and the problem still occurs once in a while.
> I think Archiva should have a configurable way to limit the number of (new) connections made per time unit, e.g. "max 60 connections / minute" to prevent this. It's kind of a potential denial of service vulnerability.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Issue Comment Edited: (MRM-1449) Remote connections for many
repos exhausts proxy limits
Posted by "Jan-Sebastian Winckelmann (JIRA)" <ji...@codehaus.org>.
[ https://jira.codehaus.org/browse/MRM-1449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=272675#comment-272675 ]
Jan-Sebastian Winckelmann edited comment on MRM-1449 at 7/8/11 9:10 AM:
------------------------------------------------------------------------
We are hit by the same problem.
was (Author: jwinckelmann):
We are hit by the some problem.
> Remote connections for many repos exhausts proxy limits
> -------------------------------------------------------
>
> Key: MRM-1449
> URL: https://jira.codehaus.org/browse/MRM-1449
> Project: Archiva
> Issue Type: New Feature
> Components: remote proxy
> Reporter: Mike R. Haller
> Fix For: Backlog
>
>
> Our Archiva installation uses a company-internal caching proxy (ISA Server) to connect to remote repositories.
> When there are many remote repositories and many developers trying to look up artifacts (existing and non-existing artifacts, e.g. often -sources and -javadoc attachments), Archiva is creating many HTTP connections to the remote repositories.
> This leads to a situation where the caching proxy thinks Archiva is creating too many connections. The ISA warning mail even suggests the host computer may be infected with a worm because it creates so many new connections and blocks the host completely for all outgoing HTTP requests.
> The policies for the remote repositories are configured for retrieving "once", "never" or "daily", depending on whether it's releases or snapshots. Caching failures is disabled and i'm trying with enabled failure caching, but it doesn't make much difference and the problem still occurs once in a while.
> I think Archiva should have a configurable way to limit the number of (new) connections made per time unit, e.g. "max 60 connections / minute" to prevent this. It's kind of a potential denial of service vulnerability.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (MRM-1449) Remote connections for many repos
exhausts proxy limits
Posted by "Mike R. Haller (JIRA)" <ji...@codehaus.org>.
[ http://jira.codehaus.org/browse/MRM-1449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=252425#action_252425 ]
Mike R. Haller commented on MRM-1449:
-------------------------------------
I'm going to try, the Archiva code base is very large.
I found the location which might be the first starting point: {{org.apache.maven.archiva.proxy.DefaultRepositoryProxyConnectors.transferFile()}} is where the Maven {{Wagon}} is asked by Archiva to retrieve files. E.g. for a HTTP Wagon, this is where the connection is opened. I assume keeping a counter in this class would be possible. (Are Plexus components stateful and singletons?)
Also, need to think about whether to
1) stall the requests e.g. just wait a little bit before actually opening the connection. This may block the thread pretty long. (Bounded queue or counter?)
2) whether to cancel the request and return a "not connected" to the caller and let the Archiva core handle the deferred retry.
For incoming connections, this could also be done with an Apache bandwidth throttling module.
> Remote connections for many repos exhausts proxy limits
> -------------------------------------------------------
>
> Key: MRM-1449
> URL: http://jira.codehaus.org/browse/MRM-1449
> Project: Archiva
> Issue Type: New Feature
> Components: remote proxy
> Reporter: Mike R. Haller
>
> Our Archiva installation uses a company-internal caching proxy (ISA Server) to connect to remote repositories.
> When there are many remote repositories and many developers trying to look up artifacts (existing and non-existing artifacts, e.g. often -sources and -javadoc attachments), Archiva is creating many HTTP connections to the remote repositories.
> This leads to a situation where the caching proxy thinks Archiva is creating too many connections. The ISA warning mail even suggests the host computer may be infected with a worm because it creates so many new connections and blocks the host completely for all outgoing HTTP requests.
> The policies for the remote repositories are configured for retrieving "once", "never" or "daily", depending on whether it's releases or snapshots. Caching failures is disabled and i'm trying with enabled failure caching, but it doesn't make much difference and the problem still occurs once in a while.
> I think Archiva should have a configurable way to limit the number of (new) connections made per time unit, e.g. "max 60 connections / minute" to prevent this. It's kind of a potential denial of service vulnerability.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (MRM-1449) Remote connections for many repos
exhausts proxy limits
Posted by "Jan-Sebastian Winckelmann (JIRA)" <ji...@codehaus.org>.
[ https://jira.codehaus.org/browse/MRM-1449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=272675#comment-272675 ]
Jan-Sebastian Winckelmann commented on MRM-1449:
------------------------------------------------
We are hit by the some problem.
> Remote connections for many repos exhausts proxy limits
> -------------------------------------------------------
>
> Key: MRM-1449
> URL: https://jira.codehaus.org/browse/MRM-1449
> Project: Archiva
> Issue Type: New Feature
> Components: remote proxy
> Reporter: Mike R. Haller
> Fix For: Backlog
>
>
> Our Archiva installation uses a company-internal caching proxy (ISA Server) to connect to remote repositories.
> When there are many remote repositories and many developers trying to look up artifacts (existing and non-existing artifacts, e.g. often -sources and -javadoc attachments), Archiva is creating many HTTP connections to the remote repositories.
> This leads to a situation where the caching proxy thinks Archiva is creating too many connections. The ISA warning mail even suggests the host computer may be infected with a worm because it creates so many new connections and blocks the host completely for all outgoing HTTP requests.
> The policies for the remote repositories are configured for retrieving "once", "never" or "daily", depending on whether it's releases or snapshots. Caching failures is disabled and i'm trying with enabled failure caching, but it doesn't make much difference and the problem still occurs once in a while.
> I think Archiva should have a configurable way to limit the number of (new) connections made per time unit, e.g. "max 60 connections / minute" to prevent this. It's kind of a potential denial of service vulnerability.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (MRM-1449) Remote connections for many repos
exhausts proxy limits
Posted by "Brett Porter (JIRA)" <ji...@codehaus.org>.
[ http://jira.codehaus.org/browse/MRM-1449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Brett Porter updated MRM-1449:
------------------------------
Fix Version/s: Backlog
any progress on this?
> Remote connections for many repos exhausts proxy limits
> -------------------------------------------------------
>
> Key: MRM-1449
> URL: http://jira.codehaus.org/browse/MRM-1449
> Project: Archiva
> Issue Type: New Feature
> Components: remote proxy
> Reporter: Mike R. Haller
> Fix For: Backlog
>
>
> Our Archiva installation uses a company-internal caching proxy (ISA Server) to connect to remote repositories.
> When there are many remote repositories and many developers trying to look up artifacts (existing and non-existing artifacts, e.g. often -sources and -javadoc attachments), Archiva is creating many HTTP connections to the remote repositories.
> This leads to a situation where the caching proxy thinks Archiva is creating too many connections. The ISA warning mail even suggests the host computer may be infected with a worm because it creates so many new connections and blocks the host completely for all outgoing HTTP requests.
> The policies for the remote repositories are configured for retrieving "once", "never" or "daily", depending on whether it's releases or snapshots. Caching failures is disabled and i'm trying with enabled failure caching, but it doesn't make much difference and the problem still occurs once in a while.
> I think Archiva should have a configurable way to limit the number of (new) connections made per time unit, e.g. "max 60 connections / minute" to prevent this. It's kind of a potential denial of service vulnerability.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (MRM-1449) Remote connections for many repos
exhausts proxy limits
Posted by "Brett Porter (JIRA)" <ji...@codehaus.org>.
[ http://jira.codehaus.org/browse/MRM-1449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=252634#action_252634 ]
Brett Porter commented on MRM-1449:
-----------------------------------
You're in the right place. We actually use spring underneath, but the plexus annotations still exist in many pieces of the code. The default for most is a singleton instance (true for DRPC, but not wagons). Instead of a counter, a list might be appropriate, as that list of requests could be inspected elsewhere for tracking the system status.
Configuration is probably good to address the two options. I'd go with a blocking queue, since most outgoing requests are short and it will free up in a reasonable time. If it grows to a certain size or timeout you might start failing to avoid spiralling out of control as it backs up.
> Remote connections for many repos exhausts proxy limits
> -------------------------------------------------------
>
> Key: MRM-1449
> URL: http://jira.codehaus.org/browse/MRM-1449
> Project: Archiva
> Issue Type: New Feature
> Components: remote proxy
> Reporter: Mike R. Haller
>
> Our Archiva installation uses a company-internal caching proxy (ISA Server) to connect to remote repositories.
> When there are many remote repositories and many developers trying to look up artifacts (existing and non-existing artifacts, e.g. often -sources and -javadoc attachments), Archiva is creating many HTTP connections to the remote repositories.
> This leads to a situation where the caching proxy thinks Archiva is creating too many connections. The ISA warning mail even suggests the host computer may be infected with a worm because it creates so many new connections and blocks the host completely for all outgoing HTTP requests.
> The policies for the remote repositories are configured for retrieving "once", "never" or "daily", depending on whether it's releases or snapshots. Caching failures is disabled and i'm trying with enabled failure caching, but it doesn't make much difference and the problem still occurs once in a while.
> I think Archiva should have a configurable way to limit the number of (new) connections made per time unit, e.g. "max 60 connections / minute" to prevent this. It's kind of a potential denial of service vulnerability.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira