You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@archiva.apache.org by Cristiano Costantini <cr...@gmail.com> on 2011/07/13 16:50:56 UTC
Archiva very slow -
Hello all,
I've set up Archiva as a mirror for many "Remote Repositories", which I've
then aggregated into two different Repository Proxy Connector (one called
"External", which holds famous repositories like maven central, jboss,
apache, and one called "Temp", which holds repositories I do not trust
completely and which I want to supervise which actual artifact I need, I
find on them).
The two are then grouped into a single Repository Group called public, so
that I mirror all to my http://localhost:9081/archiva/repository/public/ on
my maven settings.xml.
Well,
if with this configuration I try the procedure described at page 54 of the
Book "Apache Maven 2: Effective Implementation" (cit. "Let's rebuild the
project again with a clean local repository. We will delete the contents of
our <user_home>./m2/book-repository local repository and then package our
project. So, how did our build go now? For me it took just 13 seconds!"),
for me also the second build is very long!!
I inspected with wireshark the server on which I've installed Archiva, and
I've seen it sends lots of HTTP requests to the remote repositories which
extremely slow down the build!!! But I think I've set correctly archiva to
cache all release artifacts and download them only once...:-(
any "Proxy Connector" is set with:
Return error when: always
Cache failures: yes
Releases: once
On remote error: stop
Checksum: fix
Snapshots: never
(I want stable builds, I want only release artifacts! ;-) )
Two Important things I've noticed: all the request done by archiva lead to
404 Bad Request responses (then, archiva returns a cached artifact to the
maven process) and if a disable all the proxy connectors, then yes the build
is fast!
Anyone can help me on configuring Archiva to do not repeat any type of
request to remote servers (for releases) at all?
Could you point me out on which relevant configuration parameter I can send
you to investigate if I'm doing something wrong?
I'm using archiva 1.3.5.
Thank you very much,
Cristiano
Re: Archiva very slow -
Posted by Not Zippy <no...@gmail.com>.
Archiva 1.3.5 does not build the maven-metadata file automatically this
could be an issue with the speed, since it needs to then scan the repository
instead of being directed. Fixed in 1.4 though
See bug http://jira.codehaus.org/browse/MRM-1477
Nz
On Thu, Jul 14, 2011 at 11:08 PM, Brett Porter <br...@apache.org> wrote:
>
> On 14/07/2011, at 12:50 AM, Cristiano Costantini wrote:
>
> > any "Proxy Connector" is set with:
> >
> > Return error when: always
> > Cache failures: yes
> > Releases: once
> > On remote error: stop
> > Checksum: fix
> > Snapshots: never
> >
> > (I want stable builds, I want only release artifacts! ;-) )
>
> This looks right
>
> >
> > Two Important things I've noticed: all the request done by archiva lead
> to
> > 404 Bad Request responses (then, archiva returns a cached artifact to the
> > maven process) and if a disable all the proxy connectors, then yes the
> build
> > is fast!
>
> 400 remotely, or Archiva is giving back 400? (I assume you meant 400 Bad
> Request, not 404 Not Found)
>
> >
> > Anyone can help me on configuring Archiva to do not repeat any type of
> > request to remote servers (for releases) at all?
> > Could you point me out on which relevant configuration parameter I can
> send
> > you to investigate if I'm doing something wrong?
> >
> > I'm using archiva 1.3.5.
>
> It should never repeat for an artifact - but it will for maven-metadata.xml
> files when requested by the client. There's two things you can do to speed
> it up:
> - add whitelists on the proxy connectors where you know only a subset of
> artifacts should come from there - that way it won't ever look out there
> unless you intended it to
> - make sure the maven instance has similar <repository> definitions so it
> doesn't harrass Archiva for snapshots it is never going to have
>
> The other thing to check is the network speed from Archiva to the world -
> though I'm also curious about the error you listed above.
>
> HTH,
> Brett
>
> --
> Brett Porter
> brett@apache.org
> http://brettporter.wordpress.com/
> http://au.linkedin.com/in/brettporter
>
>
>
>
>
Re: Archiva very slow -
Posted by Brett Porter <br...@apache.org>.
On 16/07/2011, at 9:11 PM, Cristiano Costantini wrote:
> Hi Brett thank you for your reply (I admit it, I explicitly quoted a
> sentence from your book for raising your attention ;-) )
:)
> It seems that the Repository Group with two Managed Repositories containing
> remote Proxy Connection, does not work well as a cache: if the artifact is
> found on the second of the two Managed Repositories, requesting the artifact
> to the Repository Group makes Archiva first search the artifact on the first
> Repository, which make HTTP requests (returning 404), and then search the
> Artifact on the second Repository, where the cache works (Archiva make no
> more HTTP requests).
I think this behaviour is correct though - if we were to cache a 404, the artifact might be there later. I've generally found it to be more confusing when that is the way it is configured. Instead, we encourage ordering the repositories (both in the proxy connectors and the groups) to give the most likely hit first, and to use whitelists to increase the likelihood of a hit (or a legitimate skip) of a repository.
>
> My first set-up was intended to separate all the artifacts coming from
> unknown/untrusted repositories so that I can (try to) eliminate them from my
> dependencies.
> The second set-up solves slowness issues but now I can't no more investigate
> on the dependencies coming from these repositories...
I'm not quite sure what you mean - how did you investigate the dependencies before that you can't now? I think the pattern of one managed repo to one proxy is generally a good idea...
>
> Another question, which may be a work around for me: is there a possibility
> to know from which URL a cached artifact has been downloaded? Does Archiva
> logs this information? Does it would be easy to write a fix by myself?
If you are using one repository per remote proxy, I would have thought that would make it clear - so perhaps I'm misunderstanding what you are looking for.
- Brett
--
Brett Porter
brett@apache.org
http://brettporter.wordpress.com/
http://au.linkedin.com/in/brettporter
Re: Archiva very slow -
Posted by Cristiano Costantini <cr...@gmail.com>.
Hi Brett thank you for your reply (I admit it, I explicitly quoted a
sentence from your book for raising your attention ;-) )
well, no, it was really 404, so it was a "Not Found", and it was given back
by the remote repository to Archiva. The reason of the slowness was that
Archiva was sending out HTTP requests for artifact while they should have
been cached.
Now I've changed the configuration to a single Repository Proxy Connector
mapped to a single Managed Repository which caches ALL the artifacts and it
works fast as expected.
Considering this, I think that problem was on the Repository Groups!
It seems that the Repository Group with two Managed Repositories containing
remote Proxy Connection, does not work well as a cache: if the artifact is
found on the second of the two Managed Repositories, requesting the artifact
to the Repository Group makes Archiva first search the artifact on the first
Repository, which make HTTP requests (returning 404), and then search the
Artifact on the second Repository, where the cache works (Archiva make no
more HTTP requests).
To be more clear, my previous configuration (the slow one) was something
like this:
Repository Group: "public"
|-- Managed repository: "releases"
|-- Managed repository: "snapshots"
|-- Managed repository: "externals"
| \-- Proxy Connectors
| |-- http://repo1.maven.org/maven2
| |-- http://download.java.net/maven/2/
| |-- http://repository.jboss.com/maven2/
| |-- ...
\-- Managed repository: "temp"
\-- Proxy Connectors
|-- http://people.apache.org/repo/m2-incubating-repository/
|-- http://svn.apache.org/repos/asf/servicemix/m2-repo
|-- http://s3.amazonaws.com/maven.springframework.org/milestone
|-- ...
This is the configuration I use now which is no more slow:
Managed repository: "releases"
Managed repository: "snapshots"
Managed repository: "public"
\-- Proxy Connectors
|-- http://repo1.maven.org/maven2
|-- http://download.java.net/maven/2/
|-- http://repository.jboss.com/maven2/
|-- ...
|-- http://people.apache.org/repo/m2-incubating-repository/
|-- http://svn.apache.org/repos/asf/servicemix/m2-repo
|-- http://s3.amazonaws.com/maven.springframework.org/milestone
|-- ...
My first set-up was intended to separate all the artifacts coming from
unknown/untrusted repositories so that I can (try to) eliminate them from my
dependencies.
The second set-up solves slowness issues but now I can't no more investigate
on the dependencies coming from these repositories...
Another question, which may be a work around for me: is there a possibility
to know from which URL a cached artifact has been downloaded? Does Archiva
logs this information? Does it would be easy to write a fix by myself?
thank you all,
have a nice weekend.
Cristiano
Re: Archiva very slow -
Posted by Brett Porter <br...@apache.org>.
On 14/07/2011, at 12:50 AM, Cristiano Costantini wrote:
> any "Proxy Connector" is set with:
>
> Return error when: always
> Cache failures: yes
> Releases: once
> On remote error: stop
> Checksum: fix
> Snapshots: never
>
> (I want stable builds, I want only release artifacts! ;-) )
This looks right
>
> Two Important things I've noticed: all the request done by archiva lead to
> 404 Bad Request responses (then, archiva returns a cached artifact to the
> maven process) and if a disable all the proxy connectors, then yes the build
> is fast!
400 remotely, or Archiva is giving back 400? (I assume you meant 400 Bad Request, not 404 Not Found)
>
> Anyone can help me on configuring Archiva to do not repeat any type of
> request to remote servers (for releases) at all?
> Could you point me out on which relevant configuration parameter I can send
> you to investigate if I'm doing something wrong?
>
> I'm using archiva 1.3.5.
It should never repeat for an artifact - but it will for maven-metadata.xml files when requested by the client. There's two things you can do to speed it up:
- add whitelists on the proxy connectors where you know only a subset of artifacts should come from there - that way it won't ever look out there unless you intended it to
- make sure the maven instance has similar <repository> definitions so it doesn't harrass Archiva for snapshots it is never going to have
The other thing to check is the network speed from Archiva to the world - though I'm also curious about the error you listed above.
HTH,
Brett
--
Brett Porter
brett@apache.org
http://brettporter.wordpress.com/
http://au.linkedin.com/in/brettporter
Re: Archiva very slow -
Posted by Marius Kruger <am...@gmail.com>.
Hi,
On 13 July 2011 16:50, Cristiano Costantini
<cr...@gmail.com>wrote:
> ...
> So, how did our build go now? For me it took just 13 seconds!"),
> for me also the second build is very long!!
>
>
for this sort of performance issues, I did what is suggested here:
https://cwiki.apache.org/ARCHIVA/high-performance-archiva.html
Serve a readonly version of your repository with apache,
then getting artifacts that is already present becomes really quick.
You still use the normal archiva repo when artifacts is not there yet.
--
<>< Marius ><>