You are viewing a plain text version of this content. The canonical link for it is here.
Posted to ivy-user@ant.apache.org by Jing Xue <ji...@digizenstudio.com> on 2007/04/06 05:19:51 UTC

ranged versions and maven proxy

I have a maven-proxy, http://jabba:9999, running against the real
ibiblio repo, and the ivy ibiblio resolver pointed to the proxy. 
With this config:

<ivysettings>
  <property name="ivy.ibiblio.default.artifact.root" value="http://jabba:9999/repository" />
  <property name="ivyx.returnFirst" value="true" override="false" />

  <settings defaultResolver="main" checkUpToDate="true" />
  <resolvers>
    <chain name="main" returnFirst="${ivyx.returnFirst}">
      <cache name="cacheResolver" />
      <ibiblio name="ibiblioResolver" m2compatible="true" root="${ivy.ibiblio.default.artifact.root}" />
     </chain>
  </resolvers>
</ivysettings>

I got this error message:

[ivy:resolve] problem occured while resolving dependency: [ log4j | log4j | 1.2.+ ] {build=[default]} with cacheResolver: java.util.EmptyStackException
[ivy:resolve] 		tried http://jabba:9999/repository/log4j/log4j/1.2.+/log4j-1.2.+.pom
[ivy:resolve] CLIENT ERROR: Not Found url=http://jabba:9999/repository/log4j/log4j/1.2.+/log4j-1.2.+.pom
[ivy:resolve] 		tried http://jabba:9999/repository/log4j/log4j/[revision]/log4j-[revision].pom
[ivy:resolve] 		tried http://jabba:9999/repository/log4j/log4j/1.2.+/log4j-1.2.+.jar
[ivy:resolve] CLIENT ERROR: Not Found url=http://jabba:9999/repository/log4j/log4j/1.2.+/log4j-1.2.+.jar
[ivy:resolve] 		tried http://jabba:9999/repository/log4j/log4j/[revision]/log4j-[revision].jar
[ivy:resolve] 	ibiblioResolver: no ivy file nor artifact found for [ log4j | log4j | 1.2.+ ]

I also tried setting awaysCheckExactRevision="false", and got:

[ivy:resolve] problem occured while resolving dependency: [ log4j | log4j | 1.2.+ ] {build=[default]} with cacheResolver: java.util.EmptyStackException
[ivy:resolve] 		tried http://jabba:9999/repository/log4j/log4j/[revision]/log4j-[revision].pom
[ivy:resolve] 		tried http://jabba:9999/repository/log4j/log4j/[revision]/log4j-[revision].jar
[ivy:resolve] 	ibiblioResolver: no ivy file nor artifact found for [ log4j | log4j | 1.2.+ ]

Note that this actually happened when I _do_ have log4j 1.2.14 in the
maven proxy repository. So I looked into the code, and looks like the
problem is currently URLResolver has URLRepository hard-coded, which in
turn has the ApacheUrlLister hard-coded, which can (obviously) handle
only the authentic ibiblio directory listing. Is it possible to expose
the url lister in the settings?

BTW, on a side point unrelated to the topic of this message, please note
that in the above scenario, I also _do_ have the log4j 1.2.14 in my
local ivy cache, yet it went ahead and checked ibiblioResolver...

Thanks.
-- 
Jing Xue

Re: ranged versions and maven proxy

Posted by Xavier Hanin <xa...@gmail.com>.
On 4/6/07, Jing Xue <ji...@digizenstudio.com> wrote:
>
>
> For whatever its worth, my guess (from skimming through the code) is
> that in the ivy metadata files generated in the local cache, the
> resolver associated to an artifact is always kept as the one that
> resolved it _originally_, rather than getting updated to be the one
> that was able to resolve it _most recently_.  That's probably why if I
> commented out ibiblioResolver, it would start reporting the artifact
> being resolved by the cacheResolver, and stay that way even after I
> put the ibiblioResolver back.


It makes sense, indeed. Actually, the cache resolver is not frequently used,
so I'm not too surprised you run into some kind of trouble :-) Usually when
a resolver find a module in cache, it returns it even if it hasn't been
resolved by the same resolver before, but do not update the cache metadata
because it doesn't even know if it could find such module. But here with a
cache resolver we are facing a very specific case. Anyway, since we will
deeply review cache management before the final 2.0 release, this kind of
thing will hopefully be improved. But it would be nice to remember to test
that, so could you add a comment to the cache management related issue
(IVY-399)?

Thanks

- Xavier

Thanks.
> --
> Jing Xue
>
>
>
>


-- 
Learn Ivy at ApacheCon: http://www.eu.apachecon.com/
Manage your dependencies with Ivy!
http://incubator.apache.org/ivy/

Re: ranged versions and maven proxy

Posted by Jing Xue <ji...@digizenstudio.com>.

Hi Xavier,

Thanks for the responses. Please see below for my comment:

Quoting Xavier Hanin <xa...@gmail.com>:

> The ApacheUrlLister, depsite its name, is able to list urls on a good number
> of http server, as long as they have directory listing enabled. In the case
> of maven proxy, I guess you can't browse a directory, or the directory
> doesn't look like usual http servers directory listing.

You are right, the directory page the proxy generates is rendered  
differently, which of course throws the UrlLister off the track. :-)

> So opening the settings to be able to set the URLLister used is a good idea,
> but I think that as part of our attempt to make better integration with
> maven repository we should also make sure that the ibiblio resolver is
> compatible with maven-proxy (I think we should use maven metadata files,
> instead of browsing directories as we do).

I agree. Working with the maven metadata not only solves this problem,  
but also the more troublesome one where the proxy won't be able to  
list all the artifacts properly if it doesn't have all of them locally  
already, because proxies don't go to the underlying repository for  
directory listing.

> Could you open an issue for opening the API of URLResolver, and I'll open
> one for maven proxy compatibility.

Will do as soon as I can.

> This is really strange. I haven't tested with a cache resolver (it's not
> very often used, since Ivy itself relies on its cache as often as possible
> without any specific settings), but it's unit tested in general case, and
> I've checked the code and I don't see why if the first resolver returns
> something the chain could continue (with returnFirst=true). But if you want
> to help us investigate, could you please send a debug log, adn we'll see if
> we better understand what's happening.

Sure, will do with the latest trunk build as soon as I get to it.

For whatever its worth, my guess (from skimming through the code) is  
that in the ivy metadata files generated in the local cache, the  
resolver associated to an artifact is always kept as the one that  
resolved it _originally_, rather than getting updated to be the one  
that was able to resolve it _most recently_.  That's probably why if I  
commented out ibiblioResolver, it would start reporting the artifact  
being resolved by the cacheResolver, and stay that way even after I  
put the ibiblioResolver back.

Thanks.
-- 
Jing Xue

Re: ranged versions and maven proxy

Posted by Xavier Hanin <xa...@gmail.com>.
On 4/6/07, Xavier Hanin <xa...@gmail.com> wrote:
>
> Could you open an issue for opening the API of URLResolver, and I'll open
> one for maven proxy compatibility.
>

FYI, I've created the issue:
https://issues.apache.org/jira/browse/IVY-466

One other thing, if you use the latest trunk version, the verbose
information about what Ivy found on the server and what it really tried is
more precise, so it's easier to understand what's going on.

- Xavier

-- 
Learn Ivy at ApacheCon: http://www.eu.apachecon.com/
Manage your dependencies with Ivy!
http://incubator.apache.org/ivy/

Re: ranged versions and maven proxy

Posted by Xavier Hanin <xa...@gmail.com>.
On 4/7/07, Jing Xue <ji...@digizenstudio.com> wrote:
>
>
> Hi Xaviar,
>
> Please see the following debug output:
>
> [ivy:resolve]   using main to resolve [ org.apache.ant | ant-junit | 1.7.0]
> [ivy:resolve] pre 1.3 ivy file: using exactOrRegexp as default matcher
> [ivy:resolve]   found ivy file in cache for [ org.apache.ant | ant-junit |
> 1.7.0 ] (resolved by ibiblioResolver):
> /home/jingxue/.ivy/cache/org.apache.ant/ant-junit/ivy-1.7.0.xml
> [ivy:resolve]   cacheResolver: revision in cache: [ org.apache.ant |
> ant-junit | 1.7.0 ]
> [ivy:resolve]   found [ org.apache.ant | ant-junit | 1.7.0 ] in
> ibiblioResolver
>
> I think the cache resolver did successfully resolve it, but somehow
> prints the wrong resolver name on the last name.


Yes, I know this is disturbing, but it's the way Ivy handles cache for the
moment: to avoid going through all a chain just to figure out that the last
resolver has already resolved the module and that it is in cache, each
resolver actually checks the cache, and if one finds the module in cache, it
returns it even if the metadata in cache says that it was resolved by
another resolver the first time. And in this case Ivy keeps the information
that it was resolved by ibiblio, and show it in the log.

But this is likely to change in next version, since we will review the whole
cache management.

As for the log4j case in my previous message, I realized that ivy was
> actually going to ibiblio because the cache resolver isn't capable of
> resolving
> non-exact versions.


Ok, so this is normal. The question is: Is it really normal that cache
resolver isn't capable of resolving dynamic revisions? I don't really think
so, so we may review this in next version (as part of cache management
review). Any input on this subject is welcome.

- Xavier

Thanks.
> --
> Jing Xue
>



-- 
Learn Ivy at ApacheCon: http://www.eu.apachecon.com/
Manage your dependencies with Ivy!
http://incubator.apache.org/ivy/

Re: ranged versions and maven proxy

Posted by Jing Xue <ji...@digizenstudio.com>.
On Fri, Apr 06, 2007 at 08:42:27AM +0200, Xavier Hanin wrote:
> On 4/6/07, Jing Xue <ji...@digizenstudio.com> wrote:
> >BTW, on a side point unrelated to the topic of this message, please note
> >that in the above scenario, I also _do_ have the log4j 1.2.14 in my
> >local ivy cache, yet it went ahead and checked ibiblioResolver...
> 
> 
> This is really strange. I haven't tested with a cache resolver (it's not
> very often used, since Ivy itself relies on its cache as often as possible
> without any specific settings), but it's unit tested in general case, and
> I've checked the code and I don't see why if the first resolver returns
> something the chain could continue (with returnFirst=true). But if you want
> to help us investigate, could you please send a debug log, adn we'll see if
> we better understand what's happening.

Hi Xaviar,

Please see the following debug output:

[ivy:resolve]   using main to resolve [ org.apache.ant | ant-junit | 1.7.0 ]
[ivy:resolve] pre 1.3 ivy file: using exactOrRegexp as default matcher
[ivy:resolve]   found ivy file in cache for [ org.apache.ant | ant-junit | 1.7.0 ] (resolved by ibiblioResolver): /home/jingxue/.ivy/cache/org.apache.ant/ant-junit/ivy-1.7.0.xml
[ivy:resolve]   cacheResolver: revision in cache: [ org.apache.ant | ant-junit | 1.7.0 ]
[ivy:resolve]   found [ org.apache.ant | ant-junit | 1.7.0 ] in ibiblioResolver

I think the cache resolver did successfully resolve it, but somehow
prints the wrong resolver name on the last name.

As for the log4j case in my previous message, I realized that ivy was
actually going to ibiblio because the cache resolver isn't capable of resolving
non-exact versions.

Thanks.
-- 
Jing Xue

Re: ranged versions and maven proxy

Posted by Xavier Hanin <xa...@gmail.com>.
On 4/6/07, Jing Xue <ji...@digizenstudio.com> wrote:
>
> I have a maven-proxy, http://jabba:9999, running against the real
> ibiblio repo, and the ivy ibiblio resolver pointed to the proxy.
> With this config:
>
> <ivysettings>
>   <property name="ivy.ibiblio.default.artifact.root" value="
> http://jabba:9999/repository" />
>   <property name="ivyx.returnFirst" value="true" override="false" />
>
>   <settings defaultResolver="main" checkUpToDate="true" />
>   <resolvers>
>     <chain name="main" returnFirst="${ivyx.returnFirst}">
>       <cache name="cacheResolver" />
>       <ibiblio name="ibiblioResolver" m2compatible="true" root="${
> ivy.ibiblio.default.artifact.root}" />
>      </chain>
>   </resolvers>
> </ivysettings>
>
> I got this error message:
>
> [ivy:resolve] problem occured while resolving dependency: [ log4j | log4j
> | 1.2.+ ] {build=[default]} with cacheResolver:
> java.util.EmptyStackException
> [ivy:resolve]           tried
> http://jabba:9999/repository/log4j/log4j/1.2.+/log4j-1.2.+.pom
> [ivy:resolve] CLIENT ERROR: Not Found url=
> http://jabba:9999/repository/log4j/log4j/1.2.+/log4j-1.2.+.pom
> [ivy:resolve]           tried
> http://jabba:9999/repository/log4j/log4j/[revision]/log4j-[revision].pom
> [ivy:resolve]           tried
> http://jabba:9999/repository/log4j/log4j/1.2.+/log4j-1.2.+.jar
> [ivy:resolve] CLIENT ERROR: Not Found url=
> http://jabba:9999/repository/log4j/log4j/1.2.+/log4j-1.2.+.jar
> [ivy:resolve]           tried
> http://jabba:9999/repository/log4j/log4j/[revision]/log4j-[revision].jar
> [ivy:resolve]   ibiblioResolver: no ivy file nor artifact found for [
> log4j | log4j | 1.2.+ ]
>
> I also tried setting awaysCheckExactRevision="false", and got:
>
> [ivy:resolve] problem occured while resolving dependency: [ log4j | log4j
> | 1.2.+ ] {build=[default]} with cacheResolver:
> java.util.EmptyStackException
> [ivy:resolve]           tried
> http://jabba:9999/repository/log4j/log4j/[revision]/log4j-[revision].pom
> [ivy:resolve]           tried
> http://jabba:9999/repository/log4j/log4j/[revision]/log4j-[revision].jar
> [ivy:resolve]   ibiblioResolver: no ivy file nor artifact found for [
> log4j | log4j | 1.2.+ ]
>
> Note that this actually happened when I _do_ have log4j 1.2.14 in the
> maven proxy repository. So I looked into the code, and looks like the
> problem is currently URLResolver has URLRepository hard-coded, which in
> turn has the ApacheUrlLister hard-coded, which can (obviously) handle
> only the authentic ibiblio directory listing. Is it possible to expose
> the url lister in the settings?


The ApacheUrlLister, depsite its name, is able to list urls on a good number
of http server, as long as they have directory listing enabled. In the case
of maven proxy, I guess you can't browse a directory, or the directory
doesn't look like usual http servers directory listing.

So opening the settings to be able to set the URLLister used is a good idea,
but I think that as part of our attempt to make better integration with
maven repository we should also make sure that the ibiblio resolver is
compatible with maven-proxy (I think we should use maven metadata files,
instead of browsing directories as we do).

Could you open an issue for opening the API of URLResolver, and I'll open
one for maven proxy compatibility.

BTW, on a side point unrelated to the topic of this message, please note
> that in the above scenario, I also _do_ have the log4j 1.2.14 in my
> local ivy cache, yet it went ahead and checked ibiblioResolver...


This is really strange. I haven't tested with a cache resolver (it's not
very often used, since Ivy itself relies on its cache as often as possible
without any specific settings), but it's unit tested in general case, and
I've checked the code and I don't see why if the first resolver returns
something the chain could continue (with returnFirst=true). But if you want
to help us investigate, could you please send a debug log, adn we'll see if
we better understand what's happening.

- Xavier

Thanks.
> --
> Jing Xue
>



-- 
Learn Ivy at ApacheCon: http://www.eu.apachecon.com/
Manage your dependencies with Ivy!
http://incubator.apache.org/ivy/