You are viewing a plain text version of this content. The canonical link for it is here.
Posted to ivy-user@ant.apache.org by "Yu, Aaron (IT)" <Fa...@MorganStanley.com> on 2009/01/16 06:25:50 UTC
IvyDE cache location
Hi,
I have a problem of using Ivy and IvyDE.
At my firm we are in the process of switching to Ivy for managing our
java dependencies. We do not want thousands of developers to resolve
dependencies in different ways so we provide a centralized , read-only
ivy-settings.xml file which is used by all developers. However, this
presents us with a problem. We would like to make it possible for our
users to set their own cache locations, because if they use the same
cache location for all projects, when they clean the cache in one
project, the whole cache location will be cleaned, and this will crash
the building of other projects.
I know we can create a project-specific ivy-settings.xml file, and use
'defaultCacheDir', 'resolutionCacheDir' and 'repositoryCacheDir'
attributes in 'caches' element to set the project-based cache location.
But as I mentioned above, to make all the projects more manageable and
shareable, we do not encourage developers to have the project-specific
ivy setting file.
To get around this we have implemented a new properties page for the
ivyde plugin under the existing one called simply 'Cache'. This allows a
user to choose the cache location. The choices are: Default
($HOME/.ivy2) and Workspace (workspace_dir/.metadata/.ivy2). We then
reset the ivy.home System property and trigger re-resolution of all
project ivy classpath containers when the users changes this setting
(the change affects all projects in the workspace since ivy core reads
the ivy.home System property to find out where it should do its
caching). However, we are now wondering if what is really needed is a
private cache area for each project in the workspace, for example:
workspace_dir/.metadata/.ivy2/project_name. But this is not possible at
the moment due to the way in which ivy core decided where to place its
cache. (And, as we said, we do not want users to have to specify their
own ivy-settings for each project for this reason alone.)
Other people at our firm have also pointed out that it might be better
if ivy core used a version specific cache location by default since
there could be differences in cache metadata between versions. This
would cause cache corruption if different versions of ivy core were run
on the same system.
All this seems to indicate that the subject of caching needs rethinking
in ivy core and ivyde, and we would like to ask for some suggestions on
this.
Thanks!
Aaron Yu
Morgan Stanley | Technology
222 Yan An Road (East)
200002 Shanghai
Phone: +86 21 2321-2106
Fan.Yu@MorganStanley.com
--------------------------------------------------------
NOTICE: If received in error, please destroy and notify sender. Sender does not intend to waive confidentiality or privilege. Use of this email is prohibited when received in error.
Re: IvyDE cache location
Posted by Nicolas Lalevée <ni...@hibnet.org>.
Le 16 janv. 09 à 23:07, Mitch Gitman a écrit :
> Nicolas, correct me if I'm misstating something, but I look at an
> ivy:retrieve-like capability as a nice-to-have but not necessary
> feature for
> IvyDE.
>
> IvyDE itself really just exists as a nice convenience, so that
> classpaths
> resolve correctly in Eclipse. How it does that--whether with
> ivy:cachepath
> or with an ivy:retrieve to a specified directory--is somewhat of an
> implementation detail in this respect. Either way, the
> ClassNotFoundExceptions go away.
>
> The tool of record is still the build script itself. Once you have
> IvyDE
> doing Ivy retrieves to the lib directory that the build script is
> doing
> retrieves to, it kind of feels to me like you're letting the IDE (the
> second-class citizen) encroach into the realm of the build script (the
> first-class citizen).
I agree with you the reference should be the ant build system. For
instance eclipse can only manage one classpath by project, whereas we
usually use it to compile, run and test. So in my projects I ask IvyDE
to get 3 configurations: compile, runtime and test. But in ant the
javac task use an ivy cachepath with only the compile configuration,
for the junit task only the test configuration and for the building of
some war only the runtime configuration.
Eclipse setup should be as close as possible to the build and
deployment environment, but it is not the case.
> That concern aside, suppose the developer has blown away their Ivy
> cache. I
> don't see why the issue Aaron describes cannot be addressed simply by
> right-clicking on the Ivy icon in Package Explorer and selecting
> Resolve.
> Granted, it takes a little getting used to for developers to get in
> the
> habit of this, but this seems like part of the contract of using
> Ivy. As an
> Ivy consumer, you need to have some understanding as to when a
> resolve is
> necessary.
Well, I don't know what size is Aaron's project, how theses projects
are interconnected. But If you have 20 projects all relying on some
maven repository, it will really painful to do a resolve on every
project.
Maybe the real debatable question here is why the cache need to be
cleaned, normally it shouldn't. The cache is not aimed to modify how
the resolve process occurs, just speed it up. Actually I see only two
reasons that should stay exceptional:
* the size of the cache
* a repository has been setup with checkmodified=false but a broken
ivy.xml on that repository has been fixed
Nicolas
>
>
> On Fri, Jan 16, 2009 at 1:40 PM, Nicolas Lalevée <nicolas.lalevee@hibnet.org
>> wrote:
>
>>
>> Le 16 janv. 09 à 06:25, Yu, Aaron (IT) a écrit :
>>
>>
>> Hi,
>>>
>>> I have a problem of using Ivy and IvyDE.
>>> At my firm we are in the process of switching to Ivy for managing
>>> our
>>> java dependencies. We do not want thousands of developers to resolve
>>> dependencies in different ways so we provide a centralized ,
>>> read-only
>>> ivy-settings.xml file which is used by all developers. However,
>>> this
>>> presents us with a problem. We would like to make it possible for
>>> our
>>> users to set their own cache locations, because if they use the
>>> same
>>> cache location for all projects, when they clean the cache in one
>>> project, the whole cache location will be cleaned, and this will
>>> crash
>>> the building of other projects.
>>>
>>> I know we can create a project-specific ivy-settings.xml file, and
>>> use
>>> 'defaultCacheDir', 'resolutionCacheDir' and 'repositoryCacheDir'
>>> attributes in 'caches' element to set the project-based cache
>>> location.
>>> But as I mentioned above, to make all the projects more manageable
>>> and
>>> shareable, we do not encourage developers to have the project-
>>> specific
>>> ivy setting file.
>>>
>>> To get around this we have implemented a new properties page for the
>>> ivyde plugin under the existing one called simply 'Cache'. This
>>> allows a
>>> user to choose the cache location. The choices are: Default
>>> ($HOME/.ivy2) and Workspace (workspace_dir/.metadata/.ivy2). We then
>>> reset the ivy.home System property and trigger re-resolution of all
>>> project ivy classpath containers when the users changes this setting
>>> (the change affects all projects in the workspace since ivy core
>>> reads
>>> the ivy.home System property to find out where it should do its
>>> caching). However, we are now wondering if what is really needed
>>> is a
>>> private cache area for each project in the workspace, for example:
>>> workspace_dir/.metadata/.ivy2/project_name. But this is not
>>> possible at
>>> the moment due to the way in which ivy core decided where to place
>>> its
>>> cache. (And, as we said, we do not want users to have to specify
>>> their
>>> own ivy-settings for each project for this reason alone.)
>>>
>>> Other people at our firm have also pointed out that it might be
>>> better
>>> if ivy core used a version specific cache location by default since
>>> there could be differences in cache metadata between versions. This
>>> would cause cache corruption if different versions of ivy core
>>> were run
>>> on the same system.
>>>
>>> All this seems to indicate that the subject of caching needs
>>> rethinking
>>> in ivy core and ivyde, and we would like to ask for some
>>> suggestions on
>>> this.
>>>
>>
>> It seems to me that the way you want to use caching is actually
>> like using
>> the retrieve job.
>> The retrieve job would get the dependencies and put it in your
>> project lib
>> directory (somehow your by-project cache), and the IvyDE would
>> build the
>> classpath based on the content of that lib directory.
>> Unfortunately this feature is not implemented yet:
>> https://issues.apache.org/jira/browse/IVYDE-56
>>
>> Nicolas
>>
>>
Re: IvyDE cache location
Posted by Mitch Gitman <mg...@gmail.com>.
Nicolas, correct me if I'm misstating something, but I look at an
ivy:retrieve-like capability as a nice-to-have but not necessary feature for
IvyDE.
IvyDE itself really just exists as a nice convenience, so that classpaths
resolve correctly in Eclipse. How it does that--whether with ivy:cachepath
or with an ivy:retrieve to a specified directory--is somewhat of an
implementation detail in this respect. Either way, the
ClassNotFoundExceptions go away.
The tool of record is still the build script itself. Once you have IvyDE
doing Ivy retrieves to the lib directory that the build script is doing
retrieves to, it kind of feels to me like you're letting the IDE (the
second-class citizen) encroach into the realm of the build script (the
first-class citizen).
That concern aside, suppose the developer has blown away their Ivy cache. I
don't see why the issue Aaron describes cannot be addressed simply by
right-clicking on the Ivy icon in Package Explorer and selecting Resolve.
Granted, it takes a little getting used to for developers to get in the
habit of this, but this seems like part of the contract of using Ivy. As an
Ivy consumer, you need to have some understanding as to when a resolve is
necessary.
On Fri, Jan 16, 2009 at 1:40 PM, Nicolas Lalevée <nicolas.lalevee@hibnet.org
> wrote:
>
> Le 16 janv. 09 à 06:25, Yu, Aaron (IT) a écrit :
>
>
> Hi,
>>
>> I have a problem of using Ivy and IvyDE.
>> At my firm we are in the process of switching to Ivy for managing our
>> java dependencies. We do not want thousands of developers to resolve
>> dependencies in different ways so we provide a centralized , read-only
>> ivy-settings.xml file which is used by all developers. However, this
>> presents us with a problem. We would like to make it possible for our
>> users to set their own cache locations, because if they use the same
>> cache location for all projects, when they clean the cache in one
>> project, the whole cache location will be cleaned, and this will crash
>> the building of other projects.
>>
>> I know we can create a project-specific ivy-settings.xml file, and use
>> 'defaultCacheDir', 'resolutionCacheDir' and 'repositoryCacheDir'
>> attributes in 'caches' element to set the project-based cache location.
>> But as I mentioned above, to make all the projects more manageable and
>> shareable, we do not encourage developers to have the project-specific
>> ivy setting file.
>>
>> To get around this we have implemented a new properties page for the
>> ivyde plugin under the existing one called simply 'Cache'. This allows a
>> user to choose the cache location. The choices are: Default
>> ($HOME/.ivy2) and Workspace (workspace_dir/.metadata/.ivy2). We then
>> reset the ivy.home System property and trigger re-resolution of all
>> project ivy classpath containers when the users changes this setting
>> (the change affects all projects in the workspace since ivy core reads
>> the ivy.home System property to find out where it should do its
>> caching). However, we are now wondering if what is really needed is a
>> private cache area for each project in the workspace, for example:
>> workspace_dir/.metadata/.ivy2/project_name. But this is not possible at
>> the moment due to the way in which ivy core decided where to place its
>> cache. (And, as we said, we do not want users to have to specify their
>> own ivy-settings for each project for this reason alone.)
>>
>> Other people at our firm have also pointed out that it might be better
>> if ivy core used a version specific cache location by default since
>> there could be differences in cache metadata between versions. This
>> would cause cache corruption if different versions of ivy core were run
>> on the same system.
>>
>> All this seems to indicate that the subject of caching needs rethinking
>> in ivy core and ivyde, and we would like to ask for some suggestions on
>> this.
>>
>
> It seems to me that the way you want to use caching is actually like using
> the retrieve job.
> The retrieve job would get the dependencies and put it in your project lib
> directory (somehow your by-project cache), and the IvyDE would build the
> classpath based on the content of that lib directory.
> Unfortunately this feature is not implemented yet:
> https://issues.apache.org/jira/browse/IVYDE-56
>
> Nicolas
>
>
Re: IvyDE cache location
Posted by Nicolas Lalevée <ni...@hibnet.org>.
Le 16 janv. 09 à 06:25, Yu, Aaron (IT) a écrit :
> Hi,
>
> I have a problem of using Ivy and IvyDE.
> At my firm we are in the process of switching to Ivy for managing our
> java dependencies. We do not want thousands of developers to resolve
> dependencies in different ways so we provide a centralized , read-
> only
> ivy-settings.xml file which is used by all developers. However, this
> presents us with a problem. We would like to make it possible for our
> users to set their own cache locations, because if they use the same
> cache location for all projects, when they clean the cache in one
> project, the whole cache location will be cleaned, and this will
> crash
> the building of other projects.
>
> I know we can create a project-specific ivy-settings.xml file, and use
> 'defaultCacheDir', 'resolutionCacheDir' and 'repositoryCacheDir'
> attributes in 'caches' element to set the project-based cache
> location.
> But as I mentioned above, to make all the projects more manageable and
> shareable, we do not encourage developers to have the project-specific
> ivy setting file.
>
> To get around this we have implemented a new properties page for the
> ivyde plugin under the existing one called simply 'Cache'. This
> allows a
> user to choose the cache location. The choices are: Default
> ($HOME/.ivy2) and Workspace (workspace_dir/.metadata/.ivy2). We then
> reset the ivy.home System property and trigger re-resolution of all
> project ivy classpath containers when the users changes this setting
> (the change affects all projects in the workspace since ivy core reads
> the ivy.home System property to find out where it should do its
> caching). However, we are now wondering if what is really needed is a
> private cache area for each project in the workspace, for example:
> workspace_dir/.metadata/.ivy2/project_name. But this is not possible
> at
> the moment due to the way in which ivy core decided where to place its
> cache. (And, as we said, we do not want users to have to specify their
> own ivy-settings for each project for this reason alone.)
>
> Other people at our firm have also pointed out that it might be better
> if ivy core used a version specific cache location by default since
> there could be differences in cache metadata between versions. This
> would cause cache corruption if different versions of ivy core were
> run
> on the same system.
>
> All this seems to indicate that the subject of caching needs
> rethinking
> in ivy core and ivyde, and we would like to ask for some suggestions
> on
> this.
It seems to me that the way you want to use caching is actually like
using the retrieve job.
The retrieve job would get the dependencies and put it in your project
lib directory (somehow your by-project cache), and the IvyDE would
build the classpath based on the content of that lib directory.
Unfortunately this feature is not implemented yet: https://issues.apache.org/jira/browse/IVYDE-56
Nicolas
Re: IvyDE cache location
Posted by Mitch Gitman <mg...@gmail.com>.
If you want to give each individual developer the ability to customize the
location of their Ivy cache, one approach is to add the following line to
your global ivysettings.xml:
<include file="${ivy.default.ivy.user.dir}/ivysettings.xml"/>
Then the user can define their own Ivy settings to their heart's content in
USER_HOME/.ivy2/ivysettings.xml. Note: if you do this, this local Ivy
settings file will be required.
Now, if your concern is doing ivy:cleancache for one project and then
cleaning out the whole cache, one way to do this is by establishing
namespaces for different projects. I don't have the details on that handy.
However, I don't quite see how clearing the Ivy cache would "crash the
building of other projects." If you do ivy:cleancache on one project or just
manually delete your USER_HOME/.ivy2/cache (or whatever you customize it
to), the only complication this should introduce is the extra wait that
comes with the re-downloading of the dependent modules on the next Ivy
resolve. So, unless I'm misunderstanding something, it kinda seems like
you're going to a lot of trouble to address a non-issue.
On Thu, Jan 15, 2009 at 9:25 PM, Yu, Aaron (IT) <Fa...@morganstanley.com>wrote:
> Hi,
>
> I have a problem of using Ivy and IvyDE.
> At my firm we are in the process of switching to Ivy for managing our
> java dependencies. We do not want thousands of developers to resolve
> dependencies in different ways so we provide a centralized , read-only
> ivy-settings.xml file which is used by all developers. However, this
> presents us with a problem. We would like to make it possible for our
> users to set their own cache locations, because if they use the same
> cache location for all projects, when they clean the cache in one
> project, the whole cache location will be cleaned, and this will crash
> the building of other projects.
>
> I know we can create a project-specific ivy-settings.xml file, and use
> 'defaultCacheDir', 'resolutionCacheDir' and 'repositoryCacheDir'
> attributes in 'caches' element to set the project-based cache location.
> But as I mentioned above, to make all the projects more manageable and
> shareable, we do not encourage developers to have the project-specific
> ivy setting file.
>
> To get around this we have implemented a new properties page for the
> ivyde plugin under the existing one called simply 'Cache'. This allows a
> user to choose the cache location. The choices are: Default
> ($HOME/.ivy2) and Workspace (workspace_dir/.metadata/.ivy2). We then
> reset the ivy.home System property and trigger re-resolution of all
> project ivy classpath containers when the users changes this setting
> (the change affects all projects in the workspace since ivy core reads
> the ivy.home System property to find out where it should do its
> caching). However, we are now wondering if what is really needed is a
> private cache area for each project in the workspace, for example:
> workspace_dir/.metadata/.ivy2/project_name. But this is not possible at
> the moment due to the way in which ivy core decided where to place its
> cache. (And, as we said, we do not want users to have to specify their
> own ivy-settings for each project for this reason alone.)
>
> Other people at our firm have also pointed out that it might be better
> if ivy core used a version specific cache location by default since
> there could be differences in cache metadata between versions. This
> would cause cache corruption if different versions of ivy core were run
> on the same system.
>
> All this seems to indicate that the subject of caching needs rethinking
> in ivy core and ivyde, and we would like to ask for some suggestions on
> this.
>
> Thanks!
>
> Aaron Yu
> Morgan Stanley | Technology
> 222 Yan An Road (East)
> 200002 Shanghai
> Phone: +86 21 2321-2106
> Fan.Yu@MorganStanley.com
> --------------------------------------------------------
>
> NOTICE: If received in error, please destroy and notify sender. Sender does
> not intend to waive confidentiality or privilege. Use of this email is
> prohibited when received in error.
>