You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ant.apache.org by Carl Myers <cm...@palantir.com> on 2010/08/14 02:23:00 UTC
Publishing non-jar artifacts
Hey all,
We use ivy for dependency management but now we need to depend on things
that are not jars (as an example, a java keystore, a text file
containing a license, a .sql file describing a db schema).
Currently we have a custom build system in ant which leverages ivy ant
tasks, but we also have developers using IvyDE in Eclipse. I want to be
able to publish, and consume, non-jar dependencies in both places. This
means Ivy would need to be able to "unbundle" the contents of a
jar/zip/whatever to a specific location on resolve.
I know that ant hooks exist to do things like this, but will they work
in IvyDE? Is that the way to go? If this does require dev work, how
would you expect it to be done to be most likely to be accepted as a patch?
Thanks!
--
Carl Myers
Palantir Technologies | Internal Tools Software Engineer
cmyers@palantir.com
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@ant.apache.org
For additional commands, e-mail: dev-help@ant.apache.org
Re: Publishing non-jar artifacts
Posted by Carl Myers <cm...@palantir.com>.
Nicolas - Thanks, this info is very helpful. I will investigate and be
sure to let you know if I have any feedback.
-Carl
On 08/24/2010 02:03 AM, Nicolas Lalevée wrote:
>
> Le 24 août 2010 à 00:47, Carl Myers a écrit :
>
>> The "common and reusable" pattern is "I have a mess of files". Why does it need to be more specific?
>>
>> Some examples:
>>
>> 1. Our custom build system consists of several xml and properties files. We would like to version our build system itself so that just a small ant script is used to pull the build system, then it is used to perform the rest of the build. We would like to have the build system cached, then extracted to a certain location, after an ivy resolve, but this must work within the eclipse workflow and the CLI workflow (but, I suppose, either way it will be the dev. running an ant script probably, so maybe IvyDE doesn't need to support it explicitly)
>
> quite off topic, but this is amazingly exactly what I want to put in place for my projects. I think Easyant can help with that, but it is not yet there, so my recent long mail on easyant dev mailing list.
>
>>
>> 2. Sql files, or other generated artifacts that are not Jars. I have a directory full of Sql files that several packages need to depend upon. What I really want to do is "build" these sql files (run tests, validate them), then publish them using ivy. But when other things depend on them, they need the files in a certain location, not on their classpath. I want to be able to use a construct where I say "resolve this package and place its contents here".
>>
>> 3. Configuration only packages. This makes sense once your codebase gets big enough - and similar to #2 above, you want to have a bunch of xml or properties files or whatever and you want them to end up in a location on disk, not in the ether.
>>
>> I think this is a very general (and useful) case for Ivy to solve.
>
> So you need IvyDE to launch a resolve of some ivy.xml file, and then retrieve the appropriate files in some defined place. IvyDE can already do that:
> http://ant.apache.org/ivy/ivyde/history/latest-milestone/cpc/retrieve.html
>
> As I wrote, it is tied to the Java nature of a project, and an IvyDE classpath container has to exist even if you don't use it. It should be useable though.
>
> But for IvyDE 2.2, I have just implemented a retrieve mechanism for non java project. So you would be able to right click on a project, select the Ivy menu and then select "Retrieve 'sql files'". And with Clint Burghduff patch I integrated too, the files get properly refreshed in Eclipse. I wouldn't consider the 2.2 as production ready, but an early feedback on that would be appreciated.
>
> Nicolas
>
>
>>
>> -Carl
>>
>> On 08/19/2010 01:00 PM, Nicolas Lalevée wrote:
>>> pe, Ivy doesn't care. I have been able to make Ivy manage dependencies between flex projects. IvyDE on his side is mainly intended to be used in a Java projects (probably too tied to Java,
>>
>> --
>> Carl Myers
>> Palantir Technologies | Internal Tools Software Engineer
>> cmyers@palantir.com
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@ant.apache.org
>> For additional commands, e-mail: dev-help@ant.apache.org
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@ant.apache.org
> For additional commands, e-mail: dev-help@ant.apache.org
>
--
Carl Myers
Palantir Technologies | Internal Tools Software Engineer
cmyers@palantir.com
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@ant.apache.org
For additional commands, e-mail: dev-help@ant.apache.org
Re: Publishing non-jar artifacts
Posted by Nicolas Lalevée <ni...@hibnet.org>.
Le 24 août 2010 à 00:47, Carl Myers a écrit :
> The "common and reusable" pattern is "I have a mess of files". Why does it need to be more specific?
>
> Some examples:
>
> 1. Our custom build system consists of several xml and properties files. We would like to version our build system itself so that just a small ant script is used to pull the build system, then it is used to perform the rest of the build. We would like to have the build system cached, then extracted to a certain location, after an ivy resolve, but this must work within the eclipse workflow and the CLI workflow (but, I suppose, either way it will be the dev. running an ant script probably, so maybe IvyDE doesn't need to support it explicitly)
quite off topic, but this is amazingly exactly what I want to put in place for my projects. I think Easyant can help with that, but it is not yet there, so my recent long mail on easyant dev mailing list.
>
> 2. Sql files, or other generated artifacts that are not Jars. I have a directory full of Sql files that several packages need to depend upon. What I really want to do is "build" these sql files (run tests, validate them), then publish them using ivy. But when other things depend on them, they need the files in a certain location, not on their classpath. I want to be able to use a construct where I say "resolve this package and place its contents here".
>
> 3. Configuration only packages. This makes sense once your codebase gets big enough - and similar to #2 above, you want to have a bunch of xml or properties files or whatever and you want them to end up in a location on disk, not in the ether.
>
> I think this is a very general (and useful) case for Ivy to solve.
So you need IvyDE to launch a resolve of some ivy.xml file, and then retrieve the appropriate files in some defined place. IvyDE can already do that:
http://ant.apache.org/ivy/ivyde/history/latest-milestone/cpc/retrieve.html
As I wrote, it is tied to the Java nature of a project, and an IvyDE classpath container has to exist even if you don't use it. It should be useable though.
But for IvyDE 2.2, I have just implemented a retrieve mechanism for non java project. So you would be able to right click on a project, select the Ivy menu and then select "Retrieve 'sql files'". And with Clint Burghduff patch I integrated too, the files get properly refreshed in Eclipse. I wouldn't consider the 2.2 as production ready, but an early feedback on that would be appreciated.
Nicolas
>
> -Carl
>
> On 08/19/2010 01:00 PM, Nicolas Lalevée wrote:
>> pe, Ivy doesn't care. I have been able to make Ivy manage dependencies between flex projects. IvyDE on his side is mainly intended to be used in a Java projects (probably too tied to Java,
>
> --
> Carl Myers
> Palantir Technologies | Internal Tools Software Engineer
> cmyers@palantir.com
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@ant.apache.org
> For additional commands, e-mail: dev-help@ant.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@ant.apache.org
For additional commands, e-mail: dev-help@ant.apache.org
Re: Publishing non-jar artifacts
Posted by Carl Myers <cm...@palantir.com>.
The "common and reusable" pattern is "I have a mess of files". Why does
it need to be more specific?
Some examples:
1. Our custom build system consists of several xml and properties files.
We would like to version our build system itself so that just a small
ant script is used to pull the build system, then it is used to perform
the rest of the build. We would like to have the build system cached,
then extracted to a certain location, after an ivy resolve, but this
must work within the eclipse workflow and the CLI workflow (but, I
suppose, either way it will be the dev. running an ant script probably,
so maybe IvyDE doesn't need to support it explicitly)
2. Sql files, or other generated artifacts that are not Jars. I have a
directory full of Sql files that several packages need to depend upon.
What I really want to do is "build" these sql files (run tests, validate
them), then publish them using ivy. But when other things depend on
them, they need the files in a certain location, not on their classpath.
I want to be able to use a construct where I say "resolve this package
and place its contents here".
3. Configuration only packages. This makes sense once your codebase
gets big enough - and similar to #2 above, you want to have a bunch of
xml or properties files or whatever and you want them to end up in a
location on disk, not in the ether.
I think this is a very general (and useful) case for Ivy to solve.
-Carl
On 08/19/2010 01:00 PM, Nicolas Lalevée wrote:
> pe, Ivy doesn't care. I have been able to make Ivy manage dependencies between flex projects. IvyDE on his side is mainly intended to be used in a Java projects (probably too tied to Java,
--
Carl Myers
Palantir Technologies | Internal Tools Software Engineer
cmyers@palantir.com
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@ant.apache.org
For additional commands, e-mail: dev-help@ant.apache.org
Re: Publishing non-jar artifacts
Posted by Nicolas Lalevée <ni...@hibnet.org>.
Le 19 août 2010 à 03:15, Carl Myers a écrit :
> Thanks Jason, that info is very helpful.
>
> The reason why this is appropriate for the dev list is, I think, I am asking "shouldn't Ivy/IvyDE do this natively?"
>
> I guess you could call this a feature request (which I would be willing to implement myself) to make Ivy better support non-jars. I don't want myself (and others) to have to write stuff in our build scripts to handle this if it could be done by ivy itself. So the discussion I want to start is, "is there a reason why this isn't already included? are people deadset against supporting non-jar dependencies?"
>
> Many java projects depend upon things which are not jars, so it seems like it should be innate functionality, not something for which "a workaround happens to exist, with some work".
I think Ivy and IvyDE already do support non-jar dependencies, as explained Jason. Ivy basically manage artifacts of any type, Ivy doesn't care. I have been able to make Ivy manage dependencies between flex projects. IvyDE on his side is mainly intended to be used in a Java projects (probably too tied to Java, I'll try to improve this soon), so it tries to managed artifacts that it can do something about it: jars to be added to some classpath, attach sources and javadocs. Other than that it cannot do much than Ivy does, resolve and retrieve in some place.
The problem about non well known typed artifact, is that they are unknown, then we don't really know what to do about them.
What exactly are your non-jar artifacts, and what do you do with them ? Is there really a common and reusable pattern, then yes we can add this feature to IvyDE.
Nicolas
> Thoughts?
>
> Thanks!
> -Carl
>
> On 08/15/2010 02:28 PM, Jason Trump wrote:
>> Hi Carl,
>>
>> Not to nit-pick, but this is probably better a question for the users
>> mailing list than the dev mailing list.
>>
>> I haven't tried this myself, but here's an idea:
>>
>> 1) You can configure IvyDE to retrieve your dependencies into the
>> workspace on resolve. It might be useful to create a special Ivy
>> configuration like 'ide' for these extra dependencies so that they go
>> into 'lib/ide/**'.
>> 2) You could then add an Ant Builder to your project, which executes a
>> specified ant target whenever the 'lib/ide' folder changes. The Ant
>> Builder can also be configured to refresh the target directories in your
>> eclipse workspace where the resources have been unpacked.
>>
>> here's some aging-but-still-relevant eclipse docs on setting up Ant
>> Builder:
>> http://help.eclipse.org/ganymede/index.jsp?topic=/org.eclipse.platform.doc.user/gettingStarted/qs-93_project_builder.htm
>>
>>
>> HTH
>> jason
>>
>> On Fri, Aug 13, 2010 at 5:23 PM, Carl Myers <cmyers@palantir.com
>> <ma...@palantir.com>> wrote:
>>
>> Hey all,
>>
>> We use ivy for dependency management but now we need to depend on
>> things that are not jars (as an example, a java keystore, a text
>> file containing a license, a .sql file describing a db schema).
>>
>> Currently we have a custom build system in ant which leverages ivy
>> ant tasks, but we also have developers using IvyDE in Eclipse. I
>> want to be able to publish, and consume, non-jar dependencies in
>> both places. This means Ivy would need to be able to "unbundle" the
>> contents of a jar/zip/whatever to a specific location on resolve.
>>
>> I know that ant hooks exist to do things like this, but will they
>> work in IvyDE? Is that the way to go? If this does require dev
>> work, how would you expect it to be done to be most likely to be
>> accepted as a patch?
>>
>> Thanks!
>> --
>> Carl Myers
>> Palantir Technologies | Internal Tools Software Engineer
>> cmyers@palantir.com <ma...@palantir.com>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@ant.apache.org
>> <ma...@ant.apache.org>
>> For additional commands, e-mail: dev-help@ant.apache.org
>> <ma...@ant.apache.org>
>>
>>
>
> --
> Carl Myers
> Palantir Technologies | Internal Tools Software Engineer
> cmyers@palantir.com
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@ant.apache.org
> For additional commands, e-mail: dev-help@ant.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@ant.apache.org
For additional commands, e-mail: dev-help@ant.apache.org
Re: Publishing non-jar artifacts
Posted by Carl Myers <cm...@palantir.com>.
Thanks Jason, that info is very helpful.
The reason why this is appropriate for the dev list is, I think, I am
asking "shouldn't Ivy/IvyDE do this natively?"
I guess you could call this a feature request (which I would be willing
to implement myself) to make Ivy better support non-jars. I don't want
myself (and others) to have to write stuff in our build scripts to
handle this if it could be done by ivy itself. So the discussion I want
to start is, "is there a reason why this isn't already included? are
people deadset against supporting non-jar dependencies?"
Many java projects depend upon things which are not jars, so it seems
like it should be innate functionality, not something for which "a
workaround happens to exist, with some work".
Thoughts?
Thanks!
-Carl
On 08/15/2010 02:28 PM, Jason Trump wrote:
> Hi Carl,
>
> Not to nit-pick, but this is probably better a question for the users
> mailing list than the dev mailing list.
>
> I haven't tried this myself, but here's an idea:
>
> 1) You can configure IvyDE to retrieve your dependencies into the
> workspace on resolve. It might be useful to create a special Ivy
> configuration like 'ide' for these extra dependencies so that they go
> into 'lib/ide/**'.
> 2) You could then add an Ant Builder to your project, which executes a
> specified ant target whenever the 'lib/ide' folder changes. The Ant
> Builder can also be configured to refresh the target directories in your
> eclipse workspace where the resources have been unpacked.
>
> here's some aging-but-still-relevant eclipse docs on setting up Ant
> Builder:
> http://help.eclipse.org/ganymede/index.jsp?topic=/org.eclipse.platform.doc.user/gettingStarted/qs-93_project_builder.htm
>
>
> HTH
> jason
>
> On Fri, Aug 13, 2010 at 5:23 PM, Carl Myers <cmyers@palantir.com
> <ma...@palantir.com>> wrote:
>
> Hey all,
>
> We use ivy for dependency management but now we need to depend on
> things that are not jars (as an example, a java keystore, a text
> file containing a license, a .sql file describing a db schema).
>
> Currently we have a custom build system in ant which leverages ivy
> ant tasks, but we also have developers using IvyDE in Eclipse. I
> want to be able to publish, and consume, non-jar dependencies in
> both places. This means Ivy would need to be able to "unbundle" the
> contents of a jar/zip/whatever to a specific location on resolve.
>
> I know that ant hooks exist to do things like this, but will they
> work in IvyDE? Is that the way to go? If this does require dev
> work, how would you expect it to be done to be most likely to be
> accepted as a patch?
>
> Thanks!
> --
> Carl Myers
> Palantir Technologies | Internal Tools Software Engineer
> cmyers@palantir.com <ma...@palantir.com>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@ant.apache.org
> <ma...@ant.apache.org>
> For additional commands, e-mail: dev-help@ant.apache.org
> <ma...@ant.apache.org>
>
>
--
Carl Myers
Palantir Technologies | Internal Tools Software Engineer
cmyers@palantir.com
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@ant.apache.org
For additional commands, e-mail: dev-help@ant.apache.org
Re: Publishing non-jar artifacts
Posted by Jason Trump <tr...@gmail.com>.
Hi Carl,
Not to nit-pick, but this is probably better a question for the users
mailing list than the dev mailing list.
I haven't tried this myself, but here's an idea:
1) You can configure IvyDE to retrieve your dependencies into the workspace
on resolve. It might be useful to create a special Ivy configuration like
'ide' for these extra dependencies so that they go into 'lib/ide/**'.
2) You could then add an Ant Builder to your project, which executes a
specified ant target whenever the 'lib/ide' folder changes. The Ant Builder
can also be configured to refresh the target directories in your eclipse
workspace where the resources have been unpacked.
here's some aging-but-still-relevant eclipse docs on setting up Ant Builder:
http://help.eclipse.org/ganymede/index.jsp?topic=/org.eclipse.platform.doc.user/gettingStarted/qs-93_project_builder.htm
HTH
jason
On Fri, Aug 13, 2010 at 5:23 PM, Carl Myers <cm...@palantir.com> wrote:
> Hey all,
>
> We use ivy for dependency management but now we need to depend on things
> that are not jars (as an example, a java keystore, a text file containing a
> license, a .sql file describing a db schema).
>
> Currently we have a custom build system in ant which leverages ivy ant
> tasks, but we also have developers using IvyDE in Eclipse. I want to be
> able to publish, and consume, non-jar dependencies in both places. This
> means Ivy would need to be able to "unbundle" the contents of a
> jar/zip/whatever to a specific location on resolve.
>
> I know that ant hooks exist to do things like this, but will they work in
> IvyDE? Is that the way to go? If this does require dev work, how would you
> expect it to be done to be most likely to be accepted as a patch?
>
> Thanks!
> --
> Carl Myers
> Palantir Technologies | Internal Tools Software Engineer
> cmyers@palantir.com
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@ant.apache.org
> For additional commands, e-mail: dev-help@ant.apache.org
>
>