You are viewing a plain text version of this content. The canonical link for it is here.
Posted to ivy-user@ant.apache.org by Jim Newsham <jn...@referentia.com> on 2008/12/19 03:24:14 UTC
how to import artifacts
I'd like to integrate ivy into our development process. In order to do so,
I would need to import all the existing dependencies (typically jars) that
our project currently uses, into a shared repository. I couldn't find this
anywhere in the documentation, so. what is the magical incantation to add a
single artifact into the repository? I would expect there to be a simple
command-line invocation where you pass a few parameters (org, module,
version, filename), and the file is added to the repository.
This seems to me a fundamental requirement to set up a repository, so I'm
rather surprised that it's not documented in some obvious place, and there's
no tutorial that explains it. I saw a tutorial that shows how to copy
artifacts from one repository to another, but that's worthless if what you
have doesn't live in any repository yet.
Thanks in advance,
Jim
RE: how to import artifacts
Posted by Jim Newsham <jn...@referentia.com>.
So I have to write a script just to import an artifact into a repository?
I'm still surprised by that. I would expect to be able to invoke a
command-line tool to do a simple import. I tried to figure out how to do
that with "java -jar ivy.jar" (using the online documentation, which is
particularly weak for this usage), but couldn't get anything working.
I was a bit overwhelmed with the tutorials, and somewhat confused by how all
of the ivy ant tasks work together. A document that describes this in
detail would be very helpful. For example, I tried the publish task, but it
complained that I needed to resolve something first. I read the
documentation on the publish and resolve tasks, but was still left a bit
confused.
I eventually got something working, so this is more or less feedback from a
neophyte on where the tutorials and documentation fall short. I think the
current tuturials are great, just couldn't find what I wanted to do. Thanks
also to those who provided scripts. The fact that people have to write such
complex scripts to do seemingly core tasks also indicates an area where the
tool is lacking.
Best Regards,
Jim Newsham
> -----Original Message-----
> From: Konstantin Rekk [mailto:konstantin.rekk@gbv.de]
> Sent: Friday, December 19, 2008 1:05 AM
> To: ivy-user@ant.apache.org
> Subject: Re: how to import artifacts
>
> Not sure, if I've got your point.
> But it seems to me that you are asking about the publish task:
>
> http://ant.apache.org/ivy/history/latest-milestone/use/publish.html
>
> Studying this tutorial (the ant build files, look for publish and
> publish local targets) might help you:
>
> http://ant.apache.org/ivy/history/latest-milestone/tutorial/multiple.html
>
> Konstantin
>
> >
> >
> > This seems to me a fundamental requirement to set up a repository, so
> I'm
> > rather surprised that it's not documented in some obvious place, and
> there's
> > no tutorial that explains it. I saw a tutorial that shows how to copy
> > artifacts from one repository to another, but that's worthless if what
> you
> > have doesn't live in any repository yet.
> >
> >
> >
> > Thanks in advance,
> >
> > Jim
> >
> >
> >
> >
> >
> >
>
>
Re: how to import artifacts
Posted by Konstantin Rekk <ko...@gbv.de>.
Not sure, if I've got your point.
But it seems to me that you are asking about the publish task:
http://ant.apache.org/ivy/history/latest-milestone/use/publish.html
Studying this tutorial (the ant build files, look for publish and
publish local targets) might help you:
http://ant.apache.org/ivy/history/latest-milestone/tutorial/multiple.html
Konstantin
>
>
> This seems to me a fundamental requirement to set up a repository, so I'm
> rather surprised that it's not documented in some obvious place, and there's
> no tutorial that explains it. I saw a tutorial that shows how to copy
> artifacts from one repository to another, but that's worthless if what you
> have doesn't live in any repository yet.
>
>
>
> Thanks in advance,
>
> Jim
>
>
>
>
>
>
RE: how to import artifacts
Posted by Jim Newsham <jn...@referentia.com>.
That's no help. Read my second paragraph. From what I can tell, "install"
copies artifacts from one repository to another, rather than actually
installing a new artifact.
Thanks,
Jim
> -----Original Message-----
> From: Daniel Becheanu [mailto:daniel.becheanu@gmail.com]
> Sent: Thursday, December 18, 2008 5:59 PM
> To: ivy-user@ant.apache.org
> Subject: Re: how to import artifacts
>
> Look at
> http://ant.apache.org/ivy/history/latest-milestone/use/install.html
>
> On Thu, Dec 18, 2008 at 8:24 PM, Jim Newsham <jn...@referentia.com>
> wrote:
> >
> >
> > I'd like to integrate ivy into our development process. In order to do
> so,
> > I would need to import all the existing dependencies (typically jars)
> that
> > our project currently uses, into a shared repository. I couldn't find
> this
> > anywhere in the documentation, so. what is the magical incantation to
> add a
> > single artifact into the repository? I would expect there to be a
> simple
> > command-line invocation where you pass a few parameters (org, module,
> > version, filename), and the file is added to the repository.
> >
> >
> >
> > This seems to me a fundamental requirement to set up a repository, so
> I'm
> > rather surprised that it's not documented in some obvious place, and
> there's
> > no tutorial that explains it. I saw a tutorial that shows how to copy
> > artifacts from one repository to another, but that's worthless if what
> you
> > have doesn't live in any repository yet.
> >
> >
> >
> > Thanks in advance,
> >
> > Jim
> >
> >
> >
> >
> >
> >
> >
> >
Re: how to import artifacts
Posted by Daniel Becheanu <da...@gmail.com>.
Look at
http://ant.apache.org/ivy/history/latest-milestone/use/install.html
On Thu, Dec 18, 2008 at 8:24 PM, Jim Newsham <jn...@referentia.com> wrote:
>
>
> I'd like to integrate ivy into our development process. In order to do so,
> I would need to import all the existing dependencies (typically jars) that
> our project currently uses, into a shared repository. I couldn't find this
> anywhere in the documentation, so. what is the magical incantation to add a
> single artifact into the repository? I would expect there to be a simple
> command-line invocation where you pass a few parameters (org, module,
> version, filename), and the file is added to the repository.
>
>
>
> This seems to me a fundamental requirement to set up a repository, so I'm
> rather surprised that it's not documented in some obvious place, and there's
> no tutorial that explains it. I saw a tutorial that shows how to copy
> artifacts from one repository to another, but that's worthless if what you
> have doesn't live in any repository yet.
>
>
>
> Thanks in advance,
>
> Jim
>
>
>
>
>
>
>
>
RE: how to import artifacts
Posted by Kirby Files <Ki...@masergy.com>.
Sorry, let's try that *with* an attachment.
---
Kirby Files
Software Architect
Masergy Communications
kfiles@masergy.com
-----Original Message-----
From: Kirby Files [mailto:Kirby.Files@masergy.com]
Sent: Thursday, December 18, 2008 11:27 PM
To: ivy-user@ant.apache.org
Subject: RE: how to import artifacts
I use this script:
--------------------------------------
#!/bin/csh
if ($1 == "") then
echo "Usage: installjar <org> <module> <revision> <path-to-file>"
else
/usr/local/ant/bin/ant -Dorg="$1" -Dmodule="$2" -Drev="$3"
-Dinstall.path="$4" $5 $6 $7 $8 $9
endif
--------------------------------------
With the following build target:
--------------------------------------
<target name="installjar"
depends="init-ivy"
description="--> install jar to masrep">
<ivy:install settingsRef="masergy.settings" organisation="${org}"
module="${module}" revision="${rev}" type="jar"
from="${localjar.resolver}" to="${to.resolver}" />
</target>
--------------------------------------
Where my build.properties defines these variables:
localjar.resolver=localjars
to.resolver=masrep_sftp
And my ivysettings.xml includes:
<resolvers>
<filesystem name="localjars">
<artifact pattern="${install.path}/[artifact]-[revision].[ext]" />
<artifact pattern="${install.path}/[artifact].[ext]" />
</filesystem>
</resolvers>
This script allows me to specify a jar on the local filesystem by its
name and the directory in which it resides (install.path), add the org,
and rev metadata, and publish it to one of my resolvers (I use SFTP with
a keyfile to push files up to our webserver; you may use a filesystem
resolver or webdav as a destination).
I also have a bigger hack of a perl script which attempts to read a
directory of jars and guess the org, modul, and rev (validating against
maven2), or prompt the user when it's unclear. See the attachment.
---
Kirby Files
Software Architect
Masergy Communications
kfiles@masergy.com
-----Original Message-----
From: Jim Newsham [mailto:jnewsham@referentia.com]
Sent: Thu 12/18/2008 9:24 PM
To: ivy-user@ant.apache.org
Subject: how to import artifacts
I'd like to integrate ivy into our development process. In order to do
so,
I would need to import all the existing dependencies (typically jars)
that
our project currently uses, into a shared repository. I couldn't find
this
anywhere in the documentation, so. what is the magical incantation to
add a
single artifact into the repository? I would expect there to be a
simple
command-line invocation where you pass a few parameters (org, module,
version, filename), and the file is added to the repository.
This seems to me a fundamental requirement to set up a repository, so
I'm
rather surprised that it's not documented in some obvious place, and
there's
no tutorial that explains it. I saw a tutorial that shows how to copy
artifacts from one repository to another, but that's worthless if what
you
have doesn't live in any repository yet.
Thanks in advance,
Jim
RE: how to import artifacts
Posted by Kirby Files <Ki...@masergy.com>.
I cannot defeat our mail server's virus-scanning attachment-killer, so
here's the script inline:
--------------------------------------
#!/usr/bin/perl
# Looks for jars in the incoming directory, attemps to parse the
# needed ivy info from the filename, and if successful, publishes
# the jar to the Masergy ivrep repository.
# If using linux with package mgmt, dependencies are:
# perl-libwww-perl
# perl-XML-Xpath
# perl-Archive-Zip
use LWP::UserAgent;
use XML::XPath;
use XML::XPath::XMLParser;
use Archive::Zip;
use strict qw(vars);
use vars qw($indir $ant $rep $pre $post);
$indir = "lib";
$ant = "/usr/local/ant/bin/ant";
$rep = "newjars";
$pre="<ivy:install settingsRef=\"masergy.settings\" organisation=\"";
$post=" from=\"\${from.resolver}\" to=\"\${to.resolver}\" />";
open (DIR, "ls -1 $indir|") || die "could not open $indir\n";
while (<DIR>) {
my ($org, $module, $jar, $ver);
next unless /jar/;
chomp;
my $file = $indir . "/" . $_;
$org=$_;
$module=$_;
$jar=$_;
$ver=$_;
$module=~s/^(.*)\.jar/$1/;
$module=~s/([^.]*)-\d.*/$1/;
$ver=~s/^[^.]*-([\d.]+.*)\.jar/$1/;
$ver="unknown" unless $ver =~ /\d/;
## Figure out the org
## Ask the user, if there's ambiguity
$org = &maven_search_org( $module );
if (-1 == $org) {
$org = &guess_org($module);
print "Org guessed from module name:\n $org\n";
$org = &classes_org($file);
print "Org guessed from jar package naming:\n $org\n"
unless (-1 == $org);
do {
$org = &get_user_org();
} while ($org =~ /^$/);
}
if ($ver eq "unknown" || ($module !~ /^\w+$/) || $org !~ /\w+/) {
print "Skipping jar $jar:\n";
print " $org / $module / $ver\n";
next;
} else {
&publish_jar($jar, $org, $module, $ver);
}
#&write_ivy_conf($jar, $org, $module, $ver);
}
## Use ivy::install to publish the jar via sftp to ivyrep
sub publish_jar
{
my ($jar, $org, $module, $ver) = @_;
print "Uploading $org / $module / $jar ... ";
my $resp = `$ant -Dorg=$org -Dmodule=$module -Drev=$ver
-Dinstall.path=./$indir 2>&1`;
if ($resp =~ /failed/i) {
print STDERR "Error uploading $jar:\n", $resp;
print "\n";
} else {
print "Done.\n";
}
}
## Find a directory in the jar to propose as the org
## searches for any dirs with two path elements
sub classes_org
{
my ($file) = @_;
my $somezip = Archive::Zip->new();
unless ( $somezip->read( $file ) == AZ_OK ) {
die "Error reading jarfile $file";
}
my @members = $somezip->members();
my @dirnames = ();
foreach (@members) {
if ($_->isDirectory()) {
my $fname = $_->fileName();
if ($fname =~ /^\w+\/\w+\/$/) {
$fname =~ s/^(\w+)\/(\w+)\/$/$1.$2/;
return $fname;
}
}
}
return -1;
}
## Ask the user for the best org name
sub get_user_org
{
print "Enter preferred org:> ";
my $line = <STDIN>;
chomp($line);
return $line;
}
## Make a guess about the org name from the module name
sub guess_org
{
my ($module) = @_;
my $org = $module;
$org=~s/spring.*/springframework/;
$org=~s/hiber.*/hibernate/;
$org=~s/standard.*/taglibs/;
$org=~s/jstl.*/taglibs/;
$org=~s/persistence.*/sun/;
$org=~s/jta.*/sun/;
$org=~s/servlet-api.*/tomcat/;
$org=~s/ojdbc.*/oracle/;
$org=~s/cglib.*/cglib/;
$org=~s/javassist.*/jboss/;
$org=~s/([^.]*)-\d.*jar/$1/;
$org=~s/(.*)\.jar/$1/;
return $org;
}
## Search the maven repo for a module matching this jar
## If found, return the module's organization
## if none, or more than one, are found, ask the user
sub maven_search_org
{
my ($module) = @_;
my $ua = LWP::UserAgent->new;
$ua->agent('Maven Repo Search');
my $service=
'http://maven.ozacc.com/search?format=xml&type=jar&keyword=';
my $url=URI->new($service . $module);
my $req=HTTP::Request->new;
$req->method('GET');
$req->uri($url);
my $resp = $ua->request($req)->content;
my $org = &parse_resp( $module, $resp );
return $org;
}
## Parse the XML response from the Maven search service
sub parse_resp
{
my ($module, $data) = @_;
my $xp = XML::XPath->new(xml => $data);
my $nodeset = $xp->find('//groupId'); # find all paragraphs
# Get rid of duplicates
my %uniq=();
foreach my $node ($nodeset->get_nodelist) {
$uniq{ $xp->getNodeText( $node ) } = 1;
}
my @keys = (sort keys %uniq);
if ($#keys == 0) {
return ($keys[0]);
} elsif ($#keys == -1) {
print "Found no module matching $module in Maven repo\n";
return -1;
} else {
print "Found multiple possible orgs for $module in Maven repo:\n";
foreach (@keys) {
print " $_\n";
}
return -1;
}
}
--------------------------------------
-----Original Message-----
From: Kirby Files [mailto:Kirby.Files@masergy.com]
Sent: Thursday, December 18, 2008 11:27 PM
To: ivy-user@ant.apache.org
Subject: RE: how to import artifacts
I use this script:
--------------------------------------
#!/bin/csh
if ($1 == "") then
echo "Usage: installjar <org> <module> <revision> <path-to-file>"
else
/usr/local/ant/bin/ant -Dorg="$1" -Dmodule="$2" -Drev="$3"
-Dinstall.path="$4" $5 $6 $7 $8 $9
endif
--------------------------------------
With the following build target:
--------------------------------------
<target name="installjar"
depends="init-ivy"
description="--> install jar to masrep">
<ivy:install settingsRef="masergy.settings" organisation="${org}"
module="${module}" revision="${rev}" type="jar"
from="${localjar.resolver}" to="${to.resolver}" />
</target>
--------------------------------------
Where my build.properties defines these variables:
localjar.resolver=localjars
to.resolver=masrep_sftp
And my ivysettings.xml includes:
<resolvers>
<filesystem name="localjars">
<artifact pattern="${install.path}/[artifact]-[revision].[ext]" />
<artifact pattern="${install.path}/[artifact].[ext]" />
</filesystem>
</resolvers>
This script allows me to specify a jar on the local filesystem by its
name and the directory in which it resides (install.path), add the org,
and rev metadata, and publish it to one of my resolvers (I use SFTP with
a keyfile to push files up to our webserver; you may use a filesystem
resolver or webdav as a destination).
I also have a bigger hack of a perl script which attempts to read a
directory of jars and guess the org, modul, and rev (validating against
maven2), or prompt the user when it's unclear. See the attachment.
---
Kirby Files
Software Architect
Masergy Communications
kfiles@masergy.com
-----Original Message-----
From: Jim Newsham [mailto:jnewsham@referentia.com]
Sent: Thu 12/18/2008 9:24 PM
To: ivy-user@ant.apache.org
Subject: how to import artifacts
I'd like to integrate ivy into our development process. In order to do
so,
I would need to import all the existing dependencies (typically jars)
that
our project currently uses, into a shared repository. I couldn't find
this
anywhere in the documentation, so. what is the magical incantation to
add a
single artifact into the repository? I would expect there to be a
simple
command-line invocation where you pass a few parameters (org, module,
version, filename), and the file is added to the repository.
This seems to me a fundamental requirement to set up a repository, so
I'm
rather surprised that it's not documented in some obvious place, and
there's
no tutorial that explains it. I saw a tutorial that shows how to copy
artifacts from one repository to another, but that's worthless if what
you
have doesn't live in any repository yet.
Thanks in advance,
Jim
RE: how to import artifacts
Posted by Kirby Files <Ki...@masergy.com>.
I use this script:
--------------------------------------
#!/bin/csh
if ($1 == "") then
echo "Usage: installjar <org> <module> <revision> <path-to-file>"
else
/usr/local/ant/bin/ant -Dorg="$1" -Dmodule="$2" -Drev="$3" -Dinstall.path="$4" $5 $6 $7 $8 $9
endif
--------------------------------------
With the following build target:
--------------------------------------
<target name="installjar"
depends="init-ivy"
description="--> install jar to masrep">
<ivy:install settingsRef="masergy.settings" organisation="${org}" module="${module}" revision="${rev}" type="jar" from="${localjar.resolver}" to="${to.resolver}" />
</target>
--------------------------------------
Where my build.properties defines these variables:
localjar.resolver=localjars
to.resolver=masrep_sftp
And my ivysettings.xml includes:
<resolvers>
<filesystem name="localjars">
<artifact pattern="${install.path}/[artifact]-[revision].[ext]" />
<artifact pattern="${install.path}/[artifact].[ext]" />
</filesystem>
</resolvers>
This script allows me to specify a jar on the local filesystem by its name and the directory in which it resides (install.path), add the org, and rev metadata, and publish it to one of my resolvers (I use SFTP with a keyfile to push files up to our webserver; you may use a filesystem resolver or webdav as a destination).
I also have a bigger hack of a perl script which attempts to read a directory of jars and guess the org, modul, and rev (validating against maven2), or prompt the user when it's unclear. See the attachment.
---
Kirby Files
Software Architect
Masergy Communications
kfiles@masergy.com
-----Original Message-----
From: Jim Newsham [mailto:jnewsham@referentia.com]
Sent: Thu 12/18/2008 9:24 PM
To: ivy-user@ant.apache.org
Subject: how to import artifacts
I'd like to integrate ivy into our development process. In order to do so,
I would need to import all the existing dependencies (typically jars) that
our project currently uses, into a shared repository. I couldn't find this
anywhere in the documentation, so. what is the magical incantation to add a
single artifact into the repository? I would expect there to be a simple
command-line invocation where you pass a few parameters (org, module,
version, filename), and the file is added to the repository.
This seems to me a fundamental requirement to set up a repository, so I'm
rather surprised that it's not documented in some obvious place, and there's
no tutorial that explains it. I saw a tutorial that shows how to copy
artifacts from one repository to another, but that's worthless if what you
have doesn't live in any repository yet.
Thanks in advance,
Jim