You are viewing a plain text version of this content. The canonical link for it is here.
Posted to ivy-dev@incubator.apache.org by Xavier Hanin <xa...@gmail.com> on 2007/11/01 08:01:20 UTC
Re: How to halt on an unresolved dependency
On 10/30/07, John Rasmussen <jo...@gmail.com> wrote:
>
> I have extended org.apache.ivy.plugins.repository.AbstractRepository
> in order to build modules directly from source.
Interesting use case, feel free to share more about that with the community
if you can.
If any of the module/source contains compilation errors, I like the
> resolve process to stop.
>
> In method Resource getResource(String source) of AbstractRepository, I
> have tried calling fireTransferError, throwing an IOException and
> returning a Resource.exists = false. Thus Ivy knows the module is
> unresolved but Ivy continues resolving the next dependencies.
> Because resolving a dependency can involve building source, it gets
> time consuming to wait for the resolve process to complete all the
> dependencies - no errors are written until the process completes.
> That's why I want to stop when the first dependency fails.
> Well, one solution is to call System.exit, but like
> org.apache.ivy.util.Message() is not flushed, so this is not the best
> solution.
>
> Normally, Ivy should continue resolving dependencies (as it does), but
> in this case, I like a way to halt.
> IvyResolve.isHaltonfailure() is only read after all the dependencies
> have been resolved.
>
> I have tried Ivy 1.4.1 and 2.0.0-alpha2 but not found a way to halt on
> an unresolved dependency.
>
> Summary: Are there a "nice" way to stop the resolve process when
> failing to resolve a dependency?
Unfortunately not, Ivy resolution process tries to get as much information
as possible, like a compiler do. But I understand your use case, and since
you seems to have already scratched Ivy internals maybe opening a JIRA issue
and providing a patch to handle this would be easy enough for you. Maybe
you could add a new runtime exception, eg HaltIvyProcessException (a generic
name could be reused in other situation than resolution), and catch it
specifically to make sure it actually halts the resolution process in the
ResolveEngine.
Xavier
Regards
> John Rasmussen
>
--
Xavier Hanin - Independent Java Consultant
http://xhab.blogspot.com/
http://ant.apache.org/ivy/
http://www.xoocode.org/
Re: How to halt on an unresolved dependency
Posted by Xavier Hanin <xa...@gmail.com>.
On Nov 16, 2007 1:10 PM, John Rasmussen <jo...@gmail.com> wrote:
>
>
> Xavier Hanin wrote:
> >
> > On 10/30/07, John Rasmussen <jo...@gmail.com> wrote:
> >>
> >> I have extended org.apache.ivy.plugins.repository.AbstractRepository
> >> in order to build modules directly from source.
> >
> > Interesting use case, feel free to share more about that with the
> > community
> > if you can.
> >
> >
>
> The following might be useful.
Thanks a lot for sharing, with a lot of details, I'm pretty sure this will
help other users trying to do similar stuff!
Xavier
>
> Please notice that to build a module from source requires that all module
> are structured in a uniform way and that the implementation highly depends
> on that structure.
> Thus a lot of details regarding structure is skipped, because that depends
> on your structure.
>
> First of all, this will only work in Ivy 2.0+
> For Ivy 1.4.1, see https://issues.apache.org/jira/browse/IVY-448
>
> Some of the examples below is in "it works" state and not in "brilliant"
> state.
> Feel free to comment on that, if some can be changed or made more simple.
>
>
> First the ivysettings.xml is updated like:
> <ivysettings>
> <typedef name="my-resolver" classname="mypackage.MyResolver"/>
> <typedef name="my-latest-strategy"
> classname="mypackage.MyLatestStrategy"/>
> <latest-strategies>
> <my-latest-strategy/>
> </latest-strategies>
> <conflict-managers>
> <latest-cm name="my-latest-strategy" latest="my-latest-strategy"/>
> </conflict-managers>
> <resolvers>
> <my-resolver name="my-resolver" changingPattern="head"
> changingMatcher="exact"
> allownomd="false" checksums=""
> latest="my-latest-strategy">
> <ivy
>
> pattern="svn://myhost/[organisation]/[module]/[revision]/ivy-[module]-[revision].xml"/>
> <artifact
>
> pattern="svn://myhost/[organisation]/[module]/[revision]/[artifact]/[artifact]-[revision].[type]"/>
> </my-resolver>
> </resolvers>
> <modules>
> <module organisation="myorg" name=".*" resolver="my-resolver"
> conflict-manager="my-latest-strategy"/>
> </modules>
> </ivysettings>
>
>
>
> Two classes are required. The mypackage.MyResolver class builds the
> module,
> and the mypackage.MyLatestStrategy class evicts other revisions than the
> "build-from-source" revisions. In the ivysettings that revision is called
> 'head'.
>
> In order to load these two classes, the classes are inserted into
> myIvyPlugin.jar, and the retrieve target in ant looks like:
> Notice my.classpath should contain whatever you are using. I have module
> revisions stored in Subversion which are retrieved from the repository
> using
> svnkit, which has to be included in my.classpath.
> <path id="my.classpath">
> <pathelement location="myIvyPlugin.jar"/>
> <pathelement location="/path/to/ivy-2.0.0-alpha2-incubating.jar"/>
> </path>
>
> <taskdef name="ivy-retrieve" classname="org.apache.ivy.ant.IvyRetrieve"
> classpathref="my.classpath" loaderref="ivy.task.loader"/>
>
> <target name="retrieve">
> <ivy-retrieve/>
> </target>
>
>
>
> Now for the java stuff
>
> First MyResolver, which is simple:
> import org.apache.ivy.plugins.resolver.RepositoryResolver;
> public class MyResolver extends RepositoryResolver {
> public MyResolver() {
> setRepository(new MyRepository());
> }
>
> public String getTypeName() {
> return "myrepo";
> }
> }
>
>
> MyRepository is the next java class.
> Please notice, that the put method is unimplemented.
> I'm not using Ivy to stored revisions - historical reasons.
> Notice the uri cache, otherwise the build process is repeated a couple of
> times.
>
> import org.apache.ivy.core.module.descriptor.Artifact;
> import org.apache.ivy.plugins.repository.AbstractRepository;
> import org.apache.ivy.plugins.repository.Resource;
> import org.apache.ivy.plugins.repository.TransferEvent;
> import org.apache.ivy.util.Message;
>
> public final class MyRepository extends AbstractRepository {
> private final Map uriMap = new HashMap();
> public final void get(final String source, final File destination)
> throws IOException {
> try {
> final RepositoryResource repositoryResource =
> getRepositoryResource(source);
> fireTransferInitiated(repositoryResource,
> TransferEvent.REQUEST_GET);
> fireTransferStarted();
> repositoryResource.save(destination);
> fireTransferCompleted(repositoryResource.getContentLength());
> } catch (final MyException e) {
> Message.error("Cannot get " + source);
> fireTransferError(e);
> }
> }
>
> public final Resource getResource(final String source) throws
> IOException {
> try {
> final Resource resource = getRepositoryResource(source);
> return resource;
> } catch (final MyException e) {
> Message.error("Cannot get " + source);
> fireTransferError(e);
> throw new IOException("Failed");
> }
> }
>
> public final List list(final String source) throws IOException {
> try {
> final List list = getRepositoryResource(source).list();
> return list;
> } catch (final MyException e) {
> fireTransferError(e);
> return Collections.EMPTY_LIST;
> }
> }
>
> public final void put(final Artifact artifact, final File source, final
> String destination, final boolean overwrite) throws IOException {
> fireTransferError(new MyException("Put-operation not supported.
> Source " + source + ", destination " +
> destination));
> }
>
> private final RepositoryResource getRepositoryResource(final String
> uri)
> throws MyException {
> RepositoryResource repositoryResource = ( RepositoryResource )
> uriMap.get(uri);
> if (repositoryResource == null) {
> repositoryResource = MyResource(uri);
> uriMap.put(uri, repositoryResource);
> }
> return repositoryResource;
> }
> }
>
>
> A simple interface
> public interface RepositoryResource extends Resource {
> List list();
>
> void save(final File destination) throws IOException;
> }
>
>
> Given an uri in a format defined in ivysettings, the task is to build the
> module from source.
> This is done in MyResource, where all module structure details are placed.
> Thus this java class does not compile as is. You need to add your details
> or
> return "null" if not used.
> I have listed a few "null" methods.
> Notice, if a module build more than one artifact, then some caching/"need
> to
> update"
> functionality must be implemented somewhere to avoid that the system
> builds
> everything
> for each artifact.
> The files generated from the build is returned to ivy in the save() method
>
>
> public final class MyResource implements RepositoryResource {
> public MyResource(final String uri {
> // Build source
> }
>
> public final long getContentLength() {
> return getFile().length();
> }
>
> public final String getName() {
> return uri;
> }
>
> public final boolean isLocal() {
> return false;
> }
>
> public final List list() {
> return Collections.EMPTY_LIST;
> }
>
> public InputStream openStream() throws IOException {
> throw new IOException("openStream unsupported.");
> }
>
> public final void save(final File destination) throws IOException {
> final File source = getFile();
> final InputStream is = new FileInputStream(source);
> final OutputStream os = new FileOutputStream(destination);
> final byte[] b = new byte[BUFFER_SIZE];
> int size;
>
> while ((size = is.read(b)) != -1) {
> os.write(b, 0, size);
> }
>
> os.close();
> is.close();
> }
>
> private final File getFile() {
> // Return the file being build given the uri
> }
> }
>
>
> Other revisions than "head" should also be handled. That depends on how
> your
> revisions are stored.
> I have an svnkit implementation to retrieve from Subversion - historical
> reasons.
> Today such an implementation might already exists in Ivy??
>
> Most artifacts can be returned "as is".
> One exception is the ivy.xml artifact.
> The revision in Ivy.xml must correspond to the revision in the dependency
> tag, otherwise Ivy complains.
> Of course, this depends on how you return your ivy.xml file. (You might
> build/deliver an ivy.xml with correct version)
> I did a simple OutputStream replace:
> And use a replaced os in the save() method BUT only for the ivy.xml
> artifact!
> new StringReplaceOutputStream(os, " revision *= *\"[^\"]*\"",
> " revision=\"head\"");
>
> public final class StringReplaceOutputStream extends OutputStream {
> private int index;
> private final OutputStream outputStream;
> private final String regex;
> private final String replacement;
> private final StringBuffer strBuf;
>
> public StringReplaceOutputStream(final OutputStream outputStream, final
> String regex, final String replacement) {
> this.outputStream = outputStream;
> strBuf = new StringBuffer();
> this.regex = regex;
> this.replacement = replacement;
> index = 0;
> }
>
> public final void close() throws IOException {
> flush();
> outputStream.close();
> }
>
> public final synchronized void flush() throws IOException {
> writeFromStrBuf(strBuf.length());
> outputStream.flush();
> }
>
> public final void write(final int b) throws IOException {
> strBuf.append(( char ) (b & 0xFF));
> update();
> }
>
> public final void write(final byte[] b, final int off, final int len)
> throws IOException {
> strBuf.append(new String(b, off, len));
> update();
> }
>
> private final void update() throws IOException {
> while (index < (strBuf.length() - 1)) {
> if (strBuf.charAt(index) == 0x0D) {
> if (strBuf.charAt(index + 1) == 0x0A) {
> index++;
> }
>
> writeFromStrBuf(index + 1);
> } else {
> index++;
> }
> }
> }
>
> private final void writeFromStrBuf(final int index) throws IOException
> {
> final String str = strBuf.substring(0, index).toString();
> final String strRep = str.replaceAll(regex, replacement);
> outputStream.write(strRep.getBytes());
> strBuf.delete(0, index);
> this.index = 0;
> }
> }
>
>
> To evict other revisions than "head" add MyLatestStrategy.java
> import java.util.Comparator;
> import org.apache.ivy.plugins.latest.ArtifactInfo;
> import org.apache.ivy.plugins.latest.ComparatorLatestStrategy;
> import org.apache.ivy.plugins.latest.LatestRevisionStrategy;
>
>
> public class MyLatestStrategy extends ComparatorLatestStrategy {
> public MyLatestStrategy() {
> super(new Comparator() {
> private final Comparator comparator = new
> LatestRevisionStrategy().COMPARATOR;
>
> public int compare(Object o1, Object o2) {
> String rev1 = (( ArtifactInfo ) o1).getRevision();
> String rev2 = (( ArtifactInfo ) o2).getRevision();
>
> if (rev1.startsWith("head")) {
> return 1;
> } else if (rev2.startsWith("head")) {
> return -1;
> } else {
> return comparator.compare(o1, o2);
> }
> }
> });
> setName("my-latest-strategy");
> }
> }
>
> --
> View this message in context:
> http://www.nabble.com/How-to-halt-on-an-unresolved-dependency-tf4717519.html#a13791968
> Sent from the ivy-dev mailing list archive at Nabble.com.
>
>
--
Xavier Hanin - Independent Java Consultant
http://xhab.blogspot.com/
http://ant.apache.org/ivy/
http://www.xoocode.org/
Re: How to halt on an unresolved dependency
Posted by John Rasmussen <jo...@gmail.com>.
Xavier Hanin wrote:
>
> On 10/30/07, John Rasmussen <jo...@gmail.com> wrote:
>>
>> I have extended org.apache.ivy.plugins.repository.AbstractRepository
>> in order to build modules directly from source.
>
> Interesting use case, feel free to share more about that with the
> community
> if you can.
>
>
The following might be useful.
Please notice that to build a module from source requires that all module
are structured in a uniform way and that the implementation highly depends
on that structure.
Thus a lot of details regarding structure is skipped, because that depends
on your structure.
First of all, this will only work in Ivy 2.0+
For Ivy 1.4.1, see https://issues.apache.org/jira/browse/IVY-448
Some of the examples below is in "it works" state and not in "brilliant"
state.
Feel free to comment on that, if some can be changed or made more simple.
First the ivysettings.xml is updated like:
<ivysettings>
<typedef name="my-resolver" classname="mypackage.MyResolver"/>
<typedef name="my-latest-strategy"
classname="mypackage.MyLatestStrategy"/>
<latest-strategies>
<my-latest-strategy/>
</latest-strategies>
<conflict-managers>
<latest-cm name="my-latest-strategy" latest="my-latest-strategy"/>
</conflict-managers>
<resolvers>
<my-resolver name="my-resolver" changingPattern="head"
changingMatcher="exact"
allownomd="false" checksums=""
latest="my-latest-strategy">
<ivy
pattern="svn://myhost/[organisation]/[module]/[revision]/ivy-[module]-[revision].xml"/>
<artifact
pattern="svn://myhost/[organisation]/[module]/[revision]/[artifact]/[artifact]-[revision].[type]"/>
</my-resolver>
</resolvers>
<modules>
<module organisation="myorg" name=".*" resolver="my-resolver"
conflict-manager="my-latest-strategy"/>
</modules>
</ivysettings>
Two classes are required. The mypackage.MyResolver class builds the module,
and the mypackage.MyLatestStrategy class evicts other revisions than the
"build-from-source" revisions. In the ivysettings that revision is called
'head'.
In order to load these two classes, the classes are inserted into
myIvyPlugin.jar, and the retrieve target in ant looks like:
Notice my.classpath should contain whatever you are using. I have module
revisions stored in Subversion which are retrieved from the repository using
svnkit, which has to be included in my.classpath.
<path id="my.classpath">
<pathelement location="myIvyPlugin.jar"/>
<pathelement location="/path/to/ivy-2.0.0-alpha2-incubating.jar"/>
</path>
<taskdef name="ivy-retrieve" classname="org.apache.ivy.ant.IvyRetrieve"
classpathref="my.classpath" loaderref="ivy.task.loader"/>
<target name="retrieve">
<ivy-retrieve/>
</target>
Now for the java stuff
First MyResolver, which is simple:
import org.apache.ivy.plugins.resolver.RepositoryResolver;
public class MyResolver extends RepositoryResolver {
public MyResolver() {
setRepository(new MyRepository());
}
public String getTypeName() {
return "myrepo";
}
}
MyRepository is the next java class.
Please notice, that the put method is unimplemented.
I'm not using Ivy to stored revisions - historical reasons.
Notice the uri cache, otherwise the build process is repeated a couple of
times.
import org.apache.ivy.core.module.descriptor.Artifact;
import org.apache.ivy.plugins.repository.AbstractRepository;
import org.apache.ivy.plugins.repository.Resource;
import org.apache.ivy.plugins.repository.TransferEvent;
import org.apache.ivy.util.Message;
public final class MyRepository extends AbstractRepository {
private final Map uriMap = new HashMap();
public final void get(final String source, final File destination)
throws IOException {
try {
final RepositoryResource repositoryResource =
getRepositoryResource(source);
fireTransferInitiated(repositoryResource,
TransferEvent.REQUEST_GET);
fireTransferStarted();
repositoryResource.save(destination);
fireTransferCompleted(repositoryResource.getContentLength());
} catch (final MyException e) {
Message.error("Cannot get " + source);
fireTransferError(e);
}
}
public final Resource getResource(final String source) throws
IOException {
try {
final Resource resource = getRepositoryResource(source);
return resource;
} catch (final MyException e) {
Message.error("Cannot get " + source);
fireTransferError(e);
throw new IOException("Failed");
}
}
public final List list(final String source) throws IOException {
try {
final List list = getRepositoryResource(source).list();
return list;
} catch (final MyException e) {
fireTransferError(e);
return Collections.EMPTY_LIST;
}
}
public final void put(final Artifact artifact, final File source, final
String destination, final boolean overwrite) throws IOException {
fireTransferError(new MyException("Put-operation not supported.
Source " + source + ", destination " +
destination));
}
private final RepositoryResource getRepositoryResource(final String uri)
throws MyException {
RepositoryResource repositoryResource = ( RepositoryResource )
uriMap.get(uri);
if (repositoryResource == null) {
repositoryResource = MyResource(uri);
uriMap.put(uri, repositoryResource);
}
return repositoryResource;
}
}
A simple interface
public interface RepositoryResource extends Resource {
List list();
void save(final File destination) throws IOException;
}
Given an uri in a format defined in ivysettings, the task is to build the
module from source.
This is done in MyResource, where all module structure details are placed.
Thus this java class does not compile as is. You need to add your details or
return "null" if not used.
I have listed a few "null" methods.
Notice, if a module build more than one artifact, then some caching/"need to
update"
functionality must be implemented somewhere to avoid that the system builds
everything
for each artifact.
The files generated from the build is returned to ivy in the save() method
public final class MyResource implements RepositoryResource {
public MyResource(final String uri {
// Build source
}
public final long getContentLength() {
return getFile().length();
}
public final String getName() {
return uri;
}
public final boolean isLocal() {
return false;
}
public final List list() {
return Collections.EMPTY_LIST;
}
public InputStream openStream() throws IOException {
throw new IOException("openStream unsupported.");
}
public final void save(final File destination) throws IOException {
final File source = getFile();
final InputStream is = new FileInputStream(source);
final OutputStream os = new FileOutputStream(destination);
final byte[] b = new byte[BUFFER_SIZE];
int size;
while ((size = is.read(b)) != -1) {
os.write(b, 0, size);
}
os.close();
is.close();
}
private final File getFile() {
// Return the file being build given the uri
}
}
Other revisions than "head" should also be handled. That depends on how your
revisions are stored.
I have an svnkit implementation to retrieve from Subversion - historical
reasons.
Today such an implementation might already exists in Ivy??
Most artifacts can be returned "as is".
One exception is the ivy.xml artifact.
The revision in Ivy.xml must correspond to the revision in the dependency
tag, otherwise Ivy complains.
Of course, this depends on how you return your ivy.xml file. (You might
build/deliver an ivy.xml with correct version)
I did a simple OutputStream replace:
And use a replaced os in the save() method BUT only for the ivy.xml
artifact!
new StringReplaceOutputStream(os, " revision *= *\"[^\"]*\"",
" revision=\"head\"");
public final class StringReplaceOutputStream extends OutputStream {
private int index;
private final OutputStream outputStream;
private final String regex;
private final String replacement;
private final StringBuffer strBuf;
public StringReplaceOutputStream(final OutputStream outputStream, final
String regex, final String replacement) {
this.outputStream = outputStream;
strBuf = new StringBuffer();
this.regex = regex;
this.replacement = replacement;
index = 0;
}
public final void close() throws IOException {
flush();
outputStream.close();
}
public final synchronized void flush() throws IOException {
writeFromStrBuf(strBuf.length());
outputStream.flush();
}
public final void write(final int b) throws IOException {
strBuf.append(( char ) (b & 0xFF));
update();
}
public final void write(final byte[] b, final int off, final int len)
throws IOException {
strBuf.append(new String(b, off, len));
update();
}
private final void update() throws IOException {
while (index < (strBuf.length() - 1)) {
if (strBuf.charAt(index) == 0x0D) {
if (strBuf.charAt(index + 1) == 0x0A) {
index++;
}
writeFromStrBuf(index + 1);
} else {
index++;
}
}
}
private final void writeFromStrBuf(final int index) throws IOException {
final String str = strBuf.substring(0, index).toString();
final String strRep = str.replaceAll(regex, replacement);
outputStream.write(strRep.getBytes());
strBuf.delete(0, index);
this.index = 0;
}
}
To evict other revisions than "head" add MyLatestStrategy.java
import java.util.Comparator;
import org.apache.ivy.plugins.latest.ArtifactInfo;
import org.apache.ivy.plugins.latest.ComparatorLatestStrategy;
import org.apache.ivy.plugins.latest.LatestRevisionStrategy;
public class MyLatestStrategy extends ComparatorLatestStrategy {
public MyLatestStrategy() {
super(new Comparator() {
private final Comparator comparator = new
LatestRevisionStrategy().COMPARATOR;
public int compare(Object o1, Object o2) {
String rev1 = (( ArtifactInfo ) o1).getRevision();
String rev2 = (( ArtifactInfo ) o2).getRevision();
if (rev1.startsWith("head")) {
return 1;
} else if (rev2.startsWith("head")) {
return -1;
} else {
return comparator.compare(o1, o2);
}
}
});
setName("my-latest-strategy");
}
}
--
View this message in context: http://www.nabble.com/How-to-halt-on-an-unresolved-dependency-tf4717519.html#a13791968
Sent from the ivy-dev mailing list archive at Nabble.com.