You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@ant.apache.org by bo...@apache.org on 2009/09/23 16:54:18 UTC
svn commit: r818129 - in /ant/core/trunk: WHATSNEW
docs/manual/CoreTasks/get.html
src/main/org/apache/tools/ant/taskdefs/Get.java
Author: bodewig
Date: Wed Sep 23 14:54:18 2009
New Revision: 818129
URL: http://svn.apache.org/viewvc?rev=818129&view=rev
Log:
Make <get> support resource collections and mappers
Modified:
ant/core/trunk/WHATSNEW
ant/core/trunk/docs/manual/CoreTasks/get.html
ant/core/trunk/src/main/org/apache/tools/ant/taskdefs/Get.java
Modified: ant/core/trunk/WHATSNEW
URL: http://svn.apache.org/viewvc/ant/core/trunk/WHATSNEW?rev=818129&r1=818128&r2=818129&view=diff
==============================================================================
--- ant/core/trunk/WHATSNEW (original)
+++ ant/core/trunk/WHATSNEW Wed Sep 23 14:54:18 2009
@@ -985,6 +985,10 @@
makes it ignore differences between / and \ separators.
Bugzilla Report 47858.
+ * <get> now supports resource collections (as long as the resources
+ contained provide URLs) and can get multiple resources in a single
+ task.
+
Changes from Ant 1.7.0 TO Ant 1.7.1
=============================================
Modified: ant/core/trunk/docs/manual/CoreTasks/get.html
URL: http://svn.apache.org/viewvc/ant/core/trunk/docs/manual/CoreTasks/get.html?rev=818129&r1=818128&r2=818129&view=diff
==============================================================================
--- ant/core/trunk/docs/manual/CoreTasks/get.html (original)
+++ ant/core/trunk/docs/manual/CoreTasks/get.html Wed Sep 23 14:54:18 2009
@@ -26,31 +26,27 @@
<h2><a name="get">Get</a></h2>
<h3>Description</h3>
-<p>Gets a file from a URL. When the verbose option is "on", this task
+<p>Gets files from URLs. When the verbose option is "on", this task
displays a '.' for every 100 Kb retrieved. Any URL schema supported by
the runtime is valid here, including http:, ftp: and jar:;
-https: is only valid if the appropriate support is added to the pre-1.4 Java
-runtimes.
-
</p>
The <i>usetimestamp</i> option enables you to control downloads so that the remote file is
only fetched if newer than the local copy. If there is no local copy, the download always takes
-place. When a file is downloaded, the timestamp of the downloaded file is set to the remote timestamp,
-if the JVM is Java1.2 or later.
+place. When a file is downloaded, the timestamp of the downloaded file is set to the remote timestamp.
NB: This timestamp facility only works on downloads using the HTTP protocol.
<p>
A username and password can be specified, in which case basic 'slightly encoded
plain text' authentication is used. This is only secure over an HTTPS link.
</p>
-<p>
-<b>Proxies</b>. Since Ant1.7, Ant running on Java1.5 or later defaults to
- <a href="../proxy.html">using
- the proxy settings of the operating system</a>. There is also the
- <a href="../OptionalTasks/setproxy.html"><setproxy></a> task for
- earlier Java versions. With proxies turned on, <code><get></code> requests against
- localhost may not work as expected, if the request is relayed to the proxy.
- The <code>-noproxy</code> option can be used to turn this feature off.
-</p>
+
+<p><b>Proxies</b>. Since Ant 1.7.0, Ant running on Java1.5 or later can
+ <a href="../proxy.html">use the proxy settings of the operating
+ system</a> if enabled with the
+ <code>-autoproxy</code> option. There is also the
+ <a href="../OptionalTasks/setproxy.html"><setproxy></a> task
+ for earlier Java versions. With proxies turned
+ on, <code><get></code> requests against localhost may not work
+ as expected, if the request is relayed to the proxy.</p>
<h3>Parameters</h3>
<table border="1" cellpadding="2" cellspacing="0">
@@ -62,11 +58,12 @@
<tr>
<td valign="top">src</td>
<td valign="top">the URL from which to retrieve a file.</td>
- <td align="center" valign="top">Yes</td>
+ <td align="center" valign="top">Yes or a nested resource collection</td>
</tr>
<tr>
<td valign="top">dest</td>
- <td valign="top">the file where to store the retrieved file.</td>
+ <td valign="top">the file or directory where to store the
+ retrieved file(s).</td>
<td align="center" valign="top">Yes</td>
</tr>
<tr>
@@ -97,15 +94,15 @@
</tr>
<tr>
<td valign="top">maxtime</td>
- <td valign="top">Maximum time in seconds the download may take,
- otherwise it will be interrupted and treated like a download
+ <td valign="top">Maximum time in seconds a single download may take,
+ otherwise it will be interrupted and treated like a download
error. <em>Since Ant 1.8.0</em></td>
<td align="center" valign="top">No: default 0 which means no
maximum time</td>
</tr>
<tr>
<td valign="top">retries</td>
- <td valign="top">the number of retries on error<br/>
+ <td valign="top">the per download number of retries on error<br/>
<em>since Ant 1.8.0</em></td>
<td align="center" valign="top">No; default "3"</td>
</tr>
@@ -125,6 +122,29 @@
<td align="center" valign="top">No; default "true"</td>
</tr>
</table>
+<h3>Parameters specified as nested elements</h3>
+<h4>any resource collection</h4>
+
+<p><a href="../CoreTypes/resources.html#collection">Resource
+ Collection</a>s are used to select groups of URLs to download. If
+ the collection contains more than one resource, the dest attribute
+ must point to a directory if it exists or a directory will be
+ created if it doesn't exist. The destination file name use the
+ last part of the path of the source URL unless you also specify a
+ mapper.</p>
+
+<h4>mapper</h4>
+
+<p>You can define name transformations by using a
+ nested <a href="../CoreTypes/mapper.html">mapper</a> element. You
+ can also use any filenamemapper type in place of the mapper
+ element.</p>
+
+<p>The mapper will receive the resource's name as argument. Any
+ resource for which the mapper returns no or more than one mapped
+ name will be skipped. If the returned name is a relative path, it
+ will be considered relative to the <em>dest</em> attribute.</p>
+
<h3>Examples</h3>
<pre> <get src="http://ant.apache.org/" dest="help/index.html"/></pre>
<p>Gets the index page of http://ant.apache.org/, and stores it in the file <code>help/index.html</code>.</p>
@@ -175,6 +195,15 @@
checksum (assuming a certain naming convention for the checksum
file, of course) and validate the checksum on the fly.</p>
+<pre>
+<get dest="downloads">
+ <url url="http://ant.apache.org/index.html"/>
+ <url url="http://ant.apache.org/faq.html"/>
+</get>
+</pre>
+<p>Gets the index and FAQ pages of http://ant.apache.org/, and stores
+ them in the directory <code>downloads</code> which will be created if
+ necessary.</p>
</body>
</html>
Modified: ant/core/trunk/src/main/org/apache/tools/ant/taskdefs/Get.java
URL: http://svn.apache.org/viewvc/ant/core/trunk/src/main/org/apache/tools/ant/taskdefs/Get.java?rev=818129&r1=818128&r2=818129&view=diff
==============================================================================
--- ant/core/trunk/src/main/org/apache/tools/ant/taskdefs/Get.java (original)
+++ ant/core/trunk/src/main/org/apache/tools/ant/taskdefs/Get.java Wed Sep 23 14:54:18 2009
@@ -30,11 +30,19 @@
import java.net.URLConnection;
import java.util.Date;
import java.util.HashSet;
+import java.util.Iterator;
import java.util.Set;
import org.apache.tools.ant.BuildException;
import org.apache.tools.ant.Project;
import org.apache.tools.ant.Task;
+import org.apache.tools.ant.types.Mapper;
+import org.apache.tools.ant.types.Resource;
+import org.apache.tools.ant.types.ResourceCollection;
+import org.apache.tools.ant.types.resources.Resources;
+import org.apache.tools.ant.types.resources.URLProvider;
+import org.apache.tools.ant.types.resources.URLResource;
+import org.apache.tools.ant.util.FileNameMapper;
import org.apache.tools.ant.util.FileUtils;
/**
@@ -53,12 +61,12 @@
private static final int BIG_BUFFER_SIZE = 100 * 1024;
private static final FileUtils FILE_UTILS = FileUtils.getFileUtils();
private static final int REDIRECT_LIMIT = 25;
-
+
private static final String HTTP = "http";
private static final String HTTPS = "https";
- private URL source; // required
- private File dest; // required
+ private Resources sources = new Resources();
+ private File destination; // required
private boolean verbose = false;
private boolean useTimestamp = false; //off by default
private boolean ignoreErrors = false;
@@ -68,6 +76,7 @@
private int numberRetries = NUMBER_RETRIES;
private boolean skipExisting = false;
private boolean httpUseCaches = true; // on by default
+ private Mapper mapperElement = null;
/**
* Does the work.
@@ -75,6 +84,36 @@
* @exception BuildException Thrown in unrecoverable error.
*/
public void execute() throws BuildException {
+ checkAttributes();
+
+ for (Iterator iter = sources.iterator(); iter.hasNext(); ) {
+ Resource r = (Resource) iter.next();
+ URLProvider up = (URLProvider) r.as(URLProvider.class);
+ URL source = up.getURL();
+
+ File dest = destination;
+ if (destination.isDirectory()) {
+ if (mapperElement != null) {
+ String path = source.getPath();
+ if (path.endsWith("/")) {
+ path = path.substring(0, path.length() - 1);
+ }
+ int slash = path.lastIndexOf("/");
+ if (slash > -1) {
+ path = path.substring(slash + 1);
+ }
+ dest = new File(destination, path);
+ } else {
+ FileNameMapper mapper = mapperElement.getImplementation();
+ String[] d = mapper.mapFileName(r.getName());
+ if (d == null || d.length != 1) {
+ log("skipping " + r + " - mapper can't handle it",
+ Project.MSG_WARN);
+ continue;
+ }
+ dest = new File(destination, d[0]);
+ }
+ }
//set up logging
int logLevel = Project.MSG_INFO;
@@ -85,13 +124,14 @@
//execute the get
try {
- doGet(logLevel, progress);
+ doGet(source, dest, logLevel, progress);
} catch (IOException ioe) {
log("Error getting " + source + " to " + dest);
if (!ignoreErrors) {
throw new BuildException(ioe, getLocation());
}
}
+ }
}
/**
@@ -106,10 +146,41 @@
* @throws IOException for network trouble
* @throws BuildException for argument errors, or other trouble when ignoreErrors
* is false.
+ * @deprecated only gets the first configured resource
*/
public boolean doGet(int logLevel, DownloadProgress progress)
throws IOException {
checkAttributes();
+ for (Iterator iter = sources.iterator(); iter.hasNext(); ) {
+ Resource r = (Resource) iter.next();
+ URLProvider up = (URLProvider) r.as(URLProvider.class);
+ URL source = up.getURL();
+ return doGet(source, destination, logLevel, progress);
+ }
+ /*NOTREACHED*/
+ return false;
+ }
+
+ /**
+ * make a get request, with the supplied progress and logging info.
+ *
+ * All the other config parameters like ignoreErrors are set at
+ * the task level.
+ * @param source the URL to get
+ * @param dest the target file
+ * @param logLevel level to log at, see {@link Project#log(String, int)}
+ * @param progress progress callback; null for no-callbacks
+ * @return true for a successful download, false otherwise.
+ * The return value is only relevant when {@link #ignoreErrors} is true, as
+ * when false all failures raise BuildExceptions.
+ * @throws IOException for network trouble
+ * @throws BuildException for argument errors, or other trouble when ignoreErrors
+ * is false.
+ * @since Ant 1.8.0
+ */
+ public boolean doGet(URL source, File dest, int logLevel,
+ DownloadProgress progress)
+ throws IOException {
if (dest.exists() && skipExisting) {
log("Destination already exists (skipping): "
@@ -137,7 +208,8 @@
hasTimestamp = true;
}
- GetThread getThread = new GetThread(hasTimestamp, timestamp, progress,
+ GetThread getThread = new GetThread(source, dest,
+ hasTimestamp, timestamp, progress,
logLevel);
getThread.setDaemon(true);
getProject().registerThreadTask(getThread, this);
@@ -169,32 +241,55 @@
* Check the attributes.
*/
private void checkAttributes() {
- if (source == null) {
- throw new BuildException("src attribute is required", getLocation());
+ if (sources.size() == 0) {
+ throw new BuildException("at least one source is required",
+ getLocation());
+ }
+ for (Iterator iter = sources.iterator(); iter.hasNext(); ) {
+ Object up = ((Resource) iter.next()).as(URLProvider.class);
+ if (up == null) {
+ throw new BuildException("Only URLProvider resources are"
+ + " supported", getLocation());
+ }
}
- if (dest == null) {
+ if (destination == null) {
throw new BuildException("dest attribute is required", getLocation());
}
- if (dest.exists() && dest.isDirectory()) {
- throw new BuildException("The specified destination is a directory",
- getLocation());
+ if (destination.exists() && sources.size() > 1
+ && !destination.isDirectory()) {
+ throw new BuildException("The specified destination is not a"
+ + " directory",
+ getLocation());
}
- if (dest.exists() && !dest.canWrite()) {
- throw new BuildException("Can't write to " + dest.getAbsolutePath(),
- getLocation());
+ if (destination.exists() && !destination.canWrite()) {
+ throw new BuildException("Can't write to "
+ + destination.getAbsolutePath(),
+ getLocation());
+ }
+
+ if (sources.size() > 1 && !destination.exists()) {
+ destination.mkdirs();
}
}
/**
- * Set the URL to get.
+ * Set an URL to get.
*
* @param u URL for the file.
*/
public void setSrc(URL u) {
- this.source = u;
+ add(new URLResource(u));
+ }
+
+ /**
+ * Adds URLs to get.
+ * @since Ant 1.8.0
+ */
+ public void add(ResourceCollection rc) {
+ sources.add(rc);
}
/**
@@ -203,7 +298,7 @@
* @param dest Path to file.
*/
public void setDest(File dest) {
- this.dest = dest;
+ this.destination = dest;
}
/**
@@ -266,13 +361,6 @@
}
/**
- * Provide this for Backward Compatibility.
- */
- protected static class Base64Converter
- extends org.apache.tools.ant.util.Base64Converter {
- }
-
- /**
* The time in seconds the download is allowed to take before
* being terminated.
*
@@ -319,6 +407,37 @@
}
/**
+ * Define the mapper to map source to destination files.
+ * @return a mapper to be configured.
+ * @exception BuildException if more than one mapper is defined.
+ * @since Ant 1.8.0
+ */
+ public Mapper createMapper() throws BuildException {
+ if (mapperElement != null) {
+ throw new BuildException("Cannot define more than one mapper",
+ getLocation());
+ }
+ mapperElement = new Mapper(getProject());
+ return mapperElement;
+ }
+
+ /**
+ * Add a nested filenamemapper.
+ * @param fileNameMapper the mapper to add.
+ * @since Ant 1.8.0
+ */
+ public void add(FileNameMapper fileNameMapper) {
+ createMapper().add(fileNameMapper);
+ }
+
+ /**
+ * Provide this for Backward Compatibility.
+ */
+ protected static class Base64Converter
+ extends org.apache.tools.ant.util.Base64Converter {
+ }
+
+ /**
* Interface implemented for reporting
* progess of downloading.
*/
@@ -414,6 +533,8 @@
private class GetThread extends Thread {
+ private final URL source;
+ private final File dest;
private final boolean hasTimestamp;
private final long timestamp;
private final DownloadProgress progress;
@@ -427,7 +548,10 @@
private URLConnection connection;
private int redirections = 0;
- GetThread(boolean h, long t, DownloadProgress p, int l) {
+ GetThread(URL source, File dest,
+ boolean h, long t, DownloadProgress p, int l) {
+ this.source = source;
+ this.dest = dest;
hasTimestamp = h;
timestamp = t;
progress = p;
@@ -445,7 +569,7 @@
}
private boolean get() throws IOException, BuildException {
-
+
connection = openConnection(source);
if (connection == null)
@@ -461,7 +585,7 @@
if (downloadSucceeded && useTimestamp) {
updateTimeStamp();
}
-
+
return downloadSucceeded;
}
@@ -493,7 +617,7 @@
}
}
-
+
return true;
}
@@ -640,7 +764,7 @@
FILE_UTILS.setFileLastModified(dest, remoteTimestamp);
}
}
-
+
/**
* Has the download completed successfully?
*