You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ant.apache.org by Emma Burrows <Em...@rpharms.com> on 2011/11/29 10:23:54 UTC
Use Ant with an arbitrary date
As part of a content migration project, I am building content into a CMS on a weekly basis, and I use an Ant script to copy the content files to the build directory. Up until now, we've been wiping the CMS and reloading the whole 17,000-file set every time, which takes about 1.5 hours. But now that the content and the CMS customisations are more stable, we'd like to only upload the content files that have been modified since the previous week.
I can copy files modified since the last time I ran the Ant script using the <modified> selector:
<copy todir="changed" failonerror="no">
<fileset dir="output" includes="*.*">
<modified/>
</fileset>
</copy>
Which works very nicely. However, I would like to be able to load the files that have been modified since the last CMS build that took place on the server, typically on the previous Thursday.
So I was wondering if there was some way of using <modified>'s cache-based approach to only copy the files that have been modified since a given date/time like "17.00 last Thursday" instead of "last time this script was run".
Having researched the question, I'm guessing some kind of custom task might be required, but I'm hoping someone might have already invented that particular wheel. ;)
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@ant.apache.org
For additional commands, e-mail: user-help@ant.apache.org
RE: Use Ant with an arbitrary date
Posted by Emma Burrows <Em...@rpharms.com>.
From: Stefan Bodewig [mailto:bodewig@apache.org]
> You could copy the files that represent a version to some cache
> directory and use the different selector to select the files that are
> different from the ones inside the cache.
<snip>
> Another approach more in line with your original one is to explicitly
> set update to false on your modified selector during normal builds and
> set it to true when you actually deploy the files.
Thank you very much for those suggestions. The variations on <modified> that you suggest are exactly what I was looking for and should work well - provided I can trust myself not to accidentally run a deployment build mid-week! :*) On the other hand, the first approach sounds pretty safe since it just involves comparing the files to the existing archive. Either way, it sounds as if I'll be able to save my colleagues some build time by issuing content "patches".
Thanks,
Emma Burrows
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@ant.apache.org
For additional commands, e-mail: user-help@ant.apache.org
RE: Use Ant with an arbitrary date
Posted by Emma Burrows <Em...@rpharms.com>.
> Ant 1.7.1 has a known bug
Thanks, I'm glad to know it wasn't just me. :) I probably can update Ant, but as it's also part of our CMS, I'm nervous about writing scripts against a different version - so I have implemented your suggestion first and I'm glad to confirm that it's working now. :)
Thanks,
Emma
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@ant.apache.org
For additional commands, e-mail: user-help@ant.apache.org
Re: Use Ant with an arbitrary date
Posted by Stefan Bodewig <bo...@apache.org>.
On 2011-12-08, Emma Burrows wrote:
> Apologies for revisiting this issue a week later.
Don't worry.
>> Another approach more in line with your original one is to explicitly
>> set update to false on your modified selector during normal builds and
>> set it to true when you actually deploy the files.
> I don't seem to be able to get this to work. I am using Ant 1.7.1. and
> I have the following code:
> <copy todir="\content\Test\" includeEmptyDirs="false" failonerror="no">
> <fileset dir="../Output">
> <modified update="false" />
> </fileset>
> </copy>
Ant 1.7.1 has a known bug
<https://issues.apache.org/bugzilla/show_bug.cgi?id=32597>
If you can't upgrade Ant, this should work
<copy todir="\content\Test\" includeEmptyDirs="false" failonerror="no">
<fileset dir="../Output">
<modified>
<param name="update" value="false"/>
</modified>
</fileset>
</copy>
Stefan
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@ant.apache.org
For additional commands, e-mail: user-help@ant.apache.org
RE: Use Ant with an arbitrary date
Posted by Emma Burrows <Em...@rpharms.com>.
Apologies for revisiting this issue a week later. I've finally had a minute to test the following option you suggested:
> Another approach more in line with your original one is to explicitly
> set update to false on your modified selector during normal builds and
> set it to true when you actually deploy the files.
I don't seem to be able to get this to work. I am using Ant 1.7.1. and I have the following code:
<copy todir="\content\Test\" includeEmptyDirs="false" failonerror="no">
<fileset dir="../Output">
<modified update="false" />
</fileset>
</copy>
If I understand the process correctly, the cache.properties file that is automatically created shouldn't be updated when this attribute is set. But it is still updated every time there are modified files. Should I be doing something more to get this working?
Cheers,
Emma Burrows
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@ant.apache.org
For additional commands, e-mail: user-help@ant.apache.org
Re: Use Ant with an arbitrary date
Posted by Stefan Bodewig <bo...@apache.org>.
On 2011-11-29, Emma Burrows wrote:
> Thank you for the response. Sorry, I should probably have anticipated
> that response by expanding more thoroughly on exactly what happens. :)
8-)
> Actually, come to think of it, the date is probably a bit of a red
> herring. I suppose what I really need is some process to say "this is
> the last 'build version'" (a cache of checksums collected at a given
> point in time for instance?), and then to say "copy over only the
> files which aren't identical to the last 'build version'". This would
> also allow for us having to re-run the CMS build more than once during
> the week but still load all the files that changed since the last
> milestone build as it were.
You could copy the files that represent a version to some cache
directory and use the different selector to select the files that are
different from the ones inside the cache. The main difference between
this approach and the modified selector would be the disk space required
(full files rather than checksums are kept) but the speed is likely
similar (you need byte-by-byte comparisons but don't calculate hashes).
Another approach more in line with your original one is to explicitly
set update to false on your modified selector during normal builds and
set it to true when you actually deploy the files. This way modified
files will always be treated as modified until you deploy them. Or you
use different cache files for development and deployment (see the
cache.cachefile parameters).
Stefan
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@ant.apache.org
For additional commands, e-mail: user-help@ant.apache.org
RE: Use Ant with an arbitrary date
Posted by Emma Burrows <Em...@rpharms.com>.
Thank you for the response. Sorry, I should probably have anticipated that response by expanding more thoroughly on exactly what happens. :)
The files are all generated at the same time, whether they have been modified or not. As I am constantly tweaking the conversion process, these files are generated and date-stamped at any point during the week - so every single file will always have a date stamp later than last Thursday. But I only want to upload the ones which are actually different than the version in the last build.
Actually, come to think of it, the date is probably a bit of a red herring. I suppose what I really need is some process to say "this is the last 'build version'" (a cache of checksums collected at a given point in time for instance?), and then to say "copy over only the files which aren't identical to the last 'build version'". This would also allow for us having to re-run the CMS build more than once during the week but still load all the files that changed since the last milestone build as it were.
-----Original Message-----
From: Stefan Bodewig [mailto:bodewig@apache.org]
Sent: 29 November 2011 09:54
To: user@ant.apache.org
Subject: Re: Use Ant <modified> with an arbitrary date
On 2011-11-29, Emma Burrows wrote:
> So I was wondering if there was some way of using <modified>'s
> cache-based approach to only copy the files that have been modified
> since a given date/time like "17.00 last Thursday" instead of "last
> time this script was run".
This is a job for the date selector rather than the modified selector.
Since this one is purely based on last modified dates you wouldn't need
to calculate hashes and cache them anyway.
You may need some coding to calculate the date to pass in. <tstamp> may
help but this probably becomes brittle pretty soon.
Stefan
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@ant.apache.org
For additional commands, e-mail: user-help@ant.apache.org
______________________________________________________________________
This email has been scanned by the Symantec Email Security.cloud service.
For more information please visit http://www.symanteccloud.com
______________________________________________________________________
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@ant.apache.org
For additional commands, e-mail: user-help@ant.apache.org
Re: Use Ant with an arbitrary date
Posted by Stefan Bodewig <bo...@apache.org>.
On 2011-11-29, Emma Burrows wrote:
> So I was wondering if there was some way of using <modified>'s
> cache-based approach to only copy the files that have been modified
> since a given date/time like "17.00 last Thursday" instead of "last
> time this script was run".
This is a job for the date selector rather than the modified selector.
Since this one is purely based on last modified dates you wouldn't need
to calculate hashes and cache them anyway.
You may need some coding to calculate the date to pass in. <tstamp> may
help but this probably becomes brittle pretty soon.
Stefan
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@ant.apache.org
For additional commands, e-mail: user-help@ant.apache.org