You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ant.apache.org by Stefan Bodewig <bo...@apache.org> on 2003/01/09 09:04:22 UTC
The zip update problem
The mail I promised yesterday but didn't manage to write since the
Gump/xdoclet problem is eating too much time ...
I'm talking about bug 10755 here. The update feature of <zip> has
never worked, but it is broken in different ways in our past releases.
In Ant 1.4.x, update="true" would have recreated the archive all the
time, in 1.5.x it will never get recreated unless the files that are
to be added to the archive are newer than the archive (which means
"never" if the archive gets created during the build in the first
place).
I've been thinking about solving this in a most general way by
extending the underlying concepts in SourceFileScanner so that it can
deal with targets or sources that are not Files, but ZipEntrys (or at
a later point URLs or TarEntrys or ...). I had some initial ideas [1]
that I've not been too happy with and Magesh and Dominique have proven
that more thought needs to be put into the API. Later I ran out of
time (and my kids wouldn't allow me to code during my vacation, right
they were 8-).
In the meantime, some people have suggested patches in bugzilla that
would lead to a simpler but more specific solution (most notably
Antoine Levy-Lambert).
Putting things together, a good solution with an API that we'd want to
support in the future doesn't fit into the timeframe for a 1.5.2
release, if we are talking about end of January IMHO.
I see three options here:
(1) don't fix it in the 1.5 branch at all.
(2) delay 1.5.2 until we are happy with a general fix.
(3) fix it in the 1.5 branch using cut'n'paste and private methods and
hacks and such (so that we don't add any public API that needs to be
supported in the future) and get it right in 1.6.
I'm leaning towards (3) but want to get some feedback before I proceed
in that direction. After that I'd like to start discussion on what
"get it right" would be.
Stefan
Footnotes:
[1] <http://marc.theaimsgroup.com/?t=103849839700001&r=1&w=2>
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: The zip update problem
Posted by Antoine Levy-Lambert <le...@tiscali-dsl.de>.
I have tried the hack solution 3) myself ; I have the impression that it is
dirty [lengthy functions with a lot of ifs ], and will be more difficult to
debug and support than a clean solution 2).
I would rather go for 2). The bug has been around since the 12th of July
last year when it has been reported by Sven Karlsen.
I don't know how important it is not to delay 1.5.2 . Maybe it is possible
to solve the bug 10755 both quickly and cleanly, if all the ant community
contributes to solving it.
Antoine
----- Original Message -----
From: "Stefan Bodewig" <bo...@apache.org>
To: <an...@jakarta.apache.org>
Sent: Thursday, January 09, 2003 9:04 AM
Subject: The zip update problem
> The mail I promised yesterday but didn't manage to write since the
> Gump/xdoclet problem is eating too much time ...
>
> I'm talking about bug 10755 here. The update feature of <zip> has
> never worked, but it is broken in different ways in our past releases.
>
> In Ant 1.4.x, update="true" would have recreated the archive all the
> time, in 1.5.x it will never get recreated unless the files that are
> to be added to the archive are newer than the archive (which means
> "never" if the archive gets created during the build in the first
> place).
>
> I've been thinking about solving this in a most general way by
> extending the underlying concepts in SourceFileScanner so that it can
> deal with targets or sources that are not Files, but ZipEntrys (or at
> a later point URLs or TarEntrys or ...). I had some initial ideas [1]
> that I've not been too happy with and Magesh and Dominique have proven
> that more thought needs to be put into the API. Later I ran out of
> time (and my kids wouldn't allow me to code during my vacation, right
> they were 8-).
>
> In the meantime, some people have suggested patches in bugzilla that
> would lead to a simpler but more specific solution (most notably
> Antoine Levy-Lambert).
>
> Putting things together, a good solution with an API that we'd want to
> support in the future doesn't fit into the timeframe for a 1.5.2
> release, if we are talking about end of January IMHO.
>
> I see three options here:
>
> (1) don't fix it in the 1.5 branch at all.
>
> (2) delay 1.5.2 until we are happy with a general fix.
>
> (3) fix it in the 1.5 branch using cut'n'paste and private methods and
> hacks and such (so that we don't add any public API that needs to be
> supported in the future) and get it right in 1.6.
>
> I'm leaning towards (3) but want to get some feedback before I proceed
> in that direction. After that I'd like to start discussion on what
> "get it right" would be.
>
> Stefan
>
> Footnotes:
> [1] <http://marc.theaimsgroup.com/?t=103849839700001&r=1&w=2>
>
>
> --
> To unsubscribe, e-mail: <ma...@jakarta.apache.org>
> For additional commands, e-mail: <ma...@jakarta.apache.org>
>
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: The zip update problem
Posted by Stefan Bodewig <bo...@apache.org>.
Mails crossed, sorry.
On Fri, 10 Jan 2003, Antoine Levy-Lambert <le...@tiscali-dsl.de>
wrote:
>> > Classes interested in knowing the modification date/time of a
>> > file should ask the scanner, not the file directly any more.
>>
>> Why? They might be better of accessing the Resources directly.
>>
> The reason is that a file in a fileset can be present on the
> filesystem directly, or might be a zip entry.
I've been too terse here. I was assuming the scanner would return the
Resources in the first place instead of names -
i.e. getIncludedResources and with them, the tasks don't need to go
back to the DirectoryScanner to find the lastmodified time.
> In our new concept, the Scanner is the class which knows where are
> files or zip entries and how to read their properties.
I've taken a different twist by returning the Resources directly.
> <zip ...>
> <zipfileset src="foo.zip" id="foo"/>
> </zip>
> <copy todir="mydir">
> <fileset refid="foo"/>
> </copy>
>
> this should copy all the zip entries from foo.zip to the filesystem,
> to the subdirectory mydir
Maybe in the long run, yes ...
For this to work, Resource (I'm sticking to the name for now) would
have to provide readers to its content. We are getting close to a
full-fledged VFS layer here and would probably be better of to join
forces with the vfs code in the jakarta-commons-sandbox IMHO.
>> > one would add two new functions in DirectoryScanner
>> > long getModificationTime(String path)
>>
> this function would be to get the modification time of a file or of
> a zip entry,
redundant if we return the Resources.
>> > and
>> > Resource [ ] list(String path)
>>
>> Under which circumstances would you expect more than one resource
>> for a given path?
>>
> this function would be used in the scandir method of Directory.java,
> instead of java.io.File.list instead of String []
> newfiles=dir.list();
Would we need to expose this method outside DirectoryScanner?
> Maybe the name Resource for the class is confusing;
I'm not good at names. Throughout Ant you will find more than enough
things that back this claim. ;-)
Stefan
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: The zip update problem
Posted by Antoine Levy-Lambert <le...@tiscali-dsl.de>.
Dear ant developers,
> > Classes interested in knowing the modification date/time of a file
> > should ask the scanner, not the file directly any more.
>
> Why? They might be better of accessing the Resources directly.
>
The reason is that a file in a fileset can be present on the filesystem
directly, or might be a zip entry.
So we would like that packaging tasks (Zip) only access DateTime information
for purpose of ascertaining up to date status
of target with respect to source only through the Scanner (DirectoryScanner
or ZipScanner). In our new concept, the Scanner
is the class which knows where are files or zip entries and how to read
their properties.
<zip ...>
<zipfileset src="foo.zip" id="foo"/>
</zip>
<copy todir="mydir">
<fileset refid="foo"/>
</copy>
this should copy all the zip entries from foo.zip to the filesystem, to the
subdirectory mydir
<!--deployment procedure of foo.zip -->
<get src="ftp://somehost.acme.com:/pub/foo.zip" dest="foo.zip"/>
<zipfileset src="foo.zip" id="foodoc" dir="htdocs/manual"
prefix="docs/user-guide" />
<zipfileset src="foo.zip" id="foolib" dir="lib" prefix="lib/foo" />
<copy todir="install">
<fileset refid="foodoc"/>
<fileset refid="foolib"/>
</copy>
should copy the files located in foo.zip under htdocs/manual to
install/docs/user-guide on the hard disk
and the files from the lib subdir of foo.zip to install/lib/foo on the hard
disk
This should be useful to make installation procedures, if you want to
automate the creation of your development or build environment
> > one would add two new functions in DirectoryScanner
> > long getModificationTime(String path)
>
this function would be to get the modification time of a file or of a zip
entry, by specifying the path of the file relative to the directory of the
fileset.
>
> > and
> > Resource [ ] list(String path)
>
> Under which circumstances would you expect more than one resource for
> a given path?
>
this function would be used in the scandir method of Directory.java, instead
of java.io.File.list
instead of
String [] newfiles=dir.list(); // fist line of scandir in
DirectoryScanner.java
we would have
Resource [] newResources=list(dir);
So for a given path there are several resources (all the files and
directories below this path)
Maybe the name Resource for the class is confusing; it means basically File
or ZipEntry (or maybe in the future URL, file from a version control
repository, ...)
> Stefan
>
> --
> To unsubscribe, e-mail: <ma...@jakarta.apache.org>
> For additional commands, e-mail: <ma...@jakarta.apache.org>
>
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: The zip update problem
Posted by Stefan Bodewig <bo...@apache.org>.
On Thu, 9 Jan 2003, Antoine Levy-Lambert <le...@tiscali-dsl.de>
wrote:
> 1) ZipScanner class
>
> it is currently implemented in such a way that the
> getIncludedFiles() function answers with an array containing as only
> element the name of the zip file itself.
>
> I believe we should change the implementation of the class so that
> ZipScanner.getIncludedFiles() gives the list of entries in the
> archive which match the include and exclude patterns.
>
> Does anyone see a danger of breaking something here ?
Yes (and no).
The only classes using ZipScanner are Zip and ZipFileSet. As Zip
would get adjusted, we won't have a problem here, but ...
<zip ...>
<zipfileset src="foo.zip" id="foo"/>
</zip>
<copy ...>
<fileset refid="foo"/>
</copy>
would break. Copy would try to copy files with the names of the
ZipEntries. Then again, a construct like this would be silly, as it
wouldn't really work right now either.
> maybe we could create such a Resource class under
> org.apache.tools.ant.types it needs only two instance variables :
> String FileName and long ModificationTime or something like that
Yep.
> Then the DirectoryScanner could store the results of
> scanning internally as vectors of Resource
OK.
> Classes interested in knowing the modification date/time of a file
> should ask the scanner, not the file directly any more.
Why? They might be better of accessing the Resources directly.
> one would add two new functions in DirectoryScanner
> long getModificationTime(String path)
See above.
> and
> Resource [ ] list(String path)
Under which circumstances would you expect more than one resource for
a given path?
Stefan
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: The zip update problem
Posted by Stefan Bodewig <bo...@apache.org>.
On Thu, 9 Jan 2003, Antoine Levy-Lambert <le...@tiscali-dsl.de>
wrote:
> 1) ZipScanner class
>
> it is currently implemented in such a way that the
> getIncludedFiles() function answers with an array containing as only
> element the name of the zip file itself.
>
> I believe we should change the implementation of the class so that
> ZipScanner.getIncludedFiles() gives the list of entries in the
> archive which match the include and exclude patterns.
>
> Does anyone see a danger of breaking something here ?
Yes (and no).
The only classes using ZipScanner are Zip and ZipFileSet. As Zip
would get adjusted, we won't have a problem here, but ...
<zip ...>
<zipfileset src="foo.zip" id="foo"/>
</zip>
<copy ...>
<fileset refid="foo"/>
</copy>
would break. Copy would try to copy files with the names of the
ZipEntries. Then again, a construct like this would be silly, as it
wouldn't really work right now either.
> maybe we could create such a Resource class under
> org.apache.tools.ant.types it needs only two instance variables :
> String FileName and long ModificationTime or something like that
Yep.
> Then the DirectoryScanner could store the results of
> scanning internally as vectors of Resource
OK.
> Classes interested in knowing the modification date/time of a file
> should ask the scanner, not the file directly any more.
Why? They might be better of accessing the Resources directly.
> one would add two new functions in DirectoryScanner
> long getModificationTime(String path)
See above.
> and
> Resource [ ] list(String path)
Under which circumstances would you expect more than one resource for
a given path?
Stefan
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: The zip update problem
Posted by Antoine Levy-Lambert <le...@tiscali-dsl.de>.
Some suggestions/questions concerning the solution :
1) ZipScanner class
it is currently implemented in such a way that the getIncludedFiles()
function answers with
an array containing as only element the name of the zip file itself.
I believe we should change the implementation of the class so that
ZipScanner.getIncludedFiles() gives
the list of entries in the archive which match the include and exclude
patterns.
Does anyone see a danger of breaking something here ?
2) DirectoryScanner and ZipScanner classes
Stefan Bodewig wrote :
>Where I was thinking about changes is the SourceFileScanner class,
> instead of only working on files, it should work on "resources" that
> had a name and a last modification time. What we'd then need was to
> pass in "factories" to generate the source and target resources.
+1
maybe we could create such a Resource class under org.apache.tools.ant.types
it needs only two instance variables : String FileName and long
ModificationTime or something like that
Then the DirectoryScanner could store the results of
scanning internally as vectors of Resource
Classes interested in knowing the modification date/time of a file should
ask the scanner, not the file directly any more.
one would add two new functions in DirectoryScanner
long getModificationTime(String path)
and
Resource [ ] list(String path)
this list function would be redefined in ZipScanner to list the zip entries
and would be called from the scandir method of the DirectoryScanner class to
replace dir.list() (Java.IO.File)
the rest of the logic of scandir should work; instead of maintaining vectors
of filenames, one would maintain vectors of the new Resource type so that
the scanner "knows" what are the modification times of each files.
Afterwards, the game is to make sure that packaging tasks always find out
about the date/time of files from the scanners, not directly from the file
system (FileUtils) because the scanned files can be on the file system or in
a zip file.
How does that sound ?
Antoine
----- Original Message -----
From: "Stefan Bodewig" <bo...@apache.org>
To: <an...@jakarta.apache.org>
Sent: Thursday, January 09, 2003 9:38 AM
Subject: Re: The zip update problem
> On Thu, 09 Jan 2003, Conor MacNeill <co...@cortexebusiness.com.au>
> wrote:
>
> > MHO would be to get this done in whatever time it takes, release
> > 1.5.2
>
> Works for me as well.
>
> > and then call 1.5 done.
>
> unless an important bug gets fixed and the 1.6 release is still too
> far away.
>
> In some way I regret we didn't release a 1.4.2 that would have been
> 1.4.1 plus the WinXP/JDK 1.4 fix a year ago.
>
> > I'd be happier to do it right, first up and save the hack
> > work.
>
> fair enough.
>
> Stefan
>
> --
> To unsubscribe, e-mail: <ma...@jakarta.apache.org>
> For additional commands, e-mail: <ma...@jakarta.apache.org>
>
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: The zip update problem
Posted by Stefan Bodewig <bo...@apache.org>.
On Thu, 09 Jan 2003, Conor MacNeill <co...@cortexebusiness.com.au>
wrote:
> MHO would be to get this done in whatever time it takes, release
> 1.5.2
Works for me as well.
> and then call 1.5 done.
unless an important bug gets fixed and the 1.6 release is still too
far away.
In some way I regret we didn't release a 1.4.2 that would have been
1.4.1 plus the WinXP/JDK 1.4 fix a year ago.
> I'd be happier to do it right, first up and save the hack
> work.
fair enough.
Stefan
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: The zip update problem
Posted by Conor MacNeill <co...@cortexebusiness.com.au>.
Stefan Bodewig wrote:
>
> (1) don't fix it in the 1.5 branch at all.
-1, this issue is biting a lot of people judging by comments on
ant-user/ant-dev and the number of votes the bugs receive on BugZilla
(That's right folks, it is worth using your votes on BugZilla)
>
> (2) delay 1.5.2 until we are happy with a general fix.
+1. 1.5.1 is 3 months old now so I don't think we have to stick to a 1.5.2
deadline that is really artificial. MHO would be to get this done in
whatever time it takes, release 1.5.2 and then call 1.5 done.
>
> (3) fix it in the 1.5 branch using cut'n'paste and private methods and
> hacks and such (so that we don't add any public API that needs to be
> supported in the future) and get it right in 1.6.
I'd be happier to do it right, first up and save the hack work. There are
some work arounds being suggested with <touch> which may serve in the interim.
All MHO.
Conor
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>