You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@ant.apache.org by bu...@apache.org on 2010/11/19 02:47:24 UTC
DO NOT REPLY [Bug 50297] New: Copy task does not preserve natural
ordering of resource collections
https://issues.apache.org/bugzilla/show_bug.cgi?id=50297
Summary: Copy task does not preserve natural ordering of
resource collections
Product: Ant
Version: 1.8.1
Platform: PC
OS/Version: Windows XP
Status: NEW
Severity: major
Priority: P2
Component: Core tasks
AssignedTo: notifications@ant.apache.org
ReportedBy: isaac@shabtay.com
Various resource collections (union, etc) are designed with particular
iteration orders in mind. For example, a "union" stores items in a
LinkedHashSet to preserve ordering.
Turns out, though, that the Copy task doesn't respect the natural ordering of
items.
I am suspecting this is because "fileCopyMap" is a Hashtable; the
"doFileOperations()" method goes over the elements in "fileCopyMap" to execute
the copy instructions.
I took a few minutes (OK, more than a few) to look at the code and it appears
as if changing "fileCopyMap", "dirCopyMap" and perhaps "completeDirMap" to
"LinkedHashMap" instances would resolve this problem.
--
Configure bugmail: https://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
DO NOT REPLY [Bug 50297] Copy task does not preserve natural
ordering of resource collections
Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=50297
--- Comment #5 from Matt Benson <mb...@apache.org> 2010-11-22 12:29:49 EST ---
Much simpler would be to simply remove the unwanted resources from the source
collection by narrowing, composing, etc., using the various selectors and
collections available.
--
Configure bugmail: https://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
DO NOT REPLY [Bug 50297] Copy task does not preserve natural
ordering of resource collections
Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=50297
--- Comment #3 from Stefan Bodewig <bo...@apache.org> 2010-11-22 05:35:46 EST ---
The Copy class predates Java2 and in particular the point in time when Ant
started to require a JDK 1.2 to run, that's why it is using "old" collections.
The real BWC problem is that the fields are protected.
Even though I think you are trying to solve the problem with the wrong
approach I now understand your usecase. I've been thinking about a different
way to do what you want using built-in resource collections or selectors;
mappedresources would work if your union was in reverse order (i.e. you'd
only kee the first file of multiple files that mapped to the same name).
--
Configure bugmail: https://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
DO NOT REPLY [Bug 50297] Copy task does not preserve natural
ordering of resource collections
Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=50297
--- Comment #1 from Stefan Bodewig <bo...@apache.org> 2010-11-19 10:52:59 EST ---
While your proposed change would fix the ordering problem it would also
break the publiches API of the copy task in a backwards incompatible
way - so we can't do it that easily.
Under what circumstances is the order of file operations important to you?
--
Configure bugmail: https://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
DO NOT REPLY [Bug 50297] Copy task does not preserve natural
ordering of resource collections
Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=50297
--- Comment #12 from Stefan Bodewig <bo...@apache.org> 2010-11-30 11:42:47 EST ---
svn revision 1040627 contains LinkedHashtable but I'm running out of time and
can't test a modified Copy task using it right now - will do so tomorrow.
BTW, VectorSet couldn't wrap LinkedHashSet because of the insertAt operations
- and random access would be pretty slow as well.
--
Configure bugmail: https://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
DO NOT REPLY [Bug 50297] Copy task does not preserve natural
ordering of resource collections
Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=50297
--- Comment #4 from Isaac Shabtay <is...@shabtay.com> 2010-11-22 11:41:28 EST ---
Well I'm not sure why my approach is the wrong one; the only alternative I can
see is issuing multiple "copy" tasks, but that seems wasteful.
To fix this with a "Map", without breaking the API, I can only think of the
following:
1) Define a class that extends Hashtable (call it HashtableEx or something,
doesn't matter).
2) Have its constructor receive a reference to a Map, which will basically be
an "observer".
3) Override the "put" and "remove" method so, in addition to putting /removing
stuff to / from the Hashtable, also puts / removes stuff from / to the Map that
was provided in construction time.
4) Define an actual Map inside the Copy task - one for each Hashtable.
>From there on, any addition/removal will be "posted" to the map so the
hashtable and the map are always in sync. Internally, the Copy task should use
the Map rather than the hashtable; we can mark the hashtable instance as
"deprecated" so it can be removed in future releases.
Other than that, I can't really think of another way to resolve this issue.
--
Configure bugmail: https://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
DO NOT REPLY [Bug 50297] Copy task does not preserve natural
ordering of resource collections
Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=50297
Stefan Bodewig <bo...@apache.org> changed:
What |Removed |Added
----------------------------------------------------------------------------
Status|NEW |RESOLVED
Resolution| |FIXED
Target Milestone|--- |1.8.2
Severity|major |enhancement
--- Comment #13 from Stefan Bodewig <bo...@apache.org> 2010-12-01 00:27:29 EST ---
implemented with svn revision 1040874
--
Configure bugmail: https://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
DO NOT REPLY [Bug 50297] Copy task does not preserve natural
ordering of resource collections
Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=50297
--- Comment #14 from Isaac Shabtay <is...@shabtay.com> 2010-12-01 21:46:39 EST ---
Fantastic! Thank you both.
--
Configure bugmail: https://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
DO NOT REPLY [Bug 50297] Copy task does not preserve natural
ordering of resource collections
Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=50297
--- Comment #2 from Isaac Shabtay <is...@shabtay.com> 2010-11-19 19:00:32 EST ---
Well, both Hashtable and LinkedHashMap extend the Map interface so we could
change the variables' declaration to "Map". But you're right... it would still
break those who use these protected members within their code.
(Even though I think that's the way things should really be)
My situation is a bit tricky. Before the "copy" operation, I prepare a union of
resources to copy; the copy operation itself has a flattenmapper attached to
it. It is definitely possible for my union to contain more than one entry with
the same basename (different directories though).
So (for example):
<union>
<file name="dir1/file.txt"/>
<file name="dir2/file.txt"/>
</union>
and my intention is that "dir2/file.txt" will end-up in the target directory.
So I use a "copy" task with a flattenmapper...... therefore the order of
iteration is *VERY* important.
--
Configure bugmail: https://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
DO NOT REPLY [Bug 50297] Copy task does not preserve natural
ordering of resource collections
Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=50297
--- Comment #11 from Stefan Bodewig <bo...@apache.org> 2010-11-29 23:50:41 EST ---
First of all I'm sorry to suggest the mappedresources - this won't work since
the Hashtable is still free to reorder things.
Matt's option (1) is more or less the same we've done for VectorSet (which
might better wrap LinkedHashMap rather than use its current code). Yes, it
wouldn't help with subclasses but I'd live with that.
--
Configure bugmail: https://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
DO NOT REPLY [Bug 50297] Copy task does not preserve natural
ordering of resource collections
Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=50297
--- Comment #9 from Matt Benson <mb...@apache.org> 2010-11-29 14:39:33 EST ---
Two possibilities I have thought of:
1. Extend Hashtable such that all its methods actually defer to a wrapped
LinkedHashMap and use instances of that class in Copy's initializers. This
won't help the case where a subclass reassigns the references, however.
2. Rewrite copy/move and assign the task names to the new classes, which share
the existing introspection/XML interface. Users who extend the now deprecated
Copy/Move classes are unaffected.
Thoughts?
--
Configure bugmail: https://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
DO NOT REPLY [Bug 50297] Copy task does not preserve natural
ordering of resource collections
Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=50297
--- Comment #7 from Stefan Bodewig <bo...@apache.org> 2010-11-29 10:20:26 EST ---
After revisiting this I understood I was trong about mappedresources - it does
everything you need for your case.
Wrap a mappedresources element
http://ant.apache.org/manual/Types/resources.html#mappedresources around your
union and move the mapper from the copy task to a nested element of the
mappedresources element.
Should work or am I missing anything?
--
Configure bugmail: https://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
DO NOT REPLY [Bug 50297] Copy task does not preserve natural
ordering of resource collections
Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=50297
--- Comment #10 from Isaac Shabtay <is...@shabtay.com> 2010-11-29 14:45:20 EST ---
(In reply to comment #9)
> Two possibilities I have thought of:
>
> 1. Extend Hashtable such that all its methods actually defer to a wrapped
> LinkedHashMap and use instances of that class in Copy's initializers. This
> won't help the case where a subclass reassigns the references, however.
>
> 2. Rewrite copy/move and assign the task names to the new classes, which share
> the existing introspection/XML interface. Users who extend the now deprecated
> Copy/Move classes are unaffected.
>
> Thoughts?
I believe extending Hashtable will impact maintainability in the long term. If
it was up to me I'd go for option (2).
--
Configure bugmail: https://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
DO NOT REPLY [Bug 50297] Copy task does not preserve natural
ordering of resource collections
Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=50297
--- Comment #6 from Isaac Shabtay <is...@shabtay.com> 2010-11-22 21:53:37 EST ---
I beg to differ on that. Code-wise, there would be much less code in the
universe if the bug is fixed, than if I (or others who might encounter this
behavior later) work around it.
Plus, I don't see how I can work around this with the currently-available
resource collections, filters and such. Writing custom Java code seems to be
the only way to "filter" my list.
--
Configure bugmail: https://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
DO NOT REPLY [Bug 50297] Copy task does not preserve natural
ordering of resource collections
Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=50297
--- Comment #8 from Isaac Shabtay <is...@shabtay.com> 2010-11-29 14:24:29 EST ---
(In reply to comment #7)
> After revisiting this I understood I was trong about mappedresources - it does
> everything you need for your case.
>
Thanks, I will try that, however I just realized something. This also happens
when nested filesets exist for the "copy" task without any resource-collection
container (such as union). I just had a "copy" ask with two nested fileset
elements, and executing it I noticed that files are copied "out of order"; I
would expect the first fileset to be copied first (in whatever order), and the
second fileset to be copied afterwards.... But some files were copied from
fileset #1, then some files from fileset #2, then again some other files from
fileset #1....
I really think we have a serious problem here.
--
Configure bugmail: https://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.