You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ant.apache.org by Anakreon Mejdi <am...@ertonline.gr> on 2002/11/08 15:12:35 UTC
Apply/Exec
It seems that apply does not take under consideration the limitations
the OS implies on the number of parameters a program can receive.
I discovered this when called tidy.exe (on Windows 2000) when the
the files which will tide up are more then 13000.
If parallel attribute is set to true apply task fails to create the
process.
I did what I wanted by setting parallel to false but in this way it
takes longer since the process should be created 13000 times.
Here is the target's code:
<target name="tidy" description="Tides the html files">
<apply
dir="${cTemp}"
executable="C:/tidy/tidy.exe"
failonerror="false"
skipemptyfilesets="true"
parallel="false"
type="file"
relative="false"
output="err.txt"
>
<arg value="-config"/>
<arg value="C:/tidy.conf"/>
<arg value="-m"/>
<srcfile/>
<fileset casesensitive="false" dir="${cTemp}">
<patternset includes="**/*.asp"/>
<patternset includes="**/*.html"/>
<patternset includes="**/*.htm" />
<patternset includes="**/*.inc" />
<patternset includes="**/*.templ" />
</fileset>
</apply>
</target>
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: Apply/Exec
Posted by Stefan Bodewig <bo...@apache.org>.
On Fri, 8 Nov 2002, Kendall Collett <kc...@convio.com> wrote:
> If there was a way of specifying the maximum number and/or length of
> arguments (ala xargs' --max-args and --max-chars options; see
> <http://nodevice.com/sections/ManIndex/man1940.html>), that might
> help.
True. Has been suggested before, but nobody has found the time to
code it up (yet).
Stefan
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: Apply/Exec
Posted by Kendall Collett <kc...@convio.com>.
If there was a way of specifying the maximum number and/or length of
arguments (ala xargs' --max-args and --max-chars options; see
<http://nodevice.com/sections/ManIndex/man1940.html>), that might help.
That way, if you had a large number of files, you could pick some large
number that *usually* worked instead of having to say parallel="false" and
losing the performance benefit altogether. You could still end up picking a
number that was close to the edge (where it could fail in some future
because the fileset picked up more files), but it might be better that
nothing, (It's unfortunate that there isn't portable way to determine the
argument limits.)
Kendall
----- Original Message -----
From: "Stefan Bodewig" <bo...@apache.org>
To: <an...@jakarta.apache.org>
Sent: Friday, November 08, 2002 09:12
Subject: Re: Apply/Exec
> On Fri, 08 Nov 2002, Anakreon Mejdi <am...@ertonline.gr> wrote:
>
> > tar is just an example, but what I mean is that if by splitting the
> > parameters it wouldn't work, if parallel is false it won't work
> > either.
>
> Absolutely true.
>
> My point was that Ant might be able to detect the command line was
> getting too long (but I even doubt that, as we'd need people to report
> values for *a lot* of platforms) - but that it couldn't act
> automatically and assume things have been OK after that.
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: Apply/Exec
Posted by Stefan Bodewig <bo...@apache.org>.
On Fri, 08 Nov 2002, Anakreon Mejdi <am...@ertonline.gr> wrote:
> tar is just an example, but what I mean is that if by splitting the
> parameters it wouldn't work, if parallel is false it won't work
> either.
Absolutely true.
My point was that Ant might be able to detect the command line was
getting too long (but I even doubt that, as we'd need people to report
values for *a lot* of platforms) - but that it couldn't act
automatically and assume things have been OK after that.
Stefan
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: Apply/Exec
Posted by Anakreon Mejdi <am...@ertonline.gr>.
Stefan Bodewig wrote:
> On Fri, 08 Nov 2002, Anakreon Mejdi <am...@ertonline.gr> wrote:
>
>
>>I think it could handle it. apply knows in which OS is running and
>>from a configuration (stored in ant.jar) could know how many
>>parameters the OS can receive.
>
>
> Not really. It is the total lenght, not the number of arguments that
> is often the limit.
Params are String so length could be calculated in that case.
>
>
>>For example lets say that an OS can take 10 params and 3 are passed
>>as <arg> from the apply taks. So we have 7 params which can use.
>>>From the fileset we have 100 files. What apply could do is to split
>>the 100 files into portions of 7 and for each 7 params execute the
>>process.
>
>
> But this doesn't work for all cases. Sometimes parallel="true" cannot
> be split into sevaral invocations without causing problems.
>
> Take tar as executable. "tar cf archive.tar file1 .... fileN" creates
> archive.tar from the files. I can not simply split this into several
> commands as I'd have to use "tar rf archive.tar ...." in the second
> and subsequent calls to append instead of overwrite the existing
> archive.
Yes, this is true.
But if parallel was false wouldn't this mean that the commands would be:
tar cf archive.tar file1
tar cf archive.tar file2
.....
tar cf archive.tar fileN
In this case what would be the result?
An archive which would contain only fileN?
If the answer is yes (I am not familiar with tar to know the answer)
apply would not be the correct task any way.
Most likely tar must have a parameter to append contents into existing
archive.
tar is just an example, but what I mean is that if by splitting the
parameters it wouldn't work, if parallel is false it won't work either.
I might be wrong , so I don't insist any longer.
Thanks for answering.
Anakreon
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: Apply/Exec
Posted by Stefan Bodewig <bo...@apache.org>.
On Fri, 08 Nov 2002, Anakreon Mejdi <am...@ertonline.gr> wrote:
> I think it could handle it. apply knows in which OS is running and
> from a configuration (stored in ant.jar) could know how many
> parameters the OS can receive.
Not really. It is the total lenght, not the number of arguments that
is often the limit.
> For example lets say that an OS can take 10 params and 3 are passed
> as <arg> from the apply taks. So we have 7 params which can use.
> From the fileset we have 100 files. What apply could do is to split
> the 100 files into portions of 7 and for each 7 params execute the
> process.
But this doesn't work for all cases. Sometimes parallel="true" cannot
be split into sevaral invocations without causing problems.
Take tar as executable. "tar cf archive.tar file1 .... fileN" creates
archive.tar from the files. I can not simply split this into several
commands as I'd have to use "tar rf archive.tar ...." in the second
and subsequent calls to append instead of overwrite the existing
archive.
This is why I said it was not possible to do it automatically and
correctly at the same time.
Stefan
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: Apply/Exec
Posted by Anakreon Mejdi <am...@ertonline.gr>.
I think it could handle it.
apply knows in which OS is running and from a configuration (stored in
ant.jar) could know how many parameters the OS can receive.
For example lets say that an OS can take 10 params
and 3 are passed as <arg> from the apply taks.
So we have 7 params which can use.
From the fileset we have 100 files.
What apply could do is to split the 100 files into portions of 7
and for each 7 params execute the process.
In my case I know how many files will be passed but this is not always
possible which would make parallel attribute useless.
Stefan Bodewig wrote:
> On Fri, 08 Nov 2002, Anakreon Mejdi <am...@ertonline.gr> wrote:
>
>
>>It seems that apply does not take under consideration the
>>limitations the OS implies on the number of parameters a program can
>>receive.
>
>
> This is true. And I don't see how it could handle this situation
> automagically and correctly at the same time.
>
> You can try to use several filesets in several apply tasks, each with
> just enough files to not run into command line length problems.
>
> Stefan
>
> --
> To unsubscribe, e-mail: <ma...@jakarta.apache.org>
> For additional commands, e-mail: <ma...@jakarta.apache.org>
>
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: Apply/Exec
Posted by Stefan Bodewig <bo...@apache.org>.
On Fri, 08 Nov 2002, Anakreon Mejdi <am...@ertonline.gr> wrote:
> It seems that apply does not take under consideration the
> limitations the OS implies on the number of parameters a program can
> receive.
This is true. And I don't see how it could handle this situation
automagically and correctly at the same time.
You can try to use several filesets in several apply tasks, each with
just enough files to not run into command line length problems.
Stefan
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>