You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ant.apache.org by Nicolas Lalevée <ni...@hibnet.org> on 2010/08/15 14:22:44 UTC
bug #49119, threading and IO
I try to fix the bug #49119 [1] but I have some threading and IO issues.
The bug is about a forked <java> stealing characters from the input stream even if is finished.
This is due to the pumping thread which is reading the Ant own input stream and redirecting it to the the process. It doesn't stop at the end of the java task. It is actually blocked into the intputstream.read call.
So my first fix [2] was to use the non-blocking feature of StreamPumper, relying on inputstream.available() to know if there is something to read. It does fix my issue, the pumping thread terminate as soon as the java task finish.
But then I broke the unit test JavaTest#testRedirect1. The test is sending a string as input expecting to see it in the output. Before my patch the pumping thread was reading the input string but was also closing the stream. Relying on is.available() cannot tell us if it is actually closed. The other end of the stream JavaTest#PipeEntryPoint is expecting a closed input at some point so it doesn't know where to stop redirecting everything from input to the output. Then with my patch it is hanging indefinitely.
Then I looked into interrupting the read, it doesn't seem to be possible [3].
Another fix I thought is about closing the output if the input is empty: if is.available() == 0 and closeWhenExhausted == true, then do stop the pumping and close. But it seems quite dangerous if some input stream is quite slow.
But now I am wondering if it is a false assumption that an input stream can close. In "real life" scripting, how would an input stream be closed ? It should be opened as long as the user is typing on his keyboard, isn't it ? So maybe just the unit test is doing false assumption ?
Nicolas
[1] https://issues.apache.org/bugzilla/show_bug.cgi?id=49119
[2] http://svn.apache.org/viewvc?view=revision&revision=985632
[3] http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4514257
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@ant.apache.org
For additional commands, e-mail: dev-help@ant.apache.org
Re: bug #49119, threading and IO
Posted by Nicolas Lalevée <ni...@hibnet.org>.
Le 17 août 2010 à 18:36, Stefan Bodewig a écrit :
> On 2010-08-15, Nicolas Lalevée wrote:
>
>> The bug is about a forked <java> stealing characters from the input
>> stream even if is finished. This is due to the pumping thread which
>> is reading the Ant own input stream and redirecting it to the the
>> process. It doesn't stop at the end of the java task. It is actually
>> blocked into the intputstream.read call.
>
> This is also means Ant is leaking threads for every forked process that
> has input attached to a stream that is never closed and never provides
> input - like System.in - which means Ant leaks threads for every forked
> process by default.
>
> This is why I marked bug 49587 as a duplicate of 49119.
>
>> So my first fix [2] was to use the non-blocking feature of
>> StreamPumper, relying on inputstream.available() to know if there is
>> something to read.
>
> This likely is the only option, yes. You may need to add the stream
> pumper thread to the thread group the complete method in Redirector is
> waiting for and make PumpStreamHandler "finish" the input thread (that
> it currently isn't even aware of) as well.
>
>> It does fix my issue, the pumping thread terminate as soon as the java
>> task finish. But then I broke the unit test
>> JavaTest#testRedirect1. The test is sending a string as input
>> expecting to see it in the output. Before my patch the pumping thread
>> was reading the input string but was also closing the stream.
>
> Which side is closing the stream? I'd think the writing side should do
> once it has written the specified string.
>
>> Relying on is.available() cannot tell us if it is actually closed.
>
> With the changes above Thread.interrupted() should eventually return
> true when finish is true as well.
>
>> Then I looked into interrupting the read, it doesn't seem to be possible [3].
>
> Right, that's why we have the available() clutch.
>
>> But now I am wondering if it is a false assumption that an input
>> stream can close. In "real life" scripting, how would an input stream
>> be closed ?
>
> When input is redirected (inputstring attribute or reading from a
> resource).
>
> We likely should only set useAvailable to true on the input thread if
> input is not redirected to avoid unneeded sleeping when we know the
> input stream will be closed.
seems the best option, true.
> This also means for your failing test case useAvailable would be false
> and the test would pass.
Actually if we keep the read blocking behavior when an input file or string is provided, the end-of-stream will be still transmitted. So the unit test won't hang up.
This solution fits nicely with each use case. I'll work on the fix.
thanks.
Nicolas
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@ant.apache.org
For additional commands, e-mail: dev-help@ant.apache.org
Re: bug #49119, threading and IO
Posted by Stefan Bodewig <bo...@apache.org>.
On 2010-08-15, Nicolas Lalevée wrote:
> The bug is about a forked <java> stealing characters from the input
> stream even if is finished. This is due to the pumping thread which
> is reading the Ant own input stream and redirecting it to the the
> process. It doesn't stop at the end of the java task. It is actually
> blocked into the intputstream.read call.
This is also means Ant is leaking threads for every forked process that
has input attached to a stream that is never closed and never provides
input - like System.in - which means Ant leaks threads for every forked
process by default.
This is why I marked bug 49587 as a duplicate of 49119.
> So my first fix [2] was to use the non-blocking feature of
> StreamPumper, relying on inputstream.available() to know if there is
> something to read.
This likely is the only option, yes. You may need to add the stream
pumper thread to the thread group the complete method in Redirector is
waiting for and make PumpStreamHandler "finish" the input thread (that
it currently isn't even aware of) as well.
> It does fix my issue, the pumping thread terminate as soon as the java
> task finish. But then I broke the unit test
> JavaTest#testRedirect1. The test is sending a string as input
> expecting to see it in the output. Before my patch the pumping thread
> was reading the input string but was also closing the stream.
Which side is closing the stream? I'd think the writing side should do
once it has written the specified string.
> Relying on is.available() cannot tell us if it is actually closed.
With the changes above Thread.interrupted() should eventually return
true when finish is true as well.
> Then I looked into interrupting the read, it doesn't seem to be possible [3].
Right, that's why we have the available() clutch.
> But now I am wondering if it is a false assumption that an input
> stream can close. In "real life" scripting, how would an input stream
> be closed ?
When input is redirected (inputstring attribute or reading from a
resource).
We likely should only set useAvailable to true on the input thread if
input is not redirected to avoid unneeded sleeping when we know the
input stream will be closed.
This also means for your failing test case useAvailable would be false
and the test would pass.
Stefan
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@ant.apache.org
For additional commands, e-mail: dev-help@ant.apache.org