You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@river.apache.org by Peter Firmstone <ji...@zeus.net.au> on 2014/05/14 11:03:20 UTC

The King of all Bugs

One of the things I like about JERI is it's multiplexing and multithreaded.

What I don't like about JERI is, it passes ByteBuffers between calling 
threads and pool threads.

Who can guess what's wrong with that?

Peter.

Re: The King of all Bugs

Posted by Bryan Thompson <br...@systap.com>.
That is, assuming that the views of the buffer have distinct offsets so the read/write operations within those views can not overlap.  Brya 

> On May 14, 2014, at 6:06 AM, Peter Firmstone <ji...@zeus.net.au> wrote:
> 
> Hmm, it already does that, I wonder if this is safe for direct ByteBuffer's?
> 
> Visibility is one issue, the second is mutual exclusion.  I think mutual exclusion is ok, not sure about visibility.
> 
>> On 14/05/2014 8:01 PM, Peter Firmstone wrote:
>> Actually an alternative could be to use future send for all transfers, except for eof, which can be safely used for async send since the byte buffer doesn't change after eof.
>> 
>>> On 14/05/2014 7:55 PM, Peter Firmstone wrote:
>>> There are two methods of transfer, one, future send, where the original thread waits to be notified that the receiving thread has finished processing the ByteBuffer, this appears to be safely transferred.  This is for server mode.
>>> 
>>> Then there is async send, this is for client mode, here the calling thread doesn't wait for the receiving thread.
>>> 
>>> Have a look in com.sun.jini.jeri.internal.mux.Session, here the OutputStream uses a ByteBuffer which it shares via async send.
>>> 
>>> I think the answer is to defensively copy the async send ByteBuffer.
>>> 
>>> 
>>> 
>>> 
>>> 
>>>> On 14/05/2014 7:14 PM, Patricia Shanahan wrote:
>>>> ByteBuffer is a subclass of Buffer, whose documentation says, under "Thread Safety", "Buffers are not safe for use by multiple concurrent threads. If a buffer is to be used by more than one thread then access to the buffer should be controlled by appropriate synchronization."
>>>> 
>>>> Is access controlled by appropriate synchronization? How are the buffers communicated between the threads?
>>>> 
>>>> Patricia
>>>> 
>>>>> On 5/14/2014 2:03 AM, Peter Firmstone wrote:
>>>>> One of the things I like about JERI is it's multiplexing and multithreaded.
>>>>> 
>>>>> What I don't like about JERI is, it passes ByteBuffers between calling
>>>>> threads and pool threads.
>>>>> 
>>>>> Who can guess what's wrong with that?
>>>>> 
>>>>> Peter.
> 

Re: The King of all Bugs

Posted by Bryan Thompson <br...@systap.com>.
Excellent.

> On May 14, 2014, at 9:07 AM, Peter Firmstone <ji...@zeus.net.au> wrote:
> 
> Thanks Bryan,
> 
> That's done the trick, the OutputStream that contained the ByteBuffer was reliant on the mux output thread changing the state of position, limit and mark.
> 
> Now the duplicate has it's own state and is published safely, the OutputStream then waits for the mux output thread to finish if necessary, the original buffer is now cleared and is ready for more data, rather than relying on the mux output thread to reset it's state.
> 
> Fixed another data race while I was at it, a synchronized collection was shared outside of synchronization with a pool thread via a Runnable.
> 
> Regards,
> 
> Peter.
> 
>> On 14/05/2014 8:15 PM, Bryan Thompson wrote:
>> All you need to do is dup the buffer. That gives you independent fields for position and limit, which is what is not thread safe.  You do need a synchronization section around that dup, but that is a very fast operation.  You can also use the read only view of the buffer. Bryan
>> 
>>> On May 14, 2014, at 6:06 AM, Peter Firmstone<ji...@zeus.net.au>  wrote:
>>> 
>>> Hmm, it already does that, I wonder if this is safe for direct ByteBuffer's?
>>> 
>>> Visibility is one issue, the second is mutual exclusion.  I think mutual exclusion is ok, not sure about visibility.
>>> 
>>>> On 14/05/2014 8:01 PM, Peter Firmstone wrote:
>>>> Actually an alternative could be to use future send for all transfers, except for eof, which can be safely used for async send since the byte buffer doesn't change after eof.
>>>> 
>>>>> On 14/05/2014 7:55 PM, Peter Firmstone wrote:
>>>>> There are two methods of transfer, one, future send, where the original thread waits to be notified that the receiving thread has finished processing the ByteBuffer, this appears to be safely transferred.  This is for server mode.
>>>>> 
>>>>> Then there is async send, this is for client mode, here the calling thread doesn't wait for the receiving thread.
>>>>> 
>>>>> Have a look in com.sun.jini.jeri.internal.mux.Session, here the OutputStream uses a ByteBuffer which it shares via async send.
>>>>> 
>>>>> I think the answer is to defensively copy the async send ByteBuffer.
>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>>>> On 14/05/2014 7:14 PM, Patricia Shanahan wrote:
>>>>>> ByteBuffer is a subclass of Buffer, whose documentation says, under "Thread Safety", "Buffers are not safe for use by multiple concurrent threads. If a buffer is to be used by more than one thread then access to the buffer should be controlled by appropriate synchronization."
>>>>>> 
>>>>>> Is access controlled by appropriate synchronization? How are the buffers communicated between the threads?
>>>>>> 
>>>>>> Patricia
>>>>>> 
>>>>>>> On 5/14/2014 2:03 AM, Peter Firmstone wrote:
>>>>>>> One of the things I like about JERI is it's multiplexing and multithreaded.
>>>>>>> 
>>>>>>> What I don't like about JERI is, it passes ByteBuffers between calling
>>>>>>> threads and pool threads.
>>>>>>> 
>>>>>>> Who can guess what's wrong with that?
>>>>>>> 
>>>>>>> Peter.
> 

Re: The King of all Bugs

Posted by Peter Firmstone <ji...@zeus.net.au>.
Thanks Bryan,

That's done the trick, the OutputStream that contained the ByteBuffer 
was reliant on the mux output thread changing the state of position, 
limit and mark.

Now the duplicate has it's own state and is published safely, the 
OutputStream then waits for the mux output thread to finish if 
necessary, the original buffer is now cleared and is ready for more 
data, rather than relying on the mux output thread to reset it's state.

Fixed another data race while I was at it, a synchronized collection was 
shared outside of synchronization with a pool thread via a Runnable.

Regards,

Peter.

On 14/05/2014 8:15 PM, Bryan Thompson wrote:
> All you need to do is dup the buffer. That gives you independent fields for position and limit, which is what is not thread safe.  You do need a synchronization section around that dup, but that is a very fast operation.  You can also use the read only view of the buffer. Bryan
>
>> On May 14, 2014, at 6:06 AM, Peter Firmstone<ji...@zeus.net.au>  wrote:
>>
>> Hmm, it already does that, I wonder if this is safe for direct ByteBuffer's?
>>
>> Visibility is one issue, the second is mutual exclusion.  I think mutual exclusion is ok, not sure about visibility.
>>
>>> On 14/05/2014 8:01 PM, Peter Firmstone wrote:
>>> Actually an alternative could be to use future send for all transfers, except for eof, which can be safely used for async send since the byte buffer doesn't change after eof.
>>>
>>>> On 14/05/2014 7:55 PM, Peter Firmstone wrote:
>>>> There are two methods of transfer, one, future send, where the original thread waits to be notified that the receiving thread has finished processing the ByteBuffer, this appears to be safely transferred.  This is for server mode.
>>>>
>>>> Then there is async send, this is for client mode, here the calling thread doesn't wait for the receiving thread.
>>>>
>>>> Have a look in com.sun.jini.jeri.internal.mux.Session, here the OutputStream uses a ByteBuffer which it shares via async send.
>>>>
>>>> I think the answer is to defensively copy the async send ByteBuffer.
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>> On 14/05/2014 7:14 PM, Patricia Shanahan wrote:
>>>>> ByteBuffer is a subclass of Buffer, whose documentation says, under "Thread Safety", "Buffers are not safe for use by multiple concurrent threads. If a buffer is to be used by more than one thread then access to the buffer should be controlled by appropriate synchronization."
>>>>>
>>>>> Is access controlled by appropriate synchronization? How are the buffers communicated between the threads?
>>>>>
>>>>> Patricia
>>>>>
>>>>>> On 5/14/2014 2:03 AM, Peter Firmstone wrote:
>>>>>> One of the things I like about JERI is it's multiplexing and multithreaded.
>>>>>>
>>>>>> What I don't like about JERI is, it passes ByteBuffers between calling
>>>>>> threads and pool threads.
>>>>>>
>>>>>> Who can guess what's wrong with that?
>>>>>>
>>>>>> Peter.


Re: The King of all Bugs

Posted by Bryan Thompson <br...@systap.com>.
All you need to do is dup the buffer. That gives you independent fields for position and limit, which is what is not thread safe.  You do need a synchronization section around that dup, but that is a very fast operation.  You can also use the read only view of the buffer. Bryan

> On May 14, 2014, at 6:06 AM, Peter Firmstone <ji...@zeus.net.au> wrote:
> 
> Hmm, it already does that, I wonder if this is safe for direct ByteBuffer's?
> 
> Visibility is one issue, the second is mutual exclusion.  I think mutual exclusion is ok, not sure about visibility.
> 
>> On 14/05/2014 8:01 PM, Peter Firmstone wrote:
>> Actually an alternative could be to use future send for all transfers, except for eof, which can be safely used for async send since the byte buffer doesn't change after eof.
>> 
>>> On 14/05/2014 7:55 PM, Peter Firmstone wrote:
>>> There are two methods of transfer, one, future send, where the original thread waits to be notified that the receiving thread has finished processing the ByteBuffer, this appears to be safely transferred.  This is for server mode.
>>> 
>>> Then there is async send, this is for client mode, here the calling thread doesn't wait for the receiving thread.
>>> 
>>> Have a look in com.sun.jini.jeri.internal.mux.Session, here the OutputStream uses a ByteBuffer which it shares via async send.
>>> 
>>> I think the answer is to defensively copy the async send ByteBuffer.
>>> 
>>> 
>>> 
>>> 
>>> 
>>>> On 14/05/2014 7:14 PM, Patricia Shanahan wrote:
>>>> ByteBuffer is a subclass of Buffer, whose documentation says, under "Thread Safety", "Buffers are not safe for use by multiple concurrent threads. If a buffer is to be used by more than one thread then access to the buffer should be controlled by appropriate synchronization."
>>>> 
>>>> Is access controlled by appropriate synchronization? How are the buffers communicated between the threads?
>>>> 
>>>> Patricia
>>>> 
>>>>> On 5/14/2014 2:03 AM, Peter Firmstone wrote:
>>>>> One of the things I like about JERI is it's multiplexing and multithreaded.
>>>>> 
>>>>> What I don't like about JERI is, it passes ByteBuffers between calling
>>>>> threads and pool threads.
>>>>> 
>>>>> Who can guess what's wrong with that?
>>>>> 
>>>>> Peter.
> 

Re: The King of all Bugs

Posted by Peter Firmstone <ji...@zeus.net.au>.
Hmm, it already does that, I wonder if this is safe for direct ByteBuffer's?

Visibility is one issue, the second is mutual exclusion.  I think mutual 
exclusion is ok, not sure about visibility.

On 14/05/2014 8:01 PM, Peter Firmstone wrote:
> Actually an alternative could be to use future send for all transfers, 
> except for eof, which can be safely used for async send since the byte 
> buffer doesn't change after eof.
>
> On 14/05/2014 7:55 PM, Peter Firmstone wrote:
>> There are two methods of transfer, one, future send, where the 
>> original thread waits to be notified that the receiving thread has 
>> finished processing the ByteBuffer, this appears to be safely 
>> transferred.  This is for server mode.
>>
>> Then there is async send, this is for client mode, here the calling 
>> thread doesn't wait for the receiving thread.
>>
>> Have a look in com.sun.jini.jeri.internal.mux.Session, here the 
>> OutputStream uses a ByteBuffer which it shares via async send.
>>
>> I think the answer is to defensively copy the async send ByteBuffer.
>>
>>
>>
>>
>>
>> On 14/05/2014 7:14 PM, Patricia Shanahan wrote:
>>> ByteBuffer is a subclass of Buffer, whose documentation says, under 
>>> "Thread Safety", "Buffers are not safe for use by multiple 
>>> concurrent threads. If a buffer is to be used by more than one 
>>> thread then access to the buffer should be controlled by appropriate 
>>> synchronization."
>>>
>>> Is access controlled by appropriate synchronization? How are the 
>>> buffers communicated between the threads?
>>>
>>> Patricia
>>>
>>> On 5/14/2014 2:03 AM, Peter Firmstone wrote:
>>>> One of the things I like about JERI is it's multiplexing and 
>>>> multithreaded.
>>>>
>>>> What I don't like about JERI is, it passes ByteBuffers between calling
>>>> threads and pool threads.
>>>>
>>>> Who can guess what's wrong with that?
>>>>
>>>> Peter.
>>
>


Re: The King of all Bugs

Posted by Peter Firmstone <ji...@zeus.net.au>.
Actually an alternative could be to use future send for all transfers, 
except for eof, which can be safely used for async send since the byte 
buffer doesn't change after eof.

On 14/05/2014 7:55 PM, Peter Firmstone wrote:
> There are two methods of transfer, one, future send, where the 
> original thread waits to be notified that the receiving thread has 
> finished processing the ByteBuffer, this appears to be safely 
> transferred.  This is for server mode.
>
> Then there is async send, this is for client mode, here the calling 
> thread doesn't wait for the receiving thread.
>
> Have a look in com.sun.jini.jeri.internal.mux.Session, here the 
> OutputStream uses a ByteBuffer which it shares via async send.
>
> I think the answer is to defensively copy the async send ByteBuffer.
>
>
>
>
>
> On 14/05/2014 7:14 PM, Patricia Shanahan wrote:
>> ByteBuffer is a subclass of Buffer, whose documentation says, under 
>> "Thread Safety", "Buffers are not safe for use by multiple concurrent 
>> threads. If a buffer is to be used by more than one thread then 
>> access to the buffer should be controlled by appropriate 
>> synchronization."
>>
>> Is access controlled by appropriate synchronization? How are the 
>> buffers communicated between the threads?
>>
>> Patricia
>>
>> On 5/14/2014 2:03 AM, Peter Firmstone wrote:
>>> One of the things I like about JERI is it's multiplexing and 
>>> multithreaded.
>>>
>>> What I don't like about JERI is, it passes ByteBuffers between calling
>>> threads and pool threads.
>>>
>>> Who can guess what's wrong with that?
>>>
>>> Peter.
>


Re: The King of all Bugs

Posted by Peter Firmstone <ji...@zeus.net.au>.
There are two methods of transfer, one, future send, where the original 
thread waits to be notified that the receiving thread has finished 
processing the ByteBuffer, this appears to be safely transferred.  This 
is for server mode.

Then there is async send, this is for client mode, here the calling 
thread doesn't wait for the receiving thread.

Have a look in com.sun.jini.jeri.internal.mux.Session, here the 
OutputStream uses a ByteBuffer which it shares via async send.

I think the answer is to defensively copy the async send ByteBuffer.





On 14/05/2014 7:14 PM, Patricia Shanahan wrote:
> ByteBuffer is a subclass of Buffer, whose documentation says, under 
> "Thread Safety", "Buffers are not safe for use by multiple concurrent 
> threads. If a buffer is to be used by more than one thread then access 
> to the buffer should be controlled by appropriate synchronization."
>
> Is access controlled by appropriate synchronization? How are the 
> buffers communicated between the threads?
>
> Patricia
>
> On 5/14/2014 2:03 AM, Peter Firmstone wrote:
>> One of the things I like about JERI is it's multiplexing and 
>> multithreaded.
>>
>> What I don't like about JERI is, it passes ByteBuffers between calling
>> threads and pool threads.
>>
>> Who can guess what's wrong with that?
>>
>> Peter.


Re: The King of all Bugs

Posted by Patricia Shanahan <pa...@acm.org>.
ByteBuffer is a subclass of Buffer, whose documentation says, under 
"Thread Safety", "Buffers are not safe for use by multiple concurrent 
threads. If a buffer is to be used by more than one thread then access 
to the buffer should be controlled by appropriate synchronization."

Is access controlled by appropriate synchronization? How are the buffers 
communicated between the threads?

Patricia

On 5/14/2014 2:03 AM, Peter Firmstone wrote:
> One of the things I like about JERI is it's multiplexing and multithreaded.
>
> What I don't like about JERI is, it passes ByteBuffers between calling
> threads and pool threads.
>
> Who can guess what's wrong with that?
>
> Peter.