You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@mina.apache.org by tunca <at...@gmail.com> on 2016/03/01 15:05:44 UTC
Re: CumulativeProtocolDecoder and UDP
Hi Trustin,
We're using Mina Core 2.0.9.
The Udp Cumulative Protocol doesn't seem to accumulate.
The Decoder is working with TCP however, with UDP the
The size of the IoBuffer doesn't increase.
Best Regards
Tunca
--
View this message in context: http://apache-mina.10907.n7.nabble.com/CumulativeProtocolDecoder-and-UDP-tp18927p50272.html
Sent from the Apache MINA Developer Forum mailing list archive at Nabble.com.
Re: CumulativeProtocolDecoder and UDP
Posted by Chad Beaulac <ca...@gmail.com>.
UDP datagrams are fragmented on MTU size. So if your messages are bigger than MTU size, you need to handle multiple callbacks for MTU size in your decoder. So, it doesn't work very well with UDP.
Sent from my iPhone
> On Mar 1, 2016, at 11:37 AM, Ashish <pa...@gmail.com> wrote:
>
> TCP handles fragmentation at its level, but for UDP you have to do it
> at application layer meaning UDP data has to carry message sequences
> and then you merge them at receiving end. Here you packets can come in
> different order do you got to keep them somewhere before complete
> message is constructed.
>
>
>
>> On Tue, Mar 1, 2016 at 7:58 AM, tunca <at...@gmail.com> wrote:
>> Our customer has strange requirement about merging multiple udp/TCP packages
>> to create a single message.
>> There is a well defined protocol that defines message boundaries.
>> The decoder is working good with TCP packages. It can create a single
>> message from multiple TCP packages.
>> However when a message is fragmented into multiple packages the do doDecode
>> method always gives the same ioBuffer.
>> I'll try 2.0.13 next day.
>> Thanks
>>
>>
>>
>>
>> --
>> View this message in context: http://apache-mina.10907.n7.nabble.com/CumulativeProtocolDecoder-and-UDP-tp18927p50274.html
>> Sent from the Apache MINA Developer Forum mailing list archive at Nabble.com.
>
>
>
> --
> thanks
> ashish
>
> Blog: http://www.ashishpaliwal.com/blog
> My Photo Galleries: http://www.pbase.com/ashishpaliwal
Re: CumulativeProtocolDecoder and UDP
Posted by Ashish <pa...@gmail.com>.
Agree, that's why the protocol shift to Application layer, meaning you
emulate what TCP does like how many packets and their sequence to
order them at receiving layer for meaningful interpretation.
On Tue, Mar 1, 2016 at 11:01 AM, Mondain <mo...@gmail.com> wrote:
> Ashish, just be aware that with UDP, some of your data may never arrive;
> that's the nature of the protocol.
>
> Paul
>
> On Tue, Mar 1, 2016 at 1:48 PM Ashish <pa...@gmail.com> wrote:
>
>> TCP handles fragmentation at its level, but for UDP you have to do it
>> at application layer meaning UDP data has to carry message sequences
>> and then you merge them at receiving end. Here you packets can come in
>> different order do you got to keep them somewhere before complete
>> message is constructed.
>>
>>
>>
>> On Tue, Mar 1, 2016 at 7:58 AM, tunca <at...@gmail.com> wrote:
>> > Our customer has strange requirement about merging multiple udp/TCP
>> packages
>> > to create a single message.
>> > There is a well defined protocol that defines message boundaries.
>> > The decoder is working good with TCP packages. It can create a single
>> > message from multiple TCP packages.
>> > However when a message is fragmented into multiple packages the do
>> doDecode
>> > method always gives the same ioBuffer.
>> > I'll try 2.0.13 next day.
>> > Thanks
>> >
>> >
>> >
>> >
>> > --
>> > View this message in context:
>> http://apache-mina.10907.n7.nabble.com/CumulativeProtocolDecoder-and-UDP-tp18927p50274.html
>> > Sent from the Apache MINA Developer Forum mailing list archive at
>> Nabble.com.
>>
>>
>>
>> --
>> thanks
>> ashish
>>
>> Blog: http://www.ashishpaliwal.com/blog
>> My Photo Galleries: http://www.pbase.com/ashishpaliwal
>>
--
thanks
ashish
Blog: http://www.ashishpaliwal.com/blog
My Photo Galleries: http://www.pbase.com/ashishpaliwal
Re: CumulativeProtocolDecoder and UDP
Posted by Mondain <mo...@gmail.com>.
Ashish, just be aware that with UDP, some of your data may never arrive;
that's the nature of the protocol.
Paul
On Tue, Mar 1, 2016 at 1:48 PM Ashish <pa...@gmail.com> wrote:
> TCP handles fragmentation at its level, but for UDP you have to do it
> at application layer meaning UDP data has to carry message sequences
> and then you merge them at receiving end. Here you packets can come in
> different order do you got to keep them somewhere before complete
> message is constructed.
>
>
>
> On Tue, Mar 1, 2016 at 7:58 AM, tunca <at...@gmail.com> wrote:
> > Our customer has strange requirement about merging multiple udp/TCP
> packages
> > to create a single message.
> > There is a well defined protocol that defines message boundaries.
> > The decoder is working good with TCP packages. It can create a single
> > message from multiple TCP packages.
> > However when a message is fragmented into multiple packages the do
> doDecode
> > method always gives the same ioBuffer.
> > I'll try 2.0.13 next day.
> > Thanks
> >
> >
> >
> >
> > --
> > View this message in context:
> http://apache-mina.10907.n7.nabble.com/CumulativeProtocolDecoder-and-UDP-tp18927p50274.html
> > Sent from the Apache MINA Developer Forum mailing list archive at
> Nabble.com.
>
>
>
> --
> thanks
> ashish
>
> Blog: http://www.ashishpaliwal.com/blog
> My Photo Galleries: http://www.pbase.com/ashishpaliwal
>
Re: CumulativeProtocolDecoder and UDP
Posted by Ashish <pa...@gmail.com>.
TCP handles fragmentation at its level, but for UDP you have to do it
at application layer meaning UDP data has to carry message sequences
and then you merge them at receiving end. Here you packets can come in
different order do you got to keep them somewhere before complete
message is constructed.
On Tue, Mar 1, 2016 at 7:58 AM, tunca <at...@gmail.com> wrote:
> Our customer has strange requirement about merging multiple udp/TCP packages
> to create a single message.
> There is a well defined protocol that defines message boundaries.
> The decoder is working good with TCP packages. It can create a single
> message from multiple TCP packages.
> However when a message is fragmented into multiple packages the do doDecode
> method always gives the same ioBuffer.
> I'll try 2.0.13 next day.
> Thanks
>
>
>
>
> --
> View this message in context: http://apache-mina.10907.n7.nabble.com/CumulativeProtocolDecoder-and-UDP-tp18927p50274.html
> Sent from the Apache MINA Developer Forum mailing list archive at Nabble.com.
--
thanks
ashish
Blog: http://www.ashishpaliwal.com/blog
My Photo Galleries: http://www.pbase.com/ashishpaliwal
Re: CumulativeProtocolDecoder and UDP
Posted by tunca <at...@gmail.com>.
https://issues.apache.org/jira/browse/DIRMINA-1033
is submitted.
--
View this message in context: http://apache-mina.10907.n7.nabble.com/CumulativeProtocolDecoder-and-UDP-tp18927p50292.html
Sent from the Apache MINA Developer Forum mailing list archive at Nabble.com.
Re: CumulativeProtocolDecoder and UDP
Posted by Emmanuel Lécharny <el...@gmail.com>.
Le 03/03/16 09:09, tunca a écrit :
> Hi Emmanuel,
>
> I downloaded apache mina 2.0.13 and updated NioDatagramSession METADATA
> creation code.
> Set the 4th parameter to true, then for UDP CumulativeProtocolDecoder
> started to work as expected.
Indeed.
Although doing so you create a dependency on a patched version of MINA.
That means if someone in the future switch to a newer version of MINA,
your code will simply break.
Here is what I suggest : fill a JIRA asking for exposing the MetaData
parameter, allowing teh developper (you, here) to set the flag to
something else than the defautl value.
Re: CumulativeProtocolDecoder and UDP
Posted by tunca <at...@gmail.com>.
Hi Emmanuel,
I downloaded apache mina 2.0.13 and updated NioDatagramSession METADATA
creation code.
Set the 4th parameter to true, then for UDP CumulativeProtocolDecoder
started to work as expected.
Thanks A lot.
class NioDatagramSession extends NioSession {
static final TransportMetadata METADATA = new
DefaultTransportMetadata("nio", "datagram", true, true,
InetSocketAddress.class, DatagramSessionConfig.class,
IoBuffer.class);
--
View this message in context: http://apache-mina.10907.n7.nabble.com/CumulativeProtocolDecoder-and-UDP-tp18927p50289.html
Sent from the Apache MINA Developer Forum mailing list archive at Nabble.com.
Re: CumulativeProtocolDecoder and UDP
Posted by Emmanuel Lécharny <el...@gmail.com>.
Le 03/03/16 08:29, tunca a écrit :
> Hi Emmanuel,
>
> Thanks a lot, you find out the root cause of the problem I believe.
>
> I created my own version of CumulativeProtocolDecoder
> (http://pastebin.com/m0qmbEVS) and as you suggested removed the has
> fragmentation check.
> It still doesn't accumulate, however the second package arrives without
> accumulating for the same session.
Remove completely the initial check, includinhg the call to doDecode(),
otherwise it will be called twice, consuming the buffer.
Re: CumulativeProtocolDecoder and UDP
Posted by tunca <at...@gmail.com>.
Hi Emmanuel,
Thanks a lot, you find out the root cause of the problem I believe.
I created my own version of CumulativeProtocolDecoder
(http://pastebin.com/m0qmbEVS) and as you suggested removed the has
fragmentation check.
It still doesn't accumulate, however the second package arrives without
accumulating for the same session.
Thanks
Tunca
--
View this message in context: http://apache-mina.10907.n7.nabble.com/CumulativeProtocolDecoder-and-UDP-tp18927p50287.html
Sent from the Apache MINA Developer Forum mailing list archive at Nabble.com.
Re: CumulativeProtocolDecoder and UDP
Posted by Emmanuel Lécharny <el...@gmail.com>.
Le 02/03/16 07:38, tunca a écrit :
> Hi Emmanuel.
> http://pastebin.com/embed_iframe/EpL4WE4b
> This is the link to my doDecode method.
>
> A complete message starts with x01 and ends with x03.
> For test I send 2 udp messages.
> First Udp message starts with x01
> Second Udp messages ends with x03.
>
> When I send first message doDecode message is called with the content of the
> first Udp message.
> the doDecode method returns false as the message is not complete.
>
> Then the second Udp message is sent.
> However doDecode method is still called with only First Udp message's
> content from in IoBuffer parameter.
Ok, I think I see what's going on. The
CumulativeProtocolDecoder.decode() method starts with :
public void decode(IoSession session, IoBuffer in,
ProtocolDecoderOutput out) throws Exception {
if (!session.getTransportMetadata().hasFragmentation()) {
while (in.hasRemaining()) {
if (!doDecode(session, in, out)) {
break;
}
}
return;
}
and for a UDP session, we initialize the transport metadata this way :
static final TransportMetadata METADATA = new
DefaultTransportMetadata("nio", "datagram", true, false,
InetSocketAddress.class, DatagramSessionConfig.class,
IoBuffer.class);
where the forth parameter is 'boolean fragmentation' :
public DefaultTransportMetadata(String providerName, String name,
boolean connectionless, boolean fragmentation,
Class<? extends SocketAddress> addressType, Class<? extends
IoSessionConfig> sessionConfigType,
Class<?>... envelopeTypes) {
It's set to 'false', so the decode method simply don't accumulate the
data in one single buffer.
There is no way to change the flag once the metadata instance has been
created.
What you can do is to define your own version of the
CumulativeProtocolDecoder, copying the original code and simply discard
the test on the transportMetadata.
Re: CumulativeProtocolDecoder and UDP
Posted by tunca <at...@gmail.com>.
Hi Emmanuel.
http://pastebin.com/embed_iframe/EpL4WE4b
This is the link to my doDecode method.
A complete message starts with x01 and ends with x03.
For test I send 2 udp messages.
First Udp message starts with x01
Second Udp messages ends with x03.
When I send first message doDecode message is called with the content of the
first Udp message.
the doDecode method returns false as the message is not complete.
Then the second Udp message is sent.
However doDecode method is still called with only First Udp message's
content from in IoBuffer parameter.
Tunca
--
View this message in context: http://apache-mina.10907.n7.nabble.com/CumulativeProtocolDecoder-and-UDP-tp18927p50281.html
Sent from the Apache MINA Developer Forum mailing list archive at Nabble.com.
Re: CumulativeProtocolDecoder and UDP
Posted by Emmanuel Lécharny <el...@gmail.com>.
Le 01/03/16 16:58, tunca a écrit :
> Our customer has strange requirement about merging multiple udp/TCP packages
> to create a single message.
> There is a well defined protocol that defines message boundaries.
> The decoder is working good with TCP packages. It can create a single
> message from multiple TCP packages.
> However when a message is fragmented into multiple packages the do doDecode
> method always gives the same ioBuffer.
Define fragmented in a UDP context : how do you manage the packet ordering ?
> I'll try 2.0.13 next day.
It won't solve your issue. You most certainly handling the incoming
packets incorrectly somewhere in your IoHandler. Typically, you are not
in the same session when 2 UDP packets are coming, thus the IoBuffer,
which is stored into the session, is different for each incoming packet.
Re: CumulativeProtocolDecoder and UDP
Posted by tunca <at...@gmail.com>.
Our customer has strange requirement about merging multiple udp/TCP packages
to create a single message.
There is a well defined protocol that defines message boundaries.
The decoder is working good with TCP packages. It can create a single
message from multiple TCP packages.
However when a message is fragmented into multiple packages the do doDecode
method always gives the same ioBuffer.
I'll try 2.0.13 next day.
Thanks
--
View this message in context: http://apache-mina.10907.n7.nabble.com/CumulativeProtocolDecoder-and-UDP-tp18927p50274.html
Sent from the Apache MINA Developer Forum mailing list archive at Nabble.com.
Re: CumulativeProtocolDecoder and UDP
Posted by Emmanuel Lécharny <el...@gmail.com>.
Le 01/03/16 15:05, tunca a écrit :
> Hi Trustin,
> We're using Mina Core 2.0.9.
Have you checked with 2.0.13 ?
> The Udp Cumulative Protocol doesn't seem to accumulate.
> The Decoder is working with TCP however, with UDP the
> The size of the IoBuffer doesn't increase.
Using such a protocol decoder does not make any sense in UDP : either
you receive the full message, or you receive nothing. Moreover, there is
no guarantee that a message N will arrive *before* a message N+1 in UDP.
What exactly are you trying to do ?