You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@mina.apache.org by hezjing <he...@gmail.com> on 2009/01/09 08:22:17 UTC

How to check UDP message header length?

Hi
I'm trying to implement an UDP message that has a fixed 6 byte of header,
where the 3rd and 4th byte are the size of the message payload.

CumulativeProtocolDecoder is the choice since the header's size fixed, and
there is a payload length information (correct?).
I will have to first read the 6 bytes header, and continue to read the
subsequent bytes as specified by the payload size.

For example, the following is the IoBuffer printed in doDecode():

HeapBuffer[pos=0 lim=20 cap=20: 0E 01 00 00 00 DB 00 00 00 00 00 00 00 00 00
00...]


The IoBuffer has the limit() , capacity()  and remaining() of 20 bytes.


The question is, how do we know the header is ready to be parsed?


-- 

Hez

Re: How to check UDP message header length?

Posted by Squee <sq...@gmail.com>.
Unless you're implementing a further TCP-like protocol on top of UDP
(rarely a good idea, except with situations where UDP is the only
option), you're stuck with only single UDP packets reliably. It's far
better to use TCP if you have a choice; then you don't have to worry
about it in MINA and can just use the Cumultative filter.

If you answer the question "Can this app afford to lose data in
transmission at any point?" is a no, then 90% of the time you should
stick with TCP, even if you get a performance improvement from UDP.
Making a TCP-like protocol on top of UDP is something to be avoided.

On Sun, Jan 11, 2009 at 8:44 PM, Barrie Treloar <ba...@gmail.com> wrote:
>>> With MINA, how do we send a UDP message in one, and in multiple packets?
>> Well at this point, keep life simple, and let the stack handle it. You
>> just write data and internally stack shall
>> take care of it. For UDP, max data that can be send is 64K. So, if
>> ethernet MTU is 1500, you packet shall be fragmented
>> and reassembled at destination, and passed on as single packet back to
>> application layer. Beyond this you will run into issues of sequencing,
>> packet losses etc, so can prototype with smaller packets and take a
>> call later.
>
> As noted above, your application will see one UDP packet - regardless
> of data size (up to max).
>
> HOWEVER - if the stack is fragmenting the packets there is a chance
> that you *lose* a packet and in this case you lose the whole message.
>
> The morale of the story is not to use UDP for large message sizes or
> else you end up wasting bandwidth (a lost large packet = bandwidth
> waste)
>
> If you must use UDP and you have large packet sizes (our application
> does) then you must re-implement some of the TCP/IP features yourself.
> In our case we are manually fragmenting the packet into small chunks
> so that if we lose a packet we dont lose the whole message, we only
> lose the current packet which is much cheaper to resend.
> We are also implementing a packet ack protocol so we get guaranteed messages.
>
> We are stuck with this because we are sending messages over a Motorola
> radio network (~14kbps) and TCP/IP has too much overhead to run over
> this.
>

Re: How to check UDP message header length?

Posted by Barrie Treloar <ba...@gmail.com>.
>> With MINA, how do we send a UDP message in one, and in multiple packets?
> Well at this point, keep life simple, and let the stack handle it. You
> just write data and internally stack shall
> take care of it. For UDP, max data that can be send is 64K. So, if
> ethernet MTU is 1500, you packet shall be fragmented
> and reassembled at destination, and passed on as single packet back to
> application layer. Beyond this you will run into issues of sequencing,
> packet losses etc, so can prototype with smaller packets and take a
> call later.

As noted above, your application will see one UDP packet - regardless
of data size (up to max).

HOWEVER - if the stack is fragmenting the packets there is a chance
that you *lose* a packet and in this case you lose the whole message.

The morale of the story is not to use UDP for large message sizes or
else you end up wasting bandwidth (a lost large packet = bandwidth
waste)

If you must use UDP and you have large packet sizes (our application
does) then you must re-implement some of the TCP/IP features yourself.
In our case we are manually fragmenting the packet into small chunks
so that if we lose a packet we dont lose the whole message, we only
lose the current packet which is much cheaper to resend.
We are also implementing a packet ack protocol so we get guaranteed messages.

We are stuck with this because we are sending messages over a Motorola
radio network (~14kbps) and TCP/IP has too much overhead to run over
this.

Re: How to check UDP message header length?

Posted by Ashish <pa...@gmail.com>.
On Sun, Jan 11, 2009 at 10:19 PM, hezjing <he...@gmail.com> wrote:
> Hi
>
> I have to admit that I'm new to networking application, and I'm not sure if
> a protocol uses one or multiple packets.
> In my case, the UDP message has a fixed 6 bytes of header, and the 3rd and
> 4th byte are the size of the message payload.
> Can we tell from here if a UDP message uses one or multiple packets?

Don't worry about fragmentation, let TCP/IP stack handle it. This is
what you should do.
Once you receive, the packet
1. get the length
2. check if you have got the complete data based on length
3. if not wait for more (probably won't be the case most of the time)
4. Decode the message and process

> With MINA, how do we send a UDP message in one, and in multiple packets?
Well at this point, keep life simple, and let the stack handle it. You
just write data and internally stack shall
take care of it. For UDP, max data that can be send is 64K. So, if
ethernet MTU is 1500, you packet shall be fragmented
and reassembled at destination, and passed on as single packet back to
application layer. Beyond this you will run into issues of sequencing,
packet losses etc, so can prototype with smaller packets and take a
call later.

BTW, use Logging filter as first filter in the filter chain and see
the logs. It will give a good insight into the inner working.

Re: How to check UDP message header length?

Posted by hezjing <he...@gmail.com>.
Hi

I have to admit that I'm new to networking application, and I'm not sure if
a protocol uses one or multiple packets.

In my case, the UDP message has a fixed 6 bytes of header, and the 3rd and
4th byte are the size of the message payload.
Can we tell from here if a UDP message uses one or multiple packets?


With MINA, how do we send a UDP message in one, and in multiple packets?



On Sun, Jan 11, 2009 at 2:44 PM, Ashish <pa...@gmail.com> wrote:

> On Sun, Jan 11, 2009 at 11:27 AM, hezjing <he...@gmail.com> wrote:
> > Hi
> > I'm continuing this discussion from the developer mailing-list ...
> >
> > Is the CumulativeProtocolDecoder is really needed just for TCP (as
> mentioned
> > by Eero)?
>
>
> 1. If your protocol uses just one UDP packet, then u don't need
> CumulativeProtocolDecoder :-)
> 2. If you have a protocol in which you send data on UDP, nd in
> multiple packets, you need this kind of Decoder.
>



-- 

Hez

Re: How to check UDP message header length?

Posted by Ashish <pa...@gmail.com>.
On Sun, Jan 11, 2009 at 11:27 AM, hezjing <he...@gmail.com> wrote:
> Hi
> I'm continuing this discussion from the developer mailing-list ...
>
> Is the CumulativeProtocolDecoder is really needed just for TCP (as mentioned
> by Eero)?


1. If your protocol uses just one UDP packet, then u don't need
CumulativeProtocolDecoder :-)
2. If you have a protocol in which you send data on UDP, nd in
multiple packets, you need this kind of Decoder.

Re: How to check UDP message header length?

Posted by hezjing <he...@gmail.com>.
Hi
I'm continuing this discussion from the developer mailing-list ...

Is the CumulativeProtocolDecoder is really needed just for TCP (as mentioned
by Eero)?



On Fri, Jan 9, 2009 at 5:33 PM, Eero Nevalainen <eero.nevalainen@indagon.com
> wrote:

> Steve Ulrich wrote:
>
>> hezjing [mailto:hezjing@gmail.com] wrote:
>>>
>>> CumulativeProtocolDecoder is the choice since the header's size fixed,
>>> and
>>> there is a payload length information (correct?).
>>>
>>
>> The CumulativeProtocolDecoder is always a good idea. ;-)
>>
>
> Isn't it really needed just for TCP?
>
>  The question is, how do we know the header is ready to be parsed?
>>>
>>
>> "if (buffer.remaining() >= 6)" should work for you
>>
>
> You don't get partial UDP datagrams, so this won't happen due to network.
> It's good to check though, in case the sender is doing something wrong.
>
>  btw: Please use the user ML as you are having problems using mina for your
>> development.
>> This ml is intended for discussions for further development of mina
>> itself.
>>
>
> Yes, shutting up...
>
> --
>
> Eero
>



-- 

Hez

Re: How to check UDP message header length?

Posted by Eero Nevalainen <ee...@indagon.com>.
Steve Ulrich wrote:
>> hezjing [mailto:hezjing@gmail.com] wrote:
>>
>> CumulativeProtocolDecoder is the choice since the header's size fixed,
>> and
>> there is a payload length information (correct?).
> 
> The CumulativeProtocolDecoder is always a good idea. ;-)

Isn't it really needed just for TCP?

>> The question is, how do we know the header is ready to be parsed?
> 
> "if (buffer.remaining() >= 6)" should work for you

You don't get partial UDP datagrams, so this won't happen due to 
network. It's good to check though, in case the sender is doing 
something wrong.

> btw: Please use the user ML as you are having problems using mina for your development.
> This ml is intended for discussions for further development of mina itself.

Yes, shutting up...

-- 

Eero

RE: How to check UDP message header length?

Posted by Steve Ulrich <st...@proemion.com>.

> hezjing [mailto:hezjing@gmail.com] wrote:
>
> Hi

Hi!

> I'm trying to implement an UDP message that has a fixed 6 byte of
> header,
> where the 3rd and 4th byte are the size of the message payload.

Ok, its common to have a fixed size header.

> CumulativeProtocolDecoder is the choice since the header's size fixed,
> and
> there is a payload length information (correct?).

The CumulativeProtocolDecoder is always a good idea. ;-)

> I will have to first read the 6 bytes header, and continue to read the
> subsequent bytes as specified by the payload size.
>
> For example, the following is the IoBuffer printed in doDecode():
>
> HeapBuffer[pos=0 lim=20 cap=20: 0E 01 00 00 00 DB 00 00 00 00 00 00 00
> 00 00
> 00...]
>
>
> The IoBuffer has the limit() , capacity()  and remaining() of 20 bytes.

The capacity is the maximum number of bytes the buffer can take, without copying internal buffers.
The limit defines up to which position data was filled to the buffer.
The remaining defines how much bytes you can read before you reach limit
Or to say it with signs:
pos <= limit <= capacity
remaining = limit-pos

> The question is, how do we know the header is ready to be parsed?

"if (buffer.remaining() >= 6)" should work for you

hth

btw: Please use the user ML as you are having problems using mina for your development.
This ml is intended for discussions for further development of mina itself.

regards

Steve