NEWBIE : Writing Protocol Codecs

[Linz RB <li...@gmail.com>]

Just looking for some confirmation on my thoughts (and understanding) of
writing ProtocolCodecs
Before I start - congratulations to the MINA team - MINA 2,0 is great and
the separation of repsonsibilities is a joy to work with.

I am primarily concerned with the construction of decoder codecs (as
converting a POJO to an IO Buffer is straight forward).

I have three protocols that I am looking at - and planning to convert to
using MINA. These are DHCP, TFTP, and a bespoke protocol.

So DHCP:
I took a look at the codec provided for Apache DS - but was not confident
that it would work. The decoder codec seems to assume that an entire DHCP
message would be present in the IOBuffer. But from what I have read - you
cannot make this assumption.

Have I understoodd this correctly ?

So I have implemented a version that uses an internal state machine (not the
mina-statemachine - learning one thing at a time!) that process as much of
the received DHCP message as possible in stages : for example - process the
'static' part (first 236 bytes), then the magic cookie and lastly options
(variable length).

So TFTP:
Again - I have read the 'static' portion of the message header - and then
process the rest of the message.
The rest of the message is 0 byte terminated - so I have to look at the
IOBuffer - at the limit() position for a 0 - which indicates that the entire
message is available and can be processed (since TFTP is a strict handshake
style protocol I do no have to worry about a second message in the buffer -
I think !!)


How am I doing so far ? Is my approach incorrect - and not what MINA gurus
would recommend.

I am getting the feeling that it would be helpful to find a way to describe
a protocol and then provide a codec framework that is a close fit - is this
intention of MINA ? - so, codecs for synchronous (handshake) based protocols
(where you can only every expect a single message per IOBuffer)  v
asynchronous (streaming-like) protocols (where you may find more than one
message per IOBUffer).

Many thanks in advance - and keep up the great work.

Re: NEWBIE : Writing Protocol Codecs

[Linz RB <li...@gmail.com>]

Thanks Emmanuel

A great confidence booster.

>> There are a few interesting approaches so far to solve this problem, and
some peeps have developped some Codec generators. >> One of them is based on
Google Protocol Buffer (thomasz blachowicz).

I will go an Google this and see what they are up to

>> I have also thought that we should have an ASN.1 compiler generating code
for any kind of codec, but it's a very long task ...
This would be fantastic - but I can imagine a full project in its own right.

>> Last, not least, we discussed a bit about creating a codec library,
containing the most well known protocols (as we already
>> have a few of them, it should not be a problem)

Seems a simple proposal. Buf as the decoders are tied to the POJO
representing the message - would the POJO structure alos have to be put in
the library ?

Thanks for the help and insight - I will continue my MINA learning curve !



On Tue, Jun 30, 2009 at 10:40 AM, Emmanuel Lecharny <el...@apache.org>wrote:

> Linz RB wrote:
>
>> Just looking for some confirmation on my thoughts (and understanding) of
>> writing ProtocolCodecs
>> Before I start - congratulations to the MINA team - MINA 2,0 is great and
>> the separation of repsonsibilities is a joy to work with.
>>
>> I am primarily concerned with the construction of decoder codecs (as
>> converting a POJO to an IO Buffer is straight forward).
>>
>> I have three protocols that I am looking at - and planning to convert to
>> using MINA. These are DHCP, TFTP, and a bespoke protocol.
>>
>> So DHCP:
>> I took a look at the codec provided for Apache DS - but was not confident
>> that it would work. The decoder codec seems to assume that an entire DHCP
>> message would be present in the IOBuffer. But from what I have read - you
>> cannot make this assumption.
>>
>> Have I understoodd this correctly ?
>>
>>
>
> I suspect you are right. I have not checked the ApacheDS DHCP codec though,
> so I trust you on that.
>
>> So I have implemented a version that uses an internal state machine (not
>> the
>> mina-statemachine - learning one thing at a time!) that process as much of
>> the received DHCP message as possible in stages : for example - process
>> the
>> 'static' part (first 236 bytes), then the magic cookie and lastly options
>> (variable length).
>>
>>
> Sounds the way to go.
>
>> So TFTP:
>> Again - I have read the 'static' portion of the message header - and then
>> process the rest of the message.
>> The rest of the message is 0 byte terminated - so I have to look at the
>> IOBuffer - at the limit() position for a 0 - which indicates that the
>> entire
>> message is available and can be processed (since TFTP is a strict
>> handshake
>> style protocol I do no have to worry about a second message in the buffer
>> -
>> I think !!)
>>
>>
> Sounds ok too
>
>>
>> How am I doing so far ? Is my approach incorrect - and not what MINA gurus
>> would recommend.
>>
>>
> Well, that's the way I would have handled those protocol. May be someone
> else has a better idea ?
>
>> I am getting the feeling that it would be helpful to find a way to
>> describe
>> a protocol and then provide a codec framework that is a close fit - is
>> this
>> intention of MINA ? - so, codecs for synchronous (handshake) based
>> protocols
>> (where you can only every expect a single message per IOBuffer)  v
>> asynchronous (streaming-like) protocols (where you may find more than one
>> message per IOBUffer).
>>
>>
> There are a few interesting approaches so far to solve this problem, and
> some peeps have developped some Codec generators. One of them is based on
> Google Protocol Buffer (thomasz blachowicz).
> I have also thought that we should have an ASN.1 compiler generating code
> for any kind of codec, but it's a very long task ...
> Last, not least, we discussed a bit about creating a codec library,
> containing the most well known protocols (as we already have a few of them,
> it should not be a problem)
>
> So this is still an open area, where those who want to play can join us !
>
> --
> --
> cordialement, regards,
> Emmanuel Lécharny
> www.iktek.com
> directory.apache.org
>
>
>

Re: NEWBIE : Writing Protocol Codecs

[Emmanuel Lecharny <el...@apache.org>]

Linz RB wrote:
> Just looking for some confirmation on my thoughts (and understanding) of
> writing ProtocolCodecs
> Before I start - congratulations to the MINA team - MINA 2,0 is great and
> the separation of repsonsibilities is a joy to work with.
>
> I am primarily concerned with the construction of decoder codecs (as
> converting a POJO to an IO Buffer is straight forward).
>
> I have three protocols that I am looking at - and planning to convert to
> using MINA. These are DHCP, TFTP, and a bespoke protocol.
>
> So DHCP:
> I took a look at the codec provided for Apache DS - but was not confident
> that it would work. The decoder codec seems to assume that an entire DHCP
> message would be present in the IOBuffer. But from what I have read - you
> cannot make this assumption.
>
> Have I understoodd this correctly ?
>   

I suspect you are right. I have not checked the ApacheDS DHCP codec 
though, so I trust you on that.
> So I have implemented a version that uses an internal state machine (not the
> mina-statemachine - learning one thing at a time!) that process as much of
> the received DHCP message as possible in stages : for example - process the
> 'static' part (first 236 bytes), then the magic cookie and lastly options
> (variable length).
>   
Sounds the way to go.
> So TFTP:
> Again - I have read the 'static' portion of the message header - and then
> process the rest of the message.
> The rest of the message is 0 byte terminated - so I have to look at the
> IOBuffer - at the limit() position for a 0 - which indicates that the entire
> message is available and can be processed (since TFTP is a strict handshake
> style protocol I do no have to worry about a second message in the buffer -
> I think !!)
>   
Sounds ok too
>
> How am I doing so far ? Is my approach incorrect - and not what MINA gurus
> would recommend.
>   
Well, that's the way I would have handled those protocol. May be someone 
else has a better idea ?
> I am getting the feeling that it would be helpful to find a way to describe
> a protocol and then provide a codec framework that is a close fit - is this
> intention of MINA ? - so, codecs for synchronous (handshake) based protocols
> (where you can only every expect a single message per IOBuffer)  v
> asynchronous (streaming-like) protocols (where you may find more than one
> message per IOBUffer).
>   
There are a few interesting approaches so far to solve this problem, and 
some peeps have developped some Codec generators. One of them is based 
on Google Protocol Buffer (thomasz blachowicz).
I have also thought that we should have an ASN.1 compiler generating 
code for any kind of codec, but it's a very long task ...
Last, not least, we discussed a bit about creating a codec library, 
containing the most well known protocols (as we already have a few of 
them, it should not be a problem)

So this is still an open area, where those who want to play can join us !

-- 
--
cordialement, regards,
Emmanuel Lécharny
www.iktek.com
directory.apache.org