You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@mina.apache.org by Julien Vermillard <jv...@gmail.com> on 2012/12/31 07:55:30 UTC
codec "framework"
Hi,
Since few year, I stopped to use the MINA ProtocolCodecFilter and
associated stuff (CumulativeCodec..). for implementing my own codec
independent of MINA.
it's just a service consuming ByteBuffer and pushing decoded POJO in a
callback. The point is to be independent of MINA for example, parse & save
files using the codec, or simply implement an HTTP version of the transport
using old style servlet.
Basically a decoder looks like : https://gist.github.com/4417934
One is instantiated by session.
I'm quite happy with that and I think we should not port the old
ProtocolCodeFilter to MINA 3.0 and replace it with a independent MINA async
decoder framework (consuming BB, accumulating if needed and producing pojo).
Julien
Re: codec "framework"
Posted by Julien Vermillard <jv...@gmail.com>.
Hi and Happy new year (for those the year starts today).
Inline reply
Le 31 déc. 2012 09:02, "Ashish" <pa...@gmail.com> a écrit :
>
> On Mon, Dec 31, 2012 at 12:25 PM, Julien Vermillard
> <jv...@gmail.com>wrote:
>
> > Hi,
> >
> > Since few year, I stopped to use the MINA ProtocolCodecFilter and
> > associated stuff (CumulativeCodec..). for implementing my own codec
> > independent of MINA.
> >
>
> :)
>
>
> >
> > it's just a service consuming ByteBuffer and pushing decoded POJO in a
> > callback. The point is to be independent of MINA for example, parse &
save
> > files using the codec, or simply implement an HTTP version of the
transport
> > using old style servlet.
> >
>
> Agree. The implementation need not be tied with MINA. You can expect
> ByteBuffer(s) and can decode.
> I didn't quite get the statement " or simply implement an HTTP version of
> the transport using old style servlet."?
>
For example some protocols support both plain TCP and HTTP. If you use a
"ProtocolCodecFilter" which is a MINA filter for implementing the TCP
transport it will be tricky to reuse it in a servlet (without MINA) for
implementing an HTTP based version.
> With this approach accumulation and decoding becomes one. We can also
think
> about accumulating as one step and decoding as another, which may be
useful
> for protocols which specify length or fixed headers. User provides how to
> find length, Framework shall accumulate bytes and once done, can pass on
> complete PDU for decoding. This might be an overhead in some cases. The
> idea is to make life easy for the framework user. Or we can provide some
> helper around our core Codec framework. Thoughts?
>
Yes, that's the idea, most UDP based protocols won't need accumulation, but
most of TCP based will need it.
>
> > Basically a decoder looks like : https://gist.github.com/4417934
> > One is instantiated by session.
> >
>
> Would be worth making it accumulateAndDecode()
>
> public interface Decoder<PDU> {
>
> What is PDU? It's the custom decoded POJO or an abstraction that we want
to
> build.
>
Exactly PDU : Protocol Data Unit , your protocol message type.
> >
> > I'm quite happy with that and I think we should not port the old
> > ProtocolCodeFilter to MINA 3.0 and replace it with a independent MINA
async
> > decoder framework (consuming BB, accumulating if needed and producing
> > pojo).
> >
> > Julien
> >
>
> Not sure, but do we see scenarios where we might need reference to
> IoSession, while decoding?
> Nothing comes to my mind right now, but it's year end :)
>
You perhaps need some session attached data
like security information gathered by another filter ? Ideally you should
not depend on the sessionfor decoding, but some protocols are really
horribles:-) . I think we should try to make it independent of the session
and see if it's working for all the cases.
Julien
Re: codec "framework"
Posted by Ashish <pa...@gmail.com>.
On Mon, Dec 31, 2012 at 12:25 PM, Julien Vermillard
<jv...@gmail.com>wrote:
> Hi,
>
> Since few year, I stopped to use the MINA ProtocolCodecFilter and
> associated stuff (CumulativeCodec..). for implementing my own codec
> independent of MINA.
>
:)
>
> it's just a service consuming ByteBuffer and pushing decoded POJO in a
> callback. The point is to be independent of MINA for example, parse & save
> files using the codec, or simply implement an HTTP version of the transport
> using old style servlet.
>
Agree. The implementation need not be tied with MINA. You can expect
ByteBuffer(s) and can decode.
I didn't quite get the statement " or simply implement an HTTP version of
the transport using old style servlet."?
With this approach accumulation and decoding becomes one. We can also think
about accumulating as one step and decoding as another, which may be useful
for protocols which specify length or fixed headers. User provides how to
find length, Framework shall accumulate bytes and once done, can pass on
complete PDU for decoding. This might be an overhead in some cases. The
idea is to make life easy for the framework user. Or we can provide some
helper around our core Codec framework. Thoughts?
> Basically a decoder looks like : https://gist.github.com/4417934
> One is instantiated by session.
>
Would be worth making it accumulateAndDecode()
public interface Decoder<PDU> {
What is PDU? It's the custom decoded POJO or an abstraction that we want to
build.
>
> I'm quite happy with that and I think we should not port the old
> ProtocolCodeFilter to MINA 3.0 and replace it with a independent MINA async
> decoder framework (consuming BB, accumulating if needed and producing
> pojo).
>
> Julien
>
Not sure, but do we see scenarios where we might need reference to
IoSession, while decoding?
Nothing comes to my mind right now, but it's year end :)
--
thanks
ashish
Blog: http://www.ashishpaliwal.com/blog
My Photo Galleries: http://www.pbase.com/ashishpaliwal
Re: codec "framework"
Posted by Julien Vermillard <jv...@gmail.com>.
Here a Codec interface proposal :
https://github.com/jvermillard/mina/tree/trunk/core/src/main/java/org/apache/mina/filter/codec
PS : I'm just toying around with github, don't worry I'm not making my own
fork :)
On Thu, Jan 3, 2013 at 10:20 AM, Arnaud bourree <ar...@gmail.com>wrote:
> You're right, in same time mina-http doesn't have HttpEntity equivalent.
> We could start without them and add them in second step
>
> 2013/1/3 Julien Vermillard <jv...@gmail.com>:
> > I think we agree here, I wouldn't touch the NIO part, but the interface
> for
> > the message are nice. The only issue is with the HttpEntity stuff, I like
> > the way contents are streamed in small chunk in the mina-http codec. I'm
> > not sure it's feasable without forking the http-core code.
> >
> >
> > On Wed, Jan 2, 2013 at 10:04 PM, Arnaud bourree <
> arnaud.bourree@gmail.com>wrote:
> >
> >> My idea was to re-used http-core pojo classes like BasicHttpStatus,
> >> BasicHttpResponse, ... in mina-http codec.
> >> In other words don't re-defined Http pojo classes, just implement mina
> >> encoder/decoder
> >> IMO, http-core-nio looks more complex than existing mina-http and
> >> doesn't have clear split between pojo classes and parser we expect
> >>
> >> 2013/1/2 Julien Vermillard <jv...@gmail.com>:
> >> > Taking a look now. Looks like the code is not really commented :(
> >> >
> >> > On Wed, Jan 2, 2013 at 12:08 PM, Ashish <pa...@gmail.com>
> wrote:
> >> >
> >> >> hc has nio based implementation as well
> >> >>
> >>
> http://hc.apache.org/httpcomponents-core-ga/httpcore-nio/xref/index.html
> >> >> See nio.codecs package
> >> >>
> >> >>
> >> >>
> >> >>
> >> >> On Tue, Jan 1, 2013 at 11:53 PM, Julien Vermillard <
> >> jvermillard@gmail.com
> >> >> >wrote:
> >> >>
> >> >> > Definitively should take a look.
> >> >> > The only tricky issue is streaming large content, because MINA
> have an
> >> >> > event based paradigm where H.C. have probably a stream based
> approach.
> >> >> >
> >> >> > Julien
> >> >> > Le 1 janv. 2013 16:33, "Arnaud bourree" <ar...@gmail.com>
> a
> >> >> > écrit :
> >> >> >
> >> >> > > Hi,
> >> >> > >
> >> >> > > Happy new year 2013.
> >> >> > >
> >> >> > > I'm strongly interested in HTTP codec: I used for one project I
> did
> >> >> > > for my company.
> >> >> > > Here there are my point of view:
> >> >> > > - share codec between MINA 2 and 3: I initially take MINA 3 as
> base
> >> to
> >> >> > > implement on MINA 2 and keep classes and package structure. BTW
> >> share
> >> >> > > should be easy to do, but when I did job, I made some change due
> to
> >> >> > > JDK version 5 for MINA 2 and 6 for MINA 3
> >> >> > > - codec independent from MINA: I take a look on the web to find
> HTTP
> >> >> > > API object to convert BB to, I find that Apache HTTP Client
> >> >> > > (http-core) feet what we need.
> >> >> > > What do you think to used http-core as API for Pojo object and
> write
> >> >> > > HTTP codec to encode/decode BB from/to http-core objects and on
> top
> >> a
> >> >> > > HTTP filter for MINA framework?
> >> >> > >
> >> >> > > Regrards,
> >> >> > >
> >> >> > > Arnaud.
> >> >> > >
> >> >> > > 2013/1/1 Julien Vermillard <jv...@gmail.com>:
> >> >> > > > Hi,
> >> >> > > > I wanted to sleep, by my son wasn't agreeing :) I will probably
> >> crash
> >> >> > > later.
> >> >> > > >
> >> >> > > > Yeah we could experiment with the HTTP codec, it's in pretty
> bad
> >> >> state
> >> >> > > for
> >> >> > > > now.
> >> >> > > > It would be nice to be able to share the codec code between
> MINA 2
> >> >> and
> >> >> > 3.
> >> >> > > > Julien
> >> >> > > >
> >> >> > > >
> >> >> > > > On Tue, Jan 1, 2013 at 9:45 AM, Emmanuel Lecharny <
> >> >> > elecharny@apache.org
> >> >> > > >wrote:
> >> >> > > >
> >> >> > > >> we should think of a codec as an independant module : it
> should
> >> be
> >> >> > > >> available for any java code that just needs suh a codec for
> its
> >> own
> >> >> > > >> purpose.
> >> >> > > >>
> >> >> > > >> such a need has already been expressed for http.
> >> >> > > >>
> >> >> > > >> imo, the current impl is over-ingeniered.
> >> >> > > >>
> >> >> > > >> Btw, it seems that we are up and running at 9am on jan.
> first...
> >> >> crazy
> >> >> > > open
> >> >> > > >> source developpers...
> >> >> > > >>
> >> >> > > >> happy new year !
> >> >> > > >>
> >> >> > > >> Le 1 janv. 2013 09:26, "Julien Vermillard" <
> >> jvermillard@gmail.com>
> >> >> a
> >> >> > > >> écrit :
> >> >> > > >> >
> >> >> > > >> > It's sure 10 year after SEDA is quite smelly :-)
> >> >> > > >> > In my mind the codec code should be used by a filter for
> >> >> > transforming
> >> >> > > the
> >> >> > > >> > bytes into pojos (like today) but really not dependent of
> MINA.
> >> >> > > >> > IMHO demux handler is a piece of s..t, you should use a
> visitor
> >> >> > > pattern.
> >> >> > > >> > Much more testable.
> >> >> > > >> >
> >> >> > > >> > I like the loop until it's decoded idea, it very simple to
> >> >> > understand.
> >> >> > > >> > Le 31 déc. 2012 18:13, "Emmanuel Lécharny" <
> >> elecharny@gmail.com>
> >> >> a
> >> >> > > >> écrit
> >> >> > > >> :
> >> >> > > >> >
> >> >> > > >> > > Le 12/31/12 7:55 AM, Julien Vermillard a écrit :
> >> >> > > >> > > > Hi,
> >> >> > > >> > > >
> >> >> > > >> > > > Since few year, I stopped to use the MINA
> >> ProtocolCodecFilter
> >> >> > and
> >> >> > > >> > > > associated stuff (CumulativeCodec..). for implementing
> my
> >> own
> >> >> > > codec
> >> >> > > >> > > > independent of MINA.
> >> >> > > >> > > >
> >> >> > > >> > > > it's just a service consuming ByteBuffer and pushing
> >> decoded
> >> >> > POJO
> >> >> > > in
> >> >> > > >> a
> >> >> > > >> > > > callback. The point is to be independent of MINA for
> >> example,
> >> >> > > parse &
> >> >> > > >> > > save
> >> >> > > >> > > > files using the codec, or simply implement an HTTP
> version
> >> of
> >> >> > the
> >> >> > > >> > > transport
> >> >> > > >> > > > using old style servlet.
> >> >> > > >> > > >
> >> >> > > >> > > > Basically a decoder looks like :
> >> >> > https://gist.github.com/4417934
> >> >> > > >> > > > One is instantiated by session.
> >> >> > > >> > > >
> >> >> > > >> > > > I'm quite happy with that and I think we should not port
> >> the
> >> >> old
> >> >> > > >> > > > ProtocolCodeFilter to MINA 3.0 and replace it with a
> >> >> independent
> >> >> > > MINA
> >> >> > > >> > > async
> >> >> > > >> > > > decoder framework (consuming BB, accumulating if needed
> and
> >> >> > > producing
> >> >> > > >> > > pojo).
> >> >> > > >> > > It sounds a reasonnable proposal.
> >> >> > > >> > >
> >> >> > > >> > > If we think about it, decoding is not part of a filter
> chain
> >> :
> >> >> it
> >> >> > > >> > > introduces a change of data type being passed from one
> >> filter to
> >> >> > the
> >> >> > > >> > > other, and if we have to cumulate data, we will just stop
> >> >> > processing
> >> >> > > >> the
> >> >> > > >> > > incomming data in the middle of the chain, the handler
> being
> >> >> > > unaware of
> >> >> > > >> > > this fact.
> >> >> > > >> > >
> >> >> > > >> > >
> >> >> > > >> > > Julien's proposal seems way better : the Handler would
> have a
> >> >> > common
> >> >> > > >> > > interface for encoding and decoding, used as a service
> when a
> >> >> > > >> > > MessageReceived or a Write events are to be processed.
> This
> >> way,
> >> >> > the
> >> >> > > >> > > handler is fully in charge of all the aspects of the data
> >> >> > > processing,
> >> >> > > >> > > including the accumulation of data.
> >> >> > > >> > >
> >> >> > > >> > > It won't either eliminate the existence of pre-written
> codec,
> >> >> like
> >> >> > > the
> >> >> > > >> > > HttpCodec, or the Textline codec. We can even think about
> a
> >> >> chain
> >> >> > of
> >> >> > > >> > > codecs : one codec depends on the result of the previous
> >> codec.
> >> >> > > >> > >
> >> >> > > >> > > As far as I can tell, changing MINA this way will not
> impact
> >> >> > > ApacheDS,
> >> >> > > >> > > even if we are using a DemuxIoHandler (the handler called
> >> >> depends
> >> >> > on
> >> >> > > >> the
> >> >> > > >> > > received message) : I don't see such a handler as a
> >> >> simplification
> >> >> > > over
> >> >> > > >> > > a simple switch...
> >> >> > > >> > >
> >> >> > > >> > >
> >> >> > > >> > > Keep in mind that the exisiting MINA logic depends on an
> idea
> >> >> > which
> >> >> > > is
> >> >> > > >> > > 10 years old : SEDA, and has not proven any advantage
> against
> >> >> > > simpler
> >> >> > > >> > > implementations. It's also important to notice that SEDA
> >> implies
> >> >> > > that
> >> >> > > >> > > each process part communicates with the next process
> (read :
> >> >> > > filter) by
> >> >> > > >> > > the use of queues. This is highly costly and memory
> >> consuming.
> >> >> I'm
> >> >> > > not
> >> >> > > >> > > sure that SEDA has anything to do with MINA implementation
> >> >> > anwyay...
> >> >> > > >> > >
> >> >> > > >> > > On more thing : the current codec supposes that we pass a
> >> >> callback
> >> >> > > >> which
> >> >> > > >> > > is called as soon as something has been decoded. This make
> >> the
> >> >> > code
> >> >> > > >> > > extremely complicated to debug. I'd rather have a system
> >> where
> >> >> we
> >> >> > > can
> >> >> > > >> > > loop on the decoder, until it produces nothing. In other
> >> words,
> >> >> > > instead
> >> >> > > >> > > of having something like :
> >> >> > > >> > >
> >> >> > > >> > > void myCallback( IoSession session, Object message ) {
> >> >> > > >> > > // Do something
> >> >> > > >> > > }
> >> >> > > >> > >
> >> >> > > >> > > void decode( IoSession session, ByteBuffer buffer,
> callback
> >> ) {
> >> >> > > >> > > // Decode and call the callback
> >> >> > > >> > > }
> >> >> > > >> > >
> >> >> > > >> > >
> >> >> > > >> > > void messageReceived( IoSession session, ByteBuffer
> buffer )
> >> {
> >> >> > > >> > > decode( session, myCalback );
> >> >> > > >> > > ...
> >> >> > > >> > > }
> >> >> > > >> > >
> >> >> > > >> > >
> >> >> > > >> > > I would prefer something like :
> >> >> > > >> > >
> >> >> > > >> > > Object decode( IoSession session, ByteBuffer buffer ) {
> >> >> > > >> > > // Decode
> >> >> > > >> > >
> >> >> > > >> > > return decoded;
> >> >> > > >> > > }
> >> >> > > >> > >
> >> >> > > >> > >
> >> >> > > >> > > void messageReceived( IoSession session, ByteBuffer
> buffer )
> >> {
> >> >> > > >> > > while ( ( Object decoded = decode( session ) ) !=
> null )
> >> {
> >> >> > > >> > > // Do something
> >> >> > > >> > > }
> >> >> > > >> > > }
> >> >> > > >> > >
> >> >> > > >> > >
> >> >> > > >> > >
> >> >> > > >> > >
> >> >> > > >> > > >
> >> >> > > >> > > > Julien
> >> >> > > >> > > >
> >> >> > > >> > >
> >> >> > > >> > >
> >> >> > > >> > > --
> >> >> > > >> > > Regards,
> >> >> > > >> > > Cordialement,
> >> >> > > >> > > Emmanuel Lécharny
> >> >> > > >> > > www.iktek.com
> >> >> > > >> > >
> >> >> > > >> > >
> >> >> > > >>
> >> >> > >
> >> >> >
> >> >>
> >> >>
> >> >>
> >> >> --
> >> >> thanks
> >> >> ashish
> >> >>
> >> >> Blog: http://www.ashishpaliwal.com/blog
> >> >> My Photo Galleries: http://www.pbase.com/ashishpaliwal
> >> >>
> >>
>
Re: codec "framework"
Posted by Arnaud bourree <ar...@gmail.com>.
You're right, in same time mina-http doesn't have HttpEntity equivalent.
We could start without them and add them in second step
2013/1/3 Julien Vermillard <jv...@gmail.com>:
> I think we agree here, I wouldn't touch the NIO part, but the interface for
> the message are nice. The only issue is with the HttpEntity stuff, I like
> the way contents are streamed in small chunk in the mina-http codec. I'm
> not sure it's feasable without forking the http-core code.
>
>
> On Wed, Jan 2, 2013 at 10:04 PM, Arnaud bourree <ar...@gmail.com>wrote:
>
>> My idea was to re-used http-core pojo classes like BasicHttpStatus,
>> BasicHttpResponse, ... in mina-http codec.
>> In other words don't re-defined Http pojo classes, just implement mina
>> encoder/decoder
>> IMO, http-core-nio looks more complex than existing mina-http and
>> doesn't have clear split between pojo classes and parser we expect
>>
>> 2013/1/2 Julien Vermillard <jv...@gmail.com>:
>> > Taking a look now. Looks like the code is not really commented :(
>> >
>> > On Wed, Jan 2, 2013 at 12:08 PM, Ashish <pa...@gmail.com> wrote:
>> >
>> >> hc has nio based implementation as well
>> >>
>> http://hc.apache.org/httpcomponents-core-ga/httpcore-nio/xref/index.html
>> >> See nio.codecs package
>> >>
>> >>
>> >>
>> >>
>> >> On Tue, Jan 1, 2013 at 11:53 PM, Julien Vermillard <
>> jvermillard@gmail.com
>> >> >wrote:
>> >>
>> >> > Definitively should take a look.
>> >> > The only tricky issue is streaming large content, because MINA have an
>> >> > event based paradigm where H.C. have probably a stream based approach.
>> >> >
>> >> > Julien
>> >> > Le 1 janv. 2013 16:33, "Arnaud bourree" <ar...@gmail.com> a
>> >> > écrit :
>> >> >
>> >> > > Hi,
>> >> > >
>> >> > > Happy new year 2013.
>> >> > >
>> >> > > I'm strongly interested in HTTP codec: I used for one project I did
>> >> > > for my company.
>> >> > > Here there are my point of view:
>> >> > > - share codec between MINA 2 and 3: I initially take MINA 3 as base
>> to
>> >> > > implement on MINA 2 and keep classes and package structure. BTW
>> share
>> >> > > should be easy to do, but when I did job, I made some change due to
>> >> > > JDK version 5 for MINA 2 and 6 for MINA 3
>> >> > > - codec independent from MINA: I take a look on the web to find HTTP
>> >> > > API object to convert BB to, I find that Apache HTTP Client
>> >> > > (http-core) feet what we need.
>> >> > > What do you think to used http-core as API for Pojo object and write
>> >> > > HTTP codec to encode/decode BB from/to http-core objects and on top
>> a
>> >> > > HTTP filter for MINA framework?
>> >> > >
>> >> > > Regrards,
>> >> > >
>> >> > > Arnaud.
>> >> > >
>> >> > > 2013/1/1 Julien Vermillard <jv...@gmail.com>:
>> >> > > > Hi,
>> >> > > > I wanted to sleep, by my son wasn't agreeing :) I will probably
>> crash
>> >> > > later.
>> >> > > >
>> >> > > > Yeah we could experiment with the HTTP codec, it's in pretty bad
>> >> state
>> >> > > for
>> >> > > > now.
>> >> > > > It would be nice to be able to share the codec code between MINA 2
>> >> and
>> >> > 3.
>> >> > > > Julien
>> >> > > >
>> >> > > >
>> >> > > > On Tue, Jan 1, 2013 at 9:45 AM, Emmanuel Lecharny <
>> >> > elecharny@apache.org
>> >> > > >wrote:
>> >> > > >
>> >> > > >> we should think of a codec as an independant module : it should
>> be
>> >> > > >> available for any java code that just needs suh a codec for its
>> own
>> >> > > >> purpose.
>> >> > > >>
>> >> > > >> such a need has already been expressed for http.
>> >> > > >>
>> >> > > >> imo, the current impl is over-ingeniered.
>> >> > > >>
>> >> > > >> Btw, it seems that we are up and running at 9am on jan. first...
>> >> crazy
>> >> > > open
>> >> > > >> source developpers...
>> >> > > >>
>> >> > > >> happy new year !
>> >> > > >>
>> >> > > >> Le 1 janv. 2013 09:26, "Julien Vermillard" <
>> jvermillard@gmail.com>
>> >> a
>> >> > > >> écrit :
>> >> > > >> >
>> >> > > >> > It's sure 10 year after SEDA is quite smelly :-)
>> >> > > >> > In my mind the codec code should be used by a filter for
>> >> > transforming
>> >> > > the
>> >> > > >> > bytes into pojos (like today) but really not dependent of MINA.
>> >> > > >> > IMHO demux handler is a piece of s..t, you should use a visitor
>> >> > > pattern.
>> >> > > >> > Much more testable.
>> >> > > >> >
>> >> > > >> > I like the loop until it's decoded idea, it very simple to
>> >> > understand.
>> >> > > >> > Le 31 déc. 2012 18:13, "Emmanuel Lécharny" <
>> elecharny@gmail.com>
>> >> a
>> >> > > >> écrit
>> >> > > >> :
>> >> > > >> >
>> >> > > >> > > Le 12/31/12 7:55 AM, Julien Vermillard a écrit :
>> >> > > >> > > > Hi,
>> >> > > >> > > >
>> >> > > >> > > > Since few year, I stopped to use the MINA
>> ProtocolCodecFilter
>> >> > and
>> >> > > >> > > > associated stuff (CumulativeCodec..). for implementing my
>> own
>> >> > > codec
>> >> > > >> > > > independent of MINA.
>> >> > > >> > > >
>> >> > > >> > > > it's just a service consuming ByteBuffer and pushing
>> decoded
>> >> > POJO
>> >> > > in
>> >> > > >> a
>> >> > > >> > > > callback. The point is to be independent of MINA for
>> example,
>> >> > > parse &
>> >> > > >> > > save
>> >> > > >> > > > files using the codec, or simply implement an HTTP version
>> of
>> >> > the
>> >> > > >> > > transport
>> >> > > >> > > > using old style servlet.
>> >> > > >> > > >
>> >> > > >> > > > Basically a decoder looks like :
>> >> > https://gist.github.com/4417934
>> >> > > >> > > > One is instantiated by session.
>> >> > > >> > > >
>> >> > > >> > > > I'm quite happy with that and I think we should not port
>> the
>> >> old
>> >> > > >> > > > ProtocolCodeFilter to MINA 3.0 and replace it with a
>> >> independent
>> >> > > MINA
>> >> > > >> > > async
>> >> > > >> > > > decoder framework (consuming BB, accumulating if needed and
>> >> > > producing
>> >> > > >> > > pojo).
>> >> > > >> > > It sounds a reasonnable proposal.
>> >> > > >> > >
>> >> > > >> > > If we think about it, decoding is not part of a filter chain
>> :
>> >> it
>> >> > > >> > > introduces a change of data type being passed from one
>> filter to
>> >> > the
>> >> > > >> > > other, and if we have to cumulate data, we will just stop
>> >> > processing
>> >> > > >> the
>> >> > > >> > > incomming data in the middle of the chain, the handler being
>> >> > > unaware of
>> >> > > >> > > this fact.
>> >> > > >> > >
>> >> > > >> > >
>> >> > > >> > > Julien's proposal seems way better : the Handler would have a
>> >> > common
>> >> > > >> > > interface for encoding and decoding, used as a service when a
>> >> > > >> > > MessageReceived or a Write events are to be processed. This
>> way,
>> >> > the
>> >> > > >> > > handler is fully in charge of all the aspects of the data
>> >> > > processing,
>> >> > > >> > > including the accumulation of data.
>> >> > > >> > >
>> >> > > >> > > It won't either eliminate the existence of pre-written codec,
>> >> like
>> >> > > the
>> >> > > >> > > HttpCodec, or the Textline codec. We can even think about a
>> >> chain
>> >> > of
>> >> > > >> > > codecs : one codec depends on the result of the previous
>> codec.
>> >> > > >> > >
>> >> > > >> > > As far as I can tell, changing MINA this way will not impact
>> >> > > ApacheDS,
>> >> > > >> > > even if we are using a DemuxIoHandler (the handler called
>> >> depends
>> >> > on
>> >> > > >> the
>> >> > > >> > > received message) : I don't see such a handler as a
>> >> simplification
>> >> > > over
>> >> > > >> > > a simple switch...
>> >> > > >> > >
>> >> > > >> > >
>> >> > > >> > > Keep in mind that the exisiting MINA logic depends on an idea
>> >> > which
>> >> > > is
>> >> > > >> > > 10 years old : SEDA, and has not proven any advantage against
>> >> > > simpler
>> >> > > >> > > implementations. It's also important to notice that SEDA
>> implies
>> >> > > that
>> >> > > >> > > each process part communicates with the next process (read :
>> >> > > filter) by
>> >> > > >> > > the use of queues. This is highly costly and memory
>> consuming.
>> >> I'm
>> >> > > not
>> >> > > >> > > sure that SEDA has anything to do with MINA implementation
>> >> > anwyay...
>> >> > > >> > >
>> >> > > >> > > On more thing : the current codec supposes that we pass a
>> >> callback
>> >> > > >> which
>> >> > > >> > > is called as soon as something has been decoded. This make
>> the
>> >> > code
>> >> > > >> > > extremely complicated to debug. I'd rather have a system
>> where
>> >> we
>> >> > > can
>> >> > > >> > > loop on the decoder, until it produces nothing. In other
>> words,
>> >> > > instead
>> >> > > >> > > of having something like :
>> >> > > >> > >
>> >> > > >> > > void myCallback( IoSession session, Object message ) {
>> >> > > >> > > // Do something
>> >> > > >> > > }
>> >> > > >> > >
>> >> > > >> > > void decode( IoSession session, ByteBuffer buffer, callback
>> ) {
>> >> > > >> > > // Decode and call the callback
>> >> > > >> > > }
>> >> > > >> > >
>> >> > > >> > >
>> >> > > >> > > void messageReceived( IoSession session, ByteBuffer buffer )
>> {
>> >> > > >> > > decode( session, myCalback );
>> >> > > >> > > ...
>> >> > > >> > > }
>> >> > > >> > >
>> >> > > >> > >
>> >> > > >> > > I would prefer something like :
>> >> > > >> > >
>> >> > > >> > > Object decode( IoSession session, ByteBuffer buffer ) {
>> >> > > >> > > // Decode
>> >> > > >> > >
>> >> > > >> > > return decoded;
>> >> > > >> > > }
>> >> > > >> > >
>> >> > > >> > >
>> >> > > >> > > void messageReceived( IoSession session, ByteBuffer buffer )
>> {
>> >> > > >> > > while ( ( Object decoded = decode( session ) ) != null )
>> {
>> >> > > >> > > // Do something
>> >> > > >> > > }
>> >> > > >> > > }
>> >> > > >> > >
>> >> > > >> > >
>> >> > > >> > >
>> >> > > >> > >
>> >> > > >> > > >
>> >> > > >> > > > Julien
>> >> > > >> > > >
>> >> > > >> > >
>> >> > > >> > >
>> >> > > >> > > --
>> >> > > >> > > Regards,
>> >> > > >> > > Cordialement,
>> >> > > >> > > Emmanuel Lécharny
>> >> > > >> > > www.iktek.com
>> >> > > >> > >
>> >> > > >> > >
>> >> > > >>
>> >> > >
>> >> >
>> >>
>> >>
>> >>
>> >> --
>> >> thanks
>> >> ashish
>> >>
>> >> Blog: http://www.ashishpaliwal.com/blog
>> >> My Photo Galleries: http://www.pbase.com/ashishpaliwal
>> >>
>>
Re: codec "framework"
Posted by Julien Vermillard <jv...@gmail.com>.
I think we agree here, I wouldn't touch the NIO part, but the interface for
the message are nice. The only issue is with the HttpEntity stuff, I like
the way contents are streamed in small chunk in the mina-http codec. I'm
not sure it's feasable without forking the http-core code.
On Wed, Jan 2, 2013 at 10:04 PM, Arnaud bourree <ar...@gmail.com>wrote:
> My idea was to re-used http-core pojo classes like BasicHttpStatus,
> BasicHttpResponse, ... in mina-http codec.
> In other words don't re-defined Http pojo classes, just implement mina
> encoder/decoder
> IMO, http-core-nio looks more complex than existing mina-http and
> doesn't have clear split between pojo classes and parser we expect
>
> 2013/1/2 Julien Vermillard <jv...@gmail.com>:
> > Taking a look now. Looks like the code is not really commented :(
> >
> > On Wed, Jan 2, 2013 at 12:08 PM, Ashish <pa...@gmail.com> wrote:
> >
> >> hc has nio based implementation as well
> >>
> http://hc.apache.org/httpcomponents-core-ga/httpcore-nio/xref/index.html
> >> See nio.codecs package
> >>
> >>
> >>
> >>
> >> On Tue, Jan 1, 2013 at 11:53 PM, Julien Vermillard <
> jvermillard@gmail.com
> >> >wrote:
> >>
> >> > Definitively should take a look.
> >> > The only tricky issue is streaming large content, because MINA have an
> >> > event based paradigm where H.C. have probably a stream based approach.
> >> >
> >> > Julien
> >> > Le 1 janv. 2013 16:33, "Arnaud bourree" <ar...@gmail.com> a
> >> > écrit :
> >> >
> >> > > Hi,
> >> > >
> >> > > Happy new year 2013.
> >> > >
> >> > > I'm strongly interested in HTTP codec: I used for one project I did
> >> > > for my company.
> >> > > Here there are my point of view:
> >> > > - share codec between MINA 2 and 3: I initially take MINA 3 as base
> to
> >> > > implement on MINA 2 and keep classes and package structure. BTW
> share
> >> > > should be easy to do, but when I did job, I made some change due to
> >> > > JDK version 5 for MINA 2 and 6 for MINA 3
> >> > > - codec independent from MINA: I take a look on the web to find HTTP
> >> > > API object to convert BB to, I find that Apache HTTP Client
> >> > > (http-core) feet what we need.
> >> > > What do you think to used http-core as API for Pojo object and write
> >> > > HTTP codec to encode/decode BB from/to http-core objects and on top
> a
> >> > > HTTP filter for MINA framework?
> >> > >
> >> > > Regrards,
> >> > >
> >> > > Arnaud.
> >> > >
> >> > > 2013/1/1 Julien Vermillard <jv...@gmail.com>:
> >> > > > Hi,
> >> > > > I wanted to sleep, by my son wasn't agreeing :) I will probably
> crash
> >> > > later.
> >> > > >
> >> > > > Yeah we could experiment with the HTTP codec, it's in pretty bad
> >> state
> >> > > for
> >> > > > now.
> >> > > > It would be nice to be able to share the codec code between MINA 2
> >> and
> >> > 3.
> >> > > > Julien
> >> > > >
> >> > > >
> >> > > > On Tue, Jan 1, 2013 at 9:45 AM, Emmanuel Lecharny <
> >> > elecharny@apache.org
> >> > > >wrote:
> >> > > >
> >> > > >> we should think of a codec as an independant module : it should
> be
> >> > > >> available for any java code that just needs suh a codec for its
> own
> >> > > >> purpose.
> >> > > >>
> >> > > >> such a need has already been expressed for http.
> >> > > >>
> >> > > >> imo, the current impl is over-ingeniered.
> >> > > >>
> >> > > >> Btw, it seems that we are up and running at 9am on jan. first...
> >> crazy
> >> > > open
> >> > > >> source developpers...
> >> > > >>
> >> > > >> happy new year !
> >> > > >>
> >> > > >> Le 1 janv. 2013 09:26, "Julien Vermillard" <
> jvermillard@gmail.com>
> >> a
> >> > > >> écrit :
> >> > > >> >
> >> > > >> > It's sure 10 year after SEDA is quite smelly :-)
> >> > > >> > In my mind the codec code should be used by a filter for
> >> > transforming
> >> > > the
> >> > > >> > bytes into pojos (like today) but really not dependent of MINA.
> >> > > >> > IMHO demux handler is a piece of s..t, you should use a visitor
> >> > > pattern.
> >> > > >> > Much more testable.
> >> > > >> >
> >> > > >> > I like the loop until it's decoded idea, it very simple to
> >> > understand.
> >> > > >> > Le 31 déc. 2012 18:13, "Emmanuel Lécharny" <
> elecharny@gmail.com>
> >> a
> >> > > >> écrit
> >> > > >> :
> >> > > >> >
> >> > > >> > > Le 12/31/12 7:55 AM, Julien Vermillard a écrit :
> >> > > >> > > > Hi,
> >> > > >> > > >
> >> > > >> > > > Since few year, I stopped to use the MINA
> ProtocolCodecFilter
> >> > and
> >> > > >> > > > associated stuff (CumulativeCodec..). for implementing my
> own
> >> > > codec
> >> > > >> > > > independent of MINA.
> >> > > >> > > >
> >> > > >> > > > it's just a service consuming ByteBuffer and pushing
> decoded
> >> > POJO
> >> > > in
> >> > > >> a
> >> > > >> > > > callback. The point is to be independent of MINA for
> example,
> >> > > parse &
> >> > > >> > > save
> >> > > >> > > > files using the codec, or simply implement an HTTP version
> of
> >> > the
> >> > > >> > > transport
> >> > > >> > > > using old style servlet.
> >> > > >> > > >
> >> > > >> > > > Basically a decoder looks like :
> >> > https://gist.github.com/4417934
> >> > > >> > > > One is instantiated by session.
> >> > > >> > > >
> >> > > >> > > > I'm quite happy with that and I think we should not port
> the
> >> old
> >> > > >> > > > ProtocolCodeFilter to MINA 3.0 and replace it with a
> >> independent
> >> > > MINA
> >> > > >> > > async
> >> > > >> > > > decoder framework (consuming BB, accumulating if needed and
> >> > > producing
> >> > > >> > > pojo).
> >> > > >> > > It sounds a reasonnable proposal.
> >> > > >> > >
> >> > > >> > > If we think about it, decoding is not part of a filter chain
> :
> >> it
> >> > > >> > > introduces a change of data type being passed from one
> filter to
> >> > the
> >> > > >> > > other, and if we have to cumulate data, we will just stop
> >> > processing
> >> > > >> the
> >> > > >> > > incomming data in the middle of the chain, the handler being
> >> > > unaware of
> >> > > >> > > this fact.
> >> > > >> > >
> >> > > >> > >
> >> > > >> > > Julien's proposal seems way better : the Handler would have a
> >> > common
> >> > > >> > > interface for encoding and decoding, used as a service when a
> >> > > >> > > MessageReceived or a Write events are to be processed. This
> way,
> >> > the
> >> > > >> > > handler is fully in charge of all the aspects of the data
> >> > > processing,
> >> > > >> > > including the accumulation of data.
> >> > > >> > >
> >> > > >> > > It won't either eliminate the existence of pre-written codec,
> >> like
> >> > > the
> >> > > >> > > HttpCodec, or the Textline codec. We can even think about a
> >> chain
> >> > of
> >> > > >> > > codecs : one codec depends on the result of the previous
> codec.
> >> > > >> > >
> >> > > >> > > As far as I can tell, changing MINA this way will not impact
> >> > > ApacheDS,
> >> > > >> > > even if we are using a DemuxIoHandler (the handler called
> >> depends
> >> > on
> >> > > >> the
> >> > > >> > > received message) : I don't see such a handler as a
> >> simplification
> >> > > over
> >> > > >> > > a simple switch...
> >> > > >> > >
> >> > > >> > >
> >> > > >> > > Keep in mind that the exisiting MINA logic depends on an idea
> >> > which
> >> > > is
> >> > > >> > > 10 years old : SEDA, and has not proven any advantage against
> >> > > simpler
> >> > > >> > > implementations. It's also important to notice that SEDA
> implies
> >> > > that
> >> > > >> > > each process part communicates with the next process (read :
> >> > > filter) by
> >> > > >> > > the use of queues. This is highly costly and memory
> consuming.
> >> I'm
> >> > > not
> >> > > >> > > sure that SEDA has anything to do with MINA implementation
> >> > anwyay...
> >> > > >> > >
> >> > > >> > > On more thing : the current codec supposes that we pass a
> >> callback
> >> > > >> which
> >> > > >> > > is called as soon as something has been decoded. This make
> the
> >> > code
> >> > > >> > > extremely complicated to debug. I'd rather have a system
> where
> >> we
> >> > > can
> >> > > >> > > loop on the decoder, until it produces nothing. In other
> words,
> >> > > instead
> >> > > >> > > of having something like :
> >> > > >> > >
> >> > > >> > > void myCallback( IoSession session, Object message ) {
> >> > > >> > > // Do something
> >> > > >> > > }
> >> > > >> > >
> >> > > >> > > void decode( IoSession session, ByteBuffer buffer, callback
> ) {
> >> > > >> > > // Decode and call the callback
> >> > > >> > > }
> >> > > >> > >
> >> > > >> > >
> >> > > >> > > void messageReceived( IoSession session, ByteBuffer buffer )
> {
> >> > > >> > > decode( session, myCalback );
> >> > > >> > > ...
> >> > > >> > > }
> >> > > >> > >
> >> > > >> > >
> >> > > >> > > I would prefer something like :
> >> > > >> > >
> >> > > >> > > Object decode( IoSession session, ByteBuffer buffer ) {
> >> > > >> > > // Decode
> >> > > >> > >
> >> > > >> > > return decoded;
> >> > > >> > > }
> >> > > >> > >
> >> > > >> > >
> >> > > >> > > void messageReceived( IoSession session, ByteBuffer buffer )
> {
> >> > > >> > > while ( ( Object decoded = decode( session ) ) != null )
> {
> >> > > >> > > // Do something
> >> > > >> > > }
> >> > > >> > > }
> >> > > >> > >
> >> > > >> > >
> >> > > >> > >
> >> > > >> > >
> >> > > >> > > >
> >> > > >> > > > Julien
> >> > > >> > > >
> >> > > >> > >
> >> > > >> > >
> >> > > >> > > --
> >> > > >> > > Regards,
> >> > > >> > > Cordialement,
> >> > > >> > > Emmanuel Lécharny
> >> > > >> > > www.iktek.com
> >> > > >> > >
> >> > > >> > >
> >> > > >>
> >> > >
> >> >
> >>
> >>
> >>
> >> --
> >> thanks
> >> ashish
> >>
> >> Blog: http://www.ashishpaliwal.com/blog
> >> My Photo Galleries: http://www.pbase.com/ashishpaliwal
> >>
>
Re: codec "framework"
Posted by Arnaud bourree <ar...@gmail.com>.
My idea was to re-used http-core pojo classes like BasicHttpStatus,
BasicHttpResponse, ... in mina-http codec.
In other words don't re-defined Http pojo classes, just implement mina
encoder/decoder
IMO, http-core-nio looks more complex than existing mina-http and
doesn't have clear split between pojo classes and parser we expect
2013/1/2 Julien Vermillard <jv...@gmail.com>:
> Taking a look now. Looks like the code is not really commented :(
>
> On Wed, Jan 2, 2013 at 12:08 PM, Ashish <pa...@gmail.com> wrote:
>
>> hc has nio based implementation as well
>> http://hc.apache.org/httpcomponents-core-ga/httpcore-nio/xref/index.html
>> See nio.codecs package
>>
>>
>>
>>
>> On Tue, Jan 1, 2013 at 11:53 PM, Julien Vermillard <jvermillard@gmail.com
>> >wrote:
>>
>> > Definitively should take a look.
>> > The only tricky issue is streaming large content, because MINA have an
>> > event based paradigm where H.C. have probably a stream based approach.
>> >
>> > Julien
>> > Le 1 janv. 2013 16:33, "Arnaud bourree" <ar...@gmail.com> a
>> > écrit :
>> >
>> > > Hi,
>> > >
>> > > Happy new year 2013.
>> > >
>> > > I'm strongly interested in HTTP codec: I used for one project I did
>> > > for my company.
>> > > Here there are my point of view:
>> > > - share codec between MINA 2 and 3: I initially take MINA 3 as base to
>> > > implement on MINA 2 and keep classes and package structure. BTW share
>> > > should be easy to do, but when I did job, I made some change due to
>> > > JDK version 5 for MINA 2 and 6 for MINA 3
>> > > - codec independent from MINA: I take a look on the web to find HTTP
>> > > API object to convert BB to, I find that Apache HTTP Client
>> > > (http-core) feet what we need.
>> > > What do you think to used http-core as API for Pojo object and write
>> > > HTTP codec to encode/decode BB from/to http-core objects and on top a
>> > > HTTP filter for MINA framework?
>> > >
>> > > Regrards,
>> > >
>> > > Arnaud.
>> > >
>> > > 2013/1/1 Julien Vermillard <jv...@gmail.com>:
>> > > > Hi,
>> > > > I wanted to sleep, by my son wasn't agreeing :) I will probably crash
>> > > later.
>> > > >
>> > > > Yeah we could experiment with the HTTP codec, it's in pretty bad
>> state
>> > > for
>> > > > now.
>> > > > It would be nice to be able to share the codec code between MINA 2
>> and
>> > 3.
>> > > > Julien
>> > > >
>> > > >
>> > > > On Tue, Jan 1, 2013 at 9:45 AM, Emmanuel Lecharny <
>> > elecharny@apache.org
>> > > >wrote:
>> > > >
>> > > >> we should think of a codec as an independant module : it should be
>> > > >> available for any java code that just needs suh a codec for its own
>> > > >> purpose.
>> > > >>
>> > > >> such a need has already been expressed for http.
>> > > >>
>> > > >> imo, the current impl is over-ingeniered.
>> > > >>
>> > > >> Btw, it seems that we are up and running at 9am on jan. first...
>> crazy
>> > > open
>> > > >> source developpers...
>> > > >>
>> > > >> happy new year !
>> > > >>
>> > > >> Le 1 janv. 2013 09:26, "Julien Vermillard" <jv...@gmail.com>
>> a
>> > > >> écrit :
>> > > >> >
>> > > >> > It's sure 10 year after SEDA is quite smelly :-)
>> > > >> > In my mind the codec code should be used by a filter for
>> > transforming
>> > > the
>> > > >> > bytes into pojos (like today) but really not dependent of MINA.
>> > > >> > IMHO demux handler is a piece of s..t, you should use a visitor
>> > > pattern.
>> > > >> > Much more testable.
>> > > >> >
>> > > >> > I like the loop until it's decoded idea, it very simple to
>> > understand.
>> > > >> > Le 31 déc. 2012 18:13, "Emmanuel Lécharny" <el...@gmail.com>
>> a
>> > > >> écrit
>> > > >> :
>> > > >> >
>> > > >> > > Le 12/31/12 7:55 AM, Julien Vermillard a écrit :
>> > > >> > > > Hi,
>> > > >> > > >
>> > > >> > > > Since few year, I stopped to use the MINA ProtocolCodecFilter
>> > and
>> > > >> > > > associated stuff (CumulativeCodec..). for implementing my own
>> > > codec
>> > > >> > > > independent of MINA.
>> > > >> > > >
>> > > >> > > > it's just a service consuming ByteBuffer and pushing decoded
>> > POJO
>> > > in
>> > > >> a
>> > > >> > > > callback. The point is to be independent of MINA for example,
>> > > parse &
>> > > >> > > save
>> > > >> > > > files using the codec, or simply implement an HTTP version of
>> > the
>> > > >> > > transport
>> > > >> > > > using old style servlet.
>> > > >> > > >
>> > > >> > > > Basically a decoder looks like :
>> > https://gist.github.com/4417934
>> > > >> > > > One is instantiated by session.
>> > > >> > > >
>> > > >> > > > I'm quite happy with that and I think we should not port the
>> old
>> > > >> > > > ProtocolCodeFilter to MINA 3.0 and replace it with a
>> independent
>> > > MINA
>> > > >> > > async
>> > > >> > > > decoder framework (consuming BB, accumulating if needed and
>> > > producing
>> > > >> > > pojo).
>> > > >> > > It sounds a reasonnable proposal.
>> > > >> > >
>> > > >> > > If we think about it, decoding is not part of a filter chain :
>> it
>> > > >> > > introduces a change of data type being passed from one filter to
>> > the
>> > > >> > > other, and if we have to cumulate data, we will just stop
>> > processing
>> > > >> the
>> > > >> > > incomming data in the middle of the chain, the handler being
>> > > unaware of
>> > > >> > > this fact.
>> > > >> > >
>> > > >> > >
>> > > >> > > Julien's proposal seems way better : the Handler would have a
>> > common
>> > > >> > > interface for encoding and decoding, used as a service when a
>> > > >> > > MessageReceived or a Write events are to be processed. This way,
>> > the
>> > > >> > > handler is fully in charge of all the aspects of the data
>> > > processing,
>> > > >> > > including the accumulation of data.
>> > > >> > >
>> > > >> > > It won't either eliminate the existence of pre-written codec,
>> like
>> > > the
>> > > >> > > HttpCodec, or the Textline codec. We can even think about a
>> chain
>> > of
>> > > >> > > codecs : one codec depends on the result of the previous codec.
>> > > >> > >
>> > > >> > > As far as I can tell, changing MINA this way will not impact
>> > > ApacheDS,
>> > > >> > > even if we are using a DemuxIoHandler (the handler called
>> depends
>> > on
>> > > >> the
>> > > >> > > received message) : I don't see such a handler as a
>> simplification
>> > > over
>> > > >> > > a simple switch...
>> > > >> > >
>> > > >> > >
>> > > >> > > Keep in mind that the exisiting MINA logic depends on an idea
>> > which
>> > > is
>> > > >> > > 10 years old : SEDA, and has not proven any advantage against
>> > > simpler
>> > > >> > > implementations. It's also important to notice that SEDA implies
>> > > that
>> > > >> > > each process part communicates with the next process (read :
>> > > filter) by
>> > > >> > > the use of queues. This is highly costly and memory consuming.
>> I'm
>> > > not
>> > > >> > > sure that SEDA has anything to do with MINA implementation
>> > anwyay...
>> > > >> > >
>> > > >> > > On more thing : the current codec supposes that we pass a
>> callback
>> > > >> which
>> > > >> > > is called as soon as something has been decoded. This make the
>> > code
>> > > >> > > extremely complicated to debug. I'd rather have a system where
>> we
>> > > can
>> > > >> > > loop on the decoder, until it produces nothing. In other words,
>> > > instead
>> > > >> > > of having something like :
>> > > >> > >
>> > > >> > > void myCallback( IoSession session, Object message ) {
>> > > >> > > // Do something
>> > > >> > > }
>> > > >> > >
>> > > >> > > void decode( IoSession session, ByteBuffer buffer, callback ) {
>> > > >> > > // Decode and call the callback
>> > > >> > > }
>> > > >> > >
>> > > >> > >
>> > > >> > > void messageReceived( IoSession session, ByteBuffer buffer ) {
>> > > >> > > decode( session, myCalback );
>> > > >> > > ...
>> > > >> > > }
>> > > >> > >
>> > > >> > >
>> > > >> > > I would prefer something like :
>> > > >> > >
>> > > >> > > Object decode( IoSession session, ByteBuffer buffer ) {
>> > > >> > > // Decode
>> > > >> > >
>> > > >> > > return decoded;
>> > > >> > > }
>> > > >> > >
>> > > >> > >
>> > > >> > > void messageReceived( IoSession session, ByteBuffer buffer ) {
>> > > >> > > while ( ( Object decoded = decode( session ) ) != null ) {
>> > > >> > > // Do something
>> > > >> > > }
>> > > >> > > }
>> > > >> > >
>> > > >> > >
>> > > >> > >
>> > > >> > >
>> > > >> > > >
>> > > >> > > > Julien
>> > > >> > > >
>> > > >> > >
>> > > >> > >
>> > > >> > > --
>> > > >> > > Regards,
>> > > >> > > Cordialement,
>> > > >> > > Emmanuel Lécharny
>> > > >> > > www.iktek.com
>> > > >> > >
>> > > >> > >
>> > > >>
>> > >
>> >
>>
>>
>>
>> --
>> thanks
>> ashish
>>
>> Blog: http://www.ashishpaliwal.com/blog
>> My Photo Galleries: http://www.pbase.com/ashishpaliwal
>>
Re: codec "framework"
Posted by Julien Vermillard <jv...@gmail.com>.
Taking a look now. Looks like the code is not really commented :(
On Wed, Jan 2, 2013 at 12:08 PM, Ashish <pa...@gmail.com> wrote:
> hc has nio based implementation as well
> http://hc.apache.org/httpcomponents-core-ga/httpcore-nio/xref/index.html
> See nio.codecs package
>
>
>
>
> On Tue, Jan 1, 2013 at 11:53 PM, Julien Vermillard <jvermillard@gmail.com
> >wrote:
>
> > Definitively should take a look.
> > The only tricky issue is streaming large content, because MINA have an
> > event based paradigm where H.C. have probably a stream based approach.
> >
> > Julien
> > Le 1 janv. 2013 16:33, "Arnaud bourree" <ar...@gmail.com> a
> > écrit :
> >
> > > Hi,
> > >
> > > Happy new year 2013.
> > >
> > > I'm strongly interested in HTTP codec: I used for one project I did
> > > for my company.
> > > Here there are my point of view:
> > > - share codec between MINA 2 and 3: I initially take MINA 3 as base to
> > > implement on MINA 2 and keep classes and package structure. BTW share
> > > should be easy to do, but when I did job, I made some change due to
> > > JDK version 5 for MINA 2 and 6 for MINA 3
> > > - codec independent from MINA: I take a look on the web to find HTTP
> > > API object to convert BB to, I find that Apache HTTP Client
> > > (http-core) feet what we need.
> > > What do you think to used http-core as API for Pojo object and write
> > > HTTP codec to encode/decode BB from/to http-core objects and on top a
> > > HTTP filter for MINA framework?
> > >
> > > Regrards,
> > >
> > > Arnaud.
> > >
> > > 2013/1/1 Julien Vermillard <jv...@gmail.com>:
> > > > Hi,
> > > > I wanted to sleep, by my son wasn't agreeing :) I will probably crash
> > > later.
> > > >
> > > > Yeah we could experiment with the HTTP codec, it's in pretty bad
> state
> > > for
> > > > now.
> > > > It would be nice to be able to share the codec code between MINA 2
> and
> > 3.
> > > > Julien
> > > >
> > > >
> > > > On Tue, Jan 1, 2013 at 9:45 AM, Emmanuel Lecharny <
> > elecharny@apache.org
> > > >wrote:
> > > >
> > > >> we should think of a codec as an independant module : it should be
> > > >> available for any java code that just needs suh a codec for its own
> > > >> purpose.
> > > >>
> > > >> such a need has already been expressed for http.
> > > >>
> > > >> imo, the current impl is over-ingeniered.
> > > >>
> > > >> Btw, it seems that we are up and running at 9am on jan. first...
> crazy
> > > open
> > > >> source developpers...
> > > >>
> > > >> happy new year !
> > > >>
> > > >> Le 1 janv. 2013 09:26, "Julien Vermillard" <jv...@gmail.com>
> a
> > > >> écrit :
> > > >> >
> > > >> > It's sure 10 year after SEDA is quite smelly :-)
> > > >> > In my mind the codec code should be used by a filter for
> > transforming
> > > the
> > > >> > bytes into pojos (like today) but really not dependent of MINA.
> > > >> > IMHO demux handler is a piece of s..t, you should use a visitor
> > > pattern.
> > > >> > Much more testable.
> > > >> >
> > > >> > I like the loop until it's decoded idea, it very simple to
> > understand.
> > > >> > Le 31 déc. 2012 18:13, "Emmanuel Lécharny" <el...@gmail.com>
> a
> > > >> écrit
> > > >> :
> > > >> >
> > > >> > > Le 12/31/12 7:55 AM, Julien Vermillard a écrit :
> > > >> > > > Hi,
> > > >> > > >
> > > >> > > > Since few year, I stopped to use the MINA ProtocolCodecFilter
> > and
> > > >> > > > associated stuff (CumulativeCodec..). for implementing my own
> > > codec
> > > >> > > > independent of MINA.
> > > >> > > >
> > > >> > > > it's just a service consuming ByteBuffer and pushing decoded
> > POJO
> > > in
> > > >> a
> > > >> > > > callback. The point is to be independent of MINA for example,
> > > parse &
> > > >> > > save
> > > >> > > > files using the codec, or simply implement an HTTP version of
> > the
> > > >> > > transport
> > > >> > > > using old style servlet.
> > > >> > > >
> > > >> > > > Basically a decoder looks like :
> > https://gist.github.com/4417934
> > > >> > > > One is instantiated by session.
> > > >> > > >
> > > >> > > > I'm quite happy with that and I think we should not port the
> old
> > > >> > > > ProtocolCodeFilter to MINA 3.0 and replace it with a
> independent
> > > MINA
> > > >> > > async
> > > >> > > > decoder framework (consuming BB, accumulating if needed and
> > > producing
> > > >> > > pojo).
> > > >> > > It sounds a reasonnable proposal.
> > > >> > >
> > > >> > > If we think about it, decoding is not part of a filter chain :
> it
> > > >> > > introduces a change of data type being passed from one filter to
> > the
> > > >> > > other, and if we have to cumulate data, we will just stop
> > processing
> > > >> the
> > > >> > > incomming data in the middle of the chain, the handler being
> > > unaware of
> > > >> > > this fact.
> > > >> > >
> > > >> > >
> > > >> > > Julien's proposal seems way better : the Handler would have a
> > common
> > > >> > > interface for encoding and decoding, used as a service when a
> > > >> > > MessageReceived or a Write events are to be processed. This way,
> > the
> > > >> > > handler is fully in charge of all the aspects of the data
> > > processing,
> > > >> > > including the accumulation of data.
> > > >> > >
> > > >> > > It won't either eliminate the existence of pre-written codec,
> like
> > > the
> > > >> > > HttpCodec, or the Textline codec. We can even think about a
> chain
> > of
> > > >> > > codecs : one codec depends on the result of the previous codec.
> > > >> > >
> > > >> > > As far as I can tell, changing MINA this way will not impact
> > > ApacheDS,
> > > >> > > even if we are using a DemuxIoHandler (the handler called
> depends
> > on
> > > >> the
> > > >> > > received message) : I don't see such a handler as a
> simplification
> > > over
> > > >> > > a simple switch...
> > > >> > >
> > > >> > >
> > > >> > > Keep in mind that the exisiting MINA logic depends on an idea
> > which
> > > is
> > > >> > > 10 years old : SEDA, and has not proven any advantage against
> > > simpler
> > > >> > > implementations. It's also important to notice that SEDA implies
> > > that
> > > >> > > each process part communicates with the next process (read :
> > > filter) by
> > > >> > > the use of queues. This is highly costly and memory consuming.
> I'm
> > > not
> > > >> > > sure that SEDA has anything to do with MINA implementation
> > anwyay...
> > > >> > >
> > > >> > > On more thing : the current codec supposes that we pass a
> callback
> > > >> which
> > > >> > > is called as soon as something has been decoded. This make the
> > code
> > > >> > > extremely complicated to debug. I'd rather have a system where
> we
> > > can
> > > >> > > loop on the decoder, until it produces nothing. In other words,
> > > instead
> > > >> > > of having something like :
> > > >> > >
> > > >> > > void myCallback( IoSession session, Object message ) {
> > > >> > > // Do something
> > > >> > > }
> > > >> > >
> > > >> > > void decode( IoSession session, ByteBuffer buffer, callback ) {
> > > >> > > // Decode and call the callback
> > > >> > > }
> > > >> > >
> > > >> > >
> > > >> > > void messageReceived( IoSession session, ByteBuffer buffer ) {
> > > >> > > decode( session, myCalback );
> > > >> > > ...
> > > >> > > }
> > > >> > >
> > > >> > >
> > > >> > > I would prefer something like :
> > > >> > >
> > > >> > > Object decode( IoSession session, ByteBuffer buffer ) {
> > > >> > > // Decode
> > > >> > >
> > > >> > > return decoded;
> > > >> > > }
> > > >> > >
> > > >> > >
> > > >> > > void messageReceived( IoSession session, ByteBuffer buffer ) {
> > > >> > > while ( ( Object decoded = decode( session ) ) != null ) {
> > > >> > > // Do something
> > > >> > > }
> > > >> > > }
> > > >> > >
> > > >> > >
> > > >> > >
> > > >> > >
> > > >> > > >
> > > >> > > > Julien
> > > >> > > >
> > > >> > >
> > > >> > >
> > > >> > > --
> > > >> > > Regards,
> > > >> > > Cordialement,
> > > >> > > Emmanuel Lécharny
> > > >> > > www.iktek.com
> > > >> > >
> > > >> > >
> > > >>
> > >
> >
>
>
>
> --
> thanks
> ashish
>
> Blog: http://www.ashishpaliwal.com/blog
> My Photo Galleries: http://www.pbase.com/ashishpaliwal
>
Re: codec "framework"
Posted by Ashish <pa...@gmail.com>.
hc has nio based implementation as well
http://hc.apache.org/httpcomponents-core-ga/httpcore-nio/xref/index.html
See nio.codecs package
On Tue, Jan 1, 2013 at 11:53 PM, Julien Vermillard <jv...@gmail.com>wrote:
> Definitively should take a look.
> The only tricky issue is streaming large content, because MINA have an
> event based paradigm where H.C. have probably a stream based approach.
>
> Julien
> Le 1 janv. 2013 16:33, "Arnaud bourree" <ar...@gmail.com> a
> écrit :
>
> > Hi,
> >
> > Happy new year 2013.
> >
> > I'm strongly interested in HTTP codec: I used for one project I did
> > for my company.
> > Here there are my point of view:
> > - share codec between MINA 2 and 3: I initially take MINA 3 as base to
> > implement on MINA 2 and keep classes and package structure. BTW share
> > should be easy to do, but when I did job, I made some change due to
> > JDK version 5 for MINA 2 and 6 for MINA 3
> > - codec independent from MINA: I take a look on the web to find HTTP
> > API object to convert BB to, I find that Apache HTTP Client
> > (http-core) feet what we need.
> > What do you think to used http-core as API for Pojo object and write
> > HTTP codec to encode/decode BB from/to http-core objects and on top a
> > HTTP filter for MINA framework?
> >
> > Regrards,
> >
> > Arnaud.
> >
> > 2013/1/1 Julien Vermillard <jv...@gmail.com>:
> > > Hi,
> > > I wanted to sleep, by my son wasn't agreeing :) I will probably crash
> > later.
> > >
> > > Yeah we could experiment with the HTTP codec, it's in pretty bad state
> > for
> > > now.
> > > It would be nice to be able to share the codec code between MINA 2 and
> 3.
> > > Julien
> > >
> > >
> > > On Tue, Jan 1, 2013 at 9:45 AM, Emmanuel Lecharny <
> elecharny@apache.org
> > >wrote:
> > >
> > >> we should think of a codec as an independant module : it should be
> > >> available for any java code that just needs suh a codec for its own
> > >> purpose.
> > >>
> > >> such a need has already been expressed for http.
> > >>
> > >> imo, the current impl is over-ingeniered.
> > >>
> > >> Btw, it seems that we are up and running at 9am on jan. first... crazy
> > open
> > >> source developpers...
> > >>
> > >> happy new year !
> > >>
> > >> Le 1 janv. 2013 09:26, "Julien Vermillard" <jv...@gmail.com> a
> > >> écrit :
> > >> >
> > >> > It's sure 10 year after SEDA is quite smelly :-)
> > >> > In my mind the codec code should be used by a filter for
> transforming
> > the
> > >> > bytes into pojos (like today) but really not dependent of MINA.
> > >> > IMHO demux handler is a piece of s..t, you should use a visitor
> > pattern.
> > >> > Much more testable.
> > >> >
> > >> > I like the loop until it's decoded idea, it very simple to
> understand.
> > >> > Le 31 déc. 2012 18:13, "Emmanuel Lécharny" <el...@gmail.com> a
> > >> écrit
> > >> :
> > >> >
> > >> > > Le 12/31/12 7:55 AM, Julien Vermillard a écrit :
> > >> > > > Hi,
> > >> > > >
> > >> > > > Since few year, I stopped to use the MINA ProtocolCodecFilter
> and
> > >> > > > associated stuff (CumulativeCodec..). for implementing my own
> > codec
> > >> > > > independent of MINA.
> > >> > > >
> > >> > > > it's just a service consuming ByteBuffer and pushing decoded
> POJO
> > in
> > >> a
> > >> > > > callback. The point is to be independent of MINA for example,
> > parse &
> > >> > > save
> > >> > > > files using the codec, or simply implement an HTTP version of
> the
> > >> > > transport
> > >> > > > using old style servlet.
> > >> > > >
> > >> > > > Basically a decoder looks like :
> https://gist.github.com/4417934
> > >> > > > One is instantiated by session.
> > >> > > >
> > >> > > > I'm quite happy with that and I think we should not port the old
> > >> > > > ProtocolCodeFilter to MINA 3.0 and replace it with a independent
> > MINA
> > >> > > async
> > >> > > > decoder framework (consuming BB, accumulating if needed and
> > producing
> > >> > > pojo).
> > >> > > It sounds a reasonnable proposal.
> > >> > >
> > >> > > If we think about it, decoding is not part of a filter chain : it
> > >> > > introduces a change of data type being passed from one filter to
> the
> > >> > > other, and if we have to cumulate data, we will just stop
> processing
> > >> the
> > >> > > incomming data in the middle of the chain, the handler being
> > unaware of
> > >> > > this fact.
> > >> > >
> > >> > >
> > >> > > Julien's proposal seems way better : the Handler would have a
> common
> > >> > > interface for encoding and decoding, used as a service when a
> > >> > > MessageReceived or a Write events are to be processed. This way,
> the
> > >> > > handler is fully in charge of all the aspects of the data
> > processing,
> > >> > > including the accumulation of data.
> > >> > >
> > >> > > It won't either eliminate the existence of pre-written codec, like
> > the
> > >> > > HttpCodec, or the Textline codec. We can even think about a chain
> of
> > >> > > codecs : one codec depends on the result of the previous codec.
> > >> > >
> > >> > > As far as I can tell, changing MINA this way will not impact
> > ApacheDS,
> > >> > > even if we are using a DemuxIoHandler (the handler called depends
> on
> > >> the
> > >> > > received message) : I don't see such a handler as a simplification
> > over
> > >> > > a simple switch...
> > >> > >
> > >> > >
> > >> > > Keep in mind that the exisiting MINA logic depends on an idea
> which
> > is
> > >> > > 10 years old : SEDA, and has not proven any advantage against
> > simpler
> > >> > > implementations. It's also important to notice that SEDA implies
> > that
> > >> > > each process part communicates with the next process (read :
> > filter) by
> > >> > > the use of queues. This is highly costly and memory consuming. I'm
> > not
> > >> > > sure that SEDA has anything to do with MINA implementation
> anwyay...
> > >> > >
> > >> > > On more thing : the current codec supposes that we pass a callback
> > >> which
> > >> > > is called as soon as something has been decoded. This make the
> code
> > >> > > extremely complicated to debug. I'd rather have a system where we
> > can
> > >> > > loop on the decoder, until it produces nothing. In other words,
> > instead
> > >> > > of having something like :
> > >> > >
> > >> > > void myCallback( IoSession session, Object message ) {
> > >> > > // Do something
> > >> > > }
> > >> > >
> > >> > > void decode( IoSession session, ByteBuffer buffer, callback ) {
> > >> > > // Decode and call the callback
> > >> > > }
> > >> > >
> > >> > >
> > >> > > void messageReceived( IoSession session, ByteBuffer buffer ) {
> > >> > > decode( session, myCalback );
> > >> > > ...
> > >> > > }
> > >> > >
> > >> > >
> > >> > > I would prefer something like :
> > >> > >
> > >> > > Object decode( IoSession session, ByteBuffer buffer ) {
> > >> > > // Decode
> > >> > >
> > >> > > return decoded;
> > >> > > }
> > >> > >
> > >> > >
> > >> > > void messageReceived( IoSession session, ByteBuffer buffer ) {
> > >> > > while ( ( Object decoded = decode( session ) ) != null ) {
> > >> > > // Do something
> > >> > > }
> > >> > > }
> > >> > >
> > >> > >
> > >> > >
> > >> > >
> > >> > > >
> > >> > > > Julien
> > >> > > >
> > >> > >
> > >> > >
> > >> > > --
> > >> > > Regards,
> > >> > > Cordialement,
> > >> > > Emmanuel Lécharny
> > >> > > www.iktek.com
> > >> > >
> > >> > >
> > >>
> >
>
--
thanks
ashish
Blog: http://www.ashishpaliwal.com/blog
My Photo Galleries: http://www.pbase.com/ashishpaliwal
Re: codec "framework"
Posted by Julien Vermillard <jv...@gmail.com>.
Definitively should take a look.
The only tricky issue is streaming large content, because MINA have an
event based paradigm where H.C. have probably a stream based approach.
Julien
Le 1 janv. 2013 16:33, "Arnaud bourree" <ar...@gmail.com> a écrit :
> Hi,
>
> Happy new year 2013.
>
> I'm strongly interested in HTTP codec: I used for one project I did
> for my company.
> Here there are my point of view:
> - share codec between MINA 2 and 3: I initially take MINA 3 as base to
> implement on MINA 2 and keep classes and package structure. BTW share
> should be easy to do, but when I did job, I made some change due to
> JDK version 5 for MINA 2 and 6 for MINA 3
> - codec independent from MINA: I take a look on the web to find HTTP
> API object to convert BB to, I find that Apache HTTP Client
> (http-core) feet what we need.
> What do you think to used http-core as API for Pojo object and write
> HTTP codec to encode/decode BB from/to http-core objects and on top a
> HTTP filter for MINA framework?
>
> Regrards,
>
> Arnaud.
>
> 2013/1/1 Julien Vermillard <jv...@gmail.com>:
> > Hi,
> > I wanted to sleep, by my son wasn't agreeing :) I will probably crash
> later.
> >
> > Yeah we could experiment with the HTTP codec, it's in pretty bad state
> for
> > now.
> > It would be nice to be able to share the codec code between MINA 2 and 3.
> > Julien
> >
> >
> > On Tue, Jan 1, 2013 at 9:45 AM, Emmanuel Lecharny <elecharny@apache.org
> >wrote:
> >
> >> we should think of a codec as an independant module : it should be
> >> available for any java code that just needs suh a codec for its own
> >> purpose.
> >>
> >> such a need has already been expressed for http.
> >>
> >> imo, the current impl is over-ingeniered.
> >>
> >> Btw, it seems that we are up and running at 9am on jan. first... crazy
> open
> >> source developpers...
> >>
> >> happy new year !
> >>
> >> Le 1 janv. 2013 09:26, "Julien Vermillard" <jv...@gmail.com> a
> >> écrit :
> >> >
> >> > It's sure 10 year after SEDA is quite smelly :-)
> >> > In my mind the codec code should be used by a filter for transforming
> the
> >> > bytes into pojos (like today) but really not dependent of MINA.
> >> > IMHO demux handler is a piece of s..t, you should use a visitor
> pattern.
> >> > Much more testable.
> >> >
> >> > I like the loop until it's decoded idea, it very simple to understand.
> >> > Le 31 déc. 2012 18:13, "Emmanuel Lécharny" <el...@gmail.com> a
> >> écrit
> >> :
> >> >
> >> > > Le 12/31/12 7:55 AM, Julien Vermillard a écrit :
> >> > > > Hi,
> >> > > >
> >> > > > Since few year, I stopped to use the MINA ProtocolCodecFilter and
> >> > > > associated stuff (CumulativeCodec..). for implementing my own
> codec
> >> > > > independent of MINA.
> >> > > >
> >> > > > it's just a service consuming ByteBuffer and pushing decoded POJO
> in
> >> a
> >> > > > callback. The point is to be independent of MINA for example,
> parse &
> >> > > save
> >> > > > files using the codec, or simply implement an HTTP version of the
> >> > > transport
> >> > > > using old style servlet.
> >> > > >
> >> > > > Basically a decoder looks like : https://gist.github.com/4417934
> >> > > > One is instantiated by session.
> >> > > >
> >> > > > I'm quite happy with that and I think we should not port the old
> >> > > > ProtocolCodeFilter to MINA 3.0 and replace it with a independent
> MINA
> >> > > async
> >> > > > decoder framework (consuming BB, accumulating if needed and
> producing
> >> > > pojo).
> >> > > It sounds a reasonnable proposal.
> >> > >
> >> > > If we think about it, decoding is not part of a filter chain : it
> >> > > introduces a change of data type being passed from one filter to the
> >> > > other, and if we have to cumulate data, we will just stop processing
> >> the
> >> > > incomming data in the middle of the chain, the handler being
> unaware of
> >> > > this fact.
> >> > >
> >> > >
> >> > > Julien's proposal seems way better : the Handler would have a common
> >> > > interface for encoding and decoding, used as a service when a
> >> > > MessageReceived or a Write events are to be processed. This way, the
> >> > > handler is fully in charge of all the aspects of the data
> processing,
> >> > > including the accumulation of data.
> >> > >
> >> > > It won't either eliminate the existence of pre-written codec, like
> the
> >> > > HttpCodec, or the Textline codec. We can even think about a chain of
> >> > > codecs : one codec depends on the result of the previous codec.
> >> > >
> >> > > As far as I can tell, changing MINA this way will not impact
> ApacheDS,
> >> > > even if we are using a DemuxIoHandler (the handler called depends on
> >> the
> >> > > received message) : I don't see such a handler as a simplification
> over
> >> > > a simple switch...
> >> > >
> >> > >
> >> > > Keep in mind that the exisiting MINA logic depends on an idea which
> is
> >> > > 10 years old : SEDA, and has not proven any advantage against
> simpler
> >> > > implementations. It's also important to notice that SEDA implies
> that
> >> > > each process part communicates with the next process (read :
> filter) by
> >> > > the use of queues. This is highly costly and memory consuming. I'm
> not
> >> > > sure that SEDA has anything to do with MINA implementation anwyay...
> >> > >
> >> > > On more thing : the current codec supposes that we pass a callback
> >> which
> >> > > is called as soon as something has been decoded. This make the code
> >> > > extremely complicated to debug. I'd rather have a system where we
> can
> >> > > loop on the decoder, until it produces nothing. In other words,
> instead
> >> > > of having something like :
> >> > >
> >> > > void myCallback( IoSession session, Object message ) {
> >> > > // Do something
> >> > > }
> >> > >
> >> > > void decode( IoSession session, ByteBuffer buffer, callback ) {
> >> > > // Decode and call the callback
> >> > > }
> >> > >
> >> > >
> >> > > void messageReceived( IoSession session, ByteBuffer buffer ) {
> >> > > decode( session, myCalback );
> >> > > ...
> >> > > }
> >> > >
> >> > >
> >> > > I would prefer something like :
> >> > >
> >> > > Object decode( IoSession session, ByteBuffer buffer ) {
> >> > > // Decode
> >> > >
> >> > > return decoded;
> >> > > }
> >> > >
> >> > >
> >> > > void messageReceived( IoSession session, ByteBuffer buffer ) {
> >> > > while ( ( Object decoded = decode( session ) ) != null ) {
> >> > > // Do something
> >> > > }
> >> > > }
> >> > >
> >> > >
> >> > >
> >> > >
> >> > > >
> >> > > > Julien
> >> > > >
> >> > >
> >> > >
> >> > > --
> >> > > Regards,
> >> > > Cordialement,
> >> > > Emmanuel Lécharny
> >> > > www.iktek.com
> >> > >
> >> > >
> >>
>
Re: codec "framework"
Posted by Arnaud bourree <ar...@gmail.com>.
Hi,
Happy new year 2013.
I'm strongly interested in HTTP codec: I used for one project I did
for my company.
Here there are my point of view:
- share codec between MINA 2 and 3: I initially take MINA 3 as base to
implement on MINA 2 and keep classes and package structure. BTW share
should be easy to do, but when I did job, I made some change due to
JDK version 5 for MINA 2 and 6 for MINA 3
- codec independent from MINA: I take a look on the web to find HTTP
API object to convert BB to, I find that Apache HTTP Client
(http-core) feet what we need.
What do you think to used http-core as API for Pojo object and write
HTTP codec to encode/decode BB from/to http-core objects and on top a
HTTP filter for MINA framework?
Regrards,
Arnaud.
2013/1/1 Julien Vermillard <jv...@gmail.com>:
> Hi,
> I wanted to sleep, by my son wasn't agreeing :) I will probably crash later.
>
> Yeah we could experiment with the HTTP codec, it's in pretty bad state for
> now.
> It would be nice to be able to share the codec code between MINA 2 and 3.
> Julien
>
>
> On Tue, Jan 1, 2013 at 9:45 AM, Emmanuel Lecharny <el...@apache.org>wrote:
>
>> we should think of a codec as an independant module : it should be
>> available for any java code that just needs suh a codec for its own
>> purpose.
>>
>> such a need has already been expressed for http.
>>
>> imo, the current impl is over-ingeniered.
>>
>> Btw, it seems that we are up and running at 9am on jan. first... crazy open
>> source developpers...
>>
>> happy new year !
>>
>> Le 1 janv. 2013 09:26, "Julien Vermillard" <jv...@gmail.com> a
>> écrit :
>> >
>> > It's sure 10 year after SEDA is quite smelly :-)
>> > In my mind the codec code should be used by a filter for transforming the
>> > bytes into pojos (like today) but really not dependent of MINA.
>> > IMHO demux handler is a piece of s..t, you should use a visitor pattern.
>> > Much more testable.
>> >
>> > I like the loop until it's decoded idea, it very simple to understand.
>> > Le 31 déc. 2012 18:13, "Emmanuel Lécharny" <el...@gmail.com> a
>> écrit
>> :
>> >
>> > > Le 12/31/12 7:55 AM, Julien Vermillard a écrit :
>> > > > Hi,
>> > > >
>> > > > Since few year, I stopped to use the MINA ProtocolCodecFilter and
>> > > > associated stuff (CumulativeCodec..). for implementing my own codec
>> > > > independent of MINA.
>> > > >
>> > > > it's just a service consuming ByteBuffer and pushing decoded POJO in
>> a
>> > > > callback. The point is to be independent of MINA for example, parse &
>> > > save
>> > > > files using the codec, or simply implement an HTTP version of the
>> > > transport
>> > > > using old style servlet.
>> > > >
>> > > > Basically a decoder looks like : https://gist.github.com/4417934
>> > > > One is instantiated by session.
>> > > >
>> > > > I'm quite happy with that and I think we should not port the old
>> > > > ProtocolCodeFilter to MINA 3.0 and replace it with a independent MINA
>> > > async
>> > > > decoder framework (consuming BB, accumulating if needed and producing
>> > > pojo).
>> > > It sounds a reasonnable proposal.
>> > >
>> > > If we think about it, decoding is not part of a filter chain : it
>> > > introduces a change of data type being passed from one filter to the
>> > > other, and if we have to cumulate data, we will just stop processing
>> the
>> > > incomming data in the middle of the chain, the handler being unaware of
>> > > this fact.
>> > >
>> > >
>> > > Julien's proposal seems way better : the Handler would have a common
>> > > interface for encoding and decoding, used as a service when a
>> > > MessageReceived or a Write events are to be processed. This way, the
>> > > handler is fully in charge of all the aspects of the data processing,
>> > > including the accumulation of data.
>> > >
>> > > It won't either eliminate the existence of pre-written codec, like the
>> > > HttpCodec, or the Textline codec. We can even think about a chain of
>> > > codecs : one codec depends on the result of the previous codec.
>> > >
>> > > As far as I can tell, changing MINA this way will not impact ApacheDS,
>> > > even if we are using a DemuxIoHandler (the handler called depends on
>> the
>> > > received message) : I don't see such a handler as a simplification over
>> > > a simple switch...
>> > >
>> > >
>> > > Keep in mind that the exisiting MINA logic depends on an idea which is
>> > > 10 years old : SEDA, and has not proven any advantage against simpler
>> > > implementations. It's also important to notice that SEDA implies that
>> > > each process part communicates with the next process (read : filter) by
>> > > the use of queues. This is highly costly and memory consuming. I'm not
>> > > sure that SEDA has anything to do with MINA implementation anwyay...
>> > >
>> > > On more thing : the current codec supposes that we pass a callback
>> which
>> > > is called as soon as something has been decoded. This make the code
>> > > extremely complicated to debug. I'd rather have a system where we can
>> > > loop on the decoder, until it produces nothing. In other words, instead
>> > > of having something like :
>> > >
>> > > void myCallback( IoSession session, Object message ) {
>> > > // Do something
>> > > }
>> > >
>> > > void decode( IoSession session, ByteBuffer buffer, callback ) {
>> > > // Decode and call the callback
>> > > }
>> > >
>> > >
>> > > void messageReceived( IoSession session, ByteBuffer buffer ) {
>> > > decode( session, myCalback );
>> > > ...
>> > > }
>> > >
>> > >
>> > > I would prefer something like :
>> > >
>> > > Object decode( IoSession session, ByteBuffer buffer ) {
>> > > // Decode
>> > >
>> > > return decoded;
>> > > }
>> > >
>> > >
>> > > void messageReceived( IoSession session, ByteBuffer buffer ) {
>> > > while ( ( Object decoded = decode( session ) ) != null ) {
>> > > // Do something
>> > > }
>> > > }
>> > >
>> > >
>> > >
>> > >
>> > > >
>> > > > Julien
>> > > >
>> > >
>> > >
>> > > --
>> > > Regards,
>> > > Cordialement,
>> > > Emmanuel Lécharny
>> > > www.iktek.com
>> > >
>> > >
>>
Re: codec "framework"
Posted by Julien Vermillard <jv...@gmail.com>.
Hi,
I wanted to sleep, by my son wasn't agreeing :) I will probably crash later.
Yeah we could experiment with the HTTP codec, it's in pretty bad state for
now.
It would be nice to be able to share the codec code between MINA 2 and 3.
Julien
On Tue, Jan 1, 2013 at 9:45 AM, Emmanuel Lecharny <el...@apache.org>wrote:
> we should think of a codec as an independant module : it should be
> available for any java code that just needs suh a codec for its own
> purpose.
>
> such a need has already been expressed for http.
>
> imo, the current impl is over-ingeniered.
>
> Btw, it seems that we are up and running at 9am on jan. first... crazy open
> source developpers...
>
> happy new year !
>
> Le 1 janv. 2013 09:26, "Julien Vermillard" <jv...@gmail.com> a
> écrit :
> >
> > It's sure 10 year after SEDA is quite smelly :-)
> > In my mind the codec code should be used by a filter for transforming the
> > bytes into pojos (like today) but really not dependent of MINA.
> > IMHO demux handler is a piece of s..t, you should use a visitor pattern.
> > Much more testable.
> >
> > I like the loop until it's decoded idea, it very simple to understand.
> > Le 31 déc. 2012 18:13, "Emmanuel Lécharny" <el...@gmail.com> a
> écrit
> :
> >
> > > Le 12/31/12 7:55 AM, Julien Vermillard a écrit :
> > > > Hi,
> > > >
> > > > Since few year, I stopped to use the MINA ProtocolCodecFilter and
> > > > associated stuff (CumulativeCodec..). for implementing my own codec
> > > > independent of MINA.
> > > >
> > > > it's just a service consuming ByteBuffer and pushing decoded POJO in
> a
> > > > callback. The point is to be independent of MINA for example, parse &
> > > save
> > > > files using the codec, or simply implement an HTTP version of the
> > > transport
> > > > using old style servlet.
> > > >
> > > > Basically a decoder looks like : https://gist.github.com/4417934
> > > > One is instantiated by session.
> > > >
> > > > I'm quite happy with that and I think we should not port the old
> > > > ProtocolCodeFilter to MINA 3.0 and replace it with a independent MINA
> > > async
> > > > decoder framework (consuming BB, accumulating if needed and producing
> > > pojo).
> > > It sounds a reasonnable proposal.
> > >
> > > If we think about it, decoding is not part of a filter chain : it
> > > introduces a change of data type being passed from one filter to the
> > > other, and if we have to cumulate data, we will just stop processing
> the
> > > incomming data in the middle of the chain, the handler being unaware of
> > > this fact.
> > >
> > >
> > > Julien's proposal seems way better : the Handler would have a common
> > > interface for encoding and decoding, used as a service when a
> > > MessageReceived or a Write events are to be processed. This way, the
> > > handler is fully in charge of all the aspects of the data processing,
> > > including the accumulation of data.
> > >
> > > It won't either eliminate the existence of pre-written codec, like the
> > > HttpCodec, or the Textline codec. We can even think about a chain of
> > > codecs : one codec depends on the result of the previous codec.
> > >
> > > As far as I can tell, changing MINA this way will not impact ApacheDS,
> > > even if we are using a DemuxIoHandler (the handler called depends on
> the
> > > received message) : I don't see such a handler as a simplification over
> > > a simple switch...
> > >
> > >
> > > Keep in mind that the exisiting MINA logic depends on an idea which is
> > > 10 years old : SEDA, and has not proven any advantage against simpler
> > > implementations. It's also important to notice that SEDA implies that
> > > each process part communicates with the next process (read : filter) by
> > > the use of queues. This is highly costly and memory consuming. I'm not
> > > sure that SEDA has anything to do with MINA implementation anwyay...
> > >
> > > On more thing : the current codec supposes that we pass a callback
> which
> > > is called as soon as something has been decoded. This make the code
> > > extremely complicated to debug. I'd rather have a system where we can
> > > loop on the decoder, until it produces nothing. In other words, instead
> > > of having something like :
> > >
> > > void myCallback( IoSession session, Object message ) {
> > > // Do something
> > > }
> > >
> > > void decode( IoSession session, ByteBuffer buffer, callback ) {
> > > // Decode and call the callback
> > > }
> > >
> > >
> > > void messageReceived( IoSession session, ByteBuffer buffer ) {
> > > decode( session, myCalback );
> > > ...
> > > }
> > >
> > >
> > > I would prefer something like :
> > >
> > > Object decode( IoSession session, ByteBuffer buffer ) {
> > > // Decode
> > >
> > > return decoded;
> > > }
> > >
> > >
> > > void messageReceived( IoSession session, ByteBuffer buffer ) {
> > > while ( ( Object decoded = decode( session ) ) != null ) {
> > > // Do something
> > > }
> > > }
> > >
> > >
> > >
> > >
> > > >
> > > > Julien
> > > >
> > >
> > >
> > > --
> > > Regards,
> > > Cordialement,
> > > Emmanuel Lécharny
> > > www.iktek.com
> > >
> > >
>
Re: codec "framework"
Posted by Emmanuel Lecharny <el...@apache.org>.
we should think of a codec as an independant module : it should be
available for any java code that just needs suh a codec for its own purpose.
such a need has already been expressed for http.
imo, the current impl is over-ingeniered.
Btw, it seems that we are up and running at 9am on jan. first... crazy open
source developpers...
happy new year !
Le 1 janv. 2013 09:26, "Julien Vermillard" <jv...@gmail.com> a écrit :
>
> It's sure 10 year after SEDA is quite smelly :-)
> In my mind the codec code should be used by a filter for transforming the
> bytes into pojos (like today) but really not dependent of MINA.
> IMHO demux handler is a piece of s..t, you should use a visitor pattern.
> Much more testable.
>
> I like the loop until it's decoded idea, it very simple to understand.
> Le 31 déc. 2012 18:13, "Emmanuel Lécharny" <el...@gmail.com> a écrit
:
>
> > Le 12/31/12 7:55 AM, Julien Vermillard a écrit :
> > > Hi,
> > >
> > > Since few year, I stopped to use the MINA ProtocolCodecFilter and
> > > associated stuff (CumulativeCodec..). for implementing my own codec
> > > independent of MINA.
> > >
> > > it's just a service consuming ByteBuffer and pushing decoded POJO in a
> > > callback. The point is to be independent of MINA for example, parse &
> > save
> > > files using the codec, or simply implement an HTTP version of the
> > transport
> > > using old style servlet.
> > >
> > > Basically a decoder looks like : https://gist.github.com/4417934
> > > One is instantiated by session.
> > >
> > > I'm quite happy with that and I think we should not port the old
> > > ProtocolCodeFilter to MINA 3.0 and replace it with a independent MINA
> > async
> > > decoder framework (consuming BB, accumulating if needed and producing
> > pojo).
> > It sounds a reasonnable proposal.
> >
> > If we think about it, decoding is not part of a filter chain : it
> > introduces a change of data type being passed from one filter to the
> > other, and if we have to cumulate data, we will just stop processing the
> > incomming data in the middle of the chain, the handler being unaware of
> > this fact.
> >
> >
> > Julien's proposal seems way better : the Handler would have a common
> > interface for encoding and decoding, used as a service when a
> > MessageReceived or a Write events are to be processed. This way, the
> > handler is fully in charge of all the aspects of the data processing,
> > including the accumulation of data.
> >
> > It won't either eliminate the existence of pre-written codec, like the
> > HttpCodec, or the Textline codec. We can even think about a chain of
> > codecs : one codec depends on the result of the previous codec.
> >
> > As far as I can tell, changing MINA this way will not impact ApacheDS,
> > even if we are using a DemuxIoHandler (the handler called depends on the
> > received message) : I don't see such a handler as a simplification over
> > a simple switch...
> >
> >
> > Keep in mind that the exisiting MINA logic depends on an idea which is
> > 10 years old : SEDA, and has not proven any advantage against simpler
> > implementations. It's also important to notice that SEDA implies that
> > each process part communicates with the next process (read : filter) by
> > the use of queues. This is highly costly and memory consuming. I'm not
> > sure that SEDA has anything to do with MINA implementation anwyay...
> >
> > On more thing : the current codec supposes that we pass a callback which
> > is called as soon as something has been decoded. This make the code
> > extremely complicated to debug. I'd rather have a system where we can
> > loop on the decoder, until it produces nothing. In other words, instead
> > of having something like :
> >
> > void myCallback( IoSession session, Object message ) {
> > // Do something
> > }
> >
> > void decode( IoSession session, ByteBuffer buffer, callback ) {
> > // Decode and call the callback
> > }
> >
> >
> > void messageReceived( IoSession session, ByteBuffer buffer ) {
> > decode( session, myCalback );
> > ...
> > }
> >
> >
> > I would prefer something like :
> >
> > Object decode( IoSession session, ByteBuffer buffer ) {
> > // Decode
> >
> > return decoded;
> > }
> >
> >
> > void messageReceived( IoSession session, ByteBuffer buffer ) {
> > while ( ( Object decoded = decode( session ) ) != null ) {
> > // Do something
> > }
> > }
> >
> >
> >
> >
> > >
> > > Julien
> > >
> >
> >
> > --
> > Regards,
> > Cordialement,
> > Emmanuel Lécharny
> > www.iktek.com
> >
> >
Re: codec "framework"
Posted by Julien Vermillard <jv...@gmail.com>.
It's sure 10 year after SEDA is quite smelly :-)
In my mind the codec code should be used by a filter for transforming the
bytes into pojos (like today) but really not dependent of MINA.
IMHO demux handler is a piece of s..t, you should use a visitor pattern.
Much more testable.
I like the loop until it's decoded idea, it very simple to understand.
Le 31 déc. 2012 18:13, "Emmanuel Lécharny" <el...@gmail.com> a écrit :
> Le 12/31/12 7:55 AM, Julien Vermillard a écrit :
> > Hi,
> >
> > Since few year, I stopped to use the MINA ProtocolCodecFilter and
> > associated stuff (CumulativeCodec..). for implementing my own codec
> > independent of MINA.
> >
> > it's just a service consuming ByteBuffer and pushing decoded POJO in a
> > callback. The point is to be independent of MINA for example, parse &
> save
> > files using the codec, or simply implement an HTTP version of the
> transport
> > using old style servlet.
> >
> > Basically a decoder looks like : https://gist.github.com/4417934
> > One is instantiated by session.
> >
> > I'm quite happy with that and I think we should not port the old
> > ProtocolCodeFilter to MINA 3.0 and replace it with a independent MINA
> async
> > decoder framework (consuming BB, accumulating if needed and producing
> pojo).
> It sounds a reasonnable proposal.
>
> If we think about it, decoding is not part of a filter chain : it
> introduces a change of data type being passed from one filter to the
> other, and if we have to cumulate data, we will just stop processing the
> incomming data in the middle of the chain, the handler being unaware of
> this fact.
>
>
> Julien's proposal seems way better : the Handler would have a common
> interface for encoding and decoding, used as a service when a
> MessageReceived or a Write events are to be processed. This way, the
> handler is fully in charge of all the aspects of the data processing,
> including the accumulation of data.
>
> It won't either eliminate the existence of pre-written codec, like the
> HttpCodec, or the Textline codec. We can even think about a chain of
> codecs : one codec depends on the result of the previous codec.
>
> As far as I can tell, changing MINA this way will not impact ApacheDS,
> even if we are using a DemuxIoHandler (the handler called depends on the
> received message) : I don't see such a handler as a simplification over
> a simple switch...
>
>
> Keep in mind that the exisiting MINA logic depends on an idea which is
> 10 years old : SEDA, and has not proven any advantage against simpler
> implementations. It's also important to notice that SEDA implies that
> each process part communicates with the next process (read : filter) by
> the use of queues. This is highly costly and memory consuming. I'm not
> sure that SEDA has anything to do with MINA implementation anwyay...
>
> On more thing : the current codec supposes that we pass a callback which
> is called as soon as something has been decoded. This make the code
> extremely complicated to debug. I'd rather have a system where we can
> loop on the decoder, until it produces nothing. In other words, instead
> of having something like :
>
> void myCallback( IoSession session, Object message ) {
> // Do something
> }
>
> void decode( IoSession session, ByteBuffer buffer, callback ) {
> // Decode and call the callback
> }
>
>
> void messageReceived( IoSession session, ByteBuffer buffer ) {
> decode( session, myCalback );
> ...
> }
>
>
> I would prefer something like :
>
> Object decode( IoSession session, ByteBuffer buffer ) {
> // Decode
>
> return decoded;
> }
>
>
> void messageReceived( IoSession session, ByteBuffer buffer ) {
> while ( ( Object decoded = decode( session ) ) != null ) {
> // Do something
> }
> }
>
>
>
>
> >
> > Julien
> >
>
>
> --
> Regards,
> Cordialement,
> Emmanuel Lécharny
> www.iktek.com
>
>
Re: codec "framework"
Posted by Emmanuel Lécharny <el...@gmail.com>.
Le 12/31/12 7:55 AM, Julien Vermillard a écrit :
> Hi,
>
> Since few year, I stopped to use the MINA ProtocolCodecFilter and
> associated stuff (CumulativeCodec..). for implementing my own codec
> independent of MINA.
>
> it's just a service consuming ByteBuffer and pushing decoded POJO in a
> callback. The point is to be independent of MINA for example, parse & save
> files using the codec, or simply implement an HTTP version of the transport
> using old style servlet.
>
> Basically a decoder looks like : https://gist.github.com/4417934
> One is instantiated by session.
>
> I'm quite happy with that and I think we should not port the old
> ProtocolCodeFilter to MINA 3.0 and replace it with a independent MINA async
> decoder framework (consuming BB, accumulating if needed and producing pojo).
It sounds a reasonnable proposal.
If we think about it, decoding is not part of a filter chain : it
introduces a change of data type being passed from one filter to the
other, and if we have to cumulate data, we will just stop processing the
incomming data in the middle of the chain, the handler being unaware of
this fact.
Julien's proposal seems way better : the Handler would have a common
interface for encoding and decoding, used as a service when a
MessageReceived or a Write events are to be processed. This way, the
handler is fully in charge of all the aspects of the data processing,
including the accumulation of data.
It won't either eliminate the existence of pre-written codec, like the
HttpCodec, or the Textline codec. We can even think about a chain of
codecs : one codec depends on the result of the previous codec.
As far as I can tell, changing MINA this way will not impact ApacheDS,
even if we are using a DemuxIoHandler (the handler called depends on the
received message) : I don't see such a handler as a simplification over
a simple switch...
Keep in mind that the exisiting MINA logic depends on an idea which is
10 years old : SEDA, and has not proven any advantage against simpler
implementations. It's also important to notice that SEDA implies that
each process part communicates with the next process (read : filter) by
the use of queues. This is highly costly and memory consuming. I'm not
sure that SEDA has anything to do with MINA implementation anwyay...
On more thing : the current codec supposes that we pass a callback which
is called as soon as something has been decoded. This make the code
extremely complicated to debug. I'd rather have a system where we can
loop on the decoder, until it produces nothing. In other words, instead
of having something like :
void myCallback( IoSession session, Object message ) {
// Do something
}
void decode( IoSession session, ByteBuffer buffer, callback ) {
// Decode and call the callback
}
void messageReceived( IoSession session, ByteBuffer buffer ) {
decode( session, myCalback );
...
}
I would prefer something like :
Object decode( IoSession session, ByteBuffer buffer ) {
// Decode
return decoded;
}
void messageReceived( IoSession session, ByteBuffer buffer ) {
while ( ( Object decoded = decode( session ) ) != null ) {
// Do something
}
}
>
> Julien
>
--
Regards,
Cordialement,
Emmanuel Lécharny
www.iktek.com