Do we need to keep the encoder/decoder Provider stuff ?

[Emmanuel Lecharny <el...@gmail.com>]

  Hi,

once upon a time, ie, 8 years ago, Alex codec the network frontend by 
assuming that there will be some other codec used later. It made perfect 
sense back then, as he first used an ASN.1 based codec named Snacc, 
which was not ASL.2 compatible. Then he had to change and define its own 
version of an ASN.1 codec, but as he wanted to check that the new codec 
was valid, he designed the system to be able to switch the codec just to 
know if the new codec was responsible for the failures he met, or 
something else.

The rational was :
- if the server behaves the same way whether Snacc or Snickers (the new 
codec) is used, then if the server has an issue, it means the codec is 
not responsible
- OTOH, if the server behaves differently, then it's likey the new codec 
the cause of the problem.

That was a smart move, and it was leveraged again when I wrote Twix, the 
third codec. I was able to do the same thing : compare the results 
obtained with twix with what was obtained with Snickers.

Eventually, we decided to ditch Snickers and to keep Twix, which was 
renamed.

So as of today, we use the so-called twix internally.

Now, we still have a lot of remaining plumbery in place : all the 
classes needed to declare which codec to use. Mainly, we have :
- Provider, an abstract class in charge of class loading the LdapProvider
- LdapProvider, a class used to create the LdapEncoder and LdapDecoder

Those two classes are created when we create the FilterChain as soon as 
a new session is initialized. More specifically, the ProtocolCodecFilter 
is added into the chain, and before we can process any incoming messages 
from the client, we initialize the codec, using the 
LdapProtocolCodecFactory class to create the encoder/decoder instances.

What are the essential elements ?
- the factory
- the encoder
- the decoder

We should be able to get rid of the provider, assuming we won't even 
switch the codec in the near or distant future. That would simplify the 
code which is, to say the least, cryptic...

so, wdyt ? Should we keep the machinery that allow us to change the 
codec by using an environment variable ?

-- 
Regards,
Cordialement,
Emmanuel Lécharny
www.iktek.com

Re: Do we need to keep the encoder/decoder Provider stuff ?

[Emmanuel Lecharny <el...@gmail.com>]

  On 8/11/10 10:21 PM, Alex Karasulu wrote:
> On Wed, Aug 11, 2010 at 11:16 PM, Alex Karasulu<ak...@apache.org>wrote:
>
>> Hi Emmanuel, Kiran, guys n gals,
>>
>> On Mon, Aug 9, 2010 at 9:19 PM, Kiran Ayyagari<ka...@apache.org>wrote:
>>
>>>
>>>> so, wdyt ? Should we keep the machinery that allow us to change the
>>> codec by
>>>> using an environment variable ?
>>> IMHO let us keep it, AFAIU this is transparent to the end or API user
>>> right? if yes, then I think it is
>>> better to remove it in another release.
>>>
>>> P.S:- perhaps this will save us some debug headaches, if at all arise,
>>> as we are making great
>>>          level of refactoring for the upcoming 2.0
>>>
>>>
>> If Kiran is correct about the exposure to the API end user, then I agree
>> with Kiran here unless of course there's a lot of headache you're dealing
>> with because of this Emmanuel. If you feel it's too much of a problem just
>> nix it.
>>
> BTW reverting this change if we have to might not be so hard if in fact we
> feel that need. However if we do add some new capabilities like streaming
> content (if you remember all the Value based work we experimented with) you
> might want to leave it to test codec variation to see if bugs are due to one
> codec implementation. Then again it's not so big of a change to revert if we
> do need it. So it seems I can go either way :-).
Right now, I'm merging all the different Messages (InternaLMessage, 
MessageCodec and API messages) so that we don't have so many translation 
going on in the client and server. That will help to clarify the situation.

OTOH, I think that the provider approach is a good idea, but all those 
callback cr*pities are really a PITA, and I'd like to get rid of them. 
The ldap-api is way more simpler than the server, and it works well, so 
I think there is some room for improvement here.

But let's first clean up the place...


-- 
Regards,
Cordialement,
Emmanuel Lécharny
www.iktek.com

Re: Do we need to keep the encoder/decoder Provider stuff ?

[Alex Karasulu <ak...@apache.org>]

On Wed, Aug 11, 2010 at 11:16 PM, Alex Karasulu <ak...@apache.org>wrote:

> Hi Emmanuel, Kiran, guys n gals,
>
> On Mon, Aug 9, 2010 at 9:19 PM, Kiran Ayyagari <ka...@apache.org>wrote:
>
>>
>>
>
>> > so, wdyt ? Should we keep the machinery that allow us to change the
>> codec by
>> > using an environment variable ?
>> IMHO let us keep it, AFAIU this is transparent to the end or API user
>> right? if yes, then I think it is
>> better to remove it in another release.
>>
>> P.S:- perhaps this will save us some debug headaches, if at all arise,
>> as we are making great
>>         level of refactoring for the upcoming 2.0
>>
>>
> If Kiran is correct about the exposure to the API end user, then I agree
> with Kiran here unless of course there's a lot of headache you're dealing
> with because of this Emmanuel. If you feel it's too much of a problem just
> nix it.
>

BTW reverting this change if we have to might not be so hard if in fact we
feel that need. However if we do add some new capabilities like streaming
content (if you remember all the Value based work we experimented with) you
might want to leave it to test codec variation to see if bugs are due to one
codec implementation. Then again it's not so big of a change to revert if we
do need it. So it seems I can go either way :-).

-- 
Alex Karasulu
My Blog :: http://www.jroller.com/akarasulu/
Apache Directory Server :: http://directory.apache.org
Apache MINA :: http://mina.apache.org
To set up a meeting with me: http://tungle.me/AlexKarasulu

Re: Do we need to keep the encoder/decoder Provider stuff ?

[Alex Karasulu <ak...@apache.org>]

Hi Emmanuel, Kiran, guys n gals,

On Mon, Aug 9, 2010 at 9:19 PM, Kiran Ayyagari <ka...@apache.org> wrote:

> On Mon, Aug 9, 2010 at 10:56 PM, Emmanuel Lecharny <el...@gmail.com>
> wrote:
> >  Hi,
> >
> > once upon a time, ie, 8 years ago, Alex codec the network frontend by
> > assuming that there will be some other codec used later. It made perfect
> > sense back then, as he first used an ASN.1 based codec named Snacc, which
> > was not ASL.2 compatible. Then he had to change and define its own
> version
> > of an ASN.1 codec, but as he wanted to check that the new codec was
> valid,
> > he designed the system to be able to switch the codec just to know if the
> > new codec was responsible for the failures he met, or something else.
> >
> > The rational was :
> > - if the server behaves the same way whether Snacc or Snickers (the new
> > codec) is used, then if the server has an issue, it means the codec is
> not
> > responsible
> > - OTOH, if the server behaves differently, then it's likey the new codec
> the
> > cause of the problem.
> >
> > That was a smart move, and it was leveraged again when I wrote Twix, the
> > third codec. I was able to do the same thing : compare the results
> obtained
> > with twix with what was obtained with Snickers.
> >
> > Eventually, we decided to ditch Snickers and to keep Twix, which was
> > renamed.
> >
> > So as of today, we use the so-called twix internally.
> >
>

Real nice recap of our codec development history. I enjoyed reading it and
remembering the last 8 years.

> Now, we still have a lot of remaining plumbery in place : all the classes
> > needed to declare which codec to use. Mainly, we have :
> > - Provider, an abstract class in charge of class loading the LdapProvider
> > - LdapProvider, a class used to create the LdapEncoder and LdapDecoder
> >
> > Those two classes are created when we create the FilterChain as soon as a
> > new session is initialized. More specifically, the ProtocolCodecFilter is
> > added into the chain, and before we can process any incoming messages
> from
> > the client, we initialize the codec, using the LdapProtocolCodecFactory
> > class to create the encoder/decoder instances.
> >
> > What are the essential elements ?
> > - the factory
> > - the encoder
> > - the decoder
> >
> > We should be able to get rid of the provider, assuming we won't even
> switch
> > the codec in the near or distant future. That would simplify the code
> which
> > is, to say the least, cryptic...
>
>
Exactly how much code are we talking about getting rid of? If I remember
correctly there was not that much to this code. Hopefully as Kiran writes
below these details are not exposed to the end user of the API however they
do have the power to change the codec. It by default uses the TWIX codec.


> > so, wdyt ? Should we keep the machinery that allow us to change the codec
> by
> > using an environment variable ?
> IMHO let us keep it, AFAIU this is transparent to the end or API user
> right? if yes, then I think it is
> better to remove it in another release.
>
> P.S:- perhaps this will save us some debug headaches, if at all arise,
> as we are making great
>         level of refactoring for the upcoming 2.0
>
>
If Kiran is correct about the exposure to the API end user, then I agree
with Kiran here unless of course there's a lot of headache you're dealing
with because of this Emmanuel. If you feel it's too much of a problem just
nix it.

Thanks,
-- 
Alex Karasulu
My Blog :: http://www.jroller.com/akarasulu/
Apache Directory Server :: http://directory.apache.org
Apache MINA :: http://mina.apache.org
To set up a meeting with me: http://tungle.me/AlexKarasulu

Re: Do we need to keep the encoder/decoder Provider stuff ?

[Kiran Ayyagari <ka...@apache.org>]

On Mon, Aug 9, 2010 at 10:56 PM, Emmanuel Lecharny <el...@gmail.com> wrote:
>  Hi,
>
> once upon a time, ie, 8 years ago, Alex codec the network frontend by
> assuming that there will be some other codec used later. It made perfect
> sense back then, as he first used an ASN.1 based codec named Snacc, which
> was not ASL.2 compatible. Then he had to change and define its own version
> of an ASN.1 codec, but as he wanted to check that the new codec was valid,
> he designed the system to be able to switch the codec just to know if the
> new codec was responsible for the failures he met, or something else.
>
> The rational was :
> - if the server behaves the same way whether Snacc or Snickers (the new
> codec) is used, then if the server has an issue, it means the codec is not
> responsible
> - OTOH, if the server behaves differently, then it's likey the new codec the
> cause of the problem.
>
> That was a smart move, and it was leveraged again when I wrote Twix, the
> third codec. I was able to do the same thing : compare the results obtained
> with twix with what was obtained with Snickers.
>
> Eventually, we decided to ditch Snickers and to keep Twix, which was
> renamed.
>
> So as of today, we use the so-called twix internally.
>
> Now, we still have a lot of remaining plumbery in place : all the classes
> needed to declare which codec to use. Mainly, we have :
> - Provider, an abstract class in charge of class loading the LdapProvider
> - LdapProvider, a class used to create the LdapEncoder and LdapDecoder
>
> Those two classes are created when we create the FilterChain as soon as a
> new session is initialized. More specifically, the ProtocolCodecFilter is
> added into the chain, and before we can process any incoming messages from
> the client, we initialize the codec, using the LdapProtocolCodecFactory
> class to create the encoder/decoder instances.
>
> What are the essential elements ?
> - the factory
> - the encoder
> - the decoder
>
> We should be able to get rid of the provider, assuming we won't even switch
> the codec in the near or distant future. That would simplify the code which
> is, to say the least, cryptic...
>
> so, wdyt ? Should we keep the machinery that allow us to change the codec by
> using an environment variable ?
IMHO let us keep it, AFAIU this is transparent to the end or API user
right? if yes, then I think it is
better to remove it in another release.

P.S:- perhaps this will save us some debug headaches, if at all arise,
as we are making great
         level of refactoring for the upcoming 2.0

 thanks Emmanuel

Kiran Ayyagari