Kerberos codec and other codec, and a bit of 'state of the nation'

[Emmanuel Lecharny <el...@gmail.com>]

Hi guys,

first, I'm please to inform you all that the Kerberos protocol codec has 
been completed today. It was a painful job, with more than 30 000 slocs 
generated. Now, it's time to include this work into the kerberos server.

Atm I'm reviewing what we have done, and try to clean up the thing, as 
the code we wrote at the end is different from what we started with, as 
we improved the process a lot. There are a few points I think worth 
discussing.

- The loggers. Currently, we use a per class logger. I think it's 
inaccurate, as we won't be able to group all the logs for a specific 
protocol in one single logger. Of course, if we want to disable the 
codec logs, we can always do that by filtering on the package, but I 
think we could also decide to generate all the logs into one single 
logger, called "CODEC". wdyt ?

- The tests. Right now, testing that the decoder is correct is a burden. 
We discussed a lot about it, and we didn't found a convenient case to 
test all the possible wrong PDU we should reject. We need a tool for 
that, but I doubt a tool can be written in less than a few weeks. Thoughts ?

Otherwise, we will now have to include the codecs into the Kerberos 
server, and hopefully solve the problem we had with Kerberos on top of 
TCP. That could take a day.

We will then have to check the Kerberos server itself. I'm afraid that 
we might be surprised by the way it works, not in a positive way, sadly. 
We need some client tests, so we need to setup a test env that allows us 
to test the Kerberos server using some kerberos client we may write. In 
order to check that those kerberos client code is OK, we need to compare 
it with what Java is providing. I guess that it will take a bit of time, 
but I really see the Kerberos server as a major asset.

On the LDAP server side, we still have to work on the Administrative 
Point management. I will do that on december, and I think it can be 
available by mid-december.

We are close. Really close. The configuration management is doing OK, we 
soon will have a decent UI to administer the server. Pierre-Arnaud is 
also thinking about the best way to extend the configuration, when an 
implementer want to add his own configuration for some of the extension 
points. He should come with somethng working by mid-december too.

That's pretty much it for today, I guess that it would be valuable if 
everyone of you working on the server give some feedback about what's 
going on, as we are slowly converging to a viable 2.0 !

Thanks for listening !

-- 
Regards,
Cordialement,
Emmanuel Lécharny
www.iktek.com

Re: Kerberos codec and other codec, and a bit of 'state of the nation'

[Emmanuel Lécharny <el...@apache.org>]

On 11/27/10 6:49 PM, Alex Karasulu wrote:
> On Sat, Nov 27, 2010 at 4:15 PM, Emmanuel Lécharny<el...@apache.org>wrote:
>
>> On 11/27/10 2:32 PM, Kiran Ayyagari wrote:
>>
>>> Atm I'm reviewing what we have done, and try to clean up the thing, as the
>>>
>>>> code we wrote at the end is different from what we started with, as we
>>>> improved the process a lot. There are a few points I think worth
>>>> discussing.
>>>>
>>>> - The loggers. Currently, we use a per class logger. I think it's
>>>> inaccurate, as we won't be able to group all the logs for a specific
>>>> protocol in one single logger. Of course, if we want to disable the codec
>>>> logs, we can always do that by filtering on the package, but I think we
>>>> could also decide to generate all the logs into one single logger, called
>>>> "CODEC". wdyt ?
>>>>
>>>>   IMO this should be the way to go, how about KRB-CODEC
>> That won't work, as the classes are shared with the LDAP codec and all the
>> controls codec...
>>
>>
>>
> You know what might help is a mapped diagnostic context (MDC). You can use
> context information in these codec classes to report what it's being used
> for and filter logs accordingly.
That's something we can do when we enter into the decoder, adding either 
"LDAP" or "KERBEROS" or any needed name, as soon as we have a way to 
specialize the logs and get everything associated with Kerberos decoding 
in the same set of logs.

If MDC solve the problem, then let's go for MDC. I have currently no 
idea how it really works. If someone want to experiment, I'll be please 
to use something better than a purely static name...


-- 
Regards,
Cordialement,
Emmanuel Lécharny
www.iktek.com

Re: Kerberos codec and other codec, and a bit of 'state of the nation'

[Alex Karasulu <ak...@apache.org>]

On Sat, Nov 27, 2010 at 4:15 PM, Emmanuel Lécharny <el...@apache.org>wrote:

> On 11/27/10 2:32 PM, Kiran Ayyagari wrote:
>
>> Atm I'm reviewing what we have done, and try to clean up the thing, as the
>>
>>> code we wrote at the end is different from what we started with, as we
>>> improved the process a lot. There are a few points I think worth
>>> discussing.
>>>
>>> - The loggers. Currently, we use a per class logger. I think it's
>>> inaccurate, as we won't be able to group all the logs for a specific
>>> protocol in one single logger. Of course, if we want to disable the codec
>>> logs, we can always do that by filtering on the package, but I think we
>>> could also decide to generate all the logs into one single logger, called
>>> "CODEC". wdyt ?
>>>
>>>  IMO this should be the way to go, how about KRB-CODEC
>>
>
> That won't work, as the classes are shared with the LDAP codec and all the
> controls codec...
>
>
>
You know what might help is a mapped diagnostic context (MDC). You can use
context information in these codec classes to report what it's being used
for and filter logs accordingly.

We have support for this in MINA with the MDC filter [0]. We also have
support in Log4J for it [1].

Regards,
--Alex

----
[0] - http://alturl.com/tepw7
[1] - http://alturl.com/vq4ux

Re: Kerberos codec and other codec, and a bit of 'state of the nation'

[Emmanuel Lécharny <el...@apache.org>]

On 11/27/10 2:32 PM, Kiran Ayyagari wrote:
> Atm I'm reviewing what we have done, and try to clean up the thing, as the
>> code we wrote at the end is different from what we started with, as we
>> improved the process a lot. There are a few points I think worth discussing.
>>
>> - The loggers. Currently, we use a per class logger. I think it's
>> inaccurate, as we won't be able to group all the logs for a specific
>> protocol in one single logger. Of course, if we want to disable the codec
>> logs, we can always do that by filtering on the package, but I think we
>> could also decide to generate all the logs into one single logger, called
>> "CODEC". wdyt ?
>>
> IMO this should be the way to go, how about KRB-CODEC

That won't work, as the classes are shared with the LDAP codec and all 
the controls codec...


>> - The tests. Right now, testing that the decoder is correct is a burden. We
>> discussed a lot about it, and we didn't found a convenient case to test all
>> the possible wrong PDU we should reject. We need a tool for that, but I
>> doubt a tool can be written in less than a few weeks. Thoughts ?
>>
> rather than that am thinking that let us invest that time on writing a
> krb client and we can test it against ours and MIT krb server and
> compare the results and apply fixes if needed
That's something we have to do anyway, but it won't help us against hand 
crafted PDU. One thing I'm thinking about is the PDU where teh Length 
part is huge, but with no data coming. In fact, the main problem will be 
to protect the server against a DDOS, so we must limit the PDU size (and 
Kerberos PDU are smaller than LDAP PDU, so we must have a different 
setting), and also limit the time we keep an undecoded PDU (someone 
might draft a big PDU, but send it byte by byte, with a delay between 
each byte). A solution would be to keep a timestamp when we receive the 
first TLV, and if the end of the PDU is not received in a reasonable 
delay, then we discard the full PDU.


-- 
Regards,
Cordialement,
Emmanuel Lécharny
www.iktek.com

Re: Kerberos codec and other codec, and a bit of 'state of the nation'

[Kiran Ayyagari <ka...@apache.org>]

On Sat, Nov 27, 2010 at 3:26 PM, Emmanuel Lecharny <el...@gmail.com> wrote:
> Hi guys,
>
> first, I'm please to inform you all that the Kerberos protocol codec has
> been completed today. It was a painful job, with more than 30 000 slocs
> generated. Now, it's time to include this work into the kerberos server.
>
many thanks for the great effort
> Atm I'm reviewing what we have done, and try to clean up the thing, as the
> code we wrote at the end is different from what we started with, as we
> improved the process a lot. There are a few points I think worth discussing.
>
> - The loggers. Currently, we use a per class logger. I think it's
> inaccurate, as we won't be able to group all the logs for a specific
> protocol in one single logger. Of course, if we want to disable the codec
> logs, we can always do that by filtering on the package, but I think we
> could also decide to generate all the logs into one single logger, called
> "CODEC". wdyt ?
>
IMO this should be the way to go, how about KRB-CODEC
> - The tests. Right now, testing that the decoder is correct is a burden. We
> discussed a lot about it, and we didn't found a convenient case to test all
> the possible wrong PDU we should reject. We need a tool for that, but I
> doubt a tool can be written in less than a few weeks. Thoughts ?
>
rather than that am thinking that let us invest that time on writing a
krb client and we can test it against ours and MIT krb server and
compare the results and apply fixes if needed
> Otherwise, we will now have to include the codecs into the Kerberos server,
> and hopefully solve the problem we had with Kerberos on top of TCP. That
> could take a day.
>
> We will then have to check the Kerberos server itself. I'm afraid that we
> might be surprised by the way it works, not in a positive way, sadly. We
> need some client tests, so we need to setup a test env that allows us to
> test the Kerberos server using some kerberos client we may write. In order
> to check that those kerberos client code is OK, we need to compare it with
> what Java is providing. I guess that it will take a bit of time, but I
> really see the Kerberos server as a major asset.
>
> On the LDAP server side, we still have to work on the Administrative Point
> management. I will do that on december, and I think it can be available by
> mid-december.
>
> We are close. Really close. The configuration management is doing OK, we
> soon will have a decent UI to administer the server. Pierre-Arnaud is also
> thinking about the best way to extend the configuration, when an implementer
> want to add his own configuration for some of the extension points. He
> should come with somethng working by mid-december too.
>
> That's pretty much it for today, I guess that it would be valuable if
> everyone of you working on the server give some feedback about what's going
> on, as we are slowly converging to a viable 2.0 !
>
> Thanks for listening !
>
> --
> Regards,
> Cordialement,
> Emmanuel Lécharny
> www.iktek.com
>
>

thanks Emmanuel

-- 
Kiran Ayyagari

Re: Kerberos codec and other codec, and a bit of 'state of the nation'

[Alex Karasulu <ak...@apache.org>]

On Sat, Nov 27, 2010 at 3:26 PM, Emmanuel Lecharny <el...@gmail.com>wrote:

> Hi guys,
>
> first, I'm please to inform you all that the Kerberos protocol codec has
> been completed today. It was a painful job, with more than 30 000 slocs
> generated. Now, it's time to include this work into the kerberos server.
>
>
Great job guys. This was an incredible effort.



> Atm I'm reviewing what we have done, and try to clean up the thing, as the
> code we wrote at the end is different from what we started with, as we
> improved the process a lot. There are a few points I think worth discussing.
>
> - The loggers. Currently, we use a per class logger. I think it's
> inaccurate, as we won't be able to group all the logs for a specific
> protocol in one single logger. Of course, if we want to disable the codec
> logs, we can always do that by filtering on the package, but I think we
> could also decide to generate all the logs into one single logger, called
> "CODEC". wdyt ?
>
>
I think the current per class way works and filtering on the package name is
the way to go via the log4j properties file.


> - The tests. Right now, testing that the decoder is correct is a burden. We
> discussed a lot about it, and we didn't found a convenient case to test all
> the possible wrong PDU we should reject. We need a tool for that, but I
> doubt a tool can be written in less than a few weeks. Thoughts ?
>
>
There's a way but it's long and exhaustive. If we can find a new contributor
willing to take this on maybe as a background thread I'd be glad to guide
them on it.


> Otherwise, we will now have to include the codecs into the Kerberos server,
> and hopefully solve the problem we had with Kerberos on top of TCP. That
> could take a day.
>
> We will then have to check the Kerberos server itself. I'm afraid that we
> might be surprised by the way it works, not in a positive way, sadly.


Yes I agree - but finally we're going to have this beast tamed.


> We need some client tests, so we need to setup a test env that allows us to
> test the Kerberos server using some kerberos client we may write. In order
> to check that those kerberos client code is OK, we need to compare it with
> what Java is providing. I guess that it will take a bit of time, but I
> really see the Kerberos server as a major asset.
>
>
+1


> On the LDAP server side, we still have to work on the Administrative Point
> management. I will do that on december, and I think it can be available by
> mid-december.
>
>
I'll lend a hand on this. The AP issues are really important to get right
once and for all.

We are close. Really close. The configuration management is doing OK, we
> soon will have a decent UI to administer the server. Pierre-Arnaud is also
> thinking about the best way to extend the configuration, when an implementer
> want to add his own configuration for some of the extension points. He
> should come with somethng working by mid-december too.
>
>
Excellent.

-- 
Alex Karasulu
My Blog :: http://www.jroller.com/akarasulu/
Apache Directory Server :: http://directory.apache.org
Apache MINA :: http://mina.apache.org
To set up a meeting with me: http://tungle.me/AlexKarasulu