Some imrpovements in Filter are to be considered...

[Emmanuel Lecharny <el...@gmail.com>]

  Hi guys,

from ACI to Subtree, I'm now investigating the filter handling. As we 
allow the usage of Filters in Subtree this is a mandatoty step.

The problem was that filters weren't schema aware, thus we had to do a 
lot of manipulation to get the attributeType from the attribute, in many 
places in the server. Having a Schema aware Filter implementation would 
help a lot to avoid all those lookups.

So I modified the Filter implementation to be Schema aware.

The first attemp was a failure, as Filters are used in the server but 
also on the client side, and it may perfectly well not be schema aware. 
So I had to keep a dual form of Filters :
- Schema aware
- not schema aware

So far, so good.

The next stone in my shoe was that an old bug jumped into my face : 
DIRSERVER_971. Namely, it's about the handling of undefined Filters. For 
instance, such a filter :
(|(test=*)(objectClass=person))
will return no entries, when it should return every entry having a 
'person' ObjectClass. This is because we don't correctly handle 
Undefined evaluation of Filters (as required by RFC 4511, a filter can 
evaluate to either True, False, or Undefined).

This is bad, especially because now that the Filter is schema aware, 
this leads to some failure in tests, as unknown attributes aren't 
accepted while we parse the filter. Instead of creating an Undefined 
filter, we simply throw and exception.

Before fixing this issue, I had to fix many other inconsistencies I 
introducing in the code. Some of them are pretty nasty.

While fixing them, I discovered that when we go down to the backend, we 
don't use the Filter AttributeType to look for index, but we use the 
Attribute name. The silly thing is that inside the backend, we do a 
lookup to get back the AttributeType from its name. This is bad and has 
to be fixed too...

At the end of the day, here is what remains to be done :
- finish to fix the failing tests (almost done)
- deal with the Undefined Filter nodes
- use the AttributeType in the backend, instead of the Attribute name

Then we should be done with the filters, and it will be time to move 
back to subtree...

-- 
Regards,
Cordialement,
Emmanuel Lécharny
www.iktek.com

Re: Some improvements in Filter are to be considered...

[Emmanuel Lecharny <el...@gmail.com>]

  Some feedback :

On 7/13/10 12:57 AM, Emmanuel Lecharny wrote:
>  Hi guys,
>
> At the end of the day, here is what remains to be done :
> - finish to fix the failing tests (almost done)

Done.

> - deal with the Undefined Filter nodes
On my immediate roadmap. This will fix 2 issues ( DIRSERVER_971 & 
DIRSERVER_1525) and also unlock 2 others : DIRSERVER_955 & DIRSERVER_1061.

> - use the AttributeType in the backend, instead of the Attribute name

Done.


-- 
Regards,
Cordialement,
Emmanuel Lécharny
www.iktek.com

Re: Some imrpovements in Filter are to be considered...

[Kiran Ayyagari <ka...@apache.org>]

On Tue, Jul 13, 2010 at 4:27 AM, Emmanuel Lecharny <el...@gmail.com> wrote:
>  Hi guys,
>
> from ACI to Subtree, I'm now investigating the filter handling. As we allow
> the usage of Filters in Subtree this is a mandatoty step.
>
> The problem was that filters weren't schema aware, thus we had to do a lot
> of manipulation to get the attributeType from the attribute, in many places
> in the server. Having a Schema aware Filter implementation would help a lot
> to avoid all those lookups.
>
> So I modified the Filter implementation to be Schema aware.
>
> The first attemp was a failure, as Filters are used in the server but also
> on the client side, and it may perfectly well not be schema aware. So I had
> to keep a dual form of Filters :
> - Schema aware
> - not schema aware
>
> So far, so good.
>
> The next stone in my shoe was that an old bug jumped into my face :
> DIRSERVER_971. Namely, it's about the handling of undefined Filters. For
> instance, such a filter :
> (|(test=*)(objectClass=person))
> will return no entries, when it should return every entry having a 'person'
> ObjectClass. This is because we don't correctly handle Undefined evaluation
> of Filters (as required by RFC 4511, a filter can evaluate to either True,
> False, or Undefined).
>
> This is bad, especially because now that the Filter is schema aware, this
> leads to some failure in tests, as unknown attributes aren't accepted while
> we parse the filter. Instead of creating an Undefined filter, we simply
> throw and exception.
>
> Before fixing this issue, I had to fix many other inconsistencies I
> introducing in the code. Some of them are pretty nasty.
>
> While fixing them, I discovered that when we go down to the backend, we
> don't use the Filter AttributeType to look for index, but we use the
> Attribute name. The silly thing is that inside the backend, we do a lookup
> to get back the AttributeType from its name. This is bad and has to be fixed
> too...
>
> At the end of the day, here is what remains to be done :
> - finish to fix the failing tests (almost done)
> - deal with the Undefined Filter nodes
> - use the AttributeType in the backend, instead of the Attribute name
>
> Then we should be done with the filters, and it will be time to move back to
> subtree...

a lot of stuff.., thanks for the heads up Emmanuel

Kiran Ayyagari