JNDI vs protocol vs Core API

[Emmanuel Lecharny <el...@gmail.com>]

Hi guys,

as i'm struggling with referrals for days now, I just discovered that 
there are many places in the server which are not handling them 
correctly. Atm, the only part which is sane is the protocol layer. For 
embedded servers, not passing through the protocol layer, I have to say 
that the referrals are simply ignored.

Here are the problem we have.

A bit of history. Before we removed JNDI from the server, here was the 
chain followed by an incoming request :

1) through the network
client --> JNDI--> {network}--> ASN.1codec --> Protocol handler --> JNDI 
--> NexusProxy --> Interceptors --> ...

2) embedded
client --> JNDI--> NexusProxy --> Interceptors --> ...

Simple, clear, but messy inside the server.

We decided to remove JNDI from the server, to offer our own API 
(CoreSession) on top of which we were supposed to rewrite the JNDI 
layer. here is what are suposed to have now :
1) through the network
client --> JNDI--> {network}--> ASN.1codec --> Protocol handler --> 
CoreSession --> OperationManager --> Interceptors ...

2) embedded with JNDI
client --> JNDI--> CoreSession --> OperationManager --> Interceptors ...

3) embedded without JNDI
client --> CoreSession --> OperationManager --> Interceptors ...

As everyone can see, the JNDI provider is just offered as a thin layer 
to the users who don't want to deal with ADS inside out. This is perfect 
for those who want to migrate from version X to version Y. But they can 
also use the better API whoch does not use the cursed semantic of JNDI.

So far, so good.

Now, the problem is that for referrals, as we have also removed the 
ReferralInterceptor from the core server (a very good move, when you 
take the time to think about it - and it took me two hours of discussion 
with Alex to get it -), you have a new problem : where do we have to 
handle referrals ?

The first idea was to move it to the protocol layer. So be it. But the 
problem is that the JNDI referral handling was not anymore done (and as 
a consequence, the associated tests were disabled, until the JNDI 
provider was fixed to handle those referrals).

I'm currently re-implementing the Referral handling in this JNDI layer, 
but I suddenly discovered that this was not a good idea *at all*. And so 
was the initial idea to implement the referrals handling into the 
protocol layer. We now have 3 places where we must implement this process :
- in the protocol layer
- in the JNDI provider
- and in the CoreSession layer.

Bad...

So here is what I suggest : we should handle referrals into the 
CoreSession layer, and deal with the thrown exceptions 
(LdapReferralExceptio, LdapPartialResultException, etc) into the JNDI 
and protocol layer, otherwise, we will do the job three times (and it's 
not a funny job ...)

thoughts ?

PS: don't be too picky with the names I'm using (like CoreSession is not 
really the API we have built), but I'm sure that Alex can correct me. I 
think however that the general picture is ok.


Thanks !

-- 
--
cordialement, regards,
Emmanuel Lécharny
www.iktek.com
directory.apache.org

Re: JNDI vs protocol vs Core API

[Emmanuel Lecharny <el...@gmail.com>]

Alex Karasulu wrote:
> OK overall you have me convinced.  I was wrong.  Let's go ahead and affect
> the changes to CoreSession.
>   
Ok. I will come back with some proposal, but righjt now, I'm finishing 
to describe all the scenarii on the referral change page.

Thanks buddy !

-- 
--
cordialement, regards,
Emmanuel Lécharny
www.iktek.com
directory.apache.org

Re: JNDI vs protocol vs Core API

[Alex Karasulu <ak...@apache.org>]

OK overall you have me convinced.  I was wrong.  Let's go ahead and affect
the changes to CoreSession.

Alex

On Sat, Sep 27, 2008 at 6:12 AM, Emmanuel Lecharny <el...@gmail.com>wrote:

> Alex Karasulu wrote:
>
>> well, I don't think we should see the referral handling this way. IIf you
>>> see it from another perspective, ie, from the user POV, a referral is
>>> just
>>> an indirection. Now, the way the server manage it is pretty much a black
>>> box
>>> for the client, as soon as the client is informed that the entry is not
>>> present in the server. If you consider that you want to embed the server,
>>> but without using JNDI, you are in a dead end.
>>>
>>>
>>
>>
>> I don't think we're in a dead end.  It's up to the user of the CoreSession
>> and Entry interfaces to determine what to do with the referral if they
>> don't
>> want to use the JNDI wrapper.  It's that simple.
>>
>> What more do you think needs to be done?  For example if you do a lookup
>> via:
>>
>>    session.lookup( "ou=People,dc=foo" )
>>
>> and dc=foo is a referral to dc=bar but ou=People does not exist.  In this
>> case what exception do you throw? Something telling you the entry does not
>> exist or an exception stating a referral needs to be handled? If you want
>> referral handling to occur here in CoreSession methods then, the exception
>> thrown will depend on the presence of the ManageDsaIT control's presence.
>>
>>
> A NoSuchObjectException, from the top of my head, which is not good enough
> for the client. I think that the core session should throw a
> PartialResultException if one of the ancestor is a referral, or a
> ReferralException if the entry is a referral and there is no ManageDsaIT
> control.
>
> In fact, the little thin layer (or may be some small modification in
> CoreSession) should just do that : check if the ManageDsaIt control is
> present (it may be a simple boolean passed into the OpContext, no need to
> pass the full control), and throw the correct exception or the entry,
> depending on the cases.
>
> The JNDI and protocol layers will then just have to deal with those
> exceptions accordingly to their own semantic.
>
> One last thing is that we may have to implement the 'follow' mechanism into
> the JNDI layer, as it's very specific to JNDI.
>
>> If you don't implement referral handling here then upper levels must check
>> with the ReferralManager to see if the dn of the missing entry has a
>> referral ancestor.  I guess this is a good argument to implement some
>> minimal referral handling in the CoreSession then.
>>
>>
> Yep.
>
>> What does the Netscape API do?  Maybe that may shed some light.
>>
>>
> I can check that.
>
>

Re: JNDI vs protocol vs Core API

[Emmanuel Lecharny <el...@gmail.com>]

Alex Karasulu wrote:
>> well, I don't think we should see the referral handling this way. IIf you
>> see it from another perspective, ie, from the user POV, a referral is just
>> an indirection. Now, the way the server manage it is pretty much a black box
>> for the client, as soon as the client is informed that the entry is not
>> present in the server. If you consider that you want to embed the server,
>> but without using JNDI, you are in a dead end.
>>     
>
>
> I don't think we're in a dead end.  It's up to the user of the CoreSession
> and Entry interfaces to determine what to do with the referral if they don't
> want to use the JNDI wrapper.  It's that simple.
>
> What more do you think needs to be done?  For example if you do a lookup
> via:
>
>     session.lookup( "ou=People,dc=foo" )
>
> and dc=foo is a referral to dc=bar but ou=People does not exist.  In this
> case what exception do you throw? Something telling you the entry does not
> exist or an exception stating a referral needs to be handled? If you want
> referral handling to occur here in CoreSession methods then, the exception
> thrown will depend on the presence of the ManageDsaIT control's presence.
>   
A NoSuchObjectException, from the top of my head, which is not good 
enough for the client. I think that the core session should throw a 
PartialResultException if one of the ancestor is a referral, or a 
ReferralException if the entry is a referral and there is no ManageDsaIT 
control.

In fact, the little thin layer (or may be some small modification in 
CoreSession) should just do that : check if the ManageDsaIt control is 
present (it may be a simple boolean passed into the OpContext, no need 
to pass the full control), and throw the correct exception or the entry, 
depending on the cases.

The JNDI and protocol layers will then just have to deal with those 
exceptions accordingly to their own semantic.

One last thing is that we may have to implement the 'follow' mechanism 
into the JNDI layer, as it's very specific to JNDI.
> If you don't implement referral handling here then upper levels must check
> with the ReferralManager to see if the dn of the missing entry has a
> referral ancestor.  I guess this is a good argument to implement some
> minimal referral handling in the CoreSession then.
>   
Yep.
> What does the Netscape API do?  Maybe that may shed some light.
>   
I can check that.

Re: JNDI vs protocol vs Core API

[Alex Karasulu <ak...@apache.org>]

On Fri, Sep 26, 2008 at 8:36 PM, Emmanuel Lecharny <el...@gmail.com>wrote:

> Alex Karasulu wrote:
>
>> Emmanuel,
>>
>> On Fri, Sep 26, 2008 at 2:01 PM, Emmanuel Lecharny <elecharny@gmail.com
>> >wrote:
>>
>> Now, the problem is that for referrals, as we have also removed the
>>> ReferralInterceptor from the core server (a very good move, when you take
>>> the time to think about it - and it took me two hours of discussion with
>>> Alex to get it -), you have a new problem : where do we have to handle
>>> referrals ?
>>>
>>>
>>>
>>
>> Question is for what purpose do you want to handle referrals.  There are
>> two
>> ways to handle referrals:
>>
>> (1) Through the protocol
>> (2) Via the JNDI Provided based on how JNDI is expected to behave with
>> different values for the Context.REFERRAL property in the environment:
>> throw, ignore, follow.
>>
>>
> well, I don't think we should see the referral handling this way. IIf you
> see it from another perspective, ie, from the user POV, a referral is just
> an indirection. Now, the way the server manage it is pretty much a black box
> for the client, as soon as the client is informed that the entry is not
> present in the server. If you consider that you want to embed the server,
> but without using JNDI, you are in a dead end.


I don't think we're in a dead end.  It's up to the user of the CoreSession
and Entry interfaces to determine what to do with the referral if they don't
want to use the JNDI wrapper.  It's that simple.

What more do you think needs to be done?  For example if you do a lookup
via:

    session.lookup( "ou=People,dc=foo" )

and dc=foo is a referral to dc=bar but ou=People does not exist.  In this
case what exception do you throw? Something telling you the entry does not
exist or an exception stating a referral needs to be handled? If you want
referral handling to occur here in CoreSession methods then, the exception
thrown will depend on the presence of the ManageDsaIT control's presence.

If you don't implement referral handling here then upper levels must check
with the ReferralManager to see if the dn of the missing entry has a
referral ancestor.  I guess this is a good argument to implement some
minimal referral handling in the CoreSession then.

What does the Netscape API do?  Maybe that may shed some light.


>
>
>>
>>
>>> The first idea was to move it to the protocol layer. So be it. But the
>>> problem is that the JNDI referral handling was not anymore done (and as a
>>> consequence, the associated tests were disabled, until the JNDI provider
>>> was
>>> fixed to handle those referrals).
>>>
>>> I'm currently re-implementing the Referral handling in this JNDI layer,
>>> but
>>> I suddenly discovered that this was not a good idea *at all*. And so was
>>> the
>>> initial idea to implement the referrals handling into the protocol layer.
>>> We
>>> now have 3 places where we must implement this process :
>>> - in the protocol layer
>>> - in the JNDI provider
>>> - and in the CoreSession layer.
>>>
>>> Bad...
>>>
>>>
>>>
>>
>> I don't think it should be done in the CoreSession.  The CoreSession
>> should
>> just provide an interface to perform operations against entries.  I think
>> all entries should be treated the same with respect to this interface.
>
>
Starting to think otherwise now.


>
>>
>>
> That's fine, but then we have to add a thin layer on top of the CoreSession
> (be it the Operation manager or anything else, it's just something which
> will handle the referrals, when the plain entry is returned back, if it
> contains the Referral ObjectClass)). Otherwie, we will have to handle it in
> two places instead of one (ie, in the protocol and in JNDI).
>
>> If code needs to do something different because the entry is a referral
>> it's
>> up to that code to do so.
>>
>>
> so the need of another layer on top of the CoreSession
>
>> Referral handling is different depending on whether the protocol is
>> handling
>> it or whether the JNDI provider is handling it.
>>
> In fact, if you abstract yoruself from the e=semantic, there is not that
> much a difference : either the referral is ignored, and we return the plain
> entry (=> ManageDsaIt is present _or_ the property is set to Ignore), or we
> want to be redirected, and we want the underlying server to give us the URL
> to go to (ManageDsaIt is not present _or_ the property is set to 'throw').
> The last case (the property is set to follow) is pretty much the same,
> except that the server will automatically send a request to the other server
> (and we don't handle this case anyway)
>
>> The reason why I separated
>> these is because the concepts were getting mixed together and JNDI was
>> contaminating the server.
>>
>>
> Removing those aspects from the server was a perfect move. But I think we
> can do better than adding handlers for referrals in both JNDI and protocol
> code. This is the reason why I posted this mail.
>
>>
>>
>>
>>> So here is what I suggest : we should handle referrals into the
>>> CoreSession
>>> layer, and deal with the thrown exceptions (LdapReferralExceptio,
>>> LdapPartialResultException, etc) into the JNDI and protocol layer,
>>> otherwise, we will do the job three times (and it's not a funny job ...)
>>>
>>> thoughts ?
>>>
>>>
>>>
>>
>> We could do this but these exceptions are JNDI specific.  They've been
>> defined to deal with a specific JNDI referral handling mode (throw
>> specifically).
>>
> IMO, 'follow' is the JNDI exception, not throw. And we don't handle the
> 'follow' anyway :)
>
>> Then the protocol handler code would need to change to
>> understand these JNDI specifics.
>>
>> Perhaps we should just not deal with referrals at all in the CoreSession.
>> Just add it to the JNDI provider for it's specific modes.
>>
>>
> That's also an option, but I'm afraid that may limit the embedded version
> of the server.
>
>> My mindset my be wrong (and at times I do doubt it) but I feel the core
>> API
>> is purely a local access API to initialize and manipulate the DIT and its
>> entries.
>>
> I agree fully. The core should only return pure entries, not dealing with
> referrals as if they were special. But the layer on top of it must deal with
> referrals. The JNDI provider must do it, and my idea is to delegate this
> part to an intermediate layer, like :
>
> client --> JNDI --> server API --> referral layer --> core --> ...
> or
> client --> server API --> referral layer --> core --> ...
>
>   It does not deal with following referrals since this would go over
>> the network.  It does not deal with referrals as special entries since
>> this
>> is the responsibility of higher layers which deal with referrals in
>> different ways.
>>
>>
>>
>>
>>> PS: don't be too picky with the names I'm using (like CoreSession is not
>>> really the API we have built), but I'm sure that Alex can correct me. I
>>> think however that the general picture is ok.
>>>
>>>
>>>
>>>
>> Yeah just great - I understood easily what you meant.  I think reuse is
>> good
>> and I can certainly see how putting some referral handling functionality
>> into the CoreSession is very tempting.  As coders we always want to reuse
>> and that's good.  However sometimes reuse causes more problems than it is
>> worth.
>>
> hence my email ;)
>
>>  I think this is a good example for that kind of situation.  Also
>> sometimes we want to add additional functionality because it seems to make
>> sense to have a richer interface that handles more cases.  However more
>> complex interfaces are harder to work with as bricks to build other
>> interfaces when their rich features start to get in the way.
>>
>>
> I think that the core session interface should not be modified. But we may
> need another thin layer on top of it. Maybe.
>
>> <snip/>Then the referral handling code,
>> whether in the protocol provider or in the JNDI provider, has a much
>> easier
>> task.  The respective code is clean and shows clearly the policy of how
>> referrals are handled in their respective layers.
>>
>>
> this is the key : do we have to handle referrals in those two layers
> (protocol an JNDI) when we could manage it in an intermediate layer ? I
> think this is possoble. More than that, I think this can also be the seed
> for a new API which will replace the current JNDI API for those who would
> like to embed the server without being stuck into JNDI or to go through the
> protocol layer.
>

Ok another layer will get confusing so you're probably right that we should
put it into CoreSession.

Alex

Re: JNDI vs protocol vs Core API

[Emmanuel Lecharny <el...@gmail.com>]

Hi Alex,

I will add comments inline.

Alex Karasulu wrote:
> Emmanuel,
>
> On Fri, Sep 26, 2008 at 2:01 PM, Emmanuel Lecharny <el...@gmail.com>wrote:
>
>   
>> Hi guys,
>>
>> as i'm struggling with referrals for days now, I just discovered that there
>> are many places in the server which are not handling them correctly.
>>     
>
>
> I had removed referral handling from everywhere and placed it only at the
> protocol provider layer where it does work properly.
>   
yep, in the protocol layer, it works.
>
>   
>> Atm, the only part which is sane is the protocol layer. For embedded
>> servers, not passing through the protocol layer, I have to say that the
>> referrals are simply ignored.
>>
>>     
>
> This statement is not completely accurate.  When you embed ApacheDS as a
> server with the protocol layer then the protocol layer will handle
> referrals.  
but this is not the way people will embed the server. There is no 
advantage to pass through the protocol layer when embedding the server, IMO.
> You probably mean to say the DirectoryService and the JNDI
> Provider around the DirectoryService does not support referrals.
>   
exactly.
> Forgive me if I sound like I'm trying to correct you but just trying to make
> sure we use the right terminology to note exactly what we're talking about.
>   
Np at all. It's still a bit fuzzy so any clarification is perfect. i'm 
just trying to get as much information as possible in order to avoid to 
FU the server.
> <snip/>
>
>
>   
>> Now, the problem is that for referrals, as we have also removed the
>> ReferralInterceptor from the core server (a very good move, when you take
>> the time to think about it - and it took me two hours of discussion with
>> Alex to get it -), you have a new problem : where do we have to handle
>> referrals ?
>>
>>     
>
> Question is for what purpose do you want to handle referrals.  There are two
> ways to handle referrals:
>
> (1) Through the protocol
> (2) Via the JNDI Provided based on how JNDI is expected to behave with
> different values for the Context.REFERRAL property in the environment:
> throw, ignore, follow.
>   
well, I don't think we should see the referral handling this way. IIf 
you see it from another perspective, ie, from the user POV, a referral 
is just an indirection. Now, the way the server manage it is pretty much 
a black box for the client, as soon as the client is informed that the 
entry is not present in the server. If you consider that you want to 
embed the server, but without using JNDI, you are in a dead end. (and in 
fact, this is what we would like to do when implementing the protocol layer)
>
>   
>> The first idea was to move it to the protocol layer. So be it. But the
>> problem is that the JNDI referral handling was not anymore done (and as a
>> consequence, the associated tests were disabled, until the JNDI provider was
>> fixed to handle those referrals).
>>
>> I'm currently re-implementing the Referral handling in this JNDI layer, but
>> I suddenly discovered that this was not a good idea *at all*. And so was the
>> initial idea to implement the referrals handling into the protocol layer. We
>> now have 3 places where we must implement this process :
>> - in the protocol layer
>> - in the JNDI provider
>> - and in the CoreSession layer.
>>
>> Bad...
>>
>>     
>
> I don't think it should be done in the CoreSession.  The CoreSession should
> just provide an interface to perform operations against entries.  I think
> all entries should be treated the same with respect to this interface.
>   
That's fine, but then we have to add a thin layer on top of the 
CoreSession (be it the Operation manager or anything else, it's just 
something which will handle the referrals, when the plain entry is 
returned back, if it contains the Referral ObjectClass)). Otherwie, we 
will have to handle it in two places instead of one (ie, in the protocol 
and in JNDI).
> If code needs to do something different because the entry is a referral it's
> up to that code to do so.
>   
so the need of another layer on top of the CoreSession
> Referral handling is different depending on whether the protocol is handling
> it or whether the JNDI provider is handling it.  
In fact, if you abstract yoruself from the e=semantic, there is not that 
much a difference : either the referral is ignored, and we return the 
plain entry (=> ManageDsaIt is present _or_ the property is set to 
Ignore), or we want to be redirected, and we want the underlying server 
to give us the URL to go to (ManageDsaIt is not present _or_ the 
property is set to 'throw'). The last case (the property is set to 
follow) is pretty much the same, except that the server will 
automatically send a request to the other server (and we don't handle 
this case anyway)
> The reason why I separated
> these is because the concepts were getting mixed together and JNDI was
> contaminating the server.
>   
Removing those aspects from the server was a perfect move. But I think 
we can do better than adding handlers for referrals in both JNDI and 
protocol code. This is the reason why I posted this mail.
>
>   
>> So here is what I suggest : we should handle referrals into the CoreSession
>> layer, and deal with the thrown exceptions (LdapReferralExceptio,
>> LdapPartialResultException, etc) into the JNDI and protocol layer,
>> otherwise, we will do the job three times (and it's not a funny job ...)
>>
>> thoughts ?
>>
>>     
>
> We could do this but these exceptions are JNDI specific.  They've been
> defined to deal with a specific JNDI referral handling mode (throw
> specifically).  
IMO, 'follow' is the JNDI exception, not throw. And we don't handle the 
'follow' anyway :)
> Then the protocol handler code would need to change to
> understand these JNDI specifics.
>
> Perhaps we should just not deal with referrals at all in the CoreSession.
> Just add it to the JNDI provider for it's specific modes.
>   
That's also an option, but I'm afraid that may limit the embedded 
version of the server.
> My mindset my be wrong (and at times I do doubt it) but I feel the core API
> is purely a local access API to initialize and manipulate the DIT and its
> entries. 
I agree fully. The core should only return pure entries, not dealing 
with referrals as if they were special. But the layer on top of it must 
deal with referrals. The JNDI provider must do it, and my idea is to 
delegate this part to an intermediate layer, like :

client --> JNDI --> server API --> referral layer --> core --> ...
or
client --> server API --> referral layer --> core --> ...

>  It does not deal with following referrals since this would go over
> the network.  It does not deal with referrals as special entries since this
> is the responsibility of higher layers which deal with referrals in
> different ways.
>
>
>   
>> PS: don't be too picky with the names I'm using (like CoreSession is not
>> really the API we have built), but I'm sure that Alex can correct me. I
>> think however that the general picture is ok.
>>
>>
>>     
> Yeah just great - I understood easily what you meant.  I think reuse is good
> and I can certainly see how putting some referral handling functionality
> into the CoreSession is very tempting.  As coders we always want to reuse
> and that's good.  However sometimes reuse causes more problems than it is
> worth. 
hence my email ;)
>  I think this is a good example for that kind of situation.  Also
> sometimes we want to add additional functionality because it seems to make
> sense to have a richer interface that handles more cases.  However more
> complex interfaces are harder to work with as bricks to build other
> interfaces when their rich features start to get in the way.
>   
I think that the core session interface should not be modified. But we 
may need another thin layer on top of it. Maybe.
> <snip/>Then the referral handling code,
> whether in the protocol provider or in the JNDI provider, has a much easier
> task.  The respective code is clean and shows clearly the policy of how
> referrals are handled in their respective layers.
>   
this is the key : do we have to handle referrals in those two layers 
(protocol an JNDI) when we could manage it in an intermediate layer ? I 
think this is possoble. More than that, I think this can also be the 
seed for a new API which will replace the current JNDI API for those who 
would like to embed the server without being stuck into JNDI or to go 
through the protocol layer.


--
cordialement, regards,
Emmanuel Lécharny
www.iktek.com
directory.apache.org

Re: JNDI vs protocol vs Core API

[Alex Karasulu <ak...@apache.org>]

Emmanuel,

On Fri, Sep 26, 2008 at 2:01 PM, Emmanuel Lecharny <el...@gmail.com>wrote:

> Hi guys,
>
> as i'm struggling with referrals for days now, I just discovered that there
> are many places in the server which are not handling them correctly.


I had removed referral handling from everywhere and placed it only at the
protocol provider layer where it does work properly.


> Atm, the only part which is sane is the protocol layer. For embedded
> servers, not passing through the protocol layer, I have to say that the
> referrals are simply ignored.
>

This statement is not completely accurate.  When you embed ApacheDS as a
server with the protocol layer then the protocol layer will handle
referrals.  You probably mean to say the DirectoryService and the JNDI
Provider around the DirectoryService does not support referrals.

Forgive me if I sound like I'm trying to correct you but just trying to make
sure we use the right terminology to note exactly what we're talking about.


> Here are the problem we have.
>
> A bit of history. Before we removed JNDI from the server, here was the
> chain followed by an incoming request :
>
> 1) through the network
> client --> JNDI--> {network}--> ASN.1codec --> Protocol handler --> JNDI
> --> NexusProxy --> Interceptors --> ...
>
> 2) embedded
> client --> JNDI--> NexusProxy --> Interceptors --> ...
>
> Simple, clear, but messy inside the server.
>
> We decided to remove JNDI from the server, to offer our own API
> (CoreSession) on top of which we were supposed to rewrite the JNDI layer.
> here is what are suposed to have now :
> 1) through the network
> client --> JNDI--> {network}--> ASN.1codec --> Protocol handler -->
> CoreSession --> OperationManager --> Interceptors ...
>
> 2) embedded with JNDI
> client --> JNDI--> CoreSession --> OperationManager --> Interceptors ...
>
> 3) embedded without JNDI
> client --> CoreSession --> OperationManager --> Interceptors ...
>
> As everyone can see, the JNDI provider is just offered as a thin layer to
> the users who don't want to deal with ADS inside out.


Right. Users who do not want to deal with the internal API's can just
initialize the DirectoryService and use it to create an InitialContext.
There after all operations appear to be remote through JNDI which users are
used to.


> This is perfect for those who want to migrate from version X to version Y.
> But they can also use the better API whoch does not use the cursed semantic
> of JNDI.
>
> So far, so good.
>

Ahha.


>
> Now, the problem is that for referrals, as we have also removed the
> ReferralInterceptor from the core server (a very good move, when you take
> the time to think about it - and it took me two hours of discussion with
> Alex to get it -), you have a new problem : where do we have to handle
> referrals ?
>

Question is for what purpose do you want to handle referrals.  There are two
ways to handle referrals:

(1) Through the protocol
(2) Via the JNDI Provided based on how JNDI is expected to behave with
different values for the Context.REFERRAL property in the environment:
throw, ignore, follow.


>
> The first idea was to move it to the protocol layer. So be it. But the
> problem is that the JNDI referral handling was not anymore done (and as a
> consequence, the associated tests were disabled, until the JNDI provider was
> fixed to handle those referrals).
>
> I'm currently re-implementing the Referral handling in this JNDI layer, but
> I suddenly discovered that this was not a good idea *at all*. And so was the
> initial idea to implement the referrals handling into the protocol layer. We
> now have 3 places where we must implement this process :
> - in the protocol layer
> - in the JNDI provider
> - and in the CoreSession layer.
>
> Bad...
>

I don't think it should be done in the CoreSession.  The CoreSession should
just provide an interface to perform operations against entries.  I think
all entries should be treated the same with respect to this interface.

If code needs to do something different because the entry is a referral it's
up to that code to do so.

Referral handling is different depending on whether the protocol is handling
it or whether the JNDI provider is handling it.  The reason why I separated
these is because the concepts were getting mixed together and JNDI was
contaminating the server.


> So here is what I suggest : we should handle referrals into the CoreSession
> layer, and deal with the thrown exceptions (LdapReferralExceptio,
> LdapPartialResultException, etc) into the JNDI and protocol layer,
> otherwise, we will do the job three times (and it's not a funny job ...)
>
> thoughts ?
>

We could do this but these exceptions are JNDI specific.  They've been
defined to deal with a specific JNDI referral handling mode (throw
specifically).  Then the protocol handler code would need to change to
understand these JNDI specifics.

Perhaps we should just not deal with referrals at all in the CoreSession.
Just add it to the JNDI provider for it's specific modes.

My mindset my be wrong (and at times I do doubt it) but I feel the core API
is purely a local access API to initialize and manipulate the DIT and its
entries.  It does not deal with following referrals since this would go over
the network.  It does not deal with referrals as special entries since this
is the responsibility of higher layers which deal with referrals in
different ways.


>
> PS: don't be too picky with the names I'm using (like CoreSession is not
> really the API we have built), but I'm sure that Alex can correct me. I
> think however that the general picture is ok.
>
>
Yeah just great - I understood easily what you meant.  I think reuse is good
and I can certainly see how putting some referral handling functionality
into the CoreSession is very tempting.  As coders we always want to reuse
and that's good.  However sometimes reuse causes more problems than it is
worth.  I think this is a good example for that kind of situation.  Also
sometimes we want to add additional functionality because it seems to make
sense to have a richer interface that handles more cases.  However more
complex interfaces are harder to work with as bricks to build other
interfaces when their rich features start to get in the way.

I trust you know the best way to proceed and you'll make it work well.  I
still however prefer the core's job to be as simple as possible.  Perhaps it
can go just a little bit further and flag Entries as being referrals so
higher layers can easily see that they're dealing with a referral entry.
The entry API can be altered to check if an entry is a referral with an
isReferral() utility method which is non-essential since we can just check
if the 'ref' attribute is present.  It might be cleaner though.  If this is
done you might also add a flag indicating that the entry is subordinate to
another referral.  This information is available through the ReferralManager
and can be accessed to set this flag before returning an entry.  Perhaps
this method can be called getFarthestReferralParent() which returns an
LdapDN instead of just a boolean check.  These additions make it easier to
use the Entry API without having to access the DirectoryService's
ReferralManager to check for these.  Then the referral handling code,
whether in the protocol provider or in the JNDI provider, has a much easier
task.  The respective code is clean and shows clearly the policy of how
referrals are handled in their respective layers.

Regards,
Alex