You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@directory.apache.org by Erik Hamrin <Er...@mobeon.com> on 2007/03/15 08:56:38 UTC

Rebuilding Apache

Hi all!

I am rebuilding the apache DS to work as a proxy server for LDAP queries. It is part of a  test bed project I´m doing. The idea is that the proxy will be placed between an LDAP router and a target of this router. I need to be able to forward all queries from the router to the target and then back again. Right now I am struggling with the interceptors, I thoiught that I could just bypass the whole stack, but all the lookups and hasEntry operations makes it difficult to proceed in a straight forward way.

If anyone has done similar modifications to Apache (perhaps even gotten them to work?) I would be most grateful for hints and pointers...

Bast regards

Erik Hamrin

Re: Rebuilding Apache

Posted by Jim Yang <ji...@safehaus.org>.
All,

I want to point out a testimonial about ApacheDS in my blog here.
Please feel free to link it to directory.apache.org's testimonial
section:

http://www.idtopia.com/blog/2006/10/12/apacheds-and-opends-folks-met/

Jim

On 3/15/07, Jim Yang <ji...@safehaus.org> wrote:
> All,
>
> Penrose developers and contributors are well aware that Penrose
> leverages Apache DS in our architecture as our LDAP listener. In fact,
> it is listed in our FAQ (http://docs.safehaus.org/display/PENROSE/FAQ)
> for sometime now.
>
> With the upcoming release of Penrose 1.2, we are adding support for
> the other Open LDAP servers, such as OpenLDAP, OpenDS and Fedora DS.
>
> We, also, like to point out we don't use a straight GPL license to
> allow interoperability with the other open source license. Please read
> more here: http://docs.safehaus.org/display/PENROSE/FLOSS
>
> Cheers,
>
> Jim
>
> On 3/15/07, Alex Karasulu <ak...@apache.org> wrote:
> > Thanks for doing so.  Sounds like CA is working on a VD solution based on
> > ApacheDS much like Identyx (w/ Penrose) is working on one.  Would be good to
> > at least have some testimonials from both camps.  There's a lot of products
> > out there that use ApacheDS in one way or another.  It would be nice if more
> > of them revealed that.
> >
> > Alex
> >
> > On 3/15/07, Norval Hope <nr...@gmail.com> wrote:
> > >
> > > I'm happy to assist as like AD and really appreciate having been able to
> > use it as a base in my day job. However, the degree to which I can
> > contribute is limitted by some factors:
> > >   1. I'm flat out busy at the moment on the releases of my day job project
> > >   2. My employer (CA) is very sensitive about the IP of our current
> > solution built on top of AD, and to avoid conflict of interest they've said
> > they don't mind me sharing broad-brush insights and findings but want me to
> > tread very carefully when it comes to coding and detailed design, especially
> > in areas outside of enhancing the AD core framework itself to make it more
> > VD friendly.
> > >
> > > Hence I may be of some use as a "domain consultant" (for whatever my
> > personal experience is worth) but less so as a major VD developer as such.
> > Just thought I should put my cards on the table and make the situation
> > clear.
> > >
> > > Cheers,
> > > Norval
> >
> >
>

Re: Rebuilding Apache

Posted by Alex Karasulu <ak...@apache.org>.
Jim,

First let me clarify just in case that I'm not suggesting that you are not
saying that
Penrose uses ApacheDS.  I'm saying it would be nice to have a testimonial
from
you guys :).

On 3/16/07, Jim Yang <ji...@safehaus.org> wrote:
>
> All,
>
> Penrose developers and contributors are well aware that Penrose
> leverages Apache DS in our architecture as our LDAP listener. In fact,
> it is listed in our FAQ (http://docs.safehaus.org/display/PENROSE/FAQ)
> for sometime now.


This is great and I knew about this already when you showed it to me a while
back.

With the upcoming release of Penrose 1.2, we are adding support for
> the other Open LDAP servers, such as OpenLDAP, OpenDS and Fedora DS.


That's good for Penrose but with the recent performance enhancements to
ApacheDS
this might not be necessary for long :).

We, also, like to point out we don't use a straight GPL license to
> allow interoperability with the other open source license. Please read
> more here: http://docs.safehaus.org/display/PENROSE/FLOSS


That's great too ...

I just caught your add-on email ... We can link to this too.  Hey it's
always good to
have more positive feedback on our site about ApacheDS.

Thanks,
Alex


Cheers,
>
> Jim
>
> On 3/15/07, Alex Karasulu <ak...@apache.org> wrote:
> > Thanks for doing so.  Sounds like CA is working on a VD solution based
> on
> > ApacheDS much like Identyx (w/ Penrose) is working on one.  Would be
> good to
> > at least have some testimonials from both camps.  There's a lot of
> products
> > out there that use ApacheDS in one way or another.  It would be nice if
> more
> > of them revealed that.
> >
> > Alex
> >
> > On 3/15/07, Norval Hope <nr...@gmail.com> wrote:
> > >
> > > I'm happy to assist as like AD and really appreciate having been able
> to
> > use it as a base in my day job. However, the degree to which I can
> > contribute is limitted by some factors:
> > >   1. I'm flat out busy at the moment on the releases of my day job
> project
> > >   2. My employer (CA) is very sensitive about the IP of our current
> > solution built on top of AD, and to avoid conflict of interest they've
> said
> > they don't mind me sharing broad-brush insights and findings but want me
> to
> > tread very carefully when it comes to coding and detailed design,
> especially
> > in areas outside of enhancing the AD core framework itself to make it
> more
> > VD friendly.
> > >
> > > Hence I may be of some use as a "domain consultant" (for whatever my
> > personal experience is worth) but less so as a major VD developer as
> such.
> > Just thought I should put my cards on the table and make the situation
> > clear.
> > >
> > > Cheers,
> > > Norval
> >
> >
>

Re: Rebuilding Apache

Posted by Jim Yang <ji...@safehaus.org>.
All,

Penrose developers and contributors are well aware that Penrose
leverages Apache DS in our architecture as our LDAP listener. In fact,
it is listed in our FAQ (http://docs.safehaus.org/display/PENROSE/FAQ)
for sometime now.

With the upcoming release of Penrose 1.2, we are adding support for
the other Open LDAP servers, such as OpenLDAP, OpenDS and Fedora DS.

We, also, like to point out we don't use a straight GPL license to
allow interoperability with the other open source license. Please read
more here: http://docs.safehaus.org/display/PENROSE/FLOSS

Cheers,

Jim

On 3/15/07, Alex Karasulu <ak...@apache.org> wrote:
> Thanks for doing so.  Sounds like CA is working on a VD solution based on
> ApacheDS much like Identyx (w/ Penrose) is working on one.  Would be good to
> at least have some testimonials from both camps.  There's a lot of products
> out there that use ApacheDS in one way or another.  It would be nice if more
> of them revealed that.
>
> Alex
>
> On 3/15/07, Norval Hope <nr...@gmail.com> wrote:
> >
> > I'm happy to assist as like AD and really appreciate having been able to
> use it as a base in my day job. However, the degree to which I can
> contribute is limitted by some factors:
> >   1. I'm flat out busy at the moment on the releases of my day job project
> >   2. My employer (CA) is very sensitive about the IP of our current
> solution built on top of AD, and to avoid conflict of interest they've said
> they don't mind me sharing broad-brush insights and findings but want me to
> tread very carefully when it comes to coding and detailed design, especially
> in areas outside of enhancing the AD core framework itself to make it more
> VD friendly.
> >
> > Hence I may be of some use as a "domain consultant" (for whatever my
> personal experience is worth) but less so as a major VD developer as such.
> Just thought I should put my cards on the table and make the situation
> clear.
> >
> > Cheers,
> > Norval
>
>

Re: Rebuilding Apache

Posted by Emmanuel Lecharny <el...@gmail.com>.
Alex Karasulu a écrit :

> Thanks for doing so.  Sounds like CA is working on a VD solution based on
> ApacheDS much like Identyx (w/ Penrose) is working on one.  Would be 
> good to
> at least have some testimonials from both camps.  There's a lot of 
> products
> out there that use ApacheDS in one way or another.  It would be nice 
> if more
> of them revealed that.

I couldn't have say it better ! We don't only appreciate contributions, 
but also testimony and real life users. The more you use Apache 
products, the more we can improve it by fixing bugs, improving 
documentation, and spreading info about the server itself. Knowing that 
CA is using ADS as a backend for one of its product is really a valuable 
contribution, if we can share this information on our web site : this is 
exactly the target Apache is targeting.

Now, you are aware that it can't be a one way trip : if we are not 
enough working on the server, then the server will stale, so the 
products built on top of it !

There is a balance to find, and regarding VD, I clearly understand that 
there is a competing situation which forbid you to share all the work 
you are currently doing for CA. This is just fair :) After all, they pay 
you a salary, something Apache can't do !

Thanks for these informations, Norval.

Emmanuel

>
> Alex
>
> On 3/15/07, Norval Hope <nr...@gmail.com> wrote:
>
>>
>> I'm happy to assist as like AD and really appreciate having been able to
>> use it as a base in my day job. However, the degree to which I can
>> contribute is limitted by some factors:
>>   1. I'm flat out busy at the moment on the releases of my day job 
>> project
>>   2. My employer (CA) is very sensitive about the IP of our current
>> solution built on top of AD, and to avoid conflict of interest 
>> they've said
>> they don't mind me sharing broad-brush insights and findings but want 
>> me to
>> tread very carefully when it comes to coding and detailed design, 
>> especially
>> in areas outside of enhancing the AD core framework itself to make it 
>> more
>> VD friendly.
>>
>> Hence I may be of some use as a "domain consultant" (for whatever my
>> personal experience is worth) but less so as a major VD developer as 
>> such.
>> Just thought I should put my cards on the table and make the situation
>> clear.
>>
>> Cheers,
>> Norval
>>
>


Re: Rebuilding Apache

Posted by Alex Karasulu <ak...@apache.org>.
Thanks for doing so.  Sounds like CA is working on a VD solution based on
ApacheDS much like Identyx (w/ Penrose) is working on one.  Would be good to
at least have some testimonials from both camps.  There's a lot of products
out there that use ApacheDS in one way or another.  It would be nice if more
of them revealed that.

Alex

On 3/15/07, Norval Hope <nr...@gmail.com> wrote:
>
> I'm happy to assist as like AD and really appreciate having been able to
> use it as a base in my day job. However, the degree to which I can
> contribute is limitted by some factors:
>   1. I'm flat out busy at the moment on the releases of my day job project
>   2. My employer (CA) is very sensitive about the IP of our current
> solution built on top of AD, and to avoid conflict of interest they've said
> they don't mind me sharing broad-brush insights and findings but want me to
> tread very carefully when it comes to coding and detailed design, especially
> in areas outside of enhancing the AD core framework itself to make it more
> VD friendly.
>
> Hence I may be of some use as a "domain consultant" (for whatever my
> personal experience is worth) but less so as a major VD developer as such.
> Just thought I should put my cards on the table and make the situation
> clear.
>
> Cheers,
> Norval
>

Re: Rebuilding Apache

Posted by Norval Hope <nr...@gmail.com>.
I'm happy to assist as like AD and really appreciate having been able to use
it as a base in my day job. However, the degree to which I can contribute is
limitted by some factors:
  1. I'm flat out busy at the moment on the releases of my day job project
  2. My employer (CA) is very sensitive about the IP of our current solution
built on top of AD, and to avoid conflict of interest they've said they
don't mind me sharing broad-brush insights and findings but want me to tread
very carefully when it comes to coding and detailed design, especially in
areas outside of enhancing the AD core framework itself to make it more VD
friendly.

Hence I may be of some use as a "domain consultant" (for whatever my
personal experience is worth) but less so as a major VD developer as such.
Just thought I should put my cards on the table and make the situation
clear.

Cheers,
Norval

Re: Rebuilding Apache

Posted by Alex Karasulu <ak...@apache.org>.
It sure would be nice if we can get you Norval and Marc working together on
a VD implementation.  Perhaps the Penrose people can jump in too but I think
they're not interested in going the way of the ASL because of business
reasons.

Regardless there's a lot of interest here.  Can we reach some critical mass?

Alex


On 3/15/07, Norval Hope <nr...@gmail.com> wrote:
>
> Thanks for the info Emmanuel!
>
> We must indeed have missed something (most probably in the twix to
> snickers transformation). I'll pass on your note to the guy who was working
> on the proxy in the past.
>
> I've been buried in an alpha release, now a beta and soon a full release
> but plan to catch up with the trunk again soon (most probably when 1.5.0is released) as I'll have to work out how to retrofit my old "dynamic schema
> from .schema file" solution to all the new schema stuff you guys have been
> working away on. I can imagine that the ".schema => equivalent schema
> objects in schema partition" job won't be too hard (I already have a parser
> taking .schema to now defunct bootstrap registry objects - so I just have to
> change to a different output representation) but I'm not sure how to quickly
> and robustly remove contributions from .schema files (or not actually store
> them) so they are transient, and aren't persisted over AD server restarts.
> For VD type solutions you want to discover them at runtime, next time you
> start up there may be different VD partitions accessible with different
> schemas so you want to add them only as they are discovered.
>
> Thanks,
> Norval
>

Re: Rebuilding Apache

Posted by Norval Hope <nr...@gmail.com>.
Thanks for the info Emmanuel!

We must indeed have missed something (most probably in the twix to snickers
transformation). I'll pass on your note to the guy who was working on the
proxy in the past.

I've been buried in an alpha release, now a beta and soon a full release but
plan to catch up with the trunk again soon (most probably when 1.5.0 is
released) as I'll have to work out how to retrofit my old "dynamic schema
from .schema file" solution to all the new schema stuff you guys have been
working away on. I can imagine that the ".schema => equivalent schema
objects in schema partition" job won't be too hard (I already have a parser
taking .schema to now defunct bootstrap registry objects - so I just have to
change to a different output representation) but I'm not sure how to quickly
and robustly remove contributions from .schema files (or not actually store
them) so they are transient, and aren't persisted over AD server restarts.
For VD type solutions you want to discover them at runtime, next time you
start up there may be different VD partitions accessible with different
schemas so you want to add them only as they are discovered.

Thanks,
Norval

Re: Rebuilding Apache

Posted by Emmanuel Lecharny <el...@gmail.com>.
Hi Norval, good to see you there :)

On 3/15/07, Norval Hope <nr...@gmail.com> wrote:
>
> Correct me if I'm wrong but my understanding is that half of the LDAP
> protocol needed to write such a proxy was left behind in the Twix to
> Snickers migration (hope I got the candy bar names the right way around :-)
>

atm, we are only using Twix, but we still have two formats for manipulated
objects  (twix and snickers) in the server (we want to remove one of them,
but we don't have time :). FYI those names where taken from Snacc, which was
a "open-source" ASN.1 codec written by IBM, which was not available for
inclusion (thus the " " around open-source). So Snacc -> Snickers, and
Snickers -> Twix.

Seriously, the way ldap messages are handled in the server is the following
:
ldapMessage -> Twix -> transformation from twix to snickers format -> server
for requests
and
ldap responses -> transformation from snickers format to twix format -> twix
encoder -> client for response


In particular my understanding is the current codec handles 1. decoding
> requests and 2. encoding response, but not the two other functions required
> by a proxy (3. decoding responses and 4. encoding requests). Is that right?
>

Wrong !!! the codec works for *every* requests and *every* response :)  It
is used in LdapStudio and in the server, so it has both side completly
implemented. The sandboxed proxy would not have been working otherwise !

We wrote a prototype proxy years ago with the old protocol provider but
> found we couldn't move it forward, but perhaps we missed something.
>

I can tell for your proxy, but I can assure you that the sandboxed proxy was
working, and not only with ADS, because I tested it against other Ldap
Servers. And FYI, we will provide a working proxy in Ldaptudio in a few
weeks (it's expected for LS 0.7).

The main concern we have with a LdapProxy is about handling N clients. The
sandboxed proxy does not handle this case... So basically, we will be happy
if someone can work on this aspect :)

Norval
>

Emmanuel




-- 
Cordialement,
Emmanuel Lécharny
www.iktek.com

Re: Rebuilding Apache

Posted by Norval Hope <nr...@gmail.com>.
Correct me if I'm wrong but my understanding is that half of the LDAP
protocol needed to write such a proxy was left behind in the Twix to
Snickers migration (hope I got the candy bar names the right way around :-)

In particular my understanding is the current codec handles 1. decoding
requests and 2. encoding response, but not the two other functions required
by a proxy (3. decoding responses and 4. encoding requests). Is that right?

We wrote a prototype proxy years ago with the old protocol provider but
found we couldn't move it forward, but perhaps we missed something.

Norval

Re: Rebuilding Apache

Posted by Emmanuel Lecharny <el...@gmail.com>.
Erik Hamrin a écrit :

>Hi all!
>  
>
Hi Erik,

>I am rebuilding the apache DS to work as a proxy server for LDAP queries. It is part of a  test bed project I´m doing. The idea is that the proxy will be placed between an LDAP router and a target of this router. I need to be able to forward all queries from the router to the target and then back again. Right now I am struggling with the interceptors, I thoiught that I could just bypass the whole stack, but all the lookups and hasEntry operations makes it difficult to proceed in a straight forward way.
>  
>
If you want to write a simple proxy, just do it on the Ldap protocol 
level. We have a sandboxed proxy if you want to start with someting 
which work (even if not really clean or perfect): 
http://svn.apache.org/viewvc/directory/sandbox/trunk/proxy/
and some doc here : 
http://cwiki.apache.org/confluence/display/DIRxSBOX/Ldap+Proxy (yeah, I 
know, not very slick :)

Writing an interceptor to do so will drive you to have to encode/decode 
requests and response in the interceptor.

>If anyone has done similar modifications to Apache (perhaps even gotten them to work?) I would be most grateful for hints and pointers...
>  
>
Hope it helps

>Bast regards
>
>Erik Hamrin
>  
>
Emmanuel