Apache Directory Sync ?

[David Boreham <da...@bozemanpass.com>]

I wish that there were an open source, expandable, LDAP
sync engine. Something like a meta directory product is
what I'm thinking of (MS has one, so do Novell and
Sun although theirs are not actively marketed). There are
a few other commercial products in this space too.
AFAIK nothing like this exists in open-source-land.

Problems such a thing would be good for solving :

1. Some user data is in some old crusty data store (Exchange 5.5,
Windows NT4 domain controller, Peoplesoft...).
We want to sync some subset of schema to/from the
corporate LDAP service. Often the 'crusty data store'
is something so strange that it only exists at one
customer, and so custom code needs to be written to access it.

3. Customer has Active Directory (hard to avoid, even
I have one!). But they really want to use a nice open
source LDAP directory server. What to do : some
DS'es have native Windows sync solutions, but not all
(not Apache DS nor OpenLDAP).

3. Federation of LDAP services (for DS products that
do not have native federation, which is almost all of them).
Company A does business with company B, they want
some subset of their directory data sync'ed in two directions.
This can't be done with replication, even if the two orgs use
the same DS product, because replication implies too much
common administration for two distinct organizations.

The thing I have in mind would have a pluggable connector
architecture. It would have a general purpose module for
correlating entries/records between sources. It would be able
to abstract all the different client sync mechanisms that
the different DS'es support (DirSync for Active Directory,
persistent search for FDS, syncrepl for OL, etc).

Java seems like a good implementation language given
the lack of a high performance requirement for the task,
and the ease with which plugins can be deployed cross platform
(and also good database connectivity).

Thoughts ?

Anyone interested in working on such a thing ?

Re: Apache Directory Sync ?

[David Boreham <da...@bozemanpass.com>]

Alex Karasulu wrote:

> I'm certainly interested in working on such projects.  Perhaps we can
> start by merging some of your work on the NT4 domains as well.

I think that makes quite a bit of sense but we'd need to
talk to folk @ redhat about this since they own the rights
to that code.

Re: Apache Directory Sync ?

[Alex Karasulu <ao...@bellsouth.net>]

David Boreham wrote:
> 
> I wish that there were an open source, expandable, LDAP
> sync engine. Something like a meta directory product is
> what I'm thinking of (MS has one, so do Novell and
> Sun although theirs are not actively marketed). There are
> a few other commercial products in this space too.
> AFAIK nothing like this exists in open-source-land.
> 
> Problems such a thing would be good for solving :
> 
> 1. Some user data is in some old crusty data store (Exchange 5.5,
> Windows NT4 domain controller, Peoplesoft...).
> We want to sync some subset of schema to/from the
> corporate LDAP service. Often the 'crusty data store'
> is something so strange that it only exists at one
> customer, and so custom code needs to be written to access it.
> 
> 3. Customer has Active Directory (hard to avoid, even
> I have one!). But they really want to use a nice open
> source LDAP directory server. What to do : some
> DS'es have native Windows sync solutions, but not all
> (not Apache DS nor OpenLDAP).
> 
> 3. Federation of LDAP services (for DS products that
> do not have native federation, which is almost all of them).
> Company A does business with company B, they want
> some subset of their directory data sync'ed in two directions.
> This can't be done with replication, even if the two orgs use
> the same DS product, because replication implies too much
> common administration for two distinct organizations.
> 
> The thing I have in mind would have a pluggable connector
> architecture. It would have a general purpose module for
> correlating entries/records between sources. It would be able
> to abstract all the different client sync mechanisms that
> the different DS'es support (DirSync for Active Directory,
> persistent search for FDS, syncrepl for OL, etc).
> 
> Java seems like a good implementation language given
> the lack of a high performance requirement for the task,
> and the ease with which plugins can be deployed cross platform
> (and also good database connectivity).
> 
> Thoughts ?

I'm very interested in doing what you mentioned in various degrees here
at the ASF.  I'll try to go into each in greater detail in a week.

> Anyone interested in working on such a thing ?

I'm certainly interested in working on such projects.  Perhaps we can
start by merging some of your work on the NT4 domains as well.


Thanks,
Alex

Re: Apache Directory Sync ?

[David Boreham <da...@bozemanpass.com>]

Jim Yang wrote:

> Yes, see http://docs.safehaus.org/display/PENROSE10/LDAP+Sync+Module

Thanks, I will read and eat.

Re: Re: Apache Directory Sync ?

[Jim Yang <au...@gmail.com>]

Yes, see http://docs.safehaus.org/display/PENROSE10/LDAP+Sync+Module

and http://docs.safehaus.org/display/PENROSE10/Polling+Connector+Module

On 11/12/06, David Boreham <da...@bozemanpass.com> wrote:
> Jim Yang wrote:
>
> > Have you checked out Penrose? http://penrose.safehaus.org
>
> A bit yes. It mentioned sync on the home page, but
> so far I haven't found anything but virtual directory
> in the code (which of course is a fine thing, but not sync).
> Is there any sync functionality in penrose today ?
>
>
>

Re: Apache Directory Sync ?

[David Boreham <da...@bozemanpass.com>]

Jim Yang wrote:

> Have you checked out Penrose? http://penrose.safehaus.org

A bit yes. It mentioned sync on the home page, but
so far I haven't found anything but virtual directory
in the code (which of course is a fine thing, but not sync).
Is there any sync functionality in penrose today ?

Re: Apache Directory Sync ?

[Jim Yang <ji...@safehaus.org>]

Hi David,

Have you checked out Penrose? http://penrose.safehaus.org

-Jim

On 11/11/06, David Boreham <da...@bozemanpass.com> wrote:
>
>
> I wish that there were an open source, expandable, LDAP
> sync engine. Something like a meta directory product is
> what I'm thinking of (MS has one, so do Novell and
> Sun although theirs are not actively marketed). There are
> a few other commercial products in this space too.
> AFAIK nothing like this exists in open-source-land.
>
> Problems such a thing would be good for solving :
>
> 1. Some user data is in some old crusty data store (Exchange 5.5,
> Windows NT4 domain controller, Peoplesoft...).
> We want to sync some subset of schema to/from the
> corporate LDAP service. Often the 'crusty data store'
> is something so strange that it only exists at one
> customer, and so custom code needs to be written to access it.
>
> 3. Customer has Active Directory (hard to avoid, even
> I have one!). But they really want to use a nice open
> source LDAP directory server. What to do : some
> DS'es have native Windows sync solutions, but not all
> (not Apache DS nor OpenLDAP).
>
> 3. Federation of LDAP services (for DS products that
> do not have native federation, which is almost all of them).
> Company A does business with company B, they want
> some subset of their directory data sync'ed in two directions.
> This can't be done with replication, even if the two orgs use
> the same DS product, because replication implies too much
> common administration for two distinct organizations.
>
> The thing I have in mind would have a pluggable connector
> architecture. It would have a general purpose module for
> correlating entries/records between sources. It would be able
> to abstract all the different client sync mechanisms that
> the different DS'es support (DirSync for Active Directory,
> persistent search for FDS, syncrepl for OL, etc).
>
> Java seems like a good implementation language given
> the lack of a high performance requirement for the task,
> and the ease with which plugins can be deployed cross platform
> (and also good database connectivity).
>
> Thoughts ?
>
> Anyone interested in working on such a thing ?
>
>
>

Re: Apache Directory Sync ?

[David Boreham <da...@bozemanpass.com>]

Ersin Er wrote:

>> 3. Federation of LDAP services (for DS products that
>> do not have native federation, which is almost all of them).
>> Company A does business with company B, they want
>> some subset of their directory data sync'ed in two directions.
>> This can't be done with replication, even if the two orgs use
>> the same DS product, because replication implies too much
>> common administration for two distinct organizations.
>
>
> What about referrals?

Do you mean LDAP clients in Company A would receive
a referral and contact Company B's server directly ?
If so then that's something I've never seen deployed to
solve a federation problem. I suspect that no IT group
is going to allow random clients to access its LDAP
service from an outside organization.

> We have thoughts on virtual directories and views for LDAP. Not in the
> very near future, but we'll introduce some facilities like you
> mentioned. And also replication is hopefully coming with the next
> release. You may futher detail your requirements so we can use them to
> form our roadmap.

That all sounds great, but none of the above is a sync tool,
which is what I'm asking about here.

Re: Apache Directory Sync ?

[Ersin Er <er...@gmail.com>]

Hi,

On 11/12/06, David Boreham <da...@bozemanpass.com> wrote:
>
> I wish that there were an open source, expandable, LDAP
> sync engine. Something like a meta directory product is
> what I'm thinking of (MS has one, so do Novell and
> Sun although theirs are not actively marketed). There are
> a few other commercial products in this space too.
> AFAIK nothing like this exists in open-source-land.
>
> Problems such a thing would be good for solving :
>
> 1. Some user data is in some old crusty data store (Exchange 5.5,
> Windows NT4 domain controller, Peoplesoft...).
> We want to sync some subset of schema to/from the
> corporate LDAP service. Often the 'crusty data store'
> is something so strange that it only exists at one
> customer, and so custom code needs to be written to access it.
>
> 3. Customer has Active Directory (hard to avoid, even
> I have one!). But they really want to use a nice open
> source LDAP directory server. What to do : some
> DS'es have native Windows sync solutions, but not all
> (not Apache DS nor OpenLDAP).

BTW, not directly related but ApacheDS has full support for Kerberos
and Change Password protocols.

> 3. Federation of LDAP services (for DS products that
> do not have native federation, which is almost all of them).
> Company A does business with company B, they want
> some subset of their directory data sync'ed in two directions.
> This can't be done with replication, even if the two orgs use
> the same DS product, because replication implies too much
> common administration for two distinct organizations.

What about referrals?

> The thing I have in mind would have a pluggable connector
> architecture. It would have a general purpose module for
> correlating entries/records between sources. It would be able
> to abstract all the different client sync mechanisms that
> the different DS'es support (DirSync for Active Directory,
> persistent search for FDS, syncrepl for OL, etc).

Again BTW, ApacheDS has support for Persistent Search too.

> Java seems like a good implementation language given
> the lack of a high performance requirement for the task,
> and the ease with which plugins can be deployed cross platform
> (and also good database connectivity).
>
> Thoughts ?
>
> Anyone interested in working on such a thing ?

We have thoughts on virtual directories and views for LDAP. Not in the
very near future, but we'll introduce some facilities like you
mentioned. And also replication is hopefully coming with the next
release. You may futher detail your requirements so we can use them to
form our roadmap.

Thanks.

-- 
Ersin