You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@directory.apache.org by Pierre-Arnaud Marcelot <pa...@marcelot.net> on 2009/01/13 17:10:29 UTC
[LDAP] Mimic RDBMS auto-increment feature in LDAP
Hi guys,
I have a tricky question which is not really related specifically to
ApacheDS but more LDAP specific.
I'm working on porting to LDAP a kind of RBAC API for handling Users, Rights
and Applications which is based on a relational database.
Most of these objects are based on an integer primary key (and an
auto-increment property placed on the associated table) that I need to keep
when porting the API.
The problem is LDAP servers does not know about "auto-increment" IDs.
Do you have any recommended design pattern for solving such a problem?
Regards,
Pierre-Arnaud
Re: [LDAP] Mimic RDBMS auto-increment feature in LDAP
Posted by Aleksander Adamowski <al...@olo.org.pl>.
On Tue, Jan 13, 2009 at 5:10 PM, Pierre-Arnaud Marcelot <pa...@marcelot.net> wrote:
> Hi guys,
>
> I have a tricky question which is not really related specifically to
> ApacheDS but more LDAP specific.
>
> I'm working on porting to LDAP a kind of RBAC API for handling Users, Rights
> and Applications which is based on a relational database.
>
> Most of these objects are based on an integer primary key (and an
> auto-increment property placed on the associated table) that I need to keep
> when porting the API.
> The problem is LDAP servers does not know about "auto-increment" IDs.
>
> Do you have any recommended design pattern for solving such a problem?
In a replicated multi-master environment this is non trivial matter if
you want to retain consistency, avoid duplicate numbers being assigned
and want to have high performance.
I think that Red Hat has solved this quite elegantly in Fedora
Directory Server with their DNS (Distributed Numeric Assignment)
plugin:
http://directory.fedoraproject.org/wiki/DNA_Plugin
http://directory.fedoraproject.org/wiki/DNA_Plugin_Proposal
In general, servers that take part in replication agreements, get
numeric range assignments that they can use exclusively and renew them
when they begin to run out of numbers in those ranges.
It would be nice to see similar functionality implemented in Apache
Directory Server in the future.
--
Best Regards,
Aleksander Adamowski
http://olo.org.pl
Re: [LDAP] Mimic RDBMS auto-increment feature in LDAP
Posted by Emmanuel Lecharny <el...@gmail.com>.
David Boreham wrote:
> Emmanuel Lecharny wrote:
>> The thing is that Pierre-Arnaud is moving away from a RDBMS system ;)
>> But the applications are using Integers and Longs for IDs...
> Right, and I'd say either 1) Don't move from the RDBMS, or 2) Change
> the applications to not expect integers.
>
> Do not do 3) : attempt to make the DS behave like the RDBMS.
I see your point... In any case, it would be way better to use a unique
RDN to identify an element in LDAP. Then no need to use auto-increment
counter, with all the burden it is to manage (even in a non replicated
environment), not to mention the fact that it's not standard...
Thanks David !
--
--
cordialement, regards,
Emmanuel Lécharny
www.iktek.com
directory.apache.org
Re: [LDAP] Mimic RDBMS auto-increment feature in LDAP
Posted by David Boreham <da...@bozemanpass.com>.
Howard Chu wrote:
> There's nothing inherently "relational" about this feature.
Hence my comment that the same problems arise with a replicated RDBMS
deployment.
It is 'relational' in the sense that it's a feature available in all
popular RDBMS, and not available in any
of the popular LDAP servers (yes you can add it by changing the code,
but then it's not an 'LDAP' server
any more).
Re: [LDAP] Mimic RDBMS auto-increment feature in LDAP
Posted by Aleksander Adamowski <ap...@olo.org.pl>.
On Wed, Jan 14, 2009 at 9:20 AM, Howard Chu <hy...@symas.com> wrote:
> The DNA plugin that was written for FedoraDS uses the same strategy that
> MySQL uses for auto-incrementing numbers - allocate a span of numbers to a
> given node, where every node's span is disjoint. There's no particular
> reason not to keep going with this approach, if it's too hard to change the
> rest of the software. There's nothing inherently "relational" about this
> feature.
That's well said, I agree completely!
Also, note that some applications (like pam-ldap) cannot easily change
their data model, since they are constrained by other factors (in the
case of PAM, the architecture of POSIX systems that mandates that uid
and gid numbers be integers, and some systems even have a 16-bit size
limitation there).
--
Best Regards,
Aleksander Adamowski
http://olo.org.pl
Re: [LDAP] Mimic RDBMS auto-increment feature in LDAP
Posted by Howard Chu <hy...@symas.com>.
David Boreham wrote:
> Emmanuel Lecharny wrote:
>> The thing is that Pierre-Arnaud is moving away from a RDBMS system ;)
>> But the applications are using Integers and Longs for IDs...
> Right, and I'd say either 1) Don't move from the RDBMS, or 2) Change the
> applications to not expect integers.
>
> Do not do 3) : attempt to make the DS behave like the RDBMS.
In this particular case, there's nothing special about auto-incrementing
values that makes them more suited to RDBMSs or a directory. Like so many data
management issues, the problems (and solutions) are the same, whether you wrap
in them in relational trappings or hierarchical.
The DNA plugin that was written for FedoraDS uses the same strategy that MySQL
uses for auto-incrementing numbers - allocate a span of numbers to a given
node, where every node's span is disjoint. There's no particular reason not to
keep going with this approach, if it's too hard to change the rest of the
software. There's nothing inherently "relational" about this feature.
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: [LDAP] Mimic RDBMS auto-increment feature in LDAP
Posted by David Boreham <da...@bozemanpass.com>.
Pierre-Arnaud Marcelot wrote:
> I'd love to do option #2 but this is something we can not do as the
> project I'm working on should not have any impact on the underlying
> applications except the application which exposes the model and
> discusses with the storage.
>
> I can't do option #1 as it the aim of the project... ;)
>
> I think we'll _just_ need to integrate the handling of the incremented
> IDs in the application we're re-writing (searching for the next
> available id and then insert the entry once we've figured out what
> value to use).
> The idea is also to keep the existing API for compatibility reasons,
> but deprecate a few elements in the model and additionally provide a
> *cleaner* API that applications could use for a future migration.
This is a form of #2 (make the client not require the LDAP server to
implement small positive integer unique ID generation).
It won't work in a replicated environment, (a variant of the scheme
where you build a your own distributed id allocation service would fix
this), but you may not have that requirement.
Re: [LDAP] Mimic RDBMS auto-increment feature in LDAP
Posted by Pierre-Arnaud Marcelot <pa...@marcelot.net>.
Hi all,
Thanks for your insights and all these comments.
I'd love to do option #2 but this is something we can not do as the project
I'm working on should not have any impact on the underlying applications
except the application which exposes the model and discusses with the
storage.
I can't do option #1 as it the aim of the project... ;)
I think we'll _just_ need to integrate the handling of the incremented IDs
in the application we're re-writing (searching for the next available id and
then insert the entry once we've figured out what value to use).
The idea is also to keep the existing API for compatibility reasons, but
deprecate a few elements in the model and additionally provide a *cleaner*
API that applications could use for a future migration.
Pierre-Arnaud
On Wed, Jan 14, 2009 at 7:54 AM, David Boreham <da...@bozemanpass.com>wrote:
> Emmanuel Lecharny wrote:
>
>> The thing is that Pierre-Arnaud is moving away from a RDBMS system ;) But
>> the applications are using Integers and Longs for IDs...
>>
> Right, and I'd say either 1) Don't move from the RDBMS, or 2) Change the
> applications to not expect integers.
>
> Do not do 3) : attempt to make the DS behave like the RDBMS.
>
>
>
>
>
>
Re: [LDAP] Mimic RDBMS auto-increment feature in LDAP
Posted by David Boreham <da...@bozemanpass.com>.
Emmanuel Lecharny wrote:
> The thing is that Pierre-Arnaud is moving away from a RDBMS system ;)
> But the applications are using Integers and Longs for IDs...
Right, and I'd say either 1) Don't move from the RDBMS, or 2) Change the
applications to not expect integers.
Do not do 3) : attempt to make the DS behave like the RDBMS.
Re: [LDAP] Mimic RDBMS auto-increment feature in LDAP
Posted by Emmanuel Lecharny <el...@gmail.com>.
David Boreham wrote:
> Emmanuel Lecharny wrote:
>> David Boreham wrote:
>>> I would recommend not trying to use the auto-increment primary key
>>> approach with a DS.
>>> It won't work.
>>> Instead change the data model to use a UUID in place of the
>>> monotonic increasing integer.
>> That's an option, except when your client need to manipulate an int
>> or a long, as an UUID will be 128 bits long :/
> You can try tricks like truncating the UUID to 32-bits but it's best
> to re-design the client to not have this restriction.
>
> At some point you have go also consider giving up and answering the
> statement 'but I can do xyz with a
> relational database' with 'well, then use a relational database'.
> Except the RDBMS would have exactly the
> same problem with the integer ID when you try to deploy replication or
> partitioning...
The thing is that Pierre-Arnaud is moving away from a RDBMS system ;)
But the applications are using Integers and Longs for IDs...
--
--
cordialement, regards,
Emmanuel Lécharny
www.iktek.com
directory.apache.org
Re: [LDAP] Mimic RDBMS auto-increment feature in LDAP
Posted by David Boreham <da...@bozemanpass.com>.
Emmanuel Lecharny wrote:
> David Boreham wrote:
>> I would recommend not trying to use the auto-increment primary key
>> approach with a DS.
>> It won't work.
>> Instead change the data model to use a UUID in place of the monotonic
>> increasing integer.
> That's an option, except when your client need to manipulate an int or
> a long, as an UUID will be 128 bits long :/
You can try tricks like truncating the UUID to 32-bits but it's best to
re-design the client to not have this restriction.
At some point you have go also consider giving up and answering the
statement 'but I can do xyz with a
relational database' with 'well, then use a relational database'. Except
the RDBMS would have exactly the
same problem with the integer ID when you try to deploy replication or
partitioning...
Re: [LDAP] Mimic RDBMS auto-increment feature in LDAP
Posted by Emmanuel Lecharny <el...@gmail.com>.
David Boreham wrote:
> I would recommend not trying to use the auto-increment primary key
> approach with a DS.
> It won't work.
> Instead change the data model to use a UUID in place of the monotonic
> increasing integer.
That's an option, except when your client need to manipulate an int or a
long, as an UUID will be 128 bits long :/
--
--
cordialement, regards,
Emmanuel Lécharny
www.iktek.com
directory.apache.org
Re: [LDAP] Mimic RDBMS auto-increment feature in LDAP
Posted by Alex Karasulu <ak...@gmail.com>.
On Wed, Jan 14, 2009 at 12:29 AM, David Boreham <da...@bozemanpass.com>wrote:
> I would recommend not trying to use the auto-increment primary key approach
> with a DS.
> It won't work.
> Instead change the data model to use a UUID in place of the monotonic
> increasing integer.
>
>
>
Ahh very clever yes makes sense. The UUID is a kind of primary key for all
entries anyways.
Nice,
Alex
Re: [LDAP] Mimic RDBMS auto-increment feature in LDAP
Posted by David Boreham <da...@bozemanpass.com>.
I would recommend not trying to use the auto-increment primary key
approach with a DS.
It won't work.
Instead change the data model to use a UUID in place of the monotonic
increasing integer.
Re: [LDAP] Mimic RDBMS auto-increment feature in LDAP
Posted by Alex Karasulu <ak...@gmail.com>.
On Tue, Jan 13, 2009 at 11:20 AM, Stefan Seelmann <se...@apache.org>wrote:
> Hi Pierre-Arnaud,
>
> there is a Modify-Increment Extension, see
> http://tools.ietf.org/html/rfc4525. Not sure which LDAP servers implement
> it.
>
I think this RFC was intended to enable atomic read-modify operations to
increment attribute values rather than as a auto increment field.
One way to implement what PAM is talking about would be to use triggers but
we never really got that worked out. The trigger would increment one or
more designated auto-id attributes of a STRUCTURAL objectClass whenever a
new instance was created in the DIT.
This feature could of course be implemented without using triggers. A new
schema extension can be created to toggle an attributeType of an objectClass
as auto-increment. Don't know for certain if this should only be allowed
for STRUCTURAL objectClasses for certain but it feels like it would avoid
some issues.
Regards,
Alex
> Quoting Pierre-Arnaud Marcelot <pa...@marcelot.net>:
>
> Hi guys,
>>
>> I have a tricky question which is not really related specifically to
>> ApacheDS but more LDAP specific.
>>
>> I'm working on porting to LDAP a kind of RBAC API for handling Users,
>> Rights
>> and Applications which is based on a relational database.
>>
>> Most of these objects are based on an integer primary key (and an
>> auto-increment property placed on the associated table) that I need to
>> keep
>> when porting the API.
>> The problem is LDAP servers does not know about "auto-increment" IDs.
>>
>> Do you have any recommended design pattern for solving such a problem?
>>
>> Regards,
>> Pierre-Arnaud
>>
>>
>
>
>
Re: [LDAP] Mimic RDBMS auto-increment feature in LDAP
Posted by Emmanuel Lecharny <el...@gmail.com>.
Otherwise, an other idea would be to implement a trigger and a SP :
when an add request arrives with, say, a missing uidNumber (if it's a
mandatory AT), then it search for the biggest uidNumber, increment it,
and inject the missing uidNumber into the entry.
A bit like the OpenLDAP autoIncrement overlay do.
On Tue, Jan 13, 2009 at 5:20 PM, Stefan Seelmann <se...@apache.org> wrote:
> Hi Pierre-Arnaud,
>
> there is a Modify-Increment Extension, see
> http://tools.ietf.org/html/rfc4525. Not sure which LDAP servers implement
> it.
>
> Kind Regards,
> Stefan
>
> Quoting Pierre-Arnaud Marcelot <pa...@marcelot.net>:
>
>> Hi guys,
>>
>> I have a tricky question which is not really related specifically to
>> ApacheDS but more LDAP specific.
>>
>> I'm working on porting to LDAP a kind of RBAC API for handling Users,
>> Rights
>> and Applications which is based on a relational database.
>>
>> Most of these objects are based on an integer primary key (and an
>> auto-increment property placed on the associated table) that I need to
>> keep
>> when porting the API.
>> The problem is LDAP servers does not know about "auto-increment" IDs.
>>
>> Do you have any recommended design pattern for solving such a problem?
>>
>> Regards,
>> Pierre-Arnaud
>>
>
>
>
>
--
Regards,
Cordialement,
Emmanuel Lécharny
www.iktek.com
Re: [LDAP] Mimic RDBMS auto-increment feature in LDAP
Posted by Stefan Seelmann <se...@apache.org>.
Hi Pierre-Arnaud,
there is a Modify-Increment Extension, see
http://tools.ietf.org/html/rfc4525. Not sure which LDAP servers
implement it.
Kind Regards,
Stefan
Quoting Pierre-Arnaud Marcelot <pa...@marcelot.net>:
> Hi guys,
>
> I have a tricky question which is not really related specifically to
> ApacheDS but more LDAP specific.
>
> I'm working on porting to LDAP a kind of RBAC API for handling Users, Rights
> and Applications which is based on a relational database.
>
> Most of these objects are based on an integer primary key (and an
> auto-increment property placed on the associated table) that I need to keep
> when porting the API.
> The problem is LDAP servers does not know about "auto-increment" IDs.
>
> Do you have any recommended design pattern for solving such a problem?
>
> Regards,
> Pierre-Arnaud
>
Re: [LDAP] Mimic RDBMS auto-increment feature in LDAP
Posted by Stefan Seelmann <se...@apache.org>.
Hi Pierre-Arnaud,
there is a Modify-Increment Extension, see
http://tools.ietf.org/html/rfc4525. Not sure which LDAP servers
implement it.
Kind Regards,
Stefan
Quoting Pierre-Arnaud Marcelot <pa...@marcelot.net>:
> Hi guys,
>
> I have a tricky question which is not really related specifically to
> ApacheDS but more LDAP specific.
>
> I'm working on porting to LDAP a kind of RBAC API for handling Users, Rights
> and Applications which is based on a relational database.
>
> Most of these objects are based on an integer primary key (and an
> auto-increment property placed on the associated table) that I need to keep
> when porting the API.
> The problem is LDAP servers does not know about "auto-increment" IDs.
>
> Do you have any recommended design pattern for solving such a problem?
>
> Regards,
> Pierre-Arnaud
>
Re: [LDAP] Mimic RDBMS auto-increment feature in LDAP
Posted by Aleksander Adamowski <al...@olo.org.pl>.
On Tue, Jan 13, 2009 at 5:10 PM, Pierre-Arnaud Marcelot <pa...@marcelot.net> wrote:
> Hi guys,
>
> I have a tricky question which is not really related specifically to
> ApacheDS but more LDAP specific.
>
> I'm working on porting to LDAP a kind of RBAC API for handling Users, Rights
> and Applications which is based on a relational database.
>
> Most of these objects are based on an integer primary key (and an
> auto-increment property placed on the associated table) that I need to keep
> when porting the API.
> The problem is LDAP servers does not know about "auto-increment" IDs.
>
> Do you have any recommended design pattern for solving such a problem?
In a replicated multi-master environment this is non trivial matter if
you want to retain consistency, avoid duplicate numbers being assigned
and want to have high performance.
I think that Red Hat has solved this quite elegantly in Fedora
Directory Server with their DNS (Distributed Numeric Assignment)
plugin:
http://directory.fedoraproject.org/wiki/DNA_Plugin
http://directory.fedoraproject.org/wiki/DNA_Plugin_Proposal
In general, servers that take part in replication agreements, get
numeric range assignments that they can use exclusively and renew them
when they begin to run out of numbers in those ranges.
It would be nice to see similar functionality implemented in Apache
Directory Server in the future.
--
Best Regards,
Aleksander Adamowski
http://olo.org.pl