You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@directory.apache.org by Sebastian Oerding <se...@robotron.de> on 2014/09/17 10:24:59 UTC
Writing / integrating LDAP server
Hello,
I have very specific requirements. We want to use a LDAP server for
storing X.509 certificates. At runtime certificates are requested from
the LDAP server. However there will be the case that a certificate is
unknown by the server. In this case the certificate is requested from a
3rd pary system, stored into the LDAP server to avoid the fallback for
future requests, and returned to the client originally requesting the
certificate.
Hence I wonder to write a component forwarding client requests to a
connected LDAP server and falling back to the other approach in case of
no result on the LDAP. However this means I have to be able to "speak"
the LDAP protocol at least to return data obtained from the fallback
approach as proper LDP protocol data units or to forward responses from
the LDAP back to the client. The Java classes make it really simple to
"unwrap" data, for example to get a requested certificate as byte[] but
this gives no hint to me how to "wrap" the data into proper LDAP responses.
Unfortunately Google gives no search result providing held in writing a
LDAP server in Java. Can someone provide help by giving information
about a specific package to look into the Apache DS source code? It
would also appreciate other approaches. However my component must be
chainable. Thus extending the Apache DS by self written interceptors may
be a problem due to possible performance overhead (not offending but
having the complete logic of the server in each step of a chain feels
like an overkill and we may produce a high load such as 2.400.000
requests every 900 seconds).
Any useful hints?
With kind regards
Sebastian
Re: Writing / integrating LDAP server
Posted by Kiran Ayyagari <ka...@apache.org>.
On Wed, Sep 17, 2014 at 5:26 PM, Sebastian Oerding <
sebastian.oerding@robotron.de> wrote:
> Hello Kiran,
>
> thanks for your great reply even outlining the code for an interceptor.
> Actually I haven't really started digging into implementing an interceptor
> but I will do so after writing this mail. However currently there are two
> points left open for me.
>
> 1) Reading the outlined code to the line
> q.add(new MyCustomCertRequest(.....));
> (did the negation / notEmpty were exchanged?) I do not understand how
> responding to a client is delayed until the certificate is available in the
> fallback case (or do I have to insert it into the cursor).
>
> nah, that pseudo code is describing a scenario where when a cert is not
found then
server will not wait till you fetch the cert from a 3rd party server and
store
> 2) Due to my requirements the certificates (only X.509 certificates with
> X.509 extension SubjectKeyIdentifier) are requested by the
> subjectKeyIdentifier. Furthermore the revocation state for certificates may
> be set by LDAP requests and certificates may be stored in the LDAP. Except
> this requests no other requests are valid. Hence if I write an interceptor
> and have a chain of this component, I end up with a chain of Apache DS
are you referring to having multiple instances of ApacheDS? I don't see how
having a custom interceptor
leads to multiple ApacheDS instances, can you elaborate on what your idea
is?
> instances. Each instance in this chain will for example do the same
> verification on the request and so on. Hence I would expect this approach
> leading to a huge performance overhead (notice the worst case szenario of
> over 2.400.000 requests each 900 seconds). Furthermore there are other
> considerations which may prevent using an interceptor. Hence I wonder how
> much it would took me to write a minimal / pseudo LDAP server which only is
> capable of the two specified request types and nothing else. However having
> read RFC 4510, 4511 and having taken a look into related RFCs I do not want
> to implement even "only" an own pseudo server from scratch by myself. Thus
> the question remained whether there is some stuff I can take from the
> Apache DS / related projects to simply listen for incoming requests, do my
> simple parsing and depending on the request, forward it to the LDAP server,
> return the response if there is any and one is expected or fallback to the
> 3rd party system if no certificate is found on the LDAP server. In the
> latter case I want to take the response from the 3rd party system, store it
> on the server and transmit the response to the client as LDAP PDUs
> according to the request.
>
> your custom interceptor can perform fetching from 3rd party store if
needed and store and return to user
this will definitely have an impact on performance cause of the network
and I/O overhead.
this is one reason I suggested you use asynch approach, but when a cert is
not present in the LDAP server
then user will not receive it in the first request but in a subsequent
request, provided the interceptor's asynch
thread fetches the cert from 3rd party and stores in LDAP server.
Can you provide any information about this issue?
>
> Sebastian
>
--
Kiran Ayyagari
http://keydap.com
Re: Writing / integrating LDAP server
Posted by Sebastian Oerding <se...@robotron.de>.
Hello Kiran,
thanks for your great reply even outlining the code for an interceptor.
Actually I haven't really started digging into implementing an
interceptor but I will do so after writing this mail. However currently
there are two points left open for me.
1) Reading the outlined code to the line
q.add(new MyCustomCertRequest(.....));
(did the negation / notEmpty were exchanged?) I do not understand how
responding to a client is delayed until the certificate is available in
the fallback case (or do I have to insert it into the cursor).
2) Due to my requirements the certificates (only X.509 certificates with
X.509 extension SubjectKeyIdentifier) are requested by the
subjectKeyIdentifier. Furthermore the revocation state for certificates
may be set by LDAP requests and certificates may be stored in the LDAP.
Except this requests no other requests are valid. Hence if I write an
interceptor and have a chain of this component, I end up with a chain of
Apache DS instances. Each instance in this chain will for example do the
same verification on the request and so on. Hence I would expect this
approach leading to a huge performance overhead (notice the worst case
szenario of over 2.400.000 requests each 900 seconds). Furthermore there
are other considerations which may prevent using an interceptor. Hence I
wonder how much it would took me to write a minimal / pseudo LDAP server
which only is capable of the two specified request types and nothing
else. However having read RFC 4510, 4511 and having taken a look into
related RFCs I do not want to implement even "only" an own pseudo server
from scratch by myself. Thus the question remained whether there is
some stuff I can take from the Apache DS / related projects to simply
listen for incoming requests, do my simple parsing and depending on the
request, forward it to the LDAP server, return the response if there is
any and one is expected or fallback to the 3rd party system if no
certificate is found on the LDAP server. In the latter case I want to
take the response from the 3rd party system, store it on the server and
transmit the response to the client as LDAP PDUs according to the request.
Can you provide any information about this issue?
Sebastian
Re: Writing / integrating LDAP server
Posted by Kiran Ayyagari <ka...@apache.org>.
On Wed, Sep 17, 2014 at 1:54 PM, Sebastian Oerding <
sebastian.oerding@robotron.de> wrote:
> Hello,
>
> Hi Sebastian,
> I have very specific requirements. We want to use a LDAP server for
> storing X.509 certificates. At runtime certificates are requested from the
> LDAP server. However there will be the case that a certificate is unknown
> by the server. In this case the certificate is requested from a 3rd pary
> system, stored into the LDAP server to avoid the fallback for future
> requests, and returned to the client originally requesting the certificate.
>
> so, to be clear that we are on same page, when the certificate is
retrieved from 3rd party source
it will be stored into the LDAP server before/after returning, right?
> Hence I wonder to write a component forwarding client requests to a
> connected LDAP server and falling back to the other approach in case of no
> result on the LDAP. However this means I have to be able to "speak" the
> LDAP protocol at least to return data obtained from the fallback approach
> as proper LDP protocol data units or to forward responses from the LDAP
> back to the client. The Java classes make it really simple to "unwrap"
> data, for example to get a requested certificate as byte[] but this gives
> no hint to me how to "wrap" the data into proper LDAP responses.
>
> Unfortunately Google gives no search result providing held in writing a
> LDAP server in Java. Can someone provide help by giving information about a
> specific package to look into the Apache DS source code? It would also
> appreciate other approaches. However my component must be chainable. Thus
> extending the Apache DS by self written interceptors may be a problem due
> to possible performance overhead (not offending but having the complete
> logic of the server in each step of a chain feels like an overkill and we
> may produce a high load such as 2.400.000 requests every 900 seconds).
>
> Any useful hints?
>
> unless you have control over your client, the only way with ApacheDS is to
add a custom
interceptor and overwrite the search() operation.
The implementation will be like:
Note that I used a async mode to fetch and store the missing certs, to
avoid possible delays per each
local search that found zero certs. You may make it synchronous as per your
needs.
public class X509SearchInterceptor extends BaseInterceptor
{
public void init( DirectoryService directoryService ) throws
LdapException
{
super.init( directoryService );
// initialize a queue that holds the search requests for which no
certs wee found
LinkedBlockingQueue queue = .....
// and start a thread to process the messages pushed to this
queue from the search method
}
public EntryFilteringCursor search( SearchOperationContext
searchContext ) throws LdapException
{
// first let the search complete
EntryFilteringCursor cursor = next( searchContext );
// but before returning the cursor to read the entries
// check if the cursor has any entries
boolean notEmpty = cursor.next();
if(!notEmpty)
{
// reset the cursor position
cursor.prev();
}
else
{
// push the search details to the queue for async processing of
fetching
// from 3rd party source and storing in LDAP
q.add(new MyCustomCertRequest(.....));
}
// continue
return cursor;
}
With kind regards
> Sebastian
>
--
Kiran Ayyagari
http://keydap.com
Re: Writing / integrating LDAP server
Posted by Kiran Ayyagari <ka...@apache.org>.
On Mon, Sep 22, 2014 at 8:06 PM, Sebastian Oerding <
sebastian.oerding@robotron.de> wrote:
> Hello,
>
>
>> The best solution is to have one ApacheDS instance as a _proxy_ and cause
>> you are going to use
>> it in embedded mode then you practically have all things ready to make
>> this solution work, provided
>> you know which 3rd party server needs to be contacted for a specific
>> request when a cert is not present.
>>
>> If you can determine that information based on the search request, then
>> the best way to solve it in
>> the proxy ApacheDS instance is by adding a new in-memory partition and
>> overwrite the search() method.
>>
>
> Does that mean using an embedded Apache Ds with an own interceptor? Or is
> there a partition class whose search method to overwrite?
>
you can overwrite anywhere, but in a Partition you will have access to the
SearchEngine which is not
available in the interceptor
> Why to use a new in memory partition and is there any kind of relatioship
> between a scheme (definition) and a partition?
>
> no, there is no relationship between sheama and the custom partition
implementation.
This partition is easy to add to the server than an interceptor.
You may want to explore first with a custom interceptor.
> Sebastian
>
--
Kiran Ayyagari
http://keydap.com
Re: Writing / integrating LDAP server
Posted by Sebastian Oerding <se...@robotron.de>.
Hello,
>
> The best solution is to have one ApacheDS instance as a _proxy_ and
> cause you are going to use
> it in embedded mode then you practically have all things ready to make
> this solution work, provided
> you know which 3rd party server needs to be contacted for a specific
> request when a cert is not present.
>
> If you can determine that information based on the search request,
> then the best way to solve it in
> the proxy ApacheDS instance is by adding a new in-memory partition and
> overwrite the search() method.
Does that mean using an embedded Apache Ds with an own interceptor? Or
is there a partition class whose search method to overwrite?
Why to use a new in memory partition and is there any kind of
relatioship between a scheme (definition) and a partition?
Sebastian
Re: Writing / integrating LDAP server
Posted by Kiran Ayyagari <ka...@apache.org>.
On Wed, Sep 17, 2014 at 9:58 PM, Sebastian Oerding <
sebastian.oerding@robotron.de> wrote:
> The certificates we have to deal with are defined in a technical
> recommendation, specifying some mandatory / forbidden X.509 extensions and
> other special stuff. Especially there is some stuff which is (close to)
> violating RFC 5280.
>
> Furthermore there may be customers which require that a specific product
> is used as LDAP Server (OpenLDAP, Apache DS, MS Active Directory, Novell
> Directory Server, ...). Hence we can only rely on functionality which
> available in all of these products (or can be implemented for them with a
> reasonable effort).
>
> Currently I'm wondering about using an embedded Apache DS with
> interceptors simply forwarding each call to the corresponding stand-alone
> LDAP server (which may also be an Apache DS). However let's denote my
> component with C then we may end up with (ASCII art) the following due to
> chaining:
>
> C - C
> | |
> external LDAP external LDAP
>
> ok, this kinda answers my earlier question, but why do you need to have
more than one component?
The best solution is to have one ApacheDS instance as a _proxy_ and cause
you are going to use
it in embedded mode then you practically have all things ready to make this
solution work, provided
you know which 3rd party server needs to be contacted for a specific
request when a cert is not present.
If you can determine that information based on the search request, then the
best way to solve it in
the proxy ApacheDS instance is by adding a new in-memory partition and
overwrite the search() method.
Now each of these instances of C holding an embedded Apache DS which simply
> forwards the messages to the external LDAP seem to result in a huge
> overhead which may be unaffordable. I would prefer a better
by "overhead" you mean maintenance or performance?
performance wise there is nothing other than the overhead of proxying a
connection.
> solution but I have around five days to implement this functionality and I
> do not see any chance to implement this stuff if there is no code which
> nearly exactly fits my needs and that I can simply take and re-use.
> Unfortunately arguing for a better technical solution or more time is
> currently ignored. Hence recommendations for other approaches currently are
> of no help (just in case someone wants to argue that everything would be
> fine using JMS / protocol XYZ / ...).
>
> Sebastian
>
>
--
Kiran Ayyagari
http://keydap.com
Re: Writing / integrating LDAP server
Posted by Sebastian Oerding <se...@robotron.de>.
The certificates we have to deal with are defined in a technical
recommendation, specifying some mandatory / forbidden X.509 extensions
and other special stuff. Especially there is some stuff which is (close
to) violating RFC 5280.
Furthermore there may be customers which require that a specific product
is used as LDAP Server (OpenLDAP, Apache DS, MS Active Directory, Novell
Directory Server, ...). Hence we can only rely on functionality which
available in all of these products (or can be implemented for them with
a reasonable effort).
Currently I'm wondering about using an embedded Apache DS with
interceptors simply forwarding each call to the corresponding
stand-alone LDAP server (which may also be an Apache DS). However let's
denote my component with C then we may end up with (ASCII art) the
following due to chaining:
C - C
| |
external LDAP external LDAP
Now each of these instances of C holding an embedded Apache DS which
simply forwards the messages to the external LDAP seem to result in a
huge overhead which may be unaffordable. I would prefer a better
solution but I have around five days to implement this functionality and
I do not see any chance to implement this stuff if there is no code
which nearly exactly fits my needs and that I can simply take and
re-use. Unfortunately arguing for a better technical solution or more
time is currently ignored. Hence recommendations for other approaches
currently are of no help (just in case someone wants to argue that
everything would be fine using JMS / protocol XYZ / ...).
Sebastian
Re: Writing / integrating LDAP server
Posted by Howard Chu <hy...@symas.com>.
Sebastian Oerding wrote:
> Hello,
>
> I have very specific requirements. We want to use a LDAP server for
> storing X.509 certificates. At runtime certificates are requested from
> the LDAP server. However there will be the case that a certificate is
> unknown by the server. In this case the certificate is requested from a
> 3rd pary system, stored into the LDAP server to avoid the fallback for
> future requests, and returned to the client originally requesting the
> certificate.
As an aside, we have an overlay for OpenLDAP that generates certificates on
demand, when a search request comes in to return a userCertificate.
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/