You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@directory.apache.org by Brett Gilmer <br...@tklabs.com> on 2015/02/03 18:20:47 UTC
Replication between ApacheDS and openLDAP
Apache DS Team -
I am trying to set up replication between an openLDAP and an ApacheDS. The
ApacheDS is the provider.
What I am seeing is that openLDAP, when it tries to sync (consumer), will
often send a cookie that ApacheDS doesn't like. Once this happens, there is
no recovery.
In the example below, openLDAP is sending a cookie that is too long (looks
like 2 timestamps with a semicolon).
I also have issues where the clearing of the cookie on openLDAP sends a
cookie with an RID but no timestamp, while Apache expects a totally blank
cookie.
Has anyone else seen this? I am stuck with openLDAP on the consumer side
(it is embedded in an internet appliance).
Thanks
from LdapSession :
<0.9.2342.19200300.100.1.1=admin,2.5.4.11=system,/50.73.4.13:3960>
[14:29:37] DEBUG [org.apache.directory.server.PROVIDER_LOG] - Received a
replication request MessageType : SEARCH_REQUEST
Message ID : 1251
SearchRequest
baseDn : 'dc=example,dc=com'
filter : '(objectClass=*)'
scope : whole subtree
typesOnly : false
Size Limit : no limit
Time Limit : no limit
Deref Aliases : never Deref Aliases
attributes : '*', '+'
org.apache.directory.api.ldap.model.message.SearchRequestImpl@3f673bcb
SyncRequestValue control :
oid : 1.3.6.1.4.1.4203.1.9.1.1
critical : false
mode : 'REFRESH_AND_PERSIST'
cookie : '0x72 0x69 0x64 0x3D 0x33 0x30 0x30 0x2C 0x63
0x73 0x6E 0x3D 0x32 0x30 0x31 0x34 0x31 0x32 0x32 0x39 0x31 0x36 0x30 0x34
0x34 0x34 0
x2E 0x38 0x34 0x31 0x32 0x30 0x34 0x5A 0x23 0x30 0x30 0x30 0x30 0x30 0x30
0x23 0x30 0x30 0x30 0x23 0x30 0x30 0x30 0x30 0x30 0x30 0x3B 0x32 0x30 0x31
0x35 0x30 0
x31 0x31 0x37 0x31 0x38 0x33 0x37 0x32 0x36 0x2E 0x38 0x31 0x32 0x30 0x30
0x30 0x5A 0x23 0x30 0x30 0x30 0x30 0x30 0x30 0x23 0x30 0x30 0x31 0x23 0x30
0x30 0x30 0
x30 0x30 0x30 '
reloadHint : 'true'
ManageDsaITImpl Control
Type OID : '2.16.840.1.113730.3.4.2'
Criticality : 'true'
'
with a cookie
'rid=300,csn=20141229160444.841204Z#000000#000#000000;20150117183726.812000Z
#000000#001#000000'
[14:29:37] ERROR [org.apache.directory.server.PROVIDER_LOG] - received an
invalid cookie
rid=300,csn=20141229160444.841204Z#000000#000#000000;20150117183726.812
000Z#000000#001#000000 from the consumer with session LdapSession :
<0.9.2342.19200300.100.1.1=admin,2.5.4.11=system,/50.73.4.13:3960>
[14:29:37] DEBUG [org.apache.directory.server.PROVIDER_LOG] - Received a
Syncrepl request : MessageType : SEARCH_REQUEST
1,1 Top
Re: Replication between ApacheDS and openLDAP
Posted by Emmanuel Lécharny <el...@gmail.com>.
Le 05/02/15 16:11, Brett Gilmer a écrit :
> The OpenLDAP is 2.4.39, which is being used because it is running on a Synology DSM and this is what it installs.
>
> It was a challenge to get the sync to work at all, since it always sends at least "rid=300" for a starter-cookie instead of a blank one, so I did some tracing through the code and set up the provider side manually, which worked for about a day but then openLDAP started sending this double-csn-cookie.
Sorry for the delay...
OpenLDAP will send one CSN per server ID, something which was not
anticipated. This need to be fixed.
Can you fill a JIRA for this one ?
RE: Replication between ApacheDS and openLDAP
Posted by Brett Gilmer <br...@tklabs.com>.
The OpenLDAP is 2.4.39, which is being used because it is running on a Synology DSM and this is what it installs.
It was a challenge to get the sync to work at all, since it always sends at least "rid=300" for a starter-cookie instead of a blank one, so I did some tracing through the code and set up the provider side manually, which worked for about a day but then openLDAP started sending this double-csn-cookie.
-----Original Message-----
From: Emmanuel Lécharny [mailto:elecharny@gmail.com]
Sent: Wednesday, February 4, 2015 11:07 PM
To: users@directory.apache.org
Subject: Re: Replication between ApacheDS and openLDAP
Le 04/02/15 14:34, Brett Gilmer a écrit :
> Could this be a configuration issue on either side, or is the sync between ApacheDS and OpenLDAP not supported?
We aren't expecting OpenLDAP to send 2 CSN in the cookie, and we have to understand why it's what we get from OpenLDAP.
In any case, the cookie is opaque information not expected to be documented except by the implementers. However, we do expect that the cookie we receive from OpenLDAP is something we can deal with.
FTR, which version of OpenLDAP are you using ?
Re: Replication between ApacheDS and openLDAP
Posted by Emmanuel Lécharny <el...@gmail.com>.
Le 04/02/15 14:34, Brett Gilmer a écrit :
> Could this be a configuration issue on either side, or is the sync between ApacheDS and OpenLDAP not supported?
We aren't expecting OpenLDAP to send 2 CSN in the cookie, and we have to
understand why it's what we get from OpenLDAP.
In any case, the cookie is opaque information not expected to be
documented except by the implementers. However, we do expect that the
cookie we receive from OpenLDAP is something we can deal with.
FTR, which version of OpenLDAP are you using ?
RE: Replication between ApacheDS and openLDAP
Posted by Brett Gilmer <br...@tklabs.com>.
Could this be a configuration issue on either side, or is the sync between ApacheDS and OpenLDAP not supported? The issue, as presented, looks surmountable; however I don't want to go down a road that is known to be full of landmines.
-----Original Message-----
From: Kiran Ayyagari [mailto:kayyagari@apache.org]
Sent: Tuesday, February 03, 2015 5:59 PM
To: users@directory.apache.org
Subject: Re: Replication between ApacheDS and openLDAP
On Wed, Feb 4, 2015 at 1:20 AM, Brett Gilmer <br...@tklabs.com>
wrote:
> Apache DS Team -
>
>
>
>
>
> I am trying to set up replication between an openLDAP and an ApacheDS.
> The ApacheDS is the provider.
>
>
>
> What I am seeing is that openLDAP, when it tries to sync (consumer),
> will often send a cookie that ApacheDS doesn't like. Once this
> happens, there is no recovery.
>
>
>
> In the example below, openLDAP is sending a cookie that is too long
> (looks like 2 timestamps with a semicolon).
>
> I also have issues where the clearing of the cookie on openLDAP sends
> a cookie with an RID but no timestamp, while Apache expects a totally
> blank cookie.
>
>
>
> Has anyone else seen this? I am stuck with openLDAP on the consumer
> side (it is embedded in an internet appliance).
>
>
>
>
>
> Thanks
>
>
>
>
>
>
>
> from LdapSession :
> <0.9.2342.19200300.100.1.1=admin,2.5.4.11=system,/50.73.4.13:3960>
>
> [14:29:37] DEBUG [org.apache.directory.server.PROVIDER_LOG] - Received
> a replication request MessageType : SEARCH_REQUEST
>
> Message ID : 1251
>
> SearchRequest
>
> baseDn : 'dc=example,dc=com'
>
> filter : '(objectClass=*)'
>
> scope : whole subtree
>
> typesOnly : false
>
> Size Limit : no limit
>
> Time Limit : no limit
>
> Deref Aliases : never Deref Aliases
>
> attributes : '*', '+'
>
> org.apache.directory.api.ldap.model.message.SearchRequestImpl@3f673bcb
> SyncRequestValue control :
>
> oid : 1.3.6.1.4.1.4203.1.9.1.1
>
> critical : false
>
> mode : 'REFRESH_AND_PERSIST'
>
> cookie : '0x72 0x69 0x64 0x3D 0x33 0x30 0x30 0x2C 0x63
> 0x73 0x6E 0x3D 0x32 0x30 0x31 0x34 0x31 0x32 0x32 0x39 0x31 0x36 0x30
> 0x34
> 0x34 0x34 0
>
> x2E 0x38 0x34 0x31 0x32 0x30 0x34 0x5A 0x23 0x30 0x30 0x30 0x30 0x30
> 0x30
> 0x23 0x30 0x30 0x30 0x23 0x30 0x30 0x30 0x30 0x30 0x30 0x3B 0x32 0x30
> 0x31
> 0x35 0x30 0
>
> x31 0x31 0x37 0x31 0x38 0x33 0x37 0x32 0x36 0x2E 0x38 0x31 0x32 0x30
> 0x30
> 0x30 0x5A 0x23 0x30 0x30 0x30 0x30 0x30 0x30 0x23 0x30 0x30 0x31 0x23
> 0x30
> 0x30 0x30 0
>
> x30 0x30 0x30 '
>
> reloadHint : 'true'
>
> ManageDsaITImpl Control
>
> Type OID : '2.16.840.1.113730.3.4.2'
>
> Criticality : 'true'
>
> '
>
> with a cookie
>
> 'rid=300,csn=20141229160444.841204Z#000000#000#000000;20150117183726.8
> 12000Z
> #000000#001#000000'
>
> [14:29:37] ERROR [org.apache.directory.server.PROVIDER_LOG] - received
> an invalid cookie
> rid=300,csn=20141229160444.841204Z#000000#000#000000;20150117183726.81
> 2
>
> 000Z#000000#001#000000 from the consumer with session LdapSession :
>
here there are two CSNs in the cookie, ApacheDS is treating the entire string after RID as a CSN, which is leading to this issue.
the spec[1] is not really clear about the format of cookie, I wish we could amend this.
[1] http://tools.ietf.org/html/rfc4533#section-2.1.2
> <0.9.2342.19200300.100.1.1=admin,2.5.4.11=system,/50.73.4.13:3960>
>
> [14:29:37] DEBUG [org.apache.directory.server.PROVIDER_LOG] - Received
> a Syncrepl request : MessageType : SEARCH_REQUEST
>
> 1,1 Top
>
>
--
Kiran Ayyagari
http://keydap.com
Re: Replication between ApacheDS and openLDAP
Posted by Kiran Ayyagari <ka...@apache.org>.
On Wed, Feb 4, 2015 at 1:20 AM, Brett Gilmer <br...@tklabs.com>
wrote:
> Apache DS Team -
>
>
>
>
>
> I am trying to set up replication between an openLDAP and an ApacheDS. The
> ApacheDS is the provider.
>
>
>
> What I am seeing is that openLDAP, when it tries to sync (consumer), will
> often send a cookie that ApacheDS doesn't like. Once this happens, there
> is
> no recovery.
>
>
>
> In the example below, openLDAP is sending a cookie that is too long (looks
> like 2 timestamps with a semicolon).
>
> I also have issues where the clearing of the cookie on openLDAP sends a
> cookie with an RID but no timestamp, while Apache expects a totally blank
> cookie.
>
>
>
> Has anyone else seen this? I am stuck with openLDAP on the consumer side
> (it is embedded in an internet appliance).
>
>
>
>
>
> Thanks
>
>
>
>
>
>
>
> from LdapSession :
> <0.9.2342.19200300.100.1.1=admin,2.5.4.11=system,/50.73.4.13:3960>
>
> [14:29:37] DEBUG [org.apache.directory.server.PROVIDER_LOG] - Received a
> replication request MessageType : SEARCH_REQUEST
>
> Message ID : 1251
>
> SearchRequest
>
> baseDn : 'dc=example,dc=com'
>
> filter : '(objectClass=*)'
>
> scope : whole subtree
>
> typesOnly : false
>
> Size Limit : no limit
>
> Time Limit : no limit
>
> Deref Aliases : never Deref Aliases
>
> attributes : '*', '+'
>
> org.apache.directory.api.ldap.model.message.SearchRequestImpl@3f673bcb
> SyncRequestValue control :
>
> oid : 1.3.6.1.4.1.4203.1.9.1.1
>
> critical : false
>
> mode : 'REFRESH_AND_PERSIST'
>
> cookie : '0x72 0x69 0x64 0x3D 0x33 0x30 0x30 0x2C 0x63
> 0x73 0x6E 0x3D 0x32 0x30 0x31 0x34 0x31 0x32 0x32 0x39 0x31 0x36 0x30 0x34
> 0x34 0x34 0
>
> x2E 0x38 0x34 0x31 0x32 0x30 0x34 0x5A 0x23 0x30 0x30 0x30 0x30 0x30 0x30
> 0x23 0x30 0x30 0x30 0x23 0x30 0x30 0x30 0x30 0x30 0x30 0x3B 0x32 0x30 0x31
> 0x35 0x30 0
>
> x31 0x31 0x37 0x31 0x38 0x33 0x37 0x32 0x36 0x2E 0x38 0x31 0x32 0x30 0x30
> 0x30 0x5A 0x23 0x30 0x30 0x30 0x30 0x30 0x30 0x23 0x30 0x30 0x31 0x23 0x30
> 0x30 0x30 0
>
> x30 0x30 0x30 '
>
> reloadHint : 'true'
>
> ManageDsaITImpl Control
>
> Type OID : '2.16.840.1.113730.3.4.2'
>
> Criticality : 'true'
>
> '
>
> with a cookie
>
> 'rid=300,csn=20141229160444.841204Z#000000#000#000000;20150117183726.812000Z
> #000000#001#000000'
>
> [14:29:37] ERROR [org.apache.directory.server.PROVIDER_LOG] - received an
> invalid cookie
> rid=300,csn=20141229160444.841204Z#000000#000#000000;20150117183726.812
>
> 000Z#000000#001#000000 from the consumer with session LdapSession :
>
here there are two CSNs in the cookie, ApacheDS is treating the entire
string after RID as a CSN,
which is leading to this issue.
the spec[1] is not really clear about the format of cookie, I wish we could
amend this.
[1] http://tools.ietf.org/html/rfc4533#section-2.1.2
> <0.9.2342.19200300.100.1.1=admin,2.5.4.11=system,/50.73.4.13:3960>
>
> [14:29:37] DEBUG [org.apache.directory.server.PROVIDER_LOG] - Received a
> Syncrepl request : MessageType : SEARCH_REQUEST
>
> 1,1 Top
>
>
--
Kiran Ayyagari
http://keydap.com