You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@directory.apache.org by Ezsra McDonald <ez...@gmail.com> on 2015/06/03 23:44:06 UTC
ApacheDS Multi Master config issues
Greetings,
I am new to ApacheDS. We are running v2.0 M19. I am having issues getting
my pair of LDAP instances working in Multi Master mode. If you can answer a
few questions please:
1. Is it a problem if both instances are listening on port 10389 on
different hosts (Node 1 & Node 2)? Do they have to use different ports?
2. Do both instances need to start with the same data? Can node1 contain a
imported LDIF and will Node 1 eventual synchronize to node 2?
CONFIGURATION:
In my current setup I have imported the LDIF to both nodes. I configured
Replication as follows using the Director Studio:
* PORT: Both nodes listen on port 10389.
* ID: consumer host name
* REP MODE: Refresh and persist
* REMOTE HOST: Consumer host name
* REM PORT: 10389
* BIND: uid=admin,ou=system
* BIND PW: NOT_SHARING :-P
* USE STARTTLS: enabled
* BASE DB: dc=www,dc=somewhere,dc=com
* FILTER: (objectClass=*)
* SCOPE: SUBTREE
* ATTRIBUTES: All
SYMPTOMS:
When I setup the Replication and restart the instances I am able to login
to node 2 but node 1 won't allow me to connect. If I stop both nodes and
start only node 1 I can login to node1. By login I am using Studio to login
as uid=admin,ou=system.
If I run a netstat I can see a connection between each node. One from Node
1 to Node 2 and one from node 2 to node 1.
If I were to enable some debugging what particular logging should I enable?
-Ez
Re: ApacheDS Multi Master config issues
Posted by Kiran Ayyagari <ka...@apache.org>.
On Sat, Jun 13, 2015 at 3:58 PM, Emmanuel Lécharny <el...@gmail.com>
wrote:
> Le 13/06/15 08:41, Pontus Freyhult a écrit :
> > Hej,
> >
> >>>> 2. Do both instances need to start with the same data? Can node1
> contain
> >> a
> >>>> imported LDIF and will Node 1 eventual synchronize to node 2?
> >>>>
> >>> yes, they eventually synchronize
> >> Okay, I think I know what is happening. The node with the LDIF loaded
> in it
> >> is logging the following error:
> >>
> >> INFO | jvm 1 | 2015/06/12 14:50:35 | Exception in thread
> >> "pool-7-thread-1" java.lang.OutOfMemoryError: Java heap space
> >> INFO | jvm 1 | 2015/06/12 14:50:35 | at
> >> java.nio.HeapByteBuffer.<init>(HeapByteBuffer.java:57)
> >> INFO | jvm 1 | 2015/06/12 14:50:35 | at
> >> java.nio.ByteBuffer.allocate(ByteBuffer.java:335)
> >>
> >> This is even after making these config changes:
> >>
> >> # Initial Java Heap Size (in MB)
> >> wrapper.java.initmemory=2048
> >>
> >> # Maximum Java Heap Size (in MB)
> >> wrapper.java.maxmemory=2048
> >>
> >>
> >> Any ideas?
> > [..]
> >
> > I believe we are seeing the same issue. I've done some troubleshooting
> > but not enough that I wanted to raise it just yet, but from what we've
> > been seeing my guess is that we're hitting on some kind of leak when
> > doing replication (that we're not seeing otherwise). Raising the
> > memory available for the java environment only delays the "lockup" of
> > the first server, it does not avoid it - I believe we've tested with
> > giving it up to at least 16 Gbytes of RAM.
>
> Ok, seems quite clear. At this point, we would need some memory traces
> to be able to see what's going on.
>
> yes, a trace would be really helpful
> Can someone run |jmap -dump:format=b,file=cheap.bin <pid> on a server
> where this error appears ? (<pid> is the process' pid) and attach the
> result to a JIRA ?
>
my experience with jmap was bitter, it killed the process everytime I tried,
I suggest to use gdb instead to take the memory dump and then use jmap to
convert it to hprof format , please see this[1] page if you need some
pointers
[1] http://blogs.atlassian.com/2013/03/so-you-want-your-jvms-heap/
>
> Many thanks !
> |
>
--
Kiran Ayyagari
http://keydap.com
Re: ApacheDS Multi Master config issues
Posted by Emmanuel Lécharny <el...@gmail.com>.
Le 13/06/15 08:41, Pontus Freyhult a écrit :
> Hej,
>
>>>> 2. Do both instances need to start with the same data? Can node1 contain
>> a
>>>> imported LDIF and will Node 1 eventual synchronize to node 2?
>>>>
>>> yes, they eventually synchronize
>> Okay, I think I know what is happening. The node with the LDIF loaded in it
>> is logging the following error:
>>
>> INFO | jvm 1 | 2015/06/12 14:50:35 | Exception in thread
>> "pool-7-thread-1" java.lang.OutOfMemoryError: Java heap space
>> INFO | jvm 1 | 2015/06/12 14:50:35 | at
>> java.nio.HeapByteBuffer.<init>(HeapByteBuffer.java:57)
>> INFO | jvm 1 | 2015/06/12 14:50:35 | at
>> java.nio.ByteBuffer.allocate(ByteBuffer.java:335)
>>
>> This is even after making these config changes:
>>
>> # Initial Java Heap Size (in MB)
>> wrapper.java.initmemory=2048
>>
>> # Maximum Java Heap Size (in MB)
>> wrapper.java.maxmemory=2048
>>
>>
>> Any ideas?
> [..]
>
> I believe we are seeing the same issue. I've done some troubleshooting
> but not enough that I wanted to raise it just yet, but from what we've
> been seeing my guess is that we're hitting on some kind of leak when
> doing replication (that we're not seeing otherwise). Raising the
> memory available for the java environment only delays the "lockup" of
> the first server, it does not avoid it - I believe we've tested with
> giving it up to at least 16 Gbytes of RAM.
Ok, seems quite clear. At this point, we would need some memory traces
to be able to see what's going on.
Can someone run |jmap -dump:format=b,file=cheap.bin <pid> on a server
where this error appears ? (<pid> is the process' pid) and attach the
result to a JIRA ?
Many thanks !
|
Re: ApacheDS Multi Master config issues
Posted by Pontus Freyhult <po...@soua.net>.
Hej,
>>> 2. Do both instances need to start with the same data? Can node1 contain
> a
>>> imported LDIF and will Node 1 eventual synchronize to node 2?
>>>
>>yes, they eventually synchronize
>
> Okay, I think I know what is happening. The node with the LDIF loaded in it
> is logging the following error:
>
> INFO | jvm 1 | 2015/06/12 14:50:35 | Exception in thread
> "pool-7-thread-1" java.lang.OutOfMemoryError: Java heap space
> INFO | jvm 1 | 2015/06/12 14:50:35 | at
> java.nio.HeapByteBuffer.<init>(HeapByteBuffer.java:57)
> INFO | jvm 1 | 2015/06/12 14:50:35 | at
> java.nio.ByteBuffer.allocate(ByteBuffer.java:335)
>
> This is even after making these config changes:
>
> # Initial Java Heap Size (in MB)
> wrapper.java.initmemory=2048
>
> # Maximum Java Heap Size (in MB)
> wrapper.java.maxmemory=2048
>
>
> Any ideas?
[..]
I believe we are seeing the same issue. I've done some troubleshooting
but not enough that I wanted to raise it just yet, but from what we've
been seeing my guess is that we're hitting on some kind of leak when
doing replication (that we're not seeing otherwise). Raising the
memory available for the java environment only delays the "lockup" of
the first server, it does not avoid it - I believe we've tested with
giving it up to at least 16 Gbytes of RAM.
(On a side note; we've also seen issues with wrapper not always
passing specified arguments along, so you do need to verify that the
java process for the ApacheDS actually is actually passed the
arguments you want.)
cheers, Pontus
Re: ApacheDS Multi Master config issues
Posted by Kiran Ayyagari <ka...@apache.org>.
On Sat, Jun 13, 2015 at 5:13 AM, Ezsra McDonald <ez...@gmail.com>
wrote:
> >> 2. Do both instances need to start with the same data? Can node1 contain
> a
> >> imported LDIF and will Node 1 eventual synchronize to node 2?
> >>
> >yes, they eventually synchronize
>
> Okay, I think I know what is happening. The node with the LDIF loaded in it
> is logging the following error:
>
> INFO | jvm 1 | 2015/06/12 14:50:35 | Exception in thread
> "pool-7-thread-1" java.lang.OutOfMemoryError: Java heap space
> INFO | jvm 1 | 2015/06/12 14:50:35 | at
> java.nio.HeapByteBuffer.<init>(HeapByteBuffer.java:57)
> INFO | jvm 1 | 2015/06/12 14:50:35 | at
> java.nio.ByteBuffer.allocate(ByteBuffer.java:335)
>
> are there more stacktraces in the log, can you check apacheds.log
> This is even after making these config changes:
>
> # Initial Java Heap Size (in MB)
> wrapper.java.initmemory=2048
>
> # Maximum Java Heap Size (in MB)
> wrapper.java.maxmemory=2048
>
did you make these changes at the instance level config file (the one under
instances/default)?
or at the server installation root level?
>
>
> Any ideas?
>
>
> On Thu, Jun 4, 2015 at 1:42 AM, Kiran Ayyagari <ka...@apache.org>
> wrote:
>
> > On Thu, Jun 4, 2015 at 5:44 AM, Ezsra McDonald <ezsra.mcdonald@gmail.com
> >
> > wrote:
> >
> > > Greetings,
> > >
> > > I am new to ApacheDS. We are running v2.0 M19. I am having issues
> getting
> > > my pair of LDAP instances working in Multi Master mode. If you can
> > answer a
> > > few questions please:
> > >
> > > 1. Is it a problem if both instances are listening on port 10389 on
> > > different hosts (Node 1 & Node 2)? Do they have to use different ports?
> > >
> > > no, not needed
> >
> > > 2. Do both instances need to start with the same data? Can node1
> contain
> > a
> > > imported LDIF and will Node 1 eventual synchronize to node 2?
> > >
> > yes, they eventually synchronize
> >
> > >
> > > CONFIGURATION:
> > > In my current setup I have imported the LDIF to both nodes. I
> configured
> > > Replication as follows using the Director Studio:
> > >
> > > * PORT: Both nodes listen on port 10389.
> > > * ID: consumer host name
> > > * REP MODE: Refresh and persist
> > > * REMOTE HOST: Consumer host name
> > > * REM PORT: 10389
> > > * BIND: uid=admin,ou=system
> > > * BIND PW: NOT_SHARING :-P
> > > * USE STARTTLS: enabled
> > > * BASE DB: dc=www,dc=somewhere,dc=com
> > > * FILTER: (objectClass=*)
> > > * SCOPE: SUBTREE
> > > * ATTRIBUTES: All
> > >
> > >
> > > SYMPTOMS:
> > > When I setup the Replication and restart the instances I am able to
> login
> > > to node 2 but node 1 won't allow me to connect. If I stop both nodes
> and
> > > start only node 1 I can login to node1. By login I am using Studio to
> > login
> > > as uid=admin,ou=system.
> > >
> > what error are you getting, can you post any errors from the server log
> >
> > >
> > > If I run a netstat I can see a connection between each node. One from
> > Node
> > > 1 to Node 2 and one from node 2 to node 1.
> > >
> > > If I were to enable some debugging what particular logging should I
> > enable?
> > >
> > you can use this config http://pastebin.com/5U7NuRir
> >
> > >
> > > -Ez
> > >
> >
> >
> >
> > --
> > Kiran Ayyagari
> > http://keydap.com
> >
>
--
Kiran Ayyagari
http://keydap.com
Re: ApacheDS Multi Master config issues
Posted by Ezsra McDonald <ez...@gmail.com>.
>> 2. Do both instances need to start with the same data? Can node1 contain
a
>> imported LDIF and will Node 1 eventual synchronize to node 2?
>>
>yes, they eventually synchronize
Okay, I think I know what is happening. The node with the LDIF loaded in it
is logging the following error:
INFO | jvm 1 | 2015/06/12 14:50:35 | Exception in thread
"pool-7-thread-1" java.lang.OutOfMemoryError: Java heap space
INFO | jvm 1 | 2015/06/12 14:50:35 | at
java.nio.HeapByteBuffer.<init>(HeapByteBuffer.java:57)
INFO | jvm 1 | 2015/06/12 14:50:35 | at
java.nio.ByteBuffer.allocate(ByteBuffer.java:335)
This is even after making these config changes:
# Initial Java Heap Size (in MB)
wrapper.java.initmemory=2048
# Maximum Java Heap Size (in MB)
wrapper.java.maxmemory=2048
Any ideas?
On Thu, Jun 4, 2015 at 1:42 AM, Kiran Ayyagari <ka...@apache.org> wrote:
> On Thu, Jun 4, 2015 at 5:44 AM, Ezsra McDonald <ez...@gmail.com>
> wrote:
>
> > Greetings,
> >
> > I am new to ApacheDS. We are running v2.0 M19. I am having issues getting
> > my pair of LDAP instances working in Multi Master mode. If you can
> answer a
> > few questions please:
> >
> > 1. Is it a problem if both instances are listening on port 10389 on
> > different hosts (Node 1 & Node 2)? Do they have to use different ports?
> >
> > no, not needed
>
> > 2. Do both instances need to start with the same data? Can node1 contain
> a
> > imported LDIF and will Node 1 eventual synchronize to node 2?
> >
> yes, they eventually synchronize
>
> >
> > CONFIGURATION:
> > In my current setup I have imported the LDIF to both nodes. I configured
> > Replication as follows using the Director Studio:
> >
> > * PORT: Both nodes listen on port 10389.
> > * ID: consumer host name
> > * REP MODE: Refresh and persist
> > * REMOTE HOST: Consumer host name
> > * REM PORT: 10389
> > * BIND: uid=admin,ou=system
> > * BIND PW: NOT_SHARING :-P
> > * USE STARTTLS: enabled
> > * BASE DB: dc=www,dc=somewhere,dc=com
> > * FILTER: (objectClass=*)
> > * SCOPE: SUBTREE
> > * ATTRIBUTES: All
> >
> >
> > SYMPTOMS:
> > When I setup the Replication and restart the instances I am able to login
> > to node 2 but node 1 won't allow me to connect. If I stop both nodes and
> > start only node 1 I can login to node1. By login I am using Studio to
> login
> > as uid=admin,ou=system.
> >
> what error are you getting, can you post any errors from the server log
>
> >
> > If I run a netstat I can see a connection between each node. One from
> Node
> > 1 to Node 2 and one from node 2 to node 1.
> >
> > If I were to enable some debugging what particular logging should I
> enable?
> >
> you can use this config http://pastebin.com/5U7NuRir
>
> >
> > -Ez
> >
>
>
>
> --
> Kiran Ayyagari
> http://keydap.com
>
Re: ApacheDS Multi Master config issues
Posted by Kiran Ayyagari <ka...@apache.org>.
On Thu, Jun 4, 2015 at 5:44 AM, Ezsra McDonald <ez...@gmail.com>
wrote:
> Greetings,
>
> I am new to ApacheDS. We are running v2.0 M19. I am having issues getting
> my pair of LDAP instances working in Multi Master mode. If you can answer a
> few questions please:
>
> 1. Is it a problem if both instances are listening on port 10389 on
> different hosts (Node 1 & Node 2)? Do they have to use different ports?
>
> no, not needed
> 2. Do both instances need to start with the same data? Can node1 contain a
> imported LDIF and will Node 1 eventual synchronize to node 2?
>
yes, they eventually synchronize
>
> CONFIGURATION:
> In my current setup I have imported the LDIF to both nodes. I configured
> Replication as follows using the Director Studio:
>
> * PORT: Both nodes listen on port 10389.
> * ID: consumer host name
> * REP MODE: Refresh and persist
> * REMOTE HOST: Consumer host name
> * REM PORT: 10389
> * BIND: uid=admin,ou=system
> * BIND PW: NOT_SHARING :-P
> * USE STARTTLS: enabled
> * BASE DB: dc=www,dc=somewhere,dc=com
> * FILTER: (objectClass=*)
> * SCOPE: SUBTREE
> * ATTRIBUTES: All
>
>
> SYMPTOMS:
> When I setup the Replication and restart the instances I am able to login
> to node 2 but node 1 won't allow me to connect. If I stop both nodes and
> start only node 1 I can login to node1. By login I am using Studio to login
> as uid=admin,ou=system.
>
what error are you getting, can you post any errors from the server log
>
> If I run a netstat I can see a connection between each node. One from Node
> 1 to Node 2 and one from node 2 to node 1.
>
> If I were to enable some debugging what particular logging should I enable?
>
you can use this config http://pastebin.com/5U7NuRir
>
> -Ez
>
--
Kiran Ayyagari
http://keydap.com