You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@directory.apache.org by Da...@bosch-si.com on 2012/10/17 13:40:47 UTC
ApacheDS crashes on Startup after Backup-Restoration
Greetings,
We are using ApacheDS as LDAP provider for one of our Solutions (for user authentication).
Recently one of our ApacheDS Servers experienced a database corruption - we are not sure why.
The ApacheDS server booted up correctly, but when some client tried to retrieve a list of all users (for instance the Apache Directory Studio), the Server throws a exception and closes the socket to the client:
[05:40:05] WARN [org.apache.directory.server.ldap.LdapProtocolHandler] - Unexpected exception forcing session to close: sending disconnect notice to client.
java.lang.Error: ERR_546 CRITICAL: page header magic for block 64 not OK 0
at jdbm.recman.PageHeader.<init>(PageHeader.java:95)
at jdbm.recman.PageHeader.getView(PageHeader.java:124)
at jdbm.recman.PageManager.getNext(PageManager.java:234)
at jdbm.recman.PageCursor.next(PageCursor.java:104)
at jdbm.recman.PhysicalRowIdManager.fetch(PhysicalRowIdManager.java:158)
at jdbm.recman.BaseRecordManager.fetch(BaseRecordManager.java:324)
at jdbm.recman.CacheRecordManager.fetch(CacheRecordManager.java:262)
at jdbm.btree.BPage.loadBPage(BPage.java:899)
at jdbm.btree.BPage.childBPage(BPage.java:890)
at jdbm.btree.BPage.find(BPage.java:284)
at jdbm.btree.BTree.find(BTree.java:408)
at org.apache.directory.server.core.partition.impl.btree.jdbm.JdbmTable.get(JdbmTable.java:395)
at org.apache.directory.server.core.partition.impl.btree.jdbm.JdbmMasterTable.get(JdbmMasterTable.java:155)
at org.apache.directory.server.core.partition.impl.btree.jdbm.JdbmStore.lookup(JdbmStore.java:1332)
at org.apache.directory.server.core.partition.impl.btree.jdbm.JdbmStore.lookup(JdbmStore.java:70)
at org.apache.directory.server.xdbm.AbstractXdbmPartition.lookup(AbstractXdbmPartition.java:327)
at org.apache.directory.server.core.partition.impl.btree.ServerEntryCursorAdaptor.get(ServerEntryCursorAdaptor.java:139)
at org.apache.directory.server.core.partition.impl.btree.ServerEntryCursorAdaptor.get(ServerEntryCursorAdaptor.java:39)
at org.apache.directory.server.core.filtering.BaseEntryFilteringCursor.next(BaseEntryFilteringCursor.java:503)
at org.apache.directory.server.ldap.handlers.SearchHandler.readResults(SearchHandler.java:314)
at org.apache.directory.server.ldap.handlers.SearchHandler.doSimpleSearch(SearchHandler.java:749)
at org.apache.directory.server.ldap.handlers.SearchHandler.handleIgnoringReferrals(SearchHandler.java:978)
at org.apache.directory.server.ldap.handlers.SearchHandler.handleWithReferrals(SearchHandler.java:1054)
at org.apache.directory.server.ldap.handlers.SearchHandler.handleWithReferrals(SearchHandler.java:78)
at org.apache.directory.server.ldap.handlers.ReferralAwareRequestHandler.handle(ReferralAwareRequestHandler.java:94)
at org.apache.directory.server.ldap.handlers.ReferralAwareRequestHandler.handle(ReferralAwareRequestHandler.java:57)
at org.apache.directory.server.ldap.handlers.LdapRequestHandler.handleMessage(LdapRequestHandler.java:208)
at org.apache.directory.server.ldap.handlers.LdapRequestHandler.handleMessage(LdapRequestHandler.java:58)
at org.apache.mina.handler.demux.DemuxingIoHandler.messageReceived(DemuxingIoHandler.java:232)
at org.apache.directory.server.ldap.LdapProtocolHandler.messageReceived(LdapProtocolHandler.java:193)
at org.apache.mina.core.filterchain.DefaultIoFilterChain$TailFilter.messageReceived(DefaultIoFilterChain.java:713)
at org.apache.mina.core.filterchain.DefaultIoFilterChain.callNextMessageReceived(DefaultIoFilterChain.java:434)
at org.apache.mina.core.filterchain.DefaultIoFilterChain.access$1200(DefaultIoFilterChain.java:46)
at org.apache.mina.core.filterchain.DefaultIoFilterChain$EntryImpl$1.messageReceived(DefaultIoFilterChain.java:793)
at org.apache.mina.core.filterchain.IoFilterEvent.fire(IoFilterEvent.java:71)
at org.apache.mina.core.session.IoEvent.run(IoEvent.java:63)
at org.apache.mina.filter.executor.UnorderedThreadPoolExecutor$Worker.runTask(UnorderedThreadPoolExecutor.java:480)
at org.apache.mina.filter.executor.UnorderedThreadPoolExecutor$Worker.run(UnorderedThreadPoolExecutor.java:434)
at java.lang.Thread.run(Thread.java:662)
[05:40:05] WARN [org.apache.directory.server.ldap.LdapProtocolHandler] - Unexpected exception forcing session to close: sending disconnect notice to client.
org.apache.mina.core.write.WriteToClosedSessionException
at org.apache.mina.core.polling.AbstractPollingIoProcessor.clearWriteRequestQueue(AbstractPollingIoProcessor.java:573)
at org.apache.mina.core.polling.AbstractPollingIoProcessor.removeNow(AbstractPollingIoProcessor.java:525)
at org.apache.mina.core.polling.AbstractPollingIoProcessor.removeSessions(AbstractPollingIoProcessor.java:497)
at org.apache.mina.core.polling.AbstractPollingIoProcessor.access$600(AbstractPollingIoProcessor.java:61)
at org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.run(AbstractPollingIoProcessor.java:974)
at org.apache.mina.util.NamePreservingRunnable.run(NamePreservingRunnable.java:64)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
We do not think that there is a way to repair this corruption, therefore we tried to restore a backup (we are using Tivoli for backup of our SAN storage).
We were able to determine the Point in Time when the corruption took place, as we use a oracle database job which fails in case the LDAP query fails.
After trying to restore to the point in time when the Database was still working we experienced a new Problem:
The ApacheDS Server fails to boot up - it crashes with the following Stack trace:
[11:27:58] WARN [org.apache.directory.shared.ldap.ldif.LdifReader] - No version information : assuming version: 1
[11:27:58] WARN [org.apache.directory.shared.ldap.ldif.LdifReader] - No version information : assuming version: 1
[11:27:58] WARN [org.apache.directory.shared.ldap.ldif.LdifReader] - No version information : assuming version: 1
[11:27:58] WARN [org.apache.directory.shared.ldap.ldif.LdifReader] - No version information : assuming version: 1
[11:27:58] WARN [org.apache.directory.shared.ldap.ldif.LdifReader] - No version information : assuming version: 1
[11:27:58] WARN [org.apache.directory.shared.ldap.ldif.LdifReader] - No version information : assuming version: 1
[11:27:59] ERROR [org.apache.directory.daemon.Bootstrapper] - Failed on null.init(InstallationLayout, String[])
java.lang.NullPointerException
at org.apache.directory.server.core.entry.ClonedServerEntry.<init>(ClonedServerEntry.java:67)
at org.apache.directory.server.xdbm.AbstractXdbmPartition.lookup(AbstractXdbmPartition.java:327)
at org.apache.directory.server.core.partition.impl.btree.ServerEntryCursorAdaptor.get(ServerEntryCursorAdaptor.java:139)
at org.apache.directory.server.core.partition.impl.btree.ServerEntryCursorAdaptor.get(ServerEntryCursorAdaptor.java:39)
at org.apache.directory.server.core.filtering.BaseEntryFilteringCursor.next(BaseEntryFilteringCursor.java:503)
at org.apache.directory.server.core.authz.GroupCache.initialize(GroupCache.java:147)
at org.apache.directory.server.core.authz.GroupCache.<init>(GroupCache.java:111)
at org.apache.directory.server.core.authz.AciAuthorizationInterceptor.init(AciAuthorizationInterceptor.java:202)
at org.apache.directory.server.core.interceptor.InterceptorChain.register0(InterceptorChain.java:442)
at org.apache.directory.server.core.interceptor.InterceptorChain.register(InterceptorChain.java:398)
at org.apache.directory.server.core.interceptor.InterceptorChain.init(InterceptorChain.java:258)
at org.apache.directory.server.core.DefaultDirectoryService.initialize(DefaultDirectoryService.java:1447)
at org.apache.directory.server.core.DefaultDirectoryService.startup(DefaultDirectoryService.java:907)
at org.apache.directory.server.configuration.ApacheDS.startup(ApacheDS.java:163)
at org.apache.directory.server.Service.initLdap(Service.java:136)
at org.apache.directory.server.Service.init(Service.java:77)
at org.apache.directory.daemon.Bootstrapper.callInit(Bootstrapper.java:154)
at org.apache.directory.daemon.TanukiBootstrapper.start(TanukiBootstrapper.java:54)
at org.tanukisoftware.wrapper.WrapperManager$12.run(WrapperManager.java:2788)
I failed to archive any information in the internet about this szenario.
Does someone know what the problem is?
May the Tivoli backup restore some inconsistent state?
Is there any chance to rescue the data of this LDAP server?
Good thing it was no Productive environment this time - only QA, but it would be great to solve this in order to know how to solve it the next time.. ;-) (if there is a way to solve it)
Some additional information about our Environment: We are using Red Head Enterprise Linux release 5.8 (x86_64) on the impacted server.
ApacheDS is running on the embedded Java Virtual Machine
Mit freundlichen Grüßen / Best regards
Daniel van Maren
Bosch Software Innovations GmbH
Stuttgarter Str. 130,
71332 Waiblingen
GERMANY
www.bosch-si.de
daniel.vanmaren@bosch-si.com<ma...@bosch-si.com>
Registered office: Immenstaad, Register court: Amtsgericht Ulm, HRB 631888;
Executives: Heinz Derenbach; Erica Fölsche, Thomas Cotic, Michael Hahn, Klaus Hüftle
AW: ApacheDS crashes on Startup after Backup-Restoration
Posted by Da...@bosch-si.com.
Hello,
Unfortunately it didn't work - I guess if there are no further suggestions we will have to discard the database (and our Oracle DB as well) Anyway - as said, it's "only" an QA environment.
At least now we know that we should change our backup concept for the LDAP servers.
Anyway - Thank you for your help - very much appreciated!
Mit freundlichen Grüßen / Best regards
Daniel van Maren
Bosch Software Innovations GmbH
Stuttgarter Str. 130,
71332 Waiblingen
GERMANY
www.bosch-si.de
daniel.vanmaren@bosch-si.com
Registered office: Immenstaad, Register court: Amtsgericht Ulm, HRB 631888;
Executives: Heinz Derenbach; Erica Fölsche, Thomas Cotic, Michael Hahn, Klaus Hüftle
-----Ursprüngliche Nachricht-----
Von: ayyagarikiran@gmail.com [mailto:ayyagarikiran@gmail.com] Im Auftrag von Kiran Ayyagari
Gesendet: Mittwoch, 17. Oktober 2012 15:00
An: users@directory.apache.org
Betreff: Re: ApacheDS crashes on Startup after Backup-Restoration
can you disable the AciAuthorizationInterceptor and restart the server. Can't guarantee that it will work but just a try. If it works I would suggest you to export all the entries to an LDIF file and load it into a new server instance.
It looks like the files were backed up while the server is running and this sort of backup won't work well (especially with a JDBM based
partition)
On Wed, Oct 17, 2012 at 5:51 PM, <Da...@bosch-si.com> wrote:
> Hello,
>
> We are using ApacheDS 1.5.7.
> Thank you for your help.
>
> Mit freundlichen Grüßen / Best regards
>
> Daniel van Maren
>
> Bosch Software Innovations GmbH
> Stuttgarter Str. 130,
> 71332 Waiblingen
> GERMANY
> www.bosch-si.de
>
> daniel.vanmaren@bosch-si.com
>
> Registered office: Immenstaad, Register court: Amtsgericht Ulm, HRB
> 631888;
> Executives: Heinz Derenbach; Erica Fölsche, Thomas Cotic, Michael
> Hahn, Klaus Hüftle
>
>
> -----Ursprüngliche Nachricht-----
> Von: ayyagarikiran@gmail.com [mailto:ayyagarikiran@gmail.com] Im
> Auftrag von Kiran Ayyagari
> Gesendet: Mittwoch, 17. Oktober 2012 14:16
> An: users@directory.apache.org
> Betreff: Re: ApacheDS crashes on Startup after Backup-Restoration
>
> which version of ApacheDS you are using?
>
> On Wed, Oct 17, 2012 at 5:10 PM, <Da...@bosch-si.com> wrote:
>> Greetings,
>>
>> We are using ApacheDS as LDAP provider for one of our Solutions (for user authentication).
>> Recently one of our ApacheDS Servers experienced a database corruption - we are not sure why.
>> The ApacheDS server booted up correctly, but when some client tried to retrieve a list of all users (for instance the Apache Directory Studio), the Server throws a exception and closes the socket to the client:
>>
>> [05:40:05] WARN [org.apache.directory.server.ldap.LdapProtocolHandler] - Unexpected exception forcing session to close: sending disconnect notice to client.
>> java.lang.Error: ERR_546 CRITICAL: page header magic for block 64 not OK 0
>> at jdbm.recman.PageHeader.<init>(PageHeader.java:95)
>> at jdbm.recman.PageHeader.getView(PageHeader.java:124)
>> at jdbm.recman.PageManager.getNext(PageManager.java:234)
>> at jdbm.recman.PageCursor.next(PageCursor.java:104)
>> at jdbm.recman.PhysicalRowIdManager.fetch(PhysicalRowIdManager.java:158)
>> at jdbm.recman.BaseRecordManager.fetch(BaseRecordManager.java:324)
>> at jdbm.recman.CacheRecordManager.fetch(CacheRecordManager.java:262)
>> at jdbm.btree.BPage.loadBPage(BPage.java:899)
>> at jdbm.btree.BPage.childBPage(BPage.java:890)
>> at jdbm.btree.BPage.find(BPage.java:284)
>> at jdbm.btree.BTree.find(BTree.java:408)
>> at org.apache.directory.server.core.partition.impl.btree.jdbm.JdbmTable.get(JdbmTable.java:395)
>> at org.apache.directory.server.core.partition.impl.btree.jdbm.JdbmMasterTable.get(JdbmMasterTable.java:155)
>> at org.apache.directory.server.core.partition.impl.btree.jdbm.JdbmStore.lookup(JdbmStore.java:1332)
>> at org.apache.directory.server.core.partition.impl.btree.jdbm.JdbmStore.lookup(JdbmStore.java:70)
>> at org.apache.directory.server.xdbm.AbstractXdbmPartition.lookup(AbstractXdbmPartition.java:327)
>> at org.apache.directory.server.core.partition.impl.btree.ServerEntryCursorAdaptor.get(ServerEntryCursorAdaptor.java:139)
>> at org.apache.directory.server.core.partition.impl.btree.ServerEntryCursorAdaptor.get(ServerEntryCursorAdaptor.java:39)
>> at org.apache.directory.server.core.filtering.BaseEntryFilteringCursor.next(BaseEntryFilteringCursor.java:503)
>> at org.apache.directory.server.ldap.handlers.SearchHandler.readResults(SearchHandler.java:314)
>> at org.apache.directory.server.ldap.handlers.SearchHandler.doSimpleSearch(SearchHandler.java:749)
>> at org.apache.directory.server.ldap.handlers.SearchHandler.handleIgnoringReferrals(SearchHandler.java:978)
>> at org.apache.directory.server.ldap.handlers.SearchHandler.handleWithReferrals(SearchHandler.java:1054)
>> at org.apache.directory.server.ldap.handlers.SearchHandler.handleWithReferrals(SearchHandler.java:78)
>> at org.apache.directory.server.ldap.handlers.ReferralAwareRequestHandler.handle(ReferralAwareRequestHandler.java:94)
>> at org.apache.directory.server.ldap.handlers.ReferralAwareRequestHandler.handle(ReferralAwareRequestHandler.java:57)
>> at org.apache.directory.server.ldap.handlers.LdapRequestHandler.handleMessage(LdapRequestHandler.java:208)
>> at org.apache.directory.server.ldap.handlers.LdapRequestHandler.handleMessage(LdapRequestHandler.java:58)
>> at org.apache.mina.handler.demux.DemuxingIoHandler.messageReceived(DemuxingIoHandler.java:232)
>> at org.apache.directory.server.ldap.LdapProtocolHandler.messageReceived(LdapProtocolHandler.java:193)
>> at org.apache.mina.core.filterchain.DefaultIoFilterChain$TailFilter.messageReceived(DefaultIoFilterChain.java:713)
>> at org.apache.mina.core.filterchain.DefaultIoFilterChain.callNextMessageReceived(DefaultIoFilterChain.java:434)
>> at org.apache.mina.core.filterchain.DefaultIoFilterChain.access$1200(DefaultIoFilterChain.java:46)
>> at org.apache.mina.core.filterchain.DefaultIoFilterChain$EntryImpl$1.messageReceived(DefaultIoFilterChain.java:793)
>> at org.apache.mina.core.filterchain.IoFilterEvent.fire(IoFilterEvent.java:71)
>> at org.apache.mina.core.session.IoEvent.run(IoEvent.java:63)
>> at org.apache.mina.filter.executor.UnorderedThreadPoolExecutor$Worker.runTask(UnorderedThreadPoolExecutor.java:480)
>> at org.apache.mina.filter.executor.UnorderedThreadPoolExecutor$Worker.run(UnorderedThreadPoolExecutor.java:434)
>> at java.lang.Thread.run(Thread.java:662)
>> [05:40:05] WARN [org.apache.directory.server.ldap.LdapProtocolHandler] - Unexpected exception forcing session to close: sending disconnect notice to client.
>> org.apache.mina.core.write.WriteToClosedSessionException
>> at org.apache.mina.core.polling.AbstractPollingIoProcessor.clearWriteRequestQueue(AbstractPollingIoProcessor.java:573)
>> at org.apache.mina.core.polling.AbstractPollingIoProcessor.removeNow(AbstractPollingIoProcessor.java:525)
>> at org.apache.mina.core.polling.AbstractPollingIoProcessor.removeSessions(AbstractPollingIoProcessor.java:497)
>> at org.apache.mina.core.polling.AbstractPollingIoProcessor.access$600(AbstractPollingIoProcessor.java:61)
>> at org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.run(AbstractPollingIoProcessor.java:974)
>> at org.apache.mina.util.NamePreservingRunnable.run(NamePreservingRunnable.java:64)
>> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>> at java.lang.Thread.run(Thread.java:662)
>>
>> We do not think that there is a way to repair this corruption, therefore we tried to restore a backup (we are using Tivoli for backup of our SAN storage).
>> We were able to determine the Point in Time when the corruption took place, as we use a oracle database job which fails in case the LDAP query fails.
>> After trying to restore to the point in time when the Database was still working we experienced a new Problem:
>> The ApacheDS Server fails to boot up - it crashes with the following Stack trace:
>>
>> [11:27:58] WARN [org.apache.directory.shared.ldap.ldif.LdifReader] -
>> No version information : assuming version: 1 [11:27:58] WARN
>> [org.apache.directory.shared.ldap.ldif.LdifReader] - No version
>> information : assuming version: 1 [11:27:58] WARN
>> [org.apache.directory.shared.ldap.ldif.LdifReader] - No version
>> information : assuming version: 1 [11:27:58] WARN
>> [org.apache.directory.shared.ldap.ldif.LdifReader] - No version
>> information : assuming version: 1 [11:27:58] WARN [org.apache.directory.shared.ldap.ldif.LdifReader] - No version information : assuming version: 1 [11:27:58] WARN [org.apache.directory.shared.ldap.ldif.LdifReader] - No version information : assuming version: 1 [11:27:59] ERROR [org.apache.directory.daemon.Bootstrapper] - Failed on null.init(InstallationLayout, String[]) java.lang.NullPointerException
>> at org.apache.directory.server.core.entry.ClonedServerEntry.<init>(ClonedServerEntry.java:67)
>> at org.apache.directory.server.xdbm.AbstractXdbmPartition.lookup(AbstractXdbmPartition.java:327)
>> at org.apache.directory.server.core.partition.impl.btree.ServerEntryCursorAdaptor.get(ServerEntryCursorAdaptor.java:139)
>> at org.apache.directory.server.core.partition.impl.btree.ServerEntryCursorAdaptor.get(ServerEntryCursorAdaptor.java:39)
>> at org.apache.directory.server.core.filtering.BaseEntryFilteringCursor.next(BaseEntryFilteringCursor.java:503)
>> at org.apache.directory.server.core.authz.GroupCache.initialize(GroupCache.java:147)
>> at org.apache.directory.server.core.authz.GroupCache.<init>(GroupCache.java:111)
>> at org.apache.directory.server.core.authz.AciAuthorizationInterceptor.init(AciAuthorizationInterceptor.java:202)
>> at org.apache.directory.server.core.interceptor.InterceptorChain.register0(InterceptorChain.java:442)
>> at org.apache.directory.server.core.interceptor.InterceptorChain.register(InterceptorChain.java:398)
>> at org.apache.directory.server.core.interceptor.InterceptorChain.init(InterceptorChain.java:258)
>> at org.apache.directory.server.core.DefaultDirectoryService.initialize(DefaultDirectoryService.java:1447)
>> at org.apache.directory.server.core.DefaultDirectoryService.startup(DefaultDirectoryService.java:907)
>> at org.apache.directory.server.configuration.ApacheDS.startup(ApacheDS.java:163)
>> at org.apache.directory.server.Service.initLdap(Service.java:136)
>> at org.apache.directory.server.Service.init(Service.java:77)
>> at org.apache.directory.daemon.Bootstrapper.callInit(Bootstrapper.java:154)
>> at org.apache.directory.daemon.TanukiBootstrapper.start(TanukiBootstrapper.java:54)
>> at
>> org.tanukisoftware.wrapper.WrapperManager$12.run(WrapperManager.java:
>> 2
>> 788)
>>
>> I failed to archive any information in the internet about this szenario.
>> Does someone know what the problem is?
>> May the Tivoli backup restore some inconsistent state?
>>
>> Is there any chance to rescue the data of this LDAP server?
>> Good thing it was no Productive environment this time - only QA, but
>> it would be great to solve this in order to know how to solve it the
>> next time.. ;-) (if there is a way to solve it)
>>
>> Some additional information about our Environment: We are using Red Head Enterprise Linux release 5.8 (x86_64) on the impacted server.
>> ApacheDS is running on the embedded Java Virtual Machine
>>
>>
>> Mit freundlichen Grüßen / Best regards
>>
>> Daniel van Maren
>>
>> Bosch Software Innovations GmbH
>> Stuttgarter Str. 130,
>> 71332 Waiblingen
>> GERMANY
>> www.bosch-si.de
>>
>> daniel.vanmaren@bosch-si.com<ma...@bosch-si.com>
>>
>> Registered office: Immenstaad, Register court: Amtsgericht Ulm, HRB
>> 631888;
>> Executives: Heinz Derenbach; Erica Fölsche, Thomas Cotic, Michael
>> Hahn, Klaus Hüftle
>>
>
>
>
> --
> Kiran Ayyagari
> http://keydap.com
--
Kiran Ayyagari
http://keydap.com
Re: ApacheDS crashes on Startup after Backup-Restoration
Posted by Kiran Ayyagari <ka...@apache.org>.
can you disable the AciAuthorizationInterceptor and restart the
server. Can't guarantee that it will work
but just a try. If it works I would suggest you to export all the
entries to an LDIF file and load it into a
new server instance.
It looks like the files were backed up while the server is running and
this sort of backup won't work well (especially with a JDBM based
partition)
On Wed, Oct 17, 2012 at 5:51 PM, <Da...@bosch-si.com> wrote:
> Hello,
>
> We are using ApacheDS 1.5.7.
> Thank you for your help.
>
> Mit freundlichen Grüßen / Best regards
>
> Daniel van Maren
>
> Bosch Software Innovations GmbH
> Stuttgarter Str. 130,
> 71332 Waiblingen
> GERMANY
> www.bosch-si.de
>
> daniel.vanmaren@bosch-si.com
>
> Registered office: Immenstaad, Register court: Amtsgericht Ulm, HRB 631888;
> Executives: Heinz Derenbach; Erica Fölsche, Thomas Cotic, Michael Hahn, Klaus Hüftle
>
>
> -----Ursprüngliche Nachricht-----
> Von: ayyagarikiran@gmail.com [mailto:ayyagarikiran@gmail.com] Im Auftrag von Kiran Ayyagari
> Gesendet: Mittwoch, 17. Oktober 2012 14:16
> An: users@directory.apache.org
> Betreff: Re: ApacheDS crashes on Startup after Backup-Restoration
>
> which version of ApacheDS you are using?
>
> On Wed, Oct 17, 2012 at 5:10 PM, <Da...@bosch-si.com> wrote:
>> Greetings,
>>
>> We are using ApacheDS as LDAP provider for one of our Solutions (for user authentication).
>> Recently one of our ApacheDS Servers experienced a database corruption - we are not sure why.
>> The ApacheDS server booted up correctly, but when some client tried to retrieve a list of all users (for instance the Apache Directory Studio), the Server throws a exception and closes the socket to the client:
>>
>> [05:40:05] WARN [org.apache.directory.server.ldap.LdapProtocolHandler] - Unexpected exception forcing session to close: sending disconnect notice to client.
>> java.lang.Error: ERR_546 CRITICAL: page header magic for block 64 not OK 0
>> at jdbm.recman.PageHeader.<init>(PageHeader.java:95)
>> at jdbm.recman.PageHeader.getView(PageHeader.java:124)
>> at jdbm.recman.PageManager.getNext(PageManager.java:234)
>> at jdbm.recman.PageCursor.next(PageCursor.java:104)
>> at jdbm.recman.PhysicalRowIdManager.fetch(PhysicalRowIdManager.java:158)
>> at jdbm.recman.BaseRecordManager.fetch(BaseRecordManager.java:324)
>> at jdbm.recman.CacheRecordManager.fetch(CacheRecordManager.java:262)
>> at jdbm.btree.BPage.loadBPage(BPage.java:899)
>> at jdbm.btree.BPage.childBPage(BPage.java:890)
>> at jdbm.btree.BPage.find(BPage.java:284)
>> at jdbm.btree.BTree.find(BTree.java:408)
>> at org.apache.directory.server.core.partition.impl.btree.jdbm.JdbmTable.get(JdbmTable.java:395)
>> at org.apache.directory.server.core.partition.impl.btree.jdbm.JdbmMasterTable.get(JdbmMasterTable.java:155)
>> at org.apache.directory.server.core.partition.impl.btree.jdbm.JdbmStore.lookup(JdbmStore.java:1332)
>> at org.apache.directory.server.core.partition.impl.btree.jdbm.JdbmStore.lookup(JdbmStore.java:70)
>> at org.apache.directory.server.xdbm.AbstractXdbmPartition.lookup(AbstractXdbmPartition.java:327)
>> at org.apache.directory.server.core.partition.impl.btree.ServerEntryCursorAdaptor.get(ServerEntryCursorAdaptor.java:139)
>> at org.apache.directory.server.core.partition.impl.btree.ServerEntryCursorAdaptor.get(ServerEntryCursorAdaptor.java:39)
>> at org.apache.directory.server.core.filtering.BaseEntryFilteringCursor.next(BaseEntryFilteringCursor.java:503)
>> at org.apache.directory.server.ldap.handlers.SearchHandler.readResults(SearchHandler.java:314)
>> at org.apache.directory.server.ldap.handlers.SearchHandler.doSimpleSearch(SearchHandler.java:749)
>> at org.apache.directory.server.ldap.handlers.SearchHandler.handleIgnoringReferrals(SearchHandler.java:978)
>> at org.apache.directory.server.ldap.handlers.SearchHandler.handleWithReferrals(SearchHandler.java:1054)
>> at org.apache.directory.server.ldap.handlers.SearchHandler.handleWithReferrals(SearchHandler.java:78)
>> at org.apache.directory.server.ldap.handlers.ReferralAwareRequestHandler.handle(ReferralAwareRequestHandler.java:94)
>> at org.apache.directory.server.ldap.handlers.ReferralAwareRequestHandler.handle(ReferralAwareRequestHandler.java:57)
>> at org.apache.directory.server.ldap.handlers.LdapRequestHandler.handleMessage(LdapRequestHandler.java:208)
>> at org.apache.directory.server.ldap.handlers.LdapRequestHandler.handleMessage(LdapRequestHandler.java:58)
>> at org.apache.mina.handler.demux.DemuxingIoHandler.messageReceived(DemuxingIoHandler.java:232)
>> at org.apache.directory.server.ldap.LdapProtocolHandler.messageReceived(LdapProtocolHandler.java:193)
>> at org.apache.mina.core.filterchain.DefaultIoFilterChain$TailFilter.messageReceived(DefaultIoFilterChain.java:713)
>> at org.apache.mina.core.filterchain.DefaultIoFilterChain.callNextMessageReceived(DefaultIoFilterChain.java:434)
>> at org.apache.mina.core.filterchain.DefaultIoFilterChain.access$1200(DefaultIoFilterChain.java:46)
>> at org.apache.mina.core.filterchain.DefaultIoFilterChain$EntryImpl$1.messageReceived(DefaultIoFilterChain.java:793)
>> at org.apache.mina.core.filterchain.IoFilterEvent.fire(IoFilterEvent.java:71)
>> at org.apache.mina.core.session.IoEvent.run(IoEvent.java:63)
>> at org.apache.mina.filter.executor.UnorderedThreadPoolExecutor$Worker.runTask(UnorderedThreadPoolExecutor.java:480)
>> at org.apache.mina.filter.executor.UnorderedThreadPoolExecutor$Worker.run(UnorderedThreadPoolExecutor.java:434)
>> at java.lang.Thread.run(Thread.java:662)
>> [05:40:05] WARN [org.apache.directory.server.ldap.LdapProtocolHandler] - Unexpected exception forcing session to close: sending disconnect notice to client.
>> org.apache.mina.core.write.WriteToClosedSessionException
>> at org.apache.mina.core.polling.AbstractPollingIoProcessor.clearWriteRequestQueue(AbstractPollingIoProcessor.java:573)
>> at org.apache.mina.core.polling.AbstractPollingIoProcessor.removeNow(AbstractPollingIoProcessor.java:525)
>> at org.apache.mina.core.polling.AbstractPollingIoProcessor.removeSessions(AbstractPollingIoProcessor.java:497)
>> at org.apache.mina.core.polling.AbstractPollingIoProcessor.access$600(AbstractPollingIoProcessor.java:61)
>> at org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.run(AbstractPollingIoProcessor.java:974)
>> at org.apache.mina.util.NamePreservingRunnable.run(NamePreservingRunnable.java:64)
>> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>> at java.lang.Thread.run(Thread.java:662)
>>
>> We do not think that there is a way to repair this corruption, therefore we tried to restore a backup (we are using Tivoli for backup of our SAN storage).
>> We were able to determine the Point in Time when the corruption took place, as we use a oracle database job which fails in case the LDAP query fails.
>> After trying to restore to the point in time when the Database was still working we experienced a new Problem:
>> The ApacheDS Server fails to boot up - it crashes with the following Stack trace:
>>
>> [11:27:58] WARN [org.apache.directory.shared.ldap.ldif.LdifReader] -
>> No version information : assuming version: 1 [11:27:58] WARN
>> [org.apache.directory.shared.ldap.ldif.LdifReader] - No version
>> information : assuming version: 1 [11:27:58] WARN
>> [org.apache.directory.shared.ldap.ldif.LdifReader] - No version
>> information : assuming version: 1 [11:27:58] WARN
>> [org.apache.directory.shared.ldap.ldif.LdifReader] - No version
>> information : assuming version: 1 [11:27:58] WARN [org.apache.directory.shared.ldap.ldif.LdifReader] - No version information : assuming version: 1 [11:27:58] WARN [org.apache.directory.shared.ldap.ldif.LdifReader] - No version information : assuming version: 1 [11:27:59] ERROR [org.apache.directory.daemon.Bootstrapper] - Failed on null.init(InstallationLayout, String[]) java.lang.NullPointerException
>> at org.apache.directory.server.core.entry.ClonedServerEntry.<init>(ClonedServerEntry.java:67)
>> at org.apache.directory.server.xdbm.AbstractXdbmPartition.lookup(AbstractXdbmPartition.java:327)
>> at org.apache.directory.server.core.partition.impl.btree.ServerEntryCursorAdaptor.get(ServerEntryCursorAdaptor.java:139)
>> at org.apache.directory.server.core.partition.impl.btree.ServerEntryCursorAdaptor.get(ServerEntryCursorAdaptor.java:39)
>> at org.apache.directory.server.core.filtering.BaseEntryFilteringCursor.next(BaseEntryFilteringCursor.java:503)
>> at org.apache.directory.server.core.authz.GroupCache.initialize(GroupCache.java:147)
>> at org.apache.directory.server.core.authz.GroupCache.<init>(GroupCache.java:111)
>> at org.apache.directory.server.core.authz.AciAuthorizationInterceptor.init(AciAuthorizationInterceptor.java:202)
>> at org.apache.directory.server.core.interceptor.InterceptorChain.register0(InterceptorChain.java:442)
>> at org.apache.directory.server.core.interceptor.InterceptorChain.register(InterceptorChain.java:398)
>> at org.apache.directory.server.core.interceptor.InterceptorChain.init(InterceptorChain.java:258)
>> at org.apache.directory.server.core.DefaultDirectoryService.initialize(DefaultDirectoryService.java:1447)
>> at org.apache.directory.server.core.DefaultDirectoryService.startup(DefaultDirectoryService.java:907)
>> at org.apache.directory.server.configuration.ApacheDS.startup(ApacheDS.java:163)
>> at org.apache.directory.server.Service.initLdap(Service.java:136)
>> at org.apache.directory.server.Service.init(Service.java:77)
>> at org.apache.directory.daemon.Bootstrapper.callInit(Bootstrapper.java:154)
>> at org.apache.directory.daemon.TanukiBootstrapper.start(TanukiBootstrapper.java:54)
>> at
>> org.tanukisoftware.wrapper.WrapperManager$12.run(WrapperManager.java:2
>> 788)
>>
>> I failed to archive any information in the internet about this szenario.
>> Does someone know what the problem is?
>> May the Tivoli backup restore some inconsistent state?
>>
>> Is there any chance to rescue the data of this LDAP server?
>> Good thing it was no Productive environment this time - only QA, but
>> it would be great to solve this in order to know how to solve it the
>> next time.. ;-) (if there is a way to solve it)
>>
>> Some additional information about our Environment: We are using Red Head Enterprise Linux release 5.8 (x86_64) on the impacted server.
>> ApacheDS is running on the embedded Java Virtual Machine
>>
>>
>> Mit freundlichen Grüßen / Best regards
>>
>> Daniel van Maren
>>
>> Bosch Software Innovations GmbH
>> Stuttgarter Str. 130,
>> 71332 Waiblingen
>> GERMANY
>> www.bosch-si.de
>>
>> daniel.vanmaren@bosch-si.com<ma...@bosch-si.com>
>>
>> Registered office: Immenstaad, Register court: Amtsgericht Ulm, HRB
>> 631888;
>> Executives: Heinz Derenbach; Erica Fölsche, Thomas Cotic, Michael
>> Hahn, Klaus Hüftle
>>
>
>
>
> --
> Kiran Ayyagari
> http://keydap.com
--
Kiran Ayyagari
http://keydap.com
AW: ApacheDS crashes on Startup after Backup-Restoration
Posted by Da...@bosch-si.com.
Hello,
We are using ApacheDS 1.5.7.
Thank you for your help.
Mit freundlichen Grüßen / Best regards
Daniel van Maren
Bosch Software Innovations GmbH
Stuttgarter Str. 130,
71332 Waiblingen
GERMANY
www.bosch-si.de
daniel.vanmaren@bosch-si.com
Registered office: Immenstaad, Register court: Amtsgericht Ulm, HRB 631888;
Executives: Heinz Derenbach; Erica Fölsche, Thomas Cotic, Michael Hahn, Klaus Hüftle
-----Ursprüngliche Nachricht-----
Von: ayyagarikiran@gmail.com [mailto:ayyagarikiran@gmail.com] Im Auftrag von Kiran Ayyagari
Gesendet: Mittwoch, 17. Oktober 2012 14:16
An: users@directory.apache.org
Betreff: Re: ApacheDS crashes on Startup after Backup-Restoration
which version of ApacheDS you are using?
On Wed, Oct 17, 2012 at 5:10 PM, <Da...@bosch-si.com> wrote:
> Greetings,
>
> We are using ApacheDS as LDAP provider for one of our Solutions (for user authentication).
> Recently one of our ApacheDS Servers experienced a database corruption - we are not sure why.
> The ApacheDS server booted up correctly, but when some client tried to retrieve a list of all users (for instance the Apache Directory Studio), the Server throws a exception and closes the socket to the client:
>
> [05:40:05] WARN [org.apache.directory.server.ldap.LdapProtocolHandler] - Unexpected exception forcing session to close: sending disconnect notice to client.
> java.lang.Error: ERR_546 CRITICAL: page header magic for block 64 not OK 0
> at jdbm.recman.PageHeader.<init>(PageHeader.java:95)
> at jdbm.recman.PageHeader.getView(PageHeader.java:124)
> at jdbm.recman.PageManager.getNext(PageManager.java:234)
> at jdbm.recman.PageCursor.next(PageCursor.java:104)
> at jdbm.recman.PhysicalRowIdManager.fetch(PhysicalRowIdManager.java:158)
> at jdbm.recman.BaseRecordManager.fetch(BaseRecordManager.java:324)
> at jdbm.recman.CacheRecordManager.fetch(CacheRecordManager.java:262)
> at jdbm.btree.BPage.loadBPage(BPage.java:899)
> at jdbm.btree.BPage.childBPage(BPage.java:890)
> at jdbm.btree.BPage.find(BPage.java:284)
> at jdbm.btree.BTree.find(BTree.java:408)
> at org.apache.directory.server.core.partition.impl.btree.jdbm.JdbmTable.get(JdbmTable.java:395)
> at org.apache.directory.server.core.partition.impl.btree.jdbm.JdbmMasterTable.get(JdbmMasterTable.java:155)
> at org.apache.directory.server.core.partition.impl.btree.jdbm.JdbmStore.lookup(JdbmStore.java:1332)
> at org.apache.directory.server.core.partition.impl.btree.jdbm.JdbmStore.lookup(JdbmStore.java:70)
> at org.apache.directory.server.xdbm.AbstractXdbmPartition.lookup(AbstractXdbmPartition.java:327)
> at org.apache.directory.server.core.partition.impl.btree.ServerEntryCursorAdaptor.get(ServerEntryCursorAdaptor.java:139)
> at org.apache.directory.server.core.partition.impl.btree.ServerEntryCursorAdaptor.get(ServerEntryCursorAdaptor.java:39)
> at org.apache.directory.server.core.filtering.BaseEntryFilteringCursor.next(BaseEntryFilteringCursor.java:503)
> at org.apache.directory.server.ldap.handlers.SearchHandler.readResults(SearchHandler.java:314)
> at org.apache.directory.server.ldap.handlers.SearchHandler.doSimpleSearch(SearchHandler.java:749)
> at org.apache.directory.server.ldap.handlers.SearchHandler.handleIgnoringReferrals(SearchHandler.java:978)
> at org.apache.directory.server.ldap.handlers.SearchHandler.handleWithReferrals(SearchHandler.java:1054)
> at org.apache.directory.server.ldap.handlers.SearchHandler.handleWithReferrals(SearchHandler.java:78)
> at org.apache.directory.server.ldap.handlers.ReferralAwareRequestHandler.handle(ReferralAwareRequestHandler.java:94)
> at org.apache.directory.server.ldap.handlers.ReferralAwareRequestHandler.handle(ReferralAwareRequestHandler.java:57)
> at org.apache.directory.server.ldap.handlers.LdapRequestHandler.handleMessage(LdapRequestHandler.java:208)
> at org.apache.directory.server.ldap.handlers.LdapRequestHandler.handleMessage(LdapRequestHandler.java:58)
> at org.apache.mina.handler.demux.DemuxingIoHandler.messageReceived(DemuxingIoHandler.java:232)
> at org.apache.directory.server.ldap.LdapProtocolHandler.messageReceived(LdapProtocolHandler.java:193)
> at org.apache.mina.core.filterchain.DefaultIoFilterChain$TailFilter.messageReceived(DefaultIoFilterChain.java:713)
> at org.apache.mina.core.filterchain.DefaultIoFilterChain.callNextMessageReceived(DefaultIoFilterChain.java:434)
> at org.apache.mina.core.filterchain.DefaultIoFilterChain.access$1200(DefaultIoFilterChain.java:46)
> at org.apache.mina.core.filterchain.DefaultIoFilterChain$EntryImpl$1.messageReceived(DefaultIoFilterChain.java:793)
> at org.apache.mina.core.filterchain.IoFilterEvent.fire(IoFilterEvent.java:71)
> at org.apache.mina.core.session.IoEvent.run(IoEvent.java:63)
> at org.apache.mina.filter.executor.UnorderedThreadPoolExecutor$Worker.runTask(UnorderedThreadPoolExecutor.java:480)
> at org.apache.mina.filter.executor.UnorderedThreadPoolExecutor$Worker.run(UnorderedThreadPoolExecutor.java:434)
> at java.lang.Thread.run(Thread.java:662)
> [05:40:05] WARN [org.apache.directory.server.ldap.LdapProtocolHandler] - Unexpected exception forcing session to close: sending disconnect notice to client.
> org.apache.mina.core.write.WriteToClosedSessionException
> at org.apache.mina.core.polling.AbstractPollingIoProcessor.clearWriteRequestQueue(AbstractPollingIoProcessor.java:573)
> at org.apache.mina.core.polling.AbstractPollingIoProcessor.removeNow(AbstractPollingIoProcessor.java:525)
> at org.apache.mina.core.polling.AbstractPollingIoProcessor.removeSessions(AbstractPollingIoProcessor.java:497)
> at org.apache.mina.core.polling.AbstractPollingIoProcessor.access$600(AbstractPollingIoProcessor.java:61)
> at org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.run(AbstractPollingIoProcessor.java:974)
> at org.apache.mina.util.NamePreservingRunnable.run(NamePreservingRunnable.java:64)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
>
> We do not think that there is a way to repair this corruption, therefore we tried to restore a backup (we are using Tivoli for backup of our SAN storage).
> We were able to determine the Point in Time when the corruption took place, as we use a oracle database job which fails in case the LDAP query fails.
> After trying to restore to the point in time when the Database was still working we experienced a new Problem:
> The ApacheDS Server fails to boot up - it crashes with the following Stack trace:
>
> [11:27:58] WARN [org.apache.directory.shared.ldap.ldif.LdifReader] -
> No version information : assuming version: 1 [11:27:58] WARN
> [org.apache.directory.shared.ldap.ldif.LdifReader] - No version
> information : assuming version: 1 [11:27:58] WARN
> [org.apache.directory.shared.ldap.ldif.LdifReader] - No version
> information : assuming version: 1 [11:27:58] WARN
> [org.apache.directory.shared.ldap.ldif.LdifReader] - No version
> information : assuming version: 1 [11:27:58] WARN [org.apache.directory.shared.ldap.ldif.LdifReader] - No version information : assuming version: 1 [11:27:58] WARN [org.apache.directory.shared.ldap.ldif.LdifReader] - No version information : assuming version: 1 [11:27:59] ERROR [org.apache.directory.daemon.Bootstrapper] - Failed on null.init(InstallationLayout, String[]) java.lang.NullPointerException
> at org.apache.directory.server.core.entry.ClonedServerEntry.<init>(ClonedServerEntry.java:67)
> at org.apache.directory.server.xdbm.AbstractXdbmPartition.lookup(AbstractXdbmPartition.java:327)
> at org.apache.directory.server.core.partition.impl.btree.ServerEntryCursorAdaptor.get(ServerEntryCursorAdaptor.java:139)
> at org.apache.directory.server.core.partition.impl.btree.ServerEntryCursorAdaptor.get(ServerEntryCursorAdaptor.java:39)
> at org.apache.directory.server.core.filtering.BaseEntryFilteringCursor.next(BaseEntryFilteringCursor.java:503)
> at org.apache.directory.server.core.authz.GroupCache.initialize(GroupCache.java:147)
> at org.apache.directory.server.core.authz.GroupCache.<init>(GroupCache.java:111)
> at org.apache.directory.server.core.authz.AciAuthorizationInterceptor.init(AciAuthorizationInterceptor.java:202)
> at org.apache.directory.server.core.interceptor.InterceptorChain.register0(InterceptorChain.java:442)
> at org.apache.directory.server.core.interceptor.InterceptorChain.register(InterceptorChain.java:398)
> at org.apache.directory.server.core.interceptor.InterceptorChain.init(InterceptorChain.java:258)
> at org.apache.directory.server.core.DefaultDirectoryService.initialize(DefaultDirectoryService.java:1447)
> at org.apache.directory.server.core.DefaultDirectoryService.startup(DefaultDirectoryService.java:907)
> at org.apache.directory.server.configuration.ApacheDS.startup(ApacheDS.java:163)
> at org.apache.directory.server.Service.initLdap(Service.java:136)
> at org.apache.directory.server.Service.init(Service.java:77)
> at org.apache.directory.daemon.Bootstrapper.callInit(Bootstrapper.java:154)
> at org.apache.directory.daemon.TanukiBootstrapper.start(TanukiBootstrapper.java:54)
> at
> org.tanukisoftware.wrapper.WrapperManager$12.run(WrapperManager.java:2
> 788)
>
> I failed to archive any information in the internet about this szenario.
> Does someone know what the problem is?
> May the Tivoli backup restore some inconsistent state?
>
> Is there any chance to rescue the data of this LDAP server?
> Good thing it was no Productive environment this time - only QA, but
> it would be great to solve this in order to know how to solve it the
> next time.. ;-) (if there is a way to solve it)
>
> Some additional information about our Environment: We are using Red Head Enterprise Linux release 5.8 (x86_64) on the impacted server.
> ApacheDS is running on the embedded Java Virtual Machine
>
>
> Mit freundlichen Grüßen / Best regards
>
> Daniel van Maren
>
> Bosch Software Innovations GmbH
> Stuttgarter Str. 130,
> 71332 Waiblingen
> GERMANY
> www.bosch-si.de
>
> daniel.vanmaren@bosch-si.com<ma...@bosch-si.com>
>
> Registered office: Immenstaad, Register court: Amtsgericht Ulm, HRB
> 631888;
> Executives: Heinz Derenbach; Erica Fölsche, Thomas Cotic, Michael
> Hahn, Klaus Hüftle
>
--
Kiran Ayyagari
http://keydap.com
Re: ApacheDS crashes on Startup after Backup-Restoration
Posted by Kiran Ayyagari <ka...@apache.org>.
which version of ApacheDS you are using?
On Wed, Oct 17, 2012 at 5:10 PM, <Da...@bosch-si.com> wrote:
> Greetings,
>
> We are using ApacheDS as LDAP provider for one of our Solutions (for user authentication).
> Recently one of our ApacheDS Servers experienced a database corruption - we are not sure why.
> The ApacheDS server booted up correctly, but when some client tried to retrieve a list of all users (for instance the Apache Directory Studio), the Server throws a exception and closes the socket to the client:
>
> [05:40:05] WARN [org.apache.directory.server.ldap.LdapProtocolHandler] - Unexpected exception forcing session to close: sending disconnect notice to client.
> java.lang.Error: ERR_546 CRITICAL: page header magic for block 64 not OK 0
> at jdbm.recman.PageHeader.<init>(PageHeader.java:95)
> at jdbm.recman.PageHeader.getView(PageHeader.java:124)
> at jdbm.recman.PageManager.getNext(PageManager.java:234)
> at jdbm.recman.PageCursor.next(PageCursor.java:104)
> at jdbm.recman.PhysicalRowIdManager.fetch(PhysicalRowIdManager.java:158)
> at jdbm.recman.BaseRecordManager.fetch(BaseRecordManager.java:324)
> at jdbm.recman.CacheRecordManager.fetch(CacheRecordManager.java:262)
> at jdbm.btree.BPage.loadBPage(BPage.java:899)
> at jdbm.btree.BPage.childBPage(BPage.java:890)
> at jdbm.btree.BPage.find(BPage.java:284)
> at jdbm.btree.BTree.find(BTree.java:408)
> at org.apache.directory.server.core.partition.impl.btree.jdbm.JdbmTable.get(JdbmTable.java:395)
> at org.apache.directory.server.core.partition.impl.btree.jdbm.JdbmMasterTable.get(JdbmMasterTable.java:155)
> at org.apache.directory.server.core.partition.impl.btree.jdbm.JdbmStore.lookup(JdbmStore.java:1332)
> at org.apache.directory.server.core.partition.impl.btree.jdbm.JdbmStore.lookup(JdbmStore.java:70)
> at org.apache.directory.server.xdbm.AbstractXdbmPartition.lookup(AbstractXdbmPartition.java:327)
> at org.apache.directory.server.core.partition.impl.btree.ServerEntryCursorAdaptor.get(ServerEntryCursorAdaptor.java:139)
> at org.apache.directory.server.core.partition.impl.btree.ServerEntryCursorAdaptor.get(ServerEntryCursorAdaptor.java:39)
> at org.apache.directory.server.core.filtering.BaseEntryFilteringCursor.next(BaseEntryFilteringCursor.java:503)
> at org.apache.directory.server.ldap.handlers.SearchHandler.readResults(SearchHandler.java:314)
> at org.apache.directory.server.ldap.handlers.SearchHandler.doSimpleSearch(SearchHandler.java:749)
> at org.apache.directory.server.ldap.handlers.SearchHandler.handleIgnoringReferrals(SearchHandler.java:978)
> at org.apache.directory.server.ldap.handlers.SearchHandler.handleWithReferrals(SearchHandler.java:1054)
> at org.apache.directory.server.ldap.handlers.SearchHandler.handleWithReferrals(SearchHandler.java:78)
> at org.apache.directory.server.ldap.handlers.ReferralAwareRequestHandler.handle(ReferralAwareRequestHandler.java:94)
> at org.apache.directory.server.ldap.handlers.ReferralAwareRequestHandler.handle(ReferralAwareRequestHandler.java:57)
> at org.apache.directory.server.ldap.handlers.LdapRequestHandler.handleMessage(LdapRequestHandler.java:208)
> at org.apache.directory.server.ldap.handlers.LdapRequestHandler.handleMessage(LdapRequestHandler.java:58)
> at org.apache.mina.handler.demux.DemuxingIoHandler.messageReceived(DemuxingIoHandler.java:232)
> at org.apache.directory.server.ldap.LdapProtocolHandler.messageReceived(LdapProtocolHandler.java:193)
> at org.apache.mina.core.filterchain.DefaultIoFilterChain$TailFilter.messageReceived(DefaultIoFilterChain.java:713)
> at org.apache.mina.core.filterchain.DefaultIoFilterChain.callNextMessageReceived(DefaultIoFilterChain.java:434)
> at org.apache.mina.core.filterchain.DefaultIoFilterChain.access$1200(DefaultIoFilterChain.java:46)
> at org.apache.mina.core.filterchain.DefaultIoFilterChain$EntryImpl$1.messageReceived(DefaultIoFilterChain.java:793)
> at org.apache.mina.core.filterchain.IoFilterEvent.fire(IoFilterEvent.java:71)
> at org.apache.mina.core.session.IoEvent.run(IoEvent.java:63)
> at org.apache.mina.filter.executor.UnorderedThreadPoolExecutor$Worker.runTask(UnorderedThreadPoolExecutor.java:480)
> at org.apache.mina.filter.executor.UnorderedThreadPoolExecutor$Worker.run(UnorderedThreadPoolExecutor.java:434)
> at java.lang.Thread.run(Thread.java:662)
> [05:40:05] WARN [org.apache.directory.server.ldap.LdapProtocolHandler] - Unexpected exception forcing session to close: sending disconnect notice to client.
> org.apache.mina.core.write.WriteToClosedSessionException
> at org.apache.mina.core.polling.AbstractPollingIoProcessor.clearWriteRequestQueue(AbstractPollingIoProcessor.java:573)
> at org.apache.mina.core.polling.AbstractPollingIoProcessor.removeNow(AbstractPollingIoProcessor.java:525)
> at org.apache.mina.core.polling.AbstractPollingIoProcessor.removeSessions(AbstractPollingIoProcessor.java:497)
> at org.apache.mina.core.polling.AbstractPollingIoProcessor.access$600(AbstractPollingIoProcessor.java:61)
> at org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.run(AbstractPollingIoProcessor.java:974)
> at org.apache.mina.util.NamePreservingRunnable.run(NamePreservingRunnable.java:64)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
>
> We do not think that there is a way to repair this corruption, therefore we tried to restore a backup (we are using Tivoli for backup of our SAN storage).
> We were able to determine the Point in Time when the corruption took place, as we use a oracle database job which fails in case the LDAP query fails.
> After trying to restore to the point in time when the Database was still working we experienced a new Problem:
> The ApacheDS Server fails to boot up - it crashes with the following Stack trace:
>
> [11:27:58] WARN [org.apache.directory.shared.ldap.ldif.LdifReader] - No version information : assuming version: 1
> [11:27:58] WARN [org.apache.directory.shared.ldap.ldif.LdifReader] - No version information : assuming version: 1
> [11:27:58] WARN [org.apache.directory.shared.ldap.ldif.LdifReader] - No version information : assuming version: 1
> [11:27:58] WARN [org.apache.directory.shared.ldap.ldif.LdifReader] - No version information : assuming version: 1
> [11:27:58] WARN [org.apache.directory.shared.ldap.ldif.LdifReader] - No version information : assuming version: 1
> [11:27:58] WARN [org.apache.directory.shared.ldap.ldif.LdifReader] - No version information : assuming version: 1
> [11:27:59] ERROR [org.apache.directory.daemon.Bootstrapper] - Failed on null.init(InstallationLayout, String[])
> java.lang.NullPointerException
> at org.apache.directory.server.core.entry.ClonedServerEntry.<init>(ClonedServerEntry.java:67)
> at org.apache.directory.server.xdbm.AbstractXdbmPartition.lookup(AbstractXdbmPartition.java:327)
> at org.apache.directory.server.core.partition.impl.btree.ServerEntryCursorAdaptor.get(ServerEntryCursorAdaptor.java:139)
> at org.apache.directory.server.core.partition.impl.btree.ServerEntryCursorAdaptor.get(ServerEntryCursorAdaptor.java:39)
> at org.apache.directory.server.core.filtering.BaseEntryFilteringCursor.next(BaseEntryFilteringCursor.java:503)
> at org.apache.directory.server.core.authz.GroupCache.initialize(GroupCache.java:147)
> at org.apache.directory.server.core.authz.GroupCache.<init>(GroupCache.java:111)
> at org.apache.directory.server.core.authz.AciAuthorizationInterceptor.init(AciAuthorizationInterceptor.java:202)
> at org.apache.directory.server.core.interceptor.InterceptorChain.register0(InterceptorChain.java:442)
> at org.apache.directory.server.core.interceptor.InterceptorChain.register(InterceptorChain.java:398)
> at org.apache.directory.server.core.interceptor.InterceptorChain.init(InterceptorChain.java:258)
> at org.apache.directory.server.core.DefaultDirectoryService.initialize(DefaultDirectoryService.java:1447)
> at org.apache.directory.server.core.DefaultDirectoryService.startup(DefaultDirectoryService.java:907)
> at org.apache.directory.server.configuration.ApacheDS.startup(ApacheDS.java:163)
> at org.apache.directory.server.Service.initLdap(Service.java:136)
> at org.apache.directory.server.Service.init(Service.java:77)
> at org.apache.directory.daemon.Bootstrapper.callInit(Bootstrapper.java:154)
> at org.apache.directory.daemon.TanukiBootstrapper.start(TanukiBootstrapper.java:54)
> at org.tanukisoftware.wrapper.WrapperManager$12.run(WrapperManager.java:2788)
>
> I failed to archive any information in the internet about this szenario.
> Does someone know what the problem is?
> May the Tivoli backup restore some inconsistent state?
>
> Is there any chance to rescue the data of this LDAP server?
> Good thing it was no Productive environment this time - only QA, but it would be great to solve this in order to know how to solve it the next time.. ;-) (if there is a way to solve it)
>
> Some additional information about our Environment: We are using Red Head Enterprise Linux release 5.8 (x86_64) on the impacted server.
> ApacheDS is running on the embedded Java Virtual Machine
>
>
> Mit freundlichen Grüßen / Best regards
>
> Daniel van Maren
>
> Bosch Software Innovations GmbH
> Stuttgarter Str. 130,
> 71332 Waiblingen
> GERMANY
> www.bosch-si.de
>
> daniel.vanmaren@bosch-si.com<ma...@bosch-si.com>
>
> Registered office: Immenstaad, Register court: Amtsgericht Ulm, HRB 631888;
> Executives: Heinz Derenbach; Erica Fölsche, Thomas Cotic, Michael Hahn, Klaus Hüftle
>
--
Kiran Ayyagari
http://keydap.com