You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@jackrabbit.apache.org by Christian Zanata <ch...@wavegroup.it> on 2006/08/28 12:07:25 UTC

problems with re-indexing the workspace

Hi to all,
we have this problem on a production server:

[ERROR] 20060825 17:06:40
(org.apache.jackrabbit.core.observation.ObservationManagerFactory) -
Synchronous EventConsumer threw exception. java.lang.OutOfMemoryError

This error seems happening when the repository tries to re-index the
workspace, but we don't have more stack traces.

Our configuration is: tomcat 5.0 [256 up to 512 mb of ram] + jackrabbit
1.0.1 on jdk 1.4.2_12 on an Ibntel Xeon 3.2GHz with 2Gb of memory

an this are our libraries:
activation-1.1.jar
dom4j.jar
pdfbox-0.7.2.jar
antlr.jar
hibernate2.jar
catalina.jar
poi-3.0-alpha2-20060616.jar
poi-contrib-3.0-alpha2-20060616.jar
poi-scratchpad-3.0-alpha2-20060616.jar
cglib-full-2.0.2.jar
commons-beanutils-1.6.jar
commons-collections-3.1.jar
commons-digester-1.5.jar
commons-fileupload-1.0.jar
commons-logging-1.0.4.jar
commons-validator-1.1.3.jar
jackrabbit-core-1.0.1.jar
jackrabbit-index-filters-1.0.1.jar
jackrabbit-jcr-commons-1.0.1.jar
jakarta-oro-2.0.8.jar
slf4j-log4j12.jar
javamail-1.4.jar
struts-1.2.4.jar
jcr-1.0.jar
struts-el.jar
jta.jar
tm-extractors-0.4.jar
log4j-1.2.11.jar
xercesImpl-2.6.2.jar
concurrent.jar
lucene-1.4.3.jar
xml-apis.jar
db-ojb-1.0.4.jar
nekohtml.jar

this is also our repository.xml configuration:

<SearchIndex
class="org.apache.jackrabbit.core.query.lucene.SearchIndex">
        <param name="path" value="${wsp.home}/index"/>
        <param name="textFilterClasses"
value="org.apache.jackrabbit.core.query.lucene.TextPlainTextFilter,
org.apache.jackrabbit.core.query.MsExcelTextFilter,
org.apache.jackrabbit.core.query.MsPowerPointTextFilter, 
org.apache.jackrabbit.core.query.MsWordTextFilter,
org.apache.jackrabbit.core.query.PdfTextFilter,
org.apache.jackrabbit.core.query.HTMLTextFilter,
org.apache.jackrabbit.core.query.XMLTextFilter,
org.apache.jackrabbit.core.query.RTFTextFilter,
        		org.apache.jackrabbit.core.query.OpenOfficeTextFilter"/>
        <param name="useCompoundFile" value="true"/>
        <param name="minMergeDocs" value="100"/>
        <param name="volatileIdleTime" value="3"/>
        <param name="maxMergeDocs" value="100000"/>
        <param name="mergeFactor" value="10"/>
        <param name="bufferSize" value="10"/>
        <param name="cacheSize" value="1000"/>
        <param name="forceConsistencyCheck" value="false"/>
        <param name="autoRepair" value="true"/>
		<param name="respectDocumentOrder" value="false"/>
        <param name="analyzer"
value="org.apache.lucene.analysis.standard.StandardAnalyzer"/>
</SearchIndex>

could anybody heps us to understand what's happening?

thanks
chris


Re: problems with re-indexing the workspace

Posted by Christian Zanata <ch...@wavegroup.it>.
On Mon, 2006-08-28 at 14:50 +0300, Jukka Zitting wrote:
> Hi,
> 
> On 8/28/06, Christian Zanata <ch...@wavegroup.it> wrote:
> > [ERROR] 20060825 17:06:40
> > (org.apache.jackrabbit.core.observation.ObservationManagerFactory) -
> > Synchronous EventConsumer threw exception. java.lang.OutOfMemoryError
> >
> > This error seems happening when the repository tries to re-index the
> > workspace, but we don't have more stack traces.
> > [...]
> > could anybody heps us to understand what's happening?
> 
> There are two likely causes for that; either Lucene is running out of
> memory while merging the index segments, or one of the index filters
> runs out of memory trying to parse one of the binary documents in the
> repository. Without a complete stack trace it is difficult to
> determine the exact cause of the problems.
> 

Hi,
in our stack-trace there are other orrors, but I don't know if this
errors are related to this problem.

First one is that Lucene have some problems indexing some MsOffice
documents, but as i read on the internet it seems that the problem is
that sometimes the extension .doc isn't related to a real .doc file.

-------
[DEBUG] 20060803 08:24:01 (org.apache.jackrabbit.core.query.LazyReader)
- Dump: 
java.io.IOException: Invalid header signature; read 8656037701166316554,
expected -2226271756974174256
	at org.apache.jackrabbit.core.query.MsWordTextFilter
$1.initializeReader(MsWordTextFilter.java:78)
	at
org.apache.jackrabbit.core.query.LazyReader.read(LazyReader.java:79)
	at
org.apache.lucene.analysis.standard.FastCharStream.refill(FastCharStream.java:68)
	at
org.apache.lucene.analysis.standard.FastCharStream.readChar(FastCharStream.java:44)
	at
org.apache.lucene.analysis.standard.FastCharStream.BeginToken(FastCharStream.java:78)
	at
org.apache.lucene.analysis.standard.StandardTokenizerTokenManager.getNextToken(StandardTokenizerTokenManager.java:1105)
	at
org.apache.lucene.analysis.standard.StandardTokenizer.jj_ntk(StandardTokenizer.java:150)
	at
org.apache.lucene.analysis.standard.StandardTokenizer.next(StandardTokenizer.java:28)
	at
org.apache.lucene.analysis.standard.StandardFilter.next(StandardFilter.java:40)
	at
org.apache.lucene.analysis.LowerCaseFilter.next(LowerCaseFilter.java:32)
	at org.apache.lucene.analysis.StopFilter.next(StopFilter.java:98)
	at
org.apache.lucene.index.DocumentWriter.invertDocument(DocumentWriter.java:143)
	at
org.apache.lucene.index.DocumentWriter.addDocument(DocumentWriter.java:81)
	at
org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:307)
	at
org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:294)
	at
org.apache.jackrabbit.core.query.lucene.AbstractIndex.addDocument(AbstractIndex.java:129)
	at
org.apache.jackrabbit.core.query.lucene.VolatileIndex.commitPending(VolatileIndex.java:164)
	at
org.apache.jackrabbit.core.query.lucene.VolatileIndex.addDocument(VolatileIndex.java:86)
	at org.apache.jackrabbit.core.query.lucene.MultiIndex
$AddNode.execute(MultiIndex.java:1304)
	at
org.apache.jackrabbit.core.query.lucene.MultiIndex.executeAndLog(MultiIndex.java:763)
	at
org.apache.jackrabbit.core.query.lucene.MultiIndex.createIndex(MultiIndex.java:838)
	at
org.apache.jackrabbit.core.query.lucene.MultiIndex.createIndex(MultiIndex.java:844)
	at
org.apache.jackrabbit.core.query.lucene.MultiIndex.createIndex(MultiIndex.java:844)
	at
org.apache.jackrabbit.core.query.lucene.MultiIndex.createIndex(MultiIndex.java:844)
	at
org.apache.jackrabbit.core.query.lucene.MultiIndex.createIndex(MultiIndex.java:844)
	at
org.apache.jackrabbit.core.query.lucene.MultiIndex.createIndex(MultiIndex.java:844)
	at
org.apache.jackrabbit.core.query.lucene.MultiIndex.createIndex(MultiIndex.java:844)
	at
org.apache.jackrabbit.core.query.lucene.MultiIndex.createIndex(MultiIndex.java:844)
	at
org.apache.jackrabbit.core.query.lucene.MultiIndex.<init>(MultiIndex.java:271)
	at
org.apache.jackrabbit.core.query.lucene.SearchIndex.doInit(SearchIndex.java:218)
	at
org.apache.jackrabbit.core.query.AbstractQueryHandler.init(AbstractQueryHandler.java:43)
	at
org.apache.jackrabbit.core.SearchManager.initializeQueryHandler(SearchManager.java:529)
	at
org.apache.jackrabbit.core.SearchManager.<init>(SearchManager.java:273)
	at
org.apache.jackrabbit.core.RepositoryImpl.getSystemSearchManager(RepositoryImpl.java:560)
	at org.apache.jackrabbit.core.RepositoryImpl.access
$300(RepositoryImpl.java:84)
	at org.apache.jackrabbit.core.RepositoryImpl
$WorkspaceInfo.getSearchManager(RepositoryImpl.java:1434)
	at
org.apache.jackrabbit.core.RepositoryImpl.initWorkspace(RepositoryImpl.java:540)
	at
org.apache.jackrabbit.core.RepositoryImpl.<init>(RepositoryImpl.java:245)
	at
org.apache.jackrabbit.core.RepositoryImpl.create(RepositoryImpl.java:484)
	at
org.apache.jackrabbit.core.jndi.BindableRepository.init(BindableRepository.java:136)
	at
org.apache.jackrabbit.core.jndi.BindableRepository.create(BindableRepository.java:122)
	at
org.apache.jackrabbit.core.jndi.BindableRepositoryFactory.createInstance(BindableRepositoryFactory.java:58)
	at
org.apache.jackrabbit.core.jndi.RegistryHelper.registerRepository(RegistryHelper.java:59)
	at
it.wavegroup.imco.services.RepositoryInitService.registerRepository(Unknown Source)
	at
it.wavegroup.imco.jackrabbit.RepositoryInit.startupRepository(Unknown
Source)
	at it.wavegroup.imco.servlets.ApplicationInit.startupRepository(Unknown
Source)
	at
it.wavegroup.imco.servlets.ApplicationInit.contextInitialized(Unknown
Source)
	at
org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:3827)
	at
org.apache.catalina.core.StandardContext.start(StandardContext.java:4343)
	at
org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:823)
	at
org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:807)
	at
org.apache.catalina.core.StandardHost.addChild(StandardHost.java:595)
	at
org.apache.catalina.core.StandardHostDeployer.addChild(StandardHostDeployer.java:903)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
	at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:324)
	at
org.apache.commons.beanutils.MethodUtils.invokeMethod(MethodUtils.java:216)
	at org.apache.commons.digester.SetNextRule.end(SetNextRule.java:256)
	at org.apache.commons.digester.Rule.end(Rule.java:276)
	at org.apache.commons.digester.Digester.endElement(Digester.java:1058)
	at
org.apache.catalina.util.CatalinaDigester.endElement(CatalinaDigester.java:76)
	at org.apache.xerces.parsers.AbstractSAXParser.endElement(Unknown
Source)
	at
org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.scanEndElement(Unknown Source)
	at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl
$FragmentContentDispatcher.dispatch(Unknown Source)
	at
org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.scanDocument(Unknown Source)
	at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
	at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
	at org.apache.xerces.parsers.XMLParser.parse(Unknown Source)
	at org.apache.xerces.parsers.AbstractSAXParser.parse(Unknown Source)
	at org.apache.commons.digester.Digester.parse(Digester.java:1567)
	at
org.apache.catalina.core.StandardHostDeployer.install(StandardHostDeployer.java:488)
	at
org.apache.catalina.core.StandardHost.install(StandardHost.java:863)
	at
org.apache.catalina.startup.HostConfig.deployDescriptors(HostConfig.java:483)
	at
org.apache.catalina.startup.HostConfig.deployApps(HostConfig.java:427)
	at org.apache.catalina.startup.HostConfig.start(HostConfig.java:983)
	at
org.apache.catalina.startup.HostConfig.lifecycleEvent(HostConfig.java:349)
	at
org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSupport.java:119)
	at
org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1091)
	at org.apache.catalina.core.StandardHost.start(StandardHost.java:789)
	at
org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1083)
	at
org.apache.catalina.core.StandardEngine.start(StandardEngine.java:478)
	at
org.apache.catalina.core.StandardService.start(StandardService.java:480)
	at
org.apache.catalina.core.StandardServer.start(StandardServer.java:2313)
	at org.apache.catalina.startup.Catalina.start(Catalina.java:556)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
	at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:324)
	at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:287)
	at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:425)
[ WARN] 20060803 08:24:01 (org.apache.jackrabbit.core.query.LazyReader)
- exception initializing reader
org.apache.jackrabbit.core.query.MsWordTextFilter$1:
java.io.IOException: Invalid header signature; read 8656037701166316554,
expected -2226271756974174256



Then there are a lot of nullpointer exceptions

[DEBUG] 20060803 08:37:17 (org.apache.jackrabbit.core.ItemManager) -
removing item 8637bf5f-4689-4e75-888f-b7b89bef40c8 from cache
[ WARN] 20060803 08:40:13 (org.apache.jackrabbit.core.RepositoryImpl) -
Existing lock file at C:\Wave\Repository\.lock deteteced. Repository was
not shut down properly.
[ERROR] 20060803 09:33:14
(org.apache.jackrabbit.core.observation.ObservationManagerFactory) -
Synchronous EventConsumer threw exception.
java.lang.NullPointerException: null values not allowed
	at
org.apache.commons.collections.map.AbstractReferenceMap.put(AbstractReferenceMap.java:251)
	at
org.apache.jackrabbit.core.version.VersionManagerImpl.getItem(VersionManagerImpl.java:242)
	at
org.apache.jackrabbit.core.version.AbstractVersionManager.getVersion(AbstractVersionManager.java:62)
	at
org.apache.jackrabbit.core.version.VersionItemStateProvider.getItemState(VersionItemStateProvider.java:131)
	at
org.apache.jackrabbit.core.state.SharedItemStateManager.getItemState(SharedItemStateManager.java:212)
	at
org.apache.jackrabbit.core.state.LocalItemStateManager.getPropertyState(LocalItemStateManager.java:111)
	at
org.apache.jackrabbit.core.state.LocalItemStateManager.getItemState(LocalItemStateManager.java:143)
	at
org.apache.jackrabbit.core.state.SessionItemStateManager.getItemState(SessionItemStateManager.java:127)
	at
org.apache.jackrabbit.core.query.lucene.NodeIndexer.createDoc(NodeIndexer.java:155)
	at
org.apache.jackrabbit.core.query.lucene.NodeIndexer.createDocument(NodeIndexer.java:112)
	at
org.apache.jackrabbit.core.query.lucene.SearchIndex.createDocument(SearchIndex.java:449)
	at org.apache.jackrabbit.core.query.lucene.SearchIndex
$2.next(SearchIndex.java:292)
	at
org.apache.jackrabbit.core.query.lucene.MultiIndex.update(MultiIndex.java:313)
	at
org.apache.jackrabbit.core.query.lucene.SearchIndex.updateNodes(SearchIndex.java:279)
	at
org.apache.jackrabbit.core.SearchManager.onEvent(SearchManager.java:476)
	at
org.apache.jackrabbit.core.observation.EventConsumer.consumeEvents(EventConsumer.java:230)
	at
org.apache.jackrabbit.core.observation.ObservationManagerFactory.dispatchEvents(ObservationManagerFactory.java:218)
	at
org.apache.jackrabbit.core.observation.EventStateCollection.dispatch(EventStateCollection.java:430)
	at
org.apache.jackrabbit.core.observation.DelegatingObservationDispatcher.dispatch(DelegatingObservationDispatcher.java:105)
	at
org.apache.jackrabbit.core.observation.DelegatingObservationDispatcher.dispatchEvents(DelegatingObservationDispatcher.java:87)
	at
org.apache.jackrabbit.core.observation.EventStateCollection.dispatch(EventStateCollection.java:430)
	at org.apache.jackrabbit.core.state.SharedItemStateManager
$Update.end(SharedItemStateManager.java:607)
	at
org.apache.jackrabbit.core.state.XAItemStateManager.commit(XAItemStateManager.java:170)
	at
org.apache.jackrabbit.core.version.XAVersionManager.commit(XAVersionManager.java:440)
	at
org.apache.jackrabbit.core.TransactionContext.commit(TransactionContext.java:172)
	at
org.apache.jackrabbit.core.XASessionImpl.commit(XASessionImpl.java:324)
	at
it.wavegroup.imco.jackrabbit.transaction.UserTransactionImpl.commit(Unknown Source)
	at
it.wavegroup.imco.services.SessionFacade.commitJackrabbitTransaction(Unknown Source)
	at it.wavegroup.imco.services.RepositoryService.removeNode(Unknown
Source)
	at it.wavegroup.imco.actions.FilesAction.delete(Unknown Source)
	at it.wavegroup.imco.actions.BaseAction.selectForward(Unknown Source)
	at it.wavegroup.imco.actions.FilesAction.selectForward(Unknown Source)
	at it.wavegroup.imco.actions.BaseAction.execute(Unknown Source)
	at
org.apache.struts.action.RequestProcessor.processActionPerform(RequestProcessor.java:421)
	at
org.apache.struts.action.RequestProcessor.process(RequestProcessor.java:226)
	at
org.apache.struts.action.ActionServlet.process(ActionServlet.java:1164)
	at
org.apache.struts.action.ActionServlet.doGet(ActionServlet.java:397)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:689)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:802)
	at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:237)
	at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:157)
	at it.wavegroup.imco.servlets.AuthenticationFilter.doFilter(Unknown
Source)
	at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:186)
	at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:157)
	at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:214)
	at
org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveContext.java:104)
	at
org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:520)
	at
org.apache.catalina.core.StandardContextValve.invokeInternal(StandardContextValve.java:198)
	at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:152)
	at
org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveContext.java:104)
	at
org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:540)
	at
org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveContext.java:102)
	at
org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:520)
	at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:137)
	at
org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveContext.java:104)
	at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:118)
	at
org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveContext.java:102)
	at
org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:520)
	at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
	at
org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveContext.java:104)
	at
org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:520)
	at
org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:929)
	at
org.apache.coyote.tomcat5.CoyoteAdapter.service(CoyoteAdapter.java:160)
	at
org.apache.jk.server.JkCoyoteHandler.invoke(JkCoyoteHandler.java:300)
	at org.apache.jk.common.HandlerRequest.invoke(HandlerRequest.java:374)
	at org.apache.jk.common.ChannelSocket.invoke(ChannelSocket.java:743)
	at
org.apache.jk.common.ChannelSocket.processConnection(ChannelSocket.java:675)
	at org.apache.jk.common.SocketConnection.runIt(ChannelSocket.java:866)
	at org.apache.tomcat.util.threads.ThreadPool
$ControlRunnable.run(ThreadPool.java:683)
	at java.lang.Thread.run(Thread.java:534)

and last there are the outofMemory errors

[ERROR] 20060825 11:27:53
(org.apache.jackrabbit.core.observation.ObservationManagerFactory) -
Synchronous EventConsumer threw exception.
java.lang.OutOfMemoryError
...
[ERROR] 20060825 18:04:21
(org.apache.jackrabbit.core.observation.ObservationManagerFactory) -
Synchronous EventConsumer threw exception.
java.lang.OutOfMemoryError
[ WARN] 20060826 00:39:03 (org.apache.jackrabbit.core.RepositoryImpl) -
Existing lock file at C:\Wave\Repository\.lock deteteced. Repository was
not shut down properly.


All there errors happened a lot of times (a lot means 10, 20 times each
one), but I can't understand if all these causes the Out of Memory.
Maybe after the exception resources are never free, so after a while the
system exaust the memory, but also the number of errors is not so high
(in my opinion). I think there is something else (maybe something
similar with the sessions between jackrabbit 0.9 and 1.0).




> Jukka Zitting

chris
> 


Re: problems with re-indexing the workspace

Posted by Jukka Zitting <ju...@gmail.com>.
Hi,

On 8/28/06, Christian Zanata <ch...@wavegroup.it> wrote:
> [ERROR] 20060825 17:06:40
> (org.apache.jackrabbit.core.observation.ObservationManagerFactory) -
> Synchronous EventConsumer threw exception. java.lang.OutOfMemoryError
>
> This error seems happening when the repository tries to re-index the
> workspace, but we don't have more stack traces.
> [...]
> could anybody heps us to understand what's happening?

There are two likely causes for that; either Lucene is running out of
memory while merging the index segments, or one of the index filters
runs out of memory trying to parse one of the binary documents in the
repository. Without a complete stack trace it is difficult to
determine the exact cause of the problems.

You might want to try modifying the Lucene parameters in the
SearchIndex configuration. See the Lucene documentation for options
that affect memory usage.

BR,

Jukka Zitting

-- 
Yukatan - http://yukatan.fi/ - info@yukatan.fi
Software craftsmanship, JCR consulting, and Java development