You are viewing a plain text version of this content. The canonical link for it is here.
Posted to java-user@lucene.apache.org by Matthew Petersen <md...@gmail.com> on 2014/01/16 23:33:33 UTC
Issue with FacetFields.addFields() throwing ArrayIndexOutOfBoundsException
I’m having an issue with an index when adding category paths to a document.
They seem to be added without issue for a long period of time, then for
some unknown reason the addition fails with an ArrayIndexOutOfBounds
exception. Subsequent attempts to add category paths fail with the same
exception. I’ve run CheckIndex on both the index and the taxonomy
directory and both come back as clean with no issues. I cannot fix the
index because according to lucene it is not broken. Could this be a bug in
lucene? Below is the stack trace when the exception occurs:
Lucene v4.4.0
java.lang.ArrayIndexOutOfBoundsException: -65535
at java.util.ArrayList.elementData(ArrayList.java:371)
at java.util.ArrayList.get(ArrayList.java:384)
at
org.apache.lucene.facet.taxonomy.writercache.cl2o.CharBlockArray.charAt(CharBlockArray.java:152)
at
org.apache.lucene.facet.taxonomy.writercache.cl2o.CategoryPathUtils.equalsToSerialized(CategoryPathUtils.java:61)
at
org.apache.lucene.facet.taxonomy.writercache.cl2o.CompactLabelToOrdinal.getOrdinal(CompactLabelToOrdinal.java:257)
at
org.apache.lucene.facet.taxonomy.writercache.cl2o.CompactLabelToOrdinal.getOrdinal(CompactLabelToOrdinal.java:140)
at
org.apache.lucene.facet.taxonomy.writercache.cl2o.Cl2oTaxonomyWriterCache.get(Cl2oTaxonomyWriterCache.java:74)
at
org.apache.lucene.facet.taxonomy.directory.DirectoryTaxonomyWriter.addCategory(DirectoryTaxonomyWriter.java:455)
at org.apache.lucene.facet.index.FacetFields.addFields(FacetFields.java:175)
at
com.logrhythm.messaging.indexing.LogIndexerImpl.getDocument(LogIndexerImpl.java:478)
at
com.logrhythm.messaging.indexing.LogIndexerImpl.indexLog(LogIndexerImpl.java:392)
at
com.logrhythm.messaging.indexing.LogIndexerImpl.indexLogs(LogIndexerImpl.java:357)
at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at
com.logrhythm.tests.unit.messaging.indexing.LogIndexerTests.logIndexerLoadTest(LogIndexerTests.java:752)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
at org.junit.runner.JUnitCore.run(JUnitCore.java:157)
at
com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:74)
at
com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:202)
at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:65)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at com.intellij.rt.execution.application.AppMain.main(AppMain.java:120)
Re: Issue with FacetFields.addFields() throwing ArrayIndexOutOfBoundsException
Posted by Matthew Petersen <md...@gmail.com>.
Will do. I need to sanitize the unit test a bit so it might be a few days.
On Fri, Jan 17, 2014 at 9:24 PM, Shai Erera <se...@gmail.com> wrote:
> Can you open an issue and attach the test there?
> On Jan 18, 2014 12:41 AM, "Matthew Petersen" <md...@gmail.com> wrote:
>
> > I do have a test that will reproduce. I'm not adding categories with
> very
> > deep hierarchies, I'm adding 129 category paths per document (all docs
> have
> > paths with same label) with each path having one value. All of the
> values
> > are completely random and likely unique. It's basically a worst case
> test
> > for our app but the condition has been seen in the field (the error has
> > been encountered at less than worst case scenario). The test i have
> > reproduces it very quickly, Only have to index ~330K docs.
> >
> >
> > On Fri, Jan 17, 2014 at 3:27 PM, Shai Erera <se...@gmail.com> wrote:
> >
> > > Do you have a test which reproduces the error? Are you adding
> categories
> > > with very deep hierarchies?
> > >
> > > Shai
> > >
> > >
> > > On Fri, Jan 17, 2014 at 11:59 PM, Matthew Petersen <mdpete4@gmail.com
> > > >wrote:
> > >
> > > > I've confirmed that using the LruTaxonomyWriterCache solves the issue
> > for
> > > > me. It would appear there is in fact a bug in the
> > > Cl20TaxonomyWriterCache
> > > > or I am using it incorrectly (I use it as default, no customization).
> > > >
> > > >
> > > > On Fri, Jan 17, 2014 at 9:29 AM, Matthew Petersen <mdpete4@gmail.com
> >
> > > > wrote:
> > > >
> > > > > I'm sure. I had seen that issue and it looked similar but the
> stack
> > > > trace
> > > > > is slightly different. I've found that if I replace the
> > > > > Cl2oTaxonomyWriterCache with the LruTaxonomyWriterCache the problem
> > > seems
> > > > > to go away. I'm working right now on running a test that will
> prove
> > > this
> > > > > but it takes a while as the cache needs to get very large. If this
> > > > proves
> > > > > to solve the problem then I'd say there is still a bug in the
> > > > > Cl2oTaxonomyWriterCache implementation.
> > > > >
> > > > > Thanks for the response.
> > > > > Matt
> > > > >
> > > > >
> > > > > On Fri, Jan 17, 2014 at 6:36 AM, Michael McCandless <
> > > > > lucene@mikemccandless.com> wrote:
> > > > >
> > > > >> Are you sure you're using 4.4?
> > > > >>
> > > > >> Because ... this looks like
> > > > >> https://issues.apache.org/jira/browse/LUCENE-5048 but that was
> > > > >> supposedly fixed in 4.4.
> > > > >>
> > > > >> Mike McCandless
> > > > >>
> > > > >> http://blog.mikemccandless.com
> > > > >>
> > > > >>
> > > > >> On Thu, Jan 16, 2014 at 5:33 PM, Matthew Petersen <
> > mdpete4@gmail.com>
> > > > >> wrote:
> > > > >> > I’m having an issue with an index when adding category paths to
> a
> > > > >> document.
> > > > >> > They seem to be added without issue for a long period of time,
> > then
> > > > for
> > > > >> > some unknown reason the addition fails with an
> > ArrayIndexOutOfBounds
> > > > >> > exception. Subsequent attempts to add category paths fail with
> > the
> > > > same
> > > > >> > exception. I’ve run CheckIndex on both the index and the
> taxonomy
> > > > >> > directory and both come back as clean with no issues. I cannot
> > fix
> > > > the
> > > > >> > index because according to lucene it is not broken. Could this
> > be a
> > > > >> bug in
> > > > >> > lucene? Below is the stack trace when the exception occurs:
> > > > >> >
> > > > >> >
> > > > >> > Lucene v4.4.0
> > > > >> >
> > > > >> >
> > > > >> > java.lang.ArrayIndexOutOfBoundsException: -65535
> > > > >> >
> > > > >> > at java.util.ArrayList.elementData(ArrayList.java:371)
> > > > >> >
> > > > >> > at java.util.ArrayList.get(ArrayList.java:384)
> > > > >> >
> > > > >> > at
> > > > >> >
> > > > >>
> > > >
> > >
> >
> org.apache.lucene.facet.taxonomy.writercache.cl2o.CharBlockArray.charAt(CharBlockArray.java:152)
> > > > >> >
> > > > >> > at
> > > > >> >
> > > > >>
> > > >
> > >
> >
> org.apache.lucene.facet.taxonomy.writercache.cl2o.CategoryPathUtils.equalsToSerialized(CategoryPathUtils.java:61)
> > > > >> >
> > > > >> > at
> > > > >> >
> > > > >>
> > > >
> > >
> >
> org.apache.lucene.facet.taxonomy.writercache.cl2o.CompactLabelToOrdinal.getOrdinal(CompactLabelToOrdinal.java:257)
> > > > >> >
> > > > >> > at
> > > > >> >
> > > > >>
> > > >
> > >
> >
> org.apache.lucene.facet.taxonomy.writercache.cl2o.CompactLabelToOrdinal.getOrdinal(CompactLabelToOrdinal.java:140)
> > > > >> >
> > > > >> > at
> > > > >> >
> > > > >>
> > > >
> > >
> >
> org.apache.lucene.facet.taxonomy.writercache.cl2o.Cl2oTaxonomyWriterCache.get(Cl2oTaxonomyWriterCache.java:74)
> > > > >> >
> > > > >> > at
> > > > >> >
> > > > >>
> > > >
> > >
> >
> org.apache.lucene.facet.taxonomy.directory.DirectoryTaxonomyWriter.addCategory(DirectoryTaxonomyWriter.java:455)
> > > > >> >
> > > > >> > at
> > > > >>
> > > >
> > org.apache.lucene.facet.index.FacetFields.addFields(FacetFields.java:175)
> > > > >> >
> > > > >> > at
> > > > >> >
> > > > >>
> > > >
> > >
> >
> com.logrhythm.messaging.indexing.LogIndexerImpl.getDocument(LogIndexerImpl.java:478)
> > > > >> >
> > > > >> > at
> > > > >> >
> > > > >>
> > > >
> > >
> >
> com.logrhythm.messaging.indexing.LogIndexerImpl.indexLog(LogIndexerImpl.java:392)
> > > > >> >
> > > > >> > at
> > > > >> >
> > > > >>
> > > >
> > >
> >
> com.logrhythm.messaging.indexing.LogIndexerImpl.indexLogs(LogIndexerImpl.java:357)
> > > > >> >
> > > > >> > at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source)
> > > > >> >
> > > > >> > at
> > > > >> >
> > > > >>
> > > >
> > >
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> > > > >> >
> > > > >> > at java.lang.reflect.Method.invoke(Method.java:601)
> > > > >> >
> > > > >> > at
> > > > >> >
> > > > >>
> > > >
> > >
> >
> com.logrhythm.tests.unit.messaging.indexing.LogIndexerTests.logIndexerLoadTest(LogIndexerTests.java:752)
> > > > >> >
> > > > >> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > > > >> >
> > > > >> > at
> > > > >> >
> > > > >>
> > > >
> > >
> >
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> > > > >> >
> > > > >> > at
> > > > >> >
> > > > >>
> > > >
> > >
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> > > > >> >
> > > > >> > at java.lang.reflect.Method.invoke(Method.java:601)
> > > > >> >
> > > > >> > at
> > > > >> >
> > > > >>
> > > >
> > >
> >
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
> > > > >> >
> > > > >> > at
> > > > >> >
> > > > >>
> > > >
> > >
> >
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
> > > > >> >
> > > > >> > at
> > > > >> >
> > > > >>
> > > >
> > >
> >
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
> > > > >> >
> > > > >> > at
> > > > >> >
> > > > >>
> > > >
> > >
> >
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
> > > > >> >
> > > > >> > at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
> > > > >> >
> > > > >> > at
> > > > >> >
> > > > >>
> > > >
> > >
> >
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68)
> > > > >> >
> > > > >> > at
> > > > >> >
> > > > >>
> > > >
> > >
> >
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47)
> > > > >> >
> > > > >> > at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
> > > > >> >
> > > > >> > at
> org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
> > > > >> >
> > > > >> > at
> > org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
> > > > >> >
> > > > >> > at
> org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
> > > > >> >
> > > > >> > at
> > org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
> > > > >> >
> > > > >> > at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
> > > > >> >
> > > > >> > at org.junit.runner.JUnitCore.run(JUnitCore.java:157)
> > > > >> >
> > > > >> > at
> > > > >> >
> > > > >>
> > > >
> > >
> >
> com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:74)
> > > > >> >
> > > > >> > at
> > > > >> >
> > > > >>
> > > >
> > >
> >
> com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:202)
> > > > >> >
> > > > >> > at
> > > > >>
> > > com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:65)
> > > > >> >
> > > > >> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > > > >> >
> > > > >> > at
> > > > >> >
> > > > >>
> > > >
> > >
> >
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> > > > >> >
> > > > >> > at
> > > > >> >
> > > > >>
> > > >
> > >
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> > > > >> >
> > > > >> > at java.lang.reflect.Method.invoke(Method.java:601)
> > > > >> >
> > > > >> > at
> > > > com.intellij.rt.execution.application.AppMain.main(AppMain.java:120)
> > > > >>
> > > > >>
> > ---------------------------------------------------------------------
> > > > >> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> > > > >> For additional commands, e-mail: java-user-help@lucene.apache.org
> > > > >>
> > > > >>
> > > > >
> > > >
> > >
> >
>
Re: Issue with FacetFields.addFields() throwing ArrayIndexOutOfBoundsException
Posted by Shai Erera <se...@gmail.com>.
Can you open an issue and attach the test there?
On Jan 18, 2014 12:41 AM, "Matthew Petersen" <md...@gmail.com> wrote:
> I do have a test that will reproduce. I'm not adding categories with very
> deep hierarchies, I'm adding 129 category paths per document (all docs have
> paths with same label) with each path having one value. All of the values
> are completely random and likely unique. It's basically a worst case test
> for our app but the condition has been seen in the field (the error has
> been encountered at less than worst case scenario). The test i have
> reproduces it very quickly, Only have to index ~330K docs.
>
>
> On Fri, Jan 17, 2014 at 3:27 PM, Shai Erera <se...@gmail.com> wrote:
>
> > Do you have a test which reproduces the error? Are you adding categories
> > with very deep hierarchies?
> >
> > Shai
> >
> >
> > On Fri, Jan 17, 2014 at 11:59 PM, Matthew Petersen <mdpete4@gmail.com
> > >wrote:
> >
> > > I've confirmed that using the LruTaxonomyWriterCache solves the issue
> for
> > > me. It would appear there is in fact a bug in the
> > Cl20TaxonomyWriterCache
> > > or I am using it incorrectly (I use it as default, no customization).
> > >
> > >
> > > On Fri, Jan 17, 2014 at 9:29 AM, Matthew Petersen <md...@gmail.com>
> > > wrote:
> > >
> > > > I'm sure. I had seen that issue and it looked similar but the stack
> > > trace
> > > > is slightly different. I've found that if I replace the
> > > > Cl2oTaxonomyWriterCache with the LruTaxonomyWriterCache the problem
> > seems
> > > > to go away. I'm working right now on running a test that will prove
> > this
> > > > but it takes a while as the cache needs to get very large. If this
> > > proves
> > > > to solve the problem then I'd say there is still a bug in the
> > > > Cl2oTaxonomyWriterCache implementation.
> > > >
> > > > Thanks for the response.
> > > > Matt
> > > >
> > > >
> > > > On Fri, Jan 17, 2014 at 6:36 AM, Michael McCandless <
> > > > lucene@mikemccandless.com> wrote:
> > > >
> > > >> Are you sure you're using 4.4?
> > > >>
> > > >> Because ... this looks like
> > > >> https://issues.apache.org/jira/browse/LUCENE-5048 but that was
> > > >> supposedly fixed in 4.4.
> > > >>
> > > >> Mike McCandless
> > > >>
> > > >> http://blog.mikemccandless.com
> > > >>
> > > >>
> > > >> On Thu, Jan 16, 2014 at 5:33 PM, Matthew Petersen <
> mdpete4@gmail.com>
> > > >> wrote:
> > > >> > I’m having an issue with an index when adding category paths to a
> > > >> document.
> > > >> > They seem to be added without issue for a long period of time,
> then
> > > for
> > > >> > some unknown reason the addition fails with an
> ArrayIndexOutOfBounds
> > > >> > exception. Subsequent attempts to add category paths fail with
> the
> > > same
> > > >> > exception. I’ve run CheckIndex on both the index and the taxonomy
> > > >> > directory and both come back as clean with no issues. I cannot
> fix
> > > the
> > > >> > index because according to lucene it is not broken. Could this
> be a
> > > >> bug in
> > > >> > lucene? Below is the stack trace when the exception occurs:
> > > >> >
> > > >> >
> > > >> > Lucene v4.4.0
> > > >> >
> > > >> >
> > > >> > java.lang.ArrayIndexOutOfBoundsException: -65535
> > > >> >
> > > >> > at java.util.ArrayList.elementData(ArrayList.java:371)
> > > >> >
> > > >> > at java.util.ArrayList.get(ArrayList.java:384)
> > > >> >
> > > >> > at
> > > >> >
> > > >>
> > >
> >
> org.apache.lucene.facet.taxonomy.writercache.cl2o.CharBlockArray.charAt(CharBlockArray.java:152)
> > > >> >
> > > >> > at
> > > >> >
> > > >>
> > >
> >
> org.apache.lucene.facet.taxonomy.writercache.cl2o.CategoryPathUtils.equalsToSerialized(CategoryPathUtils.java:61)
> > > >> >
> > > >> > at
> > > >> >
> > > >>
> > >
> >
> org.apache.lucene.facet.taxonomy.writercache.cl2o.CompactLabelToOrdinal.getOrdinal(CompactLabelToOrdinal.java:257)
> > > >> >
> > > >> > at
> > > >> >
> > > >>
> > >
> >
> org.apache.lucene.facet.taxonomy.writercache.cl2o.CompactLabelToOrdinal.getOrdinal(CompactLabelToOrdinal.java:140)
> > > >> >
> > > >> > at
> > > >> >
> > > >>
> > >
> >
> org.apache.lucene.facet.taxonomy.writercache.cl2o.Cl2oTaxonomyWriterCache.get(Cl2oTaxonomyWriterCache.java:74)
> > > >> >
> > > >> > at
> > > >> >
> > > >>
> > >
> >
> org.apache.lucene.facet.taxonomy.directory.DirectoryTaxonomyWriter.addCategory(DirectoryTaxonomyWriter.java:455)
> > > >> >
> > > >> > at
> > > >>
> > >
> org.apache.lucene.facet.index.FacetFields.addFields(FacetFields.java:175)
> > > >> >
> > > >> > at
> > > >> >
> > > >>
> > >
> >
> com.logrhythm.messaging.indexing.LogIndexerImpl.getDocument(LogIndexerImpl.java:478)
> > > >> >
> > > >> > at
> > > >> >
> > > >>
> > >
> >
> com.logrhythm.messaging.indexing.LogIndexerImpl.indexLog(LogIndexerImpl.java:392)
> > > >> >
> > > >> > at
> > > >> >
> > > >>
> > >
> >
> com.logrhythm.messaging.indexing.LogIndexerImpl.indexLogs(LogIndexerImpl.java:357)
> > > >> >
> > > >> > at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source)
> > > >> >
> > > >> > at
> > > >> >
> > > >>
> > >
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> > > >> >
> > > >> > at java.lang.reflect.Method.invoke(Method.java:601)
> > > >> >
> > > >> > at
> > > >> >
> > > >>
> > >
> >
> com.logrhythm.tests.unit.messaging.indexing.LogIndexerTests.logIndexerLoadTest(LogIndexerTests.java:752)
> > > >> >
> > > >> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > > >> >
> > > >> > at
> > > >> >
> > > >>
> > >
> >
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> > > >> >
> > > >> > at
> > > >> >
> > > >>
> > >
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> > > >> >
> > > >> > at java.lang.reflect.Method.invoke(Method.java:601)
> > > >> >
> > > >> > at
> > > >> >
> > > >>
> > >
> >
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
> > > >> >
> > > >> > at
> > > >> >
> > > >>
> > >
> >
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
> > > >> >
> > > >> > at
> > > >> >
> > > >>
> > >
> >
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
> > > >> >
> > > >> > at
> > > >> >
> > > >>
> > >
> >
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
> > > >> >
> > > >> > at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
> > > >> >
> > > >> > at
> > > >> >
> > > >>
> > >
> >
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68)
> > > >> >
> > > >> > at
> > > >> >
> > > >>
> > >
> >
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47)
> > > >> >
> > > >> > at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
> > > >> >
> > > >> > at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
> > > >> >
> > > >> > at
> org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
> > > >> >
> > > >> > at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
> > > >> >
> > > >> > at
> org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
> > > >> >
> > > >> > at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
> > > >> >
> > > >> > at org.junit.runner.JUnitCore.run(JUnitCore.java:157)
> > > >> >
> > > >> > at
> > > >> >
> > > >>
> > >
> >
> com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:74)
> > > >> >
> > > >> > at
> > > >> >
> > > >>
> > >
> >
> com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:202)
> > > >> >
> > > >> > at
> > > >>
> > com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:65)
> > > >> >
> > > >> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > > >> >
> > > >> > at
> > > >> >
> > > >>
> > >
> >
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> > > >> >
> > > >> > at
> > > >> >
> > > >>
> > >
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> > > >> >
> > > >> > at java.lang.reflect.Method.invoke(Method.java:601)
> > > >> >
> > > >> > at
> > > com.intellij.rt.execution.application.AppMain.main(AppMain.java:120)
> > > >>
> > > >>
> ---------------------------------------------------------------------
> > > >> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> > > >> For additional commands, e-mail: java-user-help@lucene.apache.org
> > > >>
> > > >>
> > > >
> > >
> >
>
Re: Issue with FacetFields.addFields() throwing ArrayIndexOutOfBoundsException
Posted by Matthew Petersen <md...@gmail.com>.
I do have a test that will reproduce. I'm not adding categories with very
deep hierarchies, I'm adding 129 category paths per document (all docs have
paths with same label) with each path having one value. All of the values
are completely random and likely unique. It's basically a worst case test
for our app but the condition has been seen in the field (the error has
been encountered at less than worst case scenario). The test i have
reproduces it very quickly, Only have to index ~330K docs.
On Fri, Jan 17, 2014 at 3:27 PM, Shai Erera <se...@gmail.com> wrote:
> Do you have a test which reproduces the error? Are you adding categories
> with very deep hierarchies?
>
> Shai
>
>
> On Fri, Jan 17, 2014 at 11:59 PM, Matthew Petersen <mdpete4@gmail.com
> >wrote:
>
> > I've confirmed that using the LruTaxonomyWriterCache solves the issue for
> > me. It would appear there is in fact a bug in the
> Cl20TaxonomyWriterCache
> > or I am using it incorrectly (I use it as default, no customization).
> >
> >
> > On Fri, Jan 17, 2014 at 9:29 AM, Matthew Petersen <md...@gmail.com>
> > wrote:
> >
> > > I'm sure. I had seen that issue and it looked similar but the stack
> > trace
> > > is slightly different. I've found that if I replace the
> > > Cl2oTaxonomyWriterCache with the LruTaxonomyWriterCache the problem
> seems
> > > to go away. I'm working right now on running a test that will prove
> this
> > > but it takes a while as the cache needs to get very large. If this
> > proves
> > > to solve the problem then I'd say there is still a bug in the
> > > Cl2oTaxonomyWriterCache implementation.
> > >
> > > Thanks for the response.
> > > Matt
> > >
> > >
> > > On Fri, Jan 17, 2014 at 6:36 AM, Michael McCandless <
> > > lucene@mikemccandless.com> wrote:
> > >
> > >> Are you sure you're using 4.4?
> > >>
> > >> Because ... this looks like
> > >> https://issues.apache.org/jira/browse/LUCENE-5048 but that was
> > >> supposedly fixed in 4.4.
> > >>
> > >> Mike McCandless
> > >>
> > >> http://blog.mikemccandless.com
> > >>
> > >>
> > >> On Thu, Jan 16, 2014 at 5:33 PM, Matthew Petersen <md...@gmail.com>
> > >> wrote:
> > >> > I’m having an issue with an index when adding category paths to a
> > >> document.
> > >> > They seem to be added without issue for a long period of time, then
> > for
> > >> > some unknown reason the addition fails with an ArrayIndexOutOfBounds
> > >> > exception. Subsequent attempts to add category paths fail with the
> > same
> > >> > exception. I’ve run CheckIndex on both the index and the taxonomy
> > >> > directory and both come back as clean with no issues. I cannot fix
> > the
> > >> > index because according to lucene it is not broken. Could this be a
> > >> bug in
> > >> > lucene? Below is the stack trace when the exception occurs:
> > >> >
> > >> >
> > >> > Lucene v4.4.0
> > >> >
> > >> >
> > >> > java.lang.ArrayIndexOutOfBoundsException: -65535
> > >> >
> > >> > at java.util.ArrayList.elementData(ArrayList.java:371)
> > >> >
> > >> > at java.util.ArrayList.get(ArrayList.java:384)
> > >> >
> > >> > at
> > >> >
> > >>
> >
> org.apache.lucene.facet.taxonomy.writercache.cl2o.CharBlockArray.charAt(CharBlockArray.java:152)
> > >> >
> > >> > at
> > >> >
> > >>
> >
> org.apache.lucene.facet.taxonomy.writercache.cl2o.CategoryPathUtils.equalsToSerialized(CategoryPathUtils.java:61)
> > >> >
> > >> > at
> > >> >
> > >>
> >
> org.apache.lucene.facet.taxonomy.writercache.cl2o.CompactLabelToOrdinal.getOrdinal(CompactLabelToOrdinal.java:257)
> > >> >
> > >> > at
> > >> >
> > >>
> >
> org.apache.lucene.facet.taxonomy.writercache.cl2o.CompactLabelToOrdinal.getOrdinal(CompactLabelToOrdinal.java:140)
> > >> >
> > >> > at
> > >> >
> > >>
> >
> org.apache.lucene.facet.taxonomy.writercache.cl2o.Cl2oTaxonomyWriterCache.get(Cl2oTaxonomyWriterCache.java:74)
> > >> >
> > >> > at
> > >> >
> > >>
> >
> org.apache.lucene.facet.taxonomy.directory.DirectoryTaxonomyWriter.addCategory(DirectoryTaxonomyWriter.java:455)
> > >> >
> > >> > at
> > >>
> > org.apache.lucene.facet.index.FacetFields.addFields(FacetFields.java:175)
> > >> >
> > >> > at
> > >> >
> > >>
> >
> com.logrhythm.messaging.indexing.LogIndexerImpl.getDocument(LogIndexerImpl.java:478)
> > >> >
> > >> > at
> > >> >
> > >>
> >
> com.logrhythm.messaging.indexing.LogIndexerImpl.indexLog(LogIndexerImpl.java:392)
> > >> >
> > >> > at
> > >> >
> > >>
> >
> com.logrhythm.messaging.indexing.LogIndexerImpl.indexLogs(LogIndexerImpl.java:357)
> > >> >
> > >> > at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source)
> > >> >
> > >> > at
> > >> >
> > >>
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> > >> >
> > >> > at java.lang.reflect.Method.invoke(Method.java:601)
> > >> >
> > >> > at
> > >> >
> > >>
> >
> com.logrhythm.tests.unit.messaging.indexing.LogIndexerTests.logIndexerLoadTest(LogIndexerTests.java:752)
> > >> >
> > >> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > >> >
> > >> > at
> > >> >
> > >>
> >
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> > >> >
> > >> > at
> > >> >
> > >>
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> > >> >
> > >> > at java.lang.reflect.Method.invoke(Method.java:601)
> > >> >
> > >> > at
> > >> >
> > >>
> >
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
> > >> >
> > >> > at
> > >> >
> > >>
> >
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
> > >> >
> > >> > at
> > >> >
> > >>
> >
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
> > >> >
> > >> > at
> > >> >
> > >>
> >
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
> > >> >
> > >> > at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
> > >> >
> > >> > at
> > >> >
> > >>
> >
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68)
> > >> >
> > >> > at
> > >> >
> > >>
> >
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47)
> > >> >
> > >> > at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
> > >> >
> > >> > at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
> > >> >
> > >> > at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
> > >> >
> > >> > at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
> > >> >
> > >> > at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
> > >> >
> > >> > at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
> > >> >
> > >> > at org.junit.runner.JUnitCore.run(JUnitCore.java:157)
> > >> >
> > >> > at
> > >> >
> > >>
> >
> com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:74)
> > >> >
> > >> > at
> > >> >
> > >>
> >
> com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:202)
> > >> >
> > >> > at
> > >>
> com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:65)
> > >> >
> > >> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > >> >
> > >> > at
> > >> >
> > >>
> >
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> > >> >
> > >> > at
> > >> >
> > >>
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> > >> >
> > >> > at java.lang.reflect.Method.invoke(Method.java:601)
> > >> >
> > >> > at
> > com.intellij.rt.execution.application.AppMain.main(AppMain.java:120)
> > >>
> > >> ---------------------------------------------------------------------
> > >> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> > >> For additional commands, e-mail: java-user-help@lucene.apache.org
> > >>
> > >>
> > >
> >
>
Re: Issue with FacetFields.addFields() throwing ArrayIndexOutOfBoundsException
Posted by Shai Erera <se...@gmail.com>.
Do you have a test which reproduces the error? Are you adding categories
with very deep hierarchies?
Shai
On Fri, Jan 17, 2014 at 11:59 PM, Matthew Petersen <md...@gmail.com>wrote:
> I've confirmed that using the LruTaxonomyWriterCache solves the issue for
> me. It would appear there is in fact a bug in the Cl20TaxonomyWriterCache
> or I am using it incorrectly (I use it as default, no customization).
>
>
> On Fri, Jan 17, 2014 at 9:29 AM, Matthew Petersen <md...@gmail.com>
> wrote:
>
> > I'm sure. I had seen that issue and it looked similar but the stack
> trace
> > is slightly different. I've found that if I replace the
> > Cl2oTaxonomyWriterCache with the LruTaxonomyWriterCache the problem seems
> > to go away. I'm working right now on running a test that will prove this
> > but it takes a while as the cache needs to get very large. If this
> proves
> > to solve the problem then I'd say there is still a bug in the
> > Cl2oTaxonomyWriterCache implementation.
> >
> > Thanks for the response.
> > Matt
> >
> >
> > On Fri, Jan 17, 2014 at 6:36 AM, Michael McCandless <
> > lucene@mikemccandless.com> wrote:
> >
> >> Are you sure you're using 4.4?
> >>
> >> Because ... this looks like
> >> https://issues.apache.org/jira/browse/LUCENE-5048 but that was
> >> supposedly fixed in 4.4.
> >>
> >> Mike McCandless
> >>
> >> http://blog.mikemccandless.com
> >>
> >>
> >> On Thu, Jan 16, 2014 at 5:33 PM, Matthew Petersen <md...@gmail.com>
> >> wrote:
> >> > I’m having an issue with an index when adding category paths to a
> >> document.
> >> > They seem to be added without issue for a long period of time, then
> for
> >> > some unknown reason the addition fails with an ArrayIndexOutOfBounds
> >> > exception. Subsequent attempts to add category paths fail with the
> same
> >> > exception. I’ve run CheckIndex on both the index and the taxonomy
> >> > directory and both come back as clean with no issues. I cannot fix
> the
> >> > index because according to lucene it is not broken. Could this be a
> >> bug in
> >> > lucene? Below is the stack trace when the exception occurs:
> >> >
> >> >
> >> > Lucene v4.4.0
> >> >
> >> >
> >> > java.lang.ArrayIndexOutOfBoundsException: -65535
> >> >
> >> > at java.util.ArrayList.elementData(ArrayList.java:371)
> >> >
> >> > at java.util.ArrayList.get(ArrayList.java:384)
> >> >
> >> > at
> >> >
> >>
> org.apache.lucene.facet.taxonomy.writercache.cl2o.CharBlockArray.charAt(CharBlockArray.java:152)
> >> >
> >> > at
> >> >
> >>
> org.apache.lucene.facet.taxonomy.writercache.cl2o.CategoryPathUtils.equalsToSerialized(CategoryPathUtils.java:61)
> >> >
> >> > at
> >> >
> >>
> org.apache.lucene.facet.taxonomy.writercache.cl2o.CompactLabelToOrdinal.getOrdinal(CompactLabelToOrdinal.java:257)
> >> >
> >> > at
> >> >
> >>
> org.apache.lucene.facet.taxonomy.writercache.cl2o.CompactLabelToOrdinal.getOrdinal(CompactLabelToOrdinal.java:140)
> >> >
> >> > at
> >> >
> >>
> org.apache.lucene.facet.taxonomy.writercache.cl2o.Cl2oTaxonomyWriterCache.get(Cl2oTaxonomyWriterCache.java:74)
> >> >
> >> > at
> >> >
> >>
> org.apache.lucene.facet.taxonomy.directory.DirectoryTaxonomyWriter.addCategory(DirectoryTaxonomyWriter.java:455)
> >> >
> >> > at
> >>
> org.apache.lucene.facet.index.FacetFields.addFields(FacetFields.java:175)
> >> >
> >> > at
> >> >
> >>
> com.logrhythm.messaging.indexing.LogIndexerImpl.getDocument(LogIndexerImpl.java:478)
> >> >
> >> > at
> >> >
> >>
> com.logrhythm.messaging.indexing.LogIndexerImpl.indexLog(LogIndexerImpl.java:392)
> >> >
> >> > at
> >> >
> >>
> com.logrhythm.messaging.indexing.LogIndexerImpl.indexLogs(LogIndexerImpl.java:357)
> >> >
> >> > at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source)
> >> >
> >> > at
> >> >
> >>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> >> >
> >> > at java.lang.reflect.Method.invoke(Method.java:601)
> >> >
> >> > at
> >> >
> >>
> com.logrhythm.tests.unit.messaging.indexing.LogIndexerTests.logIndexerLoadTest(LogIndexerTests.java:752)
> >> >
> >> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> >> >
> >> > at
> >> >
> >>
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> >> >
> >> > at
> >> >
> >>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> >> >
> >> > at java.lang.reflect.Method.invoke(Method.java:601)
> >> >
> >> > at
> >> >
> >>
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
> >> >
> >> > at
> >> >
> >>
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
> >> >
> >> > at
> >> >
> >>
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
> >> >
> >> > at
> >> >
> >>
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
> >> >
> >> > at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
> >> >
> >> > at
> >> >
> >>
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68)
> >> >
> >> > at
> >> >
> >>
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47)
> >> >
> >> > at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
> >> >
> >> > at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
> >> >
> >> > at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
> >> >
> >> > at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
> >> >
> >> > at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
> >> >
> >> > at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
> >> >
> >> > at org.junit.runner.JUnitCore.run(JUnitCore.java:157)
> >> >
> >> > at
> >> >
> >>
> com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:74)
> >> >
> >> > at
> >> >
> >>
> com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:202)
> >> >
> >> > at
> >> com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:65)
> >> >
> >> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> >> >
> >> > at
> >> >
> >>
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> >> >
> >> > at
> >> >
> >>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> >> >
> >> > at java.lang.reflect.Method.invoke(Method.java:601)
> >> >
> >> > at
> com.intellij.rt.execution.application.AppMain.main(AppMain.java:120)
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> >> For additional commands, e-mail: java-user-help@lucene.apache.org
> >>
> >>
> >
>
Re: Issue with FacetFields.addFields() throwing ArrayIndexOutOfBoundsException
Posted by Matthew Petersen <md...@gmail.com>.
I've confirmed that using the LruTaxonomyWriterCache solves the issue for
me. It would appear there is in fact a bug in the Cl20TaxonomyWriterCache
or I am using it incorrectly (I use it as default, no customization).
On Fri, Jan 17, 2014 at 9:29 AM, Matthew Petersen <md...@gmail.com> wrote:
> I'm sure. I had seen that issue and it looked similar but the stack trace
> is slightly different. I've found that if I replace the
> Cl2oTaxonomyWriterCache with the LruTaxonomyWriterCache the problem seems
> to go away. I'm working right now on running a test that will prove this
> but it takes a while as the cache needs to get very large. If this proves
> to solve the problem then I'd say there is still a bug in the
> Cl2oTaxonomyWriterCache implementation.
>
> Thanks for the response.
> Matt
>
>
> On Fri, Jan 17, 2014 at 6:36 AM, Michael McCandless <
> lucene@mikemccandless.com> wrote:
>
>> Are you sure you're using 4.4?
>>
>> Because ... this looks like
>> https://issues.apache.org/jira/browse/LUCENE-5048 but that was
>> supposedly fixed in 4.4.
>>
>> Mike McCandless
>>
>> http://blog.mikemccandless.com
>>
>>
>> On Thu, Jan 16, 2014 at 5:33 PM, Matthew Petersen <md...@gmail.com>
>> wrote:
>> > I’m having an issue with an index when adding category paths to a
>> document.
>> > They seem to be added without issue for a long period of time, then for
>> > some unknown reason the addition fails with an ArrayIndexOutOfBounds
>> > exception. Subsequent attempts to add category paths fail with the same
>> > exception. I’ve run CheckIndex on both the index and the taxonomy
>> > directory and both come back as clean with no issues. I cannot fix the
>> > index because according to lucene it is not broken. Could this be a
>> bug in
>> > lucene? Below is the stack trace when the exception occurs:
>> >
>> >
>> > Lucene v4.4.0
>> >
>> >
>> > java.lang.ArrayIndexOutOfBoundsException: -65535
>> >
>> > at java.util.ArrayList.elementData(ArrayList.java:371)
>> >
>> > at java.util.ArrayList.get(ArrayList.java:384)
>> >
>> > at
>> >
>> org.apache.lucene.facet.taxonomy.writercache.cl2o.CharBlockArray.charAt(CharBlockArray.java:152)
>> >
>> > at
>> >
>> org.apache.lucene.facet.taxonomy.writercache.cl2o.CategoryPathUtils.equalsToSerialized(CategoryPathUtils.java:61)
>> >
>> > at
>> >
>> org.apache.lucene.facet.taxonomy.writercache.cl2o.CompactLabelToOrdinal.getOrdinal(CompactLabelToOrdinal.java:257)
>> >
>> > at
>> >
>> org.apache.lucene.facet.taxonomy.writercache.cl2o.CompactLabelToOrdinal.getOrdinal(CompactLabelToOrdinal.java:140)
>> >
>> > at
>> >
>> org.apache.lucene.facet.taxonomy.writercache.cl2o.Cl2oTaxonomyWriterCache.get(Cl2oTaxonomyWriterCache.java:74)
>> >
>> > at
>> >
>> org.apache.lucene.facet.taxonomy.directory.DirectoryTaxonomyWriter.addCategory(DirectoryTaxonomyWriter.java:455)
>> >
>> > at
>> org.apache.lucene.facet.index.FacetFields.addFields(FacetFields.java:175)
>> >
>> > at
>> >
>> com.logrhythm.messaging.indexing.LogIndexerImpl.getDocument(LogIndexerImpl.java:478)
>> >
>> > at
>> >
>> com.logrhythm.messaging.indexing.LogIndexerImpl.indexLog(LogIndexerImpl.java:392)
>> >
>> > at
>> >
>> com.logrhythm.messaging.indexing.LogIndexerImpl.indexLogs(LogIndexerImpl.java:357)
>> >
>> > at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source)
>> >
>> > at
>> >
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>> >
>> > at java.lang.reflect.Method.invoke(Method.java:601)
>> >
>> > at
>> >
>> com.logrhythm.tests.unit.messaging.indexing.LogIndexerTests.logIndexerLoadTest(LogIndexerTests.java:752)
>> >
>> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> >
>> > at
>> >
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>> >
>> > at
>> >
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>> >
>> > at java.lang.reflect.Method.invoke(Method.java:601)
>> >
>> > at
>> >
>> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
>> >
>> > at
>> >
>> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
>> >
>> > at
>> >
>> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
>> >
>> > at
>> >
>> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
>> >
>> > at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
>> >
>> > at
>> >
>> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68)
>> >
>> > at
>> >
>> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47)
>> >
>> > at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
>> >
>> > at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
>> >
>> > at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
>> >
>> > at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
>> >
>> > at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
>> >
>> > at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
>> >
>> > at org.junit.runner.JUnitCore.run(JUnitCore.java:157)
>> >
>> > at
>> >
>> com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:74)
>> >
>> > at
>> >
>> com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:202)
>> >
>> > at
>> com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:65)
>> >
>> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> >
>> > at
>> >
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>> >
>> > at
>> >
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>> >
>> > at java.lang.reflect.Method.invoke(Method.java:601)
>> >
>> > at com.intellij.rt.execution.application.AppMain.main(AppMain.java:120)
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>
>>
>
Re: Issue with FacetFields.addFields() throwing ArrayIndexOutOfBoundsException
Posted by Matthew Petersen <md...@gmail.com>.
I'm sure. I had seen that issue and it looked similar but the stack trace
is slightly different. I've found that if I replace the
Cl2oTaxonomyWriterCache with the LruTaxonomyWriterCache the problem seems
to go away. I'm working right now on running a test that will prove this
but it takes a while as the cache needs to get very large. If this proves
to solve the problem then I'd say there is still a bug in the
Cl2oTaxonomyWriterCache implementation.
Thanks for the response.
Matt
On Fri, Jan 17, 2014 at 6:36 AM, Michael McCandless <
lucene@mikemccandless.com> wrote:
> Are you sure you're using 4.4?
>
> Because ... this looks like
> https://issues.apache.org/jira/browse/LUCENE-5048 but that was
> supposedly fixed in 4.4.
>
> Mike McCandless
>
> http://blog.mikemccandless.com
>
>
> On Thu, Jan 16, 2014 at 5:33 PM, Matthew Petersen <md...@gmail.com>
> wrote:
> > I’m having an issue with an index when adding category paths to a
> document.
> > They seem to be added without issue for a long period of time, then for
> > some unknown reason the addition fails with an ArrayIndexOutOfBounds
> > exception. Subsequent attempts to add category paths fail with the same
> > exception. I’ve run CheckIndex on both the index and the taxonomy
> > directory and both come back as clean with no issues. I cannot fix the
> > index because according to lucene it is not broken. Could this be a bug
> in
> > lucene? Below is the stack trace when the exception occurs:
> >
> >
> > Lucene v4.4.0
> >
> >
> > java.lang.ArrayIndexOutOfBoundsException: -65535
> >
> > at java.util.ArrayList.elementData(ArrayList.java:371)
> >
> > at java.util.ArrayList.get(ArrayList.java:384)
> >
> > at
> >
> org.apache.lucene.facet.taxonomy.writercache.cl2o.CharBlockArray.charAt(CharBlockArray.java:152)
> >
> > at
> >
> org.apache.lucene.facet.taxonomy.writercache.cl2o.CategoryPathUtils.equalsToSerialized(CategoryPathUtils.java:61)
> >
> > at
> >
> org.apache.lucene.facet.taxonomy.writercache.cl2o.CompactLabelToOrdinal.getOrdinal(CompactLabelToOrdinal.java:257)
> >
> > at
> >
> org.apache.lucene.facet.taxonomy.writercache.cl2o.CompactLabelToOrdinal.getOrdinal(CompactLabelToOrdinal.java:140)
> >
> > at
> >
> org.apache.lucene.facet.taxonomy.writercache.cl2o.Cl2oTaxonomyWriterCache.get(Cl2oTaxonomyWriterCache.java:74)
> >
> > at
> >
> org.apache.lucene.facet.taxonomy.directory.DirectoryTaxonomyWriter.addCategory(DirectoryTaxonomyWriter.java:455)
> >
> > at
> org.apache.lucene.facet.index.FacetFields.addFields(FacetFields.java:175)
> >
> > at
> >
> com.logrhythm.messaging.indexing.LogIndexerImpl.getDocument(LogIndexerImpl.java:478)
> >
> > at
> >
> com.logrhythm.messaging.indexing.LogIndexerImpl.indexLog(LogIndexerImpl.java:392)
> >
> > at
> >
> com.logrhythm.messaging.indexing.LogIndexerImpl.indexLogs(LogIndexerImpl.java:357)
> >
> > at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source)
> >
> > at
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> >
> > at java.lang.reflect.Method.invoke(Method.java:601)
> >
> > at
> >
> com.logrhythm.tests.unit.messaging.indexing.LogIndexerTests.logIndexerLoadTest(LogIndexerTests.java:752)
> >
> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> >
> > at
> >
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> >
> > at
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> >
> > at java.lang.reflect.Method.invoke(Method.java:601)
> >
> > at
> >
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
> >
> > at
> >
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
> >
> > at
> >
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
> >
> > at
> >
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
> >
> > at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
> >
> > at
> >
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68)
> >
> > at
> >
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47)
> >
> > at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
> >
> > at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
> >
> > at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
> >
> > at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
> >
> > at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
> >
> > at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
> >
> > at org.junit.runner.JUnitCore.run(JUnitCore.java:157)
> >
> > at
> >
> com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:74)
> >
> > at
> >
> com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:202)
> >
> > at
> com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:65)
> >
> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> >
> > at
> >
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> >
> > at
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> >
> > at java.lang.reflect.Method.invoke(Method.java:601)
> >
> > at com.intellij.rt.execution.application.AppMain.main(AppMain.java:120)
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>
Re: Issue with FacetFields.addFields() throwing ArrayIndexOutOfBoundsException
Posted by Michael McCandless <lu...@mikemccandless.com>.
Are you sure you're using 4.4?
Because ... this looks like
https://issues.apache.org/jira/browse/LUCENE-5048 but that was
supposedly fixed in 4.4.
Mike McCandless
http://blog.mikemccandless.com
On Thu, Jan 16, 2014 at 5:33 PM, Matthew Petersen <md...@gmail.com> wrote:
> I’m having an issue with an index when adding category paths to a document.
> They seem to be added without issue for a long period of time, then for
> some unknown reason the addition fails with an ArrayIndexOutOfBounds
> exception. Subsequent attempts to add category paths fail with the same
> exception. I’ve run CheckIndex on both the index and the taxonomy
> directory and both come back as clean with no issues. I cannot fix the
> index because according to lucene it is not broken. Could this be a bug in
> lucene? Below is the stack trace when the exception occurs:
>
>
> Lucene v4.4.0
>
>
> java.lang.ArrayIndexOutOfBoundsException: -65535
>
> at java.util.ArrayList.elementData(ArrayList.java:371)
>
> at java.util.ArrayList.get(ArrayList.java:384)
>
> at
> org.apache.lucene.facet.taxonomy.writercache.cl2o.CharBlockArray.charAt(CharBlockArray.java:152)
>
> at
> org.apache.lucene.facet.taxonomy.writercache.cl2o.CategoryPathUtils.equalsToSerialized(CategoryPathUtils.java:61)
>
> at
> org.apache.lucene.facet.taxonomy.writercache.cl2o.CompactLabelToOrdinal.getOrdinal(CompactLabelToOrdinal.java:257)
>
> at
> org.apache.lucene.facet.taxonomy.writercache.cl2o.CompactLabelToOrdinal.getOrdinal(CompactLabelToOrdinal.java:140)
>
> at
> org.apache.lucene.facet.taxonomy.writercache.cl2o.Cl2oTaxonomyWriterCache.get(Cl2oTaxonomyWriterCache.java:74)
>
> at
> org.apache.lucene.facet.taxonomy.directory.DirectoryTaxonomyWriter.addCategory(DirectoryTaxonomyWriter.java:455)
>
> at org.apache.lucene.facet.index.FacetFields.addFields(FacetFields.java:175)
>
> at
> com.logrhythm.messaging.indexing.LogIndexerImpl.getDocument(LogIndexerImpl.java:478)
>
> at
> com.logrhythm.messaging.indexing.LogIndexerImpl.indexLog(LogIndexerImpl.java:392)
>
> at
> com.logrhythm.messaging.indexing.LogIndexerImpl.indexLogs(LogIndexerImpl.java:357)
>
> at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source)
>
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>
> at java.lang.reflect.Method.invoke(Method.java:601)
>
> at
> com.logrhythm.tests.unit.messaging.indexing.LogIndexerTests.logIndexerLoadTest(LogIndexerTests.java:752)
>
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>
> at java.lang.reflect.Method.invoke(Method.java:601)
>
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
>
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
>
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
>
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
>
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
>
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68)
>
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47)
>
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
>
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
>
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
>
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
>
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
>
> at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
>
> at org.junit.runner.JUnitCore.run(JUnitCore.java:157)
>
> at
> com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:74)
>
> at
> com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:202)
>
> at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:65)
>
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>
> at java.lang.reflect.Method.invoke(Method.java:601)
>
> at com.intellij.rt.execution.application.AppMain.main(AppMain.java:120)
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org