You are viewing a plain text version of this content. The canonical link for it is here.
Posted to java-user@lucene.apache.org by Jigar Shah <ji...@gmail.com> on 2014/10/07 09:03:26 UTC
Exception from FastTaxonomyFacetCounts
Intermittently while search i am getting this exception on huge index.
(FacetsConfig used while indexing and searching is same.)
java.lang.ArrayIndexOutOfBoundsException: 252554
06:28:37,954 ERROR [stderr] at
org.apache.lucene.facet.taxonomy.FastTaxonomyFacetCounts.count(FastTaxonomyFacetCounts.java:73)
06:28:37,954 ERROR [stderr] at
org.apache.lucene.facet.taxonomy.FastTaxonomyFacetCounts.<init>(FastTaxonomyFacetCounts.java:49)
06:28:37,954 ERROR [stderr] at
org.apache.lucene.facet.taxonomy.FastTaxonomyFacetCounts.<init>(FastTaxonomyFacetCounts.java:39)
06:28:37,954 ERROR [stderr] at
com.company.search.CustomDrillSideways.buildFacetsResult(LuceneDrillSideways.java:41)
06:28:37,954 ERROR [stderr] at
org.apache.lucene.facet.DrillSideways.search(DrillSideways.java:146)
06:28:37,955 ERROR [stderr] at
org.apache.lucene.facet.DrillSideways.search(DrillSideways.java:203)
Thanks,
Jigar Shah
Re: Exception from FastTaxonomyFacetCounts
Posted by Jigar Shah <ji...@gmail.com>.
Ok. Yes changing to SearcherTaxonomyManager from SearcherManager, helped
fixing the issue
Thanks very much.
On Wed, Oct 15, 2014 at 2:36 PM, Shai Erera <se...@gmail.com> wrote:
> Yes, SearcherTaxonomyManager returns a SearcherAndTaxonomy containing a
> sync'd IndexSearcher and DirectoryTaxonomyReader.
>
> Shai
>
> On Mon, Oct 13, 2014 at 12:15 PM, Jigar Shah <ji...@gmail.com>
> wrote:
>
> > In my application i have two intances of SearcherManager.
> >
> > 1) SearcherManager with 'applyAllDeletes = true' which is used by
> Indexer.
> > (Works in NRT mode, deletes should be visible to it, also i have
> > ControlledRealTimeReopenThread, which refeshes searcher)
> > 2) SearcherManager with 'applyAllDeletes = false' which is used by
> searcher
> > (Only performs search, javadoc says, we may gain some performance if
> > 'false', as it will not wait for flushing deletes,).
> >
> > I have intoduced Taxonomy Facets in my applicaiton. Should i replace both
> > SearcherManager by SearcherTaxonomyManager (one with applyAllDeletes=true
> > and another applyAllDeletes=false)
> >
> > Will "IndexSearcher" and "TaxonomyReader" be in sync, in both
> > SearcherTaxonomyManager ?
> >
> > On Fri, Oct 10, 2014 at 12:08 AM, Shai Erera <se...@gmail.com> wrote:
> >
> > > This usually means that your IndexReader and TaxonomyReader are out of
> > > sync. That is, the IndexReader sees category ordinals that the
> > > TaxonomyReader does not yet see.
> > >
> > > Do you use SearcherTaxonomyManager in your application? It ensures that
> > the
> > > two are always in sync, i.e. reopened together and that your
> application
> > > always sees a consistent view of the two.
> > >
> > > Shai
> > >
> > > On Tue, Oct 7, 2014 at 10:03 AM, Jigar Shah <ji...@gmail.com>
> > wrote:
> > >
> > > > Intermittently while search i am getting this exception on huge
> index.
> > > > (FacetsConfig used while indexing and searching is same.)
> > > >
> > > > java.lang.ArrayIndexOutOfBoundsException: 252554
> > > > 06:28:37,954 ERROR [stderr] at
> > > >
> > > >
> > >
> >
> org.apache.lucene.facet.taxonomy.FastTaxonomyFacetCounts.count(FastTaxonomyFacetCounts.java:73)
> > > > 06:28:37,954 ERROR [stderr] at
> > > >
> > > >
> > >
> >
> org.apache.lucene.facet.taxonomy.FastTaxonomyFacetCounts.<init>(FastTaxonomyFacetCounts.java:49)
> > > > 06:28:37,954 ERROR [stderr] at
> > > >
> > > >
> > >
> >
> org.apache.lucene.facet.taxonomy.FastTaxonomyFacetCounts.<init>(FastTaxonomyFacetCounts.java:39)
> > > > 06:28:37,954 ERROR [stderr] at
> > > >
> > > >
> > >
> >
> com.company.search.CustomDrillSideways.buildFacetsResult(LuceneDrillSideways.java:41)
> > > > 06:28:37,954 ERROR [stderr] at
> > > > org.apache.lucene.facet.DrillSideways.search(DrillSideways.java:146)
> > > > 06:28:37,955 ERROR [stderr] at
> > > > org.apache.lucene.facet.DrillSideways.search(DrillSideways.java:203)
> > > >
> > > > Thanks,
> > > > Jigar Shah
> > > >
> > >
> >
>
Re: Exception from FastTaxonomyFacetCounts
Posted by Shai Erera <se...@gmail.com>.
Yes, SearcherTaxonomyManager returns a SearcherAndTaxonomy containing a
sync'd IndexSearcher and DirectoryTaxonomyReader.
Shai
On Mon, Oct 13, 2014 at 12:15 PM, Jigar Shah <ji...@gmail.com> wrote:
> In my application i have two intances of SearcherManager.
>
> 1) SearcherManager with 'applyAllDeletes = true' which is used by Indexer.
> (Works in NRT mode, deletes should be visible to it, also i have
> ControlledRealTimeReopenThread, which refeshes searcher)
> 2) SearcherManager with 'applyAllDeletes = false' which is used by searcher
> (Only performs search, javadoc says, we may gain some performance if
> 'false', as it will not wait for flushing deletes,).
>
> I have intoduced Taxonomy Facets in my applicaiton. Should i replace both
> SearcherManager by SearcherTaxonomyManager (one with applyAllDeletes=true
> and another applyAllDeletes=false)
>
> Will "IndexSearcher" and "TaxonomyReader" be in sync, in both
> SearcherTaxonomyManager ?
>
> On Fri, Oct 10, 2014 at 12:08 AM, Shai Erera <se...@gmail.com> wrote:
>
> > This usually means that your IndexReader and TaxonomyReader are out of
> > sync. That is, the IndexReader sees category ordinals that the
> > TaxonomyReader does not yet see.
> >
> > Do you use SearcherTaxonomyManager in your application? It ensures that
> the
> > two are always in sync, i.e. reopened together and that your application
> > always sees a consistent view of the two.
> >
> > Shai
> >
> > On Tue, Oct 7, 2014 at 10:03 AM, Jigar Shah <ji...@gmail.com>
> wrote:
> >
> > > Intermittently while search i am getting this exception on huge index.
> > > (FacetsConfig used while indexing and searching is same.)
> > >
> > > java.lang.ArrayIndexOutOfBoundsException: 252554
> > > 06:28:37,954 ERROR [stderr] at
> > >
> > >
> >
> org.apache.lucene.facet.taxonomy.FastTaxonomyFacetCounts.count(FastTaxonomyFacetCounts.java:73)
> > > 06:28:37,954 ERROR [stderr] at
> > >
> > >
> >
> org.apache.lucene.facet.taxonomy.FastTaxonomyFacetCounts.<init>(FastTaxonomyFacetCounts.java:49)
> > > 06:28:37,954 ERROR [stderr] at
> > >
> > >
> >
> org.apache.lucene.facet.taxonomy.FastTaxonomyFacetCounts.<init>(FastTaxonomyFacetCounts.java:39)
> > > 06:28:37,954 ERROR [stderr] at
> > >
> > >
> >
> com.company.search.CustomDrillSideways.buildFacetsResult(LuceneDrillSideways.java:41)
> > > 06:28:37,954 ERROR [stderr] at
> > > org.apache.lucene.facet.DrillSideways.search(DrillSideways.java:146)
> > > 06:28:37,955 ERROR [stderr] at
> > > org.apache.lucene.facet.DrillSideways.search(DrillSideways.java:203)
> > >
> > > Thanks,
> > > Jigar Shah
> > >
> >
>
Re: Exception from FastTaxonomyFacetCounts
Posted by Jigar Shah <ji...@gmail.com>.
In my application i have two intances of SearcherManager.
1) SearcherManager with 'applyAllDeletes = true' which is used by Indexer.
(Works in NRT mode, deletes should be visible to it, also i have
ControlledRealTimeReopenThread, which refeshes searcher)
2) SearcherManager with 'applyAllDeletes = false' which is used by searcher
(Only performs search, javadoc says, we may gain some performance if
'false', as it will not wait for flushing deletes,).
I have intoduced Taxonomy Facets in my applicaiton. Should i replace both
SearcherManager by SearcherTaxonomyManager (one with applyAllDeletes=true
and another applyAllDeletes=false)
Will "IndexSearcher" and "TaxonomyReader" be in sync, in both
SearcherTaxonomyManager ?
On Fri, Oct 10, 2014 at 12:08 AM, Shai Erera <se...@gmail.com> wrote:
> This usually means that your IndexReader and TaxonomyReader are out of
> sync. That is, the IndexReader sees category ordinals that the
> TaxonomyReader does not yet see.
>
> Do you use SearcherTaxonomyManager in your application? It ensures that the
> two are always in sync, i.e. reopened together and that your application
> always sees a consistent view of the two.
>
> Shai
>
> On Tue, Oct 7, 2014 at 10:03 AM, Jigar Shah <ji...@gmail.com> wrote:
>
> > Intermittently while search i am getting this exception on huge index.
> > (FacetsConfig used while indexing and searching is same.)
> >
> > java.lang.ArrayIndexOutOfBoundsException: 252554
> > 06:28:37,954 ERROR [stderr] at
> >
> >
> org.apache.lucene.facet.taxonomy.FastTaxonomyFacetCounts.count(FastTaxonomyFacetCounts.java:73)
> > 06:28:37,954 ERROR [stderr] at
> >
> >
> org.apache.lucene.facet.taxonomy.FastTaxonomyFacetCounts.<init>(FastTaxonomyFacetCounts.java:49)
> > 06:28:37,954 ERROR [stderr] at
> >
> >
> org.apache.lucene.facet.taxonomy.FastTaxonomyFacetCounts.<init>(FastTaxonomyFacetCounts.java:39)
> > 06:28:37,954 ERROR [stderr] at
> >
> >
> com.company.search.CustomDrillSideways.buildFacetsResult(LuceneDrillSideways.java:41)
> > 06:28:37,954 ERROR [stderr] at
> > org.apache.lucene.facet.DrillSideways.search(DrillSideways.java:146)
> > 06:28:37,955 ERROR [stderr] at
> > org.apache.lucene.facet.DrillSideways.search(DrillSideways.java:203)
> >
> > Thanks,
> > Jigar Shah
> >
>
Re: Exception from FastTaxonomyFacetCounts
Posted by Shai Erera <se...@gmail.com>.
This usually means that your IndexReader and TaxonomyReader are out of
sync. That is, the IndexReader sees category ordinals that the
TaxonomyReader does not yet see.
Do you use SearcherTaxonomyManager in your application? It ensures that the
two are always in sync, i.e. reopened together and that your application
always sees a consistent view of the two.
Shai
On Tue, Oct 7, 2014 at 10:03 AM, Jigar Shah <ji...@gmail.com> wrote:
> Intermittently while search i am getting this exception on huge index.
> (FacetsConfig used while indexing and searching is same.)
>
> java.lang.ArrayIndexOutOfBoundsException: 252554
> 06:28:37,954 ERROR [stderr] at
>
> org.apache.lucene.facet.taxonomy.FastTaxonomyFacetCounts.count(FastTaxonomyFacetCounts.java:73)
> 06:28:37,954 ERROR [stderr] at
>
> org.apache.lucene.facet.taxonomy.FastTaxonomyFacetCounts.<init>(FastTaxonomyFacetCounts.java:49)
> 06:28:37,954 ERROR [stderr] at
>
> org.apache.lucene.facet.taxonomy.FastTaxonomyFacetCounts.<init>(FastTaxonomyFacetCounts.java:39)
> 06:28:37,954 ERROR [stderr] at
>
> com.company.search.CustomDrillSideways.buildFacetsResult(LuceneDrillSideways.java:41)
> 06:28:37,954 ERROR [stderr] at
> org.apache.lucene.facet.DrillSideways.search(DrillSideways.java:146)
> 06:28:37,955 ERROR [stderr] at
> org.apache.lucene.facet.DrillSideways.search(DrillSideways.java:203)
>
> Thanks,
> Jigar Shah
>