You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Cool Techi <co...@outlook.com> on 2014/04/10 13:34:39 UTC
Fails to index if unique field has special characters
Hi,
We are migrating from Solr 4.6 standalone to Solr 4.7 cloud version, while reindexing the document we are getting the following error. This is happening when the unique key has special character, this was not noticed in version 4.6 standalone mode, so we are not sure if this is a version problem or a cloud issue. Example of the unique key is given below,
http://www.mynews.in/Blog/smrity!!**)))!miami_dolphins_vs_dallas_cowboys_live_stream_on_line_nfl_football_free_video_broadcast_B142707.html
Exception Stack Trace
ERROR - 2014-04-10 10:51:44.361; org.apache.solr.common.SolrException; java.lang.ArrayIndexOutOfBoundsException: 2 at org.apache.solr.common.cloud.CompositeIdRouter$KeyParser.getHash(CompositeIdRouter.java:296) at org.apache.solr.common.cloud.CompositeIdRouter.sliceHash(CompositeIdRouter.java:58) at org.apache.solr.common.cloud.HashBasedRouter.getTargetSlice(HashBasedRouter.java:33) at org.apache.solr.update.processor.DistributedUpdateProcessor.setupRequest(DistributedUpdateProcessor.java:218) at org.apache.solr.update.processor.DistributedUpdateProcessor.processAdd(DistributedUpdateProcessor.java:550) at org.apache.solr.update.processor.LogUpdateProcessor.processAdd(LogUpdateProcessorFactory.java:100) at org.apache.solr.handler.loader.XMLLoader.processUpdate(XMLLoader.java:247) at org.apache.solr.handler.loader.XMLLoader.load(XMLLoader.java:174) at org.apache.solr.handler.UpdateRequestHandler$1.load(UpdateRequestHandler.java:92) at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:74) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135) at org.apache.solr.core.SolrCore.execute(SolrCore.java:1916) at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:780) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:427) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:217) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:455) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:557) at org.eclipse.jetty.server.session.SessionHandle
Thanks,Ayush
Re: Fails to index if unique field has special characters
Posted by Markus Jelsma <ma...@openindex.io>.
Well, this is somewhat of a problem if you have have URL's as uniqueKey that
contain exclamation marks. Isn't it an idea to allow those to be escaped and
thus ignored by CompositeIdRouter?
On Friday, April 11, 2014 11:43:31 AM Cool Techi wrote:
> Thanks, that was helpful.
> Regards,Rohit
>
> > Date: Thu, 10 Apr 2014 08:44:36 -0700
> > From: iorixxx@yahoo.com
> > Subject: Re: Fails to index if unique field has special characters
> > To: solr-user@lucene.apache.org
> >
> > Hi Ayush,
> >
> > I thinks this
> >
> > ""IBM!12345". The exclamation mark ('!') is critical here, as it
> > distinguishes the prefix used to determine which shard to direct the
> > document to."
> >
> > https://cwiki.apache.org/confluence/display/solr/Shards+and+Indexing+Data+
> > in+SolrCloud
> >
> >
> >
> >
> > On Thursday, April 10, 2014 2:35 PM, Cool Techi <co...@outlook.com>
> > wrote: Hi,
> > We are migrating from Solr 4.6 standalone to Solr 4.7 cloud version, while
> > reindexing the document we are getting the following error. This is
> > happening when the unique key has special character, this was not noticed
> > in version 4.6 standalone mode, so we are not sure if this is a version
> > problem or a cloud issue. Example of the unique key is given below,
> > http://www.mynews.in/Blog/smrity!!**)))!miami_dolphins_vs_dallas_cowboys_
> > live_stream_on_line_nfl_football_free_video_broadcast_B142707.html
> > Exception Stack Trace
> > ERROR - 2014-04-10 10:51:44.361; org.apache.solr.common.SolrException;
> > java.lang.ArrayIndexOutOfBoundsException: 2 at
> > org.apache.solr.common.cloud.CompositeIdRouter$KeyParser.getHash(Composit
> > eIdRouter.java:296) at
> > org.apache.solr.common.cloud.CompositeIdRouter.sliceHash(CompositeIdRoute
> > r.java:58) at
> > org.apache.solr.common.cloud.HashBasedRouter.getTargetSlice(HashBasedRout
> > er.java:33) at
> > org.apache.solr.update.processor.DistributedUpdateProcessor.setupRequest(
> > DistributedUpdateProcessor.java:218) at
> > org.apache.solr.update.processor.DistributedUpdateProcessor.processAdd(Di
> > stributedUpdateProcessor.java:550) at
> > org.apache.solr.update.processor.LogUpdateProcessor.processAdd(LogUpdateP
> > rocessorFactory.java:100) at
> > org.apache.solr.handler.loader.XMLLoader.processUpdate(XMLLoader.java:247
> > ) at
> > org.apache.solr.handler.loader.XMLLoader.load(XMLLoader.java:174)
> > at>
> >
> >org.apache.solr.handler.UpdateRequestHandler$1.load(UpdateRequestHandler.j
> >ava:92) at
> >org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(Content
> >StreamHandlerBase.java:74) at
> >org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBas
> >e.java:135) at
> >org.apache.solr.core.SolrCore.execute(SolrCore.java:1916) at
> >org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java
> >:780) at
> >org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.jav
> >a:427) at
> >org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.jav
> >a:217) at
> >org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandl
> >er.java:1419) at
> >org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:455)
> > at
> >org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:1
> >37) at
> >org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:557
> >) at org.eclipse.jetty.server.session.SessionHandle>
> > Thanks,Ayush
>
>
RE: Fails to index if unique field has special characters
Posted by Cool Techi <co...@outlook.com>.
Thanks, that was helpful.
Regards,Rohit
> Date: Thu, 10 Apr 2014 08:44:36 -0700
> From: iorixxx@yahoo.com
> Subject: Re: Fails to index if unique field has special characters
> To: solr-user@lucene.apache.org
>
> Hi Ayush,
>
> I thinks this
>
> ""IBM!12345". The exclamation mark ('!') is critical here, as it distinguishes the prefix used to determine which shard to direct the document to."
>
> https://cwiki.apache.org/confluence/display/solr/Shards+and+Indexing+Data+in+SolrCloud
>
>
>
>
> On Thursday, April 10, 2014 2:35 PM, Cool Techi <co...@outlook.com> wrote:
> Hi,
> We are migrating from Solr 4.6 standalone to Solr 4.7 cloud version, while reindexing the document we are getting the following error. This is happening when the unique key has special character, this was not noticed in version 4.6 standalone mode, so we are not sure if this is a version problem or a cloud issue. Example of the unique key is given below,
> http://www.mynews.in/Blog/smrity!!**)))!miami_dolphins_vs_dallas_cowboys_live_stream_on_line_nfl_football_free_video_broadcast_B142707.html
> Exception Stack Trace
> ERROR - 2014-04-10 10:51:44.361; org.apache.solr.common.SolrException; java.lang.ArrayIndexOutOfBoundsException: 2 at org.apache.solr.common.cloud.CompositeIdRouter$KeyParser.getHash(CompositeIdRouter.java:296) at org.apache.solr.common.cloud.CompositeIdRouter.sliceHash(CompositeIdRouter.java:58) at org.apache.solr.common.cloud.HashBasedRouter.getTargetSlice(HashBasedRouter.java:33) at org.apache.solr.update.processor.DistributedUpdateProcessor.setupRequest(DistributedUpdateProcessor.java:218) at org.apache.solr.update.processor.DistributedUpdateProcessor.processAdd(DistributedUpdateProcessor.java:550) at org.apache.solr.update.processor.LogUpdateProcessor.processAdd(LogUpdateProcessorFactory.java:100) at org.apache.solr.handler.loader.XMLLoader.processUpdate(XMLLoader.java:247) at org.apache.solr.handler.loader.XMLLoader.load(XMLLoader.java:174) at
> org.apache.solr.handler.UpdateRequestHandler$1.load(UpdateRequestHandler.java:92) at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:74) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135) at org.apache.solr.core.SolrCore.execute(SolrCore.java:1916) at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:780) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:427) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:217) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:455) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137) at
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:557) at org.eclipse.jetty.server.session.SessionHandle
>
> Thanks,Ayush
Re: Fails to index if unique field has special characters
Posted by Ahmet Arslan <io...@yahoo.com>.
Hi Ayush,
I thinks this
""IBM!12345". The exclamation mark ('!') is critical here, as it distinguishes the prefix used to determine which shard to direct the document to."
https://cwiki.apache.org/confluence/display/solr/Shards+and+Indexing+Data+in+SolrCloud
On Thursday, April 10, 2014 2:35 PM, Cool Techi <co...@outlook.com> wrote:
Hi,
We are migrating from Solr 4.6 standalone to Solr 4.7 cloud version, while reindexing the document we are getting the following error. This is happening when the unique key has special character, this was not noticed in version 4.6 standalone mode, so we are not sure if this is a version problem or a cloud issue. Example of the unique key is given below,
http://www.mynews.in/Blog/smrity!!**)))!miami_dolphins_vs_dallas_cowboys_live_stream_on_line_nfl_football_free_video_broadcast_B142707.html
Exception Stack Trace
ERROR - 2014-04-10 10:51:44.361; org.apache.solr.common.SolrException; java.lang.ArrayIndexOutOfBoundsException: 2 at org.apache.solr.common.cloud.CompositeIdRouter$KeyParser.getHash(CompositeIdRouter.java:296) at org.apache.solr.common.cloud.CompositeIdRouter.sliceHash(CompositeIdRouter.java:58) at org.apache.solr.common.cloud.HashBasedRouter.getTargetSlice(HashBasedRouter.java:33) at org.apache.solr.update.processor.DistributedUpdateProcessor.setupRequest(DistributedUpdateProcessor.java:218) at org.apache.solr.update.processor.DistributedUpdateProcessor.processAdd(DistributedUpdateProcessor.java:550) at org.apache.solr.update.processor.LogUpdateProcessor.processAdd(LogUpdateProcessorFactory.java:100) at org.apache.solr.handler.loader.XMLLoader.processUpdate(XMLLoader.java:247) at org.apache.solr.handler.loader.XMLLoader.load(XMLLoader.java:174) at
org.apache.solr.handler.UpdateRequestHandler$1.load(UpdateRequestHandler.java:92) at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:74) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135) at org.apache.solr.core.SolrCore.execute(SolrCore.java:1916) at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:780) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:427) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:217) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:455) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137) at
org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:557) at org.eclipse.jetty.server.session.SessionHandle
Thanks,Ayush