You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by "Jason, Kim" <hi...@gmail.com> on 2011/10/15 11:51:42 UTC
Phrase search error
Hi, all
When I queried a phrase search "test mp3", I got some error below.
I think that the problem is because of WordDelimiterFilter.
In WordDelimiterFilter 'mp3' is splited pos1:mp, pos2:(3, mp3).
In such a case, the positions of subword and catenateword are incremented.
If this is not phrase search or WordDelimiterFilterFactory options just set
catenateAll=1, no problems.
But If WordDelimiterFilterFactory options set like below 'My Schema.xml',
occured error.
How can I solve this problem?
Give me any idea.
Thanks in advance.
Jason
[Error Message]
==============================================================================
Unknown query type "org.apache.lucene.search.MultiPhraseQuery" found in
phrase query string "test mp3"
java.lang.IllegalArgumentException: Unknown query type
"org.apache.lucene.search.MultiPhraseQuery" found in phrase query string
"test mp3"
at
org.apache.lucene.queryParser.ComplexPhraseQueryParser$ComplexPhraseQuery.rewrite(ComplexPhraseQueryParser.java:300)
at org.apache.lucene.search.IndexSearcher.rewrite(IndexSearcher.java:307)
at org.apache.lucene.search.Query.weight(Query.java:98)
at org.apache.lucene.search.Searcher.createWeight(Searcher.java:230)
at org.apache.lucene.search.Searcher.search(Searcher.java:171)
at
org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:988)
at
org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:884)
at
org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:341)
at
org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:182)
at
org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:195)
at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:1316)
at
org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:341)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:244)
at
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1089)
at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:365)
at
org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:181)
at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:712)
at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:405)
at
org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:211)
at
org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114)
at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:139)
at org.mortbay.jetty.Server.handle(Server.java:285)
at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:502)
at
org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:821)
at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:513)
at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:208)
at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:378)
at
org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:226)
at
org.mortbay.thread.BoundedThreadPool$PoolThread.run(BoundedThreadPool.java:442)
==============================================================================
[My Schema.xml]
==============================================================================
<fieldType name="text" class="solr.TextField" positionIncrementGap="100">
<analyzer type="index">
<charFilter class="solr.MappingCharFilterFactory"
mapping="mapping-FoldToASCII.txt" />
<tokenizer class="solr.WhitespaceTokenizerFactory"/>
<filter class="solr.StopFilterFactory" ignoreCase="true"
words="stopwords.txt" enablePositionIncrements="true" />
<filter class="solr.WordDelimiterFilterFactory"
generateWordParts="1" generateNumberParts="0" catenateWords="1"
catenateNumbers="1" catenateAll="1" splitOnCaseChange="1" />
<filter class="solr.LowerCaseFilterFactory" />
<filter class="solr.SnowballPorterWithUnstemFilterFactory"
language="English" protected="protwords.txt" />
</analyzer>
<analyzer type="query">
<tokenizer class="solr.WhitespaceTokenizerFactory"/>
<filter class="solr.StopFilterFactory" ignoreCase="true"
words="stopwords.txt" enablePositionIncrements="true" />
<filter class="solr.WordDelimiterFilterFactory"
generateWordParts="1" generateNumberParts="0" catenateWords="1"
catenateNumbers="1" catenateAll="1" splitOnCaseChange="0" />
<filter class="solr.LowerCaseFilterFactory" />
<filter class="solr.SnowballPorterFilterFactory" language="English"
protected="protwords.txt" />
</analyzer>
</fieldType>
==============================================================================
--
View this message in context: http://lucene.472066.n3.nabble.com/Phrase-search-error-tp3423799p3423799.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Phrase search error
Posted by "Jason, Kim" <hi...@gmail.com>.
Hi, Ludovic
That's just what I'm looking for.
You're been a big help.
Thank you so much.
Jason.
--
View this message in context: http://lucene.472066.n3.nabble.com/Phrase-search-error-tp3423799p3423916.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Phrase search error
Posted by lboutros <bo...@gmail.com>.
Hi Jason,
you could add this filter to the end of your analyzer :
http://wiki.apache.org/solr/AnalyzersTokenizersTokenFilters#solr.PositionFilterFactory
That should solve your problem.
Ludovic.
-----
Jouve
France.
--
View this message in context: http://lucene.472066.n3.nabble.com/Phrase-search-error-tp3423799p3423893.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Multi CPU Cores
Posted by Chris Hostetter <ho...@fucit.org>.
: Subject: Multi CPU Cores
: In-Reply-To: <13...@n3.nabble.com>
: References: <13...@n3.nabble.com>
http://people.apache.org/~hossman/#threadhijack
Thread Hijacking on Mailing Lists
When starting a new discussion on a mailing list, please do not reply to
an existing message, instead start a fresh email. Even if you change the
subject line of your email, other mail headers still track which thread
you replied to and your question is "hidden" in that thread and gets less
attention. It makes following discussions in the mailing list archives
particularly difficult.
-Hoss
Re: Multi CPU Cores
Posted by Li Li <fa...@gmail.com>.
for indexing, your can make use of multi cores easily by call
IndexWriter.addDocument with multi-threads
as far as I know, for searching, if there is only one request, you can't
make good use of cpus.
On Sat, Oct 15, 2011 at 9:37 PM, Rob Brown <ro...@intelcompute.com> wrote:
> Hi,
>
> I'm running Solr on a machine with 16 CPU cores, yet watching "top" shows
> that java is only apparently using 1 and maxing it out.
>
> Is there anything that can be done to take advantage of more CPU cores?
>
> Solr 3.4 under Tomcat
>
> [root@solr01 ~]# java -version
> java version "1.6.0_20"
> OpenJDK Runtime Environment (IcedTea6 1.9.8) (rhel-1.22.1.9.8.el5_6-x86_64)
> OpenJDK 64-Bit Server VM (build 19.0-b09, mixed mode)
>
>
> top - 14:36:18 up 22 days, 21:54, 4 users, load average: 1.89, 1.24, 1.08
> Tasks: 317 total, 1 running, 315 sleeping, 0 stopped, 1 zombie
> Cpu0 : 0.0%us, 0.0%sy, 0.0%ni, 99.6%id, 0.4%wa, 0.0%hi, 0.0%si,
> 0.0%st
> Cpu1 : 0.0%us, 0.0%sy, 0.0%ni,100.0%id, 0.0%wa, 0.0%hi, 0.0%si,
> 0.0%st
> Cpu2 : 0.0%us, 0.0%sy, 0.0%ni,100.0%id, 0.0%wa, 0.0%hi, 0.0%si,
> 0.0%st
> Cpu3 : 0.0%us, 0.0%sy, 0.0%ni,100.0%id, 0.0%wa, 0.0%hi, 0.0%si,
> 0.0%st
> Cpu4 : 0.0%us, 0.0%sy, 0.0%ni,100.0%id, 0.0%wa, 0.0%hi, 0.0%si,
> 0.0%st
> Cpu5 : 0.0%us, 0.0%sy, 0.0%ni,100.0%id, 0.0%wa, 0.0%hi, 0.0%si,
> 0.0%st
> Cpu6 : 99.6%us, 0.4%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si,
> 0.0%st
> Cpu7 : 0.0%us, 0.0%sy, 0.0%ni,100.0%id, 0.0%wa, 0.0%hi, 0.0%si,
> 0.0%st
> Cpu8 : 0.0%us, 0.0%sy, 0.0%ni,100.0%id, 0.0%wa, 0.0%hi, 0.0%si,
> 0.0%st
> Cpu9 : 0.0%us, 0.0%sy, 0.0%ni,100.0%id, 0.0%wa, 0.0%hi, 0.0%si,
> 0.0%st
> Cpu10 : 0.0%us, 0.0%sy, 0.0%ni,100.0%id, 0.0%wa, 0.0%hi, 0.0%si,
> 0.0%st
> Cpu11 : 0.0%us, 0.0%sy, 0.0%ni,100.0%id, 0.0%wa, 0.0%hi, 0.0%si,
> 0.0%st
> Cpu12 : 0.0%us, 0.0%sy, 0.0%ni,100.0%id, 0.0%wa, 0.0%hi, 0.0%si,
> 0.0%st
> Cpu13 : 0.7%us, 0.0%sy, 0.0%ni, 99.3%id, 0.0%wa, 0.0%hi, 0.0%si,
> 0.0%st
> Cpu14 : 0.0%us, 0.0%sy, 0.0%ni,100.0%id, 0.0%wa, 0.0%hi, 0.0%si,
> 0.0%st
> Cpu15 : 0.0%us, 0.0%sy, 0.0%ni,100.0%id, 0.0%wa, 0.0%hi, 0.0%si,
> 0.0%st
> Mem: 132088928k total, 23760584k used, 108328344k free, 318228k buffers
> Swap: 25920868k total, 0k used, 25920868k free, 18371128k cached
>
> PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
> 4466 tomcat 20 0 31.2g 4.0g 171m S 101.0 3.2 2909:38 java
> 6495 root 15 0 42416 3892 1740 S 0.4 0.0 9:34.71 openvpn
> 11456 root 16 0 12892 1312 836 R 0.4 0.0 0:00.08 top
> 1 root 15 0 10368 632 536 S 0.0 0.0 0:04.69 init
>
>
Multi CPU Cores
Posted by Rob Brown <ro...@intelcompute.com>.
Hi,
I'm running Solr on a machine with 16 CPU cores, yet watching "top" shows that java is only apparently using 1 and maxing it out.
Is there anything that can be done to take advantage of more CPU cores?
Solr 3.4 under Tomcat
[root@solr01 ~]# java -version
java version "1.6.0_20"
OpenJDK Runtime Environment (IcedTea6 1.9.8) (rhel-1.22.1.9.8.el5_6-x86_64)
OpenJDK 64-Bit Server VM (build 19.0-b09, mixed mode)
top - 14:36:18 up 22 days, 21:54, 4 users, load average: 1.89, 1.24, 1.08
Tasks: 317 total, 1 running, 315 sleeping, 0 stopped, 1 zombie
Cpu0 : 0.0%us, 0.0%sy, 0.0%ni, 99.6%id, 0.4%wa, 0.0%hi, 0.0%si, 0.0%st
Cpu1 : 0.0%us, 0.0%sy, 0.0%ni,100.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Cpu2 : 0.0%us, 0.0%sy, 0.0%ni,100.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Cpu3 : 0.0%us, 0.0%sy, 0.0%ni,100.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Cpu4 : 0.0%us, 0.0%sy, 0.0%ni,100.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Cpu5 : 0.0%us, 0.0%sy, 0.0%ni,100.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Cpu6 : 99.6%us, 0.4%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Cpu7 : 0.0%us, 0.0%sy, 0.0%ni,100.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Cpu8 : 0.0%us, 0.0%sy, 0.0%ni,100.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Cpu9 : 0.0%us, 0.0%sy, 0.0%ni,100.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Cpu10 : 0.0%us, 0.0%sy, 0.0%ni,100.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Cpu11 : 0.0%us, 0.0%sy, 0.0%ni,100.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Cpu12 : 0.0%us, 0.0%sy, 0.0%ni,100.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Cpu13 : 0.7%us, 0.0%sy, 0.0%ni, 99.3%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Cpu14 : 0.0%us, 0.0%sy, 0.0%ni,100.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Cpu15 : 0.0%us, 0.0%sy, 0.0%ni,100.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Mem: 132088928k total, 23760584k used, 108328344k free, 318228k buffers
Swap: 25920868k total, 0k used, 25920868k free, 18371128k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
4466 tomcat 20 0 31.2g 4.0g 171m S 101.0 3.2 2909:38 java
6495 root 15 0 42416 3892 1740 S 0.4 0.0 9:34.71 openvpn
11456 root 16 0 12892 1312 836 R 0.4 0.0 0:00.08 top
1 root 15 0 10368 632 536 S 0.0 0.0 0:04.69 init