You are viewing a plain text version of this content. The canonical link for it is here.
Posted to java-user@lucene.apache.org by Wilson Wu <so...@gmail.com> on 2009/10/23 08:18:08 UTC
help needed improving lucene concurret search performance
Dear Friend,
I have encountered some performance problems recently in lucene
search 2.9. I use a single IndexSearcher in the whole system, It seems
perfect when there is less than 10 threads doing search concurrenty.
Bu if there is more than 100 threads doing concurrent search,the
average response time is becoming bigger(>1s),and the max response
time reaches 299s. I really don't know how to improve,can you help me?
Thanks a lot !
Wilson
2009.10.23
The profiling result about 400 concurret search is at:
http://i3.6.cn/cvbnm/aa/f5/00/63521d982a469f5063b82268eee91d08.gif
it seems a lot of time consumed by TermScorer.score.
Follewing is my servlet class which is reponse to search request:
public final class DispatchServlet extends
javax.servlet.http.HttpServlet implements javax.servlet.Servlet
{
private static final long serialVersionUID = -5547647006004900451L;
protected final Log log = LogFactory.getLog(getClass());
protected Searcher searcher;
protected Directory dir;
protected RAMDirectory ram;
public DispatchServlet() {
super();
}
public void init() throws ServletException {
super.init();
try {
dir = FSDirectory.open(new
File("/usr/bestv/search_engin_index/index/program"));
ram = new RAMDirectory(dir);
searcher = new IndexSearcher(ram,true);
int h = searcher.search(tq,null,1).totalHits;
System.out.println("the searcher has warmed and
searched " + h + " docs" );
}
} catch (IOException e) {
log.error(e);
}
}
protected void doPost(HttpServletRequest request, HttpServletResponse
response) throws ServletException, IOException {
response.setContentType("text/html");
doExecute(request.getParameter("q"),response);
}
protected void doGet(HttpServletRequest request, HttpServletResponse
response) throws ServletException, IOException {
response.setContentType("text/html");
try{
String schCon =
URLDecoder.decode(request.getParameter("q"),"UTF-8");
doExecute(schCon,response);
}catch(Exception e){
response.getWriter().write("Parameter
Error,please send param 'q'");
}
}
public void doExecute(String schCon,HttpServletResponse response)
throws ServletException,IOException{
response.getWriter().write(new SearchCommand().search(searcher));
}
}
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: help needed improving lucene concurret search performance
Posted by Yonik Seeley <yo...@lucidimagination.com>.
How many processors do you have on this system?
If you are CPU bound, 100 threads is going to be 10 times slower (at a
minimum) than 10 threads (unless you have more than 10 CPUs).
-Yonik
http://www.lucidimagination.com
On Fri, Oct 23, 2009 at 2:18 AM, Wilson Wu <so...@gmail.com> wrote:
> Dear Friend,
> I have encountered some performance problems recently in lucene
> search 2.9. I use a single IndexSearcher in the whole system, It seems
> perfect when there is less than 10 threads doing search concurrenty.
> Bu if there is more than 100 threads doing concurrent search,the
> average response time is becoming bigger(>1s),and the max response
> time reaches 299s. I really don't know how to improve,can you help me?
> Thanks a lot !
>
> Wilson
> 2009.10.23
> The profiling result about 400 concurret search is at:
> http://i3.6.cn/cvbnm/aa/f5/00/63521d982a469f5063b82268eee91d08.gif
> it seems a lot of time consumed by TermScorer.score.
> Follewing is my servlet class which is reponse to search request:
> public final class DispatchServlet extends
> javax.servlet.http.HttpServlet implements javax.servlet.Servlet
> {
> private static final long serialVersionUID = -5547647006004900451L;
> protected final Log log = LogFactory.getLog(getClass());
> protected Searcher searcher;
> protected Directory dir;
> protected RAMDirectory ram;
>
> public DispatchServlet() {
> super();
> }
>
> public void init() throws ServletException {
> super.init();
> try {
>
> dir = FSDirectory.open(new
> File("/usr/bestv/search_engin_index/index/program"));
> ram = new RAMDirectory(dir);
> searcher = new IndexSearcher(ram,true);
> int h = searcher.search(tq,null,1).totalHits;
> System.out.println("the searcher has warmed and
> searched " + h + " docs" );
> }
> } catch (IOException e) {
> log.error(e);
> }
> }
>
> protected void doPost(HttpServletRequest request, HttpServletResponse
> response) throws ServletException, IOException {
> response.setContentType("text/html");
> doExecute(request.getParameter("q"),response);
> }
>
> protected void doGet(HttpServletRequest request, HttpServletResponse
> response) throws ServletException, IOException {
> response.setContentType("text/html");
> try{
> String schCon =
> URLDecoder.decode(request.getParameter("q"),"UTF-8");
> doExecute(schCon,response);
> }catch(Exception e){
> response.getWriter().write("Parameter
> Error,please send param 'q'");
> }
> }
>
> public void doExecute(String schCon,HttpServletResponse response)
> throws ServletException,IOException{
> response.getWriter().write(new SearchCommand().search(searcher));
> }
>
> }
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org