You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "tom liu (JIRA)" <ji...@apache.org> on 2010/10/10 09:28:31 UTC
[jira] Issue Comment Edited: (SOLR-1395) Integrate Katta
[ https://issues.apache.org/jira/browse/SOLR-1395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12919478#action_12919478 ]
tom liu edited comment on SOLR-1395 at 10/10/10 3:26 AM:
---------------------------------------------------------
i use solr-4.0 newest code trunk, and katta 0.6.2, hadoop-0.20.2, zookeeper-3.3.1, after fixed some bugs , i run it.
the bugs is :
1. solr's ShardDoc.java, ShardFieldSortedHitQueue line 210 :
<pre>
final float f1 = e1.score==null?0.00f:e1.score;
final float f2 = e2.score==null?0.00f:e2.score;
</pre>
2. KattaSearchHandler.java, KattaMultiShardHandler may be return more results, so must include any results:
<pre>
if (results.isEmpty()) {
ssr.setResponse(new NamedList<Object>());
return;
}
+
+ NamedList<Object> nl = new NamedList<Object>();
+ NamedListCollection nlc = new NamedListCollection(nl);
+ for(KattaResponse kr : results){
+ nl = nlc.add(kr.getRsp().getResponse());
+ }
ssr.setResponse(nl);
}
+ private class NamedListCollection {
+ private NamedList<Object> _nl;
+ NamedListCollection(NamedList<Object> nl){
+ _nl = nl;
+ }
+ NamedList<Object> add(NamedList<Object> nl){
+ Iterator<Entry<String,Object>> it = nl.iterator();
+ while (it.hasNext()){
+ Entry<String,Object> entry = it.next();
+ String key = entry.getKey();
+ Object obj = entry.getValue();
+ Object old = _nl.remove(key);
+ if(old != null){
+ add(key, obj , old );
+ }else{
+ _nl.add(key, obj);
+ }
+ }
+ return _nl;
+ }
+ void add(String key,Object obj,Object old){
+ if(key.equals("response")){
+ SolrDocumentList doca = (SolrDocumentList)obj;
+ SolrDocumentList docb = (SolrDocumentList)old;
+ SolrDocumentList docs = new SolrDocumentList();
+ docs.setNumFound(doca.getNumFound()+docb.getNumFound());
+ //doca.setStart(doca.getStart()+docb.getStart());
+ docs.setMaxScore(Math.max(doca.getMaxScore(), docb.getMaxScore()));
+ docs.addAll(doca);
+ docs.addAll(docb);
+ _nl.add(key,docs);
+ }else if(key.equals("QueriedShards")){
+ Collection<String> qsa = (ArrayList<String>)obj;
+ Collection<String> qsb = (ArrayList<String>)old;
+ Collection<String> qs = new ArrayList<String>();
+ qs.addAll(qsa);
+ qs.addAll(qsb);
+ _nl.add(key, qs);
+ }
+ }
+ }
</pre>
was (Author: tom_lt):
i use solr-4.0 newest code trunk, and katta 0.6.2, hadoop-0.20.2, zookeeper-3.3.1, after fixed some bugs , i run it.
the bugs is :
1. solr's ShardDoc.java, ShardFieldSortedHitQueue line 210 :
final float f1 = e1.score==null?0.00f:e1.score;
final float f2 = e2.score==null?0.00f:e2.score;
2. KattaSearchHandler.java, KattaMultiShardHandler may be return more results, so must include any results:
if (results.isEmpty()) {
ssr.setResponse(new NamedList<Object>());
return;
}
+
+ NamedList<Object> nl = new NamedList<Object>();
+ NamedListCollection nlc = new NamedListCollection(nl);
+ for(KattaResponse kr : results){
+ nl = nlc.add(kr.getRsp().getResponse());
+ }
ssr.setResponse(nl);
}
+ private class NamedListCollection {
+ private NamedList<Object> _nl;
+ NamedListCollection(NamedList<Object> nl){
+ _nl = nl;
+ }
+ NamedList<Object> add(NamedList<Object> nl){
+ Iterator<Entry<String,Object>> it = nl.iterator();
+ while (it.hasNext()){
+ Entry<String,Object> entry = it.next();
+ String key = entry.getKey();
+ Object obj = entry.getValue();
+ Object old = _nl.remove(key);
+ if(old != null){
+ add(key, obj , old );
+ }else{
+ _nl.add(key, obj);
+ }
+ }
+ return _nl;
+ }
+ void add(String key,Object obj,Object old){
+ if(key.equals("response")){
+ SolrDocumentList doca = (SolrDocumentList)obj;
+ SolrDocumentList docb = (SolrDocumentList)old;
+ SolrDocumentList docs = new SolrDocumentList();
+ docs.setNumFound(doca.getNumFound()+docb.getNumFound());
+ //doca.setStart(doca.getStart()+docb.getStart());
+ docs.setMaxScore(Math.max(doca.getMaxScore(), docb.getMaxScore()));
+ docs.addAll(doca);
+ docs.addAll(docb);
+ _nl.add(key,docs);
+ }else if(key.equals("QueriedShards")){
+ Collection<String> qsa = (ArrayList<String>)obj;
+ Collection<String> qsb = (ArrayList<String>)old;
+ Collection<String> qs = new ArrayList<String>();
+ qs.addAll(qsa);
+ qs.addAll(qsb);
+ _nl.add(key, qs);
+ }
+ }
+ }
> Integrate Katta
> ---------------
>
> Key: SOLR-1395
> URL: https://issues.apache.org/jira/browse/SOLR-1395
> Project: Solr
> Issue Type: New Feature
> Affects Versions: 1.4
> Reporter: Jason Rutherglen
> Priority: Minor
> Fix For: Next
>
> Attachments: back-end.log, front-end.log, hadoop-core-0.19.0.jar, katta-core-0.6-dev.jar, katta.node.properties, katta.zk.properties, log4j-1.2.13.jar, solr-1395-1431-3.patch, solr-1395-1431-4.patch, solr-1395-1431-katta0.6.patch, solr-1395-1431-katta0.6.patch, solr-1395-1431.patch, SOLR-1395.patch, SOLR-1395.patch, SOLR-1395.patch, test-katta-core-0.6-dev.jar, zkclient-0.1-dev.jar, zookeeper-3.2.1.jar
>
> Original Estimate: 336h
> Remaining Estimate: 336h
>
> We'll integrate Katta into Solr so that:
> * Distributed search uses Hadoop RPC
> * Shard/SolrCore distribution and management
> * Zookeeper based failover
> * Indexes may be built using Hadoop
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org