You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-dev@lucene.apache.org by "Jason Rutherglen (JIRA)" <ji...@apache.org> on 2009/08/30 01:15:33 UTC
[jira] Created: (SOLR-1395) Integrate Katta
Integrate Katta
---------------
Key: SOLR-1395
URL: https://issues.apache.org/jira/browse/SOLR-1395
Project: Solr
Issue Type: New Feature
Affects Versions: 1.4
Reporter: Jason Rutherglen
Priority: Minor
Fix For: 1.5
We'll integrate Katta into Solr so that:
* Distributed search uses Hadoop RPC
* Shard/SolrCore distribution and management
* Zookeeper based failover
* Indexes may be built using Hadoop
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-1395) Integrate Katta
Posted by "Thomas Koch (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12834273#action_12834273 ]
Thomas Koch commented on SOLR-1395:
-----------------------------------
I'd also need katta integration at least for search, since my frontend is PHP (sorry...) and so I can't communicate as easily from PHP to Java as from PHP to SOLR.
Has anybody already done an updated patch or could help me to do it?
> Integrate Katta
> ---------------
>
> Key: SOLR-1395
> URL: https://issues.apache.org/jira/browse/SOLR-1395
> Project: Solr
> Issue Type: New Feature
> Affects Versions: 1.4
> Reporter: Jason Rutherglen
> Priority: Minor
> Fix For: 1.5
>
> Attachments: hadoop-core-0.19.0.jar, katta-core-0.6-dev.jar, katta.node.properties, katta.zk.properties, log4j-1.2.13.jar, solr-1395-1431-3.patch, solr-1395-1431-4.patch, solr-1395-1431.patch, SOLR-1395.patch, SOLR-1395.patch, SOLR-1395.patch, test-katta-core-0.6-dev.jar, zkclient-0.1-dev.jar, zookeeper-3.2.1.jar
>
> Original Estimate: 336h
> Remaining Estimate: 336h
>
> We'll integrate Katta into Solr so that:
> * Distributed search uses Hadoop RPC
> * Shard/SolrCore distribution and management
> * Zookeeper based failover
> * Indexes may be built using Hadoop
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-1395) Integrate Katta
Posted by "Jason Rutherglen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12832587#action_12832587 ]
Jason Rutherglen commented on SOLR-1395:
----------------------------------------
shyjuThomas,
It'd be good to update this patch to the latest Katta... You're welcome to do so... For my project I only need what'll be in SOLR-1724...
> Integrate Katta
> ---------------
>
> Key: SOLR-1395
> URL: https://issues.apache.org/jira/browse/SOLR-1395
> Project: Solr
> Issue Type: New Feature
> Affects Versions: 1.4
> Reporter: Jason Rutherglen
> Priority: Minor
> Fix For: 1.5
>
> Attachments: hadoop-core-0.19.0.jar, katta-core-0.6-dev.jar, katta.node.properties, katta.zk.properties, log4j-1.2.13.jar, solr-1395-1431-3.patch, solr-1395-1431-4.patch, solr-1395-1431.patch, SOLR-1395.patch, SOLR-1395.patch, SOLR-1395.patch, test-katta-core-0.6-dev.jar, zkclient-0.1-dev.jar, zookeeper-3.2.1.jar
>
> Original Estimate: 336h
> Remaining Estimate: 336h
>
> We'll integrate Katta into Solr so that:
> * Distributed search uses Hadoop RPC
> * Shard/SolrCore distribution and management
> * Zookeeper based failover
> * Indexes may be built using Hadoop
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-1395) Integrate Katta
Posted by "Jason Rutherglen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12760955#action_12760955 ]
Jason Rutherglen commented on SOLR-1395:
----------------------------------------
Jason, Can you upload a SOLR-1395 only patch? That will help in seeing the SOLR-1395 specific changes.
I think the next step is to remove the dependency on separate property files, as I find these hard to manage (they are too numerous).
> Integrate Katta
> ---------------
>
> Key: SOLR-1395
> URL: https://issues.apache.org/jira/browse/SOLR-1395
> Project: Solr
> Issue Type: New Feature
> Affects Versions: 1.4
> Reporter: Jason Rutherglen
> Priority: Minor
> Fix For: 1.5
>
> Attachments: hadoop-core-0.19.0.jar, katta-core-0.6-dev.jar, katta.node.properties, katta.zk.properties, log4j-1.2.13.jar, solr-1395-1431-3.patch, solr-1395-1431.patch, SOLR-1395.patch, SOLR-1395.patch, SOLR-1395.patch, test-katta-core-0.6-dev.jar, zkclient-0.1-dev.jar, zookeeper-3.2.1.jar
>
> Original Estimate: 336h
> Remaining Estimate: 336h
>
> We'll integrate Katta into Solr so that:
> * Distributed search uses Hadoop RPC
> * Shard/SolrCore distribution and management
> * Zookeeper based failover
> * Indexes may be built using Hadoop
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-1395) Integrate Katta
Posted by "Jason Venner (www.prohadoop.com) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12766077#action_12766077 ]
Jason Venner (www.prohadoop.com) commented on SOLR-1395:
--------------------------------------------------------
My apologies, I think it is in the katta tree,
Katta-80?
http://oss.101tec.com/jira/browse/KATTA-80
--
Jason Venner
Author: Pro Hadoop A howto guide to learning and using hadoop and map/reduce
http://www.prohadoopbook.com/ a Ning network for Hadoop using Professionals
> Integrate Katta
> ---------------
>
> Key: SOLR-1395
> URL: https://issues.apache.org/jira/browse/SOLR-1395
> Project: Solr
> Issue Type: New Feature
> Affects Versions: 1.4
> Reporter: Jason Rutherglen
> Priority: Minor
> Fix For: 1.5
>
> Attachments: hadoop-core-0.19.0.jar, katta-core-0.6-dev.jar, katta.node.properties, katta.zk.properties, log4j-1.2.13.jar, solr-1395-1431-3.patch, solr-1395-1431-4.patch, solr-1395-1431.patch, SOLR-1395.patch, SOLR-1395.patch, SOLR-1395.patch, test-katta-core-0.6-dev.jar, zkclient-0.1-dev.jar, zookeeper-3.2.1.jar
>
> Original Estimate: 336h
> Remaining Estimate: 336h
>
> We'll integrate Katta into Solr so that:
> * Distributed search uses Hadoop RPC
> * Shard/SolrCore distribution and management
> * Zookeeper based failover
> * Indexes may be built using Hadoop
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Issue Comment Edited: (SOLR-1395) Integrate Katta
Posted by "jv ning (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12760635#action_12760635 ]
jv ning edited comment on SOLR-1395 at 9/29/09 9:07 AM:
--------------------------------------------------------
the file /tmp/solr-1395-1431.patch is a combined patch of 1431 and 1395.
A small api change in the query string parser required a small code change.
was (Author: jv_ning):
the file /tmp/solr-1395-1431.patch is a combined patch of 1431 and 1395.
A small api change in the query parser required a small code change.
> Integrate Katta
> ---------------
>
> Key: SOLR-1395
> URL: https://issues.apache.org/jira/browse/SOLR-1395
> Project: Solr
> Issue Type: New Feature
> Affects Versions: 1.4
> Reporter: Jason Rutherglen
> Priority: Minor
> Fix For: 1.5
>
> Attachments: hadoop-core-0.19.0.jar, katta-core-0.6-dev.jar, log4j-1.2.13.jar, solr-1395-1431.patch, SOLR-1395.patch, SOLR-1395.patch, SOLR-1395.patch, test-katta-core-0.6-dev.jar, zkclient-0.1-dev.jar, zookeeper-3.2.1.jar
>
> Original Estimate: 336h
> Remaining Estimate: 336h
>
> We'll integrate Katta into Solr so that:
> * Distributed search uses Hadoop RPC
> * Shard/SolrCore distribution and management
> * Zookeeper based failover
> * Indexes may be built using Hadoop
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-1395) Integrate Katta
Posted by "Jason Rutherglen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12754241#action_12754241 ]
Jason Rutherglen commented on SOLR-1395:
----------------------------------------
I added a wiki page at: http://wiki.apache.org/solr/KattaIntegration
> Integrate Katta
> ---------------
>
> Key: SOLR-1395
> URL: https://issues.apache.org/jira/browse/SOLR-1395
> Project: Solr
> Issue Type: New Feature
> Affects Versions: 1.4
> Reporter: Jason Rutherglen
> Priority: Minor
> Fix For: 1.5
>
> Attachments: KATTA-SOLR.patch, SOLR-1395.patch
>
> Original Estimate: 336h
> Remaining Estimate: 336h
>
> We'll integrate Katta into Solr so that:
> * Distributed search uses Hadoop RPC
> * Shard/SolrCore distribution and management
> * Zookeeper based failover
> * Indexes may be built using Hadoop
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Issue Comment Edited: (SOLR-1395) Integrate Katta
Posted by "jv ning (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12760635#action_12760635 ]
jv ning edited comment on SOLR-1395 at 9/29/09 9:08 AM:
--------------------------------------------------------
the file /tmp/solr-1395-1431.patch is a combined patch of 1431 and 1395.
A small api change in the query string creator required a small code change.
ClientUtils.toQueryString, now prefixes the returned query string with a '?' character
was (Author: jv_ning):
the file /tmp/solr-1395-1431.patch is a combined patch of 1431 and 1395.
A small api change in the query string parser required a small code change.
> Integrate Katta
> ---------------
>
> Key: SOLR-1395
> URL: https://issues.apache.org/jira/browse/SOLR-1395
> Project: Solr
> Issue Type: New Feature
> Affects Versions: 1.4
> Reporter: Jason Rutherglen
> Priority: Minor
> Fix For: 1.5
>
> Attachments: hadoop-core-0.19.0.jar, katta-core-0.6-dev.jar, log4j-1.2.13.jar, solr-1395-1431.patch, SOLR-1395.patch, SOLR-1395.patch, SOLR-1395.patch, test-katta-core-0.6-dev.jar, zkclient-0.1-dev.jar, zookeeper-3.2.1.jar
>
> Original Estimate: 336h
> Remaining Estimate: 336h
>
> We'll integrate Katta into Solr so that:
> * Distributed search uses Hadoop RPC
> * Shard/SolrCore distribution and management
> * Zookeeper based failover
> * Indexes may be built using Hadoop
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Issue Comment Edited: (SOLR-1395) Integrate Katta
Posted by "Jason Venner (at ning) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12760726#action_12760726 ]
Jason Venner (at ning) edited comment on SOLR-1395 at 9/29/09 3:09 PM:
-----------------------------------------------------------------------
/tmp/solr-1395-1431-3.patch contains an additional unit test for the query string serialization code, and two additional classes that allow for deployment to katta.
WIth this jar, a katta client node may be started via
katta-daemon.sh start katta\ startNode org.apache.solr.katta.DeployableSolrKattaServer
The system properties that control the node startup are
solr.server.name - the property to look for the server name, default proxy
solr.home - the property to look for the server root, default solrHome
solr.config.file - the property to look for the server config file name, default solr.xml
These will be used to find a solr configuration to run the embedded server which will search the deployed shards.
Index shards may be deployed via the standard katta mechanism of katta addIndex index-name shared-path-to-index
I use the zip files produced by SOLR-1301 and deploy from hdfs.
For searching, create a solr configuration with a handler:
<requestHandler name="standard" class="solr.KattaRequestHandler" default="true">
<!-- default values for query parameters -->
<lst name="defaults">
<str name="echoParams">explicit</str>
<!--
<int name="rows">10</int>
<str name="fl">*</str>
<str name="version">2.1</str>
-->
<str name="shards">*</str>
</lst>
</requestHandler>
This will search all deployed shards, replace the shards parameter with an explicit shard list if you only wish to query an explicit subset with this query handler.
The solr instance for search will need the zookeeper information.
conf/katta.node.properties
conf/katta.zk.properties, replace the zookeeper nodes with your clusters nodes
I tend to run java -d64 -Xmx2g -Dkatta.request.timeout=100000 start.jar
for my testing work as my cluster is on the far side of a couple of firewallss
I also have to store my katta.zk.properties file in the start.jar for some reason
was (Author: jv_ning):
/tmp/solr-1395-1431-3.patch contains an additional unit test for the query string serialization code, and two additional classes that allow for deployment to katta.
WIth this jar, a katta client node may be started via
katta-daemon.sh start katta\ startNode org.apache.solr.katta.DeployableSolrKattaServer
The system properties that control the node startup are
solr.server.name - the property to look for the server name, default proxy
solr.home - the property to look for the server root, default solrHome
solr.config.file - the property to look for the server config file name, default solr.xml
These will be used to find a solr configuration to run the embedded server which will search the deployed shards.
Index shards may be deployed via the standard katta mechanism of katta addIndex index-name shared-path-to-index
I use the zip files produced by SOLR-1301 and deploy from hdfs.
For searching, create a solr configuration with a handler:
<requestHandler name="standard" class="solr.KattaRequestHandler" default="true">
<!-- default values for query parameters -->
<lst name="defaults">
<str name="echoParams">explicit</str>
<!--
<int name="rows">10</int>
<str name="fl">*</str>
<str name="version">2.1</str>
-->
<str name="shards">*</str>
</lst>
</requestHandler>
This will search all deployed shards, replace the shards parameter with an explicit shard list if you only wish to query an explicit subset with this query handler.
The solr instance for search will need the zookeeper information.
conf/katta.node.properties
conf/katta.zk.properties, replace the zookeeper nodes with your clusters nodes
I tend to run java -d64 -Xmx2g -Dkatta.request.timeout=100000 start.jar
for my testing work as my cluster is on the far side of a couple of firewals
> Integrate Katta
> ---------------
>
> Key: SOLR-1395
> URL: https://issues.apache.org/jira/browse/SOLR-1395
> Project: Solr
> Issue Type: New Feature
> Affects Versions: 1.4
> Reporter: Jason Rutherglen
> Priority: Minor
> Fix For: 1.5
>
> Attachments: hadoop-core-0.19.0.jar, katta-core-0.6-dev.jar, katta.node.properties, katta.zk.properties, log4j-1.2.13.jar, solr-1395-1431-3.patch, solr-1395-1431.patch, SOLR-1395.patch, SOLR-1395.patch, SOLR-1395.patch, test-katta-core-0.6-dev.jar, zkclient-0.1-dev.jar, zookeeper-3.2.1.jar
>
> Original Estimate: 336h
> Remaining Estimate: 336h
>
> We'll integrate Katta into Solr so that:
> * Distributed search uses Hadoop RPC
> * Shard/SolrCore distribution and management
> * Zookeeper based failover
> * Indexes may be built using Hadoop
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-1395) Integrate Katta
Posted by "Jason Rutherglen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12755228#action_12755228 ]
Jason Rutherglen commented on SOLR-1395:
----------------------------------------
Noble, great idea! I opened an issue at SOLR-1431.
> Integrate Katta
> ---------------
>
> Key: SOLR-1395
> URL: https://issues.apache.org/jira/browse/SOLR-1395
> Project: Solr
> Issue Type: New Feature
> Affects Versions: 1.4
> Reporter: Jason Rutherglen
> Priority: Minor
> Fix For: 1.5
>
> Attachments: hadoop-core-0.19.0.jar, KATTA-SOLR.patch, log4j-1.2.13.jar, SOLR-1395.patch, zookeeper-3.2.1.jar
>
> Original Estimate: 336h
> Remaining Estimate: 336h
>
> We'll integrate Katta into Solr so that:
> * Distributed search uses Hadoop RPC
> * Shard/SolrCore distribution and management
> * Zookeeper based failover
> * Indexes may be built using Hadoop
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-1395) Integrate Katta
Posted by "Jason Venner (at ning) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jason Venner (at ning) updated SOLR-1395:
-----------------------------------------
Attachment: katta.zk.properties
katta.node.properties
solr-1395-1431-3.patch
/tmp/solr-1395-1431-3.patch contains an additional unit test for the query string serialization code, and two additional classes that allow for deployment to katta.
WIth this jar, a katta client node may be started via
katta-daemon.sh start katta\ startNode org.apache.solr.katta.DeployableSolrKattaServer
The system properties that control the node startup are
solr.server.name - the property to look for the server name, default proxy
solr.home - the property to look for the server root, default solrHome
solr.config.file - the property to look for the server config file name, default solr.xml
These will be used to find a solr configuration to run the embedded server which will search the deployed shards.
Index shards may be deployed via the standard katta mechanism of katta addIndex index-name shared-path-to-index
I use the zip files produced by SOLR-1301 and deploy from hdfs.
For searching, create a solr configuration with a handler:
<requestHandler name="standard" class="solr.KattaRequestHandler" default="true">
<!-- default values for query parameters -->
<lst name="defaults">
<str name="echoParams">explicit</str>
<!--
<int name="rows">10</int>
<str name="fl">*</str>
<str name="version">2.1</str>
-->
<str name="shards">*</str>
</lst>
</requestHandler>
This will search all deployed shards, replace the shards parameter with an explicit shard list if you only wish to query an explicit subset with this query handler.
The solr instance for search will need the zookeeper information.
conf/katta.node.properties
conf/katta.zk.properties, replace the zookeeper nodes with your clusters nodes
I tend to run java -d64 -Xmx2g -Dkatta.request.timeout=100000 start.jar
for my testing work as my cluster is on the far side of a couple of firewals
> Integrate Katta
> ---------------
>
> Key: SOLR-1395
> URL: https://issues.apache.org/jira/browse/SOLR-1395
> Project: Solr
> Issue Type: New Feature
> Affects Versions: 1.4
> Reporter: Jason Rutherglen
> Priority: Minor
> Fix For: 1.5
>
> Attachments: hadoop-core-0.19.0.jar, katta-core-0.6-dev.jar, katta.node.properties, katta.zk.properties, log4j-1.2.13.jar, solr-1395-1431-3.patch, solr-1395-1431.patch, SOLR-1395.patch, SOLR-1395.patch, SOLR-1395.patch, test-katta-core-0.6-dev.jar, zkclient-0.1-dev.jar, zookeeper-3.2.1.jar
>
> Original Estimate: 336h
> Remaining Estimate: 336h
>
> We'll integrate Katta into Solr so that:
> * Distributed search uses Hadoop RPC
> * Shard/SolrCore distribution and management
> * Zookeeper based failover
> * Indexes may be built using Hadoop
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-1395) Integrate Katta
Posted by "Jason Rutherglen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jason Rutherglen updated SOLR-1395:
-----------------------------------
Attachment: hadoop-core-0.19.0.jar
log4j-1.2.13.jar
zookeeper-3.2.1.jar
These are the external libraries necessary to run the test
> Integrate Katta
> ---------------
>
> Key: SOLR-1395
> URL: https://issues.apache.org/jira/browse/SOLR-1395
> Project: Solr
> Issue Type: New Feature
> Affects Versions: 1.4
> Reporter: Jason Rutherglen
> Priority: Minor
> Fix For: 1.5
>
> Attachments: hadoop-core-0.19.0.jar, KATTA-SOLR.patch, log4j-1.2.13.jar, SOLR-1395.patch, zookeeper-3.2.1.jar
>
> Original Estimate: 336h
> Remaining Estimate: 336h
>
> We'll integrate Katta into Solr so that:
> * Distributed search uses Hadoop RPC
> * Shard/SolrCore distribution and management
> * Zookeeper based failover
> * Indexes may be built using Hadoop
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-1395) Integrate Katta
Posted by "Jason Rutherglen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jason Rutherglen updated SOLR-1395:
-----------------------------------
Attachment: (was: KATTA-SOLR.patch)
> Integrate Katta
> ---------------
>
> Key: SOLR-1395
> URL: https://issues.apache.org/jira/browse/SOLR-1395
> Project: Solr
> Issue Type: New Feature
> Affects Versions: 1.4
> Reporter: Jason Rutherglen
> Priority: Minor
> Fix For: 1.5
>
> Attachments: hadoop-core-0.19.0.jar, katta-core-0.6-dev.jar, log4j-1.2.13.jar, SOLR-1395.patch, SOLR-1395.patch, test-katta-core-0.6-dev.jar, zkclient-0.1-dev.jar, zookeeper-3.2.1.jar
>
> Original Estimate: 336h
> Remaining Estimate: 336h
>
> We'll integrate Katta into Solr so that:
> * Distributed search uses Hadoop RPC
> * Shard/SolrCore distribution and management
> * Zookeeper based failover
> * Indexes may be built using Hadoop
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-1395) Integrate Katta
Posted by "Jason Rutherglen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jason Rutherglen updated SOLR-1395:
-----------------------------------
Attachment: KATTA-SOLR.patch
SOLR-1395.patch
This is our first cut at integrating Katta with Solr. The
KattaClientTest test case shows a Katta cluster being created
locally, a couple of cores/shards being placed into the cluster,
then a query being executed that returns the correct number of
results. It takes about 30s - 1.5min to run (hopefully that can
be reduced?).
Today Solr shards map to Solr servers. Here we map shards to
cores, where there can be multiple shards per server or in Katta
parlance a node. We assume the shards exist in Hadoop HDFS.
Katta copies the shards to a local Solr server to make them
searchable (and incrementally updateable).
h3. Instructions for Installation
* Download Katta trunk "svn co
https://katta.svn.sourceforge.net/svnroot/katta/trunk
kattatrunk". Download the KATTA-SOLR.patch to kattatrunk. run "patch
-p 0 -i KATTA-SOLR.patch", "ant -jar", "ant jar-test".
* Download a Solr trunk "svn co
http://svn.apache.org/repos/asf/lucene/solr/trunk solrtrunk".
Copy from kattatrunk: lib/log4j-1.2.13.jar
lib/zookeeper-3.1.1.jar lib/hadoop-core-0.19.0.jar
build/katta-core-0.6-dev.jar build/test-katta-core-0.6-dev.jar
to solrtrunk/lib
* Download SOLR-1395.patch to solrtrunk. Run "patch -p 0 -i
SOLR-1395.patch".
* Run a test while in solrtrunk "ant test-core
-Dtestcase=KattaClientTest"
h3. General Notes
* SearchHandler's HttpCommComponent has been abstracted out.
There's a CommComponent interface, AbstractCommComponent
implements the generic multithreading ShardRequest ->
ShardResponse logic. EmbeddedSearchHandler executes requests on
a set of local cores. HttpCommComponent implements requests over
HTTP. KattaCommComponent distributes requests using Katta's
Hadoop RPC mechanism.
* The patch enables all of Solr's distributed request types. All
current distributed requests should work as is with no
modifications.
* Shards/Solr cores may be managed dynamically and remotely
administered from a centralized location (whereas today Solr
typically requires SSHing and manually editing files etc)
* Solr Katta has built in failover, this is tested in
KattaClientFailoverTest
* When a shard is deployed to a Solr server, the schema and
solrconfig are deployed with it. This begs the question of how
updates to the solrconfig and schema are deployed. Redeploying
solrconfig is fairly simple, whereas a schema change implies
recreating the entire shard.
* Maybe there's an easy way to interface with Hadoop index
creation (i.e. as easy as Solr's HTTP based update command)
The patch was created by Jason Venner and Jason Rutherglen
> Integrate Katta
> ---------------
>
> Key: SOLR-1395
> URL: https://issues.apache.org/jira/browse/SOLR-1395
> Project: Solr
> Issue Type: New Feature
> Affects Versions: 1.4
> Reporter: Jason Rutherglen
> Priority: Minor
> Fix For: 1.5
>
> Attachments: KATTA-SOLR.patch, SOLR-1395.patch
>
> Original Estimate: 336h
> Remaining Estimate: 336h
>
> We'll integrate Katta into Solr so that:
> * Distributed search uses Hadoop RPC
> * Shard/SolrCore distribution and management
> * Zookeeper based failover
> * Indexes may be built using Hadoop
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-1395) Integrate Katta
Posted by "Jason Rutherglen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jason Rutherglen updated SOLR-1395:
-----------------------------------
Attachment: SOLR-1395.patch
New patch updated to Katta's latest from Git. It's slimmed down a bit, removing the various extraneous config files etc.
> Integrate Katta
> ---------------
>
> Key: SOLR-1395
> URL: https://issues.apache.org/jira/browse/SOLR-1395
> Project: Solr
> Issue Type: New Feature
> Affects Versions: 1.4
> Reporter: Jason Rutherglen
> Priority: Minor
> Fix For: 1.5
>
> Attachments: hadoop-core-0.19.0.jar, KATTA-SOLR.patch, log4j-1.2.13.jar, SOLR-1395.patch, SOLR-1395.patch, zookeeper-3.2.1.jar
>
> Original Estimate: 336h
> Remaining Estimate: 336h
>
> We'll integrate Katta into Solr so that:
> * Distributed search uses Hadoop RPC
> * Shard/SolrCore distribution and management
> * Zookeeper based failover
> * Indexes may be built using Hadoop
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-1395) Integrate Katta
Posted by "Jason Venner (www.prohadoop.com) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12766081#action_12766081 ]
Jason Venner (www.prohadoop.com) commented on SOLR-1395:
--------------------------------------------------------
AFIK this was committed also, so it is in katta trunk now.
--
Jason Venner
Author: Pro Hadoop A howto guide to learning and using hadoop and map/reduce
http://www.prohadoopbook.com/ a Ning network for Hadoop using Professionals
Stefan Groschupf
<http://oss.101tec.com/jira/secure/ViewProfile.jspa?name=sg> added a
comment - 13/Oct/09 09:15 PM
Just committed that, thanks Jason.
> Integrate Katta
> ---------------
>
> Key: SOLR-1395
> URL: https://issues.apache.org/jira/browse/SOLR-1395
> Project: Solr
> Issue Type: New Feature
> Affects Versions: 1.4
> Reporter: Jason Rutherglen
> Priority: Minor
> Fix For: 1.5
>
> Attachments: hadoop-core-0.19.0.jar, katta-core-0.6-dev.jar, katta.node.properties, katta.zk.properties, log4j-1.2.13.jar, solr-1395-1431-3.patch, solr-1395-1431-4.patch, solr-1395-1431.patch, SOLR-1395.patch, SOLR-1395.patch, SOLR-1395.patch, test-katta-core-0.6-dev.jar, zkclient-0.1-dev.jar, zookeeper-3.2.1.jar
>
> Original Estimate: 336h
> Remaining Estimate: 336h
>
> We'll integrate Katta into Solr so that:
> * Distributed search uses Hadoop RPC
> * Shard/SolrCore distribution and management
> * Zookeeper based failover
> * Indexes may be built using Hadoop
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-1395) Integrate Katta
Posted by "pravin karne (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12765983#action_12765983 ]
pravin karne commented on SOLR-1395:
------------------------------------
from where i can download KATTA-SOLR.patch
> Integrate Katta
> ---------------
>
> Key: SOLR-1395
> URL: https://issues.apache.org/jira/browse/SOLR-1395
> Project: Solr
> Issue Type: New Feature
> Affects Versions: 1.4
> Reporter: Jason Rutherglen
> Priority: Minor
> Fix For: 1.5
>
> Attachments: hadoop-core-0.19.0.jar, katta-core-0.6-dev.jar, katta.node.properties, katta.zk.properties, log4j-1.2.13.jar, solr-1395-1431-3.patch, solr-1395-1431-4.patch, solr-1395-1431.patch, SOLR-1395.patch, SOLR-1395.patch, SOLR-1395.patch, test-katta-core-0.6-dev.jar, zkclient-0.1-dev.jar, zookeeper-3.2.1.jar
>
> Original Estimate: 336h
> Remaining Estimate: 336h
>
> We'll integrate Katta into Solr so that:
> * Distributed search uses Hadoop RPC
> * Shard/SolrCore distribution and management
> * Zookeeper based failover
> * Indexes may be built using Hadoop
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
Re: [jira] Commented: (SOLR-1395) Integrate Katta
Posted by Jason Venner <jv...@ning.com>.
I seem to remember that Jason Rutherglen refactored it into 2 patches, but I do not remember at present the jira for the other patch.
1 patch provided an abstract handler with an http implementation that was equivalent to the standard http implementation
2 1395 provided a pair of katta handlers
On 2/17/10 11:12 AM, "Tatsuya Hayashi (JIRA)" <ji...@apache.org> wrote:
[ https://issues.apache.org/jira/browse/SOLR-1395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12834935#action_12834935 ]
Tatsuya Hayashi commented on SOLR-1395:
---------------------------------------
I downloaded a Solr trunk and copied necessary jar files to solrtrunk/lib.
And tried to apply the SOLR-1395.patch (patch -p 0 -i SOLR-1395.patch)
But I saw a failure message on the console.
patching file src/java/org/apache/solr/handler/component/SearchHandler.java
Hunk #1 FAILED at 17.
1 out of 4 hunks FAILED -- saving rejects to file src/java/org/apache/solr/handler/component/SearchHandler.java.rej
patching file src/solrj/org/apache/solr/client/solrj/request/QueryRequest.java
Could anyone give me any suggestion to solve it?
> Integrate Katta
> ---------------
>
> Key: SOLR-1395
> URL: https://issues.apache.org/jira/browse/SOLR-1395
> Project: Solr
> Issue Type: New Feature
> Affects Versions: 1.4
> Reporter: Jason Rutherglen
> Priority: Minor
> Fix For: 1.5
>
> Attachments: hadoop-core-0.19.0.jar, katta-core-0.6-dev.jar, katta.node.properties, katta.zk.properties, log4j-1.2.13.jar, solr-1395-1431-3.patch, solr-1395-1431-4.patch, solr-1395-1431.patch, SOLR-1395.patch, SOLR-1395.patch, SOLR-1395.patch, test-katta-core-0.6-dev.jar, zkclient-0.1-dev.jar, zookeeper-3.2.1.jar
>
> Original Estimate: 336h
> Remaining Estimate: 336h
>
> We'll integrate Katta into Solr so that:
> * Distributed search uses Hadoop RPC
> * Shard/SolrCore distribution and management
> * Zookeeper based failover
> * Indexes may be built using Hadoop
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-1395) Integrate Katta
Posted by "Tatsuya Hayashi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12834935#action_12834935 ]
Tatsuya Hayashi commented on SOLR-1395:
---------------------------------------
I downloaded a Solr trunk and copied necessary jar files to solrtrunk/lib.
And tried to apply the SOLR-1395.patch (patch -p 0 -i SOLR-1395.patch)
But I saw a failure message on the console.
patching file src/java/org/apache/solr/handler/component/SearchHandler.java
Hunk #1 FAILED at 17.
1 out of 4 hunks FAILED -- saving rejects to file src/java/org/apache/solr/handler/component/SearchHandler.java.rej
patching file src/solrj/org/apache/solr/client/solrj/request/QueryRequest.java
Could anyone give me any suggestion to solve it?
> Integrate Katta
> ---------------
>
> Key: SOLR-1395
> URL: https://issues.apache.org/jira/browse/SOLR-1395
> Project: Solr
> Issue Type: New Feature
> Affects Versions: 1.4
> Reporter: Jason Rutherglen
> Priority: Minor
> Fix For: 1.5
>
> Attachments: hadoop-core-0.19.0.jar, katta-core-0.6-dev.jar, katta.node.properties, katta.zk.properties, log4j-1.2.13.jar, solr-1395-1431-3.patch, solr-1395-1431-4.patch, solr-1395-1431.patch, SOLR-1395.patch, SOLR-1395.patch, SOLR-1395.patch, test-katta-core-0.6-dev.jar, zkclient-0.1-dev.jar, zookeeper-3.2.1.jar
>
> Original Estimate: 336h
> Remaining Estimate: 336h
>
> We'll integrate Katta into Solr so that:
> * Distributed search uses Hadoop RPC
> * Shard/SolrCore distribution and management
> * Zookeeper based failover
> * Indexes may be built using Hadoop
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-1395) Integrate Katta
Posted by "Jason Rutherglen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jason Rutherglen updated SOLR-1395:
-----------------------------------
Attachment: SOLR-1395.patch
Updated the KattaRequest class to properly serialize the SolrParams.
> Integrate Katta
> ---------------
>
> Key: SOLR-1395
> URL: https://issues.apache.org/jira/browse/SOLR-1395
> Project: Solr
> Issue Type: New Feature
> Affects Versions: 1.4
> Reporter: Jason Rutherglen
> Priority: Minor
> Fix For: 1.5
>
> Attachments: hadoop-core-0.19.0.jar, katta-core-0.6-dev.jar, log4j-1.2.13.jar, SOLR-1395.patch, SOLR-1395.patch, SOLR-1395.patch, test-katta-core-0.6-dev.jar, zkclient-0.1-dev.jar, zookeeper-3.2.1.jar
>
> Original Estimate: 336h
> Remaining Estimate: 336h
>
> We'll integrate Katta into Solr so that:
> * Distributed search uses Hadoop RPC
> * Shard/SolrCore distribution and management
> * Zookeeper based failover
> * Indexes may be built using Hadoop
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-1395) Integrate Katta
Posted by "Sumit (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12849674#action_12849674 ]
Sumit commented on SOLR-1395:
-----------------------------
In First step of installation when jar created using ant -jar does not contain any file related to integrated solr-katta. Also many test cases are failing like NodeMockTest.java, ShardManagerTest.java etc giving an error as invalid URI for shard indexes..... Can some one help me out?
> Integrate Katta
> ---------------
>
> Key: SOLR-1395
> URL: https://issues.apache.org/jira/browse/SOLR-1395
> Project: Solr
> Issue Type: New Feature
> Affects Versions: 1.4
> Reporter: Jason Rutherglen
> Priority: Minor
> Fix For: 1.5
>
> Attachments: hadoop-core-0.19.0.jar, katta-core-0.6-dev.jar, katta.node.properties, katta.zk.properties, log4j-1.2.13.jar, solr-1395-1431-3.patch, solr-1395-1431-4.patch, solr-1395-1431-katta0.6.patch, solr-1395-1431.patch, SOLR-1395.patch, SOLR-1395.patch, SOLR-1395.patch, test-katta-core-0.6-dev.jar, zkclient-0.1-dev.jar, zookeeper-3.2.1.jar
>
> Original Estimate: 336h
> Remaining Estimate: 336h
>
> We'll integrate Katta into Solr so that:
> * Distributed search uses Hadoop RPC
> * Shard/SolrCore distribution and management
> * Zookeeper based failover
> * Indexes may be built using Hadoop
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-1395) Integrate Katta
Posted by "Jason Venner (at ning) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jason Venner (at ning) updated SOLR-1395:
-----------------------------------------
Attachment: solr-1395-1431-4.patch
solr-1395-1431-4.patch contains a number of repairs, and now facet count aggregation works.
The one down side, is that this patch REQUIRES that the shards paramter explicitly list the shards to be queried, using a wild card does not work.
I have this up and running nicely over 9 katta nodes and 65million documents.
> Integrate Katta
> ---------------
>
> Key: SOLR-1395
> URL: https://issues.apache.org/jira/browse/SOLR-1395
> Project: Solr
> Issue Type: New Feature
> Affects Versions: 1.4
> Reporter: Jason Rutherglen
> Priority: Minor
> Fix For: 1.5
>
> Attachments: hadoop-core-0.19.0.jar, katta-core-0.6-dev.jar, katta.node.properties, katta.zk.properties, log4j-1.2.13.jar, solr-1395-1431-3.patch, solr-1395-1431-4.patch, solr-1395-1431.patch, SOLR-1395.patch, SOLR-1395.patch, SOLR-1395.patch, test-katta-core-0.6-dev.jar, zkclient-0.1-dev.jar, zookeeper-3.2.1.jar
>
> Original Estimate: 336h
> Remaining Estimate: 336h
>
> We'll integrate Katta into Solr so that:
> * Distributed search uses Hadoop RPC
> * Shard/SolrCore distribution and management
> * Zookeeper based failover
> * Indexes may be built using Hadoop
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-1395) Integrate Katta
Posted by "pravin karne (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12766066#action_12766066 ]
pravin karne commented on SOLR-1395:
------------------------------------
hi
For solr patch i used following command:
patch -p 0 -i solr-1395-1431-4.patch // this is for solrt trunk and its working
but for katt trunk i have to use KATTA-SOLR.patch
"KATTA-SOLR.patch" is not on jira .Shall i use same above patch i.e. solr-1395-1431-4.patch
can plz tell me name of that patch file?
> Integrate Katta
> ---------------
>
> Key: SOLR-1395
> URL: https://issues.apache.org/jira/browse/SOLR-1395
> Project: Solr
> Issue Type: New Feature
> Affects Versions: 1.4
> Reporter: Jason Rutherglen
> Priority: Minor
> Fix For: 1.5
>
> Attachments: hadoop-core-0.19.0.jar, katta-core-0.6-dev.jar, katta.node.properties, katta.zk.properties, log4j-1.2.13.jar, solr-1395-1431-3.patch, solr-1395-1431-4.patch, solr-1395-1431.patch, SOLR-1395.patch, SOLR-1395.patch, SOLR-1395.patch, test-katta-core-0.6-dev.jar, zkclient-0.1-dev.jar, zookeeper-3.2.1.jar
>
> Original Estimate: 336h
> Remaining Estimate: 336h
>
> We'll integrate Katta into Solr so that:
> * Distributed search uses Hadoop RPC
> * Shard/SolrCore distribution and management
> * Zookeeper based failover
> * Indexes may be built using Hadoop
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-1395) Integrate Katta
Posted by "Noble Paul (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12749389#action_12749389 ]
Noble Paul commented on SOLR-1395:
----------------------------------
Why should this be a Solr issue? What is missing in Solr which prevents you from integrating katta into Solr as a some kind of plugin?
> Integrate Katta
> ---------------
>
> Key: SOLR-1395
> URL: https://issues.apache.org/jira/browse/SOLR-1395
> Project: Solr
> Issue Type: New Feature
> Affects Versions: 1.4
> Reporter: Jason Rutherglen
> Priority: Minor
> Fix For: 1.5
>
> Original Estimate: 336h
> Remaining Estimate: 336h
>
> We'll integrate Katta into Solr so that:
> * Distributed search uses Hadoop RPC
> * Shard/SolrCore distribution and management
> * Zookeeper based failover
> * Indexes may be built using Hadoop
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-1395) Integrate Katta
Posted by "jv ning (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12754662#action_12754662 ]
jv ning commented on SOLR-1395:
-------------------------------
Jason and I have a couple of small changes that make this simpler to use, and a first faq entry.
If you get a NullPointerException in mergeId's a likely cause is a schema missmatch on the unique id field between an index served by a shard, and the top level solr instance performing the search.
> Integrate Katta
> ---------------
>
> Key: SOLR-1395
> URL: https://issues.apache.org/jira/browse/SOLR-1395
> Project: Solr
> Issue Type: New Feature
> Affects Versions: 1.4
> Reporter: Jason Rutherglen
> Priority: Minor
> Fix For: 1.5
>
> Attachments: hadoop-core-0.19.0.jar, KATTA-SOLR.patch, log4j-1.2.13.jar, SOLR-1395.patch, zookeeper-3.2.1.jar
>
> Original Estimate: 336h
> Remaining Estimate: 336h
>
> We'll integrate Katta into Solr so that:
> * Distributed search uses Hadoop RPC
> * Shard/SolrCore distribution and management
> * Zookeeper based failover
> * Indexes may be built using Hadoop
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-1395) Integrate Katta
Posted by "Jason Venner (www.prohadoop.com) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12766054#action_12766054 ]
Jason Venner (www.prohadoop.com) commented on SOLR-1395:
--------------------------------------------------------
It should all be attached to the jira.
> Integrate Katta
> ---------------
>
> Key: SOLR-1395
> URL: https://issues.apache.org/jira/browse/SOLR-1395
> Project: Solr
> Issue Type: New Feature
> Affects Versions: 1.4
> Reporter: Jason Rutherglen
> Priority: Minor
> Fix For: 1.5
>
> Attachments: hadoop-core-0.19.0.jar, katta-core-0.6-dev.jar, katta.node.properties, katta.zk.properties, log4j-1.2.13.jar, solr-1395-1431-3.patch, solr-1395-1431-4.patch, solr-1395-1431.patch, SOLR-1395.patch, SOLR-1395.patch, SOLR-1395.patch, test-katta-core-0.6-dev.jar, zkclient-0.1-dev.jar, zookeeper-3.2.1.jar
>
> Original Estimate: 336h
> Remaining Estimate: 336h
>
> We'll integrate Katta into Solr so that:
> * Distributed search uses Hadoop RPC
> * Shard/SolrCore distribution and management
> * Zookeeper based failover
> * Indexes may be built using Hadoop
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-1395) Integrate Katta
Posted by "Sumit (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12849155#action_12849155 ]
Sumit commented on SOLR-1395:
-----------------------------
Can some one help me out in integrating katta with solr. I am facing lot of challenges in integrating using above steps.
> Integrate Katta
> ---------------
>
> Key: SOLR-1395
> URL: https://issues.apache.org/jira/browse/SOLR-1395
> Project: Solr
> Issue Type: New Feature
> Affects Versions: 1.4
> Reporter: Jason Rutherglen
> Priority: Minor
> Fix For: 1.5
>
> Attachments: hadoop-core-0.19.0.jar, katta-core-0.6-dev.jar, katta.node.properties, katta.zk.properties, log4j-1.2.13.jar, solr-1395-1431-3.patch, solr-1395-1431-4.patch, solr-1395-1431-katta0.6.patch, solr-1395-1431.patch, SOLR-1395.patch, SOLR-1395.patch, SOLR-1395.patch, test-katta-core-0.6-dev.jar, zkclient-0.1-dev.jar, zookeeper-3.2.1.jar
>
> Original Estimate: 336h
> Remaining Estimate: 336h
>
> We'll integrate Katta into Solr so that:
> * Distributed search uses Hadoop RPC
> * Shard/SolrCore distribution and management
> * Zookeeper based failover
> * Indexes may be built using Hadoop
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-1395) Integrate Katta
Posted by "Jason Venner (www.prohadoop.com) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12834292#action_12834292 ]
Jason Venner (www.prohadoop.com) commented on SOLR-1395:
--------------------------------------------------------
I run this by having a set of front end solr instances with jetty (1 or more), then you can speak http to these solr instances, which will allow you to query via php.
What I typically do, is just hack the solrconfig.xml in the solr/examples/solr/conf directory and drop in my schema.xml into the same directory.
Then run java ..... -jar start.jar from the examples directory.
Tweek this out for your production requirements ...
> Integrate Katta
> ---------------
>
> Key: SOLR-1395
> URL: https://issues.apache.org/jira/browse/SOLR-1395
> Project: Solr
> Issue Type: New Feature
> Affects Versions: 1.4
> Reporter: Jason Rutherglen
> Priority: Minor
> Fix For: 1.5
>
> Attachments: hadoop-core-0.19.0.jar, katta-core-0.6-dev.jar, katta.node.properties, katta.zk.properties, log4j-1.2.13.jar, solr-1395-1431-3.patch, solr-1395-1431-4.patch, solr-1395-1431.patch, SOLR-1395.patch, SOLR-1395.patch, SOLR-1395.patch, test-katta-core-0.6-dev.jar, zkclient-0.1-dev.jar, zookeeper-3.2.1.jar
>
> Original Estimate: 336h
> Remaining Estimate: 336h
>
> We'll integrate Katta into Solr so that:
> * Distributed search uses Hadoop RPC
> * Shard/SolrCore distribution and management
> * Zookeeper based failover
> * Indexes may be built using Hadoop
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-1395) Integrate Katta
Posted by "Noble Paul (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12754700#action_12754700 ]
Noble Paul commented on SOLR-1395:
----------------------------------
Jason , why don't you separate issue for the CommComponent.It is useful for Solr even w/o Katta
> Integrate Katta
> ---------------
>
> Key: SOLR-1395
> URL: https://issues.apache.org/jira/browse/SOLR-1395
> Project: Solr
> Issue Type: New Feature
> Affects Versions: 1.4
> Reporter: Jason Rutherglen
> Priority: Minor
> Fix For: 1.5
>
> Attachments: hadoop-core-0.19.0.jar, KATTA-SOLR.patch, log4j-1.2.13.jar, SOLR-1395.patch, zookeeper-3.2.1.jar
>
> Original Estimate: 336h
> Remaining Estimate: 336h
>
> We'll integrate Katta into Solr so that:
> * Distributed search uses Hadoop RPC
> * Shard/SolrCore distribution and management
> * Zookeeper based failover
> * Indexes may be built using Hadoop
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-1395) Integrate Katta
Posted by "Stefan Groschupf (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12751980#action_12751980 ]
Stefan Groschupf commented on SOLR-1395:
----------------------------------------
Jason please note that the latest katta code is actually in sourceforges git repo not in svn.
> Integrate Katta
> ---------------
>
> Key: SOLR-1395
> URL: https://issues.apache.org/jira/browse/SOLR-1395
> Project: Solr
> Issue Type: New Feature
> Affects Versions: 1.4
> Reporter: Jason Rutherglen
> Priority: Minor
> Fix For: 1.5
>
> Attachments: KATTA-SOLR.patch, SOLR-1395.patch
>
> Original Estimate: 336h
> Remaining Estimate: 336h
>
> We'll integrate Katta into Solr so that:
> * Distributed search uses Hadoop RPC
> * Shard/SolrCore distribution and management
> * Zookeeper based failover
> * Indexes may be built using Hadoop
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-1395) Integrate Katta
Posted by "Jason Venner (at ning) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12760968#action_12760968 ]
Jason Venner (at ning) commented on SOLR-1395:
----------------------------------------------
I was unable to separate them cleanly, so no.
> Integrate Katta
> ---------------
>
> Key: SOLR-1395
> URL: https://issues.apache.org/jira/browse/SOLR-1395
> Project: Solr
> Issue Type: New Feature
> Affects Versions: 1.4
> Reporter: Jason Rutherglen
> Priority: Minor
> Fix For: 1.5
>
> Attachments: hadoop-core-0.19.0.jar, katta-core-0.6-dev.jar, katta.node.properties, katta.zk.properties, log4j-1.2.13.jar, solr-1395-1431-3.patch, solr-1395-1431.patch, SOLR-1395.patch, SOLR-1395.patch, SOLR-1395.patch, test-katta-core-0.6-dev.jar, zkclient-0.1-dev.jar, zookeeper-3.2.1.jar
>
> Original Estimate: 336h
> Remaining Estimate: 336h
>
> We'll integrate Katta into Solr so that:
> * Distributed search uses Hadoop RPC
> * Shard/SolrCore distribution and management
> * Zookeeper based failover
> * Indexes may be built using Hadoop
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-1395) Integrate Katta
Posted by "pravin karne (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12769917#action_12769917 ]
pravin karne commented on SOLR-1395:
------------------------------------
Hi,
i have integrate above path successfully.But when i tried to run "ant test-core-Dtestcase=KattaClientTest" test ,its failed.
Is there any katta configuration required?
As this path uses katta internally How to deploy indexes on solr with above patch ?
can i run katta and solr run on different machine ? how to configure this?
can you please provide details configurations steps for katta/solr integrations
Thanks
> Integrate Katta
> ---------------
>
> Key: SOLR-1395
> URL: https://issues.apache.org/jira/browse/SOLR-1395
> Project: Solr
> Issue Type: New Feature
> Affects Versions: 1.4
> Reporter: Jason Rutherglen
> Priority: Minor
> Fix For: 1.5
>
> Attachments: hadoop-core-0.19.0.jar, katta-core-0.6-dev.jar, katta.node.properties, katta.zk.properties, log4j-1.2.13.jar, solr-1395-1431-3.patch, solr-1395-1431-4.patch, solr-1395-1431.patch, SOLR-1395.patch, SOLR-1395.patch, SOLR-1395.patch, test-katta-core-0.6-dev.jar, zkclient-0.1-dev.jar, zookeeper-3.2.1.jar
>
> Original Estimate: 336h
> Remaining Estimate: 336h
>
> We'll integrate Katta into Solr so that:
> * Distributed search uses Hadoop RPC
> * Shard/SolrCore distribution and management
> * Zookeeper based failover
> * Indexes may be built using Hadoop
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-1395) Integrate Katta
Posted by "shyjuThomas (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12832465#action_12832465 ]
shyjuThomas commented on SOLR-1395:
-----------------------------------
Now Katta 0.6 version has been released, and there are many changes present in the katta-core-0.6.0.jar compared to the katta-core-0.6-dev.jar present along with this. The patch provided for this issue will not work with this latest katta release version.
> Integrate Katta
> ---------------
>
> Key: SOLR-1395
> URL: https://issues.apache.org/jira/browse/SOLR-1395
> Project: Solr
> Issue Type: New Feature
> Affects Versions: 1.4
> Reporter: Jason Rutherglen
> Priority: Minor
> Fix For: 1.5
>
> Attachments: hadoop-core-0.19.0.jar, katta-core-0.6-dev.jar, katta.node.properties, katta.zk.properties, log4j-1.2.13.jar, solr-1395-1431-3.patch, solr-1395-1431-4.patch, solr-1395-1431.patch, SOLR-1395.patch, SOLR-1395.patch, SOLR-1395.patch, test-katta-core-0.6-dev.jar, zkclient-0.1-dev.jar, zookeeper-3.2.1.jar
>
> Original Estimate: 336h
> Remaining Estimate: 336h
>
> We'll integrate Katta into Solr so that:
> * Distributed search uses Hadoop RPC
> * Shard/SolrCore distribution and management
> * Zookeeper based failover
> * Indexes may be built using Hadoop
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-1395) Integrate Katta
Posted by "Thomas Koch (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Thomas Koch updated SOLR-1395:
------------------------------
Attachment: solr-1395-1431-katta0.6.patch
This patch implements searching over a set of indices specified by a regular expression (in the shards= parameter of the query). For this patch to work, you also need to patch katta: http://oss.101tec.com/jira/browse/KATTA-91
> Integrate Katta
> ---------------
>
> Key: SOLR-1395
> URL: https://issues.apache.org/jira/browse/SOLR-1395
> Project: Solr
> Issue Type: New Feature
> Affects Versions: 1.4
> Reporter: Jason Rutherglen
> Priority: Minor
> Fix For: 1.5
>
> Attachments: hadoop-core-0.19.0.jar, katta-core-0.6-dev.jar, katta.node.properties, katta.zk.properties, log4j-1.2.13.jar, solr-1395-1431-3.patch, solr-1395-1431-4.patch, solr-1395-1431-katta0.6.patch, solr-1395-1431-katta0.6.patch, solr-1395-1431.patch, SOLR-1395.patch, SOLR-1395.patch, SOLR-1395.patch, test-katta-core-0.6-dev.jar, zkclient-0.1-dev.jar, zookeeper-3.2.1.jar
>
> Original Estimate: 336h
> Remaining Estimate: 336h
>
> We'll integrate Katta into Solr so that:
> * Distributed search uses Hadoop RPC
> * Shard/SolrCore distribution and management
> * Zookeeper based failover
> * Indexes may be built using Hadoop
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-1395) Integrate Katta
Posted by "Jason Rutherglen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12771704#action_12771704 ]
Jason Rutherglen commented on SOLR-1395:
----------------------------------------
Pravin,
I'll review the test case when I can. Did you download and apply the latest patch?
> Integrate Katta
> ---------------
>
> Key: SOLR-1395
> URL: https://issues.apache.org/jira/browse/SOLR-1395
> Project: Solr
> Issue Type: New Feature
> Affects Versions: 1.4
> Reporter: Jason Rutherglen
> Priority: Minor
> Fix For: 1.5
>
> Attachments: hadoop-core-0.19.0.jar, katta-core-0.6-dev.jar, katta.node.properties, katta.zk.properties, log4j-1.2.13.jar, solr-1395-1431-3.patch, solr-1395-1431-4.patch, solr-1395-1431.patch, SOLR-1395.patch, SOLR-1395.patch, SOLR-1395.patch, test-katta-core-0.6-dev.jar, zkclient-0.1-dev.jar, zookeeper-3.2.1.jar
>
> Original Estimate: 336h
> Remaining Estimate: 336h
>
> We'll integrate Katta into Solr so that:
> * Distributed search uses Hadoop RPC
> * Shard/SolrCore distribution and management
> * Zookeeper based failover
> * Indexes may be built using Hadoop
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-1395) Integrate Katta
Posted by "Jason Rutherglen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jason Rutherglen updated SOLR-1395:
-----------------------------------
Attachment: zkclient-0.1-dev.jar
katta-core-0.6-dev.jar
test-katta-core-0.6-dev.jar
Copy these libraries into lib/ before executing the test. The Katta jars are somewhat custom. I'll post a patch there shortly.
> Integrate Katta
> ---------------
>
> Key: SOLR-1395
> URL: https://issues.apache.org/jira/browse/SOLR-1395
> Project: Solr
> Issue Type: New Feature
> Affects Versions: 1.4
> Reporter: Jason Rutherglen
> Priority: Minor
> Fix For: 1.5
>
> Attachments: hadoop-core-0.19.0.jar, katta-core-0.6-dev.jar, KATTA-SOLR.patch, log4j-1.2.13.jar, SOLR-1395.patch, SOLR-1395.patch, test-katta-core-0.6-dev.jar, zkclient-0.1-dev.jar, zookeeper-3.2.1.jar
>
> Original Estimate: 336h
> Remaining Estimate: 336h
>
> We'll integrate Katta into Solr so that:
> * Distributed search uses Hadoop RPC
> * Shard/SolrCore distribution and management
> * Zookeeper based failover
> * Indexes may be built using Hadoop
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-1395) Integrate Katta
Posted by "jv ning (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
jv ning updated SOLR-1395:
--------------------------
Attachment: solr-1395-1431.patch
the file /tmp/solr-1395-1431.patch is a combined patch of 1431 and 1395.
A small api change in the query parser required a small code change.
> Integrate Katta
> ---------------
>
> Key: SOLR-1395
> URL: https://issues.apache.org/jira/browse/SOLR-1395
> Project: Solr
> Issue Type: New Feature
> Affects Versions: 1.4
> Reporter: Jason Rutherglen
> Priority: Minor
> Fix For: 1.5
>
> Attachments: hadoop-core-0.19.0.jar, katta-core-0.6-dev.jar, log4j-1.2.13.jar, solr-1395-1431.patch, SOLR-1395.patch, SOLR-1395.patch, SOLR-1395.patch, test-katta-core-0.6-dev.jar, zkclient-0.1-dev.jar, zookeeper-3.2.1.jar
>
> Original Estimate: 336h
> Remaining Estimate: 336h
>
> We'll integrate Katta into Solr so that:
> * Distributed search uses Hadoop RPC
> * Shard/SolrCore distribution and management
> * Zookeeper based failover
> * Indexes may be built using Hadoop
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-1395) Integrate Katta
Posted by "Thomas Koch (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Thomas Koch updated SOLR-1395:
------------------------------
Attachment: solr-1395-1431-katta0.6.patch
I've updated the patch for katta 0.6 however I deleted the SolrIndexer class since I don't need it and it relies on the indexer contribution to katta which seems to be deprecated.
I still need to work on this patch, because I need the functionality to search all registered indexes. I'd appreciate any help!
> Integrate Katta
> ---------------
>
> Key: SOLR-1395
> URL: https://issues.apache.org/jira/browse/SOLR-1395
> Project: Solr
> Issue Type: New Feature
> Affects Versions: 1.4
> Reporter: Jason Rutherglen
> Priority: Minor
> Fix For: 1.5
>
> Attachments: hadoop-core-0.19.0.jar, katta-core-0.6-dev.jar, katta.node.properties, katta.zk.properties, log4j-1.2.13.jar, solr-1395-1431-3.patch, solr-1395-1431-4.patch, solr-1395-1431-katta0.6.patch, solr-1395-1431.patch, SOLR-1395.patch, SOLR-1395.patch, SOLR-1395.patch, test-katta-core-0.6-dev.jar, zkclient-0.1-dev.jar, zookeeper-3.2.1.jar
>
> Original Estimate: 336h
> Remaining Estimate: 336h
>
> We'll integrate Katta into Solr so that:
> * Distributed search uses Hadoop RPC
> * Shard/SolrCore distribution and management
> * Zookeeper based failover
> * Indexes may be built using Hadoop
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.