You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-dev@lucene.apache.org by "Linbin Chen (JIRA)" <ji...@apache.org> on 2009/06/03 11:24:07 UTC

[jira] Created: (SOLR-1197) memcached implement solr cache for queryresultCache

memcached implement solr cache for queryresultCache 
----------------------------------------------------

                 Key: SOLR-1197
                 URL: https://issues.apache.org/jira/browse/SOLR-1197
             Project: Solr
          Issue Type: New Feature
    Affects Versions: 1.3
         Environment: multiple slave solr replication environment
            Reporter: Linbin Chen
             Fix For: 1.4


multiple slave create query result together, and share that slaves.

implement  memcached cache instead LRUCache


my implement:

solr-memcache.zip http://solr-side.googlecode.com/files/solr-memcache.zip 

object transport by net need Serializable, so need patch solr 1.3, DocSetBase implements Serializable, see http://code.google.com/p/solr-side/issues/detail?id=1&can=1




-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Unsubscribe my mail id muralihb@hcl.in

Posted by "Murali Hari Babu Chinnampuri, TLS, Chennai" <mu...@hcl.in>.
Can u please unsubscribe my mail id

Regards,
Murali Hari Babu.C


-----Original Message-----
From: Noble Paul (JIRA) [mailto:jira@apache.org] 
Sent: Friday, June 05, 2009 12:08 PM
To: solr-dev@lucene.apache.org
Subject: [jira] Updated: (SOLR-1197) memcached implement solr cache for queryresultCache


     [ https://issues.apache.org/jira/browse/SOLR-1197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Noble Paul updated SOLR-1197:
-----------------------------

    Fix Version/s:     (was: 1.4)
                   1.5

moving to 1.5. it is not a good idea to accept new features for 1.4

> memcached implement solr cache for queryresultCache 
> ----------------------------------------------------
>
>                 Key: SOLR-1197
>                 URL: https://issues.apache.org/jira/browse/SOLR-1197
>             Project: Solr
>          Issue Type: New Feature
>    Affects Versions: 1.3
>         Environment: multiple slave solr replication environment
>            Reporter: Linbin Chen
>             Fix For: 1.5
>
>         Attachments: SOLR_1197-solr-memcache.patch
>
>
> multiple slave create query result together, and slaves can share that.
> implement  memcached cache instead LRUCache
> my implement:
> solr-memcache.zip http://solr-side.googlecode.com/files/solr-memcache.zip 
> object transport by net need Serializable, so need patch solr 1.3, DocSetBase implements Serializable, see http://code.google.com/p/solr-side/issues/detail?id=1&can=1

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


DISCLAIMER:
-----------------------------------------------------------------------------------------------------------------------

The contents of this e-mail and any attachment(s) are confidential and intended for the named recipient(s) only.
It shall not attach any liability on the originator or HCL or its affiliates. Any views or opinions presented in 
this email are solely those of the author and may not necessarily reflect the opinions of HCL or its affiliates.
Any form of reproduction, dissemination, copying, disclosure, modification, distribution and / or publication of 
this message without the prior written consent of the author of this e-mail is strictly prohibited. If you have
received this email in error please delete it and notify the sender immediately. Before opening any mail and 
attachments please check them for viruses and defect.

-----------------------------------------------------------------------------------------------------------------------

[jira] Updated: (SOLR-1197) memcached implement solr cache for queryresultCache

Posted by "Linbin Chen (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/SOLR-1197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Linbin Chen updated SOLR-1197:
------------------------------

    Attachment: SOLR_1197-solr-memcache.patch

Solr 1.3 

only DocSetBase  implement Serializable

> memcached implement solr cache for queryresultCache 
> ----------------------------------------------------
>
>                 Key: SOLR-1197
>                 URL: https://issues.apache.org/jira/browse/SOLR-1197
>             Project: Solr
>          Issue Type: New Feature
>    Affects Versions: 1.3
>         Environment: multiple slave solr replication environment
>            Reporter: Linbin Chen
>             Fix For: 1.4
>
>         Attachments: SOLR_1197-solr-memcache.patch
>
>
> multiple slave create query result together, and slaves can share that.
> implement  memcached cache instead LRUCache
> my implement:
> solr-memcache.zip http://solr-side.googlecode.com/files/solr-memcache.zip 
> object transport by net need Serializable, so need patch solr 1.3, DocSetBase implements Serializable, see http://code.google.com/p/solr-side/issues/detail?id=1&can=1

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Updated: (SOLR-1197) memcached implement solr cache for queryresultCache

Posted by "Linbin Chen (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/SOLR-1197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Linbin Chen updated SOLR-1197:
------------------------------

    Description: 
multiple slave create query result together, and slaves can share that.

implement  memcached cache instead LRUCache


my implement:

solr-memcache.zip http://solr-side.googlecode.com/files/solr-memcache.zip 

object transport by net need Serializable, so need patch solr 1.3, DocSetBase implements Serializable, see http://code.google.com/p/solr-side/issues/detail?id=1&can=1




  was:
multiple slave create query result together, and share that slaves.

implement  memcached cache instead LRUCache


my implement:

solr-memcache.zip http://solr-side.googlecode.com/files/solr-memcache.zip 

object transport by net need Serializable, so need patch solr 1.3, DocSetBase implements Serializable, see http://code.google.com/p/solr-side/issues/detail?id=1&can=1





> memcached implement solr cache for queryresultCache 
> ----------------------------------------------------
>
>                 Key: SOLR-1197
>                 URL: https://issues.apache.org/jira/browse/SOLR-1197
>             Project: Solr
>          Issue Type: New Feature
>    Affects Versions: 1.3
>         Environment: multiple slave solr replication environment
>            Reporter: Linbin Chen
>             Fix For: 1.4
>
>
> multiple slave create query result together, and slaves can share that.
> implement  memcached cache instead LRUCache
> my implement:
> solr-memcache.zip http://solr-side.googlecode.com/files/solr-memcache.zip 
> object transport by net need Serializable, so need patch solr 1.3, DocSetBase implements Serializable, see http://code.google.com/p/solr-side/issues/detail?id=1&can=1

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Updated: (SOLR-1197) memcached implement solr cache for queryresultCache

Posted by "Noble Paul (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/SOLR-1197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Noble Paul updated SOLR-1197:
-----------------------------

    Fix Version/s:     (was: 1.4)
                   1.5

moving to 1.5. it is not a good idea to accept new features for 1.4

> memcached implement solr cache for queryresultCache 
> ----------------------------------------------------
>
>                 Key: SOLR-1197
>                 URL: https://issues.apache.org/jira/browse/SOLR-1197
>             Project: Solr
>          Issue Type: New Feature
>    Affects Versions: 1.3
>         Environment: multiple slave solr replication environment
>            Reporter: Linbin Chen
>             Fix For: 1.5
>
>         Attachments: SOLR_1197-solr-memcache.patch
>
>
> multiple slave create query result together, and slaves can share that.
> implement  memcached cache instead LRUCache
> my implement:
> solr-memcache.zip http://solr-side.googlecode.com/files/solr-memcache.zip 
> object transport by net need Serializable, so need patch solr 1.3, DocSetBase implements Serializable, see http://code.google.com/p/solr-side/issues/detail?id=1&can=1

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Assigned: (SOLR-1197) memcached implement solr cache for queryresultCache

Posted by "Noble Paul (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/SOLR-1197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Noble Paul reassigned SOLR-1197:
--------------------------------

    Assignee: Noble Paul

> memcached implement solr cache for queryresultCache 
> ----------------------------------------------------
>
>                 Key: SOLR-1197
>                 URL: https://issues.apache.org/jira/browse/SOLR-1197
>             Project: Solr
>          Issue Type: New Feature
>    Affects Versions: 1.3
>         Environment: multiple slave solr replication environment
>            Reporter: Linbin Chen
>            Assignee: Noble Paul
>             Fix For: 1.5
>
>         Attachments: SOLR_1197-solr-memcache.patch
>
>
> multiple slave create query result together, and slaves can share that.
> implement  memcached cache instead LRUCache
> my implement:
> solr-memcache.zip http://solr-side.googlecode.com/files/solr-memcache.zip 
> object transport by net need Serializable, so need patch solr 1.3, DocSetBase implements Serializable, see http://code.google.com/p/solr-side/issues/detail?id=1&can=1

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Issue Comment Edited: (SOLR-1197) memcached implement solr cache for queryresultCache

Posted by "Linbin Chen (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/SOLR-1197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12715879#action_12715879 ] 

Linbin Chen edited comment on SOLR-1197 at 6/3/09 2:42 AM:
-----------------------------------------------------------

patch for Solr 1.3 

this patch only DocSetBase  implement Serializable

solr-memcache 

=====================readme.txt=========================
 

MemcachedCache instead of solr queryresultCache (default LRUCache)
 

config in solrconfig.xml to use solr-memcache

add newSearcher and firstSearcher Listener, such as:
 


<listener event="newSearcher" class="solr.MemcachedCache" />
<listener event="firstSearcher" class="solr.MemcachedCache" />
 

use listener only for get index version, to create memcached key

indexVersion is static long field of MemcachedCache.java.
 

//originalKey is QueryResultKey
memcached key = keyPrefix+indexVersion+"-"+originalKey.hashCode() 
 


<!--

MemcachedCache params:
 

memcachedHosts (required), "," split.
name (optional) no default.
expTime (optional) default 1800 s (= 30 minute)
defaultPort (optional) default 11211
keyPrefix (optional) default ""
 

-->

<queryResultCache

class="solr.MemcachedCache"
memcachedHosts="192.168.0.100,192.168.0.101:1234,192.168.0.103"
expTime="21600"
defaultPort="11511"
keyPrefix="shard-1-"/>
 

dep jar:

memcached-2.2.jar
spy-2.4.jar
 

solr-memcache.patch for solr 1.3
 

if download and unzip to d:/apache-solr-1.3.0
 

copy patch-build.xml and solr-memcache.patch to (d:/apache-solr-1.3.0)
 


D:\apache-solr-1.3.0>ant -f patch-build.xml -Dpatch.file=solr-memcache.patch
Buildfile: patch-build.xml

apply-patch:
    [patch] patching file src/java/org/apache/solr/search/DocSet.java

BUILD SUCCESSFUL
Total time: 0 seconds
 

if exist d:/apache-solr-1.3.0/contrib/solr-memcache, if no exist you can unzip solr-memcache.zip to that dir

and dist
 


D:\apache-solr-1.3.0>ant dist
...
 

look D:\apache-solr-1.3.0\dist\apache-solr-memcache-1.3.0.jar

      was (Author: chenlb):
    patch for Solr 1.3 

this patch only DocSetBase  implement Serializable
  
> memcached implement solr cache for queryresultCache 
> ----------------------------------------------------
>
>                 Key: SOLR-1197
>                 URL: https://issues.apache.org/jira/browse/SOLR-1197
>             Project: Solr
>          Issue Type: New Feature
>    Affects Versions: 1.3
>         Environment: multiple slave solr replication environment
>            Reporter: Linbin Chen
>             Fix For: 1.4
>
>         Attachments: SOLR_1197-solr-memcache.patch
>
>
> multiple slave create query result together, and slaves can share that.
> implement  memcached cache instead LRUCache
> my implement:
> solr-memcache.zip http://solr-side.googlecode.com/files/solr-memcache.zip 
> object transport by net need Serializable, so need patch solr 1.3, DocSetBase implements Serializable, see http://code.google.com/p/solr-side/issues/detail?id=1&can=1

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Updated: (SOLR-1197) memcached implement solr cache for queryresultCache

Posted by "Linbin Chen (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/SOLR-1197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Linbin Chen updated SOLR-1197:
------------------------------

    Attachment: SOLR_1197-solr-memcache-0.2.patch

solr sint hashcode conflict cause solr memcached bug

solr-memcached-0.2

solr sint fieldType sint.hashcode conflict very hight above 99%

such as id:44232808 and id:44090533 hashcode conflict, so q=id:44232808 
maybe response id:44090533 doc result. error!

http://code.google.com/p/solr-side/issues/detail?id=2&can=1
solr-memcached-0.2 patch for create memcache key strategy:

QueryResultKey add method

  public String toString() {
	  return query.toString();
  }

for MemcachedCache md5(QueryResultKey.toString())

now memcache key strategy = md5(keyPrefix+indexVersion+QueryResultKey.hashCode()+md5(QueryResultKey.toString()))

double md5

solr-memcache-0.2.zip http://solr-side.googlecode.com/files/solr-memcache-0.2.zip


> memcached implement solr cache for queryresultCache 
> ----------------------------------------------------
>
>                 Key: SOLR-1197
>                 URL: https://issues.apache.org/jira/browse/SOLR-1197
>             Project: Solr
>          Issue Type: New Feature
>    Affects Versions: 1.3
>         Environment: multiple slave solr replication environment
>            Reporter: Linbin Chen
>            Assignee: Noble Paul
>             Fix For: 1.5
>
>         Attachments: SOLR_1197-solr-memcache-0.2.patch, SOLR_1197-solr-memcache.patch
>
>
> multiple slave create query result together, and slaves can share that.
> implement  memcached cache instead LRUCache
> my implement:
> solr-memcache.zip http://solr-side.googlecode.com/files/solr-memcache.zip 
> object transport by net need Serializable, so need patch solr 1.3, DocSetBase implements Serializable, see http://code.google.com/p/solr-side/issues/detail?id=1&can=1

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Issue Comment Edited: (SOLR-1197) memcached implement solr cache for queryresultCache

Posted by "Linbin Chen (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/SOLR-1197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12715879#action_12715879 ] 

Linbin Chen edited comment on SOLR-1197 at 6/3/09 2:41 AM:
-----------------------------------------------------------

patch for Solr 1.3 

this patch only DocSetBase  implement Serializable

      was (Author: chenlb):
    Solr 1.3 

only DocSetBase  implement Serializable
  
> memcached implement solr cache for queryresultCache 
> ----------------------------------------------------
>
>                 Key: SOLR-1197
>                 URL: https://issues.apache.org/jira/browse/SOLR-1197
>             Project: Solr
>          Issue Type: New Feature
>    Affects Versions: 1.3
>         Environment: multiple slave solr replication environment
>            Reporter: Linbin Chen
>             Fix For: 1.4
>
>         Attachments: SOLR_1197-solr-memcache.patch
>
>
> multiple slave create query result together, and slaves can share that.
> implement  memcached cache instead LRUCache
> my implement:
> solr-memcache.zip http://solr-side.googlecode.com/files/solr-memcache.zip 
> object transport by net need Serializable, so need patch solr 1.3, DocSetBase implements Serializable, see http://code.google.com/p/solr-side/issues/detail?id=1&can=1

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.