You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by "O. Olson" <ol...@yahoo.it> on 2013/12/11 00:06:34 UTC
Replacing Deprecated CachedSqlEntityProcessor with SqlEntityProcessor with a cacheImpl parameter
Hi,
I am
looking to replace the Deprecated CachedSqlEntityProcessor with SqlEntityProcessor
with a cacheImpl parameter but I cannot find documentation.
The Deprecated note at the top of http://lucene.apache.org/solr/3_6_0/org/apache/solr/handler/dataimport/CachedSqlEntityProcessor.html says that we need to replace the CachedSqlEntityProcessor with
SqlEntityProcessor with a cacheImpl parameter. The wiki here, does not mention
the cacheImpl parameter, or it's possible values: https://cwiki.apache.org/confluence/display/solr/Uploading+Structured+Data+Store+Data+with+the+Data+Import+Handler#UploadingStructuredDataStoreDatawiththeDataImportHandler-EntityProcessors
An abbreviated version of my db-data-config.xml looks like:
<entity name="Doc"
query="SELECT DocID, Title FROM solr.DOCS_TABLE">
<field column="DocID" name="DocID" />
<field column="Title" name="Title" />
<entity name="Cat1"
query="SELECT CategoryName, DocID FROM solr.CAT_DOCS_MAP
WHERE CategoryLevel=1"
cacheKey="DocID" cacheLookup="Doc.DocID" processor="CachedSqlEntityProcessor">
<field column="CategoryName" name="Category1" />
</entity>
</entity>
I am curious how I would use SqlEntityProcessor and turn on
caching (because I really need it). Or is that even possible? Can I do something like:
<entity name="Doc"
query="SELECT DocID, Title FROM solr.DOCS_TABLE">
<field column="DocID" name="DocID" />
<field column="Title" name="Title" />
<entity name="Cat1"
query="SELECT CategoryName, DocID FROM solr.CAT_DOCS_MAP
WHERE CategoryLevel=1"
cacheKey="DocID" cacheLookup="Doc.DocID" processor="SqlEntityProcessor" cacheImpl=???>
<field column="CategoryName" name="Category1" />
</entity>
</entity>
What do I put in for cacheImpl? What are the possible values
for cacheImpl.
Thank you in advance for your help.
O. O.
Re: Replacing Deprecated CachedSqlEntityProcessor with
SqlEntityProcessor with a cacheImpl parameter
Posted by Furkan KAMACI <fu...@gmail.com>.
Hi;
Good to hear that I could help you.
Thanks;
Furkan KAMACI
11 Aralık 2013 Çarşamba tarihinde O. Olson <ol...@yahoo.it> adlı
kullanıcı şöyle yazdı:
> Thank you very much Furkan for the detailed explanation. I simply set the
> processor to SqlEntityProcessor and cacheImpl to SortedMapBackedCache, and
> it worked for me i.e. from my original post I had/did:
>
> <entity name="Doc"
> query="SELECT DocID, Title FROM solr.DOCS_TABLE">
> <field column="DocID" name="DocID" />
> <field column="Title" name="Title" />
> <entity name="Cat1"
> query="SELECT CategoryName, DocID FROM
solr.CAT_DOCS_MAP
> WHERE CategoryLevel=1"
> cacheKey="DocID" cacheLookup="Doc.DocID"
> processor="SqlEntityProcessor" cacheImpl="SortedMapBackedCache">
> <field column="CategoryName" name="Category1" />
> </entity>
> </entity>
>
> Thanks again,
> O. O.
>
>
>
>
> --
> View this message in context:
http://lucene.472066.n3.nabble.com/Replacing-Deprecated-CachedSqlEntityProcessor-with-SqlEntityProcessor-with-a-cacheImpl-parameter-tp4106018p4106227.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>
Re: Replacing Deprecated CachedSqlEntityProcessor with
SqlEntityProcessor with a cacheImpl parameter
Posted by "O. Olson" <ol...@yahoo.it>.
Thank you very much Furkan for the detailed explanation. I simply set the
processor to SqlEntityProcessor and cacheImpl to SortedMapBackedCache, and
it worked for me i.e. from my original post I had/did:
<entity name="Doc"
query="SELECT DocID, Title FROM solr.DOCS_TABLE">
<field column="DocID" name="DocID" />
<field column="Title" name="Title" />
<entity name="Cat1"
query="SELECT CategoryName, DocID FROM solr.CAT_DOCS_MAP
WHERE CategoryLevel=1"
cacheKey="DocID" cacheLookup="Doc.DocID"
processor="SqlEntityProcessor" cacheImpl="SortedMapBackedCache">
<field column="CategoryName" name="Category1" />
</entity>
</entity>
Thanks again,
O. O.
--
View this message in context: http://lucene.472066.n3.nabble.com/Replacing-Deprecated-CachedSqlEntityProcessor-with-SqlEntityProcessor-with-a-cacheImpl-parameter-tp4106018p4106227.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Replacing Deprecated CachedSqlEntityProcessor with
SqlEntityProcessor with a cacheImpl parameter
Posted by Furkan KAMACI <fu...@gmail.com>.
Hi;
I suggest you to read this Jira issue:
https://issues.apache.org/jira/browse/SOLR-2382 it makes clear why it is
deprecated. Here is one thing about the issue:
*Provide a means to temporarily cache a child Entity's data without needing
to create a special cached implementation of the Entity Processor (such as
CachedSqlEntityProcessor)*
Also here is an explanation:
http://comments.gmane.org/gmane.comp.jakarta.lucene.solr.user/67782
*"There were 2 major changes to DIH Cache functionality in Solr 3.6, only 1
of which was carried to Solr 4.0:*
*- Solr 3.6 had 2 MAJOR changes:*
*1. We support pluggable caches so that you can write your own cache
implemetations and cache however you*
*want. The goal here is to allow you to cache to disk when you had to do
large, complex joins and an in-memory*
*cache could result in an OOM. Also, you can specify "cacheImpl" with any
EntityProcessor, not just*
*SqlEntityProcessor. So you can join child entities that come from XML,
flat files, etc. *
*CachedSqlEntityProcessor is technically deprecated as using it is the same
as SqlEntityProcessor with*
*cacheImpl="SortedMapBackedCache" specified. This does a simple in-memory
cache very similar to*
*Solr3.5 and prior. (see https://issues.apache.org/jira/browse/SOLR-2382
<https://issues.apache.org/jira/browse/SOLR-2382>)"*
Given example uses that: SortedMapBackedCache as cacheImpl. And if you
debug the code you can see that it implements DIHCache interface and there
is not any other implementations within Solr 3.6.0 code. However as you can
see from the source code of Solr there is a pluggable cache mechanism
inside it. On the other hand SOLR-2382 starts with that description:
*Provide a pluggable caching framework for DIH so that users can choose a
cache implementation that best suits their data and application.*
Thanks;
Furkan KAMACI
2013/12/11 O. Olson <ol...@yahoo.it>
> Hi,
>
> I am
> looking to replace the Deprecated CachedSqlEntityProcessor with
> SqlEntityProcessor
> with a cacheImpl parameter but I cannot find documentation.
>
> The Deprecated note at the top of
> http://lucene.apache.org/solr/3_6_0/org/apache/solr/handler/dataimport/CachedSqlEntityProcessor.htmlsays that we need to replace the CachedSqlEntityProcessor with
> SqlEntityProcessor with a cacheImpl parameter. The wiki here, does not
> mention
> the cacheImpl parameter, or it's possible values:
> https://cwiki.apache.org/confluence/display/solr/Uploading+Structured+Data+Store+Data+with+the+Data+Import+Handler#UploadingStructuredDataStoreDatawiththeDataImportHandler-EntityProcessors
>
> An abbreviated version of my db-data-config.xml looks like:
>
> <entity name="Doc"
> query="SELECT DocID, Title FROM solr.DOCS_TABLE">
> <field column="DocID" name="DocID" />
> <field column="Title" name="Title" />
> <entity name="Cat1"
> query="SELECT CategoryName, DocID FROM
> solr.CAT_DOCS_MAP
> WHERE CategoryLevel=1"
> cacheKey="DocID" cacheLookup="Doc.DocID"
> processor="CachedSqlEntityProcessor">
> <field column="CategoryName" name="Category1" />
> </entity>
> </entity>
>
>
> I am curious how I would use SqlEntityProcessor and turn on
> caching (because I really need it). Or is that even possible? Can I do
> something like:
>
>
> <entity name="Doc"
> query="SELECT DocID, Title FROM solr.DOCS_TABLE">
> <field column="DocID" name="DocID" />
> <field column="Title" name="Title" />
> <entity name="Cat1"
> query="SELECT CategoryName, DocID FROM
> solr.CAT_DOCS_MAP
> WHERE CategoryLevel=1"
> cacheKey="DocID" cacheLookup="Doc.DocID"
> processor="SqlEntityProcessor" cacheImpl=???>
> <field column="CategoryName" name="Category1" />
> </entity>
> </entity>
>
> What do I put in for cacheImpl? What are the possible values
> for cacheImpl.
>
> Thank you in advance for your help.
> O. O.
>