how to retrieve only the updated data from database using solr?

[nagarjuna <na...@gmail.com>]

Hi everybody...........
           
          i would like to know how to get only the updated data from the
database using solr instead of getting total results ........can anybody
explain me the procedure 


Thanx in advance.......


--
View this message in context: http://lucene.472066.n3.nabble.com/how-to-retrieve-only-the-updated-data-from-database-using-solr-tp3411622p3411622.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: how to retrieve only the updated data from database using solr?

[Shawn Heisey <so...@elyograg.org>]

On 10/11/2011 12:23 AM, nagarjuna wrote:
> Hi pravesh Thank u for ur reply..............
>
>              i am using DIH,but right now i dont have timestamp in my
> database....instead of that i have datecreated column which wont updated for
> the changes .....when ever i created some thing it just store the created
> time.....is there any chance to do with this column instead of the timestamp
> ....

By passing variables in via the DIH url and tracking it yourself, you 
can use arbitrary information outside of Solr for keeping track of 
what's new for delta-imports.

Here's the URL template that I use, and the DIH config that goes with 
it.  I keep track of the minDid and maxDid values outside of Solr and 
pass them in on each request with values appropriate for the shard 
that's being updated.  Note that < and > are encode as &lt; and &gt; as 
required by XML.

http://HOST:PORT/solr/CORE/dataimport?command=COMMAND&dbHost=DBSERVER&dbSchema=DBSCHEMA&dataView=DATAVIEW&numShards=NUMSHARDS&modVal=MODVAL&minDid=MINDID&maxDid=MAXDID&extraWhere=EXTRAWHERE

       query="
         SELECT * FROM ${dataimporter.request.dataView}
         WHERE (
           (
             did &gt; ${dataimporter.request.minDid}
             AND did &lt;= ${dataimporter.request.maxDid}
           )
           ${dataimporter.request.extraWhere}
         ) AND (crc32(did) % ${dataimporter.request.numShards})
           IN (${dataimporter.request.modVal})
         "
       deltaImportQuery="
         SELECT * FROM ${dataimporter.request.dataView}
         WHERE (
           (
             did &gt; ${dataimporter.request.minDid}
             AND did &lt;= ${dataimporter.request.maxDid}
           )
           ${dataimporter.request.extraWhere}
         ) AND (crc32(did) % ${dataimporter.request.numShards})
           IN (${dataimporter.request.modVal})
         "
       deltaQuery="SELECT 1 AS did"

Re: how to retrieve only the updated data from database using solr?

[nagarjuna <na...@gmail.com>]

Hi pravesh Thank u for ur reply..............

            i am using DIH,but right now i dont have timestamp in my
database....instead of that i have datecreated column which wont updated for
the changes .....when ever i created some thing it just store the created
time.....is there any chance to do with this column instead of the timestamp
....


Thanx in advance 

--
View this message in context: http://lucene.472066.n3.nabble.com/how-to-retrieve-only-the-updated-data-from-database-using-solr-tp3411622p3411755.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: how to retrieve only the updated data from database using solr?

[pravesh <su...@yahoo.com>]

If you mean using DIH? Then you need to have a timestamp column in your DB,
which has to be updated to current timestamp, whenever you are modifying the
record in DB.

For rest just go through the  http://wiki.apache.org/solr/DataImportHandler
DIH wiki here 

Thanx
Pravesh

--
View this message in context: http://lucene.472066.n3.nabble.com/how-to-retrieve-only-the-updated-data-from-database-using-solr-tp3411622p3411748.html
Sent from the Solr - User mailing list archive at Nabble.com.