You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Romi <ro...@gmail.com> on 2011/06/24 07:43:02 UTC
how to index data in solr form database automatically
I have MySql database for my application. i implemented solr search and used
dataimporthandler(DIH)to index data from database into solr. my question is:
is there any way that if database gets updated then my solr indexes
automatically gets update for new data added in the database. . It means i
need not to run index process manually every time data base tables
changes.If yes then please tell me how can i achieve this.
-----
Thanks & Regards
Romi
--
View this message in context: http://lucene.472066.n3.nabble.com/how-to-index-data-in-solr-form-database-automatically-tp3102893p3102893.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: how to index data in solr form database automatically
Posted by Shawn Heisey <ap...@elyograg.org>.
On 2/4/2015 12:01 AM, nvmai wrote:
> I use crontab
> * */1 * * * wget http://blog.nhipvang.com
> :8983/solr/imuzik/dataimport?command=full-import&clean=false
That will attempt to start a full-import once a minute. Requests sent
while an import is already underway will fail, but this basically means
that your solr instance will *always* be doing a full import.
That's probably not what you want.
A time specification of "0 * * * *" would do the import once an hour.
Thanks,
Shawn
Re: how to index data in solr form database automatically
Posted by nvmai <di...@yahoo.com>.
I use crontab
* */1 * * * wget http://blog.nhipvang.com
:8983/solr/imuzik/dataimport?command=full-import&clean=false
-----
mẹo làm đẹp
--
View this message in context: http://lucene.472066.n3.nabble.com/how-to-index-data-in-solr-form-database-automatically-tp3102893p4183792.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: how to index data in solr form database automatically
Posted by Renato Eschini <r....@inera.it>.
Why don't you use DataImportHandler?
We use DIH, we have a wget based bash script that is runned by cron
every about 2 minutes. DIH is called in delta-query mode.
The bash script waork in this way:
1) first call a wget on DIH status
2) analyze wget DIH status "/dataimport?command=status"
2.1) if status is **busy** do nothing and exit (beacuse DIH is already
running)
2.2) if status is **idle** do "/dataimport?command=delta-import&clean=false"
3) exit
On 24/06/11 15:20, Mohammad Shariq wrote:
> First write a Script in Python ( or JAVA or PHP or anyLanguage) which reads
> the data from database and index into Solr.
>
> Now setup this script as cron-job to run automatically at certain interval.
--
Renato Eschini
Inera srl
Via Mazzini 138
56100 Pisa (PI)
Tel: (+39) (0)50 9911800
Fax: (+39) (0)50 9911830
Int: (+39) (0)50 9911819
Email: r.eschini@inera.it
Msn: r_eschini@hotmail.com
Skype: renato.eschini
WWW: http://www.inera.it
Rispetta l'ambiente - è veramente necessario stampare questa e-mail?
Please consider the environment - do you really need to print this e-mail?
Re: how to index data in solr form database automatically
Posted by Renato Eschini <r....@inera.it>.
On 27/06/11 11:01, Romi wrote:
> Can i not use it for full-import ??
yes!!!
You must only change
urlCmd='http://localhost:8080/solr/mycore/dataimport?command=delta-import&clean=false'
in
urlCmd='http://localhost:8080/solr/mycore/dataimport?command=full-import&clean=false'
or
urlCmd='http://localhost:8080/solr/mycore/dataimport?command=full-import&clean=true'
or as you desire!!
Consult DIH handler wiki for the complete list of support params......
--
Renato Eschini
Inera srl
Via Mazzini 138
56100 Pisa (PI)
Tel: (+39) (0)50 9911800
Fax: (+39) (0)50 9911830
Int: (+39) (0)50 9911819
Email: r.eschini@inera.it
Msn: r_eschini@hotmail.com
Skype: renato.eschini
WWW: http://www.inera.it
Rispetta l'ambiente - è veramente necessario stampare questa e-mail?
Please consider the environment - do you really need to print this e-mail?
Re: how to index data in solr form database automatically
Posted by Romi <ro...@gmail.com>.
Can i not use it for full-import ??
-----
Thanks & Regards
Romi
--
View this message in context: http://lucene.472066.n3.nabble.com/how-to-index-data-in-solr-form-database-automatically-tp3102893p3113002.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: how to index data in solr form database automatically
Posted by Renato Eschini <r....@inera.it>.
Start from this point:
http://wiki.apache.org/solr/DataImportHandler
So, you can try to use this script:
#!/bin/bash
wgetProgram=wget
urlCmd='http://localhost:8080/solr/mycore/dataimport?command=delta-import&clean=false'
statusCmd='http://localhost:8080/solr/mycore/dataimport?command=status'
outputDir=.
$wgetProgram -O $outputDir/check_status_update_index.txt ${statusCmd}
2>/dev/null
status=`fgrep idle $outputDir/check_status_update_index.txt`
if [[ ${status} == *idle* ]]
then
$wgetProgram -O $outputDir/status_update_index.txt ${urlCmd}
2>/dev/null
fi
save it somewhere (ex. /root/script_solr) like "update-solr.sh" and
chmod +x.....
Now you can config it from root user in crontab:
linux:~# crontab -e
and insert for run every 2 minutes:
*/2 * * * * cd /root/script_solr && /root/script_solr/update-solr.sh
On 27/06/11 09:01, Romi wrote:
> Thanks for answering but i don't know how to write script for fetching data
> from database and index it in to solr and how to setup this script as
> cron-job to run automatically at certain interval.
>
>
> Please suggest
--
Renato Eschini
Inera srl
Via Mazzini 138
56100 Pisa (PI)
Tel: (+39) (0)50 9911800
Fax: (+39) (0)50 9911830
Int: (+39) (0)50 9911819
Email: r.eschini@inera.it
Msn: r_eschini@hotmail.com
Skype: renato.eschini
WWW: http://www.inera.it
Rispetta l'ambiente - è veramente necessario stampare questa e-mail?
Please consider the environment - do you really need to print this e-mail?
Re: how to index data in solr form database automatically
Posted by Romi <ro...@gmail.com>.
Thanks for answering but i don't know how to write script for fetching data
from database and index it in to solr and how to setup this script as
cron-job to run automatically at certain interval.
Please suggest .
-----
Thanks & Regards
Romi
--
View this message in context: http://lucene.472066.n3.nabble.com/how-to-index-data-in-solr-form-database-automatically-tp3102893p3112772.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: how to index data in solr form database automatically
Posted by Mohammad Shariq <sh...@gmail.com>.
First write a Script in Python ( or JAVA or PHP or anyLanguage) which reads
the data from database and index into Solr.
Now setup this script as cron-job to run automatically at certain interval.
On 24 June 2011 17:23, Romi <ro...@gmail.com> wrote:
> would you please tell me how can i use Cron for auto index my database
> tables
> in solr
>
> -----
> Thanks & Regards
> Romi
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/how-to-index-data-in-solr-form-database-automatically-tp3102893p3103768.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>
--
Thanks and Regards
Mohammad Shariq
Re: how to index data in solr form database automatically
Posted by Romi <ro...@gmail.com>.
would you please tell me how can i use Cron for auto index my database tables
in solr
-----
Thanks & Regards
Romi
--
View this message in context: http://lucene.472066.n3.nabble.com/how-to-index-data-in-solr-form-database-automatically-tp3102893p3103768.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: how to index data in solr form database automatically
Posted by Pranav Prakash <pr...@gmail.com>.
Cron is a time-based job scheduler in Unix-like computer operating systems.
en.wikipedia.org/wiki/Cron
*Pranav Prakash*
"temet nosce"
Twitter <http://twitter.com/pranavprakash> | Blog <http://blog.myblive.com> |
Google <http://www.google.com/profiles/pranny>
On Fri, Jun 24, 2011 at 12:26, Romi <ro...@gmail.com> wrote:
> Yeah i am using data-import to get data from database and indexing it. but
> what is cron can you please provide a link for it
>
> -----
> Thanks & Regards
> Romi
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/how-to-index-data-in-solr-form-database-automatically-tp3102893p3103072.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>
Re: how to index data in solr form database automatically
Posted by Romi <ro...@gmail.com>.
Yeah i am using data-import to get data from database and indexing it. but
what is cron can you please provide a link for it
-----
Thanks & Regards
Romi
--
View this message in context: http://lucene.472066.n3.nabble.com/how-to-index-data-in-solr-form-database-automatically-tp3102893p3103072.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: how to index data in solr form database automatically
Posted by Anshum <an...@gmail.com>.
How about having a delta-import and a cron to trigger the post?
--
Anshum Gupta
http://ai-cafe.blogspot.com
On Fri, Jun 24, 2011 at 11:13 AM, Romi <ro...@gmail.com> wrote:
> I have MySql database for my application. i implemented solr search and
> used
> dataimporthandler(DIH)to index data from database into solr. my question
> is:
> is there any way that if database gets updated then my solr indexes
> automatically gets update for new data added in the database. . It means i
> need not to run index process manually every time data base tables
> changes.If yes then please tell me how can i achieve this.
>
> -----
> Thanks & Regards
> Romi
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/how-to-index-data-in-solr-form-database-automatically-tp3102893p3102893.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>