You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by "kiran.bodigam" <ki...@gmail.com> on 2011/09/08 15:27:19 UTC
can indexing information stored in db rather than filesystem?
SOLR will keep the indexed information in directory called
index(${DATA_DIR}/index) after clicking the commit from admin console and
also we can override the existing index files.similarly can we keep (or)
store these index related information in permanent location like db table
rather than a file system?
--
View this message in context: http://lucene.472066.n3.nabble.com/can-indexing-information-stored-in-db-rather-than-filesystem-tp3319687p3319687.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: can indexing information stored in db rather than filesystem?
Posted by Walter Underwood <wu...@wunderwood.org>.
Lucene isn't even close to a DBMS. The batch commits make it less of of database than Berkeley DB.
Lucene is a search engine.
wunder
On Sep 8, 2011, at 8:54 AM, Jaeger, Jay - DOT wrote:
> If you think about it, Lucene (upon which Solr is build) *is* a kind of DBMS - just not an RDBMS. After all, in the end, a DBMS stores its stuff in files, too. If you then turned around and mapped the stuff that Solr does into database tables, you would lose all of the performance advantages that the Lucene/Solr design brings to the table. There would be no point.
>
> I think this is just a matter of having been "brainwashed" into thinking that a database is the only "safe" place to put stuff.
>
> JRJ
>
> -----Original Message-----
> From: kiran.bodigam [mailto:kiran.bodigam@gmail.com]
> Sent: Thursday, September 08, 2011 8:27 AM
> To: solr-user@lucene.apache.org
> Subject: can indexing information stored in db rather than filesystem?
>
> SOLR will keep the indexed information in directory called
> index(${DATA_DIR}/index) after clicking the commit from admin console and
> also we can override the existing index files.similarly can we keep (or)
> store these index related information in permanent location like db table
> rather than a file system?
>
> --
> View this message in context: http://lucene.472066.n3.nabble.com/can-indexing-information-stored-in-db-rather-than-filesystem-tp3319687p3319687.html
> Sent from the Solr - User mailing list archive at Nabble.com.
RE: can indexing information stored in db rather than filesystem?
Posted by "Jaeger, Jay - DOT" <Ja...@dot.wi.gov>.
If you think about it, Lucene (upon which Solr is build) *is* a kind of DBMS - just not an RDBMS. After all, in the end, a DBMS stores its stuff in files, too. If you then turned around and mapped the stuff that Solr does into database tables, you would lose all of the performance advantages that the Lucene/Solr design brings to the table. There would be no point.
I think this is just a matter of having been "brainwashed" into thinking that a database is the only "safe" place to put stuff.
JRJ
-----Original Message-----
From: kiran.bodigam [mailto:kiran.bodigam@gmail.com]
Sent: Thursday, September 08, 2011 8:27 AM
To: solr-user@lucene.apache.org
Subject: can indexing information stored in db rather than filesystem?
SOLR will keep the indexed information in directory called
index(${DATA_DIR}/index) after clicking the commit from admin console and
also we can override the existing index files.similarly can we keep (or)
store these index related information in permanent location like db table
rather than a file system?
--
View this message in context: http://lucene.472066.n3.nabble.com/can-indexing-information-stored-in-db-rather-than-filesystem-tp3319687p3319687.html
Sent from the Solr - User mailing list archive at Nabble.com.
RE: can indexing information stored in db rather than filesystem?
Posted by "Jaeger, Jay - DOT" <Ja...@dot.wi.gov>.
I don't think you understand. Solr does not have the code to do that. It just isn't there, nor would I expect it would ever be there.
Solr is open source though. You could look at the code and figure out how to do it (though why anyone would do that remains beyond my ability to understand). As the saying goes: "Knock yourself out".
(Happy programmer's day to all. http://en.wikipedia.org/wiki/Programmers'_Day ).
JRJ
-----Original Message-----
From: kiran.bodigam [mailto:kiran.bodigam@gmail.com]
Sent: Tuesday, September 13, 2011 8:51 AM
To: solr-user@lucene.apache.org
Subject: Re: can indexing information stored in db rather than filesystem?
Thanks for u r reply guys....
As suggested i agree that we are losing many of the benefits of Solr/Lucene
but i still want to store the index output (index files) in db table please
suggest what are the steps i need to follow to configure the db with SOLR
Engine (As how we done in it solrconfig.xml
<dataDir>${solr.data.dir:}</dataDir> similarly i would like to give the path
for db table)..?
--
View this message in context: http://lucene.472066.n3.nabble.com/can-indexing-information-stored-in-db-rather-than-filesystem-tp3319687p3332663.html
Sent from the Solr - User mailing list archive at Nabble.com.
RE: can indexing information stored in db rather than filesystem?
Posted by "Jaeger, Jay - DOT" <Ja...@dot.wi.gov>.
Nicely put. ;^)
-----Original Message-----
From: Walter Underwood [mailto:wunder@wunderwood.org]
Sent: Tuesday, September 13, 2011 9:16 AM
To: solr-user@lucene.apache.org
Subject: Re: can indexing information stored in db rather than filesystem?
On Sep 13, 2011, at 6:51 AM, kiran.bodigam wrote:
> As suggested i agree that we are losing many of the benefits of Solr/Lucene
> but i still want to store the index output (index files) in db table please
> suggest what are the steps i need to follow to configure the db with SOLR
> Engine
The steps are:
1. write the Java code to do that
2. submit it as contrib, because it is such a bad idea that I doubt it will be added to the common code
wunder
--
Walter Underwood
Re: can indexing information stored in db rather than filesystem?
Posted by Walter Underwood <wu...@wunderwood.org>.
On Sep 13, 2011, at 6:51 AM, kiran.bodigam wrote:
> As suggested i agree that we are losing many of the benefits of Solr/Lucene
> but i still want to store the index output (index files) in db table please
> suggest what are the steps i need to follow to configure the db with SOLR
> Engine
The steps are:
1. write the Java code to do that
2. submit it as contrib, because it is such a bad idea that I doubt it will be added to the common code
wunder
--
Walter Underwood
Re: can indexing information stored in db rather than filesystem?
Posted by Markus Jelsma <ma...@openindex.io>.
I'm curious; what benefits do you think you'll get by storing the files in
some DB table?
On Tuesday 13 September 2011 15:51:19 kiran.bodigam wrote:
> Thanks for u r reply guys....
>
> As suggested i agree that we are losing many of the benefits of Solr/Lucene
> but i still want to store the index output (index files) in db table please
> suggest what are the steps i need to follow to configure the db with SOLR
> Engine (As how we done in it solrconfig.xml
> <dataDir>${solr.data.dir:}</dataDir> similarly i would like to give the
> path for db table)..?
>
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/can-indexing-information-stored-in-db-r
> ather-than-filesystem-tp3319687p3332663.html Sent from the Solr - User
> mailing list archive at Nabble.com.
--
Markus Jelsma - CTO - Openindex
http://www.linkedin.com/in/markus17
050-8536620 / 06-50258350
Re: can indexing information stored in db rather than filesystem?
Posted by "kiran.bodigam" <ki...@gmail.com>.
Thanks for u r reply guys....
As suggested i agree that we are losing many of the benefits of Solr/Lucene
but i still want to store the index output (index files) in db table please
suggest what are the steps i need to follow to configure the db with SOLR
Engine (As how we done in it solrconfig.xml
<dataDir>${solr.data.dir:}</dataDir> similarly i would like to give the path
for db table)..?
--
View this message in context: http://lucene.472066.n3.nabble.com/can-indexing-information-stored-in-db-rather-than-filesystem-tp3319687p3332663.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: can indexing information stored in db rather than filesystem?
Posted by Gora Mohanty <go...@mimirtech.com>.
On Thu, Sep 8, 2011 at 6:57 PM, kiran.bodigam <ki...@gmail.com> wrote:
> SOLR will keep the indexed information in directory called
> index(${DATA_DIR}/index) after clicking the commit from admin console and
> also we can override the existing index files. similarly can we keep (or)
> store these index related information in permanent location like db table
> rather than a file system?
What do you mean by index-related information? If you are referring
to dataimport.properties, the file that stores information about the
last time the dataimporthandler (DIH) ran, then DIH will only pick
up information from that file. However, you could push that data into
a database, and retrieve then when reindexing, if so desired.
If you are referring to the index files themselves, it makes no sense
to do that, as one would be losing many of the benefits of Solr/Lucene.
Regards,
Gora