You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Moritz Mädler <ma...@moritz-maedler.de> on 2010/03/17 09:49:59 UTC
Will Solr fit our needs?
Hi List,
we are running a marketplace which has about a comparable functionality like ebay (auctions, fixed-price items etc).
The items are placed on the market by users who want to sell their goods.
Currently we are using Sphinx as an indexing engine, but, as Sphinx returns only document ids we have to make a
database-query to fetch the data to display. This massively decreases performance as we have to do two requests to
display data.
I heard that Solr is able to return a complete dataset and we hope a switch to Solr can boost perfomance.
A critical question is left and i was not able to find a solution for it in the docs: Is it possible to update attributes directly in the
index?
An example for better illustration:
We have an index which holds all the auctions (containing auctionid, auction title) with its current prices(field: current_price). When a user places a new bid,
is it possible to update the attribute 'current_price' directly in the index so that we can fetch the current_price from Solr and not from the database?
I hope you understood my problem. It would be kind if someone can point me to the right direction.
Thanks alot!
Moritz
Re: Will Solr fit our needs?
Posted by Lukáš Vlček <lu...@gmail.com>.
On Thu, Mar 18, 2010 at 8:45 AM, Moritz Maedler <ma...@moritz-maedler.de>wrote:
> Hi guys!
>
> Thanks alot for your suggestions and help - I really appreciate that!
> As we need e.g. the the price for sorting I think it must be in the index?
> Thus, I'm not shure that a key-value-store is the thing we are looking for
> as we
> need a good searchengine.
> Currently we are using serveral indices items, bids, etc. I think that ZOIE
> looks
> kind of interesting - I will check that out.
>
> BTW do/can you recommend any "newbe" literature or resources for getting a
> bit
> more familiar w/ Solr/Lucene ?
>
Probably the best way how to get your questions answered it participating in
mail lists. Documentation and wiki pages as well. There are also some books:
check Lucene in Action (http://www.manning.com/hatcher3/) and Solr 1.4
Enterprise Search Server (
http://www.packtpub.com/solr-1-4-enterprise-search-server). Other books
about Lucene and Solr are WIP but unfortunately not released yet.
>
> Thanks a lot again guys!
>
>
> -----Ursprüngliche Nachricht-----
> Von: Lance Norskog [mailto:goksron@gmail.com]
> Gesendet: Donnerstag, 18. März 2010 00:38
> An: solr-user@lucene.apache.org
> Betreff: Re: Will Solr fit our needs?
>
> Another option is the ExternalFileField:
>
>
> http://www.lucidimagination.com/search/document/CDRG_ch04_4.4.4?q=ExternalFileField
>
> This lets you store the current prices for all items in a separate
> file. You can only use it in a function query, that is. But it does
> allow you to maintain one Solr index, which is very very worthwhile.
>
> On Wed, Mar 17, 2010 at 4:19 AM, Geert-Jan Brits <gb...@gmail.com> wrote:
> > If you dont' plan on filtering/ sorting and/or faceting on fast-changing
> > fields it would be better to store them outside of solr/lucene in my
> > opinion.
> >
> > If you must: for indexing-performance reasons you will probably end up
> with
> > maintaining seperate indices (1 for slow-changing/static fields and 1 for
> > fast-changing-fields) .
> > You frequently commit the fast-changing -index to incorporate the changes
> > in current_price. Afterwards you have 2 options I believe:
> >
> > 1. use parallelreader to query the seperate indices directly. Afaik, this
> is
> > not (completely) integrated in Solr... I wouldn't recommend it.
> > 2. after you commit the fast-changing-index, merge with the static-index.
> > You're left with 1 fresh index, which you can push to your slave-servers.
> > (all this in regular interverals)
> >
> > Disadvatages:
> > - In any way, you must be very careful with maintaining multiple parallel
> > indexes with the purpose of treating them as one. For instance document
> > inserts must be done exactly in the same order, otherwise the indices go
> > 'out-of-sync' and are unusable.
> > - higher maintenance
> > - there is always a time-window in which the current_price values are
> stale.
> > If that's within reqs that's ok.
> >
> > The other path, which I recommend, would be to store the current_price
> > outside of solr (like you're currently doing) but instead of using a
> > relational db, try looking into persistent key-value stores. Many of them
> > exist and a lot of progress has been made in the last couple of years.
> For
> > simple key-lookups (what you need as far as I can tell) they really blow
> > every relational db out of the water (considering the same hardware of
> > course)
> >
> > We're currently using Tokyo Cabinet with the server-frontend Tokyo Tyrant
> > and seeing almost a 5x increased in lookup performance compared to our
> > previous kv-store memcachedDB which is based on BerkelyDB. Memcachedb was
> > already several times faster than our mysql-setup (although not optimally
> > tuned) .
> >
> > to sum things up: use the best tools for what they were meant to do.
> >
> > - index/search --> solr/ lucene without a doubt.
> >
> > - kv-lookup --> consensus is still forming, and a lot of players (with a
> lot
> > of different types of functionality) but if all you need is simple
> > key-value-lookup, I would go for Tokyo Cabinet (TC) / Tyrant at the
> moment.
> > Please note that TC and competitors aren't just some code/ hobby
> projects
> > but are usually born out of a real need at huge websites / social
> networks
> > such as TC which is born from mixi (big social network in Japan) . So at
> > least you're in good company..
> >
> > for kv-stores I would suggest to begin your research at:
> >
> http://www.metabrew.com/article/anti-rdbms-a-list-of-distributed-key-value-stores/
> > (beginning
> > 2009)
> > http://randomfoo.net/2009/04/20/some-notes-on-distributed-key-stores(half
> > 2009)
> > and get a feel of the kv-playing field.
> >
> > Hope this (pretty long) post helps,
> > Geert-Jan
> >
> >
> > 2010/3/17 Krzysztof Grodzicki <kr...@iterate.pl>
> >
> >> Hi Mortiz,
> >>
> >> You can take a look on the project ZOIE -
> >> http://code.google.com/p/zoie/. I think it's that what are you looking
> >> for.
> >>
> >> br
> >> Krzysztof
> >>
> >> On Wed, Mar 17, 2010 at 9:49 AM, Moritz Mädler <ma...@moritz-maedler.de>
> >> wrote:
> >> > Hi List,
> >> >
> >> > we are running a marketplace which has about a comparable
> functionality
> >> like ebay (auctions, fixed-price items etc).
> >> > The items are placed on the market by users who want to sell their
> goods.
> >> >
> >> > Currently we are using Sphinx as an indexing engine, but, as Sphinx
> >> returns only document ids we have to make a
> >> > database-query to fetch the data to display. This massively decreases
> >> performance as we have to do two requests to
> >> > display data.
> >> >
> >> > I heard that Solr is able to return a complete dataset and we hope a
> >> switch to Solr can boost perfomance.
> >> > A critical question is left and i was not able to find a solution for
> it
> >> in the docs: Is it possible to update attributes directly in the
> >> > index?
> >> > An example for better illustration:
> >> > We have an index which holds all the auctions (containing auctionid,
> >> auction title) with its current prices(field: current_price). When a
> user
> >> places a new bid,
> >> > is it possible to update the attribute 'current_price' directly in the
> >> index so that we can fetch the current_price from Solr and not from the
> >> database?
> >> >
> >> > I hope you understood my problem. It would be kind if someone can
> point
> >> me to the right direction.
> >> >
> >> > Thanks alot!
> >> >
> >> > Moritz
> >>
> >
>
>
>
> --
> Lance Norskog
> goksron@gmail.com
>
>
AW: Will Solr fit our needs?
Posted by Moritz Maedler <ma...@moritz-maedler.de>.
Hi guys!
Thanks alot for your suggestions and help - I really appreciate that!
As we need e.g. the the price for sorting I think it must be in the index?
Thus, I'm not shure that a key-value-store is the thing we are looking for as we
need a good searchengine.
Currently we are using serveral indices items, bids, etc. I think that ZOIE looks
kind of interesting - I will check that out.
BTW do/can you recommend any "newbe" literature or resources for getting a bit
more familiar w/ Solr/Lucene ?
Thanks a lot again guys!
-----Ursprüngliche Nachricht-----
Von: Lance Norskog [mailto:goksron@gmail.com]
Gesendet: Donnerstag, 18. März 2010 00:38
An: solr-user@lucene.apache.org
Betreff: Re: Will Solr fit our needs?
Another option is the ExternalFileField:
http://www.lucidimagination.com/search/document/CDRG_ch04_4.4.4?q=ExternalFileField
This lets you store the current prices for all items in a separate
file. You can only use it in a function query, that is. But it does
allow you to maintain one Solr index, which is very very worthwhile.
On Wed, Mar 17, 2010 at 4:19 AM, Geert-Jan Brits <gb...@gmail.com> wrote:
> If you dont' plan on filtering/ sorting and/or faceting on fast-changing
> fields it would be better to store them outside of solr/lucene in my
> opinion.
>
> If you must: for indexing-performance reasons you will probably end up with
> maintaining seperate indices (1 for slow-changing/static fields and 1 for
> fast-changing-fields) .
> You frequently commit the fast-changing -index to incorporate the changes
> in current_price. Afterwards you have 2 options I believe:
>
> 1. use parallelreader to query the seperate indices directly. Afaik, this is
> not (completely) integrated in Solr... I wouldn't recommend it.
> 2. after you commit the fast-changing-index, merge with the static-index.
> You're left with 1 fresh index, which you can push to your slave-servers.
> (all this in regular interverals)
>
> Disadvatages:
> - In any way, you must be very careful with maintaining multiple parallel
> indexes with the purpose of treating them as one. For instance document
> inserts must be done exactly in the same order, otherwise the indices go
> 'out-of-sync' and are unusable.
> - higher maintenance
> - there is always a time-window in which the current_price values are stale.
> If that's within reqs that's ok.
>
> The other path, which I recommend, would be to store the current_price
> outside of solr (like you're currently doing) but instead of using a
> relational db, try looking into persistent key-value stores. Many of them
> exist and a lot of progress has been made in the last couple of years. For
> simple key-lookups (what you need as far as I can tell) they really blow
> every relational db out of the water (considering the same hardware of
> course)
>
> We're currently using Tokyo Cabinet with the server-frontend Tokyo Tyrant
> and seeing almost a 5x increased in lookup performance compared to our
> previous kv-store memcachedDB which is based on BerkelyDB. Memcachedb was
> already several times faster than our mysql-setup (although not optimally
> tuned) .
>
> to sum things up: use the best tools for what they were meant to do.
>
> - index/search --> solr/ lucene without a doubt.
>
> - kv-lookup --> consensus is still forming, and a lot of players (with a lot
> of different types of functionality) but if all you need is simple
> key-value-lookup, I would go for Tokyo Cabinet (TC) / Tyrant at the moment.
> Please note that TC and competitors aren't just some code/ hobby projects
> but are usually born out of a real need at huge websites / social networks
> such as TC which is born from mixi (big social network in Japan) . So at
> least you're in good company..
>
> for kv-stores I would suggest to begin your research at:
> http://www.metabrew.com/article/anti-rdbms-a-list-of-distributed-key-value-stores/
> (beginning
> 2009)
> http://randomfoo.net/2009/04/20/some-notes-on-distributed-key-stores (half
> 2009)
> and get a feel of the kv-playing field.
>
> Hope this (pretty long) post helps,
> Geert-Jan
>
>
> 2010/3/17 Krzysztof Grodzicki <kr...@iterate.pl>
>
>> Hi Mortiz,
>>
>> You can take a look on the project ZOIE -
>> http://code.google.com/p/zoie/. I think it's that what are you looking
>> for.
>>
>> br
>> Krzysztof
>>
>> On Wed, Mar 17, 2010 at 9:49 AM, Moritz Mädler <ma...@moritz-maedler.de>
>> wrote:
>> > Hi List,
>> >
>> > we are running a marketplace which has about a comparable functionality
>> like ebay (auctions, fixed-price items etc).
>> > The items are placed on the market by users who want to sell their goods.
>> >
>> > Currently we are using Sphinx as an indexing engine, but, as Sphinx
>> returns only document ids we have to make a
>> > database-query to fetch the data to display. This massively decreases
>> performance as we have to do two requests to
>> > display data.
>> >
>> > I heard that Solr is able to return a complete dataset and we hope a
>> switch to Solr can boost perfomance.
>> > A critical question is left and i was not able to find a solution for it
>> in the docs: Is it possible to update attributes directly in the
>> > index?
>> > An example for better illustration:
>> > We have an index which holds all the auctions (containing auctionid,
>> auction title) with its current prices(field: current_price). When a user
>> places a new bid,
>> > is it possible to update the attribute 'current_price' directly in the
>> index so that we can fetch the current_price from Solr and not from the
>> database?
>> >
>> > I hope you understood my problem. It would be kind if someone can point
>> me to the right direction.
>> >
>> > Thanks alot!
>> >
>> > Moritz
>>
>
--
Lance Norskog
goksron@gmail.com
Re: Will Solr fit our needs?
Posted by Lance Norskog <go...@gmail.com>.
Another option is the ExternalFileField:
http://www.lucidimagination.com/search/document/CDRG_ch04_4.4.4?q=ExternalFileField
This lets you store the current prices for all items in a separate
file. You can only use it in a function query, that is. But it does
allow you to maintain one Solr index, which is very very worthwhile.
On Wed, Mar 17, 2010 at 4:19 AM, Geert-Jan Brits <gb...@gmail.com> wrote:
> If you dont' plan on filtering/ sorting and/or faceting on fast-changing
> fields it would be better to store them outside of solr/lucene in my
> opinion.
>
> If you must: for indexing-performance reasons you will probably end up with
> maintaining seperate indices (1 for slow-changing/static fields and 1 for
> fast-changing-fields) .
> You frequently commit the fast-changing -index to incorporate the changes
> in current_price. Afterwards you have 2 options I believe:
>
> 1. use parallelreader to query the seperate indices directly. Afaik, this is
> not (completely) integrated in Solr... I wouldn't recommend it.
> 2. after you commit the fast-changing-index, merge with the static-index.
> You're left with 1 fresh index, which you can push to your slave-servers.
> (all this in regular interverals)
>
> Disadvatages:
> - In any way, you must be very careful with maintaining multiple parallel
> indexes with the purpose of treating them as one. For instance document
> inserts must be done exactly in the same order, otherwise the indices go
> 'out-of-sync' and are unusable.
> - higher maintenance
> - there is always a time-window in which the current_price values are stale.
> If that's within reqs that's ok.
>
> The other path, which I recommend, would be to store the current_price
> outside of solr (like you're currently doing) but instead of using a
> relational db, try looking into persistent key-value stores. Many of them
> exist and a lot of progress has been made in the last couple of years. For
> simple key-lookups (what you need as far as I can tell) they really blow
> every relational db out of the water (considering the same hardware of
> course)
>
> We're currently using Tokyo Cabinet with the server-frontend Tokyo Tyrant
> and seeing almost a 5x increased in lookup performance compared to our
> previous kv-store memcachedDB which is based on BerkelyDB. Memcachedb was
> already several times faster than our mysql-setup (although not optimally
> tuned) .
>
> to sum things up: use the best tools for what they were meant to do.
>
> - index/search --> solr/ lucene without a doubt.
>
> - kv-lookup --> consensus is still forming, and a lot of players (with a lot
> of different types of functionality) but if all you need is simple
> key-value-lookup, I would go for Tokyo Cabinet (TC) / Tyrant at the moment.
> Please note that TC and competitors aren't just some code/ hobby projects
> but are usually born out of a real need at huge websites / social networks
> such as TC which is born from mixi (big social network in Japan) . So at
> least you're in good company..
>
> for kv-stores I would suggest to begin your research at:
> http://www.metabrew.com/article/anti-rdbms-a-list-of-distributed-key-value-stores/
> (beginning
> 2009)
> http://randomfoo.net/2009/04/20/some-notes-on-distributed-key-stores (half
> 2009)
> and get a feel of the kv-playing field.
>
> Hope this (pretty long) post helps,
> Geert-Jan
>
>
> 2010/3/17 Krzysztof Grodzicki <kr...@iterate.pl>
>
>> Hi Mortiz,
>>
>> You can take a look on the project ZOIE -
>> http://code.google.com/p/zoie/. I think it's that what are you looking
>> for.
>>
>> br
>> Krzysztof
>>
>> On Wed, Mar 17, 2010 at 9:49 AM, Moritz Mädler <ma...@moritz-maedler.de>
>> wrote:
>> > Hi List,
>> >
>> > we are running a marketplace which has about a comparable functionality
>> like ebay (auctions, fixed-price items etc).
>> > The items are placed on the market by users who want to sell their goods.
>> >
>> > Currently we are using Sphinx as an indexing engine, but, as Sphinx
>> returns only document ids we have to make a
>> > database-query to fetch the data to display. This massively decreases
>> performance as we have to do two requests to
>> > display data.
>> >
>> > I heard that Solr is able to return a complete dataset and we hope a
>> switch to Solr can boost perfomance.
>> > A critical question is left and i was not able to find a solution for it
>> in the docs: Is it possible to update attributes directly in the
>> > index?
>> > An example for better illustration:
>> > We have an index which holds all the auctions (containing auctionid,
>> auction title) with its current prices(field: current_price). When a user
>> places a new bid,
>> > is it possible to update the attribute 'current_price' directly in the
>> index so that we can fetch the current_price from Solr and not from the
>> database?
>> >
>> > I hope you understood my problem. It would be kind if someone can point
>> me to the right direction.
>> >
>> > Thanks alot!
>> >
>> > Moritz
>>
>
--
Lance Norskog
goksron@gmail.com
Re: Will Solr fit our needs?
Posted by Geert-Jan Brits <gb...@gmail.com>.
If you dont' plan on filtering/ sorting and/or faceting on fast-changing
fields it would be better to store them outside of solr/lucene in my
opinion.
If you must: for indexing-performance reasons you will probably end up with
maintaining seperate indices (1 for slow-changing/static fields and 1 for
fast-changing-fields) .
You frequently commit the fast-changing -index to incorporate the changes
in current_price. Afterwards you have 2 options I believe:
1. use parallelreader to query the seperate indices directly. Afaik, this is
not (completely) integrated in Solr... I wouldn't recommend it.
2. after you commit the fast-changing-index, merge with the static-index.
You're left with 1 fresh index, which you can push to your slave-servers.
(all this in regular interverals)
Disadvatages:
- In any way, you must be very careful with maintaining multiple parallel
indexes with the purpose of treating them as one. For instance document
inserts must be done exactly in the same order, otherwise the indices go
'out-of-sync' and are unusable.
- higher maintenance
- there is always a time-window in which the current_price values are stale.
If that's within reqs that's ok.
The other path, which I recommend, would be to store the current_price
outside of solr (like you're currently doing) but instead of using a
relational db, try looking into persistent key-value stores. Many of them
exist and a lot of progress has been made in the last couple of years. For
simple key-lookups (what you need as far as I can tell) they really blow
every relational db out of the water (considering the same hardware of
course)
We're currently using Tokyo Cabinet with the server-frontend Tokyo Tyrant
and seeing almost a 5x increased in lookup performance compared to our
previous kv-store memcachedDB which is based on BerkelyDB. Memcachedb was
already several times faster than our mysql-setup (although not optimally
tuned) .
to sum things up: use the best tools for what they were meant to do.
- index/search --> solr/ lucene without a doubt.
- kv-lookup --> consensus is still forming, and a lot of players (with a lot
of different types of functionality) but if all you need is simple
key-value-lookup, I would go for Tokyo Cabinet (TC) / Tyrant at the moment.
Please note that TC and competitors aren't just some code/ hobby projects
but are usually born out of a real need at huge websites / social networks
such as TC which is born from mixi (big social network in Japan) . So at
least you're in good company..
for kv-stores I would suggest to begin your research at:
http://www.metabrew.com/article/anti-rdbms-a-list-of-distributed-key-value-stores/
(beginning
2009)
http://randomfoo.net/2009/04/20/some-notes-on-distributed-key-stores (half
2009)
and get a feel of the kv-playing field.
Hope this (pretty long) post helps,
Geert-Jan
2010/3/17 Krzysztof Grodzicki <kr...@iterate.pl>
> Hi Mortiz,
>
> You can take a look on the project ZOIE -
> http://code.google.com/p/zoie/. I think it's that what are you looking
> for.
>
> br
> Krzysztof
>
> On Wed, Mar 17, 2010 at 9:49 AM, Moritz Mädler <ma...@moritz-maedler.de>
> wrote:
> > Hi List,
> >
> > we are running a marketplace which has about a comparable functionality
> like ebay (auctions, fixed-price items etc).
> > The items are placed on the market by users who want to sell their goods.
> >
> > Currently we are using Sphinx as an indexing engine, but, as Sphinx
> returns only document ids we have to make a
> > database-query to fetch the data to display. This massively decreases
> performance as we have to do two requests to
> > display data.
> >
> > I heard that Solr is able to return a complete dataset and we hope a
> switch to Solr can boost perfomance.
> > A critical question is left and i was not able to find a solution for it
> in the docs: Is it possible to update attributes directly in the
> > index?
> > An example for better illustration:
> > We have an index which holds all the auctions (containing auctionid,
> auction title) with its current prices(field: current_price). When a user
> places a new bid,
> > is it possible to update the attribute 'current_price' directly in the
> index so that we can fetch the current_price from Solr and not from the
> database?
> >
> > I hope you understood my problem. It would be kind if someone can point
> me to the right direction.
> >
> > Thanks alot!
> >
> > Moritz
>
Re: Will Solr fit our needs?
Posted by Krzysztof Grodzicki <kr...@iterate.pl>.
Hi Mortiz,
You can take a look on the project ZOIE -
http://code.google.com/p/zoie/. I think it's that what are you looking
for.
br
Krzysztof
On Wed, Mar 17, 2010 at 9:49 AM, Moritz Mädler <ma...@moritz-maedler.de> wrote:
> Hi List,
>
> we are running a marketplace which has about a comparable functionality like ebay (auctions, fixed-price items etc).
> The items are placed on the market by users who want to sell their goods.
>
> Currently we are using Sphinx as an indexing engine, but, as Sphinx returns only document ids we have to make a
> database-query to fetch the data to display. This massively decreases performance as we have to do two requests to
> display data.
>
> I heard that Solr is able to return a complete dataset and we hope a switch to Solr can boost perfomance.
> A critical question is left and i was not able to find a solution for it in the docs: Is it possible to update attributes directly in the
> index?
> An example for better illustration:
> We have an index which holds all the auctions (containing auctionid, auction title) with its current prices(field: current_price). When a user places a new bid,
> is it possible to update the attribute 'current_price' directly in the index so that we can fetch the current_price from Solr and not from the database?
>
> I hope you understood my problem. It would be kind if someone can point me to the right direction.
>
> Thanks alot!
>
> Moritz
Re: Will Solr fit our needs?
Posted by Lukáš Vlček <lu...@gmail.com>.
Having been thinking about your questions again and I think that if you are
expecting that the price value will be changing a lot, especially when
talking about auctions then you should consider not storing the actual price
into the full text index but into some fast datastore. Some kind of scalable
in-memory hash map with journal based backup would do this job better I
think.
Just my 2 cents.
Regards,
Lukas
On Wed, Mar 17, 2010 at 10:36 AM, Lukáš Vlček <lu...@gmail.com> wrote:
> Hi,
>
> Solr is running on top of Lucene and as far as I know Lucene knows only one
> approach how to update the document field content: that is delete first and
> then (re)index with new values.
> However, saying this it does not mean you can not implement what you need.
> Take a look at ParallelReader API
> http://lucene.apache.org/java/3_0_1/api/all/org/apache/lucene/index/ParallelReader.html
>
> I am not sure if this functionality is directly exposed via Solr API. Try
> digging mail lists (search-lucene.com or lucidimagination.com can be of
> good help while you can narrow search to Solr only:
> http://search-lucene.com/?q=ParallelReader&fc_project=Solr or
> http://www.lucidimagination.com/search/?q=ParallelReader#/p:solr). For
> example the following mail thread seems to be relevant:
> http://search-lucene.com/m/iT2hMvtDt5 (though it is bit dated)
>
> Do you really use only one physical index for all auctions? If yes then you
> might consider using ParallelReader but if the index is large then I am not
> sure about the performance. If you are planning to partition your index then
> it can get more complex but faster.
>
> Regards,
> Lukas
>
>
> On Wed, Mar 17, 2010 at 9:49 AM, Moritz Mädler <ma...@moritz-maedler.de>wrote:
>
>> Hi List,
>>
>> we are running a marketplace which has about a comparable functionality
>> like ebay (auctions, fixed-price items etc).
>> The items are placed on the market by users who want to sell their goods.
>>
>> Currently we are using Sphinx as an indexing engine, but, as Sphinx
>> returns only document ids we have to make a
>> database-query to fetch the data to display. This massively decreases
>> performance as we have to do two requests to
>> display data.
>>
>> I heard that Solr is able to return a complete dataset and we hope a
>> switch to Solr can boost perfomance.
>> A critical question is left and i was not able to find a solution for it
>> in the docs: Is it possible to update attributes directly in the
>> index?
>> An example for better illustration:
>> We have an index which holds all the auctions (containing auctionid,
>> auction title) with its current prices(field: current_price). When a user
>> places a new bid,
>> is it possible to update the attribute 'current_price' directly in the
>> index so that we can fetch the current_price from Solr and not from the
>> database?
>>
>> I hope you understood my problem. It would be kind if someone can point me
>> to the right direction.
>>
>> Thanks alot!
>>
>> Moritz
>
>
>
Re: Will Solr fit our needs?
Posted by Lukáš Vlček <lu...@gmail.com>.
Hi,
Solr is running on top of Lucene and as far as I know Lucene knows only one
approach how to update the document field content: that is delete first and
then (re)index with new values.
However, saying this it does not mean you can not implement what you need.
Take a look at ParallelReader API
http://lucene.apache.org/java/3_0_1/api/all/org/apache/lucene/index/ParallelReader.html
I am not sure if this functionality is directly exposed via Solr API. Try
digging mail lists (search-lucene.com or lucidimagination.com can be of good
help while you can narrow search to Solr only:
http://search-lucene.com/?q=ParallelReader&fc_project=Solr or
http://www.lucidimagination.com/search/?q=ParallelReader#/p:solr). For
example the following mail thread seems to be relevant:
http://search-lucene.com/m/iT2hMvtDt5 (though it is bit dated)
Do you really use only one physical index for all auctions? If yes then you
might consider using ParallelReader but if the index is large then I am not
sure about the performance. If you are planning to partition your index then
it can get more complex but faster.
Regards,
Lukas
On Wed, Mar 17, 2010 at 9:49 AM, Moritz Mädler <ma...@moritz-maedler.de>wrote:
> Hi List,
>
> we are running a marketplace which has about a comparable functionality
> like ebay (auctions, fixed-price items etc).
> The items are placed on the market by users who want to sell their goods.
>
> Currently we are using Sphinx as an indexing engine, but, as Sphinx returns
> only document ids we have to make a
> database-query to fetch the data to display. This massively decreases
> performance as we have to do two requests to
> display data.
>
> I heard that Solr is able to return a complete dataset and we hope a switch
> to Solr can boost perfomance.
> A critical question is left and i was not able to find a solution for it in
> the docs: Is it possible to update attributes directly in the
> index?
> An example for better illustration:
> We have an index which holds all the auctions (containing auctionid,
> auction title) with its current prices(field: current_price). When a user
> places a new bid,
> is it possible to update the attribute 'current_price' directly in the
> index so that we can fetch the current_price from Solr and not from the
> database?
>
> I hope you understood my problem. It would be kind if someone can point me
> to the right direction.
>
> Thanks alot!
>
> Moritz