You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Uwe Clement <uw...@exxcellent.de> on 2013/01/15 09:59:45 UTC
Solr Query | Loading documents with large content (Performance)
Hi there,
sometimes we have to load very big documents, 1-2 multi-value-fields of it
can contain 10.000 items. And unfortunately we need this informations.
We have to load 50 documents in order to show to the result table in the
UI.
The query takes around 50 seconds. I guess 48 seconds of it, is just to
transfer the content of the documents over the net.
What can I do here?
- I know, can take out this long informations outside of the document.
But this is also not really a solution
- Then I was thinking about compressed-fields. They come with solr 4.1
again, right?
How is it with compressed field. As I understood the stored field will be
stored in a compressed way. Ok, but when they will be uncompressed?
- Before sending back to the client on server-side?
- Or, on the clientside? I am using solrJ.
Any other ideas? Can it work to increase the query performance using
compressed fields?
Thanks a lot for your ideas and answers!
Regards
Uwe
--
Uwe Clement
Software Architect
Project Manager
___ |X__________________________________________________________
X|
eXXcellent solutions gmbh
Beim Alten Fritz 2
D-89075 Ulm
e | <ma...@exxcellent.de> uwe.clement@exxcellent.de
m | +49 [0]151-275 692 27
i | <http://www.exxcellent.de> http://www.exxcellent.de
________________________________________________________________
Geschäftsführer: Dr. Martina Burgetsmeier, Wilhelm Zorn, Gerhard Gruber
Sitz der Gesellschaft: Ulm, Registergericht: Ulm HRB 4309
Re: Solr Query | Loading documents with large content (Performance)
Posted by Otis Gospodnetic <ot...@gmail.com>.
Hi,
Have a look under
http://wiki.apache.org/solr/UpdateCSV#Methods_of_uploading_CSV_recordsabout
uploading a *local* file.
Otis
--
Solr & ElasticSearch Support
http://sematext.com/
On Tue, Jan 15, 2013 at 3:59 AM, Uwe Clement <uw...@exxcellent.de>wrote:
> Hi there,
>
>
>
> sometimes we have to load very big documents, 1-2 multi-value-fields of it
> can contain 10.000 items. And unfortunately we need this informations.
>
>
>
> We have to load 50 documents in order to show to the result table in the
> UI.
>
>
>
> The query takes around 50 seconds. I guess 48 seconds of it, is just to
> transfer the content of the documents over the net.
>
>
>
> What can I do here?
>
> - I know, can take out this long informations outside of the document.
> But this is also not really a solution
>
> - Then I was thinking about compressed-fields. They come with solr 4.1
> again, right?
>
>
>
> How is it with compressed field. As I understood the stored field will be
> stored in a compressed way. Ok, but when they will be uncompressed?
>
> - Before sending back to the client on server-side?
>
> - Or, on the clientside? I am using solrJ.
>
>
>
> Any other ideas? Can it work to increase the query performance using
> compressed fields?
>
>
>
> Thanks a lot for your ideas and answers!
>
>
>
> Regards
>
> Uwe
>
>
>
>
>
>
>
> --
>
> Uwe Clement
>
> Software Architect
>
> Project Manager
>
>
>
> ___ |X__________________________________________________________
>
> X|
>
>
>
>
>
> eXXcellent solutions gmbh
>
> Beim Alten Fritz 2
>
>
>
> D-89075 Ulm
>
>
>
> e | <ma...@exxcellent.de> uwe.clement@exxcellent.de
>
> m | +49 [0]151-275 692 27
>
> i | <http://www.exxcellent.de> http://www.exxcellent.de
>
>
>
> ________________________________________________________________
>
>
>
> Geschäftsführer: Dr. Martina Burgetsmeier, Wilhelm Zorn, Gerhard Gruber
> Sitz der Gesellschaft: Ulm, Registergericht: Ulm HRB 4309
>
>
>
>