You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Denis Kuzmenok <fo...@ukr.net> on 2011/05/27 13:01:39 UTC
Documents update
Hi.
I have and indexed database which is indexed few times a day and
contain tinyint flag (like is_enabled, is_active, etc), and content
isn't changed too often, but flags are.
So if i index via post.jar only flags then entire document is deleted
and there's only unique key and flags.
Is there any way to index certain columns, and not to change all
document?
Thanks
Re: Documents update
Posted by Alexey Serba <as...@gmail.com>.
> Will it be slow if there are 3-5 million key/value rows?
AFAIK it shouldn't affect search time significantly as Solr caches it
in memory after you reloading Solr core / issuing commit.
But obviously you need more memory and commit/reload will take more time.
Re: Documents update
Posted by Denis Kuzmenok <fo...@ukr.net>.
Will it be slow if there are 3-5 million key/value rows?
> http://lucene.apache.org/solr/api/org/apache/solr/schema/ExternalFileField.html
> On Tuesday 31 May 2011 15:41:32 Denis Kuzmenok wrote:
>> Flags are stored to filter results and it's pretty highloaded, it's
>> working fine, but i can't update index very often just to make flags
>> up to time =\
>> Where can i read about using external fields / files?
Re: Documents update
Posted by Denis Kuzmenok <fo...@ukr.net>.
Created file, reloaded solr - externalfilefield works fine, if i
change change external files and do "curl
http://127.0.0.1:4900/solr/site/update -H "Content-Type: text/xml" --data-binary '<commit />'"
then no thanges are made. If i start solr without external files and
then create them - they are not working..
What is wrong?
PS: Solr 3.2
> http://lucene.apache.org/solr/api/org/apache/solr/schema/ExternalFileField.html
> On Tuesday 31 May 2011 15:41:32 Denis Kuzmenok wrote:
>> Flags are stored to filter results and it's pretty highloaded, it's
>> working fine, but i can't update index very often just to make flags
>> up to time =\
>> Where can i read about using external fields / files?
>>
>> > And it wouldn't work unless all the data is stored anyway. Currently
>> > there's no way to update a single field in a document, although there's
>> > work being done in that direction (see the "column stride" JIRA).
>> >
>> > What do you want to do with these fields? If it's to influence scoring,
>> > you could look at external fields.
>> >
>> > If the flags are a selection criteria, it's...harder. What are the flags
>> > used for? Could you consider essentially storing a map of the
>> > uniqueKey's and flags in a "special" document and having your app
>> > read that document and merge the results with the output? If this seems
>> > irrelevant, a more complete statement of the use-case would be helpful.
>> >
>> > Best
>> > Erick
Re: Documents update
Posted by Markus Jelsma <ma...@openindex.io>.
http://lucene.apache.org/solr/api/org/apache/solr/schema/ExternalFileField.html
On Tuesday 31 May 2011 15:41:32 Denis Kuzmenok wrote:
> Flags are stored to filter results and it's pretty highloaded, it's
> working fine, but i can't update index very often just to make flags
> up to time =\
> Where can i read about using external fields / files?
>
> > And it wouldn't work unless all the data is stored anyway. Currently
> > there's no way to update a single field in a document, although there's
> > work being done in that direction (see the "column stride" JIRA).
> >
> > What do you want to do with these fields? If it's to influence scoring,
> > you could look at external fields.
> >
> > If the flags are a selection criteria, it's...harder. What are the flags
> > used for? Could you consider essentially storing a map of the
> > uniqueKey's and flags in a "special" document and having your app
> > read that document and merge the results with the output? If this seems
> > irrelevant, a more complete statement of the use-case would be helpful.
> >
> > Best
> > Erick
--
Markus Jelsma - CTO - Openindex
http://www.linkedin.com/in/markus17
050-8536620 / 06-50258350
Re: Documents update
Posted by Denis Kuzmenok <fo...@ukr.net>.
Flags are stored to filter results and it's pretty highloaded, it's
working fine, but i can't update index very often just to make flags
up to time =\
Where can i read about using external fields / files?
> And it wouldn't work unless all the data is stored anyway. Currently there's
> no way to update a single field in a document, although there's work being
> done in that direction (see the "column stride" JIRA).
> What do you want to do with these fields? If it's to influence scoring, you
> could look at external fields.
> If the flags are a selection criteria, it's...harder. What are the flags
> used for? Could you consider essentially storing a map of the
> uniqueKey's and flags in a "special" document and having your app
> read that document and merge the results with the output? If this seems
> irrelevant, a more complete statement of the use-case would be helpful.
> Best
> Erick
Re: Documents update
Posted by Erick Erickson <er...@gmail.com>.
And it wouldn't work unless all the data is stored anyway. Currently there's
no way to update a single field in a document, although there's work being
done in that direction (see the "column stride" JIRA).
What do you want to do with these fields? If it's to influence scoring, you
could look at external fields.
If the flags are a selection criteria, it's...harder. What are the flags
used for? Could you consider essentially storing a map of the
uniqueKey's and flags in a "special" document and having your app
read that document and merge the results with the output? If this seems
irrelevant, a more complete statement of the use-case would be helpful.
Best
Erick
On Fri, May 27, 2011 at 4:33 AM, Denis Kuzmenok <fo...@ukr.net> wrote:
> I'm using 3.1 now. Indexing lasts for a few hours, and have big
> plain size. Getting all documents would be rather slow :(
>
>
>> Not with 1.4, but apparently there is a patch for trunk. Not
>> sure if it is in 3.1.
>
>> If you are on 1.4, you could first query Solr to get the data
>> for the document to be changed, change the modified values,
>> and make a complete XML, including all fields, for post.jar.
>
>> Regards,
>> Gora
>
>
>
>
>
Re: Documents update
Posted by Denis Kuzmenok <fo...@ukr.net>.
I'm using 3.1 now. Indexing lasts for a few hours, and have big
plain size. Getting all documents would be rather slow :(
> Not with 1.4, but apparently there is a patch for trunk. Not
> sure if it is in 3.1.
> If you are on 1.4, you could first query Solr to get the data
> for the document to be changed, change the modified values,
> and make a complete XML, including all fields, for post.jar.
> Regards,
> Gora
Re: Documents update
Posted by Gora Mohanty <go...@mimirtech.com>.
2011/5/27 Denis Kuzmenok <fo...@ukr.net>:
> Hi.
>
> I have and indexed database which is indexed few times a day and
> contain tinyint flag (like is_enabled, is_active, etc), and content
> isn't changed too often, but flags are.
> So if i index via post.jar only flags then entire document is deleted
> and there's only unique key and flags.
> Is there any way to index certain columns, and not to change all
> document?
[...]
Not with 1.4, but apparently there is a patch for trunk. Not
sure if it is in 3.1.
If you are on 1.4, you could first query Solr to get the data
for the document to be changed, change the modified values,
and make a complete XML, including all fields, for post.jar.
Regards,
Gora