You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Oliver Schrenk <os...@elmar.nl> on 2015/11/17 13:08:10 UTC
Sold 4.10.4 dropping index on shutdown
Hi,
since we upgraded our cluster from 4.7 to 4.10.4 we are experiencing issues. When shutting down the service (with a confirmed graceful shutdown in the logs), the index is dropped, with only one lonely `segments.gen` file left for each shard and all other files being deleted.
There is no message in the logs, other than graceful shutdown. Did anybody have a similar issues and has some advice?
Cheers,
Oliver
Re: Solr 5.2 child documents
Posted by Novin <no...@temetra.com>.
Thanks Mikhali.
On 23/11/15 17:44, Mikhail Khludnev wrote:
> Novin,
>
> As it's stated here http://yonik.com/solr-nested-objects/ there is no
> requirements for schema. Perhaps
> http://blog.griddynamics.com/2013/09/solr-block-join-support.html might be
> useful too.
>
> I suppose LogUpdateProcessor messages might answer about the root cause of
> your problem.
>
> On Mon, Nov 23, 2015 at 4:45 PM, Novin <no...@temetra.com> wrote:
>
>> Could somebody please give schema example for creating child document?
>>
>>
>> On 23/11/15 12:57, Novin wrote:
>>
>>> When I query using solr admin page. I can see the parent document there
>>> in results, but now child document.
>>>
>>> On 23/11/15 12:24, Alexandre Rafalovitch wrote:
>>>
>>>> Do you get a parent doc? If not, maybe you forgot to commit the whole
>>>> lot.
>>>> On 23 Nov 2015 6:42 am, "Novin" <no...@temetra.com> wrote:
>>>>
>>>> Hi,
>>>>> When I query q=*:* I can't get child documents back, Below is
>>>>> configuration I am using for child Document to index in solr.
>>>>>
>>>>> Am I missing something?
>>>>>
>>>>> note: child document are using dynamic fields.
>>>>>
>>>>>
>>>>>
>>>>> SolrInputDocument childDoc = new SolrInputDocument();
>>>>> childDoc.addfield("index_l", 20000);
>>>>> childDoc.addfield("datetime_dt", DateTime.now().toIsoDateTime());
>>>>>
>>>>> parentDoc. .addChildDocument(childDoc);
>>>>>
>>>>> schema.xml
>>>>> <field name="_root_" type="string" indexed="true" stored="true"/>
>>>>>
>>>>> solrconfig.xml
>>>>> <cache name="perSegFilter"
>>>>> class="solr.search.LRUCache"
>>>>> size="10"
>>>>> initialSize="0"
>>>>> autowarmCount="10" />
>>>>>
>>>>> If needed more info I can provide that either.
>>>>>
>>>>> Cheeers,
>>>>> Novin
>>>>>
>>>>>
>>>>>
>
Re: Solr 5.2 child documents
Posted by Mikhail Khludnev <mk...@griddynamics.com>.
Novin,
As it's stated here http://yonik.com/solr-nested-objects/ there is no
requirements for schema. Perhaps
http://blog.griddynamics.com/2013/09/solr-block-join-support.html might be
useful too.
I suppose LogUpdateProcessor messages might answer about the root cause of
your problem.
On Mon, Nov 23, 2015 at 4:45 PM, Novin <no...@temetra.com> wrote:
> Could somebody please give schema example for creating child document?
>
>
> On 23/11/15 12:57, Novin wrote:
>
>> When I query using solr admin page. I can see the parent document there
>> in results, but now child document.
>>
>> On 23/11/15 12:24, Alexandre Rafalovitch wrote:
>>
>>> Do you get a parent doc? If not, maybe you forgot to commit the whole
>>> lot.
>>> On 23 Nov 2015 6:42 am, "Novin" <no...@temetra.com> wrote:
>>>
>>> Hi,
>>>>
>>>> When I query q=*:* I can't get child documents back, Below is
>>>> configuration I am using for child Document to index in solr.
>>>>
>>>> Am I missing something?
>>>>
>>>> note: child document are using dynamic fields.
>>>>
>>>>
>>>>
>>>> SolrInputDocument childDoc = new SolrInputDocument();
>>>> childDoc.addfield("index_l", 20000);
>>>> childDoc.addfield("datetime_dt", DateTime.now().toIsoDateTime());
>>>>
>>>> parentDoc. .addChildDocument(childDoc);
>>>>
>>>> schema.xml
>>>> <field name="_root_" type="string" indexed="true" stored="true"/>
>>>>
>>>> solrconfig.xml
>>>> <cache name="perSegFilter"
>>>> class="solr.search.LRUCache"
>>>> size="10"
>>>> initialSize="0"
>>>> autowarmCount="10" />
>>>>
>>>> If needed more info I can provide that either.
>>>>
>>>> Cheeers,
>>>> Novin
>>>>
>>>>
>>>>
>>
>
--
Sincerely yours
Mikhail Khludnev
Principal Engineer,
Grid Dynamics
<http://www.griddynamics.com>
<mk...@griddynamics.com>
Re: Solr 5.2 child documents
Posted by Novin <no...@temetra.com>.
Could somebody please give schema example for creating child document?
On 23/11/15 12:57, Novin wrote:
> When I query using solr admin page. I can see the parent document
> there in results, but now child document.
>
> On 23/11/15 12:24, Alexandre Rafalovitch wrote:
>> Do you get a parent doc? If not, maybe you forgot to commit the whole
>> lot.
>> On 23 Nov 2015 6:42 am, "Novin" <no...@temetra.com> wrote:
>>
>>> Hi,
>>>
>>> When I query q=*:* I can't get child documents back, Below is
>>> configuration I am using for child Document to index in solr.
>>>
>>> Am I missing something?
>>>
>>> note: child document are using dynamic fields.
>>>
>>>
>>>
>>> SolrInputDocument childDoc = new SolrInputDocument();
>>> childDoc.addfield("index_l", 20000);
>>> childDoc.addfield("datetime_dt", DateTime.now().toIsoDateTime());
>>>
>>> parentDoc. .addChildDocument(childDoc);
>>>
>>> schema.xml
>>> <field name="_root_" type="string" indexed="true" stored="true"/>
>>>
>>> solrconfig.xml
>>> <cache name="perSegFilter"
>>> class="solr.search.LRUCache"
>>> size="10"
>>> initialSize="0"
>>> autowarmCount="10" />
>>>
>>> If needed more info I can provide that either.
>>>
>>> Cheeers,
>>> Novin
>>>
>>>
>
Re: Solr 5.2 child documents
Posted by Novin <no...@temetra.com>.
When I query using solr admin page. I can see the parent document there
in results, but now child document.
On 23/11/15 12:24, Alexandre Rafalovitch wrote:
> Do you get a parent doc? If not, maybe you forgot to commit the whole lot.
> On 23 Nov 2015 6:42 am, "Novin" <no...@temetra.com> wrote:
>
>> Hi,
>>
>> When I query q=*:* I can't get child documents back, Below is
>> configuration I am using for child Document to index in solr.
>>
>> Am I missing something?
>>
>> note: child document are using dynamic fields.
>>
>>
>>
>> SolrInputDocument childDoc = new SolrInputDocument();
>> childDoc.addfield("index_l", 20000);
>> childDoc.addfield("datetime_dt", DateTime.now().toIsoDateTime());
>>
>> parentDoc. .addChildDocument(childDoc);
>>
>> schema.xml
>> <field name="_root_" type="string" indexed="true" stored="true"/>
>>
>> solrconfig.xml
>> <cache name="perSegFilter"
>> class="solr.search.LRUCache"
>> size="10"
>> initialSize="0"
>> autowarmCount="10" />
>>
>> If needed more info I can provide that either.
>>
>> Cheeers,
>> Novin
>>
>>
Re: Solr 5.2 child documents
Posted by Alexandre Rafalovitch <ar...@gmail.com>.
Do you get a parent doc? If not, maybe you forgot to commit the whole lot.
On 23 Nov 2015 6:42 am, "Novin" <no...@temetra.com> wrote:
> Hi,
>
> When I query q=*:* I can't get child documents back, Below is
> configuration I am using for child Document to index in solr.
>
> Am I missing something?
>
> note: child document are using dynamic fields.
>
>
>
> SolrInputDocument childDoc = new SolrInputDocument();
> childDoc.addfield("index_l", 20000);
> childDoc.addfield("datetime_dt", DateTime.now().toIsoDateTime());
>
> parentDoc. .addChildDocument(childDoc);
>
> schema.xml
> <field name="_root_" type="string" indexed="true" stored="true"/>
>
> solrconfig.xml
> <cache name="perSegFilter"
> class="solr.search.LRUCache"
> size="10"
> initialSize="0"
> autowarmCount="10" />
>
> If needed more info I can provide that either.
>
> Cheeers,
> Novin
>
>
Re: Solr 5.2 child documents
Posted by Chris Hostetter <ho...@fucit.org>.
: Subject: Solr 5.2 child documents
: References: <A0...@elmar.nl>
: <CA...@mail.gmail.com>
: <A9...@elmar.nl>
: In-Reply-To: <A9...@elmar.nl>
https://people.apache.org/~hossman/#threadhijack
Thread Hijacking on Mailing Lists
When starting a new discussion on a mailing list, please do not reply to
an existing message, instead start a fresh email. Even if you change the
subject line of your email, other mail headers still track which thread
you replied to and your question is "hidden" in that thread and gets less
attention. It makes following discussions in the mailing list archives
particularly difficult.
-Hoss
http://www.lucidworks.com/
Solr 5.2 child documents
Posted by Novin <no...@temetra.com>.
Hi,
When I query q=*:* I can't get child documents back, Below is
configuration I am using for child Document to index in solr.
Am I missing something?
note: child document are using dynamic fields.
SolrInputDocument childDoc = new SolrInputDocument();
childDoc.addfield("index_l", 20000);
childDoc.addfield("datetime_dt", DateTime.now().toIsoDateTime());
parentDoc. .addChildDocument(childDoc);
schema.xml
<field name="_root_" type="string" indexed="true" stored="true"/>
solrconfig.xml
<cache name="perSegFilter"
class="solr.search.LRUCache"
size="10"
initialSize="0"
autowarmCount="10" />
If needed more info I can provide that either.
Cheeers,
Novin
Re: Sold 4.10.4 dropping index on shutdown
Posted by Erick Erickson <er...@gmail.com>.
Oliver:
It's really got to be some kind of oddity in your environment. It's,
shall we say, highly unlikely that this is a fundamental problem in the
code and you're the _only_ one it's affected.
Best,
Erick
On Mon, Nov 23, 2015 at 3:21 AM, Upayavira <uv...@odoko.co.uk> wrote:
> Can you confirm that the indexes *are* in place on disk before you shut
> down? Whilst it may seem like indexes are being removed, this would seem
> extremely unlikely, and the explanation is likely something else.
>
> Try looking at your index files, post some content, commit, look again
> at your index files, then shut down. What changes have you seen? Did the
> commit 'take' and cause the files to be written?
>
> Upayavira
>
> On Mon, Nov 23, 2015, at 10:25 AM, Oliver Schrenk wrote:
>> Yes. Hard commit using SolrJ 4.10.4 client with UpdateResponse.commit()
>> which defaults to waitFlush=true and waitSearcher=true
>>
>> We reverted back to Solr 4.7 for the time being, which is a pity as we
>> really benefitted from the concurrent commit fix.
>>
>> > On 17 Nov 2015, at 18:00, Erick Erickson <er...@gmail.com> wrote:
>> >
>> > Did you commit after indexing and before shutting down? Even if you didn't, I'm
>> > still a bit surprised, but that's one possible explanation.
>> >
>> > But this is the first time I've seen this problem mentioned...
>> >
>> > Best,
>> > Erick
>> >
>> > On Tue, Nov 17, 2015 at 4:08 AM, Oliver Schrenk <os...@elmar.nl> wrote:
>> >> Hi,
>> >>
>> >> since we upgraded our cluster from 4.7 to 4.10.4 we are experiencing issues. When shutting down the service (with a confirmed graceful shutdown in the logs), the index is dropped, with only one lonely `segments.gen` file left for each shard and all other files being deleted.
>> >>
>> >> There is no message in the logs, other than graceful shutdown. Did anybody have a similar issues and has some advice?
>> >>
>> >> Cheers,
>> >> Oliver
>>
Re: Sold 4.10.4 dropping index on shutdown
Posted by Oliver Schrenk <os...@elmar.nl>.
It will take a while before we try out 4.10 again. We might just jump directly to Solr 5 (waiting for some fixes regarding faceting).
One thing I noticed today that luceneMatchVersion was still set to 4.7 and not to 4.10.4. But I would expect Solr 4.10 to handle this.
> On 23 Nov 2015, at 12:21, Upayavira <uv...@odoko.co.uk> wrote:
>
> Can you confirm that the indexes *are* in place on disk before you shut
> down? Whilst it may seem like indexes are being removed, this would seem
> extremely unlikely, and the explanation is likely something else.
>
> Try looking at your index files, post some content, commit, look again
> at your index files, then shut down. What changes have you seen? Did the
> commit 'take' and cause the files to be written?
>
> Upayavira
>
> On Mon, Nov 23, 2015, at 10:25 AM, Oliver Schrenk wrote:
>> Yes. Hard commit using SolrJ 4.10.4 client with UpdateResponse.commit()
>> which defaults to waitFlush=true and waitSearcher=true
>>
>> We reverted back to Solr 4.7 for the time being, which is a pity as we
>> really benefitted from the concurrent commit fix.
>>
>>> On 17 Nov 2015, at 18:00, Erick Erickson <er...@gmail.com> wrote:
>>>
>>> Did you commit after indexing and before shutting down? Even if you didn't, I'm
>>> still a bit surprised, but that's one possible explanation.
>>>
>>> But this is the first time I've seen this problem mentioned...
>>>
>>> Best,
>>> Erick
>>>
>>> On Tue, Nov 17, 2015 at 4:08 AM, Oliver Schrenk <os...@elmar.nl> wrote:
>>>> Hi,
>>>>
>>>> since we upgraded our cluster from 4.7 to 4.10.4 we are experiencing issues. When shutting down the service (with a confirmed graceful shutdown in the logs), the index is dropped, with only one lonely `segments.gen` file left for each shard and all other files being deleted.
>>>>
>>>> There is no message in the logs, other than graceful shutdown. Did anybody have a similar issues and has some advice?
>>>>
>>>> Cheers,
>>>> Oliver
>>
Re: Sold 4.10.4 dropping index on shutdown
Posted by Upayavira <uv...@odoko.co.uk>.
Can you confirm that the indexes *are* in place on disk before you shut
down? Whilst it may seem like indexes are being removed, this would seem
extremely unlikely, and the explanation is likely something else.
Try looking at your index files, post some content, commit, look again
at your index files, then shut down. What changes have you seen? Did the
commit 'take' and cause the files to be written?
Upayavira
On Mon, Nov 23, 2015, at 10:25 AM, Oliver Schrenk wrote:
> Yes. Hard commit using SolrJ 4.10.4 client with UpdateResponse.commit()
> which defaults to waitFlush=true and waitSearcher=true
>
> We reverted back to Solr 4.7 for the time being, which is a pity as we
> really benefitted from the concurrent commit fix.
>
> > On 17 Nov 2015, at 18:00, Erick Erickson <er...@gmail.com> wrote:
> >
> > Did you commit after indexing and before shutting down? Even if you didn't, I'm
> > still a bit surprised, but that's one possible explanation.
> >
> > But this is the first time I've seen this problem mentioned...
> >
> > Best,
> > Erick
> >
> > On Tue, Nov 17, 2015 at 4:08 AM, Oliver Schrenk <os...@elmar.nl> wrote:
> >> Hi,
> >>
> >> since we upgraded our cluster from 4.7 to 4.10.4 we are experiencing issues. When shutting down the service (with a confirmed graceful shutdown in the logs), the index is dropped, with only one lonely `segments.gen` file left for each shard and all other files being deleted.
> >>
> >> There is no message in the logs, other than graceful shutdown. Did anybody have a similar issues and has some advice?
> >>
> >> Cheers,
> >> Oliver
>
Re: Sold 4.10.4 dropping index on shutdown
Posted by Oliver Schrenk <os...@elmar.nl>.
Yes. Hard commit using SolrJ 4.10.4 client with UpdateResponse.commit() which defaults to waitFlush=true and waitSearcher=true
We reverted back to Solr 4.7 for the time being, which is a pity as we really benefitted from the concurrent commit fix.
> On 17 Nov 2015, at 18:00, Erick Erickson <er...@gmail.com> wrote:
>
> Did you commit after indexing and before shutting down? Even if you didn't, I'm
> still a bit surprised, but that's one possible explanation.
>
> But this is the first time I've seen this problem mentioned...
>
> Best,
> Erick
>
> On Tue, Nov 17, 2015 at 4:08 AM, Oliver Schrenk <os...@elmar.nl> wrote:
>> Hi,
>>
>> since we upgraded our cluster from 4.7 to 4.10.4 we are experiencing issues. When shutting down the service (with a confirmed graceful shutdown in the logs), the index is dropped, with only one lonely `segments.gen` file left for each shard and all other files being deleted.
>>
>> There is no message in the logs, other than graceful shutdown. Did anybody have a similar issues and has some advice?
>>
>> Cheers,
>> Oliver
Re: Sold 4.10.4 dropping index on shutdown
Posted by Erick Erickson <er...@gmail.com>.
Did you commit after indexing and before shutting down? Even if you didn't, I'm
still a bit surprised, but that's one possible explanation.
But this is the first time I've seen this problem mentioned...
Best,
Erick
On Tue, Nov 17, 2015 at 4:08 AM, Oliver Schrenk <os...@elmar.nl> wrote:
> Hi,
>
> since we upgraded our cluster from 4.7 to 4.10.4 we are experiencing issues. When shutting down the service (with a confirmed graceful shutdown in the logs), the index is dropped, with only one lonely `segments.gen` file left for each shard and all other files being deleted.
>
> There is no message in the logs, other than graceful shutdown. Did anybody have a similar issues and has some advice?
>
> Cheers,
> Oliver