You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Siddhartha Singh Sandhu <sa...@gmail.com> on 2015/10/05 22:52:49 UTC
Pressed optimize and now SOLR is not indexing while optimize is going on
Hi,
I pressed the optimize switch. Wasn't the best decision I made today. The
aftermath of it was that when I tried to index more documents the curl just
waited and waited.
I pinged my SOLR and all is well. I am able to access the admin console
also. I can query the SOLR machine too. But, I cannot insert into the
index. Is this behavior normal?
Regards,
Sid.
Re: Pressed optimize and now SOLR is not indexing while optimize is
going on
Posted by Erick Erickson <er...@gmail.com>.
bq: Would having an SSD make considerable difference in
speed
Almost certainly. Optimize is rarely necessary, especially if
you're indexing relatively constantly so just avoiding that might
do the trick ;).
But reloading shouldn't be taking 10 minutes. Before 5.2, if you
had suggesters configured in your solrconfig.xml (and in some
versions, it was enabled by default), then a _very_ expensive
rebuild would happen whenever the core loaded, you might
want to check that.
see: https://issues.apache.org/jira/browse/SOLR-6845
The other possibility is that you had a very large transaction log
and abnormally shut Solr down (kill -9 for instance) and the tlog
would be replayed. You get a large tlog by indexing a bunch
of stuff and not doing a hard commit, see:
https://lucidworks.com/blog/understanding-transaction-logs-softcommit-and-commit-in-sorlcloud/
Best,
Erick
On Mon, Oct 5, 2015 at 2:26 PM, Siddhartha Singh Sandhu
<sa...@gmail.com> wrote:
> Thank you Eric. I think your explanation was the reason.
>
> Following up on that: Would having an SSD make considerable difference in
> speed?
>
> On Mon, Oct 5, 2015 at 5:18 PM, Siddhartha Singh Sandhu <
> sandhusolr@gmail.com> wrote:
>
>> Scrap the last one. It just took 10 mins to load. I panicked too quick.
>>
>> On Mon, Oct 5, 2015 at 5:15 PM, Siddhartha Singh Sandhu <
>> sandhusolr@gmail.com> wrote:
>>
>>> I restarted my SOLR and it is now not reloading the configuration.
>>> Is my solr index corrupted?
>>>
>>> Sid.
>>>
>>> On Mon, Oct 5, 2015 at 5:09 PM, Erick Erickson <er...@gmail.com>
>>> wrote:
>>>
>>>> You should be able to insert while optimizing. Do be aware that
>>>> optimize will probably require that your disk have free _at least_ as
>>>> much space as the index takes up.
>>>>
>>>> It may just be that the disk is so busy with the optimize (it's mostly
>>>> just writing from one file to another) that it's appearing to be
>>>> slow...
>>>>
>>>> Best,
>>>> Erick
>>>>
>>>> On Mon, Oct 5, 2015 at 1:52 PM, Siddhartha Singh Sandhu
>>>> <sa...@gmail.com> wrote:
>>>> > Hi,
>>>> >
>>>> > I pressed the optimize switch. Wasn't the best decision I made today.
>>>> The
>>>> > aftermath of it was that when I tried to index more documents the curl
>>>> just
>>>> > waited and waited.
>>>> >
>>>> > I pinged my SOLR and all is well. I am able to access the admin console
>>>> > also. I can query the SOLR machine too. But, I cannot insert into the
>>>> > index. Is this behavior normal?
>>>> >
>>>> > Regards,
>>>> >
>>>> > Sid.
>>>>
>>>
>>>
>>
Re: Pressed optimize and now SOLR is not indexing while optimize is going on
Posted by Walter Underwood <wu...@wunderwood.org>.
Unix has a “buffer cache”, often called a file cache. This chapter discusses the Linux buffer cache, which is very similar to other Unix implementations. Essentially, all unused RAM is used to make disk access faster.
http://www.tldp.org/LDP/sag/html/buffer-cache.html <http://www.tldp.org/LDP/sag/html/buffer-cache.html>
wunder
Walter Underwood
wunder@wunderwood.org
http://observer.wunderwood.org/ (my blog)
> On Oct 7, 2015, at 3:40 AM, Toke Eskildsen <te...@statsbiblioteket.dk> wrote:
>
> On Wed, 2015-10-07 at 07:03 -0300, Eric Torti wrote:
>> I'm sorry to diverge this thread a little bit. But could please point me to
>> resources that explain deeply how this process of OS using the non-java
>> memory to cache index data?
>
> http://blog.thetaphi.de/2012/07/use-lucenes-mmapdirectory-on-64bit.html
>
> Shawn Heisey:
>>> Whatever RAM is left over after you give 12GB to Java for Solr will be
>>> used automatically by the operating system to cache index data on the
>>> disk. Solr is completely reliant on that caching for good performance.
>>
>> I'm puzzled as to why the physical memory of solr's host machine is always
>> used up and I think some resources on that would help me understand it.
>
> It is not used up as such: Add "Disk cache" and "Free space" (or
> whatever your monitoring tool calls them) and you will have the amount
> of memory available for new processes. If you start a new and
> memory-hungry process, it will take the memory from the free pool first,
> then from the disk cache.
>
>
> - Toke Eskildsen, State and University Library, Denmark
>
>
Re: Pressed optimize and now SOLR is not indexing while optimize is
going on
Posted by Toke Eskildsen <te...@statsbiblioteket.dk>.
On Wed, 2015-10-07 at 07:03 -0300, Eric Torti wrote:
> I'm sorry to diverge this thread a little bit. But could please point me to
> resources that explain deeply how this process of OS using the non-java
> memory to cache index data?
http://blog.thetaphi.de/2012/07/use-lucenes-mmapdirectory-on-64bit.html
Shawn Heisey:
> > Whatever RAM is left over after you give 12GB to Java for Solr will be
> > used automatically by the operating system to cache index data on the
> > disk. Solr is completely reliant on that caching for good performance.
>
> I'm puzzled as to why the physical memory of solr's host machine is always
> used up and I think some resources on that would help me understand it.
It is not used up as such: Add "Disk cache" and "Free space" (or
whatever your monitoring tool calls them) and you will have the amount
of memory available for new processes. If you start a new and
memory-hungry process, it will take the memory from the free pool first,
then from the disk cache.
- Toke Eskildsen, State and University Library, Denmark
Re: Pressed optimize and now SOLR is not indexing while optimize is
going on
Posted by Siddhartha Singh Sandhu <sa...@gmail.com>.
Thanks from my end too. And thanks for the question Eric that added a lot
to my understanding as well.
Regards.
Sid.
On Wed, Oct 7, 2015 at 10:04 AM, Eric Torti <er...@gmail.com> wrote:
> Cool, Toke and Shawn!
>
> That's exactly what I was looking for. I'll have a look at those resources
> and if something is yet unclear I'll open a thread for it.
>
> Thanks for the information,
>
> Eric
>
> On Wed, Oct 7, 2015 at 10:29 AM, Shawn Heisey <ap...@elyograg.org> wrote:
>
> > On 10/7/2015 4:03 AM, Eric Torti wrote:
> > > I'm sorry to diverge this thread a little bit. But could please point
> me
> > to
> > > resources that explain deeply how this process of OS using the non-java
> > > memory to cache index data?
> > >
> > >> Whatever RAM is left over after you give 12GB to Java for Solr will be
> > >> used automatically by the operating system to cache index data on the
> > >> disk. Solr is completely reliant on that caching for good
> performance.
> > >
> > > I'm puzzled as to why the physical memory of solr's host machine is
> > always
> > > used up and I think some resources on that would help me understand it.
> >
> > Toke's reply is excellent, and describes the situation from Lucene's
> > perspective. Solr is a Lucene program, so the same information applies.
> >
> > Here's more generic information on how the OS uses memory for caching
> > for most programs:
> >
> > https://en.wikipedia.org/wiki/Page_cache
> >
> > Note that some programs, like MySQL and Microsoft Exchange, skip the OS
> > cache and take care of caching internally.
> >
> > Thanks,
> > Shawn
> >
> >
>
Re: Pressed optimize and now SOLR is not indexing while optimize is
going on
Posted by Eric Torti <er...@gmail.com>.
Cool, Toke and Shawn!
That's exactly what I was looking for. I'll have a look at those resources
and if something is yet unclear I'll open a thread for it.
Thanks for the information,
Eric
On Wed, Oct 7, 2015 at 10:29 AM, Shawn Heisey <ap...@elyograg.org> wrote:
> On 10/7/2015 4:03 AM, Eric Torti wrote:
> > I'm sorry to diverge this thread a little bit. But could please point me
> to
> > resources that explain deeply how this process of OS using the non-java
> > memory to cache index data?
> >
> >> Whatever RAM is left over after you give 12GB to Java for Solr will be
> >> used automatically by the operating system to cache index data on the
> >> disk. Solr is completely reliant on that caching for good performance.
> >
> > I'm puzzled as to why the physical memory of solr's host machine is
> always
> > used up and I think some resources on that would help me understand it.
>
> Toke's reply is excellent, and describes the situation from Lucene's
> perspective. Solr is a Lucene program, so the same information applies.
>
> Here's more generic information on how the OS uses memory for caching
> for most programs:
>
> https://en.wikipedia.org/wiki/Page_cache
>
> Note that some programs, like MySQL and Microsoft Exchange, skip the OS
> cache and take care of caching internally.
>
> Thanks,
> Shawn
>
>
Re: Pressed optimize and now SOLR is not indexing while optimize is
going on
Posted by Shawn Heisey <ap...@elyograg.org>.
On 10/7/2015 4:03 AM, Eric Torti wrote:
> I'm sorry to diverge this thread a little bit. But could please point me to
> resources that explain deeply how this process of OS using the non-java
> memory to cache index data?
>
>> Whatever RAM is left over after you give 12GB to Java for Solr will be
>> used automatically by the operating system to cache index data on the
>> disk. Solr is completely reliant on that caching for good performance.
>
> I'm puzzled as to why the physical memory of solr's host machine is always
> used up and I think some resources on that would help me understand it.
Toke's reply is excellent, and describes the situation from Lucene's
perspective. Solr is a Lucene program, so the same information applies.
Here's more generic information on how the OS uses memory for caching
for most programs:
https://en.wikipedia.org/wiki/Page_cache
Note that some programs, like MySQL and Microsoft Exchange, skip the OS
cache and take care of caching internally.
Thanks,
Shawn
Re: Pressed optimize and now SOLR is not indexing while optimize is
going on
Posted by Eric Torti <er...@gmail.com>.
Hello Shawn,
I'm sorry to diverge this thread a little bit. But could please point me to
resources that explain deeply how this process of OS using the non-java
memory to cache index data?
> Whatever RAM is left over after you give 12GB to Java for Solr will be
> used automatically by the operating system to cache index data on the
> disk. Solr is completely reliant on that caching for good performance.
I'm puzzled as to why the physical memory of solr's host machine is always
used up and I think some resources on that would help me understand it.
Thanks
On Tue, Oct 6, 2015 at 5:07 PM, Siddhartha Singh Sandhu <
sandhusolr@gmail.com> wrote:
> Thank you for helping out.
>
> Further inquiry: I am committing records to my solr implementation and they
> are not getting showing up in my search. I am search on the default id.
> Is this related to the fact that I dont have enough memory so my SOLR is
> taking a lot of time to actually making the indexed documents available
> instantly.
>
> I also looked at the solr log when I sent in my curl commit with my
> record(which I can not see in the SOLR instance even after sending it
> repeatedly), but it didn't through an error.
>
> I got this as my response on insertion of that record:
>
> {"responseHeader":{"status":0,"QTime":57}}
>
> Thank you.
>
> Sid.
>
> On Tue, Oct 6, 2015 at 3:21 PM, Shawn Heisey <ap...@elyograg.org> wrote:
>
> > On 10/6/2015 8:18 AM, Siddhartha Singh Sandhu wrote:
> > > A have a few questions about optimize. Is the search index fully
> > searchable
> > > after a commit?
> >
> > If openSearcher is true on the commit, then changes to the index
> > (additions, replacements, deletions) will be visible when the commit
> > completes.
> >
> > > How much time does one have to wait in case of a hard commit for the
> > index
> > > to be available?
> >
> > This is impossible to answer. It will take as long as it takes, and the
> > time will depend on many factors, so it is nearly impossible to
> > predict. The only way to know is to try it ... and the number you get
> > on one test may be very different than what you actually see once the
> > system is in production.
> >
> > > I have an index of 180G. Do I need to hit the optimize on this chunk.
> > This
> > > is a single core. Say I cannot get in a cloud env because of cost but
> > this
> > > is a fairly large
> > > amazon machine where I have given SOLR 12G of memory.
> >
> > Whatever RAM is left over after you give 12GB to Java for Solr will be
> > used automatically by the operating system to cache index data on the
> > disk. Solr is completely reliant on that caching for good performance.
> > A perfectly ideal system for that index and heap size would have 192GB
> > of RAM, which is enough to entirely cache the index. I personally
> > wouldn't expect good performance with less than 96GB. Some systems with
> > a 180GB index and a 12GB heap might be OK with 64GBtotal memory, while
> > others with the same size index will require more.
> >
> >
> >
> https://lucidworks.com/blog/sizing-hardware-in-the-abstract-why-we-dont-have-a-definitive-answer/
> >
> > If the index is on SSD, then RAM is *slightly* less important, and
> > performance usually goes up with SSD ... but an SSD cannot completely
> > replace RAM, because RAM is much faster. With SSD, you can get away
> > with less RAM than you can on a spinning disk system, but depending on a
> > bunch of factors, it may not be a LOT less RAM.
> >
> > https://wiki.apache.org/solr/SolrPerformanceProblems
> >
> > Optimizing the index is almost never necessary with recent versions. In
> > almost all cases optimizing will get you a performance increase, but it
> > comes at a huge cost in terms of resource utilization to DO the
> > optimize. While the optimize is happening performance will likely be
> > worse, possibly a LOT worse. Newer versions of Solr (Lucene) have
> > closed the gap on performance with non-optimized indexes, so it doesn't
> > gain you as much in performance as it did in earlier versions.
> >
> > Thanks,
> > Shawn
> >
> >
>
Re: Pressed optimize and now SOLR is not indexing while optimize is
going on
Posted by Siddhartha Singh Sandhu <sa...@gmail.com>.
Thank you for helping out.
Further inquiry: I am committing records to my solr implementation and they
are not getting showing up in my search. I am search on the default id.
Is this related to the fact that I dont have enough memory so my SOLR is
taking a lot of time to actually making the indexed documents available
instantly.
I also looked at the solr log when I sent in my curl commit with my
record(which I can not see in the SOLR instance even after sending it
repeatedly), but it didn't through an error.
I got this as my response on insertion of that record:
{"responseHeader":{"status":0,"QTime":57}}
Thank you.
Sid.
On Tue, Oct 6, 2015 at 3:21 PM, Shawn Heisey <ap...@elyograg.org> wrote:
> On 10/6/2015 8:18 AM, Siddhartha Singh Sandhu wrote:
> > A have a few questions about optimize. Is the search index fully
> searchable
> > after a commit?
>
> If openSearcher is true on the commit, then changes to the index
> (additions, replacements, deletions) will be visible when the commit
> completes.
>
> > How much time does one have to wait in case of a hard commit for the
> index
> > to be available?
>
> This is impossible to answer. It will take as long as it takes, and the
> time will depend on many factors, so it is nearly impossible to
> predict. The only way to know is to try it ... and the number you get
> on one test may be very different than what you actually see once the
> system is in production.
>
> > I have an index of 180G. Do I need to hit the optimize on this chunk.
> This
> > is a single core. Say I cannot get in a cloud env because of cost but
> this
> > is a fairly large
> > amazon machine where I have given SOLR 12G of memory.
>
> Whatever RAM is left over after you give 12GB to Java for Solr will be
> used automatically by the operating system to cache index data on the
> disk. Solr is completely reliant on that caching for good performance.
> A perfectly ideal system for that index and heap size would have 192GB
> of RAM, which is enough to entirely cache the index. I personally
> wouldn't expect good performance with less than 96GB. Some systems with
> a 180GB index and a 12GB heap might be OK with 64GBtotal memory, while
> others with the same size index will require more.
>
>
> https://lucidworks.com/blog/sizing-hardware-in-the-abstract-why-we-dont-have-a-definitive-answer/
>
> If the index is on SSD, then RAM is *slightly* less important, and
> performance usually goes up with SSD ... but an SSD cannot completely
> replace RAM, because RAM is much faster. With SSD, you can get away
> with less RAM than you can on a spinning disk system, but depending on a
> bunch of factors, it may not be a LOT less RAM.
>
> https://wiki.apache.org/solr/SolrPerformanceProblems
>
> Optimizing the index is almost never necessary with recent versions. In
> almost all cases optimizing will get you a performance increase, but it
> comes at a huge cost in terms of resource utilization to DO the
> optimize. While the optimize is happening performance will likely be
> worse, possibly a LOT worse. Newer versions of Solr (Lucene) have
> closed the gap on performance with non-optimized indexes, so it doesn't
> gain you as much in performance as it did in earlier versions.
>
> Thanks,
> Shawn
>
>
Re: Pressed optimize and now SOLR is not indexing while optimize is
going on
Posted by Shawn Heisey <ap...@elyograg.org>.
On 10/6/2015 8:18 AM, Siddhartha Singh Sandhu wrote:
> A have a few questions about optimize. Is the search index fully searchable
> after a commit?
If openSearcher is true on the commit, then changes to the index
(additions, replacements, deletions) will be visible when the commit
completes.
> How much time does one have to wait in case of a hard commit for the index
> to be available?
This is impossible to answer. It will take as long as it takes, and the
time will depend on many factors, so it is nearly impossible to
predict. The only way to know is to try it ... and the number you get
on one test may be very different than what you actually see once the
system is in production.
> I have an index of 180G. Do I need to hit the optimize on this chunk. This
> is a single core. Say I cannot get in a cloud env because of cost but this
> is a fairly large
> amazon machine where I have given SOLR 12G of memory.
Whatever RAM is left over after you give 12GB to Java for Solr will be
used automatically by the operating system to cache index data on the
disk. Solr is completely reliant on that caching for good performance.
A perfectly ideal system for that index and heap size would have 192GB
of RAM, which is enough to entirely cache the index. I personally
wouldn't expect good performance with less than 96GB. Some systems with
a 180GB index and a 12GB heap might be OK with 64GBtotal memory, while
others with the same size index will require more.
https://lucidworks.com/blog/sizing-hardware-in-the-abstract-why-we-dont-have-a-definitive-answer/
If the index is on SSD, then RAM is *slightly* less important, and
performance usually goes up with SSD ... but an SSD cannot completely
replace RAM, because RAM is much faster. With SSD, you can get away
with less RAM than you can on a spinning disk system, but depending on a
bunch of factors, it may not be a LOT less RAM.
https://wiki.apache.org/solr/SolrPerformanceProblems
Optimizing the index is almost never necessary with recent versions. In
almost all cases optimizing will get you a performance increase, but it
comes at a huge cost in terms of resource utilization to DO the
optimize. While the optimize is happening performance will likely be
worse, possibly a LOT worse. Newer versions of Solr (Lucene) have
closed the gap on performance with non-optimized indexes, so it doesn't
gain you as much in performance as it did in earlier versions.
Thanks,
Shawn
Re: Pressed optimize and now SOLR is not indexing while optimize is
going on
Posted by Siddhartha Singh Sandhu <sa...@gmail.com>.
Nice. Will port it onto an SSD.
A have a few questions about optimize. Is the search index fully searchable
after a commit?
How much time does one have to wait in case of a hard commit for the index
to be available?
I have an index of 180G. Do I need to hit the optimize on this chunk. This
is a single core. Say I cannot get in a cloud env because of cost but this
is a fairly large
amazon machine where I have given SOLR 12G of memory.
In context of my index if I had say 20G more data per month onto it. how
much time before it is fully available for search?
And when should I hit the optimize button?
Thanks
Sid.
On Tue, Oct 6, 2015 at 6:55 AM, Toke Eskildsen <te...@statsbiblioteket.dk>
wrote:
> On Mon, 2015-10-05 at 17:26 -0400, Siddhartha Singh Sandhu wrote:
> > Following up on that: Would having an SSD make considerable difference in
> > speed?
>
> Yes, but only to a point.
>
> The UK Web Archive has done some tests on optimizing indexes on both
> spinning drives and SSDs:
> https://github.com/ukwa/shine/tree/master/python/test-logs
>
> With spinning drives, their machines maxed out on IOWait. With SSD, the
> machine maxed out on CPU. That might sound great, but the problem is
> that optimizing on a single shard is single threaded (at least for Solr
> 4.10.x), so if there is only a single shard on the machine, only 1 CPU
> is running at full tilt. There is always a bottleneck.
>
> What might help is that the SSD (probably) does not get bogged down by
> the process, so it should be much better at handling other requests
> while the optimization is running.
>
> - Toke Eskildsen, State and University Library, Denmark
>
>
>
Re: Pressed optimize and now SOLR is not indexing while optimize is
going on
Posted by Toke Eskildsen <te...@statsbiblioteket.dk>.
On Mon, 2015-10-05 at 17:26 -0400, Siddhartha Singh Sandhu wrote:
> Following up on that: Would having an SSD make considerable difference in
> speed?
Yes, but only to a point.
The UK Web Archive has done some tests on optimizing indexes on both
spinning drives and SSDs:
https://github.com/ukwa/shine/tree/master/python/test-logs
With spinning drives, their machines maxed out on IOWait. With SSD, the
machine maxed out on CPU. That might sound great, but the problem is
that optimizing on a single shard is single threaded (at least for Solr
4.10.x), so if there is only a single shard on the machine, only 1 CPU
is running at full tilt. There is always a bottleneck.
What might help is that the SSD (probably) does not get bogged down by
the process, so it should be much better at handling other requests
while the optimization is running.
- Toke Eskildsen, State and University Library, Denmark
Re: Pressed optimize and now SOLR is not indexing while optimize is
going on
Posted by Siddhartha Singh Sandhu <sa...@gmail.com>.
Thank you Eric. I think your explanation was the reason.
Following up on that: Would having an SSD make considerable difference in
speed?
On Mon, Oct 5, 2015 at 5:18 PM, Siddhartha Singh Sandhu <
sandhusolr@gmail.com> wrote:
> Scrap the last one. It just took 10 mins to load. I panicked too quick.
>
> On Mon, Oct 5, 2015 at 5:15 PM, Siddhartha Singh Sandhu <
> sandhusolr@gmail.com> wrote:
>
>> I restarted my SOLR and it is now not reloading the configuration.
>> Is my solr index corrupted?
>>
>> Sid.
>>
>> On Mon, Oct 5, 2015 at 5:09 PM, Erick Erickson <er...@gmail.com>
>> wrote:
>>
>>> You should be able to insert while optimizing. Do be aware that
>>> optimize will probably require that your disk have free _at least_ as
>>> much space as the index takes up.
>>>
>>> It may just be that the disk is so busy with the optimize (it's mostly
>>> just writing from one file to another) that it's appearing to be
>>> slow...
>>>
>>> Best,
>>> Erick
>>>
>>> On Mon, Oct 5, 2015 at 1:52 PM, Siddhartha Singh Sandhu
>>> <sa...@gmail.com> wrote:
>>> > Hi,
>>> >
>>> > I pressed the optimize switch. Wasn't the best decision I made today.
>>> The
>>> > aftermath of it was that when I tried to index more documents the curl
>>> just
>>> > waited and waited.
>>> >
>>> > I pinged my SOLR and all is well. I am able to access the admin console
>>> > also. I can query the SOLR machine too. But, I cannot insert into the
>>> > index. Is this behavior normal?
>>> >
>>> > Regards,
>>> >
>>> > Sid.
>>>
>>
>>
>
Re: Pressed optimize and now SOLR is not indexing while optimize is
going on
Posted by Siddhartha Singh Sandhu <sa...@gmail.com>.
Scrap the last one. It just took 10 mins to load. I panicked too quick.
On Mon, Oct 5, 2015 at 5:15 PM, Siddhartha Singh Sandhu <
sandhusolr@gmail.com> wrote:
> I restarted my SOLR and it is now not reloading the configuration.
> Is my solr index corrupted?
>
> Sid.
>
> On Mon, Oct 5, 2015 at 5:09 PM, Erick Erickson <er...@gmail.com>
> wrote:
>
>> You should be able to insert while optimizing. Do be aware that
>> optimize will probably require that your disk have free _at least_ as
>> much space as the index takes up.
>>
>> It may just be that the disk is so busy with the optimize (it's mostly
>> just writing from one file to another) that it's appearing to be
>> slow...
>>
>> Best,
>> Erick
>>
>> On Mon, Oct 5, 2015 at 1:52 PM, Siddhartha Singh Sandhu
>> <sa...@gmail.com> wrote:
>> > Hi,
>> >
>> > I pressed the optimize switch. Wasn't the best decision I made today.
>> The
>> > aftermath of it was that when I tried to index more documents the curl
>> just
>> > waited and waited.
>> >
>> > I pinged my SOLR and all is well. I am able to access the admin console
>> > also. I can query the SOLR machine too. But, I cannot insert into the
>> > index. Is this behavior normal?
>> >
>> > Regards,
>> >
>> > Sid.
>>
>
>
Re: Pressed optimize and now SOLR is not indexing while optimize is
going on
Posted by Siddhartha Singh Sandhu <sa...@gmail.com>.
I restarted my SOLR and it is now not reloading the configuration.
Is my solr index corrupted?
Sid.
On Mon, Oct 5, 2015 at 5:09 PM, Erick Erickson <er...@gmail.com>
wrote:
> You should be able to insert while optimizing. Do be aware that
> optimize will probably require that your disk have free _at least_ as
> much space as the index takes up.
>
> It may just be that the disk is so busy with the optimize (it's mostly
> just writing from one file to another) that it's appearing to be
> slow...
>
> Best,
> Erick
>
> On Mon, Oct 5, 2015 at 1:52 PM, Siddhartha Singh Sandhu
> <sa...@gmail.com> wrote:
> > Hi,
> >
> > I pressed the optimize switch. Wasn't the best decision I made today. The
> > aftermath of it was that when I tried to index more documents the curl
> just
> > waited and waited.
> >
> > I pinged my SOLR and all is well. I am able to access the admin console
> > also. I can query the SOLR machine too. But, I cannot insert into the
> > index. Is this behavior normal?
> >
> > Regards,
> >
> > Sid.
>
Re: Pressed optimize and now SOLR is not indexing while optimize is
going on
Posted by Erick Erickson <er...@gmail.com>.
You should be able to insert while optimizing. Do be aware that
optimize will probably require that your disk have free _at least_ as
much space as the index takes up.
It may just be that the disk is so busy with the optimize (it's mostly
just writing from one file to another) that it's appearing to be
slow...
Best,
Erick
On Mon, Oct 5, 2015 at 1:52 PM, Siddhartha Singh Sandhu
<sa...@gmail.com> wrote:
> Hi,
>
> I pressed the optimize switch. Wasn't the best decision I made today. The
> aftermath of it was that when I tried to index more documents the curl just
> waited and waited.
>
> I pinged my SOLR and all is well. I am able to access the admin console
> also. I can query the SOLR machine too. But, I cannot insert into the
> index. Is this behavior normal?
>
> Regards,
>
> Sid.