You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Nitin Solanki <ni...@gmail.com> on 2015/02/02 09:11:54 UTC
Solr ate all Memory and throws -bash: cannot create temp file for
here-document: No space left on device on Server
I have been started *solr* for *long time approx 2 weeks* then I saw that
Solr ate around *22 GB* from *28 GB RAM* of my *Server*.
Please check explanation of error on
http://stackoverflow.com/questions/28273340/solr-ate-all-memory-and-throws-bash-cannot-create-temp-file-for-here-document
Re: Solr ate all Memory and throws -bash: cannot create temp file for
here-document: No space left on device on Server
Posted by Nitin Solanki <ni...@gmail.com>.
Before stopping solr, I checked the space using free -m
free = 5000 approx. out of 28000
After stopping solr, Again checked the space using free -m
Then, it shows - free = 20000 approx of 28000
Sorry for 22GB. It is actually 15GB.
Any help Toke
On Mon, Feb 2, 2015 at 3:29 PM, Toke Eskildsen <te...@statsbiblioteket.dk>
wrote:
> On Mon, 2015-02-02 at 10:36 +0100, Nitin Solanki wrote:
> > Which 20GB, You are asking?
> > Indexed data size is 2.(dot)04GB.
>
> Yes, but at
>
> http://stackoverflow.com/questions/28273340/solr-ate-all-memory-and-throws-bash-cannot-create-temp-file-for-here-document
> you stated that Solr allocated 22GB RAM.
>
> 2GB JVM heap + 2GB MMapDirectory virtual memory allocation is far less
> than 22GB. Maybe someone else can help with that part?
>
> - Toke Eskildsen, State and University Library, Denmark
>
>
>
>
Re: Solr ate all Memory and throws -bash: cannot create temp file
for here-document: No space left on device on Server
Posted by Toke Eskildsen <te...@statsbiblioteket.dk>.
On Mon, 2015-02-02 at 10:36 +0100, Nitin Solanki wrote:
> Which 20GB, You are asking?
> Indexed data size is 2.(dot)04GB.
Yes, but at
http://stackoverflow.com/questions/28273340/solr-ate-all-memory-and-throws-bash-cannot-create-temp-file-for-here-document
you stated that Solr allocated 22GB RAM.
2GB JVM heap + 2GB MMapDirectory virtual memory allocation is far less
than 22GB. Maybe someone else can help with that part?
- Toke Eskildsen, State and University Library, Denmark
Re: Solr ate all Memory and throws -bash: cannot create temp file for
here-document: No space left on device on Server
Posted by Nitin Solanki <ni...@gmail.com>.
Which 20GB, You are asking?
Indexed data size is 2.(dot)04GB.
I checked index data using sudo du -hs
test_azure_mapping/nitin/solr/node*/solr/wikingram_shard*
On Mon, Feb 2, 2015 at 2:54 PM, Toke Eskildsen <te...@statsbiblioteket.dk>
wrote:
> On Mon, 2015-02-02 at 09:59 +0100, Nitin Solanki wrote:
> > Ran this command ulimit -a and found open files (-n) 1024
>
> That explains your error message.
>
> > size of Xmx = 2048MB
> >
> > Total Indexed data = 2.04GB
>
> That was surprising. Solr allocation should not reach 20GB with that
> setup. How did you get the 20GB number?
>
> > I checked my solrconfig.xml but didn't find any value of
> > MMapDirectory. I don't know the default size of
> > MMapDirectory.
>
> MMapDirectory is used per default under Linux, as far as I remember.
> You don't have to change anything.
>
> > How to increase the size of open files?
>
> It is probably best to follow a guide, for example:
>
> http://www.cyberciti.biz/faq/linux-increase-the-maximum-number-of-open-files/
>
> - Toke Eskildsen, State and University Library, Denmark
>
>
>
Re: Solr ate all Memory and throws -bash: cannot create temp file
for here-document: No space left on device on Server
Posted by Toke Eskildsen <te...@statsbiblioteket.dk>.
On Mon, 2015-02-02 at 09:59 +0100, Nitin Solanki wrote:
> Ran this command ulimit -a and found open files (-n) 1024
That explains your error message.
> size of Xmx = 2048MB
>
> Total Indexed data = 2.04GB
That was surprising. Solr allocation should not reach 20GB with that
setup. How did you get the 20GB number?
> I checked my solrconfig.xml but didn't find any value of
> MMapDirectory. I don't know the default size of
> MMapDirectory.
MMapDirectory is used per default under Linux, as far as I remember.
You don't have to change anything.
> How to increase the size of open files?
It is probably best to follow a guide, for example:
http://www.cyberciti.biz/faq/linux-increase-the-maximum-number-of-open-files/
- Toke Eskildsen, State and University Library, Denmark
Re: Solr ate all Memory and throws -bash: cannot create temp file for
here-document: No space left on device on Server
Posted by Nitin Solanki <ni...@gmail.com>.
Ran this command *ulimit -a *and found *open files (-n) 1024*
size of Xmx = 2048MB
Total Indexed data = 2.04GB
I checked my solrconfig.xml but didn't find any value of
*MMapDirectory. *I don't
know the default size of
MMapDirectory.
How to increase the size of open files?
On Mon, Feb 2, 2015 at 2:10 PM, Toke Eskildsen <te...@statsbiblioteket.dk>
wrote:
> On Mon, 2015-02-02 at 09:11 +0100, Nitin Solanki wrote:
> > I have been started *solr* for *long time approx 2 weeks* then I saw that
> > Solr ate around *22 GB* from *28 GB RAM* of my *Server*.
>
> I am guessing your index is about 20GB? Solr's MMapDirectory allocates
> virtual memory equal to the size of the index. If there is enough free
> physical memory, that will be used as a cache.
>
> Sanity check: What is your Xmx?
>
>
> The "No space left on device" error is probably because there is not
> enough free file handles on your system. If it is a Linux box, try
> running 'ulimit -a' and look for "open files". If that number is 1024 or
> lower, you should increase it.
>
>
> - Toke Eskildsen, State and University Library, Denmark
>
>
>
Re: Solr ate all Memory and throws -bash: cannot create temp file
for here-document: No space left on device on Server
Posted by Toke Eskildsen <te...@statsbiblioteket.dk>.
On Mon, 2015-02-02 at 09:11 +0100, Nitin Solanki wrote:
> I have been started *solr* for *long time approx 2 weeks* then I saw that
> Solr ate around *22 GB* from *28 GB RAM* of my *Server*.
I am guessing your index is about 20GB? Solr's MMapDirectory allocates
virtual memory equal to the size of the index. If there is enough free
physical memory, that will be used as a cache.
Sanity check: What is your Xmx?
The "No space left on device" error is probably because there is not
enough free file handles on your system. If it is a Linux box, try
running 'ulimit -a' and look for "open files". If that number is 1024 or
lower, you should increase it.
- Toke Eskildsen, State and University Library, Denmark