You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Isart Montane <is...@gmail.com> on 2007/11/09 13:03:01 UTC
2GB limit on 32 bits
Hi all,
i'm experiencing some trouble when i'm trying to lauch solr with more
than 1.6GB. My server is a FC5 with 8GB RAM but when I start solr like this
java -Xmx2000m -jar start.jar
i get the following errors:
Error occurred during initialization of VM
Could not reserve enough space for object heap
Could not create the Java virtual machine.
I've tried to start a virtual machine like this
java -Xmx2000m -version
but i get the same errors.
I've read there's a kernel limitation for a 32 bits architecture of 2Gb
per process, and i just wanna know if anybody knows an alternative to
get a new 64bits server.
Thanks
Isart
Re: 2GB limit on 32 bits
Posted by Isart Montane <is...@gmail.com>.
OK!
i will try to reinstall the SO to 64bits and i will let you know
Thanks!
Norberto Meijome wrote:
> On Fri, 9 Nov 2007 11:58:53 -0300
> "Isart Montane" <is...@gmail.com> wrote:
>
>
>> More info.
>>
>> The kernel is compiled with HIGHMEM64 and PAE
>>
>
> Sorry, i havent dealt with linux kernel options for years.
>
> PAE will give you 36 bits of address. but if the kernel is still limiting the user space to 2 GB / proc, there isn't much PAE will do. Check your OS documentation.
>
> and, let me say it one more time - 64 bit platform. :)
> B
>
> _________________________
> {Beto|Norberto|Numard} Meijome
>
> "All parts should go together without forcing. You must remember that the parts you are reassembling were disassembled by you.
> Therefore, if you can't get them together again, there must be a reason.
> By all means, do not use hammer."
> IBM maintenance manual, 1975
>
> I speak for myself, not my employer. Contents may be hot. Slippery when wet. Reading disclaimers makes you go blind. Writing them is worse. You have been Warned.
>
>
Re: 2GB limit on 32 bits
Posted by Norberto Meijome <fr...@meijome.net>.
On Fri, 9 Nov 2007 11:58:53 -0300
"Isart Montane" <is...@gmail.com> wrote:
> More info.
>
> The kernel is compiled with HIGHMEM64 and PAE
Sorry, i havent dealt with linux kernel options for years.
PAE will give you 36 bits of address. but if the kernel is still limiting the user space to 2 GB / proc, there isn't much PAE will do. Check your OS documentation.
and, let me say it one more time - 64 bit platform. :)
B
_________________________
{Beto|Norberto|Numard} Meijome
"All parts should go together without forcing. You must remember that the parts you are reassembling were disassembled by you.
Therefore, if you can't get them together again, there must be a reason.
By all means, do not use hammer."
IBM maintenance manual, 1975
I speak for myself, not my employer. Contents may be hot. Slippery when wet. Reading disclaimers makes you go blind. Writing them is worse. You have been Warned.
Re: 2GB limit on 32 bits
Posted by Isart Montane <is...@gmail.com>.
More info.
The kernel is compiled with HIGHMEM64 and PAE
On Nov 9, 2007 11:05 AM, Isart Montane <is...@gmail.com> wrote:
> Hi norberto,
>
> i've tried a simple C app to maloc 2GB and it doesn't works (the same with
> 1.5Gb works) so it seems to be a kernel problem.
>
> The server is a FC5 with this uname -a
> Linux XXXXX 2.6.18-1.2239.fc5smp #1 SMP Fri Nov 10 13:22:44 EST 2006 i686
> i686 i386 GNU/Linux
>
> Any ideas how to reach the 4GB?
>
>
> On Nov 9, 2007 10:41 AM, Norberto Meijome <freebsd@meijome.net > wrote:
>
> > On Fri, 9 Nov 2007 10:30:16 -0300
> > "Isart Montane" < isart.montane@gmail.com> wrote:
> >
> > > I've got a dual Xeon. Here you are my cpuinfo. I've read the limit on
> > > a 2.6linux kernel is 4GB on user space and 4GB for kernel... that's
> > > why I asked
> > > if there's any way to reach 4GB per process.
> >
> > ok - i'm obviously too tired - 32 bit should allow you up to 4 GB /
> > proc. If hte kernel doesnt let allow you more than that, that's an issue
> > with your kernel.
> >
> > u need to know first why u cant reach over 2 GB - java limit, OS limit ?
> >
> > i'll sit in a corner very quietly now....
> >
> > _________________________
> > {Beto|Norberto|Numard} Meijome
> >
> > "People demand freedom of speech to make up for the freedom of thought
> > which they avoid. "
> > Soren Aabye Kierkegaard
> >
> > I speak for myself, not my employer. Contents may be hot. Slippery when
> > wet. Reading disclaimers makes you go blind. Writing them is worse. You have
> > been Warned.
> >
>
>
Re: 2GB limit on 32 bits
Posted by Isart Montane <is...@gmail.com>.
Hi norberto,
i've tried a simple C app to maloc 2GB and it doesn't works (the same with
1.5Gb works) so it seems to be a kernel problem.
The server is a FC5 with this uname -a
Linux XXXXX 2.6.18-1.2239.fc5smp #1 SMP Fri Nov 10 13:22:44 EST 2006 i686
i686 i386 GNU/Linux
Any ideas how to reach the 4GB?
On Nov 9, 2007 10:41 AM, Norberto Meijome <fr...@meijome.net> wrote:
> On Fri, 9 Nov 2007 10:30:16 -0300
> "Isart Montane" <is...@gmail.com> wrote:
>
> > I've got a dual Xeon. Here you are my cpuinfo. I've read the limit on
> > a 2.6linux kernel is 4GB on user space and 4GB for kernel... that's
> > why I asked
> > if there's any way to reach 4GB per process.
>
> ok - i'm obviously too tired - 32 bit should allow you up to 4 GB / proc.
> If hte kernel doesnt let allow you more than that, that's an issue with your
> kernel.
>
> u need to know first why u cant reach over 2 GB - java limit, OS limit ?
>
> i'll sit in a corner very quietly now....
>
> _________________________
> {Beto|Norberto|Numard} Meijome
>
> "People demand freedom of speech to make up for the freedom of thought
> which they avoid. "
> Soren Aabye Kierkegaard
>
> I speak for myself, not my employer. Contents may be hot. Slippery when
> wet. Reading disclaimers makes you go blind. Writing them is worse. You have
> been Warned.
>
Re: 2GB limit on 32 bits
Posted by Walter Underwood <wu...@netflix.com>.
Some OSs split that 4GB into a 2GB data space and a 2GB instruction
space. To get a 64bit address space, the CPU, OS, and JVM all need
to support 64 bits. There have been 64 bit Xeon chips since 2004,
the Linux 2.6 kernel supports 64 bit, and recent JVMs do, too.
If your Xeon supports 64 bits, you should be able to get the rest
of it to do 64 bits.
I'm not an expert on configuring that stuff, though.
wunder
On 11/9/07 5:41 AM, "Norberto Meijome" <fr...@meijome.net> wrote:
> On Fri, 9 Nov 2007 10:30:16 -0300
> "Isart Montane" <is...@gmail.com> wrote:
>
>> I've got a dual Xeon. Here you are my cpuinfo. I've read the limit on
>> a 2.6linux kernel is 4GB on user space and 4GB for kernel... that's
>> why I asked
>> if there's any way to reach 4GB per process.
>
> ok - i'm obviously too tired - 32 bit should allow you up to 4 GB / proc. If
> hte kernel doesnt let allow you more than that, that's an issue with your
> kernel.
>
> u need to know first why u cant reach over 2 GB - java limit, OS limit ?
>
> i'll sit in a corner very quietly now....
>
> _________________________
> {Beto|Norberto|Numard} Meijome
>
> "People demand freedom of speech to make up for the freedom of thought which
> they avoid. "
> Soren Aabye Kierkegaard
>
> I speak for myself, not my employer. Contents may be hot. Slippery when wet.
> Reading disclaimers makes you go blind. Writing them is worse. You have been
> Warned.
Re: 2GB limit on 32 bits
Posted by Norberto Meijome <fr...@meijome.net>.
On Fri, 9 Nov 2007 10:30:16 -0300
"Isart Montane" <is...@gmail.com> wrote:
> I've got a dual Xeon. Here you are my cpuinfo. I've read the limit on
> a 2.6linux kernel is 4GB on user space and 4GB for kernel... that's
> why I asked
> if there's any way to reach 4GB per process.
ok - i'm obviously too tired - 32 bit should allow you up to 4 GB / proc. If hte kernel doesnt let allow you more than that, that's an issue with your kernel.
u need to know first why u cant reach over 2 GB - java limit, OS limit ?
i'll sit in a corner very quietly now....
_________________________
{Beto|Norberto|Numard} Meijome
"People demand freedom of speech to make up for the freedom of thought which they avoid. "
Soren Aabye Kierkegaard
I speak for myself, not my employer. Contents may be hot. Slippery when wet. Reading disclaimers makes you go blind. Writing them is worse. You have been Warned.
Re: 2GB limit on 32 bits
Posted by Walter Ferrara <wa...@gmail.com>.
Isn't Xeon5110 64bit? Maybe you could just put a 64 bit OS in you box.
Also, take a look at http://www.spack.org/wiki/LinuxRamLimits
--
Walter
Isart Montane wrote:
> I've got a dual Xeon. Here you are my cpuinfo. I've read the limit on
> a 2.6linux kernel is 4GB on user space and 4GB for kernel... that's
> why I asked
> if there's any way to reach 4GB per process.
>
> Thanks anyway :(
>
> cat /proc/cpuinfo
> processor : 0
> vendor_id : GenuineIntel
> cpu family : 6
> model : 15
> model name : Intel(R) Xeon(R) CPU 5110 @ 1.60GHz
> stepping : 6
> cpu MHz : 1596.192
> cache size : 4096 KB
> physical id : 0
> siblings : 2
> core id : 0
> cpu cores : 2
> fdiv_bug : no
> hlt_bug : no
> f00f_bug : no
> coma_bug : no
> fpu : yes
> fpu_exception : yes
> cpuid level : 10
> wp : yes
> flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca
> cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe nx lm
> constant_tsc pni monitor ds_cpl vmx tm2 cx16 xtpr lahf_lm
> bogomips : 3194.21
>
> processor : 1
> vendor_id : GenuineIntel
> cpu family : 6
> model : 15
> model name : Intel(R) Xeon(R) CPU 5110 @ 1.60GHz
> stepping : 6
> cpu MHz : 1596.192
> cache size : 4096 KB
> physical id : 0
> siblings : 2
> core id : 1
> cpu cores : 2
> fdiv_bug : no
> hlt_bug : no
> f00f_bug : no
> coma_bug : no
> fpu : yes
> fpu_exception : yes
> cpuid level : 10
> wp : yes
> flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca
> cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe nx lm
> constant_tsc pni monitor ds_cpl vmx tm2 cx16 xtpr lahf_lm
> bogomips : 3192.09
>
> processor : 2
> vendor_id : GenuineIntel
> cpu family : 6
> model : 15
> model name : Intel(R) Xeon(R) CPU 5110 @ 1.60GHz
> stepping : 6
> cpu MHz : 1596.192
> cache size : 4096 KB
> physical id : 3
> siblings : 2
> core id : 0
> cpu cores : 2
> fdiv_bug : no
> hlt_bug : no
> f00f_bug : no
> coma_bug : no
> fpu : yes
> fpu_exception : yes
> cpuid level : 10
> wp : yes
> flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca
> cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe nx lm
> constant_tsc pni monitor ds_cpl vmx tm2 cx16 xtpr lahf_lm
> bogomips : 3192.13
>
> processor : 3
> vendor_id : GenuineIntel
> cpu family : 6
> model : 15
> model name : Intel(R) Xeon(R) CPU 5110 @ 1.60GHz
> stepping : 6
> cpu MHz : 1596.192
> cache size : 4096 KB
> physical id : 3
> siblings : 2
> core id : 1
> cpu cores : 2
> fdiv_bug : no
> hlt_bug : no
> f00f_bug : no
> coma_bug : no
> fpu : yes
> fpu_exception : yes
> cpuid level : 10
> wp : yes
> flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca
> cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe nx lm
> constant_tsc pni monitor ds_cpl vmx tm2 cx16 xtpr lahf_lm
> bogomips : 3192.12
>
> On Nov 9, 2007 9:26 AM, Norberto Meijome <fr...@meijome.net> wrote:
>
>
>> On Fri, 9 Nov 2007 09:03:01 -0300
>> "Isart Montane" <is...@gmail.com> wrote:
>>
>>
>>> I've read there's a kernel limitation for a 32 bits architecture of 2Gb
>>> per process, and i just wanna know if anybody knows an alternative to
>>> get a new 64bits server.
>>>
>> You don't say what CPU you have. But the 32 bit limit is real (it's an
>> architecture issue, not a kernel limitation...). You could try running
>> several servers on different ports, each managing part of your index, each
>> up to 2 GB RAM - but you may be pushing your CPU / disks too much and hit
>> other issues - try and see how it goes.
>>
>> If I were you, i'd seriously look into getting a new (64 bit) server .
>> B
>>
>> _________________________
>> {Beto|Norberto|Numard} Meijome
>>
>> "Too bad ignorance isn't painful."
>> Don Lindsay
>>
>> I speak for myself, not my employer. Contents may be hot. Slippery when
>> wet. Reading disclaimers makes you go blind. Writing them is worse. You have
>> been Warned.
>>
>>
>
>
Re: 2GB limit on 32 bits
Posted by Isart Montane <is...@gmail.com>.
I've got a dual Xeon. Here you are my cpuinfo. I've read the limit on
a 2.6linux kernel is 4GB on user space and 4GB for kernel... that's
why I asked
if there's any way to reach 4GB per process.
Thanks anyway :(
cat /proc/cpuinfo
processor : 0
vendor_id : GenuineIntel
cpu family : 6
model : 15
model name : Intel(R) Xeon(R) CPU 5110 @ 1.60GHz
stepping : 6
cpu MHz : 1596.192
cache size : 4096 KB
physical id : 0
siblings : 2
core id : 0
cpu cores : 2
fdiv_bug : no
hlt_bug : no
f00f_bug : no
coma_bug : no
fpu : yes
fpu_exception : yes
cpuid level : 10
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca
cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe nx lm
constant_tsc pni monitor ds_cpl vmx tm2 cx16 xtpr lahf_lm
bogomips : 3194.21
processor : 1
vendor_id : GenuineIntel
cpu family : 6
model : 15
model name : Intel(R) Xeon(R) CPU 5110 @ 1.60GHz
stepping : 6
cpu MHz : 1596.192
cache size : 4096 KB
physical id : 0
siblings : 2
core id : 1
cpu cores : 2
fdiv_bug : no
hlt_bug : no
f00f_bug : no
coma_bug : no
fpu : yes
fpu_exception : yes
cpuid level : 10
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca
cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe nx lm
constant_tsc pni monitor ds_cpl vmx tm2 cx16 xtpr lahf_lm
bogomips : 3192.09
processor : 2
vendor_id : GenuineIntel
cpu family : 6
model : 15
model name : Intel(R) Xeon(R) CPU 5110 @ 1.60GHz
stepping : 6
cpu MHz : 1596.192
cache size : 4096 KB
physical id : 3
siblings : 2
core id : 0
cpu cores : 2
fdiv_bug : no
hlt_bug : no
f00f_bug : no
coma_bug : no
fpu : yes
fpu_exception : yes
cpuid level : 10
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca
cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe nx lm
constant_tsc pni monitor ds_cpl vmx tm2 cx16 xtpr lahf_lm
bogomips : 3192.13
processor : 3
vendor_id : GenuineIntel
cpu family : 6
model : 15
model name : Intel(R) Xeon(R) CPU 5110 @ 1.60GHz
stepping : 6
cpu MHz : 1596.192
cache size : 4096 KB
physical id : 3
siblings : 2
core id : 1
cpu cores : 2
fdiv_bug : no
hlt_bug : no
f00f_bug : no
coma_bug : no
fpu : yes
fpu_exception : yes
cpuid level : 10
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca
cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe nx lm
constant_tsc pni monitor ds_cpl vmx tm2 cx16 xtpr lahf_lm
bogomips : 3192.12
On Nov 9, 2007 9:26 AM, Norberto Meijome <fr...@meijome.net> wrote:
> On Fri, 9 Nov 2007 09:03:01 -0300
> "Isart Montane" <is...@gmail.com> wrote:
>
> > I've read there's a kernel limitation for a 32 bits architecture of 2Gb
> > per process, and i just wanna know if anybody knows an alternative to
> > get a new 64bits server.
>
> You don't say what CPU you have. But the 32 bit limit is real (it's an
> architecture issue, not a kernel limitation...). You could try running
> several servers on different ports, each managing part of your index, each
> up to 2 GB RAM - but you may be pushing your CPU / disks too much and hit
> other issues - try and see how it goes.
>
> If I were you, i'd seriously look into getting a new (64 bit) server .
> B
>
> _________________________
> {Beto|Norberto|Numard} Meijome
>
> "Too bad ignorance isn't painful."
> Don Lindsay
>
> I speak for myself, not my employer. Contents may be hot. Slippery when
> wet. Reading disclaimers makes you go blind. Writing them is worse. You have
> been Warned.
>
Re: 2GB limit on 32 bits
Posted by Norberto Meijome <fr...@meijome.net>.
On Fri, 9 Nov 2007 09:03:01 -0300
"Isart Montane" <is...@gmail.com> wrote:
> I've read there's a kernel limitation for a 32 bits architecture of 2Gb
> per process, and i just wanna know if anybody knows an alternative to
> get a new 64bits server.
You don't say what CPU you have. But the 32 bit limit is real (it's an architecture issue, not a kernel limitation...). You could try running several servers on different ports, each managing part of your index, each up to 2 GB RAM - but you may be pushing your CPU / disks too much and hit other issues - try and see how it goes.
If I were you, i'd seriously look into getting a new (64 bit) server .
B
_________________________
{Beto|Norberto|Numard} Meijome
"Too bad ignorance isn't painful."
Don Lindsay
I speak for myself, not my employer. Contents may be hot. Slippery when wet. Reading disclaimers makes you go blind. Writing them is worse. You have been Warned.