Anyway to set more max ram to Tomcat 32bit?

[ar...@tutanota.com]

Something wierd seems to be going on... I have an application on tomcat that used to be on a physical windows server 2008 standard 32-bit which I p2v'd with vmware converter over as a VM. I'm upgrading the server to windows server 2008 r2 64bit and instead of doing an inplace, I'm just building another one up and doing parallel side by side, and its a fresh VM from the start, no p2v stuff.... 

The strange part is I was able to set a max memory in the tomcat9w.exe configuration tool to about 1600 Megabytes and that was the hitting of the wall on that setup (the original physical one that I p2v'd to a VM), I can set less ram, but I cannot set it more than 1600 otherwise when I try to start the tomcat service in services.msc it will just crap outs. 

So my thinking was since I have to stay on 32-bit of tomcat and 32-bit of Java JVM anyhow (long story...) that it might be better to put it in a 64-bit OS (server 2008r2) rather than the standard 32bit of server 2008. Long story short, from a fresh VM, on this box I'm only able to set a max total ram in tomcatw as around 1000 m (1GB) and if I set anymore it will not start, as in I cannot get tomcat service to start... so the wall is around 1GB...

As a test, I installed the tomcat (zip extract, not using msi or exe installer) to my own client workstation laptop running the latest windows 10 1809 etc and once again I hit the 1GB wall and cannot even get it to go up to 1.6GB, even though I have like 16GB ram on my laptop machine...

I have googled this for a long time and apparently it has to do with the way the operating system itself (in this case windows, be it windows server 2008 std, 2008 r2, or win10 etc) handles per process memory limitations and thread memory allocations that by default only allow around 1.5 to 2 g of ram per process and that this is what is constraining the tomcat apache jvm and not the programs itself.... 

But if this is indeed an OS difference, I checked on my original 32bit server2008 std, the one that I did P2V, and it didn't have anything special set in the boot.ini or any bcdedit parameters that allow it to see more ram per process nor anything like a /3GB flag etc etc

The only other thing I can think of is that maybe there is a difference between a VM and a physical raw bare metal box in how it allocates ram, and that I can get more contigous memory that tomcat/jvm requires when running on a 32bit OS on a physical box as opposed to and compared to a 64bit OS in a VM?

Why is it I can get the exact same version of Tomcat (bit by bit the same) to handle up to 1600m in 32bit version of server 2008 but on server 2008r2 64bit or even server 2016 64bit I can only get tomcat to do up to around 1GB total max? They are both using the same Java version jvm etc etc etc

Re: Anyway to set more max ram to Tomcat 32bit?

[Christopher Schultz <ch...@christopherschultz.net>]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Shawn,

On 1/4/19 13:07, Shawn Heisey wrote:
> On 1/4/2019 10:44 AM, John Dale wrote:
>> Interesting note: "likely includes memory that Java requires
>> beyond the Java heap size itself"
>> 
>> Can you expound on this?
> 
> In the majority of cases, most of the memory that Java allocates
> from the system is its heap.
> 
> But there are other memory structures required.  One of those is
> the space where Java loads things like classes.  Java is a very
> complicated piece of software that would require a number of
> in-memory structures for various operations.  The one that comes to
> mind most readily is garbage collection.

Also the actual compiled code from all that Java bytecode. It's gotta
go somewhere.

> The OP said the max heap they could get allocated was 1600MB.
> Which suggests that Java is requiring a few hundred megabytes to be
> allocated for its own operation, fitting inside the 2GB max.
> 
> If more memory is needed in the heap, the only option will be to
> switch to 64-bit Java.

If you *absolutely need* to run 32-bit JVM, could you perhaps switch
to another OS? I think that many folks have been able to squeeze a
little more RAM from Linux than they have been able to from Windows.
Maybe it's time to move to Docker :)

- -chris
-----BEGIN PGP SIGNATURE-----
Comment: Using GnuPG with Thunderbird - https://www.enigmail.net/

iQIzBAEBCAAdFiEEMmKgYcQvxMe7tcJcHPApP6U8pFgFAlwxE7AACgkQHPApP6U8
pFi7hBAAwFPtOz0mOAAEhVrzl3tSUNi4rksbymnHkHPb62vzkA9NtGw4cHlAMy2F
5ksbCfT1IbLoJkm5+GZx3PdUWnO7w/jggyATeuoGyTakq7XPJPqwRXNnu1pMgejJ
S4YHHlkeJ2lV0A0nnq154OPrES/8Y4FWLE8G+CgCMuEwGMY+1PYIGMurxPGhS/1l
UU7ZXYZKcJFzeBRSFxeva3fidsBRpO60JU/E+eaKlq8ScZVnvLiqsPsb/3RYHo2F
fWd4KxXUUwUuaemoMLHfMWbNYWY7MGYI/fR9i0jkMFELirv8eHtSDVb1pweffSzh
ZbrSAltB38EhkZkt2wyPptot18vyDstLUnaLj9CQkY+k7Dl/SLmS24AezDFwgT3x
Jb7bS4Mk9mKmw0+RY6GoSsQ7cdOWG23lxIK7CT3ebLaiWat5/sI1K/830uo0Wm4s
2aUnqkMvcrkrUI0XbKjtP2MEGZ4QRsMdIVQ5qW5+2844GONZwMEUJdYkLG3WMuid
SWOH8CNAWli0cw5P/RO9cAeVbgH9L4MFTuCJfwxWdVhQ7ME3nWgl6oUa23IQRF0i
Kem0RoHZPa40ucwKvIHM6ErrIqXCUppYb+HBhNJKWSBQSGtx5IjuQcOQSdjmCGoy
ZRvjnnDIB4Ho3on0SQowIbX5dMFa9eDoAY8pXIDsyQUWmZvT7rw=
=zo7q
-----END PGP SIGNATURE-----

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org

Re: Anyway to set more max ram to Tomcat 32bit?

[Shawn Heisey <ap...@elyograg.org>]

On 1/4/2019 10:44 AM, John Dale wrote:
> Interesting note: "likely includes memory that Java requires beyond
> the Java heap size itself"
>
> Can you expound on this?

In the majority of cases, most of the memory that Java allocates from 
the system is its heap.

But there are other memory structures required.  One of those is the 
space where Java loads things like classes.  Java is a very complicated 
piece of software that would require a number of in-memory structures 
for various operations.  The one that comes to mind most readily is 
garbage collection.

The OP said the max heap they could get allocated was 1600MB. Which 
suggests that Java is requiring a few hundred megabytes to be allocated 
for its own operation, fitting inside the 2GB max.

If more memory is needed in the heap, the only option will be to switch 
to 64-bit Java.

Thanks,
Shawn


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org

Re: Anyway to set more max ram to Tomcat 32bit?

[John Dale <jc...@gmail.com>]

Good answer, but 32 bit may be a reality for some folks.  I agree with
you regarding 64 bit.  Cloud instances are 64 bit.  My workstations
are 64 bit.  It's great.  I did some work with a bank awhile back ..
they had some *ahem* limitations for security reasons and had to use
32 bit on many of their systems.  They were also using *cough*
Internet Explorer 8.  They were happy to be upgrading to IE 9.

Interesting note: "likely includes memory that Java requires beyond
the Java heap size itself"

Can you expound on this?

On 1/4/19, Shawn Heisey <ap...@elyograg.org> wrote:
> On 1/3/2019 6:40 PM, arkboy@tutanota.com wrote:
>> The strange part is I was able to set a max memory in the tomcat9w.exe
>> configuration tool to about 1600 Megabytes and that was the hitting of the
>> wall on that setup (the original physical one that I p2v'd to a VM), I can
>> set less ram, but I cannot set it more than 1600 otherwise when I try to
>> start the tomcat service in services.msc it will just crap outs.
>
> Tomcat itself is not 32 bit or 64 bit.  It is *Java* that determines this.
>
> If you have a 32-bit Java, then that Java cannot address more than 2GB
> of memory, and this likely includes memory that Java requires beyond the
> Java heap size itself.  This is a limitation that can only be overcome
> by running a 64-bit Java.  Naturally this requires a 64-bit operating
> system, which requires a 64-bit CPU.
>
> For the server side, readily available Intel processors have been all
> 64-bit since 2005.  For the client side, readily available Intel
> processors have been all 64-bit since 2006.  If you've purchased a
> computer in the last ten years, it is virtually guaranteed to have
> 64-bit hardware.
>
> I think it was either Java 9 or Java 10 where Oracle stopped making
> 32-bit versions easy to find.
>
> Run a 64-bit Java.  You won't have any memory limits.
>
> Thanks,
> Shawn
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org

Re: Anyway to set more max ram to Tomcat 32bit?

[Christopher Schultz <ch...@christopherschultz.net>]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Shawn,

On 1/4/19 12:20, Shawn Heisey wrote:
> On 1/3/2019 6:40 PM, arkboy@tutanota.com wrote:
>> The strange part is I was able to set a max memory in the
>> tomcat9w.exe configuration tool to about 1600 Megabytes and that
>> was the hitting of the wall on that setup (the original physical
>> one that I p2v'd to a VM), I can set less ram, but I cannot set
>> it more than 1600 otherwise when I try to start the tomcat
>> service in services.msc it will just crap outs.
> 
> Tomcat itself is not 32 bit or 64 bit.  It is *Java* that
> determines this.
> 
> If you have a 32-bit Java, then that Java cannot address more than
> 2GB of memory, and this likely includes memory that Java requires
> beyond the Java heap size itself.

In a 32-bit process space, there are 4GiB worth of addressable memory,
but Windows typically splits that space into two chunks of 2GiB each:
one for the OS's usage and one for the application's usage. That means
that the application can only address/allocate a maximum of 2GiB, and
for Java that means, as you say, both heap and "other stuff".

> This is a limitation that can only be overcome by running a 64-bit 
> Java.  Naturally this requires a 64-bit operating system, which 
> requires a 64-bit CPU.
You can also adjust the OS/app split to be 1GiB/3GiB, but usually it's
not worth it when 64-bit is a possibility.

> For the server side, readily available Intel processors have been
> all 64-bit since 2005.  For the client side, readily available
> Intel processors have been all 64-bit since 2006.  If you've
> purchased a computer in the last ten years, it is virtually
> guaranteed to have 64-bit hardware.

Oddly enough, AMD produced the first x86-64 chips back in 2003 when
Intel was betting on IA64 which is now all but dead. The relationship
between Intel and AMD over the years has been ... interesting.

Sun released the UltraSPARC in 1995, and it was beautiful. And it ran
Java. I'm not sure when a 64-bit JVM was availabile for it, though. I
*think* there was a Windows build for it, but nobody cared.

And never forget the DEC Alpha which was released in 1992, and also a
thing of beauty. I'm not sure exactly when the 64-bit JVM became
available for the Alpha, but it was ... long ago.

64-bit computing has been around for *decades* at this point. OS
vendors are dropping 32-bit support all overt the place. There really
is no reason to stick with 32-bit at this point.

> I think it was either Java 9 or Java 10 where Oracle stopped
> making 32-bit versions easy to find.
> 
> Run a 64-bit Java.  You won't have any memory limits.

+1

You still have the physical limit, but meh. RAM is cheap.

- -chris
-----BEGIN PGP SIGNATURE-----
Comment: Using GnuPG with Thunderbird - https://www.enigmail.net/

iQIzBAEBCAAdFiEEMmKgYcQvxMe7tcJcHPApP6U8pFgFAlwxEyIACgkQHPApP6U8
pFgiohAAhCF5rb5Izqbh5DOmlAhqpWLPwHnw5PuI2Hlgdx57QVitLfXNCnwIu0YW
ptgMP+rcYRjtgmY1xvHqjAlnRX3XXoIVV1IcALgc+85mlXtO32whQ8ciRkOTipvx
x08d6pIPHr4NqQeDoJllU7Mc9Xxm2bc1kY4DYgzSPnXyYYu7rVpKvSSufzylu78x
6MjgfczwwQYJvVQLJ2RdkRn9P7t9K7CZEng0DnBUQfQHeIZzjNumrmegAWZM/iTO
2Vr82DRXNidOjlN7TJ6zt0oUIOWmL2kJ/WC5p2GHdC4xkVJb/0oRD+4lJMSyDoLP
lB5G6uDrD5Zi4JUG1kDOH2Os42s7ph1dUHKZomzCaKWXBT1bkmRVYpUEMYTTH1d6
P8B6XG+S/cTzQ/2a22EW+P2mNUQ7VAOOUJZ0Cd1rMwCUacDWZq0n3uGcTbtp7+2X
Jjpug5xXhURdcWLi7ZWdB739JnmdBBr0RRbE+tPpJ0lIA3eXj/HRsEv0zsF+Mmta
McGWe//x3dHUEiKO4I9+M9uwVcXdwIEoi1WawDiRUtBu1t9WwQcqjr9JMnTQgzc5
yEXZhz2RO9fe4roRAQXuh9bWZD1sqV44V2a1/eHIa+TQ+MGpTpIjF1gfBxkvn/QB
x30xmAFDQn1dYXIE9IMWkjo8h0xWBA8oVQkou/e8zjsa5b3WRcg=
=rfHP
-----END PGP SIGNATURE-----

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org

Re: Anyway to set more max ram to Tomcat 32bit?

[Shawn Heisey <ap...@elyograg.org>]

On 1/3/2019 6:40 PM, arkboy@tutanota.com wrote:
> The strange part is I was able to set a max memory in the tomcat9w.exe configuration tool to about 1600 Megabytes and that was the hitting of the wall on that setup (the original physical one that I p2v'd to a VM), I can set less ram, but I cannot set it more than 1600 otherwise when I try to start the tomcat service in services.msc it will just crap outs.

Tomcat itself is not 32 bit or 64 bit.  It is *Java* that determines this.

If you have a 32-bit Java, then that Java cannot address more than 2GB 
of memory, and this likely includes memory that Java requires beyond the 
Java heap size itself.  This is a limitation that can only be overcome 
by running a 64-bit Java.  Naturally this requires a 64-bit operating 
system, which requires a 64-bit CPU.

For the server side, readily available Intel processors have been all 
64-bit since 2005.  For the client side, readily available Intel 
processors have been all 64-bit since 2006.  If you've purchased a 
computer in the last ten years, it is virtually guaranteed to have 
64-bit hardware.

I think it was either Java 9 or Java 10 where Oracle stopped making 
32-bit versions easy to find.

Run a 64-bit Java.  You won't have any memory limits.

Thanks,
Shawn


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org