A problem creating Java VM using JNI_CreateJavaVM

[sagi nachum <sa...@gmail.com>]

Hi,
I tried to change to prunsrv.c code so that after it fails creating a JVM
due to heap problem it will try again using a lower -Xmx parameter.
I have a problem doing it since after the first JVM creation failure, it
fails in all subsequent calls, specifically:

I tried to create it with -Xmx1700m and failed on a heap size problem (As
expected), right after that failure I change the -Xmx parameter to -Xmx800
(Which is expected to succeed) and try to create it again - And here I get a
failure again (Just a failure, no indication on heap size problem).
(Needless to say, when I try to create it with -Xmx800m from the first place
it succeeds).

The code change I did is at javajni.c at apxJavaInitialize (I know it looks
a bit hard-coded, its just for demo simplicity...):

if (DYNLOAD_FPTR(JNI_CreateJavaVM)(&(lpJava->lpJvm),
                                           (void **)&(lpJava->lpEnv),
                                           &vmArgs) != JNI_OK)

{
            apxLogWrite(APXLOG_MARK_ERROR "CreateJavaVM Failed");

            wsprintfA(iB[1], "-Xmx800m");
            lpJvmOptions[XMX_OPTION].optionString = iB[1];

            vmArgs.options  = lpJvmOptions;

            if (DYNLOAD_FPTR(JNI_CreateJavaVM)(&(lpJava->lpJvm),
                                                                       (void
**)&(lpJava->lpEnv),

&vmArgs) != JNI_OK)

            {

                       apxLogWrite(APXLOG_MARK_ERROR "CreateJavaVM Failed
Again!!!");

            }

            rv = FALSE;
}

As I said before, the first call is with -Xmx1700 (This is the default
conifguration of the service) and it fails on heap size allocation, but the
second call is with -Xmx800, and it should succeed. The second call also
fails (Not on heap size problem, it simply fails...)
I don't know what the problem is, I suspect it has something to do with
cleanups after the first failure, but I could not get it to work.

Any one have a solution?