You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@tomcat.apache.org by Jean-Frederic <jf...@gmail.com> on 2007/02/22 10:42:12 UTC
snprintf/vsnprintf in mod_jk
Hi,
We have those routines twice (in jk_util.c and jk_status.c).
We are also assuming that the platforms always have snprintf() in
jk_lb_worker.c for example.
In jk_global.h there is:
+++
/* XXXX There is a snprintf() and vsnprintf() in jk_util.c */
/* if those work remove the #define. */
#if defined(NETWARE) || defined(AS400)
#define USE_SPRINTF
#define USE_VSPRINTF
#endif
+++
Could someone check that the snprintf() and snprintf() of jk_util.c are
working on NETWARE and AS400?
Cheers
Jean-Frederic
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
Re: snprintf/vsnprintf in mod_jk
Posted by Mladen Turk <mt...@apache.org>.
Guenter Knauf wrote:
> Hi,
> Therefore I would like to suggest another approach: let us import ap_snprintf.c from Apache 1.3.x into the mod_jk project.
> comments welcome!
>
Just commit if you think it's more portable.
Regards,
Mladen.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
Re: snprintf/vsnprintf in mod_jk
Posted by Guenter Knauf <fu...@apache.org>.
Hi Rainer,
> What's the cycle of the leak:
> - per request
> - per restart
> - ??
per start/stop (restart isnt possible with AP 1.3 on NetWare) - request or no request doesnt matter;
leak is reported after unload...; unfortunately AP 1.3 is build upon old clib, and there I dont have the cool debug features as with new system; but will see if I can further track down when I have some more time for that...
Guenter.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
Re: snprintf/vsnprintf in mod_jk
Posted by Rainer Jung <ra...@kippdata.de>.
What's the cycle of the leak:
- per request
- per restart
- ??
Guenter Knauf wrote:
> Hi,
>> yes, very interesting, it could explain why there is core under i5/OS,
>> this platform is very sensible when memory leak or buffer overflow
>> happens...
> I can add that I currently assume the leak in Apache-1.3/mod_jk.c since I dont see the leak when I build the Apache2 module....
>
> NetWare is also that sensible normally, just in this case it seems that only a buffer is not free'd; its sufficient that I just load / unload to see this leak; and such things are only reported with NetWare when I enable developer option...; otherwise such is silently ignored.
>
> Guenter.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
Re: snprintf/vsnprintf in mod_jk
Posted by Henri Gomez <he...@gmail.com>.
i5/OS is Apache 2.x only, so it's probably not the same problem
2007/4/19, Guenter Knauf <fu...@apache.org>:
> Hi,
> > yes, very interesting, it could explain why there is core under i5/OS,
> > this platform is very sensible when memory leak or buffer overflow
> > happens...
> I can add that I currently assume the leak in Apache-1.3/mod_jk.c since I dont see the leak when I build the Apache2 module....
>
> NetWare is also that sensible normally, just in this case it seems that only a buffer is not free'd; its sufficient that I just load / unload to see this leak; and such things are only reported with NetWare when I enable developer option...; otherwise such is silently ignored.
>
> Guenter.
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: dev-help@tomcat.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
Re: snprintf/vsnprintf in mod_jk
Posted by Guenter Knauf <fu...@apache.org>.
Hi,
> yes, very interesting, it could explain why there is core under i5/OS,
> this platform is very sensible when memory leak or buffer overflow
> happens...
I can add that I currently assume the leak in Apache-1.3/mod_jk.c since I dont see the leak when I build the Apache2 module....
NetWare is also that sensible normally, just in this case it seems that only a buffer is not free'd; its sufficient that I just load / unload to see this leak; and such things are only reported with NetWare when I enable developer option...; otherwise such is silently ignored.
Guenter.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
Re: snprintf/vsnprintf in mod_jk
Posted by Henri Gomez <he...@gmail.com>.
yes, very interesting, it could explain why there is core under i5/OS,
this platform is very sensible when memory leak or buffer overflow
happens...
2007/4/19, Guenter Knauf <fu...@apache.org>:
> Hi,
> > Well the latest mod_jk working on i5/OS v5R3 was 1.2.19. 1.2.20,
> > 1.2.21 and 1.2.22 didn't works and you could see on the dev list the
> > various problems encoutered.
> hmm, interesting....;
> I did just apply my recent sp_snprintf() hack to 1.1.19 and 1.2.20 so that I could now compile it for Apache 1.3.x, and while 1.1.19 loads/unloads fine, with 1.2.20 I see the memory leak of 8280 bytes introduced; that was just a quick test to make sure I didnt introduce it with linking the compiler runtime.
> So there was definitely a memory leak introduced between 1.2.19 and 1.2.20, and its still present in trunk...
>
> Guenter.
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: dev-help@tomcat.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
Re: snprintf/vsnprintf in mod_jk
Posted by Guenter Knauf <fu...@apache.org>.
Hi,
> Well the latest mod_jk working on i5/OS v5R3 was 1.2.19. 1.2.20,
> 1.2.21 and 1.2.22 didn't works and you could see on the dev list the
> various problems encoutered.
hmm, interesting....;
I did just apply my recent sp_snprintf() hack to 1.1.19 and 1.2.20 so that I could now compile it for Apache 1.3.x, and while 1.1.19 loads/unloads fine, with 1.2.20 I see the memory leak of 8280 bytes introduced; that was just a quick test to make sure I didnt introduce it with linking the compiler runtime.
So there was definitely a memory leak introduced between 1.2.19 and 1.2.20, and its still present in trunk...
Guenter.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
Re: snprintf/vsnprintf in mod_jk
Posted by Henri Gomez <he...@gmail.com>.
Well the latest mod_jk working on i5/OS v5R3 was 1.2.19. 1.2.20,
1.2.21 and 1.2.22 didn't works and you could see on the dev list the
various problems encoutered.
With i5/OS v5R4, the module are no more in EBCDIC land but UTF and as
such need a serious code review.
I just asked one IBM representative in Rochester Labs if they could
help us (ASF) in such area. The future of mod_jk on i5/OS depends on
the answer, since it will be very difficult and time consuming to have
a functionnal mod_jk without help from the Labs.
Stay tuned
2007/4/16, Guenter Knauf <fu...@apache.org>:
> Hi Henri,
> >> anybody out there who can test this with AS400?
>
> > I could test on AS400 but mod_jk didn't works on AS400, seems to me a
> > thread/log problem and I can't figure it where
> I've just put the stuff into SVN, perhaps you can check it that builds at least;
> if you see linkage errors then you might have to add ap_spnprintf.c somwhere to the buid process...
>
> does mod_jk 1.2.15 work on AS400? I see currently with this 1.2.23-dev a memory leak of 8280 bytes;
> with 1.2.15 I dont see this; but I've not further tracked down yet where this might be - however its _not_ from the HUGE_BUFFER_SIZE buffer; I did a quick test with this buffer doubled in size, but the leak kept same...
>
> Guenter.
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: dev-help@tomcat.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
Re: snprintf/vsnprintf in mod_jk
Posted by Guenter Knauf <fu...@apache.org>.
Hi Henri,
>> anybody out there who can test this with AS400?
> I could test on AS400 but mod_jk didn't works on AS400, seems to me a
> thread/log problem and I can't figure it where
I've just put the stuff into SVN, perhaps you can check it that builds at least;
if you see linkage errors then you might have to add ap_spnprintf.c somwhere to the buid process...
does mod_jk 1.2.15 work on AS400? I see currently with this 1.2.23-dev a memory leak of 8280 bytes;
with 1.2.15 I dont see this; but I've not further tracked down yet where this might be - however its _not_ from the HUGE_BUFFER_SIZE buffer; I did a quick test with this buffer doubled in size, but the leak kept same...
Guenter.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
Re: snprintf/vsnprintf in mod_jk
Posted by Henri Gomez <he...@gmail.com>.
2007/4/14, Guenter Knauf <fu...@apache.org>:
> Hi,
> > We have those routines twice (in jk_util.c and jk_status.c).
> > We are also assuming that the platforms always have snprintf() in
> > jk_lb_worker.c for example.
> yes, and that was one reason why I couldnt build post-1.2.15 versions for NetWare Apache 1.3.x and Netscape.
>
> > In jk_global.h there is:
> > +++
> > /* XXXX There is a snprintf() and vsnprintf() in jk_util.c */
> > /* if those work remove the #define. */
> > #if defined(NETWARE) || defined(AS400)
> > #define USE_SPRINTF
> > #define USE_VSPRINTF
> > #endif
> > +++
>
> > Could someone check that the snprintf() and snprintf() of jk_util.c are
> > working on NETWARE and AS400?
> I've just checked with these, and they dont work for NetWare because we have no /dev/null support.
> Therefore I would like to suggest another approach: let us import ap_snprintf.c from Apache 1.3.x into the mod_jk project.
> I think this is a good solution; we have there a ready-to-use stable implementation, and every platform which lacks of an own snprintf() / vsnprintf() implementation can then pick this up only when not build with/for Apache; when build with Apache 1.3.x then its even not needed to compile this external ap_snprintf.c because then we can directly import ap_snprintf() / ap_vsnprint() from Apache, and only need the defines in jk_global.h.
>
> I have hacked a new header ap_snprintf.h:
> http://people.apache.org/~fuankg/diffs/ap_snprintf.h
>
> and patched ap_snprintf.c from Apache 1.3.37 to build standalone:
> http://people.apache.org/~fuankg/diffs/ap_snprintf.c.diff
>
> then changed the defines in jk_global.h:
> http://people.apache.org/~fuankg/diffs/jk_global.h.diff
>
> these changes let me again build mod_jk for Apache 1.3 and also the Netscape plugin for NetWare platform.
>
> Also we can then remove the other snprintf() / vsnprintf() functions, and remove the USE_SPRINTF and USE_VSPRINTF hacks completely which is also important because it removes a potential buffer overflow.
>
> anybody out there who can test this with AS400?
I could test on AS400 but mod_jk didn't works on AS400, seems to me a
thread/log problem and I can't figure it where
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
Re: snprintf/vsnprintf in mod_jk
Posted by Henri Gomez <he...@gmail.com>.
> But it will be supported by the Apache community :)
Well ASF should first find an System I (new AS400 name) and play with it
> I'm sure there are many users that use the original
> Apache Httpd on AS400.
Nope, System I users are very conservative, I wonder I'm not the only
one to work on Apache modules on this OS :(
BTW, without support from IBM Rochester Labs to know how to update
mod_jk with new UTF8 support on v5R4 and what differ from original
Apache and IBM implementation, we're stuck in the unholy land without
knowledge of what should be done.
> > The fair solution is to use a Linux partition on AS400 and get back in
> > OSS land :)
> >
>
> +1
A Linux partition on System I works well with Redhat and Suse
specifics distributions ;)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
Re: snprintf/vsnprintf in mod_jk
Posted by Mladen Turk <mt...@apache.org>.
Henri Gomez wrote:
>> Right, but one can compile Apache Httpd from the ASF thought
>> and run it on AS400. I suppose all the EBCDIC we added to
>> mod_jk will allow it compile with standard httpd like before.
>
> On AS400, it will be more than difficult and not supported by IBM.
>
But it will be supported by the Apache community :)
I'm sure there are many users that use the original
Apache Httpd on AS400.
> The fair solution is to use a Linux partition on AS400 and get back in
> OSS land :)
>
+1
Regards,
Mladen.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
Re: snprintf/vsnprintf in mod_jk
Posted by Henri Gomez <he...@gmail.com>.
> Right, but one can compile Apache Httpd from the ASF thought
> and run it on AS400. I suppose all the EBCDIC we added to
> mod_jk will allow it compile with standard httpd like before.
On AS400, it will be more than difficult and not supported by IBM.
The fair solution is to use a Linux partition on AS400 and get back in
OSS land :)
> > So since there is no more IBM support for this kind of OpenSource
> > stuff in there proprietary OS, we may drop our support of mod_jk. I
> > don't have the time to work on v5R4 adaptation and since there is
> > little or no interest in user land, a pragmatic approch is to stop
> > this OS.
> >
>
> Fair enough. A simple note in BUILDING would be OK.
>
> Regards,
> Mladen.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: dev-help@tomcat.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
Re: snprintf/vsnprintf in mod_jk
Posted by Mladen Turk <mt...@apache.org>.
Henri Gomez wrote:
> 2007/4/16, Mladen Turk <mt...@apache.org>:
>>
>> You mean the support for the IBM HTTP Server.
>> I suppose the AS400 is still able to have genuine Apache Httpd
>> and mod_jk compiled and running.
>
> Nope on AS400, there is an IBM HTTP Server (powered by Apache), so
> it's not a genuine Apache 2.x
>
Right, but one can compile Apache Httpd from the ASF thought
and run it on AS400. I suppose all the EBCDIC we added to
mod_jk will allow it compile with standard httpd like before.
> So since there is no more IBM support for this kind of OpenSource
> stuff in there proprietary OS, we may drop our support of mod_jk. I
> don't have the time to work on v5R4 adaptation and since there is
> little or no interest in user land, a pragmatic approch is to stop
> this OS.
>
Fair enough. A simple note in BUILDING would be OK.
Regards,
Mladen.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
Re: snprintf/vsnprintf in mod_jk
Posted by Henri Gomez <he...@gmail.com>.
2007/4/16, Mladen Turk <mt...@apache.org>:
> Henri Gomez wrote:
> > Still on mod_jk and this kind of stuff.
> >
> > We switch our dev servers from i5/OS v5R4 and many changes occurs in
> > IBM HTTP Server powered by Apache, and for instance this one :
> >
> > If I could'nt fix/adapt for i5/OS, we may have to remove the whole
> > AS400 support in next release of mod_jk ;(
> >
>
> You mean the support for the IBM HTTP Server.
> I suppose the AS400 is still able to have genuine Apache Httpd
> and mod_jk compiled and running.
Nope on AS400, there is an IBM HTTP Server (powered by Apache), so
it's not a genuine Apache 2.x
So since there is no more IBM support for this kind of OpenSource
stuff in there proprietary OS, we may drop our support of mod_jk. I
don't have the time to work on v5R4 adaptation and since there is
little or no interest in user land, a pragmatic approch is to stop
this OS.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
Re: snprintf/vsnprintf in mod_jk
Posted by Mladen Turk <mt...@apache.org>.
Henri Gomez wrote:
> Still on mod_jk and this kind of stuff.
>
> We switch our dev servers from i5/OS v5R4 and many changes occurs in
> IBM HTTP Server powered by Apache, and for instance this one :
>
> If I could'nt fix/adapt for i5/OS, we may have to remove the whole
> AS400 support in next release of mod_jk ;(
>
You mean the support for the IBM HTTP Server.
I suppose the AS400 is still able to have genuine Apache Httpd
and mod_jk compiled and running.
Regards,
Mladen.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
Re: snprintf/vsnprintf in mod_jk
Posted by Henri Gomez <he...@gmail.com>.
Still on mod_jk and this kind of stuff.
We switch our dev servers from i5/OS v5R4 and many changes occurs in
IBM HTTP Server powered by Apache, and for instance this one :
As of i5/OS™ V5R4, modules must be recompiled with a UTF locale. This
creates an environment where locale-dependent C runtime functions
assume that string data is encoded in UTF-8. Any hardcoded constants
can be encoded in UTF-8 by adding a #pragma convert(1208) statement in
the module. Additionally, input data from the client will no longer be
converted to EBCDIC but will be passed as-is. Output data sent from
the module is not converted either so it must be encoded in ASCII or
UTF8 as required. APR and HTTP APIs as of V5R4, expect data in UTF-8.
Note that several APIs have additional functions that allow a CCSID to
be set to indicate the encoding of the parameters being passed.
Conversion functions between UTF-8 and EBCDIC have been added. Be sure
to review APIs used by your module to be aware of current changes.
Consequence, there is many works in i5/OS (AS400) implementation of mod_jk.
I hope to get some time to works on these adaptations and help is more
than welcome (did there is other i5/OS users around ?).
If I could'nt fix/adapt for i5/OS, we may have to remove the whole
AS400 support in next release of mod_jk ;(
Regards
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
Re: snprintf/vsnprintf in mod_jk
Posted by Jim Jagielski <ji...@jaguNET.com>.
On Apr 14, 2007, at 9:24 AM, Guenter Knauf wrote:
> I've just checked with these, and they dont work for NetWare
> because we have no /dev/null support.
> Therefore I would like to suggest another approach: let us import
> ap_snprintf.c from Apache 1.3.x into the mod_jk project.
> I think this is a good solution; we have there a ready-to-use
> stable implementation, and every platform which lacks of an own
> snprintf() / vsnprintf() implementation can then pick this up only
> when not build with/for Apache;
+1
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
Re: snprintf/vsnprintf in mod_jk
Posted by Guenter Knauf <fu...@apache.org>.
Hi,
> We have those routines twice (in jk_util.c and jk_status.c).
> We are also assuming that the platforms always have snprintf() in
> jk_lb_worker.c for example.
yes, and that was one reason why I couldnt build post-1.2.15 versions for NetWare Apache 1.3.x and Netscape.
> In jk_global.h there is:
> +++
> /* XXXX There is a snprintf() and vsnprintf() in jk_util.c */
> /* if those work remove the #define. */
> #if defined(NETWARE) || defined(AS400)
> #define USE_SPRINTF
> #define USE_VSPRINTF
> #endif
> +++
> Could someone check that the snprintf() and snprintf() of jk_util.c are
> working on NETWARE and AS400?
I've just checked with these, and they dont work for NetWare because we have no /dev/null support.
Therefore I would like to suggest another approach: let us import ap_snprintf.c from Apache 1.3.x into the mod_jk project.
I think this is a good solution; we have there a ready-to-use stable implementation, and every platform which lacks of an own snprintf() / vsnprintf() implementation can then pick this up only when not build with/for Apache; when build with Apache 1.3.x then its even not needed to compile this external ap_snprintf.c because then we can directly import ap_snprintf() / ap_vsnprint() from Apache, and only need the defines in jk_global.h.
I have hacked a new header ap_snprintf.h:
http://people.apache.org/~fuankg/diffs/ap_snprintf.h
and patched ap_snprintf.c from Apache 1.3.37 to build standalone:
http://people.apache.org/~fuankg/diffs/ap_snprintf.c.diff
then changed the defines in jk_global.h:
http://people.apache.org/~fuankg/diffs/jk_global.h.diff
these changes let me again build mod_jk for Apache 1.3 and also the Netscape plugin for NetWare platform.
Also we can then remove the other snprintf() / vsnprintf() functions, and remove the USE_SPRINTF and USE_VSPRINTF hacks completely which is also important because it removes a potential buffer overflow.
anybody out there who can test this with AS400?
comments welcome!
greets, Guenter,
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org