running websh as a cgi in apache 1.3

[David McTavish <dm...@sandvine.com>]

hi all,
I'm running into some horrible memory issues when using tcl/websh/apache
1.3 on freebsd.  I'm basically leaking 200MB of memory when iterating
through a list (basically a resultset from an ODBC query) that is 5MB in
size. I've tried upgrading to the 3.6.0b4 build of websh without much
luck. I do know that if I run the same code within a tcl shell, I do not
encounter this memory bloating issue, so I'm fairly confident that the
code is sound.  its similar to the following pseudo-code:

set data [list ]
foreach row $resultset \
{ 
	if {[lindex $row $x] eq "someConstraint"} \
	{
		lappend data $row
	}
}
unset $resultset

so, my question is:
1. how can I configure websh within apache to run in cgi-mode instead of
being integrated with the apache process? and if I do this, what are the
ramifications? I'm concerned that alot of the mime/cookie manipulation I
currently do will be rendered inoperable.
2. is it possible that there is a memory leak within websh? or can this
be explained in some way of how apache + tcl handle their memory
utilization? and if the latter, is this possible to be fixed?

if desired, I can provide a code samples that highlights this problem.

thx,
d.

---------------------------------------------------------------------
To unsubscribe, e-mail: websh-user-unsubscribe@tcl.apache.org
For additional commands, e-mail: websh-user-help@tcl.apache.org

Re: running websh as a cgi in apache 1.3

[Ronnie Brunner <ro...@netcetera.ch>]

Hi David

> I'm running into some horrible memory issues when using tcl/websh/apache
> 1.3 on freebsd.  I'm basically leaking 200MB of memory when iterating
> through a list (basically a resultset from an ODBC query) that is 5MB in
> size. I've tried upgrading to the 3.6.0b4 build of websh without much
> luck. I do know that if I run the same code within a tcl shell, I do not
> encounter this memory bloating issue, so I'm fairly confident that the
> code is sound.  its similar to the following pseudo-code:

If you have a threaded Tcl, Tcl itself will not return any memory
even when it's "free". (That's only passing on second hand info)
But still: Tcl is then at the so called "high water mark", which
means it will use as much memory as it will ever have used.

> set data [list ]
> foreach row $resultset \
> { 
> 	if {[lindex $row $x] eq "someConstraint"} \
> 	{
> 		lappend data $row
> 	}
> }
> unset $resultset

As there is no Websh related code here, I assume that you rally talk
about a Tcl issue, not a Websh memory leak issue. (Note that if $data
is your 200MB in size, as notetd above it can easily be that Tcl will
never return them to the system even when data is unset.)

> so, my question is:
> 1. how can I configure websh within apache to run in cgi-mode instead of
> being integrated with the apache process? and if I do this, what are the
> ramifications? I'm concerned that alot of the mime/cookie manipulation I
> currently do will be rendered inoperable.

I don't think that you should have any problems with your cookies at
all. Since you're using Apache 1.3, which runs in prefork only, I
guess that you don't have anything regarding the session in memory
as you don't know which child will handle a request -> serving
a request using CGI will not change a thing. You can also easily share
the same code base for your CGI script and the rest of your
application(that would be recommended anyway). (BTW: by default, an
intepreter in mod_websh will handle only one request anyway to prevent
session cross-talk) -> If you didn't explicitly make sure you share
interpreters and really thought about consequences, the CGI is not a
problem. On the other hand, if you really did think about all that
stuff, then your code is safe and CGI is not a problem either :-)

I assume that you also have a websh binary, which is jall you
need for the CGI setup (except fot the Tcl library, which you have
anyway, otherwise your mod_websh wouldn't run). Take the typical Tcl
approach for your CGI script:

 #!/bin/sh
 # \
 LD_LIBRARY_PATH=/.../lib; export LD_LIBRARY_PATH;
 # \
 exec /.../bin/websh3.6.0b4 "$0" "$@"

Setup your Apache with a ScriptAlias directive, set the x-flag on your
script and there you go.

In the script you'd just source the common code you also use for your
mod_websh pages...

 source /.../other/code

That's all there is to do.

> 2. is it possible that there is a memory leak within websh? or can this
> be explained in some way of how apache + tcl handle their memory
> utilization? and if the latter, is this possible to be fixed?

I did some extensive testing regarding memory. I'm pretty sure there
are some cases pending, but nothing serious. Currently I don't know of
any leaks, but if anyone can show me one, I'll gladly try to fix it.

Note that there was a significant leak fixed in websh3.6.0b4, but you
already wrote that this didn't help.

If you rally use a threaded Tcl with your Apache 1.3, you could try
using a non-threaded one. This suposedly would change the memory
allocation/freeing from Tcl internal to standard OS allcoation. But
I'm not sure, what the exact setup needs be (depends on OS I
guess). comp.lang.tcl would probably be a better place to ask whether
and how this works. (I don't even know to which versions of Tcl these
assumptions apply...)

> if desired, I can provide a code samples that highlights this problem.

If you can strip it down to a couple of relevant lines it might help,
but only if you still insist it might be a Websh problem and not just
the way Tcl handles memory. ;-)

hth
Ronnie
--
Ronnie Brunner | ronnie.brunner@netcetera.ch
phone +41 (0)44 247 79 79 | fax +41 (0)44 247 70 75
Netcetera AG | 8040 Zürich | Switzerland | http://netcetera.ch

---------------------------------------------------------------------
To unsubscribe, e-mail: websh-user-unsubscribe@tcl.apache.org
For additional commands, e-mail: websh-user-help@tcl.apache.org