You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@tomcat.apache.org by bu...@apache.org on 2004/08/12 17:38:15 UTC
DO NOT REPLY [Bug 30620] New: -
incorrent handling of last_access and elapsed
DO NOT REPLY TO THIS EMAIL, BUT PLEASE POST YOUR BUG
RELATED COMMENTS THROUGH THE WEB INTERFACE AVAILABLE AT
<http://issues.apache.org/bugzilla/show_bug.cgi?id=30620>.
ANY REPLY MADE TO THIS MESSAGE WILL NOT BE COLLECTED AND
INSERTED IN THE BUG DATABASE.
http://issues.apache.org/bugzilla/show_bug.cgi?id=30620
incorrent handling of last_access and elapsed
Summary: incorrent handling of last_access and elapsed
Product: Tomcat 5
Version: 5.0.27
Platform: All
OS/Version: All
Status: NEW
Severity: Normal
Priority: Other
Component: Native:JK
AssignedTo: tomcat-dev@jakarta.apache.org
ReportedBy: rainer.jung@kippdata.de
I have to small bugs and patches to report for mod_jk
jk/native/common/jk_ajp_common.c. They are present at least since 1.2.6 and
still in the CVS version. Both bugs are in method ajp_get_endoint.
1) Initialization of "now"
The variable is only initialized, if cache_timeout>0 or socket_timeout>0 (line
1754). But in lines 1783 and 1784 it is used every time an endpoint with
existing socket connection is found.
2) Format string in jk_log
The variable elapsed is unsigned, but the format string says %d instead of %u
(lines 1771 and 1787)
Please consider the small patch I will attach to this bug report.
---------------------------------------------------------------------
To unsubscribe, e-mail: tomcat-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: tomcat-dev-help@jakarta.apache.org
Re: Enhancement: mod_jk-Logging
Posted by Henri Gomez <hg...@apache.org>.
Rainer Jung wrote:
> I have a proposal for a slight enhancement concerning the log format of
> mod_jk. As suggested by Henri Gomez, I sent the proposal to the
> tomcat-dev list.
>
> 1) Include the log level of a message in the log line. That should be
> easy and is very helpful to find relevant messages. It is a pretty
> standard feature.
>
> 2) Include the PID of the logging process in the log file. That helps a
> lot, because one can also log the PID in Apaches access log. So it is
> easier to relate access log lines and mod_jk log lines to each other.
> Also it helps to unmangle, if log lines from several parallel requests
> are mixed. Of course this argumentation mainly is for apache 1.3, but
> most people use mod_jk for it.
>
> Both enhancements can be done in jk/native/common/jk_util.c.
>
> I attach a small patch, which I also include at the end of the message.
>
> Since I'm not an experienced C developer please
> check. I borrowed a define for Netware for getpid() from Apache's code.
> Not sure If it will work for Netware, or if configure needs to be
> enhanced, if we use getpid(). It build well on Solaris and it is
> done in the same way inside core apache.
>
> The patch is in "diff -Nru"-format and is based on
> jk/native/common/jk_util.c revision 1.28. I hope this is still up to date.
>
> Thank's for considering.
>
> Rainer Jung
>
> --- jk_util.c Tue Jul 13 15:58:10 2004
> +++ jk_util.c.new Thu Aug 12 18:11:50 2004
> @@ -85,6 +85,14 @@
>
> const char * jk_log_fmt = JK_TIME_FORMAT;
>
> +int log_level_max = 3;
> +static const char *log_levels[] = {
> + JK_LOG_DEBUG_VERB,
> + JK_LOG_INFO_VERB,
> + JK_LOG_ERROR_VERB,
> + JK_LOG_EMERG_VERB
> +};
> +
> static void set_time_str(char * str, int len)
> {
> time_t t = time(NULL);
> @@ -250,19 +258,32 @@
> f++;
> }
>
> + const char *log_level;
> + if ( level >=0 && level <= log_level_max ) {
> + log_level=log_levels[level];
> + } else {
> + log_level="unknown";
> + }
Did there is an interest in seeing DEBUG, INFO, ERROR or EMERG in
log file ?
> +#ifdef NETWARE
> +#define getpid() ((pid_t)GetThreadGroupID())
> +#endif
> +
> #ifdef USE_SPRINTF /* until we get a snprintf function */
> #ifdef NETWARE
> buf = (char *) malloc(HUGE_BUFFER_SIZE);
> if (NULL == buf)
> return -1;
> #endif
> - set_time_str(buf, HUGE_BUFFER_SIZE);
> - used = strlen(buf);
> + set_time_str(buf, HUGE_BUFFER_SIZE);
> + used = strlen(buf);
> + used += sprintf(&buf[used], "[%s] [%ld]", log_level,
> (long)getpid());
> if(line)
> used += sprintf(&buf[used], " [%s (%d)]: ", f, line);
> #else
> - set_time_str(buf, HUGE_BUFFER_SIZE);
> - used = strlen(buf);
> + set_time_str(buf, HUGE_BUFFER_SIZE);
> + used = strlen(buf);
> + used += snprintf(&buf[used], HUGE_BUFFER_SIZE, "[%s] [%ld]",
> log_level, (long)getpid());
> if(line)
> used += snprintf(&buf[used], HUGE_BUFFER_SIZE, " [%s
> (%d)]: ", f, line);
> #endif
>
>
>
>
>
> ------------------------------------------------------------------------
>
> --- jk_util.c Tue Jul 13 15:58:10 2004
> +++ jk_util.c.new Thu Aug 12 18:11:50 2004
> @@ -85,6 +85,14 @@
>
> const char * jk_log_fmt = JK_TIME_FORMAT;
>
> +int log_level_max = 3;
> +static const char *log_levels[] = {
> + JK_LOG_DEBUG_VERB,
> + JK_LOG_INFO_VERB,
> + JK_LOG_ERROR_VERB,
> + JK_LOG_EMERG_VERB
> +};
> +
> static void set_time_str(char * str, int len)
> {
> time_t t = time(NULL);
> @@ -250,19 +258,32 @@
> f++;
> }
>
> + const char *log_level;
> + if ( level >=0 && level <= log_level_max ) {
> + log_level=log_levels[level];
> + } else {
> + log_level="unknown";
> + }
> +
> +#ifdef NETWARE
> +#define getpid() ((pid_t)GetThreadGroupID())
> +#endif
> +
> #ifdef USE_SPRINTF /* until we get a snprintf function */
> #ifdef NETWARE
> buf = (char *) malloc(HUGE_BUFFER_SIZE);
> if (NULL == buf)
> return -1;
> #endif
> - set_time_str(buf, HUGE_BUFFER_SIZE);
> - used = strlen(buf);
> + set_time_str(buf, HUGE_BUFFER_SIZE);
> + used = strlen(buf);
> + used += sprintf(&buf[used], "[%s] [%ld]", log_level, (long)getpid());
> if(line)
> used += sprintf(&buf[used], " [%s (%d)]: ", f, line);
> #else
> - set_time_str(buf, HUGE_BUFFER_SIZE);
> - used = strlen(buf);
> + set_time_str(buf, HUGE_BUFFER_SIZE);
> + used = strlen(buf);
> + used += snprintf(&buf[used], HUGE_BUFFER_SIZE, "[%s] [%ld]", log_level, (long)getpid());
> if(line)
> used += snprintf(&buf[used], HUGE_BUFFER_SIZE, " [%s (%d)]: ", f, line);
> #endif
>
Well on multi-thread OS, getpid won't be usefull.
Also I wonder what happen on Windows.
---------------------------------------------------------------------
To unsubscribe, e-mail: tomcat-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: tomcat-dev-help@jakarta.apache.org
Enhancement: mod_jk-Logging
Posted by Rainer Jung <to...@kippdata.de>.
I have a proposal for a slight enhancement concerning the log format of
mod_jk. As suggested by Henri Gomez, I sent the proposal to the
tomcat-dev list.
1) Include the log level of a message in the log line. That should be
easy and is very helpful to find relevant messages. It is a pretty
standard feature.
2) Include the PID of the logging process in the log file. That helps a
lot, because one can also log the PID in Apaches access log. So it is
easier to relate access log lines and mod_jk log lines to each other.
Also it helps to unmangle, if log lines from several parallel requests
are mixed. Of course this argumentation mainly is for apache 1.3, but
most people use mod_jk for it.
Both enhancements can be done in jk/native/common/jk_util.c.
I attach a small patch, which I also include at the end of the message.
Since I'm not an experienced C developer please
check. I borrowed a define for Netware for getpid() from Apache's code.
Not sure If it will work for Netware, or if configure needs to be
enhanced, if we use getpid(). It build well on Solaris and it is
done in the same way inside core apache.
The patch is in "diff -Nru"-format and is based on
jk/native/common/jk_util.c revision 1.28. I hope this is still up to date.
Thank's for considering.
Rainer Jung
--- jk_util.c Tue Jul 13 15:58:10 2004
+++ jk_util.c.new Thu Aug 12 18:11:50 2004
@@ -85,6 +85,14 @@
const char * jk_log_fmt = JK_TIME_FORMAT;
+int log_level_max = 3;
+static const char *log_levels[] = {
+ JK_LOG_DEBUG_VERB,
+ JK_LOG_INFO_VERB,
+ JK_LOG_ERROR_VERB,
+ JK_LOG_EMERG_VERB
+};
+
static void set_time_str(char * str, int len)
{
time_t t = time(NULL);
@@ -250,19 +258,32 @@
f++;
}
+ const char *log_level;
+ if ( level >=0 && level <= log_level_max ) {
+ log_level=log_levels[level];
+ } else {
+ log_level="unknown";
+ }
+
+#ifdef NETWARE
+#define getpid() ((pid_t)GetThreadGroupID())
+#endif
+
#ifdef USE_SPRINTF /* until we get a snprintf function */
#ifdef NETWARE
buf = (char *) malloc(HUGE_BUFFER_SIZE);
if (NULL == buf)
return -1;
#endif
- set_time_str(buf, HUGE_BUFFER_SIZE);
- used = strlen(buf);
+ set_time_str(buf, HUGE_BUFFER_SIZE);
+ used = strlen(buf);
+ used += sprintf(&buf[used], "[%s] [%ld]", log_level,
(long)getpid());
if(line)
used += sprintf(&buf[used], " [%s (%d)]: ", f, line);
#else
- set_time_str(buf, HUGE_BUFFER_SIZE);
- used = strlen(buf);
+ set_time_str(buf, HUGE_BUFFER_SIZE);
+ used = strlen(buf);
+ used += snprintf(&buf[used], HUGE_BUFFER_SIZE, "[%s] [%ld]",
log_level, (long)getpid());
if(line)
used += snprintf(&buf[used], HUGE_BUFFER_SIZE, " [%s
(%d)]: ", f, line);
#endif
mod_jk enhancement discussion
Posted by Rainer Jung <to...@kippdata.de>.
Hi Mladen,
great! I don't know how hard the following is to achieve, but it is the
most prominent problem around mod_jk-Logging I know of:
mod_jk does neither support Apaches rotatelogs nor does it have a log
rotation capability on its own. If you or any other committer has an
idea how to introduce rotating logfiles, that would be a major achievement.
I have 3 more features I would be willing to provide. Maybe you or the
other committers can comment, if they think that would be useful. All of
them are already implemented for mod_jk/apache 1.3 and in production
under high load.
1) Limiting new application sessions if load is to high.
You can configure a start URL for your application (regexp based). If a
request X matches this URL, then mod_jk checks the scoreboard how many
apache requests are being processed simultaneously at that moment. There
is a configurable limti, and if that limit is reached, the request X
will not be sent to tomcat, but instead be answered by some configurable
"local" static response (containing the info, that the load is to high
and the user should try again later). Alternatively one can configure X
to be answered by some external redirect.
2) Multi-Cluster-Routing
Tomcat-Cluster does only allow session replication to all nodes in the
cluster. Once you work with more than 3-4 nodes there is too much
overhead and risk in replicating sessions to all nodes. We split all
nodes into clustered groups. I introduced a new worker attribute
"domain" to let get_most_suitable_worker know, to which other nodes the
session gets replicated (the workers in the same domain).
This enhancement has another benefit: Once mod_jk connects an apache
process to a tomcat instance, the tomcat jk connector will need one
thread for this connection as long as the apache process is alive.
Now assume you have multiple Apaches and Tomcats. The Tomcats are
clustered and mod_jk uses sticky sessions. Now you are going to shut
down (maintenance) one tomcat. All Apache will start connections to all
tomcats. You end up with all tomcats getting connections from all apache
processes, so the number of threads needed inside the tomcats will explode.
If you group the tomcats the connections normally will stay inside the
groups (combine 2) with a routing load balancer to the apaches).
3) Idle connection disconnect
This one also comes from the fact, that a mod_jk connection consumes one
tomcat connector thread, even if its idle for a long time. The idle
connection check in mod_jk only works, if the connection is going to be
used. Then it is checked if it was idle for too long and probably is
beeing closed and reconnected.
I implemented a feature, where in configurable intervals all workers of
an lb worker are checked for idle connections, even the ones which are
not returned by get_most_suitable_worker. The rationale behind it is the
same as in the second part of 2).
4) Open Problem
I didn't check your new code, but at least before there was the problem,
that a recovered worker that was offline a long time (in means of load)
got all the work after recovery. Of course there was a fixed conatsnt of
load added to it after recovery to compensate, but working with a
constant was not really solving the problem.
It looks like one should virtually add load to a worker, even during
it's error time, so that it will get normal load after recovering.
Thanks for any comments!
Rainer
Mladen Turk wrote:
> Rainer Jung wrote:
>
>> Hi Mladen, hi everyone,
>>
>> i have a proposal for a slight enhancement concerning the log format of
>> mod_jk. Maybe you could consider including it in your recently revised
>> code. It contains only changes to jk_util.c and I attach a patch
>> relativ to version 1.32 of the file.
>>
>
> Hi Rainer.
>
> You are definitely on my track :).
>
> I'm planning to add one additional log level, and clear all the log
> messages.
>
> 1. ERROR - fatal operations
> 2. WARN - problems but not fatal
> 3. INFO - production logging
> 4. DEBUG - protocol tracing
> 5. TRACE - function call logging
>
>> 1) Include the log level of a message in the log line.
>> 2) Include the PID of the logging process in the log file.
>
>
> I'd like to make a customizable log line, in the JkRequestLogFormat
> fashion to be able to add PID/ThreadId, level, etc...
>
> MT.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: tomcat-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: tomcat-dev-help@jakarta.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: tomcat-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: tomcat-dev-help@jakarta.apache.org
Re: Enhancement: mod_jk-Logging
Posted by Mladen Turk <mt...@apache.org>.
Rainer Jung wrote:
> Hi Mladen, hi everyone,
>
> i have a proposal for a slight enhancement concerning the log format of
> mod_jk. Maybe you could consider including it in your recently revised
> code. It contains only changes to jk_util.c and I attach a patch relativ
> to version 1.32 of the file.
>
Hi Rainer.
You are definitely on my track :).
I'm planning to add one additional log level, and clear all the log
messages.
1. ERROR - fatal operations
2. WARN - problems but not fatal
3. INFO - production logging
4. DEBUG - protocol tracing
5. TRACE - function call logging
> 1) Include the log level of a message in the log line.
> 2) Include the PID of the logging process in the log file.
I'd like to make a customizable log line, in the JkRequestLogFormat
fashion to be able to add PID/ThreadId, level, etc...
MT.
---------------------------------------------------------------------
To unsubscribe, e-mail: tomcat-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: tomcat-dev-help@jakarta.apache.org
Enhancement: mod_jk-Logging
Posted by Rainer Jung <to...@kippdata.de>.
Hi Mladen, hi everyone,
i have a proposal for a slight enhancement concerning the log format of
mod_jk. Maybe you could consider including it in your recently revised
code. It contains only changes to jk_util.c and I attach a patch relativ
to version 1.32 of the file.
1) Include the log level of a message in the log line. That should be
easy and is very helpful to find relevant messages. It is pretty
standard and much more safe than relying on the fact, that most error
messages actually contain the word error in the message text.
2) Include the PID of the logging process in the log file. That helps a
lot at least for Apache 1.3, because one can also log the PID in Apaches
access log. So it is easier to relate request info from access log lines
and errors from mod_jk log lines to each other. Also it helps to
unmangle, if log lines from several parallel requests are mixed inside
mod_jk-log.
The patch is small, but since I'm not a very experienced C developer
please check. I borrowed a define for Netware for getpid() from Apache's
code. Not sure if it will work for Netware, or if configure needs to be
enhanced, if we use getpid(). It builds well on Solaris.
Thank's for considering.
Enhancement: mod_jk-Logging
Posted by Rainer Jung <to...@kippdata.de>.
I have a proposal for a slight enhancement concerning the log format of
mod_jk.
1) Include the log level of a message in the log line. That should be
easy and is very helpful to find relevant messages. It is pretty standard.
2) Include the PID of the logging process in the log file. That helps a
lot, because one can also log the PID in Apaches access log. So it is
easier to relate access log lines and mod_jk log lines to each other.
Also it helps to unmangle, if log lines from several parallel requests
are mixed. Of course this argumentation mainly is for apache 1.3, but
most people use mod_jk for it.
Both enhancements can be done in jk_util.c.
I attach a small patch. Since I'm not an experienced C developer please
check. I borrowed a define for Netware for getpid() from Apache's code.
Not sure If it will work for Netware, or if configure needs to be
enhanced, if we use getpid(). It build well on Solaris.
Thank's for considering.