Re: Short term roller/blogs suggestion

[Dave <sn...@gmail.com>]

Good ideas for blogs.apache.org. My comments are below.


On Tue, Jul 15, 2014 at 4:45 AM, Rainer Jung <ra...@kippdata.de>
wrote:

> I suggest the following changes to our roller installation in order to
> hunt down the instability problems we observe. I'm ready to do the changes
> and investigations myself but wanted to first check, whether there are any
> objections.
>
> 1) Trivial: Add response duration to Tomcat access log
>
> Add "%D" at end of pattern string in server.xml access log configuration.
>
> 2) Medium: Add GC log
>
> To rule out GC problems (we don't expect them, but I would like to make
> sure), add a couple of startup parameters that write sensible GC data to a
> log.
>
> Note that GC logs have no nice way of rotation. Even the new JVM flags
> allowing GC log rotation will overwrite GC log files when the process
> starts. So the theoretical risk is writing big log files, but since roller
> is restarted relatively frequently, getting rid of the log by removing the
> new startup parameters is easy.
>
> 3) Medium: Add analysis code
>
> I'd like to deploy an additional app inside the roller Tomcat, that tracks
> long running requests and writes our stack information only for the threads
> that have long running requests on them. The app is not open source but my
> own IP, so no licensing issue here. The protocol is written as a rotating
> log file, that can then be investigated.
>
> The app is packaged as a normal webapp (war) but it doesn't actually
> provide a useful web interface. Packaging as a war is just an easy way of
> getting it deployed inside Tomcat.
>
> I have that code being running in important and high-load applications
> since a few years, so i see no risk of negatively ifluencinf stability or
> performance. It is not doing a tracing, it only collects snapshots of
> threads doing long running since in regular time intervals like once per
> minute.
>
> 4) Trivial: Turn off the https connector
>
> The Tomcat https connector is currently configured only partially and
> complains on startup. We don't currently use it, because the ssl endpoint
> is on erebus-ssl which forward to the normal http port.
>
> 5) Medium: Enabling Tomcat Manager Webapp with role manager-jmx
>
> Enabling Tomcat Manager webapp with a user in role manager-jmx with a
> hashed password in tomcat-users.xml.
>
> This gives me the opportunity to collect JMX information on the fly via
> simple lightweight scripts.
>
> 6) More complex: Caching
>
> Since roller has only small content, caching would be technically cheap.
> Unfortunately currently the response cache headers are not nice to help
> caching. So if we want to go down this road, we would have to configura or
> fix roller to set useful caching headers (or overwrite them by Tomcat).
> Plus we would need to setup the cache, being it inside the Apache reverse
> proxy in front of roller, or a separate component.
>
> I'd like to execute quickly on 1-5 and investigate the collected data.
>
> Note that recently the db connection pooling was again turned off, because
> problems were observed.
>


These all sound like good ideas and I'd be happy to help with preparing
builds, deploying changes, etc. Also, there is a new Roller 5.1 release on
the way that fixes the bug that prevented us from using database connection
pooling.

For caching, note Roller has built-in support for caching for pages and
feeds. The cache sizes can be adjusted and it's also possible to plugin
memcached as the caching mechanism.

Thanks,
- Dave