Tomcat 8.x - Servlets Become Available Before Container Fully Init'd

[Nicholas Hagen <ni...@znetdevelopment.com>]

We are updating our applications to Spring Boot using Tomcat Embedded.
Previously we were on the 7.x version running standalone Tomcat with WAR
deployments.  In our application we have two main servlets:  a general
purpose health check  (used in all our services) and the specific web
application.  We set the web application with "load-on-startup" set as 1
and set the health check "load-on-startup" to be 2.  The web applications
take upwards of a minute to start and we want the health checks to not
return until the applications are fully init'd.  This is required so that
our routing tier does not add a member to its pool to early.  Within Tomcat
7.x this seems to work correctly and the health check is not available
until after the init method of the web application servlet has completed.
Jetty appears to behave the same way as well.

In 8.x this behavior seems to have changed, or at least it does with Spring
Boot.  In my debugging into the core Tomcat libraries, it seems to suggest
it's on the Tomcat side.  If I let the application startup with no traffic,
I properly see the web application start its "init" method on the main
application thread.  Once complete it properly starts the health check
servlet's "init" method as well.

However, if I run again and then request the health check during the minute
it takes to launch the web application servlet's "init" method, the health
check will "init" and return its content.  In essence, the web application
is still init'ing on the main application but the health check then inits
on the Tomcat worker thread as if lazy loaded.  So, it seems that in Tomcat
8 the "load-on-startup" order is properly used to start servlets, but if
requested they are lazy loaded regardless of that order.  This causes our
health check to become immediately available even though the web
application is still init'ing.

Looking through the code it seems to start the worker thread pool, then the
acceptor threads and then the servlets with positive "load-on-startup"
values.  However since the protocol handler and workers are running, they
are allowed to take traffic and lazy load.

Is this the expected behavior now with Tomcat?  If so, are there better
recommendations on how to have a general purpose health check wait until
the web applications are fully init'd?

A simple test is with the following two servlets:

public static class ServletA extends HttpServlet {

    public static final Logger LOG = LoggerFactory.getLogger(ServletA.class
);

    protected void doGet(HttpServletRequest req, HttpServletResponse resp)

        throws ServletException, IOException {

        LOG.info("A - GET DATA");

        throw new ServletException("nothing available");

    }

    public void init(ServletConfig config) throws ServletException {

         LOG.info("A - INIT W CONFIG");

         try { Thread.sleep(60000); } catch (Exception e) {
LOG.error("error", e); }

         LOG.info("A - INIT W CONFIG END");

    }

}


public static class ServletB extends HttpServlet {

    public static final Logger LOG = LoggerFactory.getLogger(ServletA.class
);

    protected void doGet(HttpServletRequest req, HttpServletResponse resp)

        throws ServletException, IOException {

        LOG.info("B - GET DATA");

        throw new ServletException("nothing available");

    }

    public void init(ServletConfig config) throws ServletException {

        LOG.info("B - INIT W CONFIG");

        LOG.info("B - INIT W CONFIG END");

    }

}

Set servletA to have load-on-startup set to 1 with url mapping "/test/A"
and servletB to have load-on-startup set to 2 with "/test/B" mapping.  Now
when starting up Servlet A will block init'ing for 60 seconds.  During that
time, hit /test/B and it should init and return ServletB on a worker thread
rather than waiting for servletA to finish init'ing.

Thanks,
Nicholas Hagen


-- 
=================================
Nicholas Hagen
Software Engineer
Twitter:  nicholas_hagen
=================================

Re: Tomcat 8.x - Servlets Become Available Before Container Fully Init'd

[Mark Thomas <ma...@apache.org>]

On 19/08/17 20:02, Nicholas Hagen wrote:

<snip/>

> Is this the expected behavior now with Tomcat?  If so, are there better
> recommendations on how to have a general purpose health check wait until
> the web applications are fully init'd?

I do not see the behaviour you describe with the latest 8.5.x build.
Neither am I aware of anything that has changed with the processing of
load-on-startup that would have changed the behaviour between 7.0.x and 8.x.

If I deploy the test application to a running instance, I get 404s for
ServletA and ServletB while ServletA is processing the init call. This
is as expected. The app deployment is not complete so the requests are
routed to the default web application which returns the 404.

If I perform the test during start-up, the user agent hangs on the
request to ServletB until ServletA init completes, Tomcat starts and
then the request is correctly processed.

This looks to be an issue somewhere between your application and Spring
Boot.

Mark

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org