[DISCUSS] fixing the tomcat situation, post 4.1?

[Noa Resare <no...@spotify.com>]

So, when working with the 4.1 packaging issues I have more than once been
intensely frustrated with the current tomcat situation.

To recap: Some cloustack components are set up as web applications and
requires a servlet container as their execution environment. The current
solution to provide such as servlet container is to have the cloudstack
package depend on an Apache Tomcat 6 package provided by the OS
distribution. Cloudstack then ships copies of the configuration files
needed by tomcat to run along with init scripts that bootstraps tomcat in a
way that references the cloudstack shipped artifacts. This setup has many
problems.

# Cloudstack assumes that it's configuration files will be enough in sync
with what the externally provided tomcat expects.

# Lots and lots of fairly esoteric config files included in cloudstack
makes the learning curve steep

# Tomcat setup, as it stands without trying any specific hackery to make it
do other stuff, is a labyrinth of shell scripts

# Things gets even more complicated when cloudstack invokes tomcat provided
scripts.

# tomcat has it's own log files for the early parts of it's startup.

It shouldn't need to be this hard. webapps are standardized for a reason,
and the requirements we have on a servlet execution environments are not
that special (one webapp on /client listening one port, and possibly
another webapp for the aws stuff listening on another port)

As a proof of concept I hacked together the starting points of
a replacement for this whole mess yesterday, a few tens of lines of code
setting up an embedded jetty web container. A few lines of code to parse a
config file, set up logging and spawn an embedded servlet container.

Points of discussion:

# What features do we need for our servlet container? Do I miss
anything obvious?

# Do we have some kind of loyalty towards tomcat, it being one of the main
apache projects after all? 2 minutes of googling seems to indicate that
there are embedded tomcat options as well, but I know jetty better so that
seemed like a better stating point for my proof of concept.

My proof of concept code is available here:
https://github.com/noaresare/incubator-cloudstack/tree/noa/runner

/noa




-- 
Engineering Experience, Infrastructure tribe, Spotify

RE: [DISCUSS] fixing the tomcat situation, post 4.1?

[Alex Huang <Al...@citrix.com>]

Hi Noa,

I like your proposal a lot.  This was exactly our question when we saw how this is was done in waf.  It was obvious to us at the time that waf tried to make cloudstack look like a linux native application by spreading it to /etc, /var, /usr/share etc and took it away from its webapp origins.    I have a few questions/comments.

- Are you sure to get into distros it doesn't have to look like a linux app?  That was why we agreed to it so I want to double check here.
- How would it look in distros if it is a webapp?
- I don't think we have any loyalty to tomcat.   Nor should we.
- What about the system.iso?  Is that packaged in the war too?

Thanks.

--Alex

> -----Original Message-----
> From: Noa Resare [mailto:noa@spotify.com]
> Sent: Sunday, March 3, 2013 5:04 AM
> To: cloudstack-dev@incubator.apache.org
> Subject: [DISCUSS] fixing the tomcat situation, post 4.1?
> 
> So, when working with the 4.1 packaging issues I have more than once been
> intensely frustrated with the current tomcat situation.
> 
> To recap: Some cloustack components are set up as web applications and
> requires a servlet container as their execution environment. The current
> solution to provide such as servlet container is to have the cloudstack
> package depend on an Apache Tomcat 6 package provided by the OS
> distribution. Cloudstack then ships copies of the configuration files needed
> by tomcat to run along with init scripts that bootstraps tomcat in a way that
> references the cloudstack shipped artifacts. This setup has many problems.
> 
> # Cloudstack assumes that it's configuration files will be enough in sync with
> what the externally provided tomcat expects.
> 
> # Lots and lots of fairly esoteric config files included in cloudstack makes the
> learning curve steep
> 
> # Tomcat setup, as it stands without trying any specific hackery to make it do
> other stuff, is a labyrinth of shell scripts
> 
> # Things gets even more complicated when cloudstack invokes tomcat
> provided scripts.
> 
> # tomcat has it's own log files for the early parts of it's startup.
> 
> It shouldn't need to be this hard. webapps are standardized for a reason, and
> the requirements we have on a servlet execution environments are not that
> special (one webapp on /client listening one port, and possibly another
> webapp for the aws stuff listening on another port)
> 
> As a proof of concept I hacked together the starting points of a replacement
> for this whole mess yesterday, a few tens of lines of code setting up an
> embedded jetty web container. A few lines of code to parse a config file, set
> up logging and spawn an embedded servlet container.
> 
> Points of discussion:
> 
> # What features do we need for our servlet container? Do I miss anything
> obvious?
> 
> # Do we have some kind of loyalty towards tomcat, it being one of the main
> apache projects after all? 2 minutes of googling seems to indicate that there
> are embedded tomcat options as well, but I know jetty better so that
> seemed like a better stating point for my proof of concept.
> 
> My proof of concept code is available here:
> https://github.com/noaresare/incubator-cloudstack/tree/noa/runner
> 
> /noa
> 
> 
> 
> 
> --
> Engineering Experience, Infrastructure tribe, Spotify

Re: [DISCUSS] fixing the tomcat situation, post 4.1?

[KAWAI Hiroaki <ka...@stratosphere.co.jp>]

Hi,

 From the side of technology, what we need to run cloudstack management
server is J2EE container (JPA, JTX and so on).
As for J2EE contianer, there's a TomEE in apache project which
is less generally available in common distribution packages.
I think Tomcat is an appropriate option referring the availability.
# But at the same time, I think TomEE would be appropriate because
# cloudstack is also an apache project now.

We would need servlet because we need web UI to take control.

For configuration files, I think whatever container we choose,
those files would be look like as tomcat at last. There would
be no big difference.

(2013/03/03 22:04), Noa Resare wrote:
> So, when working with the 4.1 packaging issues I have more than once been
> intensely frustrated with the current tomcat situation.
>
> To recap: Some cloustack components are set up as web applications and
> requires a servlet container as their execution environment. The current
> solution to provide such as servlet container is to have the cloudstack
> package depend on an Apache Tomcat 6 package provided by the OS
> distribution. Cloudstack then ships copies of the configuration files
> needed by tomcat to run along with init scripts that bootstraps tomcat in a
> way that references the cloudstack shipped artifacts. This setup has many
> problems.
>
> # Cloudstack assumes that it's configuration files will be enough in sync
> with what the externally provided tomcat expects.
>
> # Lots and lots of fairly esoteric config files included in cloudstack
> makes the learning curve steep
>
> # Tomcat setup, as it stands without trying any specific hackery to make it
> do other stuff, is a labyrinth of shell scripts
>
> # Things gets even more complicated when cloudstack invokes tomcat provided
> scripts.
>
> # tomcat has it's own log files for the early parts of it's startup.
>
> It shouldn't need to be this hard. webapps are standardized for a reason,
> and the requirements we have on a servlet execution environments are not
> that special (one webapp on /client listening one port, and possibly
> another webapp for the aws stuff listening on another port)
>
> As a proof of concept I hacked together the starting points of
> a replacement for this whole mess yesterday, a few tens of lines of code
> setting up an embedded jetty web container. A few lines of code to parse a
> config file, set up logging and spawn an embedded servlet container.
>
> Points of discussion:
>
> # What features do we need for our servlet container? Do I miss
> anything obvious?
>
> # Do we have some kind of loyalty towards tomcat, it being one of the main
> apache projects after all? 2 minutes of googling seems to indicate that
> there are embedded tomcat options as well, but I know jetty better so that
> seemed like a better stating point for my proof of concept.
>
> My proof of concept code is available here:
> https://github.com/noaresare/incubator-cloudstack/tree/noa/runner
>
> /noa
>
>
>
>

RE: [DISCUSS] fixing the tomcat situation, post 4.1?

[Hugo Trippaers <HT...@schubergphilis.com>]

> -----Original Message-----
> From: David Nalley [mailto:david@gnsa.us]
> Sent: maandag 4 maart 2013 6:03
> To: cloudstack-dev@incubator.apache.org
> Subject: Re: [DISCUSS] fixing the tomcat situation, post 4.1?
> 
> On Sun, Mar 3, 2013 at 8:04 AM, Noa Resare <no...@spotify.com> wrote:
> > So, when working with the 4.1 packaging issues I have more than once
> > been intensely frustrated with the current tomcat situation.
> >
> > To recap: Some cloustack components are set up as web applications and
> > requires a servlet container as their execution environment. The
> > current solution to provide such as servlet container is to have the
> > cloudstack package depend on an Apache Tomcat 6 package provided by
> > the OS distribution. Cloudstack then ships copies of the configuration
> > files needed by tomcat to run along with init scripts that bootstraps
> > tomcat in a way that references the cloudstack shipped artifacts. This
> > setup has many problems.
> >
> > # Cloudstack assumes that it's configuration files will be enough in
> > sync with what the externally provided tomcat expects.
> >
> > # Lots and lots of fairly esoteric config files included in cloudstack
> > makes the learning curve steep
> >
> > # Tomcat setup, as it stands without trying any specific hackery to
> > make it do other stuff, is a labyrinth of shell scripts
> >
> > # Things gets even more complicated when cloudstack invokes tomcat
> > provided scripts.
> >
> > # tomcat has it's own log files for the early parts of it's startup.
> >
> > It shouldn't need to be this hard. webapps are standardized for a
> > reason, and the requirements we have on a servlet execution
> > environments are not that special (one webapp on /client listening one
> > port, and possibly another webapp for the aws stuff listening on
> > another port)
> >
> > As a proof of concept I hacked together the starting points of a
> > replacement for this whole mess yesterday, a few tens of lines of code
> > setting up an embedded jetty web container. A few lines of code to
> > parse a config file, set up logging and spawn an embedded servlet
> container.
> >
> > Points of discussion:
> >
> > # What features do we need for our servlet container? Do I miss
> > anything obvious?
> >
> > # Do we have some kind of loyalty towards tomcat, it being one of the
> > main apache projects after all? 2 minutes of googling seems to
> > indicate that there are embedded tomcat options as well, but I know
> > jetty better so that seemed like a better stating point for my proof of
> concept.
> >
> > My proof of concept code is available here:
> > https://github.com/noaresare/incubator-cloudstack/tree/noa/runner
> >
> > /noa
> >
> >
> 
> I don't think that we have any loyalties to Tomcat.
> 
> You aren't the first person to voice frustration at how things are deployed.
> I'd be happy if we ended up with a truly deployable war.

The current war should be completely deployable. I tested this with a jetty container (which is also what the development build is using ) and glassfish a while back. If it is not at this point, this is a bug that should be solved.

> 
> Thanks for starting this work.
> 
> --David

Re: [DISCUSS] fixing the tomcat situation, post 4.1?

[David Nalley <da...@gnsa.us>]

On Sun, Mar 3, 2013 at 8:04 AM, Noa Resare <no...@spotify.com> wrote:
> So, when working with the 4.1 packaging issues I have more than once been
> intensely frustrated with the current tomcat situation.
>
> To recap: Some cloustack components are set up as web applications and
> requires a servlet container as their execution environment. The current
> solution to provide such as servlet container is to have the cloudstack
> package depend on an Apache Tomcat 6 package provided by the OS
> distribution. Cloudstack then ships copies of the configuration files
> needed by tomcat to run along with init scripts that bootstraps tomcat in a
> way that references the cloudstack shipped artifacts. This setup has many
> problems.
>
> # Cloudstack assumes that it's configuration files will be enough in sync
> with what the externally provided tomcat expects.
>
> # Lots and lots of fairly esoteric config files included in cloudstack
> makes the learning curve steep
>
> # Tomcat setup, as it stands without trying any specific hackery to make it
> do other stuff, is a labyrinth of shell scripts
>
> # Things gets even more complicated when cloudstack invokes tomcat provided
> scripts.
>
> # tomcat has it's own log files for the early parts of it's startup.
>
> It shouldn't need to be this hard. webapps are standardized for a reason,
> and the requirements we have on a servlet execution environments are not
> that special (one webapp on /client listening one port, and possibly
> another webapp for the aws stuff listening on another port)
>
> As a proof of concept I hacked together the starting points of
> a replacement for this whole mess yesterday, a few tens of lines of code
> setting up an embedded jetty web container. A few lines of code to parse a
> config file, set up logging and spawn an embedded servlet container.
>
> Points of discussion:
>
> # What features do we need for our servlet container? Do I miss
> anything obvious?
>
> # Do we have some kind of loyalty towards tomcat, it being one of the main
> apache projects after all? 2 minutes of googling seems to indicate that
> there are embedded tomcat options as well, but I know jetty better so that
> seemed like a better stating point for my proof of concept.
>
> My proof of concept code is available here:
> https://github.com/noaresare/incubator-cloudstack/tree/noa/runner
>
> /noa
>
>

I don't think that we have any loyalties to Tomcat.

You aren't the first person to voice frustration at how things are deployed.
I'd be happy if we ended up with a truly deployable war.

Thanks for starting this work.

--David