You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@tomcat.apache.org by Jesse Long <jp...@unknown.za.net> on 2009/10/12 13:01:44 UTC
Persistent Storage for Webapps
Hi,
In my webapp I need persistent storage. I am building my webapp as a
.war file, and copying it into $CATALINE_HOME/webapps/
Tomcat extracts $CATALINA_HOME/webapps/app.war to
$CATALINA_HOME/webapps/app/
The war file contains an empty directory /WEB-INF/data . During
operations, data files are created under /WEB-INF/data/
Today I created a new release, new.war and upgraded like: cat
/tmp/new.war >$CATALINA_HOME/webapps/app.war
app.war was re-extracted, and all of my data under /WEB-INF/data/
disappeared.
Question: id /WEB-INF/ in the extracted directory the correct place for
persistent data storage? If not, where is?
My goal is to find a consistent way of creating persistent data relative
to the context. I dont want to have to configure context parameters with
data directory paths for each installation.
I saw some code referencing the temp directory obtained from
ServletContext.getAttribute("javax.servlet.context.tempdir"), but I also
saw that this data is only guaranteed to last while the app is running.
Where can I find documentation of this behavior? grepping the servlet
spec returns no matches for "tempdir".
I'd appreciate any pointers.
Thanks,
Jesse
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Persistent Storage for Webapps
Posted by Tobias Crefeld <tc...@cataneo.eu>.
Am Mon, 12 Oct 2009 18:57:43 +0200
schrieb Jesse Long <jp...@unknown.za.net>:
> Thanks for your answers Tobias. Unfortunately we have a very real use
> case for multiple instances on one host, with different data paths.
> So, /etc/app.properties is not usable, but /etc/app-<context path
> derived
> data>.properties might be but virtual hosting would break. No way
> data>to
> identify unique host consistently.
It depends on how you implement virtual hosting.
I have no experience with Tomcats virtual hosting but a
Tomcat-application can always can request the hostname from the URI and
use it to build a path name. We are using this hostname to give users
via VPN other privileges than users via public internet. Not really
secure but within our business it's enough.
We're using Apache2-vhosts and run one Tomcat (in its own JVM) per
vhost. This wastes RAM but avoids that a
crash|deadlock|overload|whatever in one vhost affects the other vhosts
seriously.
In our scenario we have one "home"-directory per vhost
(e.g. /srv/vhost1) where we install Tomcat (mostly by symbolic links to
unpacked Tomcats-archives) incl. startup/shutdown-scripts, cronjobs,
logs, Apache2-document-root, Tomcat-ROOT and persistent data for
import/export. Tomcat's webapps-concept is only used to deploy some
standard applications like Lambda probe, Webservice or j4p.
This "vhost-home" also contains some *.properties-files with
parameters, e.g. for database-access (we don't use JNDI due to historic
reasons), paths for import/export, etc.
If your using only one Tomcat with several name-based vhosts it
probably will be enough if you read hostname from URI and use this
information to construct the path to a tree
with /srv/vhost1, /srv/vhost2, etc. as data directories.
Gruß,
Tobias.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Persistent Storage for Webapps
Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Jesse,
On 10/12/2009 12:57 PM, Jesse Long wrote:
> Unfortunately we have a very real use case for multiple instances on
> one host, with different data paths. So, /etc/app.properties is not
> usable, but /etc/app-<context path derived data>.properties might be
> but virtual hosting would break. No way to identify unique host
> consistently.
I think that the previous suggestion of using JNDI to resolve either the
path to a configuration file, or simply to determine the configuration
itself (that is, have config values directly available through JNDI) is
probably the right way to go: they can be specified by the server
administrator, are context-specific, and do not require the developer to
worry about them at all (except to read their values and act accordingly).
> Also, I could use a path name derived from the temp directory name,
> but nothing says the temp directory name will be consistent. For now
> I will reluctantly use context parameters.
Forget about the temp directory: the servlet spec is basically
preventing that from being useful to you. Also, as someone pointed out,
'temporary' and 'persistent' are usually regarded as antonyms.
- -chris
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iEYEARECAAYFAkrTbZkACgkQ9CaO5/Lv0PD9LwCfXjMj4Gg9pq3lZRkBpu2gHAfu
IZ4An0pV3erpLLiLaKDer+yb7muMwOc3
=T9JR
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Persistent Storage for Webapps
Posted by David Fisher <df...@jmlafferty.com>.
> So you have multiple contexts on a host, each of which needs separate,
> persistent storage?
>
> If you don't want to keep track of a property files, you could write
> a bean
> with a method that takes a HttpRequest as a argument, and returns a
> file
> path based on the info in the request. You would have to dedicate a
> directory on your server as the root for the persistence system.
> Then build
> subdirectories based on info available in the request object.
>
> I could go on if you want, but basically I'm trying to say there is
> more
> than enough info in each HttpRequest to allow you build a flexible,
> scalable
> persistence system without having to resort to .properties files which
> you'll then have to squirrel away somewhere and remember to update
> each time
> you change your Context name, etc. Just look at the javadocs for
> HttpRequest. See especially getContextPath() and getLocalName().
Yes, these are good points about managing persistent data.
The OP's original issue with updating his WAR and deleting the data
directory - followed by trying to use temporary points out a
misconception about persistence that system administrators live to
solve. Persistent data has no business being in the webapps.
Webapps need to find the correct data. The system admin needs to
backup the data and perhaps restore it.
If Tomcat server A dies then does the service need to reappear on
Tomcat server B where is the data? This suggests that you should have
your data on a network mount point.
What if your webapps is insanely sucessful and you need to put your
server in the cloud? Where are you going to find your user's data? Did
you hear what happened to T-Mobile and the Sidekick - they just lost
most every client's contacts stored in their "cloud"
If you are stubborn and I've been myself with virtual host config on
the Tomcat 3.3 model, you will later spend dear effort programming
yourself out of the box. In my case we are nearly ready to go to
Tomcat 6 and a single Webapps completely in a single WAR able to run
multiple hosts (not SSL). Network storage and file-locking are
required, you can never assume you are the only one touching a file.
But now we are off topic, managing persistent data can take numerous
forms - see Apache Lucene, Hadoop, CouchDB, ... and then there are the
SQL...
Regards,
Dave
>
>
>
> On Mon, Oct 12, 2009 at 12:57 PM, Jesse Long <jp...@unknown.za.net>
> wrote:
>
>> Tobias Crefeld wrote:
>>
>>> Am Mon, 12 Oct 2009 13:01:44 +0200
>>> schrieb Jesse Long <jp...@unknown.za.net>:
>>>
>>>
>>>
>>>> In my webapp I need persistent storage. I am building my webapp
>>>> as a
>>>> .war file, and copying it into $CATALINE_HOME/webapps/
>>>>
>>>> Tomcat extracts $CATALINA_HOME/webapps/app.war to
>>>> $CATALINA_HOME/webapps/app/
>>>>
>>>> The war file contains an empty directory /WEB-INF/data . During
>>>> operations, data files are created under /WEB-INF/data/
>>>>
>>>>
>>>
>>> WEB-INF should contain only static files which shouldn't be
>>> accessible
>>> by Webclients (directly), e.g. by /app/data/filename . So it seems
>>> to
>>> be a bad place
>>>
>>>
>>>
>>>
>>>> Question: id /WEB-INF/ in the extracted directory the correct place
>>>> for persistent data storage? If not, where is?
>>>>
>>>>
>>>
>>> AFAIK there is no general advise where to put dynamic, persistent
>>> data.
>>> As system administrator I prefer solutions that give me the
>>> opportunity
>>> to decide myself where to put dynamic data. There are a lot of
>>> reasons
>>> for this: securing against disk space overflow, backup, priviledge
>>> separation, etc.
>>>
>>> Following this way you might create a "properties-"file outside the
>>> Tomcat-hierarchy at a sensible place (e.g. /etc/app.properties) with
>>> the paths your application should use.
>>> Another way is to define a Tomcat-parameter during JVM-startup
>>> that will
>>> contain the path to your data-directory. As this might not survive
>>> the
>>> next update of Tomcat itself I don't like this kind of configuration
>>> but some applications and classes are working this way.
>>>
>>>
>>>
>>>
>>>> My goal is to find a consistent way of creating persistent data
>>>> relative to the context. I dont want to have to configure context
>>>> parameters with data directory paths for each installation.
>>>>
>>>>
>>>
>>> untested idea: If you have no control over the filesystem of the
>>> target Tomcat you could create a mini-application named data.war
>>> that
>>> you are deploying. Of course there is always the danger of
>>> accidentally
>>> erasing the data by re-deploying this pseudo-application. And you
>>> will
>>> have to find a way by valves, etc. to preserve this path from being
>>> accessed by web-clients. Maybe it is enough if you leave out the
>>> context-entry.
>>>
>>>
>> Thanks for your answers Tobias. Unfortunately we have a very real
>> use case
>> for multiple instances on one host, with different data paths. So,
>> /etc/app.properties is not usable, but /etc/app-<context path derived
>> data>.properties might be but virtual hosting would break. No way to
>> identify unique host consistently.
>>
>> Also, I could use a path name derived from the temp directory name,
>> but
>> nothing says the temp directory name will be consistent. For now I
>> will
>> reluctantly use context parameters.
>>
>> Thanks,
>> Jesse
>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>> For additional commands, e-mail: users-help@tomcat.apache.org
>>
>>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Persistent Storage for Webapps
Posted by Jonathan Mast <jh...@gmail.com>.
So you have multiple contexts on a host, each of which needs separate,
persistent storage?
If you don't want to keep track of a property files, you could write a bean
with a method that takes a HttpRequest as a argument, and returns a file
path based on the info in the request. You would have to dedicate a
directory on your server as the root for the persistence system. Then build
subdirectories based on info available in the request object.
I could go on if you want, but basically I'm trying to say there is more
than enough info in each HttpRequest to allow you build a flexible, scalable
persistence system without having to resort to .properties files which
you'll then have to squirrel away somewhere and remember to update each time
you change your Context name, etc. Just look at the javadocs for
HttpRequest. See especially getContextPath() and getLocalName().
On Mon, Oct 12, 2009 at 12:57 PM, Jesse Long <jp...@unknown.za.net> wrote:
> Tobias Crefeld wrote:
>
>> Am Mon, 12 Oct 2009 13:01:44 +0200
>> schrieb Jesse Long <jp...@unknown.za.net>:
>>
>>
>>
>>> In my webapp I need persistent storage. I am building my webapp as a
>>> .war file, and copying it into $CATALINE_HOME/webapps/
>>>
>>> Tomcat extracts $CATALINA_HOME/webapps/app.war to
>>> $CATALINA_HOME/webapps/app/
>>>
>>> The war file contains an empty directory /WEB-INF/data . During
>>> operations, data files are created under /WEB-INF/data/
>>>
>>>
>>
>> WEB-INF should contain only static files which shouldn't be accessible
>> by Webclients (directly), e.g. by /app/data/filename . So it seems to
>> be a bad place
>>
>>
>>
>>
>>> Question: id /WEB-INF/ in the extracted directory the correct place
>>> for persistent data storage? If not, where is?
>>>
>>>
>>
>> AFAIK there is no general advise where to put dynamic, persistent data.
>> As system administrator I prefer solutions that give me the opportunity
>> to decide myself where to put dynamic data. There are a lot of reasons
>> for this: securing against disk space overflow, backup, priviledge
>> separation, etc.
>>
>> Following this way you might create a "properties-"file outside the
>> Tomcat-hierarchy at a sensible place (e.g. /etc/app.properties) with
>> the paths your application should use.
>> Another way is to define a Tomcat-parameter during JVM-startup that will
>> contain the path to your data-directory. As this might not survive the
>> next update of Tomcat itself I don't like this kind of configuration
>> but some applications and classes are working this way.
>>
>>
>>
>>
>>> My goal is to find a consistent way of creating persistent data
>>> relative to the context. I dont want to have to configure context
>>> parameters with data directory paths for each installation.
>>>
>>>
>>
>> untested idea: If you have no control over the filesystem of the
>> target Tomcat you could create a mini-application named data.war that
>> you are deploying. Of course there is always the danger of accidentally
>> erasing the data by re-deploying this pseudo-application. And you will
>> have to find a way by valves, etc. to preserve this path from being
>> accessed by web-clients. Maybe it is enough if you leave out the
>> context-entry.
>>
>>
> Thanks for your answers Tobias. Unfortunately we have a very real use case
> for multiple instances on one host, with different data paths. So,
> /etc/app.properties is not usable, but /etc/app-<context path derived
> data>.properties might be but virtual hosting would break. No way to
> identify unique host consistently.
>
> Also, I could use a path name derived from the temp directory name, but
> nothing says the temp directory name will be consistent. For now I will
> reluctantly use context parameters.
>
> Thanks,
> Jesse
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
>
Re: Persistent Storage for Webapps
Posted by Jesse Long <jp...@unknown.za.net>.
Tobias Crefeld wrote:
> Am Mon, 12 Oct 2009 13:01:44 +0200
> schrieb Jesse Long <jp...@unknown.za.net>:
>
>
>> In my webapp I need persistent storage. I am building my webapp as a
>> .war file, and copying it into $CATALINE_HOME/webapps/
>>
>> Tomcat extracts $CATALINA_HOME/webapps/app.war to
>> $CATALINA_HOME/webapps/app/
>>
>> The war file contains an empty directory /WEB-INF/data . During
>> operations, data files are created under /WEB-INF/data/
>>
>
> WEB-INF should contain only static files which shouldn't be accessible
> by Webclients (directly), e.g. by /app/data/filename . So it seems to
> be a bad place
>
>
>
>> Question: id /WEB-INF/ in the extracted directory the correct place
>> for persistent data storage? If not, where is?
>>
>
> AFAIK there is no general advise where to put dynamic, persistent data.
> As system administrator I prefer solutions that give me the opportunity
> to decide myself where to put dynamic data. There are a lot of reasons
> for this: securing against disk space overflow, backup, priviledge
> separation, etc.
>
> Following this way you might create a "properties-"file outside the
> Tomcat-hierarchy at a sensible place (e.g. /etc/app.properties) with
> the paths your application should use.
>
> Another way is to define a Tomcat-parameter during JVM-startup that will
> contain the path to your data-directory. As this might not survive the
> next update of Tomcat itself I don't like this kind of configuration
> but some applications and classes are working this way.
>
>
>
>> My goal is to find a consistent way of creating persistent data
>> relative to the context. I dont want to have to configure context
>> parameters with data directory paths for each installation.
>>
>
> untested idea: If you have no control over the filesystem of the
> target Tomcat you could create a mini-application named data.war that
> you are deploying. Of course there is always the danger of accidentally
> erasing the data by re-deploying this pseudo-application. And you will
> have to find a way by valves, etc. to preserve this path from being
> accessed by web-clients. Maybe it is enough if you leave out the
> context-entry.
>
Thanks for your answers Tobias. Unfortunately we have a very real use
case for multiple instances on one host, with different data paths. So,
/etc/app.properties is not usable, but /etc/app-<context path derived
data>.properties might be but virtual hosting would break. No way to
identify unique host consistently.
Also, I could use a path name derived from the temp directory name, but
nothing says the temp directory name will be consistent. For now I will
reluctantly use context parameters.
Thanks,
Jesse
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Persistent Storage for Webapps
Posted by Tobias Crefeld <tc...@cataneo.eu>.
Am Mon, 12 Oct 2009 13:01:44 +0200
schrieb Jesse Long <jp...@unknown.za.net>:
> In my webapp I need persistent storage. I am building my webapp as a
> .war file, and copying it into $CATALINE_HOME/webapps/
>
> Tomcat extracts $CATALINA_HOME/webapps/app.war to
> $CATALINA_HOME/webapps/app/
>
> The war file contains an empty directory /WEB-INF/data . During
> operations, data files are created under /WEB-INF/data/
WEB-INF should contain only static files which shouldn't be accessible
by Webclients (directly), e.g. by /app/data/filename . So it seems to
be a bad place
> Question: id /WEB-INF/ in the extracted directory the correct place
> for persistent data storage? If not, where is?
AFAIK there is no general advise where to put dynamic, persistent data.
As system administrator I prefer solutions that give me the opportunity
to decide myself where to put dynamic data. There are a lot of reasons
for this: securing against disk space overflow, backup, priviledge
separation, etc.
Following this way you might create a "properties-"file outside the
Tomcat-hierarchy at a sensible place (e.g. /etc/app.properties) with
the paths your application should use.
Another way is to define a Tomcat-parameter during JVM-startup that will
contain the path to your data-directory. As this might not survive the
next update of Tomcat itself I don't like this kind of configuration
but some applications and classes are working this way.
> My goal is to find a consistent way of creating persistent data
> relative to the context. I dont want to have to configure context
> parameters with data directory paths for each installation.
untested idea: If you have no control over the filesystem of the
target Tomcat you could create a mini-application named data.war that
you are deploying. Of course there is always the danger of accidentally
erasing the data by re-deploying this pseudo-application. And you will
have to find a way by valves, etc. to preserve this path from being
accessed by web-clients. Maybe it is enough if you leave out the
context-entry.
Gruss,
Tobias.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Persistent Storage for Webapps
Posted by Mikolaj Rydzewski <mi...@ceti.pl>.
Jesse Long wrote:
> Thanks for your answers. I still like the concept of having the
> container allocate persistent storage space. Can I configure tomcat to
> not delete from the "javax.servlet.context.tempdir" directory?
'persistent' and 'temp' sounds like opposite.
--
Mikolaj Rydzewski <mi...@ceti.pl>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Persistent Storage for Webapps
Posted by Jesse Long <jp...@unknown.za.net>.
Mikolaj Rydzewski wrote:
> Jesse Long wrote:
>> Question: id /WEB-INF/ in the extracted directory the correct place for
>> persistent data storage? If not, where is?
> No, WEB-INF is not a correct place app generated files.
> You can use any directory outside of context root.
>> My goal is to find a consistent way of creating persistent data relative
>> to the context. I dont want to have to configure context parameters with
>> data directory paths for each installation.
> It depends :-)
> You can have a code that generates path to storage directory based on
> host name, based on app name, based on server's IP, reads it from JNDI
> context, etc. With such solution you don't need any context
> parameters, you need one war file (I mean war file is not environment
> dependant).
>
Thanks for your answers. I still like the concept of having the
container allocate persistent storage space. Can I configure tomcat to
not delete from the "javax.servlet.context.tempdir" directory?
Thanks,
Jesse
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Persistent Storage for Webapps
Posted by Mikolaj Rydzewski <mi...@ceti.pl>.
Jesse Long wrote:
> Question: id /WEB-INF/ in the extracted directory the correct place for
> persistent data storage? If not, where is?
No, WEB-INF is not a correct place app generated files.
You can use any directory outside of context root.
> My goal is to find a consistent way of creating persistent data relative
> to the context. I dont want to have to configure context parameters with
> data directory paths for each installation.
It depends :-)
You can have a code that generates path to storage directory based on
host name, based on app name, based on server's IP, reads it from JNDI
context, etc. With such solution you don't need any context parameters,
you need one war file (I mean war file is not environment dependant).
--
Mikolaj Rydzewski <mi...@ceti.pl>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org