You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@tomcat.apache.org by bu...@apache.org on 2014/01/23 16:16:04 UTC

[Bug 56056] New: Tomcat fail to redeploy application, when webapps is on NFS file system

https://issues.apache.org/bugzilla/show_bug.cgi?id=56056

            Bug ID: 56056
           Summary: Tomcat fail to redeploy application, when webapps is
                    on NFS file system
           Product: Tomcat 7
           Version: 7.0.42
          Hardware: All
                OS: BeOS
            Status: NEW
          Severity: normal
          Priority: P2
         Component: Catalina
          Assignee: dev@tomcat.apache.org
          Reporter: yair.lenga@citi.com

When a new WAR file is copied to the 'webapps' in redepoyment mode
(unpackWar=true), tomcat is unable to complete redeployment in many case, where
the NFS unpacked applications can not be completed.

The reason for the partial delete is that NFS file delete will sometimes rename
the removed file to ".nfsNNNNN". This is usually result of some process the
file open (can be a thread in the application, leaked resources, or a user on
any machine that can view/inspect files/folders in the unpack tree.

While it is bad for any application to have leaked open connections to
resources in the WAR, it is impossible to identify and fix all resource
leakages in third part libraries, etc.

If any file is still open in the unpacked WAR, the directory where the file use
to exists can not be removed. If tomcat can not removed the top level directory
of the old app, it will not redeploy the new WAR. leaving the server without
any running instance of the application.

The requested fix is to add an option (either with passing '-d' to catalina.sh,
or new option in the container XML (similar to antiJARLocking). The new feature
(possible name: 'redeployNFS' ?), will use the following logic for redeployt:

Try to remove the unpacked WAR
If the removal was not completed, sleep 3 seconds, then try again (NFS removal
are not always instant, because some NFS data may be cached on the client).
If the removal is still incomplete, rename the unpacked war (e.g.,
nfs-WAR.YYYY-MM-DDTHH:MM:SS) folder
If the rename works, continue with redeploy. Otherwise stop.

-- 
You are receiving this mail because:
You are the assignee for the bug.

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org


[Bug 56056] Tomcat fail to redeploy application, when webapps is on NFS file system

Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=56056

--- Comment #3 from Konstantin Kolinko <kn...@gmail.com> ---
(In reply to Yair Lenga from comment #0)
> This is usually result of some
> process the file open (can be a thread in the application, leaked resources,
> or a user on any machine that can view/inspect files/folders in the unpack
> tree.
>

In all the above cases waiting for 3 seconds (or for any reasonable longer
time) is futile, as that would not be enough for an user to close the file
or for a resource leak to disappear.

> sleep 3 seconds, then try again

It is not that simple. A retry must be performed at the top-most possible
level, at caller's discretion and consent. Automatic retries are evil when they
sum up.

-- 
You are receiving this mail because:
You are the assignee for the bug.

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org


[Bug 56056] Tomcat fail to redeploy application, when webapps is on NFS file system

Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=56056

--- Comment #2 from Yair Lenga <ya...@citi.com> ---
Mark,

Thank for your feedback. I do not believe that the suggested alternatives
address the problem with NFS removals. The issue is not a tomcat problem
problem, it is just the way Nfs works. The temporary .nfs files may be a result
of bad third party code, background file scanning jobs, or just a user that
does a 'chdir' and forgot to leave. Triggering the problem does not even
require read write access to tomcat.

While tomcat is not the source of the problem, it can help solve the it. I will
appreciate if you will take a second look. Solving this problem will provide
significant improvement to many deployment situation. I will be happy to help
with additional information.

Yair

-- 
You are receiving this mail because:
You are the assignee for the bug.

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org


[Bug 56056] Tomcat fail to redeploy application, when webapps is on NFS file system

Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=56056

Mark Thomas <ma...@apache.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |RESOLVED
         Resolution|---                         |WONTFIX

--- Comment #1 from Mark Thomas <ma...@apache.org> ---
There is no need for yet another deployment option to work around this issue.
You can use parallel deployment to work-around it.

Note with Tomcat 8 you can track open files and Tomcat will forcibly close them
on web application shutdown (assuming the app goes via the Servlet API rather
than directly to the file system).

-- 
You are receiving this mail because:
You are the assignee for the bug.

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org


[Bug 56056] Tomcat fail to redeploy application, when webapps is on NFS file system

Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=56056

--- Comment #4 from Mark Thomas <ma...@apache.org> ---
I'll happily re-open this if you explain how parallel deployment doesn't solve
this problem. If you use parallel deployment:
- you can deploy a new version before deleting the old one
- new users will be directed to the new version
- current users will be directed to the old version until you undeploy it after
which they will be directed to the new one
- if deleting the old one fails users continue to be directed to the new one
- you can repeat the attempt to undeploy the old one as many times as you like
and whenever you like
- the old version will remain listed (in a failed state) in the Manager app so
any tools that work via the Manager app will continue to be able to see it and
attempt to delete it

-- 
You are receiving this mail because:
You are the assignee for the bug.

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org


[Bug 56056] Tomcat fail to redeploy application, when webapps is on NFS file system

Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=56056

--- Comment #5 from Yair Lenga <ya...@citi.com> ---
Many thanks for considering the issue further.

I need few days to reconfigure the system back into the NFS deployment, and
replicate the buggy situation. This time I will make sure to capture the
sequence of events and the exact messages from the varius log files. I will
also try the parallel deployment option.

-- 
You are receiving this mail because:
You are the assignee for the bug.

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org