You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@tomcat.apache.org by bu...@apache.org on 2013/10/29 17:41:54 UTC

[Bug 55719] New: WAR's deployed via Tomcat Manager using Cargo deployer are getting corrupted.

https://issues.apache.org/bugzilla/show_bug.cgi?id=55719

            Bug ID: 55719
           Summary: WAR's deployed via Tomcat Manager using Cargo deployer
                    are getting corrupted.
           Product: Tomcat 7
           Version: 7.0.42
          Hardware: PC
                OS: Mac OS X 10.4
            Status: NEW
          Severity: normal
          Priority: P2
         Component: Manager
          Assignee: dev@tomcat.apache.org
          Reporter: dan@danshome.net

We've been using the same cargo deployer (v 1.2.4) from Jenkins to deploy WAR's
to Tomcat 7.0.30 for almost a year with no issues at all, then after we
upgraded to Tomcat 7.0.42 and JDK 1.7.0_40 we started getting random failures
with WAR deployments.  We've seen similar issues with someone tries to manually
deploy a WAR to the webapps folder when autodeploy is enabled and Tomcat tries
to unzip the WAR before the file copy is complete, but we've never seen this
when using the Tomcat Manager /deploy. We do have auto deploy enabled at this
time.  Is it possible that a change between 7.0.30 and 7.0.42 could be causing
Tomcat with autodeploy enabled to try to decompress the WAR before the Tomcat
Manager has finished dropping the WAR in the webapps folder?  

The problem happens at random, and in most cases simply re-running the
deployment works.


Oct 24, 2013 9:23:24 PM org.apache.catalina.startup.HostConfig
deleteRedeployResources
INFO: Undeploying context [/CrashMapBuilder]
Oct 24, 2013 9:34:03 PM org.apache.catalina.startup.HostConfig deployWAR
INFO: Deploying web application archive
E:\servers\tomcat\inst\tomcatsrv1\webapps\CrashMapBuilder.war
Oct 24, 2013 9:34:03 PM org.apache.catalina.startup.ContextConfig init
SEVERE: Exception fixing docBase for context [/CrashMapBuilder]
java.util.zip.ZipException: error in opening zip file
    at java.util.zip.ZipFile.open(Native Method)
    at java.util.zip.ZipFile.<init>(ZipFile.java:215)
    at java.util.zip.ZipFile.<init>(ZipFile.java:145)
    at java.util.jar.JarFile.<init>(JarFile.java:153)
    at java.util.jar.JarFile.<init>(JarFile.java:90)
    at sun.net.www.protocol.jar.URLJarFile.<init>(URLJarFile.java:93)
    at sun.net.www.protocol.jar.URLJarFile.getJarFile(URLJarFile.java:69)
    at sun.net.www.protocol.jar.JarFileFactory.get(JarFileFactory.java:109)
    at
sun.net.www.protocol.jar.JarURLConnection.connect(JarURLConnection.java:122)
    at
sun.net.www.protocol.jar.JarURLConnection.getJarFile(JarURLConnection.java:89)
    at org.apache.catalina.startup.ExpandWar.expand(ExpandWar.java:113)
    at
org.apache.catalina.startup.ContextConfig.fixDocBase(ContextConfig.java:722)
    at org.apache.catalina.startup.ContextConfig.init(ContextConfig.java:843)
    at
org.apache.catalina.startup.ContextConfig.lifecycleEvent(ContextConfig.java:387)
    at
org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSupport.java:119)
    at
org.apache.catalina.util.LifecycleBase.fireLifecycleEvent(LifecycleBase.java:90)
    at
org.apache.catalina.util.LifecycleBase.setStateInternal(LifecycleBase.java:402)
    at org.apache.catalina.util.LifecycleBase.init(LifecycleBase.java:110)
    at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:139)
    at
org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:901)
    at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:877)
    at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:633)
    at org.apache.catalina.startup.HostConfig.deployWAR(HostConfig.java:976)
    at
org.apache.catalina.startup.HostConfig$DeployWar.run(HostConfig.java:1653)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
    at java.util.concurrent.FutureTask.run(FutureTask.java:262)
    at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:724)

Oct 24, 2013 9:34:03 PM org.apache.catalina.core.StandardContext resourcesStart
SEVERE: Error starting static Resources
java.lang.IllegalArgumentException: Invalid or unreadable WAR file : error in
opening zip file
    at
org.apache.naming.resources.WARDirContext.setDocBase(WARDirContext.java:138)
    at
org.apache.catalina.core.StandardContext.resourcesStart(StandardContext.java:5055)
    at
org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5235)
    at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150)
    at
org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:901)
    at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:877)
    at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:633)
    at org.apache.catalina.startup.HostConfig.deployWAR(HostConfig.java:976)
    at
org.apache.catalina.startup.HostConfig$DeployWar.run(HostConfig.java:1653)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
    at java.util.concurrent.FutureTask.run(FutureTask.java:262)
    at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:724)

Oct 24, 2013 9:34:03 PM org.apache.catalina.core.StandardContext startInternal
SEVERE: Error in resourceStart()
Oct 24, 2013 9:34:03 PM org.apache.catalina.core.StandardContext startInternal
SEVERE: Error getConfigured
Oct 24, 2013 9:34:03 PM org.apache.catalina.core.StandardContext startInternal
SEVERE: Context [/CrashMapBuilder] startup failed due to previous errors

-- 
You are receiving this mail because:
You are the assignee for the bug.

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org


[Bug 55719] WAR's deployed via Tomcat Manager using Cargo deployer are getting corrupted.

Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=55719

--- Comment #4 from Mark Thomas <ma...@apache.org> ---
The manager app does the following:
- marks the context as services so the auto deployer leaves it alone
- removes the old war
- reads the input stream and writes the WAR directly to the appBase
- triggers a deploy for the new WAR
- removes the serviced marker

Given that auto deployment is disabled in your environment anyway, the only way
I can see the problem you are having arising is if the file upload somehow goes
wrong and a partial file is written to the appBase.

Let me look though the source code to check if all the possible errors are
correctly handled.

-- 
You are receiving this mail because:
You are the assignee for the bug.

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org


[Bug 55719] WAR's deployed via Tomcat Manager using Cargo deployer are getting corrupted.

Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=55719

--- Comment #7 from Mark Thomas <ma...@apache.org> ---
I been through the code and if the upload fails there should be something
logged to $CATALINA_BASE/logs/manager-<timestamp>.log

Do you still have those logs? If so, could you check to see if anything was
logged?

-- 
You are receiving this mail because:
You are the assignee for the bug.

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org


[Bug 55719] WAR's deployed via Tomcat Manager using Cargo deployer are getting corrupted.

Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=55719

dan@danshome.net changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 OS|                            |All

--- Comment #1 from dan@danshome.net ---
One thing that is important to mention is that I just found out that we had
several backup processes starting on the same SAN that is used by the
underlying storage where the WAR's are deployed. This was creating an extreme
mount if I/O contention and slowed down the deployment of the WAR from Tomcat
manager. So you may not see this issue unless you are able to create enough I/O
contention to slow the deployment down to a crawl.  Regardless of how slow it
takes for Tomcat Manager to receive and copy the WAR to the web apps folder, it
still seems like autodeploy should be smart enough to know that Tomcat Manager
is in the middle of a WAR deployment and not to try to to extract the WAR.

-- 
You are receiving this mail because:
You are the assignee for the bug.

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org


[Bug 55719] WAR's deployed via Tomcat Manager using Cargo deployer are getting corrupted.

Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=55719

--- Comment #3 from dan@danshome.net ---
Only a single Tomcat instance.

-- 
You are receiving this mail because:
You are the assignee for the bug.

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org


[Bug 55719] WAR's deployed via Tomcat Manager using Cargo deployer are getting corrupted.

Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=55719

Mark Thomas <ma...@apache.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEEDINFO                    |RESOLVED
         Resolution|---                         |WONTFIX

-- 
You are receiving this mail because:
You are the assignee for the bug.

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org


[Bug 55719] WAR's deployed via Tomcat Manager using Cargo deployer are getting corrupted.

Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=55719

dan@danshome.net changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 OS|Mac OS X 10.4               |Windows Server 2008 R2

-- 
You are receiving this mail because:
You are the assignee for the bug.

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org


[Bug 55719] WAR's deployed via Tomcat Manager using Cargo deployer are getting corrupted.

Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=55719

--- Comment #8 from Mark Thomas <ma...@apache.org> ---
Without further input this issue will eventually get resolved as WONTFIX. If
that does happen, it can always be re-opened once further information is
available.

-- 
You are receiving this mail because:
You are the assignee for the bug.

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org


[Bug 55719] WAR's deployed via Tomcat Manager using Cargo deployer are getting corrupted.

Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=55719

--- Comment #2 from Mark Thomas <ma...@apache.org> ---
How many Tomcat instances are using the WARs that are being deployed to this
SAN?

-- 
You are receiving this mail because:
You are the assignee for the bug.

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org


[Bug 55719] WAR's deployed via Tomcat Manager using Cargo deployer are getting corrupted.

Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=55719

--- Comment #6 from Mark Thomas <ma...@apache.org> ---
Even with auto deployment enabled, the use of the serviced flag by the manager
means it should not be a factor here.

I suspect some form of timeout as well. I agree it should be handled. I want to
check to make sure that it is handled.

-- 
You are receiving this mail because:
You are the assignee for the bug.

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org


[Bug 55719] WAR's deployed via Tomcat Manager using Cargo deployer are getting corrupted.

Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=55719

--- Comment #5 from Dan McLaughlin <da...@danshome.net> ---
Hi Mark,

As I stated in my original posting "We DO have auto deploy enabled at this
time."  We moved our SAN backups last night so they didn't run during the
nightly deployment and there were no zip failures this time.  We'll have to
watch things over the the next few days, but it seems that the I/O contention
somehow allowed for the Tomcat Manager deployment and autodeploy to collide. 
The idea that the transfer of the WAR failed sounds reasonable.  Even so, if we
hit some timeout that caused the WAR to abort the upload, shouldn't Tomcat
Manager have caught the disconnect and removed the incomplete WAR?

-- 
You are receiving this mail because:
You are the assignee for the bug.

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org


[Bug 55719] WAR's deployed via Tomcat Manager using Cargo deployer are getting corrupted.

Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=55719

Mark Thomas <ma...@apache.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |NEEDINFO

-- 
You are receiving this mail because:
You are the assignee for the bug.

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org