QA tested it on Tomcat 6 using basic sample application (Glimmer revision #20012 )

The NullPointerException cannot be reproduced when redeploying basic sample on top of itself by hot deployment.

After working for a while, it comes back and even when the server is cleaned off entirely, start up the server clean (all temporary files, etc). Application cannot be deployed for some period of time after shutting down server and keep cleaning it....
SEVERE: Exception sending context destroyed event to listener instance of class org.icefaces.push.servlet.ServletEnvironmentListener
java.lang.NullPointerException
at org.icefaces.push.servlet.ICEfacesResourceHandler.notifyContextShutdown(ICEfacesResourceHandler.java:175)
at org.icefaces.push.servlet.ServletEnvironmentListener.contextDestroyed(ServletEnvironmentListener.java:38)
at org.apache.catalina.core.StandardContext.listenerStop(StandardContext.java:3973)
at org.apache.catalina.core.StandardContext.stop(StandardContext.java:4577)
at org.apache.catalina.core.StandardContext.start(StandardContext.java:4474)
at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:791)
at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:771)
at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:526)
at org.apache.catalina.startup.HostConfig.deployWAR(HostConfig.java:850)
at org.apache.catalina.startup.HostConfig.deployWARs(HostConfig.java:724)
at org.apache.catalina.startup.HostConfig.deployApps(HostConfig.java:493)
at org.apache.catalina.startup.HostConfig.check(HostConfig.java:1274)
at org.apache.catalina.startup.HostConfig.lifecycleEvent(HostConfig.java:296)
at org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSupport.java:119)
at org.apache.catalina.core.ContainerBase.backgroundProcess(ContainerBase.java:1337)
at org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.processChildren(ContainerBase.java:1601)
at org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.processChildren(ContainerBase.java:1610)
at org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.run(ContainerBase.java:1590)
at java.lang.Thread.run(Thread.java:619)

Turned off session persistence by uncommenting the following line in /conf/context.xml of the Tomcat6.0.20 AS:-
<Manager pathname="" />
Now the server no longer freezes, but the client does. (seems to be better on the windows machine than my mac).

Also put session timeout in web.xml down to 3 minutes as once the session times out, I can get the client to unfreeze again. Otherwise, it is still blocking. Chrome will allow another tab to open but the application is unactive (until it times out).

Note that this doesn't seem to be a problem on the mac. Just windows machine (I have XP) seems to be the problem.

I even have the session persistence on for the mac and no problem.

File locking has been a long standing problem with redeploying .war files when running Tomcat on Windows. So much so that they provide a couple of configuration parameters to deal with it:

http://tomcat.apache.org/tomcat-6.0-doc/config/context.html

antiJARLocking

If true, the Tomcat classloader will take extra measures to avoid JAR file locking when resources are accessed inside JARs through URLs. This will impact startup time of applications, but could prove to be useful on platforms or configurations where file locking can occur. If not specified, the default value is false.

antiResourceLocking

If true, Tomcat will prevent any file locking. This will significantly impact startup time of applications, but allows full webapp hot deploy and undeploy on platforms or configurations where file locking can occur. If not specified, the default value is false.

Please note that setting this to true has some side effects, including the disabling of JSP reloading in a running server: see Bugzilla 37668.

Please note that setting this flag to true in applications that are outside the appBase for the Host (the webapps directory by default) will cause the application to be deleted on Tomcat shutdown. You probably don't want to do this, so think twice before setting antiResourceLocking=true on a webapp that's outside the appBase for its Host.

So I added a context.xml file and modified the common build file to add it to the component-showcase:

<Context antiJarLocking="true" antiResourceLocking="true" path="/component-showcase" >
</Context>

After that I was easily able to re-deploy the component-showcase.war file. So I can assume that the problem is related to file locking of some sort. We may not need both of them set to true and it may not be desirable to have antiResourceLocking set but we'd need to do more testing to ensure it works and to determine what exactly we're doing to cause the problem. It's likely some resource that either we are JSF is accessing inside a JAR file and perhaps not releasing properly. I think we should consider this a workaround because I don't believe this has been required in the past.

Need to doc. this as a Known Issue for Alpha 2.

I haven't been able to track down the cause of this yet for this build. It's a bit transient as well which doesn't help. For the time being, we'll just have to document the anti*Locking attributes as the required workaround for Tomcat on Windows.

Replacing the application works fine if you don't actually point a browser at it. File locking is definitely in play here. Turning it off within context.xml allows it to be replaced. The first time I removed it, I opened up the initial page and these .jars could not get deleted:

icefaces.jar
icefaces-compat.jar
jsf-impl.jar

The next time, I went to the Connection Status demo and the following .jars could not be deleted:

icefaces.jar
icefaces-compat.jar
jsf-impl.jar
krysalis-.jar

Then I removed icepush.jar. There were a couple of INFO messages indicating that push wasn't available. When I navigated to the first page then tried to redeploy, the same basic list of .jars was hanging around so it's not related to push:

icefaces.jar
icefaces-compat.jar
jsf-impl.jar

Even trying to manually delete them from Windows Explorer wouldn't work until I shut Tomcat down. I ran the non-compat version of Auction as well and had no trouble re-deploying it so it seems to be more related to the icefaces-compat.jar

So next I turned on verbose classloading. The following are the classes loaded from icefaces-compat.jar:

[Loaded com.icesoft.faces.context.effects.CurrentStyle from file:/C:/apps/apache-tomcat-6.0.18/webapps/component-showcase/WEB-INF/lib/icefaces-compat.jar]

[Loaded com.icesoft.util.pooling.ClientIdPool from file:/C:/apps/apache-tomcat-6.0.18/webapps/component-showcase/WEB-INF/lib/icefaces-compat.jar]

[Loaded com.icesoft.util.pooling.StringInternMapLRU from file:/C:/apps/apache-tomcat-6.0.18/webapps/component-showcase/WEB-INF/lib/icefaces-compat.jar]

[Loaded com.icesoft.util.pooling.StringInternMapLRU$1 from file:/C:/apps/apache-tomcat-6.0.18/webapps/component-showcase/WEB-INF/lib/icefaces-compat.jar]

[Loaded com.icesoft.faces.util.CoreUtils from file:/C:/apps/apache-tomcat-6.0.18/webapps/component-showcase/WEB-INF/lib/icefaces-compat.jar]

[Loaded com.icesoft.util.pooling.CSSNamePool from file:/C:/apps/apache-tomcat-6.0.18/webapps/component-showcase/WEB-INF/lib/icefaces-compat.jar]

[Loaded com.icesoft.faces.util.Debug from file:/C:/apps/apache-tomcat-6.0.18/webapps/component-showcase/WEB-INF/lib/icefaces-compat.jar]

[Loaded com.icesoft.faces.renderkit.dom_html_basic.PassThruAttributeRenderer from file:/C:/apps/apache-tomcat-6.0.18/webapps/component-showcase/WEB-INF/lib/icefaces-compat.jar]

[Loaded com.icesoft.faces.webapp.parser.ImplementationUtil from file:/C:/apps/apache-tomcat-6.0.18/webapps/component-showcase/WEB-INF/lib/icefaces-compat.jar]

[Loaded com.icesoft.faces.context.effects.JavascriptContext from file:/C:/apps/apache-tomcat-6.0.18/webapps/component-showcase/WEB-INF/lib/icefaces-compat.jar]

[Loaded com.icesoft.faces.application.StartupTime from file:/C:/apps/apache-tomcat-6.0.18/webapps/component-showcase/WEB-INF/lib/icefaces-compat.jar]

[Loaded com.icesoft.util.SeamUtilities from file:/C:/apps/apache-tomcat-6.0.18/webapps/component-showcase/WEB-INF/lib/icefaces-compat.jar]

[Loaded com.icesoft.faces.context.ResourceLinker$Handler from file:/C:/apps/apache-tomcat-6.0.18/webapps/component-showcase/WEB-INF/lib/icefaces-compat.jar]

[Loaded com.icesoft.faces.context.JarResource from file:/C:/apps/apache-tomcat-6.0.18/webapps/component-showcase/WEB-INF/lib/icefaces-compat.jar]

[Loaded com.icesoft.faces.renderkit.dom_html_basic.PassThruAttributeWriter from file:/C:/apps/apache-tomcat-6.0.18/webapps/component-showcase/WEB-INF/lib/icefaces-compat.jar]

[Loaded com.icesoft.faces.webapp.ServeCSSResource$1 from file:/C:/apps/apache-tomcat-6.0.18/webapps/component-showcase/WEB-INF/lib/icefaces-compat.jar]

I noted this bug has some very recent additions and has not yet been fixed. It's related to using getResourcesAsStream() having no way to close() and how this is a problem specifically for Windows:

http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=5041014

Searching on our own usages of getResourceAsStream() (narrowed down to compat, components, and showcase), I find the following:

Compat Components (1 usage)
com.icesoft.faces.component.inputrichtext (1 usage)
InputRichText.java (1 usage)
(38, 67) InputStream in = InputRichText.class.getClassLoader().getResourceAsStream(FCK_EDITOR_ZIP);
Compat Core (4 usages)
com.icesoft.faces.context (1 usage)
JarResource.java (1 usage)
(24, 49) return this.getClass().getClassLoader().getResourceAsStream(path);
com.icesoft.faces.webapp (1 usage)
ServeCSSResource.java (1 usage)
(26, 39) final InputStream in = loader.getResourceAsStream(Package + file);
com.icesoft.jasper.xmlparser (2 usages)
CachedEntityResolver.java (1 usage)
(35, 41) this.getClass().getResourceAsStream(resourcePath);
ParserUtils.java (1 usage)
(186, 23) this.getClass().getResourceAsStream(resourcePath);
Component-showcase (4 usages)
OutputResourceBean.java (1 usage)
(130, 45) InputStream stream = extContext.getResourceAsStream(OutputResourceBean.RESOURCE_PATH + resourceName);
CityDictionary.java (1 usage)
(197, 33) InputStream is = ec.getResourceAsStream(DATA_RESOURCE_PATH);
org.icefaces.application.showcase.view.bean.examples.component.outputResource (1 usage)
OutputResourceBean.java (1 usage)
(130, 45) InputStream stream = extContext.getResourceAsStream(OutputResourceBean.RESOURCE_PATH + resourceName);
org.icefaces.application.showcase.view.bean.examples.component.selectInputText (1 usage)
CityDictionary.java (1 usage)
(197, 33) InputStream is = ec.getResourceAsStream(DATA_RESOURCE_PATH);

After a bunch of trial and error debugging, I changed the ServeCSSResource.service method to put in a fake inputStream to replace the real inputStream based on the classloader and it seemed to fix the problem.

public void service(Request request) throws Exception {
final String path = request.getURI().getPath();
String file = path.substring(path.lastIndexOf("css/") + 4, path.length());
final ByteArrayInputStream in = new ByteArrayInputStream(new String("blah").getBytes());
// final InputStream in = loader.getResourceAsStream(Package + file);

Merging the icefaces-compat.jar and icefaces-compat-comps.jar into a single jar may alleviate this issue.

Now that the compat jars have been coalesced into a single library, I ran the compat version of Component Showcase, exercised it, and re-deployed a couple of times on Windows. There is still an issue of jar locking there. The server log reports:

Jul 12, 2010 11:20:14 AM org.apache.catalina.startup.ExpandWar deleteDir
SEVERE: [C:\apps\apache-tomcat-6.0.28\apache-tomcat-6.0.28\webapps\component-showcase\WEB-INF\lib] could not be completely deleted. The presence of the remaining files may cause problems
Jul 12, 2010 11:20:14 AM org.apache.catalina.startup.ExpandWar deleteDir
SEVERE: [C:\apps\apache-tomcat-6.0.28\apache-tomcat-6.0.28\webapps\component-showcase\WEB-INF] could not be completely deleted. The presence of the remaining files may cause problems
Jul 12, 2010 11:20:14 AM org.apache.catalina.startup.ExpandWar deleteDir
SEVERE: [C:\apps\apache-tomcat-6.0.28\apache-tomcat-6.0.28\webapps\component-showcase] could not be completely deleted. The presence of the remaining files may cause problems
Jul 12, 2010 11:20:14 AM org.apache.catalina.startup.ExpandWar delete
SEVERE: [C:\apps\apache-tomcat-6.0.28\apache-tomcat-6.0.28\webapps\component-showcase] could not be completely deleted. The presence of the remaining files may cause problems
Jul 12, 2010 11:20:14 AM org.apache.catalina.startup.HostConfig deployWAR
INFO: Deploying web application archive component-showcase.war

Looking in WEB-INF/lib, it shows that

icefaces.jar
icefaces-compat.jar

are still in there so there is likely still a cross-jar URL loading dependency between these two libraries that is causing the issue. We still have a valid workaround for this so we'll just have to live with that for the time being.

Please retest with latest icefaces2/trunk.

Icefaces2 revision# 22166
Server: Tomcat6

Tested successfully, problem could not be reproduced after trying several times using auctionMonitor & auction applications.