[ICE-4799] Deadlock in Push Server - ICEsoft JIRA Issue Tracker

Details

Type: Bug
Status: Closed
Priority: Critical
Resolution: Fixed
Affects Version/s: 1.8.1
Fix Version/s: 1.8.2-RC1, 1.8.2
Component/s: None
Labels:
None
Environment:
ICEfaces Core, ICEfaces Push Server, Component Showcase, JBoss 4.0.3.SP1

Description

There's a deadlock in Push Server when the Bridge sends out another blocking request next to an outstanding blocking request just as the session expires:

Java stack information for the threads listed above:
===================================================
"TP-Processor1663":
        at org.icefaces.push.server.SessionManager.isValid(SessionManager.java:182)
        - waiting to lock <0x53183048> (a java.util.HashMap)
        - locked <0x53183020> (a java.util.HashMap)
        at org.icefaces.push.server.SessionManager.isValid(SessionManager.java:172)
        - locked <0x53183020> (a java.util.HashMap)
        at org.icefaces.push.server.ReceiveUpdatedViewsHandler.run(ReceiveUpdatedViewsHandler.java:175)
        at org.icefaces.push.server.AbstractHandler.handle(AbstractHandler.java:76)
        at org.icefaces.push.server.IDVerifier.run(IDVerifier.java:38)
        at org.icefaces.push.server.AbstractHandler.handle(AbstractHandler.java:76)
        at org.icefaces.push.server.SessionBoundServlet$1.handle(SessionBoundServlet.java:79)
        at org.icefaces.push.server.SendUpdatedViewsServer.service(SendUpdatedViewsServer.java:87)
        at com.icesoft.faces.webapp.http.servlet.ThreadBlockingAdaptingServlet.service(ThreadBlockingAdaptingServlet.java:25)
        at com.icesoft.faces.webapp.http.servlet.EnvironmentAdaptingServlet.service(EnvironmentAdaptingServlet.java:63)
        at com.icesoft.faces.webapp.http.servlet.PathDispatcher.service(PathDispatcher.java:23)
        at com.icesoft.faces.webapp.http.servlet.SessionDispatcher.service(SessionDispatcher.java:47)
        at com.icesoft.faces.webapp.http.servlet.PathDispatcher.service(PathDispatcher.java:23)
        at org.icefaces.push.server.PushServlet.service(PushServlet.java:143)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:810)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:252)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:173)
        at org.jboss.web.tomcat.filters.ReplyHeaderFilter.doFilter(ReplyHeaderFilter.java:81)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:202)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:173)
        at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:213)
        at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:178)
        at org.jboss.web.tomcat.security.CustomPrincipalValve.invoke(CustomPrincipalValve.java:39)
        at org.jboss.web.tomcat.security.SecurityAssociationValve.invoke(SecurityAssociationValve.java:159)
        at org.jboss.web.tomcat.security.JaccContextValve.invoke(JaccContextValve.java:59)
        at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:126)
        at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:105)
        at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:107)
        at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:148)
        at org.apache.jk.server.JkCoyoteHandler.invoke(JkCoyoteHandler.java:307)
        at org.apache.jk.common.HandlerRequest.invoke(HandlerRequest.java:385)
        at org.apache.jk.common.ChannelSocket.invoke(ChannelSocket.java:748)
        at org.apache.jk.common.ChannelSocket.processConnection(ChannelSocket.java:678)
        at org.apache.jk.common.SocketConnection.runIt(ChannelSocket.java:871)
        at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:684)
        at java.lang.Thread.run(Thread.java:595)
"http-0.0.0.0-18080-2":
        at org.icefaces.push.server.SessionManager.isValid(SessionManager.java:167)
        - waiting to lock <0x53183020> (a java.util.HashMap)
        at org.icefaces.push.server.ReceiveUpdatedViewsHandler.run(ReceiveUpdatedViewsHandler.java:175)
        at org.icefaces.push.server.AbstractHandler.handle(AbstractHandler.java:76)
        at org.icefaces.push.server.SessionManager.iceFacesIdDisposed(SessionManager.java:120)
        - locked <0x53183048> (a java.util.HashMap)
        - locked <0x53183070> (a org.icefaces.push.server.RequestManager)
        at org.icefaces.push.server.ContextEventMessageHandler.handle(ContextEventMessageHandler.java:80)
        at com.icesoft.net.messaging.MessageSeparator.handle(MessageSeparator.java:85)
        at com.icesoft.net.messaging.http.HttpAdapter$1.dispatch(HttpAdapter.java:101)
        at com.icesoft.net.messaging.http.HttpAdapter$1.service(HttpAdapter.java:143)
        at com.icesoft.faces.webapp.http.servlet.ThreadBlockingAdaptingServlet.service(ThreadBlockingAdaptingServlet.java:25)
        at com.icesoft.faces.webapp.http.servlet.EnvironmentAdaptingServlet.service(EnvironmentAdaptingServlet.java:63)
        at com.icesoft.faces.webapp.http.servlet.PathDispatcher.service(PathDispatcher.java:23)
        at org.icefaces.push.server.PushServlet.service(PushServlet.java:143)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:810)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:252)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:173)
        at org.jboss.web.tomcat.filters.ReplyHeaderFilter.doFilter(ReplyHeaderFilter.java:81)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:202)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:173)
        at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:213)
        at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:178)
        at org.jboss.web.tomcat.security.CustomPrincipalValve.invoke(CustomPrincipalValve.java:39)
        at org.jboss.web.tomcat.security.SecurityAssociationValve.invoke(SecurityAssociationValve.java:159)
        at org.jboss.web.tomcat.security.JaccContextValve.invoke(JaccContextValve.java:59)
        at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:126)
        at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:105)
        at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:107)
        at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:148)
        at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:856)
        at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.processConnection(Http11Protocol.java:744)
        at org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.java:527)
        at org.apache.tomcat.util.net.MasterSlaveWorkerThread.run(MasterSlaveWorkerThread.java:112)
        at java.lang.Thread.run(Thread.java:595)

Found 1 deadlock.

Activity

Ascending order - Click to sort in descending order

Repository	Revision	Date	User	Message
ICEsoft Public SVN Repository	#19140	Mon Aug 03 15:31:53 MDT 2009	jack.van.ooststroom	Fixed JIRA ~~ICE-4799~~ : Deadlock in Push Server
				Files Changed
				MODIFY /icefaces/trunk/icefaces/push-server/src/org/icefaces/push/server/SessionManager.java

Jack Van Ooststroom created issue - 03/Aug/09 4:39 PM

Jack Van Ooststroom made changes - 03/Aug/09 4:52 PM

Field	Original Value	New Value
Assignee		Jack Van Ooststroom [ jack.van.ooststroom ]

Jack Van Ooststroom made changes - 03/Aug/09 4:52 PM

Status

Open [ 1 ]

In Progress [ 3 ]

Hide

Permalink

Jack Van Ooststroom added a comment - 03/Aug/09 4:53 PM

Changed Fix Version(s) to 1.8.2

Show

Jack Van Ooststroom added a comment - 03/Aug/09 4:53 PM Changed Fix Version(s) to 1.8.2

Jack Van Ooststroom made changes - 03/Aug/09 4:53 PM

Salesforce Case		[]
Fix Version/s		1.8.2 [ 10190 ]

Hide

Permalink

Jack Van Ooststroom added a comment - 12/Aug/09 7:00 PM

Deadlocks can be hard to reproduce and this one is a good example. Based on the analysis of the stacktraces and the code involved I made a couple of changes to the locking strategy:

1. Simplified the locking strategy
2. Reduce the time locks are being held (in some cases we could release the lock earlier than we did)
3. Ensure the order of requiring the locks are consistent throughout the code.

Marking this one as FIXED.

Show

Jack Van Ooststroom added a comment - 12/Aug/09 7:00 PM Deadlocks can be hard to reproduce and this one is a good example. Based on the analysis of the stacktraces and the code involved I made a couple of changes to the locking strategy: 1. Simplified the locking strategy 2. Reduce the time locks are being held (in some cases we could release the lock earlier than we did) 3. Ensure the order of requiring the locks are consistent throughout the code. Marking this one as FIXED.

Jack Van Ooststroom made changes - 12/Aug/09 7:00 PM

Status	In Progress [ 3 ]	Resolved [ 5 ]
Resolution		Fixed [ 1 ]

Ken Fyten made changes - 10/Sep/09 1:01 PM

Salesforce Case		[]
Priority	Major [ 3 ]	Critical [ 2 ]

Ken Fyten made changes - 15/Sep/09 11:42 AM

Fix Version/s

1.8.2-RC1 [ 10210 ]

Ken Fyten made changes - 07/Jan/10 12:04 PM

Status

Resolved [ 5 ]

Closed [ 6 ]

People

Assignee:

Jack Van Ooststroom

Reporter:

Jack Van Ooststroom

Votes:

0 Vote for this issue

Watchers:

0 Start watching this issue

Dates

Created:

03/Aug/09 4:39 PM

Updated:

07/Jan/10 12:04 PM

Resolved:

12/Aug/09 7:00 PM