ICEfaces
  1. ICEfaces
  2. ICE-4799

Deadlock in Push Server

          Details

          • Type: Bug Bug
          • Status: Closed
          • Priority: Critical Critical
          • Resolution: Fixed
          • Affects Version/s: 1.8.1
          • Fix Version/s: 1.8.2-RC1, 1.8.2
          • Component/s: None
          • Labels:
            None
          • Environment:
            ICEfaces Core, ICEfaces Push Server, Component Showcase, JBoss 4.0.3.SP1

            Description

            There's a deadlock in Push Server when the Bridge sends out another blocking request next to an outstanding blocking request just as the session expires:

            Java stack information for the threads listed above:
            ===================================================
            "TP-Processor1663":
                    at org.icefaces.push.server.SessionManager.isValid(SessionManager.java:182)
                    - waiting to lock <0x53183048> (a java.util.HashMap)
                    - locked <0x53183020> (a java.util.HashMap)
                    at org.icefaces.push.server.SessionManager.isValid(SessionManager.java:172)
                    - locked <0x53183020> (a java.util.HashMap)
                    at org.icefaces.push.server.ReceiveUpdatedViewsHandler.run(ReceiveUpdatedViewsHandler.java:175)
                    at org.icefaces.push.server.AbstractHandler.handle(AbstractHandler.java:76)
                    at org.icefaces.push.server.IDVerifier.run(IDVerifier.java:38)
                    at org.icefaces.push.server.AbstractHandler.handle(AbstractHandler.java:76)
                    at org.icefaces.push.server.SessionBoundServlet$1.handle(SessionBoundServlet.java:79)
                    at org.icefaces.push.server.SendUpdatedViewsServer.service(SendUpdatedViewsServer.java:87)
                    at com.icesoft.faces.webapp.http.servlet.ThreadBlockingAdaptingServlet.service(ThreadBlockingAdaptingServlet.java:25)
                    at com.icesoft.faces.webapp.http.servlet.EnvironmentAdaptingServlet.service(EnvironmentAdaptingServlet.java:63)
                    at com.icesoft.faces.webapp.http.servlet.PathDispatcher.service(PathDispatcher.java:23)
                    at com.icesoft.faces.webapp.http.servlet.SessionDispatcher.service(SessionDispatcher.java:47)
                    at com.icesoft.faces.webapp.http.servlet.PathDispatcher.service(PathDispatcher.java:23)
                    at org.icefaces.push.server.PushServlet.service(PushServlet.java:143)
                    at javax.servlet.http.HttpServlet.service(HttpServlet.java:810)
                    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:252)
                    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:173)
                    at org.jboss.web.tomcat.filters.ReplyHeaderFilter.doFilter(ReplyHeaderFilter.java:81)
                    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:202)
                    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:173)
                    at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:213)
                    at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:178)
                    at org.jboss.web.tomcat.security.CustomPrincipalValve.invoke(CustomPrincipalValve.java:39)
                    at org.jboss.web.tomcat.security.SecurityAssociationValve.invoke(SecurityAssociationValve.java:159)
                    at org.jboss.web.tomcat.security.JaccContextValve.invoke(JaccContextValve.java:59)
                    at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:126)
                    at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:105)
                    at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:107)
                    at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:148)
                    at org.apache.jk.server.JkCoyoteHandler.invoke(JkCoyoteHandler.java:307)
                    at org.apache.jk.common.HandlerRequest.invoke(HandlerRequest.java:385)
                    at org.apache.jk.common.ChannelSocket.invoke(ChannelSocket.java:748)
                    at org.apache.jk.common.ChannelSocket.processConnection(ChannelSocket.java:678)
                    at org.apache.jk.common.SocketConnection.runIt(ChannelSocket.java:871)
                    at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:684)
                    at java.lang.Thread.run(Thread.java:595)
            "http-0.0.0.0-18080-2":
                    at org.icefaces.push.server.SessionManager.isValid(SessionManager.java:167)
                    - waiting to lock <0x53183020> (a java.util.HashMap)
                    at org.icefaces.push.server.ReceiveUpdatedViewsHandler.run(ReceiveUpdatedViewsHandler.java:175)
                    at org.icefaces.push.server.AbstractHandler.handle(AbstractHandler.java:76)
                    at org.icefaces.push.server.SessionManager.iceFacesIdDisposed(SessionManager.java:120)
                    - locked <0x53183048> (a java.util.HashMap)
                    - locked <0x53183070> (a org.icefaces.push.server.RequestManager)
                    at org.icefaces.push.server.ContextEventMessageHandler.handle(ContextEventMessageHandler.java:80)
                    at com.icesoft.net.messaging.MessageSeparator.handle(MessageSeparator.java:85)
                    at com.icesoft.net.messaging.http.HttpAdapter$1.dispatch(HttpAdapter.java:101)
                    at com.icesoft.net.messaging.http.HttpAdapter$1.service(HttpAdapter.java:143)
                    at com.icesoft.faces.webapp.http.servlet.ThreadBlockingAdaptingServlet.service(ThreadBlockingAdaptingServlet.java:25)
                    at com.icesoft.faces.webapp.http.servlet.EnvironmentAdaptingServlet.service(EnvironmentAdaptingServlet.java:63)
                    at com.icesoft.faces.webapp.http.servlet.PathDispatcher.service(PathDispatcher.java:23)
                    at org.icefaces.push.server.PushServlet.service(PushServlet.java:143)
                    at javax.servlet.http.HttpServlet.service(HttpServlet.java:810)
                    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:252)
                    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:173)
                    at org.jboss.web.tomcat.filters.ReplyHeaderFilter.doFilter(ReplyHeaderFilter.java:81)
                    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:202)
                    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:173)
                    at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:213)
                    at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:178)
                    at org.jboss.web.tomcat.security.CustomPrincipalValve.invoke(CustomPrincipalValve.java:39)
                    at org.jboss.web.tomcat.security.SecurityAssociationValve.invoke(SecurityAssociationValve.java:159)
                    at org.jboss.web.tomcat.security.JaccContextValve.invoke(JaccContextValve.java:59)
                    at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:126)
                    at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:105)
                    at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:107)
                    at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:148)
                    at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:856)
                    at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.processConnection(Http11Protocol.java:744)
                    at org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.java:527)
                    at org.apache.tomcat.util.net.MasterSlaveWorkerThread.run(MasterSlaveWorkerThread.java:112)
                    at java.lang.Thread.run(Thread.java:595)

            Found 1 deadlock.

              Activity

              Ascending order - Click to sort in descending order
              Hide
              Permalink
              Jack Van Ooststroom added a comment -

              Changed Fix Version(s) to 1.8.2

              Show
              Jack Van Ooststroom added a comment - Changed Fix Version(s) to 1.8.2
              Hide
              Permalink
              Jack Van Ooststroom added a comment -

              Deadlocks can be hard to reproduce and this one is a good example. Based on the analysis of the stacktraces and the code involved I made a couple of changes to the locking strategy:

              1. Simplified the locking strategy
              2. Reduce the time locks are being held (in some cases we could release the lock earlier than we did)
              3. Ensure the order of requiring the locks are consistent throughout the code.

              Marking this one as FIXED.

              Show
              Jack Van Ooststroom added a comment - Deadlocks can be hard to reproduce and this one is a good example. Based on the analysis of the stacktraces and the code involved I made a couple of changes to the locking strategy: 1. Simplified the locking strategy 2. Reduce the time locks are being held (in some cases we could release the lock earlier than we did) 3. Ensure the order of requiring the locks are consistent throughout the code. Marking this one as FIXED.

                People

                • Assignee:
                  Jack Van Ooststroom
                  Reporter:
                  Jack Van Ooststroom
                • Votes:
                  0 Vote for this issue
                  Watchers:
                  0 Start watching this issue

                  Dates

                  • Created:
                    Updated:
                    Resolved: