ICEfaces
  1. ICEfaces
  2. ICE-5212

Concurrency issue when using Spring Web Flow and ICEfaces under heavy load

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 1.8, 1.8.2-RC1, 1.8.2
    • Fix Version/s: 1.8.2-EE-GA_P01
    • Component/s: Framework
    • Labels:
      None
    • Environment:
      WebSphere Application Server 6.0 and 7.0.

      Description

      When using ICEfaces and Spring Web Flow in web applications under moderate to heavy load we would start to see Nullpointer exceptions somewhere deep in the webserver stack. Now, a lot of bad things can (and should! ;-) ) be said about WebSphere, but I doubt that an error of this kind could go undetected for so long.

      Here is an example of a stacktrace:

      Caused by: java.lang.NullPointerException
                       at java.util.Hashtable.get(Hashtable.java(Compiled Code))
                       at com.ibm.ws.webcontainer.srt.SRTRequestContext.encodeURL(SRTRequestContext.java:132)
                       at com.ibm.ws.webcontainer.webapp.WebAppDispatcherContext.encodeURL(WebAppDispatcherContext.java:317)
                       at com.ibm.ws.webcontainer.srt.SRTServletResponse.encodeURL(SRTServletResponse.java:709)
                       at com.ibm.ws.webcontainer.srt.SRTServletResponse.encodeRedirectURL(SRTServletResponse.java:701)
                       at com.icesoft.faces.webapp.http.core.ServletFlowExecutionHandler.sendRedirect(ServletFlowExecutionHandler.java:164)
                       at com.icesoft.faces.webapp.http.core.ServletFlowExecutionHandler.sendFlowExecutionRedirect(ServletFlowExecutionHandler.java:98)
                       at com.icesoft.faces.webapp.http.core.ServletFlowExecutionHandler.handleFlowExecutionResult(ServletFlowExecutionHandler.java:51)
                       at com.icesoft.faces.webapp.http.core.SwfLifecycleExecutor.apply(SwfLifecycleExecutor.java:67)
                       at com.icesoft.faces.webapp.http.core.ReceiveSendUpdates.renderCycle(ReceiveSendUpdates.java:114)
                       at com.icesoft.faces.webapp.http.core.ReceiveSendUpdates.service(ReceiveSendUpdates.java:66)
                       at com.icesoft.faces.webapp.http.core.RequestVerifier.service(RequestVerifier.java:28)
                       at com.icesoft.faces.webapp.http.common.standard.PathDispatcherServer.service(PathDispatcherServer.java(Compiled Code))
                       at com.icesoft.faces.webapp.http.servlet.MainSessionBoundServlet.service(MainSessionBoundServlet.java:160)
                       at com.icesoft.faces.webapp.http.servlet.SessionDispatcher$1.service(SessionDispatcher.java:42)
                       at com.icesoft.faces.webapp.http.servlet.ThreadBlockingAdaptingServlet.service(ThreadBlockingAdaptingServlet.java:19)
                       at com.icesoft.faces.webapp.http.servlet.EnvironmentAdaptingServlet.service(EnvironmentAdaptingServlet.java:63)
                       at com.icesoft.faces.webapp.http.servlet.SessionDispatcher.service(SessionDispatcher.java:62)
                       at com.icesoft.faces.webapp.http.servlet.SessionVerifier.service(SessionVerifier.java:22)
                       at com.icesoft.faces.webapp.http.servlet.PathDispatcher.service(PathDispatcher.java(Compiled Code))
                       at com.icesoft.faces.webapp.http.servlet.MainServlet.service(MainServlet.java:153)

      After days and days of debugging we managed to locate the error and how to reproduce it easily (as in, without launching a stress test every time).

      You need a web application, using Spring Web Flow and ICEfaces, and you need a flow containing more than one page.

      - In one browser you enter your web app. Then you add a breakpoint in the apply method of SwfLifecycleExecutor just before determining whether to resume or launch execution. This line would be a good place:

      FlowExecutionResult result = null;

      - Now navigate to page two in your web app. You're now stuck in your breakpoint. Take a note at the flow execution key, let's say it's e1s1.

      - Next, disable your breakpoint, BUT DO NOT RELEASE your current thread, we're interested in simulating that this thread is suspended and then resumed later.

      - Now it's time to enter your web app in another browser, as the breakpoint is disabled you should go straight through. Navigate a few times and see that everything is working. Note what the execution key is (let's for an example say that it is e4s2)

      - Finally it's time for the fun part. Go back to your debugger, keep an eye on the flow execution key, it is now e1s1. Now step one line forward, and take a look at the flow execution key again....is it still e1s1? No, it is now e4s2!! Somehow the two SwfLifecycleExecutor instances manage to use and update the same values (actually, it is always the same SwfLifecycleExecutor that is in use)

      The problem exists (as far as we can see) in the Lifecycle class. Here's the getLifecycleExecutor method:

      public static LifecycleExecutor getLifecycleExecutor(FacesContext context) {
               init(context);
               Map appMap = context.getExternalContext().getApplicationMap();
               Object swfExecObj = appMap.get(SWF_EXEC);
               if (null != swfExecObj) {
                   //Spring Web Flow URLs do not typically contain file extensions
                   //this is not the correct way to determine whether to delegate
                   //these requests
                  if (!isExtensionMapped(context)) {
                       return (LifecycleExecutor)swfExecObj;
                   }
               }
                return (LifecycleExecutor)appMap.get(JSF_EXEC);
            }

      It seems that it's always the same SWF lifecycle executor that is returned. I'm sure this works with the "normal" ICEFaces Lifecycle executor, but when using Spring Web Flow it leads to unpredictable results if the currently running thread is suspended when executing the apply method.

      Our solution is to return a new SWFLifecycleExecutor every time the getLifecycleExecutor method is called. I'm not sure how much of a performance slowdown (if any), this is, but that way we're sure that nobody steps on each others toes.

      I've attached a patch with our solution, we took the liberty of cleaning up the code a bit, maybe we went overboard but feel free to use which parts suits you.

        Activity

        Hide
        Greg Dick added a comment -

        Changes to the LifecycleExecutor were indeed necessary.

        Show
        Greg Dick added a comment - Changes to the LifecycleExecutor were indeed necessary.
        Hide
        Greg Dick added a comment -

        This was repaired in version 20226

        Show
        Greg Dick added a comment - This was repaired in version 20226
        Hide
        Krashan Brahmanjara added a comment -

        In rev 20214 was patched file SwfLifecycleExecutor.java end file is compiled with error.

        You forget add one line :
        import org.apache.commons.logging.Log;

        Show
        Krashan Brahmanjara added a comment - In rev 20214 was patched file SwfLifecycleExecutor.java end file is compiled with error. You forget add one line : import org.apache.commons.logging.Log;
        Hide
        Peter Bødskov added a comment -

        Here's the patch. I've patched against your trunk as of the14th of December

        Show
        Peter Bødskov added a comment - Here's the patch. I've patched against your trunk as of the14th of December

          People

          • Assignee:
            Unassigned
            Reporter:
            Peter Bødskov
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved: