ICEpush
  1. ICEpush
  2. PUSH-138

ConcurrentModificationException in LocalPushGroupManager

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Invalid
    • Affects Version/s: None
    • Fix Version/s: EE-3.2.0.GA
    • Component/s: None
    • Labels:
      None
    • Environment:
      ICEfaces, ICEpush, load test

      Description

      ConcurrentModificationException has been observed in ICEpush under load.
      1. load.sh
        0.6 kB
        Ted Goddard

        Activity

        Ted Goddard created issue -
        Hide
        Ted Goddard added a comment -

        java.util.ConcurrentModificationException
        at java.util.HashMap$HashIterator.nextEntry(HashMap.java:793)
        at java.util.HashMap$KeyIterator.next(HashMap.java:828)
        at java.util.AbstractCollection.toArray(AbstractCollection.java:171)
        at org.icepush.LocalPushGroupManager$Group.getPushIDs(LocalPushGroupManager.java:276)
        at org.icepush.LocalPushGroupManager$Group.access$1300(LocalPushGroupManager.java:241)
        at org.icepush.LocalPushGroupManager.push(LocalPushGroupManager.java:156)
        at org.icepush.PushContext.push(PushContext.java:69)

        Show
        Ted Goddard added a comment - java.util.ConcurrentModificationException at java.util.HashMap$HashIterator.nextEntry(HashMap.java:793) at java.util.HashMap$KeyIterator.next(HashMap.java:828) at java.util.AbstractCollection.toArray(AbstractCollection.java:171) at org.icepush.LocalPushGroupManager$Group.getPushIDs(LocalPushGroupManager.java:276) at org.icepush.LocalPushGroupManager$Group.access$1300(LocalPushGroupManager.java:241) at org.icepush.LocalPushGroupManager.push(LocalPushGroupManager.java:156) at org.icepush.PushContext.push(PushContext.java:69)
        Hide
        Ted Goddard added a comment -

        java.util.ConcurrentModificationException
        at java.util.HashMap$HashIterator.nextEntry(HashMap.java:793)
        at java.util.HashMap$ValueIterator.next(HashMap.java:822)
        at java.util.AbstractCollection.toArray(AbstractCollection.java:124)
        at java.util.ArrayList.<init>(ArrayList.java:131)
        at org.icepush.LocalPushGroupManager.scanForExpiry(LocalPushGroupManager.java:220)
        at org.icepush.LocalPushGroupManager.addMember(LocalPushGroupManager.java:116)
        at org.icepush.PushContext.addGroupMember(PushContext.java:77)
        at org.icefaces.impl.push.SessionViewManager.addView(SessionViewManager.java:39)
        at org.icefaces.impl.event.BridgeSetup$2.encode(BridgeSetup.java:304)
        at org.icefaces.impl.event.UIOutputWriter.encodeBegin(UIOutputWriter.java:31)
        at javax.faces.component.UIComponent.encodeAll(UIComponent.java:1754)
        at com.sun.faces.renderkit.html_basic.BodyRenderer.encodeEnd(BodyRenderer.java:101)
        at javax.faces.component.UIComponentBase.encodeEnd(UIComponentBase.java:875)
        at javax.faces.component.UIComponent.encodeAll(UIComponent.java:1763)
        at org.icefaces.impl.context.DOMPartialViewContext.processPartial(DOMPartialViewContext.java:136)
        at javax.faces.component.UIViewRoot.encodeChildren(UIViewRoot.java:981)
        at javax.faces.component.UIComponent.encodeAll(UIComponent.java:1756)
        at com.sun.faces.application.view.FaceletViewHandlingStrategy.renderView(FaceletViewHandlingStrategy.java:390)
        at com.sun.faces.application.view.MultiViewHandler.renderView(MultiViewHandler.java:131)
        at com.sun.faces.lifecycle.RenderResponsePhase.execute(RenderResponsePhase.java:121)
        at com.sun.faces.lifecycle.Phase.doPhase(Phase.java:101)
        at com.sun.faces.lifecycle.LifecycleImpl.render(LifecycleImpl.java:139)
        at javax.faces.webapp.FacesServlet.service(FacesServlet.java:410)

        Show
        Ted Goddard added a comment - java.util.ConcurrentModificationException at java.util.HashMap$HashIterator.nextEntry(HashMap.java:793) at java.util.HashMap$ValueIterator.next(HashMap.java:822) at java.util.AbstractCollection.toArray(AbstractCollection.java:124) at java.util.ArrayList.<init>(ArrayList.java:131) at org.icepush.LocalPushGroupManager.scanForExpiry(LocalPushGroupManager.java:220) at org.icepush.LocalPushGroupManager.addMember(LocalPushGroupManager.java:116) at org.icepush.PushContext.addGroupMember(PushContext.java:77) at org.icefaces.impl.push.SessionViewManager.addView(SessionViewManager.java:39) at org.icefaces.impl.event.BridgeSetup$2.encode(BridgeSetup.java:304) at org.icefaces.impl.event.UIOutputWriter.encodeBegin(UIOutputWriter.java:31) at javax.faces.component.UIComponent.encodeAll(UIComponent.java:1754) at com.sun.faces.renderkit.html_basic.BodyRenderer.encodeEnd(BodyRenderer.java:101) at javax.faces.component.UIComponentBase.encodeEnd(UIComponentBase.java:875) at javax.faces.component.UIComponent.encodeAll(UIComponent.java:1763) at org.icefaces.impl.context.DOMPartialViewContext.processPartial(DOMPartialViewContext.java:136) at javax.faces.component.UIViewRoot.encodeChildren(UIViewRoot.java:981) at javax.faces.component.UIComponent.encodeAll(UIComponent.java:1756) at com.sun.faces.application.view.FaceletViewHandlingStrategy.renderView(FaceletViewHandlingStrategy.java:390) at com.sun.faces.application.view.MultiViewHandler.renderView(MultiViewHandler.java:131) at com.sun.faces.lifecycle.RenderResponsePhase.execute(RenderResponsePhase.java:121) at com.sun.faces.lifecycle.Phase.doPhase(Phase.java:101) at com.sun.faces.lifecycle.LifecycleImpl.render(LifecycleImpl.java:139) at javax.faces.webapp.FacesServlet.service(FacesServlet.java:410)
        Ted Goddard made changes -
        Field Original Value New Value
        Attachment load.sh [ 13336 ]
        Hide
        Ted Goddard added a comment -

        Open attached load.sh in a few windows.

        Show
        Ted Goddard added a comment - Open attached load.sh in a few windows.
        Ted Goddard made changes -
        Assignee Mircea Toma [ mircea.toma ]
        Hide
        Ted Goddard added a comment -

        Encountered while investigating ICE-6129.

        Show
        Ted Goddard added a comment - Encountered while investigating ICE-6129.
        Hide
        Ted Goddard added a comment -

        java.lang.NegativeArraySizeException
        at java.util.AbstractCollection.toArray(AbstractCollection.java:119)
        at java.util.ArrayList.<init>(ArrayList.java:131)
        at org.icepush.LocalPushGroupManager.scanForExpiry(LocalPushGroupManager.java:220)
        at org.icepush.LocalPushGroupManager.push(LocalPushGroupManager.java:162)
        at org.icepush.PushContext.push(PushContext.java:69)
        at org.icefaces.application.PushRenderer$1.render(PushRenderer.java:185)
        at org.icefaces.demo.auction.view.controllers.IntervalPushRenderer$1.run(IntervalPushRenderer.java:91)
        at java.util.TimerThread.mainLoop(Timer.java:512)
        at java.util.TimerThread.run(Timer.java:462)

        Show
        Ted Goddard added a comment - java.lang.NegativeArraySizeException at java.util.AbstractCollection.toArray(AbstractCollection.java:119) at java.util.ArrayList.<init>(ArrayList.java:131) at org.icepush.LocalPushGroupManager.scanForExpiry(LocalPushGroupManager.java:220) at org.icepush.LocalPushGroupManager.push(LocalPushGroupManager.java:162) at org.icepush.PushContext.push(PushContext.java:69) at org.icefaces.application.PushRenderer$1.render(PushRenderer.java:185) at org.icefaces.demo.auction.view.controllers.IntervalPushRenderer$1.run(IntervalPushRenderer.java:91) at java.util.TimerThread.mainLoop(Timer.java:512) at java.util.TimerThread.run(Timer.java:462)
        Hide
        Deryk Sinotte added a comment -

        Ted to verify whether this is still a concern.

        Show
        Deryk Sinotte added a comment - Ted to verify whether this is still a concern.
        Deryk Sinotte made changes -
        Assignee Mircea Toma [ mircea.toma ] Ted Goddard [ ted.goddard ]
        Hide
        Ted Goddard added a comment -

        Need to repeat load test.

        Show
        Ted Goddard added a comment - Need to repeat load test.
        Ted Goddard made changes -
        Salesforce Case []
        Fix Version/s 3.0.1 [ 10311 ]
        Fix Version/s 3.0 [ 10259 ]
        Ted Goddard made changes -
        Salesforce Case []
        Fix Version/s 3.1 [ 10310 ]
        Fix Version/s 3.0.1 [ 10311 ]
        Hide
        Jack Van Ooststroom added a comment - - edited

        Using ICEfaces3 trunk I re-ran the load test using 2, 4 and 8 windows. I didn't see the exceptions mentioned here but did encounter the following exception a number of times:
        Sep 25, 2012 7:14:55 PM org.icefaces.impl.event.BridgeSetup getBodyResources
        WARNING: Unable to find WindowScope for view /auction.xhtml
        java.lang.NullPointerException
        at org.icefaces.impl.event.WindowAndViewIDSetup$1.encode(WindowAndViewIDSetup.java:65)
        at org.icefaces.impl.event.UIOutputWriter.encodeBegin(UIOutputWriter.java:27)
        at javax.faces.component.UIComponent.encodeAll(UIComponent.java:1777)
        at javax.faces.render.Renderer.encodeChildren(Renderer.java:168)
        at org.icefaces.impl.renderkit.RendererWrapper.encodeChildren(RendererWrapper.java:49)
        at javax.faces.component.UIComponentBase.encodeChildren(UIComponentBase.java:845)
        at javax.faces.component.UIComponent.encodeAll(UIComponent.java:1779)
        at javax.faces.component.UIComponent.encodeAll(UIComponent.java:1782)
        at org.icefaces.impl.context.DOMPartialViewContext.processPartial(DOMPartialViewContext.java:142)
        at javax.faces.component.UIViewRoot.encodeChildren(UIViewRoot.java:981)
        at javax.faces.component.UIComponent.encodeAll(UIComponent.java:1779)
        at com.sun.faces.application.view.FaceletViewHandlingStrategy.renderView(FaceletViewHandlingStrategy.java:409)
        at com.sun.faces.application.view.MultiViewHandler.renderView(MultiViewHandler.java:124)
        at com.sun.faces.lifecycle.RenderResponsePhase.execute(RenderResponsePhase.java:121)
        at com.sun.faces.lifecycle.Phase.doPhase(Phase.java:101)
        at com.sun.faces.lifecycle.LifecycleImpl.render(LifecycleImpl.java:139)
        at javax.faces.webapp.FacesServlet.service(FacesServlet.java:594)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:305)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
        at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:225)
        at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123)
        at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:472)
        at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:168)
        at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:98)
        at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:927)
        at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
        at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:407)
        at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1001)
        at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:585)
        at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:310)
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
        at java.lang.Thread.run(Thread.java:662)

        Show
        Jack Van Ooststroom added a comment - - edited Using ICEfaces3 trunk I re-ran the load test using 2, 4 and 8 windows. I didn't see the exceptions mentioned here but did encounter the following exception a number of times: Sep 25, 2012 7:14:55 PM org.icefaces.impl.event.BridgeSetup getBodyResources WARNING: Unable to find WindowScope for view /auction.xhtml java.lang.NullPointerException at org.icefaces.impl.event.WindowAndViewIDSetup$1.encode(WindowAndViewIDSetup.java:65) at org.icefaces.impl.event.UIOutputWriter.encodeBegin(UIOutputWriter.java:27) at javax.faces.component.UIComponent.encodeAll(UIComponent.java:1777) at javax.faces.render.Renderer.encodeChildren(Renderer.java:168) at org.icefaces.impl.renderkit.RendererWrapper.encodeChildren(RendererWrapper.java:49) at javax.faces.component.UIComponentBase.encodeChildren(UIComponentBase.java:845) at javax.faces.component.UIComponent.encodeAll(UIComponent.java:1779) at javax.faces.component.UIComponent.encodeAll(UIComponent.java:1782) at org.icefaces.impl.context.DOMPartialViewContext.processPartial(DOMPartialViewContext.java:142) at javax.faces.component.UIViewRoot.encodeChildren(UIViewRoot.java:981) at javax.faces.component.UIComponent.encodeAll(UIComponent.java:1779) at com.sun.faces.application.view.FaceletViewHandlingStrategy.renderView(FaceletViewHandlingStrategy.java:409) at com.sun.faces.application.view.MultiViewHandler.renderView(MultiViewHandler.java:124) at com.sun.faces.lifecycle.RenderResponsePhase.execute(RenderResponsePhase.java:121) at com.sun.faces.lifecycle.Phase.doPhase(Phase.java:101) at com.sun.faces.lifecycle.LifecycleImpl.render(LifecycleImpl.java:139) at javax.faces.webapp.FacesServlet.service(FacesServlet.java:594) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:305) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:225) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123) at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:472) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:168) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:98) at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:927) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:407) at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1001) at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:585) at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:310) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662)
        Hide
        Jack Van Ooststroom added a comment - - edited

        I unlinked this case from ICE-8511 because of the following.
        I believe the usage of the load.sh test as described in this JIRA is a flawed/unfair test. The load.sh uses a fixed location for the cookie bin. When starting the test in an x number of console windows, this cookie bin will be used by all the used console windows. This results in using the same Window ID for all console windows, which seems unfair to me. In the case of using 4 console windows, you would now have 4 console windows hammering the server with a 1000 requests each all using the same Window ID.
        The auction application doesn't have any window scoped beans in it. This results in the WindowScope to be constantly activated and deactivated. During this test it could happen that at some point the WindowScope is deactivated while it's trying to retrieve it resulting in the mentioned NullPointerException.
        At this point it's unclear if this is the same situation as ICE-8511, though it looks similar.
        (Restricted to faces-core group)

        Show
        Jack Van Ooststroom added a comment - - edited I unlinked this case from ICE-8511 because of the following. I believe the usage of the load.sh test as described in this JIRA is a flawed/unfair test. The load.sh uses a fixed location for the cookie bin. When starting the test in an x number of console windows, this cookie bin will be used by all the used console windows. This results in using the same Window ID for all console windows, which seems unfair to me. In the case of using 4 console windows, you would now have 4 console windows hammering the server with a 1000 requests each all using the same Window ID. The auction application doesn't have any window scoped beans in it. This results in the WindowScope to be constantly activated and deactivated. During this test it could happen that at some point the WindowScope is deactivated while it's trying to retrieve it resulting in the mentioned NullPointerException. At this point it's unclear if this is the same situation as ICE-8511 , though it looks similar. (Restricted to faces-core group)
        Hide
        Ted Goddard added a comment - - edited

        This doesn't seem any different from a rare startup case with two windows, potentially leading to the same race condition with WindowScope create/destroy.

        Show
        Ted Goddard added a comment - - edited This doesn't seem any different from a rare startup case with two windows, potentially leading to the same race condition with WindowScope create/destroy.
        Migration made changes -
        Fix Version/s 3.2 [ 10340 ]
        Fix Version/s 3.1 [ 10310 ]
        Migration made changes -
        Fix Version/s EE-3.2.0.GA [ 10323 ]
        Fix Version/s 3.2 [ 10340 ]
        Migration made changes -
        Fix Version/s 3.2 [ 10340 ]
        Fix Version/s EE-3.2.0.GA [ 10323 ]
        Migration made changes -
        Fix Version/s EE-3.2.0.GA [ 10323 ]
        Fix Version/s 3.2 [ 10340 ]
        Hide
        Ted Goddard added a comment -

        ICEpush has been considerably refactored, invalidating these Exceptions. The result is that it is simply necessary to conduct a load test.

        Show
        Ted Goddard added a comment - ICEpush has been considerably refactored, invalidating these Exceptions. The result is that it is simply necessary to conduct a load test.
        Ted Goddard made changes -
        Status Open [ 1 ] Resolved [ 5 ]
        Resolution Invalid [ 6 ]
        Ken Fyten made changes -
        Status Resolved [ 5 ] Closed [ 6 ]

          People

          • Assignee:
            Ted Goddard
            Reporter:
            Ted Goddard
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved: