Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: Bridge-it 1.0
    • Fix Version/s: Bridge-it 1.0
    • Component/s: Push Server
    • Labels:
      None
    • Environment:
      ICEpush as a service

      Description

      ICEpush connection reliability issues have been seen with the service deployment.

        Activity

        Ted Goddard created issue -
        Repository Revision Date User Message
        ICEsoft Public SVN Repository #38785 Wed Nov 13 16:34:06 MST 2013 ted.goddard allow 10 retries during failure condition (PUSH-291)
        Files Changed
        Commit graph MODIFY /icepush/trunk/icepush/core/src/main/javascript/connection.async.js
        Ted Goddard made changes -
        Field Original Value New Value
        Fix Version/s Bridge-it 1.0 [ 11374 ]
        Hide
        Ted Goddard added a comment -

        Increased the allowed retries to 10 from 3. This is hard-coded so it is worthwhile making it configurable.

        Show
        Ted Goddard added a comment - Increased the allowed retries to 10 from 3. This is hard-coded so it is worthwhile making it configurable.
        Hide
        Ted Goddard added a comment -

        Slightly more than 3 retries seems to be sufficient – once the connection went to 4 retries, but has been otherwise stable and accepting push notifications for 20 minutes.

        Show
        Ted Goddard added a comment - Slightly more than 3 retries seems to be sufficient – once the connection went to 4 retries, but has been otherwise stable and accepting push notifications for 20 minutes.
        Ted Goddard made changes -
        Assignee Mircea Toma [ mircea.toma ]
        Hide
        Ted Goddard added a comment -

        The push client should include the current ice.push.heartbeat value it is using so that the server does not block for longer than the expected time.

        Show
        Ted Goddard added a comment - The push client should include the current ice.push.heartbeat value it is using so that the server does not block for longer than the expected time.
        Hide
        Mircea Toma added a comment -

        Most probably the cause of this issue is ICE-9689.

        Show
        Mircea Toma added a comment - Most probably the cause of this issue is ICE-9689 .
        Hide
        Mircea Toma added a comment -

        Modified listen.icepush request to send ice.push.heartbeat parameter with the value of the current heartbeat interval.

        Show
        Mircea Toma added a comment - Modified listen.icepush request to send ice.push.heartbeat parameter with the value of the current heartbeat interval.
        Mircea Toma made changes -
        Status Open [ 1 ] Resolved [ 5 ]
        Resolution Fixed [ 1 ]
        Repository Revision Date User Message
        ICEsoft Public SVN Repository #38792 Thu Nov 14 15:47:01 MST 2013 mircea.toma PUSH-291 Modified listen.icepush request to send ice.push.heartbeat parameter with the value of the current heartbeat interval.
        Files Changed
        Commit graph MODIFY /icepush/trunk/icepush/core/src/main/javascript/connection.async.js
        Hide
        Jack Van Ooststroom added a comment -

        Sending core/src/main/java/org/icepush/BlockingConnectionServer.java
        Sending core/src/main/java/org/icepush/http/PushRequest.java
        Sending core/src/main/java/org/icepush/http/standard/PushRequestProxy.java
        Sending core/src/main/java/org/icepush/servlet/ServletPushRequest.java
        Sending core/src/main/java/org/icepush/servlet/ServletPushRequestResponse.java
        Transmitting file data .....
        Committed revision 38795.

        Show
        Jack Van Ooststroom added a comment - Sending core/src/main/java/org/icepush/BlockingConnectionServer.java Sending core/src/main/java/org/icepush/http/PushRequest.java Sending core/src/main/java/org/icepush/http/standard/PushRequestProxy.java Sending core/src/main/java/org/icepush/servlet/ServletPushRequest.java Sending core/src/main/java/org/icepush/servlet/ServletPushRequestResponse.java Transmitting file data ..... Committed revision 38795.
        Repository Revision Date User Message
        ICEsoft Public SVN Repository #38795 Fri Nov 15 07:44:08 MST 2013 jack.van.ooststroom Fixed JIRA PUSH-291 : Naas Reliability
        Files Changed
        Commit graph MODIFY /icepush/trunk/icepush/core/src/main/java/org/icepush/http/standard/PushRequestProxy.java
        Commit graph MODIFY /icepush/trunk/icepush/core/src/main/java/org/icepush/servlet/ServletPushRequest.java
        Commit graph MODIFY /icepush/trunk/icepush/core/src/main/java/org/icepush/BlockingConnectionServer.java
        Commit graph MODIFY /icepush/trunk/icepush/core/src/main/java/org/icepush/http/PushRequest.java
        Commit graph MODIFY /icepush/trunk/icepush/core/src/main/java/org/icepush/servlet/ServletPushRequestResponse.java
        Hide
        Jack Van Ooststroom added a comment -

        I added the following logic server-side:

        • Get maxHeartbeatInterval and minHeartbeatInterval from the configuation
        • If incoming listen.icepush request has the ice.push.heartbeat parameter assign it to clientSideHeartbeatInterval.
        • If incoming listen.icepush request does not have the ice.push.heartbeat parameter assign Long.MAX_VALUE to clientSideHeartbeatInterval.
        • Assign heartbeatInterval's (Slot) long value to serverSideHeartbeatInterval
        • ResponseTimeoutTime is calculated as follows:
              System.currentTimeMillis() +
              Math.min(
                  Math.max(
                      Math.min(clientSideHeartbeatInterval, serverSideHeartbeatInterval),
                      minHeartbeatInterval),
                  maxHeartbeatInterval)
          

        This should result in the server responding within what the timeframe the client expects within reason by keeping the server-side configured maximum and minimum in mind to avoid malicious clients.

        Show
        Jack Van Ooststroom added a comment - I added the following logic server-side: Get maxHeartbeatInterval and minHeartbeatInterval from the configuation If incoming listen.icepush request has the ice.push.heartbeat parameter assign it to clientSideHeartbeatInterval. If incoming listen.icepush request does not have the ice.push.heartbeat parameter assign Long.MAX_VALUE to clientSideHeartbeatInterval. Assign heartbeatInterval's (Slot) long value to serverSideHeartbeatInterval ResponseTimeoutTime is calculated as follows: System .currentTimeMillis() + Math .min( Math .max( Math .min(clientSideHeartbeatInterval, serverSideHeartbeatInterval), minHeartbeatInterval), maxHeartbeatInterval) This should result in the server responding within what the timeframe the client expects within reason by keeping the server-side configured maximum and minimum in mind to avoid malicious clients.
        Hide
        Jack Van Ooststroom added a comment -

        Mircea,

        We regularly see a response with the ice.push.heartbeat header trying to tell the client what the new value should be and the client not respecting this value. For instance, the server tells the client to use a heartbeat interval of 25000 while the client tells the server it expects a response within its specified heartbeat interval of 15000. This happens without a reload. After running for a certain amount of time we just see the occasional duplicate response messages.

        Show
        Jack Van Ooststroom added a comment - Mircea, We regularly see a response with the ice.push.heartbeat header trying to tell the client what the new value should be and the client not respecting this value. For instance, the server tells the client to use a heartbeat interval of 25000 while the client tells the server it expects a response within its specified heartbeat interval of 15000. This happens without a reload. After running for a certain amount of time we just see the occasional duplicate response messages.
        Hide
        Ted Goddard added a comment -

        Technique for testing bridgeit.mobi/demo/geotrack.html

        Using JavaScript developer console, enter "localStorage"
        Select value of bridgeit.jguid, say, acec8377-5768-48ef-af6e-2984376c23a7
        Run georand acec8377-5768-48ef-af6e-2984376c23a7

        where georand is:

        #!/bin/zsh
        curl --header "Content-Type:application/json" -d '{ "geometry" : { "type" : "Point", "coordinates" : [ '-114.$

        {RANDOM}','51.${RANDOM}

        ', 1091.083740234375 ] }, "type" : "Feature", "properties" :

        { "jguid" : "bafe2b76-08cc-4a6d-9c17-9a1bebe65037" }

        }' http://api.bridgeit.mobi/echo/service/echoput/geotrack-$1

        Show
        Ted Goddard added a comment - Technique for testing bridgeit.mobi/demo/geotrack.html Using JavaScript developer console, enter "localStorage" Select value of bridgeit.jguid, say, acec8377-5768-48ef-af6e-2984376c23a7 Run georand acec8377-5768-48ef-af6e-2984376c23a7 where georand is: #!/bin/zsh curl --header "Content-Type:application/json" -d '{ "geometry" : { "type" : "Point", "coordinates" : [ '-114.$ {RANDOM}','51.${RANDOM} ', 1091.083740234375 ] }, "type" : "Feature", "properties" : { "jguid" : "bafe2b76-08cc-4a6d-9c17-9a1bebe65037" } }' http://api.bridgeit.mobi/echo/service/echoput/geotrack-$1
        Ken Fyten made changes -
        Status Resolved [ 5 ] Closed [ 6 ]

          People

          • Assignee:
            Mircea Toma
            Reporter:
            Ted Goddard
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved: