Since the fixes to PUSH-244 and PUSH-245 I haven't been able to reproduce this anymore. However, when I was originally able to reproduce it, I didn't see the ConcurrentModificationException nor was I using logging at Level.FINE level. Therefore I'm in doubt that the fixes to PUSH-244 and PUSH-245 could have resolved this issue. But as I can't reproduce it anymore I'll mark this one as Cannot Reproduce. If Ted and/or Steve can reproduce it again we can re-open this one.
As Ted noted that he suspects a Cloud Push not having been confirmed correctly upon a subsequent listen.icepush request (either by the lack of that request or something at the server-side going awry) it could get into the case where the server decides not to schedule the ConfirmationTimeout due to the following if-statement:
long now = System.currentTimeMillis();
long timeout = status.getConnectionRecreationTimeout() * 2;
if (notifyBackURI.getTimestamp() + minCloudPushInterval <= now + timeout)
{
return startConfirmationTimeout(sequenceNumber, timeout);
}
else {
if (LOGGER.isLoggable(Level.FINE))
{
LOGGER.log(
Level.FINE,
"Timeout is within the minimum Cloud Push interval for URI '" + notifyBackURI + "'. (" +
"timestamp: '" + notifyBackURI.getTimestamp() + "', " +
"minCloudPushInterval: '" + minCloudPushInterval + "', " +
"now: '" + now + "', " +
"timeout: '" + timeout + "'" +
")");
}
}
If the NotifyBackURI instance doesn't get touched when expected it could happen that, especially with the 5 second Priority Push, the Server decides not to schedule the ConfirmationTimeout as part of its flood prevention logic.
The culprit here was that once the NotifyBackURI was set on the BlockingConnectionServer new PushID objects wouldn't get a NotifyBackURI set on it. The BlockingConnectionServer now always invokes the PushGroupManager.setNotifyBackURI if the incoming Request contains a ice.notifyBack. The com.icesoft.push.LocalPushGroupManager already ensures that a SetNotifyBackURI message is only send if a change has been detected. As it iterates over the Push IDs, if a Push ID's NotifyBackURI didn't change, it will be touched. Finally, the Push ID's NotifyBackURI reference is only updated if there is indeed a change. Marking this one as FIXED.