ICEpdf
  1. ICEpdf
  2. PDF-218

Update Memory Manager so maxMemory is configurable.

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Trivial Trivial
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 4.2
    • Component/s: Core/Parsing
    • Labels:
      None
    • Environment:
      Embedded ICEpdf component in a large application

      Description

      We have had a couple requests for the ability to set the max memory limit of the library. Currently we use Runtime.getRuntime().maxMemory(); which is ok for the standalone Viewer RI but not so great when you use the viewer component in a larger application and want to limit the amount of memory ICEpdf uses.

      Simple to address, create a new system property org.icepdf.core.maxMemory and use Runtime.getRuntime().maxMemory(); as a default value.

        Activity

        Hide
        Patrick Corless added a comment -

        Should note that these changes should be made to the MemoryManager Class.

        Show
        Patrick Corless added a comment - Should note that these changes should be made to the MemoryManager Class.
        Hide
        Patrick Corless added a comment -

        This turned out to be a bit of a rabit hole. The memory manager system properties work a little differently then one might first thing. The maxMemory property dates back to JDK 1.1 where there was no way of getting the max memory assigned to the JDK, infact maxMemory has no effect on ICEpdf 3.x > as we can get the max memory of the jvm via JDK 1.5 support. The minMemory specifies the amount of memory that should be kept free on the heap which is a bit misleading.

        While in the memory code base it became apparent that we weren't freeing up as much memory as we could when a uses the call checkMemory(). The checkMemory() is called before a large resource is to be put into memory. However it may be some time before this call will be made leading to significant memory creep far above that set by minMemory. To help keep the memory spikes to a minimum the method MemoryManger.isLowMemory() is now called before a page is initialized. This helps keep the memory contract specified by minMemory in check.

        So if a developer is using ICEpdf in their application that has a -Xmx512m and they want ICEpdf to only ever use 128MB of memory then they would set -Dorg.icepdf.core.minMemory=368m. I did quite a bit of profiling and the core library seem to respect his. Then only exception is during a large resource load which might cause a slight spike above the desired thresh hold but the memory manager will quickly come in and straighten things out.

        Show
        Patrick Corless added a comment - This turned out to be a bit of a rabit hole. The memory manager system properties work a little differently then one might first thing. The maxMemory property dates back to JDK 1.1 where there was no way of getting the max memory assigned to the JDK, infact maxMemory has no effect on ICEpdf 3.x > as we can get the max memory of the jvm via JDK 1.5 support. The minMemory specifies the amount of memory that should be kept free on the heap which is a bit misleading. While in the memory code base it became apparent that we weren't freeing up as much memory as we could when a uses the call checkMemory(). The checkMemory() is called before a large resource is to be put into memory. However it may be some time before this call will be made leading to significant memory creep far above that set by minMemory. To help keep the memory spikes to a minimum the method MemoryManger.isLowMemory() is now called before a page is initialized. This helps keep the memory contract specified by minMemory in check. So if a developer is using ICEpdf in their application that has a -Xmx512m and they want ICEpdf to only ever use 128MB of memory then they would set -Dorg.icepdf.core.minMemory=368m. I did quite a bit of profiling and the core library seem to respect his. Then only exception is during a large resource load which might cause a slight spike above the desired thresh hold but the memory manager will quickly come in and straighten things out.
        Hide
        Patrick Corless added a comment -

        While in the MemoryManger code one more tweak became apparent with regards to the Library class's cache of Ref's. The Cache stores PDF object by PDF object number as they are loaded by the library class. This cache is never cleared resulting in a slow memory leak. We do a punch of clean up with the dispose() method on various PDF classes but never take the time to remove the objects from the library. So I tried removing key classes from the Library's refs cache. The result were pretty good, there is still little plead do to how we load the some object but it is greatly reduced to depending on the memory configurations.

        I did one more test to see how the memory manger handles two Documents instances. A performance hit was diffently seen as a lot more time was spend in gc but the Memory manager seemed to respect the minMemory setting.

        Show
        Patrick Corless added a comment - While in the MemoryManger code one more tweak became apparent with regards to the Library class's cache of Ref's. The Cache stores PDF object by PDF object number as they are loaded by the library class. This cache is never cleared resulting in a slow memory leak. We do a punch of clean up with the dispose() method on various PDF classes but never take the time to remove the objects from the library. So I tried removing key classes from the Library's refs cache. The result were pretty good, there is still little plead do to how we load the some object but it is greatly reduced to depending on the memory configurations. I did one more test to see how the memory manger handles two Documents instances. A performance hit was diffently seen as a lot more time was spend in gc but the Memory manager seemed to respect the minMemory setting.
        Hide
        Patrick Corless added a comment -

        I wanted to try and quantify the saving of being a little more proactive about when we flush pages as well as clearing the library ref cache programatically. When comparing memory usage between 4.1.4 and the trunk the results are about the same, the memory manager system properties do what they say they do.

        However when you look at time spent doing Garbage Collection the differences quickly become apparent. 4.1.4 spends between 60 and 70 percent of the time doing collection. Where the trunk spends 10 and 30 percent doing collection. This difference can really bee seen when scrolling though a large documents, the trunk feels a lot snappier.

        Closing issue.

        Show
        Patrick Corless added a comment - I wanted to try and quantify the saving of being a little more proactive about when we flush pages as well as clearing the library ref cache programatically. When comparing memory usage between 4.1.4 and the trunk the results are about the same, the memory manager system properties do what they say they do. However when you look at time spent doing Garbage Collection the differences quickly become apparent. 4.1.4 spends between 60 and 70 percent of the time doing collection. Where the trunk spends 10 and 30 percent doing collection. This difference can really bee seen when scrolling though a large documents, the trunk feels a lot snappier. Closing issue.

          People

          • Assignee:
            Patrick Corless
            Reporter:
            Patrick Corless
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved: