Managing Off-Heap Memory
Tanzu GemFire can be configured to store region values in off-heap memory, which is memory within the JVM that is not subject to Java garbage collection.
Garbage collection (GC) within a JVM can prove to be a performance impediment. A server cannot exert control over when garbage collection within the JVM heap memory takes place, and the server has little control over the triggers for invocation. Off-heap memory offloads values to a storage area that is not subject to Java GC. By taking advantage of off-heap storage, an application can reduce the amount of heap storage that is subject to GC overhead.
Off-heap memory works in conjunction with the heap, it does not replace it. The keys are stored in heap memory space. Tanzu GemFire’s own memory manager handles the off-heap memory with better performance than the Java garbage collector would for certain sets of region data.
The resource manager monitors the contents of off-heap memory and invokes memory management operations in accordance with two thresholds similar to those used for monitoring the JVM heap:
On-heap and Off-heap Objects
The following objects are always stored in the JVM heap:
- Region metadata
- Entry metadata
- Subscription queue elements
The following objects can be stored in off-heap memory:
- Values - maximum value size is 2GB
- Reference counts
- List of free memory blocks
- WAN queue elements
Note: Do not use functional range indexes with off-heap data, as they are not supported. An attempt to do so generates an exception.
Off-heap storage is best suited to data patterns where:
- Stored values are relatively uniform in size
- Stored values are mostly less than 128K in size
- The usage patterns involve cycles of many creates followed by destroys or clear
- The values do not need to be frequently deserialized
- Many of the values are long-lived reference data
Be aware that Tanzu GemFire has to perform extra work to access the data stored in off-heap memory since it is stored in serialized form. This extra work may cause some use cases to run slower in an off-heap configuration, even though they use less memory and avoid garbage collection overhead. However, even with the extra deserialization, off-heap storage may give you the best performance. Features that may increase overhead include
- frequent updates
- stored values of widely varying sizes
The off-heap memory manager is efficient at handling region data values that are all the same size or are of fixed sizes. With fixed and same-sized data values allocated within the off-heap memory, freed chunks can often be re-used, and there is little or no need to devote cycles to defragmentation.
Region values that are less than or equal to eight bytes in size will not reside in off-heap memory, even if the region is configured to use off-heap memory. These very small size region values reside in the JVM heap in place of a reference to an off-heap location. This performance enhancement saves space and load time.
Controlling Off-heap Use with the Resource Manager
The Tanzu GemFire resource manager controls off-heap memory by means of two thresholds, in much the same way as it does JVM heap memory. See Using the Tanzu GemFire Resource Manager. The resource manager prevents the cache from consuming too much off-heap memory by evicting old data. If the off-heap memory manager is unable to keep up, the resource manager refuses additions to the cache until the off-heap memory manager has freed an adequate amount of memory.
The resource manager has two threshold settings, each expressed as a percentage of the total off-heap memory. Both are disabled by default.
Eviction Threshold. The percentage of off-heap memory at which eviction should begin. Evictions continue until the resource manager determines that off-heap memory use is again below the eviction threshold. Set the eviction threshold with the
eviction-off-heap-percentageregion attribute. The resource manager enforces an eviction threshold only on regions with the HEAP_LRU characteristic. If critical threshold is non-zero, the default eviction threshold is 5% below the critical threshold. If critical threshold is zero, the default eviction threshold is 80% of total off-heap memory.
The resource manager enforces eviction thresholds only on regions whose LRU eviction policies are based on heap percentage. Regions whose eviction policies based on entry count or memory size use other mechanisms to manage evictions. See Eviction for more detail regarding eviction policies.
Critical Threshold. The percentage of off-heap memory at which the cache is at risk of becoming inoperable. When cache use exceeds the critical threshold, all activity that might add data to the cache is refused. Any operation that would increase consumption of off-heap memory throws a
LowMemoryExceptioninstead of completing its operation. Set the critical threshold with the
Critical threshold is enforced on all regions, regardless of LRU eviction policy, though it can be set to zero to disable its effect.
Specifying Off-heap Memory
To use off-heap memory, specify the following options when setting up servers and regions:
- Start the JVM as described in Tuning the JVM’s Garbage Collection Parameters. In particular, set the initial and maximum heap sizes to the same value. Sizes less than 32GB are optimal when you plan to use off-heap memory.
From gfsh, start each server that will support off-heap memory with a non-zero
off-heap-memory-sizevalue, specified in megabytes (m) or gigabytes (g). If you plan to use the resource manager, specify critical threshold, eviction threshold, or (in most cases) both.
gfsh> start server --name=server1 -–initial-heap=10G -–max-heap=10G -–off-heap-memory-size=200G \ -–lock-memory=true -–critical-off-heap-percentage=90 -–eviction-off-heap-percentage=80
Mark regions whose entry values should be stored off-heap by setting the
off-heapregion attribute to
trueConfigure other region attributes uniformly for all members that host data for the same region. .
gfsh>create region --name=region1 --type=PARTITION_HEAP_LRU --off-heap=true
gfsh Off-heap Support
gfsh supports off-heap memory in server and region creation operations and in reporting functions:
--off-heap=(true | false) resets the off-heap attribute for the specified region. See alter disk-store for details.
--off-heap=(true | false)sets the off-heap attribute for the specified region. See create region for details.
displays off-heap size
shows if an off-line region is off-heap
displays the value of a region’s off-heap attribute
includes off-heap metrics
supports off-heap options
‑‑eviction-off-heap-percentage See start server for details.
org.apache.geode.cache.control.ResourceManager interface defines methods that support off-heap use:
public void setCriticalOffHeapPercentage(float Percentage)
public float getCriticalOffHeapPercentage()
public void setEvictionOffHeapPercentage(float Percentage)
public float getEvictionOffHeapPercentage()
The gemfire.properties file supports one off-heap property:
Specifies the size of off-heap memory in megabytes (m) or gigabytes (g). For example:
The cache.xml file supports one region attribute:
off-heap(=true | false)
Specifies that the region uses off-heap memory; defaults to
false. For example:
<region-attributes off-heap="true"> </region-attributes>
See <region-attributes> for details.
The cache.xml file supports two resource manager attributes:
Specifies the percentage of off-heap memory at or above which the cache is considered in danger of becoming inoperable due to out of memory exceptions. See <resource-manager> for details.
Specifies the percentage of off-heap memory at or above which eviction should begin. Can be set for any region, but actively operates only in regions configured for HEAP_LRU eviction. See <resource-manager> for details.
<cache> ... <resource-manager critical-off-heap-percentage="99.9" eviction-off-heap=-percentage="85"/> ... </cache>
Tanzu GemFire collects statistics on off-heap memory usage which you can view with the gfsh
show metrics command. See Off-Heap (OffHeapMemoryStats) for a description of available off-heap statistics.
Off-heap memory is optimized, by default, for storing values of 128 KB in size. This figure is known as the “maximum optimized stored value size,” which we will denote here by maxOptStoredValSize. If your data typically runs larger, you can enhance performance by increasing the OFF_HEAP_FREE_LIST_COUNT system parameter to a number larger than
maxOptStoredValSize/8, where maxOptStoredValSize is expressed in KB (1024 bytes). So, the default values correspond to:
128 KB / 8 = (128 * 1024) / 8 = 131,072 / 8 = 16,384 -Dgemfire.OFF_HEAP_FREE_LIST_COUNT=16384
To optimize for a maximum optimized stored value size that is twice the default, or 256 KB, the free list count should be doubled:
During the tuning process, you can toggle the
off-heap region attribute on and off, leaving other off-heap settings and parameters in place, in order to compare your application’s on-heap and off-heap performance.