How Disk Stores Work

How Disk Stores Work

The overflow feature uses disk stores as an extension of the in-memory cache. Persistence uses disk stores to store a redundant copy of the data.

These two options can be used individually or together.

Disk storage is available for these cached data types:
  • Cached regions. Persist and/or overflow data from your cached data regions.
  • Server’s client subscription queues. Overflow the messaging queues to control memory use.
  • Gateway sender queues. Persist these for high availability. These queues always overflow.
  • PDX serialization metadata. Persist metadata about objects you serialize using GemFire PDX serialization.

Each member has its own set of disk stores, completely separate from the disk stores of any other member. For each disk store, you define where and how the data is stored to disk. You can store data from multiple regions and queues in a single disk store.

This figure shows a member with disk stores D through R defined. The member has two persistent regions using disk store D and an overflow region and an overflow queue using disk store R.

What GemFire Writes to the Disk Store

The following list describes the items that GemFire writes to the disk store:
  • List of members that host the store and information on their status, such as running or offline and time stamps.
  • List of regions in the disk store. For cached regions, these are the names you give the regions. For all other data, these are GemFire internal region names.
  • For each region:
    • Region configuration attributes pertaining to loading and capacity management, used to load the data quickly on startup.
    • Region data operations.

GemFire does not write indexes to disk.

Disk Store State

Disk store access and management differs according to whether the store is online or offline.

While a member is running, its disk stores are online in the GemFire system. When the member closes its cache and exits, its disk stores go offline. When the member starts up again, its disk stores come back online.
  • Online, a disk store is owned and managed by its member process. To run operations on an online disk store, use API calls in the member process or use the gfsh command-line interface. The gfsh command-line interface joins the distributed system and sends requests to members that have disk stores.
  • Offline, the disk store is just a collection of files in your host file system. The files are open to access by anyone with the right file system permissions. You can copy the files using your file system commands, for backup or to move your member’s disk store location. You can also run some maintenance operations on the offline disk store, like file compaction and validation by using the gfsh command-line interface.
    Note: The files for a disk store are used by GemFire as a group. Treat them as a single entity. If you copy them, copy them all together. Do not change the file names.
    When a disk store is offline, its data is unavailable to the GemFire distributed system. For partitioned regions, the data is split between multiple members, so you may be able to access the region, but have some of its data only present in an offline disk store. If you try to access an entry whose only copy is stored on disk by an offline member, the operation returns a PartitionOfflineException.