Cluster Configuration Files and Troubleshooting

When you use the cluster configuration service in GemFire, you can examine the generated configuration files in the cluster_config directory on the locator. gfsh saves configuration files at the cluster-level and at the individual group-level.

The following directories and configuration files are available on the locator running the cluster configuration service:

Cluster-level configuration
For configurations that apply to all members of a cluster, the locator creates a cluster subdirectory within the cluster_config directory (or in the cluster configuration directory when starting up the locator with the --cluster-config-dir=value parameter) specified. All servers receive this configuration when they are started using gfsh. This directory contains:

  • cluster.xml – A GemFire cache.xml file containing configuration common to all members
  • – a GemFire file containing properties common to all members
  • Jar files that are intended for deployment to all members

Group-level configuration
When you specify the --groups parameter in a gfsh command, (for example, start server or create region) the locator writes the configurations for each group in a subdirectory with the same name as the group. When you start a server that specifies one or more group names, the server receives both the cluster-level configurations and the configurations from all groups specified. This subdirectory contains:

  • <group-name>.xml – A GemFire cache.xml file containing configurations common to all members of the group
  • <group-name>.properties – A GemFire file containing properties common to all members of the group
  • Jar files that are intended for deployment to all members of the group

You can export a zip file that contains all artifacts of a cluster configuration. The zip file contains all of the files in the cluster_config (or otherwise specified) subdirectory of a locator. You can import this configuration to a new cluster. See Exporting and Importing Cluster Configurations.

Troubleshooting Tips

  • When you start a locator using gfsh, you should see the following message:

    Cluster configuration service is up and running.

    If you do not see this message, there may be a problem with the cluster configuration service. Use the status cluster-config-service command to check the status of the cluster configuration.

    • If the command returns RUNNING, the cluster configuration is running normally.
    • If the command returns WAITING, run the status locator command. The output of this command returns the cause of the WAITING status.
  • When using a cache.xml file for configuration, there is a specific order to the application of the configuration in these files. GemFire applies the cluster-wide configuration files first. Group-level configurations follow. Last will be the configuration in a member’s own configuration files (cache.xml and files).

  • If a server start fails with the following exception: ClusterConfigurationNotAvailableException, the cluster configuration service may not be in the RUNNING state. Because the server requests the cluster configuration from the locator, which is not available, the start server command fails.

  • You can determine what configurations a server received from a locator by examining the server’s log file. See Logging.

  • If a start server command specifies a cache.xml file that conflicts with the existing cluster configuration, the server startup may fail.

  • If a gfsh command fails because the cluster configuration cannot be saved, the following message displays:

    Failed to persist the configuration changes due to this command, 
    Revert the command to maintain consistency. Please use "status cluster-config-service" 
    to determine whether Cluster configuration service is RUNNING."
  • There are some types of configurations that cannot be made using gfsh. See gfsh Limitations.