Maintenance Mode

Minimum Required Role: Configurator (also provided by Cluster Administrator, Full Administrator)

Maintenance mode allows you to suppress alerts for a host, service, role, or an entire cluster. This can be useful when you need to take actions in your cluster (make configuration changes and restart various elements) and do not want to see the alerts that will be generated due to those actions.

Putting an entity into maintenance mode does not prevent events from being logged; it only suppresses the alerts that those events would otherwise generate. You can see a history of all the events that were recorded for entities during the period that those entities were in maintenance mode.

Explicit and Effective Maintenance Mode

When you enter maintenance mode on an entity (cluster, service, or host) that has subordinate entities (for example, the roles for a service) the subordinate entities are also put into maintenance mode. These are considered to be in effective maintenance mode, as they have inherited the setting from the higher-level entity.

For example:
  • If you set the HBase service into maintenance mode, then its roles (HBase Master and all RegionServers) are put into effective maintenance mode.
  • If you set a host into maintenance mode, then any roles running on that host are put into effective maintenance mode.

Entities that have been explicitly put into maintenance mode show the icon . Entities that have entered effective maintenance mode as a result of inheritance from a higher-level entity show the icon .

When an entity (role, host or service) is in effective maintenance mode, it can only be removed from maintenance mode when the higher-level entity exits maintenance mode. For example, if you put a service into maintenance mode, the roles associated with that service are entered into effective maintenance mode, and remain in effective maintenance mode until the service exits maintenance mode. You cannot remove them from maintenance mode individually.

Alternatively, an entity that is in effective maintenance mode can be put into explicit maintenance mode. In this case, the entity remains in maintenance mode even when the higher-level entity exits maintenance mode. For example, suppose you put a host into maintenance mode, (which puts all the roles on that host into effective maintenance mode). You then select one of the roles on that host and put it explicitly into maintenance mode. When you have the host exit maintenance mode, that one role remains in maintenance mode. You need to select it individually and specifically have it exit maintenance mode.

Viewing Maintenance Mode Status

You can view the status of Maintenance Mode in your cluster by clicking to the right of the cluster name and selecting View Maintenance Mode Status.

Entering Maintenance Mode

You can enable maintenance mode for a cluster, service, role, or host.

Putting a Cluster into Maintenance Mode

  1. Click to the right of the cluster name and select Enter Maintenance Mode.
  2. Confirm that you want to do this.

The cluster is put into explicit maintenance mode, as indicated by the icon. All services and roles in the cluster are entered into effective maintenance mode, as indicated by the icon.

Putting a Service into Maintenance Mode

  1. Go to the service page in Cloudera Manager.
  2. Click Actions > Enter Maintenance Mode.
  3. Confirm that you want to do this.

The service is put into explicit maintenance mode, as indicated by the icon. All roles for the service are entered into effective maintenance mode, as indicated by the icon.

Putting Roles into Maintenance Mode

  1. Go to the service page that includes the role.
  2. Go to the Instances tab.
  3. Select the role(s) you want to put into maintenance mode.
  4. From the Actions for Selected menu, select Enter Maintenance Mode.
  5. Confirm that you want to do this.

The roles will be put in explicit maintenance mode. If the roles were already in effective maintenance mode (because its service or host was put into maintenance mode) the roles will now be in explicit maintenance mode. This means that they will not exit maintenance mode automatically if their host or service exits maintenance mode; they must be explicitly removed from maintenance mode.

Putting Hosts into Maintenance Mode

  1. In Cloudera Manager, select the cluster where you want to decommission hosts.
  2. Click Hosts > All Hosts.
  3. Select the hosts that you want to put into Maintenance Mode.
  4. Select Actions for Selected > Begin Maintenance (Suppress Alerts/Decommission.

    The Begin Maintenance (Suppress Alerts/Decommission) dialog box opens. The role instances running on the hosts display at the top. You can also use this dialog box to decommission the host. See Tuning and Troubleshooting Host Decommissioning.

  5. Deselect the Decommission Host(s) option to put the host into Maintenance Mode. In this mode, alerts from the hosts are suppressed until the host exits Maintenance Mode. The events, however, are still logged. Hosts that are currently in Maintenance Mode display the icon.
  6. Click Begin Maintenance.

    The Host Decommission Command dialog box opens and displays the progress of the command.

Exiting Maintenance Mode

When you exit maintenance mode, the maintenance mode icons are removed and alert notification resumes.

Exiting a Cluster from Maintenance Mode

  1. Click to the right of the cluster name and select Exit Maintenance Mode.
  2. Confirm that you want to do this.

Exiting a Service from Maintenance Mode

  1. Click to the right of the service name and select Exit Maintenance Mode.
  2. Confirm that you want to do this.

Exiting Roles from Maintenance Mode

  1. Go to the services page that includes the role.
  2. Go to the Instances tab.
  3. Select the role(s) you want to exit from maintenance mode.
  4. From the Actions for Selected menu, select Exit Maintenance Mode.
  5. Confirm that you want to do this.

Taking Hosts out of Maintenance Mode

  1. In Cloudera Manager, to go the cluster with the hosts you want to take out of Maintenance Mode.
  2. Click Hosts > All Hosts.
  3. Select the hosts that are ready to exit Maintenance Mode.
  4. Select Actions for Selected > End Maintenance (Suppress Alerts/Decommission.

    The End Maintenance (Suppress Alerts/Decommission) dialog box opens. The role instances running on the hosts display at the top.

  5. Deselect the Recommission Host(s) option to take the host out of Maintenance Mode and re-enable alerts from the hosts. Hosts that are currently in Maintenance Mode display the icon on the All Hosts page.
  6. Click End Maintenance.