Role Instances

Continue reading:

Adding a Role Instance

Minimum Required Role: Cluster Administrator (also provided by Full Administrator)

After creating services, you can add role instances to the services. For example, after initial installation in which you created the HDFS service, you can add a DataNode role instance to a host where one was not previously running. Upon upgrading a cluster to a new version of CDH you might want to create a role instance for a role added in the new version.

  1. Go to the service for which you want to add a role instance. For example, to add a DataNode role instance, go to the HDFS service.
  2. Click the Instances tab.
  3. Click the Add Role Instances button.
  4. Customize the assignment of role instances to hosts. The wizard evaluates the hardware configurations of the hosts to determine the best hosts for each role. The wizard assigns all worker roles to the same set of hosts to which the HDFS DataNode role is assigned. You can reassign role instances.

    Click a field below a role to display a dialog box containing a list of hosts. If you click a field containing multiple hosts, you can also select All Hosts to assign the role to all hosts, or Custom to display the hosts dialog box.

    The following shortcuts for specifying hostname patterns are supported:
    • Range of hostnames (without the domain portion)
      Range Definition Matching Hosts
      10.1.1.[1-4] 10.1.1.1, 10.1.1.2, 10.1.1.3, 10.1.1.4
      host[1-3].company.com host1.company.com, host2.company.com, host3.company.com
      host[07-10].company.com host07.company.com, host08.company.com, host09.company.com, host10.company.com
    • IP addresses
    • Rack name

    Click the View By Host button for an overview of the role assignment by hostname ranges.

  5. Click Continue.
  6. In the Review Changes page, review the configuration changes to be applied. Confirm the settings entered for file system paths. The file paths required vary based on the services to be installed. For example, you might confirm the NameNode Data Directory and the DataNode Data Directory for HDFS. Click Continue. The wizard finishes by performing any actions necessary to prepare the cluster for the new role instances. For example, new DataNodes are added to the NameNode dfs_hosts_allow.txt file. The new role instance is configured with the default role group for its role type, even if there are multiple role groups for the role type. If you want to use a different role group, follow the instructions in Managing Role Groups for moving role instances to a different role group. The new role instances are not started automatically.

Starting, Stopping, and Restarting Role Instances

Minimum Required Role: Operator (also provided by Configurator, Cluster Administrator, Full Administrator)

If the host for the role instance is currently decommissioned, you will not be able to start the role until the host has been recommissioned.
  1. Go to the service that contains the role instances to start, stop, or restart.
  2. Click the Instances tab.
  3. Check the checkboxes next to the role instances to start, stop, or restart (such as a DataNode instance).
  4. Select Actions for Selected > Start, Stop, or Restart, and then click Start, Stop, or Restart again to start the process. When you see a Finished status, the process has finished.
Also see Rolling Restart.

Decommissioning Role Instances

Minimum Required Role: Operator (also provided by Configurator, Cluster Administrator, Full Administrator)

You can remove a role instance such as a DataNode from a cluster while the cluster is running by decommissioning the role instance. When you decommission a role instance, Cloudera Manager performs a procedure so that you can safely retire a host without losing data. Role decommissioning applies to HDFS DataNode, MapReduce TaskTracker, YARN NodeManager, and HBase RegionServer roles.

Hosts with DataNodes and DataNode roles themselves can only be decommissioned if the resulting action leaves enough DataNodes commissioned to maintain the configured HDFS replication factor (by default 3). If you attempt to decommission a DataNode or a host with a DataNode in such situations, the decommission process will not complete and must be aborted.

A role will be decommissioned if its host is decommissioned. See Tuning and Troubleshooting Host Decommissioning for more details.

To remove a DataNode from the cluster, you decommission the DataNode role as described here and then perform a few additional steps to remove the role. See Removing a DataNode.

To decommission role instances:
  1. If you are decommissioning DataNodes, perform the steps in Tuning HDFS Prior to Decommissioning DataNodes.
  2. Click the service instance that contains the role instance you want to decommission.
  3. Click the Instances tab.
  4. Check the checkboxes next to the role instances to decommission.
  5. Select Actions for Selected > Decommission, and then click Decommission again to start the process. A Decommission Command pop-up displays that shows each step or decommission command as it is run. In the Details area, click to see the subcommands that are run. Depending on the role, the steps may include adding the host to an "exclusions list" and refreshing the NameNode, JobTracker, or NodeManager; stopping the Balancer (if it is running); and moving data blocks or regions. Roles that do not have specific decommission actions are stopped.

    You can abort the decommission process by clicking the Abort button, but you must recommission and restart the role.

    The Commission State facet in the Filters list displays Decommissioning while decommissioning is in progress, and Decommissioned when the decommissioning process has finished. When the process is complete, a is added in front of Decommission Command.

Recommissioning Role Instances

Minimum Required Role: Operator (also provided by Configurator, Cluster Administrator, Full Administrator)

  1. Click the service that contains the role instance you want to recommission.
  2. Click the Instances tab.
  3. Check the checkboxes next to the decommissioned role instances to recommission.
  4. Select Actions for Selected > Recommission, and then click Recommission to start the process. A Recommission Command pop-up displays that shows each step or recommission command as it is run. When the process is complete, a is added in front of Recommission Command.
  5. Restart the role instance.

Deleting Role Instances

Minimum Required Role: Cluster Administrator (also provided by Full Administrator)

  1. Click the service instance that contains the role instance you want to delete. For example, if you want to delete a DataNode role instance, click an HDFS service instance.
  2. Click the Instances tab.
  3. Check the checkboxes next to the role instances you want to delete.
  4. If the role instance is running, select Actions for Selected > Stop and click Stop to confirm the action.
  5. Select Actions for Selected > Delete. Click Delete to confirm the deletion.

Configuring Roles to Use a Custom Garbage Collection Parameter

Every Java-based role in Cloudera Manager has a configuration setting called Java Configuration Options for role where you can enter command line options. Commonly, garbage collection flags or extra debugging flags would be passed here. To find the appropriate configuration setting, select the service you want to modify in the Cloudera Manager Admin Console, then use the Search box to search for Java Configuration Options.

You can add configuration options for all instances of a given role by making this configuration change at the service level. For example, to modify the setting for all DataNodes, select the HDFS service, then modify the Java Configuration Options for DataNode setting.

To modify a configuration option for a given instance of a role, select the service, then select the particular role instance (for example, a specific DataNode). The configuration settings you modify will apply to the selected role instance only.

For detailed instructions see Modifying Configuration Properties Using Cloudera Manager.