Enabling and Disabling Fair Scheduler Preemption

You can enable the Fair Scheduler to preempt applications in other pools if a pool's fair or minimum share is not met for some period of time. When you create a pool, you can specify whether preemption is allowed and whether a specific pool can be preempted. Fair scheduler preemption is controlled by several properties. For more information, see Configuring the Fair Scheduler.

Enabling and Disabling Preemption Using Cloudera Manager

Minimum Required Role: Configurator (also provided by Cluster Administrator, Full Administrator)

Enabling Preemption

  1. Select Clusters > Cluster name > Dynamic Resource Pool Configuration. If the cluster has a YARN service, the YARN > Resource Pools tab displays.
  2. Click Default Settings.
  3. Click the Enable Fair Scheduler Preemption link.
  4. Select the ResourceManager Default Group checkbox.
  5. Click Save Changes to commit the changes.
  6. Return to the Home page by clicking the Cloudera Manager logo.
  7. Click the icon next to any stale services to invoke the cluster restart wizard.
  8. Click Restart Stale Services.
  9. Click Restart Now.
  10. Click Finish.

Disabling Preemption

  1. Select Clusters > Cluster name > Dynamic Resource Pool Configuration. If the cluster has a YARN service, the YARN > Resource Pools tab displays.
  2. In Cloudera Manager select the YARN service.
  3. Clear the Enable Fair Scheduler Preemption checkbox.
  4. Click Save Changes to commit the changes.
  5. Return to the Home page by clicking the Cloudera Manager logo.
  6. Click the icon next to any stale services to invoke the cluster restart wizard.
  7. Click Restart Stale Services.
  8. Click Restart Now.
  9. Click Finish.

Enabling and Disabling Preemption for a Pool

  1. Select Clusters > Cluster name > Dynamic Resource Pool Configuration. If the cluster has a YARN service, the YARN > Resource Pools tab displays.
  2. In a pool row, click Edit.
  3. Click the Preemption tab.
  4. Select or clear Preemptable.
  5. Click Save.
  6. Click Refresh Dynamic Resource Pools.

Enabling and Disabling Preemption Using the Command Line

To enable and disable preemption, set yarn.scheduler.fair.preemption in yarn-site.xml: 
<property>
    <name>yarn.scheduler.fair.preemption</name>
    <value>true</value>
</property>
To enable or disable a specific pool from being preempted, set the allowPreemptionFrom property in fair-scheduler.xml. The following example disables preemption from the important pool:
<queue name="important">
    <weight>1.0</weight>
    <allowPreemptionFrom>false</allowPreemptionFrom>
    <schedulingPolicy>drf</schedulingPolicy>
    <aclSubmitApps>*</aclSubmitApps>
    <aclAdministerApps>*</aclAdministerApps>
</queue>