Upgrading Cloudera Manager 5 Using Tarballs

Before beginning a Cloudera Manager upgrade, do the following:

1. Review the CDH 5 and Cloudera Manager 5 Requirements and Supported Versions for the new versions you are upgrading to.
2. Read the Cloudera Manager 5 Release Notes.
3. Read the Cloudera Security Bulletins.
4. Note the following:
   • Cloudera Management Service TLS/SSL configuration

     If you have enabled TLS security for the Cloudera Manager Admin Console, as of Cloudera Manager 5.1, Cloudera Management Service roles communicate with Cloudera Manager using TLS, and fail to start until TLS/SSL properties have been configured.

   • Navigator

     If you have enabled auditing with Cloudera Navigator, during the upgrade to Cloudera Manager 5, auditing is suspended and is only restarted when you restart the roles of audited services. You will be instructed to stop some services in a later step.

5. If you have previously installed Kafka 1.2, and are upgrading from Cloudera Manager 5.4 or lower, remove the Kafka CSD:
   1. Determine the location of the CSD directory:
      1. Select Administration > Settings.
      2. Click the Custom Service Descriptors category.
      3. Retrieve the directory from the Local Descriptor Repository Path property.
   2. Delete the Kafka CSD from the directory.
6. Review package (RPM) dependencies. A Cloudera Manager upgrade might introduce new package dependencies. If your organization has restrictions or requires prior approval for installation of packages, see the list of Package Dependencies before upgrading Cloudera Manager.

If the Cloudera Manager host does not have access to the internet, or you install a version lower than the latest version of Cloudera Manager, configure access to the Cloudera Manager software from either the Cloudera public repository or a local package repository that you create.

See: Creating and Using a Package Repository for Cloudera Manager

If you are upgrading from Cloudera Navigator 2.6 or lower, upgrade the Cloudera Navigator data management component.

See Upgrading the Cloudera Navigator Data Management Component

1. If your cluster is running the embedded PostgreSQL database, stop all services that are using the embedded database. These can include:
   • Hive service and all services such as Impala and Hue that use the Hive metastore
   • Oozie
   • Sentry
2. Stop Cloudera Manager Server, Database, and Agent:
   1. Use the Cloudera Manager Admin Console to stop any running commands. These include user commands and commands Cloudera Manager automatically triggers in response to a state change or a schedule. You can either wait for commands to complete, or stop any running commands. For more information on viewing and stopping running commands, see Viewing Running and Recent Commands.
   2. On the host running the Cloudera Manager Server, stop the Cloudera Manager Server:

      sudo service cloudera-scm-server stop

   3. If you are using the embedded PostgreSQL database with Cloudera Manager, stop the database on the host where the database runs, usually the Cloudera Manager Server host:

      sudo service cloudera-scm-server-db stop

   4. If the Cloudera Manager host is also running the Cloudera Manager Agent, stop the Cloudera Manager Agent:

      sudo service cloudera-scm-agent stop

3. Download tarballs from the locations listed in Cloudera Manager Version and Download Information.
4. Copy the tarballs and unpack them on all hosts on which you intend to install Cloudera Manager Server and Cloudera Manager Agents, in a directory of your choosing. If necessary, create a new directory to accommodate the files you extract from the tarball. For instance, if /opt/cloudera-manager does not exist, create it using a command similar to:

   $ sudo mkdir /opt/cloudera-manager

5. Extract the contents of the tarball to this directory. For example, to copy a tar file to your home directory and extract the contents of all tar files to the /opt/ directory, use a command similar to the following:

   $ sudo tar xzf cloudera-manager*.tar.gz -C /opt/cloudera-manager

   The files are extracted to a subdirectory named according to the Cloudera Manager version being extracted. For example, files could be extracted to /opt/cloudera-manager/cm-5.0/. This full path is needed later and is referred to as tarball_root directory.

6. On every Cloudera Manager Agent host, configure the Cloudera Manager Agent to point to the Cloudera Manager Server by setting the following properties in the tarball_root/etc/cloudera-scm-agent/config.ini configuration file:

   Property	Description
   server_host	Name of the host where Cloudera Manager Server is running.
   server_port	Port on the host where Cloudera Manager Server is running.

7. By default, a tarball installation has a var subdirectory where state is stored. In a non-tarball installation, state is stored in /var. Cloudera recommends that you configure the tarball installation to use an external directory as the /var equivalent (/var or any other directory outside the tarball) so that when you upgrade Cloudera Manager, the new tarball installation can access this state. Configure the installation to use an external directory for storing state by editing tarball_root/etc/default/cloudera-scm-agent and setting the CMF_VAR variable to the location of the /var equivalent. If you do not reuse the state directory between different tarball installations, duplicate Cloudera Manager Agent entries can occur in the Cloudera Manager database.
8. Start Cloudera Manager Server.
   The way in which you start the Cloudera Manager Server varies depending on which account you want the Server to run under:

   • As root:

     sudo tarball_root/etc/init.d/cloudera-scm-server start 

   • As another user. If you run as another user, ensure the user you created for Cloudera Manager owns the location to which you extracted the tarball including the newly created database files. If you followed the earlier examples and created the directory /opt/cloudera-manager and the user cloudera-scm, you could use the following command to change ownership of the directory:

     sudo chown -R cloudera-scm:cloudera-scm /opt/cloudera-manager

     Once you have established ownership of directory locations, you can start Cloudera Manager Server using the user account you chose. For example, you might run the Cloudera Manager Server as cloudera-service. In this case, you have the following options:

     • Run the following command:

       $ sudo -u cloudera-service tarball_root/etc/init.d/cloudera-scm-server start 

     • Edit the configuration files so the script internally changes the user. Then run the script as root:
       1. Remove the following line from tarball_root/etc/default/cloudera-scm-server:

          export CMF_SUDO_CMD=" "

       2. Change the user and group in tarball_root/etc/init.d/cloudera-scm-server to the user you want the server to run as. For example, to run as cloudera-service, change the user and group as follows:

          USER=cloudera-service
          GROUP=cloudera-service

       3. Run the server script as root:

          $ sudo tarball_root/etc/init.d/cloudera-scm-server start 

   • To start the Cloudera Manager Server automatically after a reboot:
     1. Run the following commands on the Cloudera Manager Server host:
        • RHEL-compatible and SLES

          $ cp tarball_root/etc/init.d/cloudera-scm-server /etc/init.d/cloudera-scm-server
          $ chkconfig cloudera-scm-server on

        • Debian/Ubuntu

          $ cp tarball_root/etc/init.d/cloudera-scm-server /etc/init.d/cloudera-scm-server
          $ update-rc.d cloudera-scm-server defaults

     2. On the Cloudera Manager Server host, open the /etc/init.d/cloudera-scm-server file and change the value of CMF_DEFAULTS from ${CMF_DEFAULTS:-/etc/default} to tarball_root/etc/default.
9. To stop the Cloudera Manager Agent, run this command on each Agent host:

   $ sudo tarball_root/etc/init.d/cloudera-scm-agent hard_stop_confirmed

   If you are running single user mode, stop Cloudera Manager Agent using the user account you chose. For example, if you are running the Cloudera Manager Agent as cloudera-scm, you have the following options:
   • Run the following command:

     $ sudo -u cloudera-scm tarball_root/etc/init.d/cloudera-scm-agent hard_stop_confirmed 

   • Edit the configuration files so the script internally changes the user, and then run the script as root:
     1. Remove the following line from tarball_root/etc/default/cloudera-scm-agent:

        export CMF_SUDO_CMD=" "

     2. Change the user and group in tarball_root/etc/init.d/cloudera-scm-agent to the user you want the Agent to run as. For example, to run as cloudera-scm, change the user and group as follows:

        USER=cloudera-scm
        GROUP=cloudera-scm

     3. Run the Agent script as root:

        $ sudo tarball_root/etc/init.d/cloudera-scm-agent hard_stop_confirmed 

10. Start the Cloudera Manager Agent according to the account you want the Agent to run under:
    • To start the Cloudera Manager Agent, run this command on each Agent host:

      $ sudo tarball_root/etc/init.d/cloudera-scm-agent start

      When the Agent starts, it contacts the Cloudera Manager Server.
    • If you are running single user mode, start Cloudera Manager Agent using the user account you chose. For example, to run the Cloudera Manager Agent as cloudera-scm, you have the following options:
      • Run the following command:

        $ sudo -u cloudera-scm tarball_root/etc/init.d/cloudera-scm-agent start 

      • Edit the configuration files so the script internally changes the user, and then run the script as root:
        1. Remove the following line from tarball_root/etc/default/cloudera-scm-agent:

           export CMF_SUDO_CMD=" "

        2. Change the user and group in tarball_root/etc/init.d/cloudera-scm-agent to the user you want the Agent to run as. For example, to run as cloudera-scm, change the user and group as follows:

           USER=cloudera-scm
           GROUP=cloudera-scm

        3. Run the Agent script as root:

           $ sudo tarball_root/etc/init.d/cloudera-scm-agent start 

    • To start the Cloudera Manager Agents automatically after a reboot:
      1. Run the following commands on each Agent host:
         • RHEL-compatible and SLES

           $ cp tarball_root/etc/init.d/cloudera-scm-agent /etc/init.d/cloudera-scm-agent
           $ chkconfig cloudera-scm-agent on

         • Debian/Ubuntu

           $ cp tarball_root/etc/init.d/cloudera-scm-agent /etc/init.d/cloudera-scm-agent
           $ update-rc.d cloudera-scm-agent defaults

      2. On each Agent, open the tarball_root/etc/init.d/cloudera-scm-agent file and change the value of CMF_DEFAULTS from ${CMF_DEFAULTS:-/etc/default} to tarball_root/etc/default.
11. Log in to the Cloudera Manager Admin Console.
12. Restart all services:
    1. On the Home > Status tab, click next to the cluster name and select Restart.
    2. In the confirmation dialog box that displays, click Restart.
13. Upgrade any required Cloudera Navigator components:
    • Upgrading the Cloudera Navigator Data Management Component
    • Upgrading Cloudera Navigator Key Trustee Server
    • Upgrading Cloudera Navigator Key HSM
    • Upgrading Key Trustee KMS
    • Upgrading Cloudera Navigator Encrypt

See Upgrading Cloudera Navigator Components.

The Cloudera Navigator Data Management Component is upgraded automatically when you upgrade Cloudera Manager.

To upgrade CDH, see Upgrading CDH and Managed Services Using Cloudera Manager.