Setting Up KMS Using the Command Line

Hadoop Key Management Server (KMS) is a cryptographic key management server based on the Hadoop KeyProvider API. It provides a KeyProvider implementation client that interacts with the KMS using the HTTP REST API. Both the KMS and its client support HTTP SPNEGO Kerberos authentication and TLS/SSL-secured communication. The KMS is a Java-based web application.

Cloudera provides the following implementations of the Hadoop KMS:
  • Java KeyStore KMS - The default Hadoop KMS included in CDH that uses a file-based Java KeyStore (JKS) for its backing keystore. For parcel-based installations, no additional action is required to install or upgrade the KMS. For package-based installations, you must install additional packages. For more information, see Installing and Upgrading Java KeyStore KMS. Cloudera strongly recommends not using Java Keystore KMS in production environments.
  • Key Trustee KMS - A custom KMS that uses Cloudera Navigator Key Trustee Server for its backing keystore instead of the file-based Java KeyStore (JKS) used by the default Hadoop KMS. Cloudera strongly recommends using Key Trustee KMS in production environments to improve the security, durability, and scalability of your cryptographic key management. For more information about the architecture and components involved in encrypting data at rest for production environments, see Cloudera Navigator Data Encryption Overview and Data at Rest Encryption Reference Architecture. For instructions on installing and upgrading Key Trustee KMS, see: Also, integrating Key Trustee Server with Cloudera Navigator Key HSM provides an additional layer of protection.
  • Navigator KMS Services backed by Thales HSM - A custom KMS that uses a supported Thales Hardware Security Module (HSM) as its backing keystore. This KMS service provides the highest level of key isolation to customers who require it.

    For installation information about Navigator KMS Services backed by Thales HSM, see Installing Navigator HSM KMS Backed by Thales HSM.

  • Navigator KMS Services backed by Luna HSM - A custom KMS that uses a supported Luna Hardware Security Module (HSM) as its backing keystore. This KMS provides the highest level of key isolation to customers who require it.

    For installation information about Navigator KMS Services backed by Luna HSM, see Installing Navigator HSM KMS Backed by Luna HSM.

Installing and Upgrading Java KeyStore KMS

To install or upgrade Java KeyStore KMS on a RHEL-compatible system:

$ sudo yum install hadoop-kms hadoop-kms-server

To install or upgrade Java KeyStore KMS on a SLES system:

$ sudo zypper install hadoop-kms hadoop-kms-server

To install or upgrade Java KeyStore KMS on an Ubuntu or Debian system:

$ sudo apt-get install hadoop-kms hadoop-kms-server

Troubleshooting: Upgrading hadoop-kms from 5.2.x and 5.3.x Releases on SLES

This section describes issues that affect SLES upgrades from 5.2.x releases earlier than 5.2.4, and from 5.3.x releases earlier than 5.3.2.

Problem

The problem occurs when you try to upgrade the hadoop-kms package, for example:

Installing: hadoop-kms-2.5.0+cdh5.3.2+801-1.cdh5.3.2.p0.224.sles11 [error]
12:54:19 Installation of hadoop-kms-2.5.0+cdh5.3.2+801-1.cdh5.3.2.p0.224.sles11 failed:
12:54:19 (with --nodeps --force) Error: Subprocess failed. Error: RPM failed: warning: /var/cache/zypp/packages/cdh/RPMS/x86_64/hadoop-kms-2.5.0+cdh5.3.2+801-1.cdh5.3.2.p0.224.sles11.x86_64.rpm: Header V4 DSA signature: NOKEY, key ID e8f86acd
12:54:19 error: %postun(hadoop-kms-2.5.0+cdh5.3.1+791-1.cdh5.3.1.p0.17.sles11.x86_64) scriptlet failed, exit status 1
12:54:19

What to Do

If you see an error similar to the one in the example above, proceed as follows:
  1. Abort or ignore the error (either option works): 
    Abort, retry, ignore? [a/r/i] (a): i
  2. Perform cleanup:
    1. # rpm -qa hadoop-kms
      You will see two versions of hadoop-kms; for example: 
      hadoop-kms-2.5.0+cdh5.3.1+791-1.cdh5.3.1.p0.17.sles11
hadoop-kms-2.5.0+cdh5.3.2+801-1.cdh5.3.2.p0.224.sles11
    2. Remove the older version, in this example hadoop-kms-2.5.0+cdh5.3.1+791-1.cdh5.3.1.p0.17.sles11: 
       # rpm -e --noscripts hadoop-kms-2.5.0+cdh5.3.1+791-1.cdh5.3.1.p0.17.sles11
  3. Verify that the older version of the package has been removed: 
    # rpm -qa hadoop-kms
    You should now see only the newer package: 
    hadoop-kms-2.5.0+cdh5.3.2+801-1.cdh5.3.2.p0.224.sles11