Configuring the Hive Metastore to Use HDFS High Availability in CDH

In this task, you first set HIVE_CONF_DIR, HADOOP_CREDSTORE_PASSWORD, and AUX_CLASSPATH environmental variables for the hive metatool to reference. This action is required to successfully connect to the database. Next, you add configuration properties to hive-site.xml for connecting to the backend database. This can be done on any Hive Gateway host, Hive Metastore (HMS), or HiveServer (HS2) host. Adding these configuration properties to a limited number of hosts is recommended to limit exposing the backend Hive database username, password, and connection string.

  1. Set HIVE_CONF_DIR to the Hive MetaStore process directory
  2. Set HADOOP_CREDSTORE_PASSWORD to the same value defined in the supervisor.conf file (which is located in the Hive MetaStore process directory)
  3. Set AUX_CLASSPATH to value defined in logs/stderr.log (which is also in your Hive MetaStore process directory)
    For example: 
    export HIVE_CONF_DIR=/var/run/cloudera-scm-agent/process/4595-hive-HIVEMETASTORE
export HADOOP_CREDSTORE_PASSWORD=abcdefg1234...
export AUX_CLASSPATH=/opt/cloudera/parcels/CDH-5.13.3-1.cdh5.13.3.p0.2....
  4. Connect to any Hive Gateway, HMS, or HS2 host.
  5. Open the hive-site.xml file in /etc/hive/conf/. Add the following properties to the hive-site.xml: 
    <property>
  <name>javax.jdo.option.ConnectionURL</name> 
  <value>jdbc:mysql://[ENTER BACKEND DATABASE HOSTNAME]:[ENTER PORT]/[ENTER HIVE BACKEND DATABASE USERNAME]?useUnicode=true&characterEncoding=UTF-8</value>
</property>
<property>
  <name>javax.jdo.option.ConnectionUserName</name>
  <value>[ENTER BACKEND DATABASE USERNAME]</value>
</property>
<property>
  <name>javax.jdo.option.ConnectionPassword</name>
  <value>[ENTER BACKEND DATABASE PASSWORD]</value>
</property>

    To determine what the backend database host, port, username, or password is, in Cloudera Manager, go to Hive > Configurations. Set Category Filter to Hive Metastore Database. The password is not exposed in plaintext.

Hive metatool is now able to connect to the backend database on the hosts and return results. You do not need to export environmental variables.

For example, run the following command to connect to the database and list FS roots: 
hive --service metatool -listFSRoot
The output is: 
Listing FS Roots..
hdfs://[hostname]:8020/user/hive/warehouse

To configure other CDH components to use HDFS high availability, see Configuring Other CDH Components to Use HDFS HA.