Configuring Lily HBase Indexer Security

To configure the Lily HBase Indexer to use Kerberos authentication, you must create principals and keytabs and then modify certain configuration properties. If you are using Cloudera Manager to manage your cluster, much of this is handled automatically. For unmanaged environments, you must generate the Kerberos principals and keytabs manually.

For more an overview of Kerberos concepts (including principals and keytabs), see Kerberos Security Artifacts Overview.

To enable Kerberos authentication for the Lily HBase Indexer service:

Cloudera Manager:

1. Go to Key-Value Store Indexer service > Configuration > Category > Security.
2. Select the kerberos option for HBase Indexer Secure Authentication.
3. Click Save Changes.
4. Go to Administration > Security > Kerberos Credentials.
5. Click Generate Missing Credentials.
6. Restart the indexer service (Key-Value Store Indexer service > Actions > Restart).

Unmanaged:

1. Create principals and keytabs. To complete these steps you must have Kerberos administrator credentials. Perform this procedure on each Lily HBase Indexer host:
   1. Launch the kadmin utility after authenticating with your Kerberos administrator credentials. For example:

      $ kinit jdoe/admin@EXAMPLE.COM
      Password for jdoe/admin@EXAMPLE.COM:
      $ kadmin
      Authenticating as principal jdoe/admin@EXAMPLE.COM with password.
      Password for jdoe/admin@EXAMPLE.COM:
      kadmin:  

   2. Create Lily HBase Indexer service user principals using the format hbase/lily01.example.com@EXAMPLE.COM. This principal is used to authenticate with the Hadoop cluster. Replace lily01.example.com with the Lily HBase Indexer host, and EXAMPLE.COM with your Kerberos realm name:

      kadmin: addprinc -randkey hbase/lily01.example.com@EXAMPLE.COM

   3. Create an HTTP service user principal using the format HTTP/lily01.example.com@EXAMPLE.COM. This principal is used to authenticate user requests coming to the Lily HBase Indexer web services. Replace lily01.example.com with the Lily HBase Indexer host, and EXAMPLE.COM with your Kerberos realm name:

      kadmin: addprinc -randkey HTTP/lily01.example.com@EXAMPLE.COM

   4. Create a keytab file containing both the HTTP/ and hbase/ principals. For example:

      kadmin: xst -norandkey -k /tmp/hbase.keytab hbase/lily01.example.com@EXAMPLE.COM \
      HTTP/lily01.example.com@EXAMPLE.COM

   5. Exit the kadmin utility by using the quit or exit command.
   6. Check the keytabs to make sure that both the HTTP/ and hbase/ principals are present, and that they have the same hostname component. For example:

      $ klist -ekt /tmp/hbase.keytab

   7. Copy the hbase.keytab file to the Lily HBase Indexer configuration directory. Make sure that the owner of the hbase.keytab file is the hbase user and the file has owner-only read permissions.
   8. Repeat this procedure on each Lily HBase Indexer host.
2. Configure each Lily HBase Indexer host to use Kerberos authentication:
   1. Modify the hbase-indexer-site.xml file as follows:

        <property>
          <name>hbaseindexer.authentication.type</name>
          <value>kerberos</value>
        </property>
        <property>
          <name>hbaseindexer.authentication.kerberos.keytab</name>
          <value>hbase.keytab</value>
        </property>
        <property>
          <name>hbaseindexer.authentication.kerberos.principal</name>
          <value>HTTP/lily01.example.com@EXAMPLE.COM</value>
        </property>
        <property>
          <name>hbaseindexer.authentication.kerberos.name.rules</name>
          <value>DEFAULT</value>
        </property>

   2. Set up the Java Authentication and Authorization Service (JAAS) configuration file. Create a jaas.conf file in the configuration directory containing the following settings. Replace the values with the correct ones for your environment:

      Client {
        com.sun.security.auth.module.Krb5LoginModule required
        useKeyTab=true
        useTicketCache=false
        keyTab="/etc/hbase-solr/conf/hbase.keytab"
        principal="hbase/lily01.example.com@EXAMPLE.COM";
      };

   3. Modify hbase-indexer-env.sh in the configuration directory to add the JAAS configuration to the system properties. You can do this by adding -Djava.security.auth.login.config to the HBASE_INDEXER_OPTS. For example:

      HBASE_INDEXER_OPTS = "$HBASE_INDEXER_OPTS -Djava.security.auth.login.config=/etc/hbase-solr/conf/jaas.conf"

   4. Make these changes on each Lily HBase Indexer host.

The Lily HBase Indexer service uses Apache Sentry for authorization. To use Sentry for authorization, you must use the indexer HTTP interface.

If you are using policy files for Sentry, and want to switch to the Sentry service, see Migrating From Sentry Policy Files to the Sentry Service.

To configure the Lily HBase Indexer to use the Sentry Service:

Cloudera Manager:

1. Go to Key-Value Store Indexer service > Configuration > Category > Policy File Based Sentry.
2. Make sure that the box labeled Enable Sentry Authorization using Policy Files is unchecked.
3. Click the Main category in the left pane.
4. Select the Sentry Service for the cluster (SENTRY-1 by default).
5. Click Save Changes.
6. Restart stale services (Key-Value Store Indexer service > Actions > Restart).

Unmanaged:

1. Add the following properties to hbase-indexer-site.xml on each Lily HBase Indexer host:

     <property>
       <name>sentry.hbaseindexer.sentry.site</name>
       <value>/path/to/sentry-site.xml</value>
     </property>
     <property>
       <name>hbaseindexer.rest.resource.package</name>
       <value>org/apache/sentry/binding/hbaseindexer/rest</value>
     </property>

2. Edit the specified sentry-site.xml file (on each Lily HBase Indexer host) to add or modify the following properties:

     <property>
       <name>sentry.service.server.principal</name>
       <value>sentry/_HOST@EXAMPLE.COM</value>
     </property>
     <property>
       <name>sentry.service.security.mode</name>
       <value>kerberos</value>
     </property>
     <property>
       <name>sentry.service.client.server.rpc-address</name>
       <value>sentry-1.example.com</value>
     </property>
     <property>
       <name>sentry.service.client.server.rpc-port</name>
       <value>8038</value>
     </property>
     <property>
       <name>sentry.hbaseindexer.provider.backend</name>
       <value>org.apache.sentry.provider.db.generic.SentryGenericProviderBackend</value>
     </property>
     <property>
       <name>sentry.provider.backend.generic.service-name</name>
       <value>hbase-indexer-1</value>
     </property>

3. Restart the Lily HBase Indexer service on each host:

   $ sudo service hbase-solr-indexer restart

The Lily HBase Indexer service uses Apache Sentry for authorization. To use Sentry for authorization, you must use the indexer HTTP interface.

Before CDH 5.14.0, Lily HBase Indexer supported only Sentry policy files. In CDH 5.14.0 and higher, it supports the Sentry service, and includes a command line utility (hbase-indexer-sentry) for configuring Sentry. Cloudera recommends using the Sentry Service. To migrate your policy files to the Sentry Service, see Migrating From Sentry Policy Files to the Sentry Service.

To configure Sentry policy files for the Lily HBase Indexer:

Cloudera Manager:

1. Go to Key-Value Store Indexer service > Configuration > Category > Policy File Based Sentry.
2. Check the box labeled Enable Sentry Authorization using Policy Files.
3. If necessary, edit Sentry Global Policy File to change the HDFS location of the sentry-provider.ini file.
4. Click Save Changes.
5. Restart the service (Key-Value Store Indexer service > Actions > Restart).
6. Upload the sentry-provider.ini file to the specified location in HDFS. For example:
   • Security Enabled:

     $ kinit hdfs@EXAMPLE.COM
     $ hdfs dfs -mkdir -p /user/hbaseindexer/sentry/
     $ hdfs dfs -put /path/to/local/sentry-provider.ini /user/hbaseindexer/sentry/
     $ hdfs dfs -chown -R hbase:hbase /user/hbaseindexer

   • Security Disabled:

     $ sudo -u hdfs hdfs dfs -mkdir -p /user/hbaseindexer/sentry/
     $ sudo -u hdfs hdfs dfs -put /path/to/local/sentry-provider.ini /user/hbaseindexer/sentry/
     $ sudo -u hdfs hdfs dfs -chown -R hbase:hbase /user/hbaseindexer

Unmanaged:

1. Add the following properties to hbase-indexer-site.xml on each Lily HBase Indexer host:

     <property>
       <name>sentry.service.server.principal</name>
       <value>sentry/_HOST@EXAMPLE.COM</value>
     </property>
     <property>
       <name>sentry.service.security.mode</name>
       <value>kerberos</value>
     </property>
     <property>
       <name>sentry.hbaseindexer.sentry.site</name>
       <value>/path/to/sentry-site.xml</value>
     </property>
     <property>
       <name>hbaseindexer.rest.resource.package</name>
       <value>org/apache/sentry/binding/hbaseindexer/rest</value>
     </property>

2. Edit the referenced sentry-site.xml file to set the HDFS location of the sentry-provider.ini policy file:

     <property>
       <name>sentry.hbaseindexer.provider.resource</name>
       <value>/user/hbaseindexer/sentry/sentry-provider.ini</value>
     </property>
   

3. Upload the sentry-provider.ini file to the specified location in HDFS. For example:
   • Security Enabled:

     $ kinit hdfs@EXAMPLE.COM
     $ hdfs dfs -mkdir -p /user/hbaseindexer/sentry/
     $ hdfs dfs -put /path/to/local/sentry-provider.ini /user/hbaseindexer/sentry/
     $ hdfs dfs -chown -R hbase:hbase /user/hbaseindexer

   • Security Disabled:

     $ sudo -u hdfs hdfs dfs -mkdir -p /user/hbaseindexer/sentry/
     $ sudo -u hdfs hdfs dfs -put /path/to/local/sentry-provider.ini /user/hbaseindexer/sentry/
     $ sudo -u hdfs hdfs dfs -chown -R hbase:hbase /user/hbaseindexer

4. Restart the Lily HBase Indexer service on each host:

   $ sudo service hbase-solr-indexer restart

The Lily HBase Indexer privilege model specifies READ and WRITE privileges for each indexer. The privileges work as follows:

• If a role has WRITE privilege for indexer1, that role can create, update, or delete an indexer named indexer1, using the hbase-indexer command.
• READ privileges control what a user can see when running the hbase-indexer list-indexers command. If a role has READ privileges for indexer1, the command output lists indexer1 if it exists. If an indexer named indexer2 exists, but the role does not have READ privileges for it, that indexer is omitted from the response.

For example, consider the following privileges defined in a policy file:

[groups]
jdoe = admin
psherman = readonly

[roles]
admin = indexer=*
readonly = indexer=*->action=READ

This policy file grants the jdoe user full access to all indexers, and the psherman user read access to all indexers. User psherman can see all indexers, but cannot create new ones or modify existing ones.

Before CDH 5.14.0, Lily HBase Indexer supported only Sentry policy files. In CDH 5.14.0 and higher, it supports the Sentry service, and includes a command line utility (hbase-indexer-sentry) for configuring Sentry. The command syntax is as follows:

/opt/cloudera/parcels/CDH/bin/hbase-indexer-sentry
Missing required option: [-lp List privilege, -rpr Revoke privilege from role, -lr List role, -arg Add role to group, -drg Delete role from group, -gpr Grant privilege to role, -mgr Migrate ini file to Sentry service, -dr Drop role, -cr Create role]
usage: sentryShell
 -arg,--add_role_group          Add role to group
 -c,--checkcompat               Check compatibility with Sentry Service
 -conf,--sentry_conf <arg>      sentry-site file path
 -cr,--create_role              Create role
 -dr,--drop_role                Drop role
 -drg,--delete_role_group       Delete role from group
 -f,--policy_ini <arg>          Policy file path
 -g,--groupname <arg>           Group name
 -gpr,--grant_privilege_role    Grant privilege to role
 -h,--help                      Shell usage
 -i,--import                    Import policy file
 -lp,--list_privilege           List privilege
 -lr,--list_role                List role
 -mgr,--migrate                 Migrate ini file to Sentry service
 -p,--privilege <arg>           Privilege string
 -r,--rolename <arg>            Role name
 -rpr,--revoke_privilege_role   Revoke privilege from role
 -s,--service <arg>             Name of the service being managed
 -t,--type <arg>                [hive|solr|sqoop|.....]
 -v,--validate                  Validate policy file

If you have upgraded to CDH 5.14.0 or higher, and want to use the Sentry Service, you can migrate your policy files using the hbase-indexer-sentry utility:

/opt/cloudera/parcels/CDH/bin/hbase-indexer-sentry -mgr -i -v -f "file://path/to/sentry-provider.ini"

This command validates and imports the specified policy file to the Sentry Service. For more information on the command usage and syntax, see Managing Sentry Permissions for the Lily HBase Indexer.