Step 15: Start up the Secondary NameNode (if used)

At this point, you should be able to start the Secondary NameNode if you are using one:

$ sudo service hadoop-hdfs-secondarynamenode start

You'll see some extra information in the logs such as:

10/10/26 12:03:18 INFO security.UserGroupInformation: 
Login successful for user hdfs/fully.qualified.domain.name@YOUR-REALM using keytab file /etc/hadoop/conf/hdfs.keytab

and:

12/05/23 18:33:06 INFO http.HttpServer: Adding Kerberos (SPNEGO) filter to getimage
12/05/23 18:33:06 INFO http.HttpServer: Jetty bound to port 50090
12/05/23 18:33:06 INFO mortbay.log: jetty-6.1.26
12/05/23 18:33:06 INFO server.KerberosAuthenticationHandler: Login using keytab /etc/hadoop/conf/hdfs.keytab, for principal HTTP/fully.qualified.domain.name@YOUR-REALM.COM
12/05/23 18:33:06 INFO server.KerberosAuthenticationHandler: Initialized, principal [HTTP/fully.qualified.domain.name@YOUR-REALM.COM] from keytab [/etc/hadoop/conf/hdfs.keytab]

You should make sure that the Secondary NameNode not only starts, but that it is successfully checkpointing.

If you're using the service command to start the Secondary NameNode from the /etc/init.d scripts, Cloudera recommends setting the property fs.checkpoint.period in the hdfs-site.xml file to a very low value (such as 5), and then monitoring the Secondary NameNode logs for a successful startup and checkpoint. Once you are satisfied that the Secondary NameNode is checkpointing properly, you should reset the fs.checkpoint.period to a reasonable value, or return it to the default, and then restart the Secondary NameNode.

You can make the Secondary NameNode perform a checkpoint by doing the following:

$ sudo -u hdfs hdfs secondarynamenode -checkpoint force

Note that this will not cause a running Secondary NameNode to checkpoint, but rather will start up a Secondary NameNode that will immediately perform a checkpoint and then shut down. This can be useful for debugging.