Using an Internal Package Repository

You can create a package repository for Cloudera Manager either by hosting an internal web repository or by manually copying the repository files to the Cloudera Manager Server host for distribution to Cloudera Manager Agent hosts.

Loading Filters ...

6.0.0 6.0.0

Continue reading:

Creating a Permanent Internal Repository

The following sections describe how to create a permanent internal repository using Apache HTTP Server:

Setting Up a Web Server

To host an internal repository, you must install or use an existing web server on an internal host that is reachable by the Cloudera Manager host, and then download the repository files to the web server host. The examples in this section use Apache HTTP Server as the web server. If you already have a web server in your organization, you can skip to Downloading and Publishing the Package Repository.

  1. Install Apache HTTP Server:
    RHEL / CentOS
    sudo yum install httpd
    SLES
    sudo zypper install httpd
    Debian
    sudo apt-get install httpd
  2. Start Apache HTTP Server:
    RHEL 7
    sudo systemctl start httpd
    RHEL 6 or lower
    sudo service httpd start
    SLES 12, Ubuntu 16.04, Debian 8
    sudo systemctl start apache2
    SLES 11, Ubuntu 14.04, Debian 7 or lower
    sudo service apache2 start

Downloading and Publishing the Package Repository

  1. Download the package repository for the product you want to install:
    Cloudera Manager 6
    To download the files for the latest Cloudera Manager 6.0 release, run the following commands on the web server host. 
    sudo mkdir -p /var/www/html/cloudera-repos
sudo wget --recursive --no-parent --no-host-directories https://username:password@archive.cloudera.com/p/cm6/6.0.0/<operating_system>/ -P /var/www/html/cloudera-repos
sudo wget https://archive.cloudera.com/cm6/6.0.0/allkeys.asc -P /var/www/html/cloudera-repos/cm6/6.0.0/
    sudo chmod -R ugo+rX /var/www/html/cloudera-repos/cm6

    If you want to create a repository for a different Cloudera Manager 6 release, replace 6.0.0 with the CDH 6 version that you want. For more information, see Cloudera Manager 6 Version and Download Information.

    CDH 6
    To download the files for the latest CDH 6.0 release, run the following commands on the web server host. 
    sudo mkdir -p /var/www/html/cloudera-repos
sudo wget --recursive --no-parent --no-host-directories https://username:password@archive.cloudera.com/p//cdh6/6.0.0/<operating_system>/ -P /var/www/html/cloudera-repos
    sudo wget --recursive --no-parent --no-host-directories https://archive.cloudera.com/gplextras6/6.0.0/<operating_system>/ -P /var/www/html/cloudera-repos
    sudo chmod -R ugo+rX /var/www/html/cloudera-repos/cdh6
sudo chmod -R ugo+rX /var/www/html/cloudera-repos/gplextras6

    If you want to create a repository for a different CDH 6 release, replace 6.0.0 with the CDH 6 version that you want. For more information, see CDH 6 Download Information.

    Cloudera Manager 5
    To download the files for a Cloudera Manager release, download the repository tarball for your operating system. 
    sudo mkdir -p /var/www/html/cloudera-repos/cm5
    Redhat/Centos:
    wget https://username:password@archive.cloudera.com/p/cm5/repo-as-tarball/5.14.4/cm5.14.4-centos7.tar.gz
    tar xvfz cm5.14.4-centos7.tar.gz -C /var/www/html/cloudera-repos/cm5 --strip-components=1
    Debian:
    wget https://username:password@archive.cloudera.com/p/cm5/repo-as-tarball/5.14.4/cm5.14.4-debian-jessie.tar.gz
    tar xvfz cm5.14.4-debian-jessie.tar.gz -C /var/www/html/cloudera-repos/cm5 --strip-components=1
    SLES:
    wget https://username:password@archive.cloudera.com/p/cm5/repo-as-tarball/5.14.4/cm5.14.4-sles12.tar.gz
    tar xvfz cm5.14.4-sles12.tar.gz -C /var/www/html/cloudera-repos/cm5 --strip-components=1
    Ubuntu:
    wget https://username:password@archive.cloudera.com/p/cm5/repo-as-tarball/5.14.4/cm5.14.4-ubuntu16-04.tar.gz
    tar xvfz cm5.14.4-ubuntu16-04.tar.gz -C /var/www/html/cloudera-repos/cm5 --strip-components=1
    sudo chmod -R ugo+rX /var/www/html/cloudera-repos/cm5

    If you want to create a repository for a different Cloudera Manager release or operating system, start in the repo-as-tarball parent directory, select the Cloudera Manager version you want to use, and then copy the .tar.gz link for your OS.

    CDH 5
    To download the files for a CDH release, download the repository tarball for your operating system. 
    sudo mkdir -p /var/www/html/cloudera-repos/cdh5
    Redhat/Centos:
    wget https://username:password@archive.cloudera.com/p/cdh5/repo-as-tarball/5.14.4/cdh5.14.4-centos7.tar.gz
    tar xvfz cdh5.14.4-centos7.tar.gz -C /var/www/html/cloudera-repos/cdh5 --strip-components=1
    Debian:
    wget https://username:password@archive.cloudera.com/p/cdh5/repo-as-tarball/5.14.4/cdh5.14.4-debian-jessie.tar.gz
    tar xvfz cdh5.14.4-debian-jessie.tar.gz -C /var/www/html/cloudera-repos/cdh5 --strip-components=1
    SLES:
    wget https://username:password@archive.cloudera.com/p/cdh5/repo-as-tarball/5.14.4/cdh5.14.4-sles12.tar.gz
    tar xvfz cdh5.14.4-sles12.tar.gz -C /var/www/html/cloudera-repos/cdh5 --strip-components=1
    Ubuntu:
    wget https://username:password@archive.cloudera.com/p/cdh5/repo-as-tarball/5.14.4/cdh5.14.4-ubuntu16-04.tar.gz
    tar xvfz cdh5.14.4-ubuntu16-04.tar.gz -C /var/www/html/cloudera-repos/cdh5 --strip-components=1
    sudo chmod -R ugo+rX /var/www/html/cloudera-repos/cdh5

    If you want to create a repository for a different CDH release or operating system, start in the repo-as-tarball parent directory, select the CDH version you want to use, and then copy the .tar.gz link for your OS.

    Apache Accumulo for CDH
    To download the files for an Accumulo release for CDH, run the following commands on the web server host. 
    sudo mkdir -p /var/www/html/cloudera-repos
sudo wget --recursive --no-parent --no-host-directories https://archive.cloudera.com/accumulo-c5/<operating_system>/ -P /var/www/html/cloudera-repos
    sudo chmod -R ugo+rX /var/www/html/cloudera-repos/accumulo-c5
    Cloudera Navigator Key Trustee Server
    Go to the Key Trustee Server download page. Select Packages from the CHOOSE DOWNLOAD TYPE drop-down menu, select your operating system from the CHOOSE AN OS drop-down menu, and then click DOWNLOAD NOW. This downloads the Key Trustee Server package files in a .tar.gz file. Copy the file to your web server, and extract the files with the tar xvfz filename.tar.gz command. This example uses Key Trustee Server 5.14.0: 
    sudo mkdir -p /var/www/html/cloudera-repos/keytrustee-server
sudo tar xvfz /path/to/keytrustee-server-5.14.0-parcels.tar.gz -C /var/www/html/cloudera-repos/keytrustee-server --strip-components=1
    sudo chmod -R ugo+rX /var/www/html/cloudera-repos/keytrustee-server
    Cloudera Navigator Key Trustee KMS and HSM KMS
    Go to the Key Trustee KMS download page. Select Package from the CHOOSE DOWNLOAD TYPE drop-down menu, select your operating system from the OPERATING SYSTEM drop-down menu, and then click DOWNLOAD NOW. This downloads the Key Trustee KMS package files in a .tar.gz file. Copy the file to your web server, and extract the files with the tar xvfz filename.tar.gz command. This example uses Key Trustee KMS 5.14.0: 
    sudo mkdir -p /var/www/html/cloudera-repos/keytrustee-kms
sudo tar xvfz /path/to/keytrustee-kms-5.14.0-parcels.tar.gz -C /var/www/html/cloudera-repos/keytrustee-kms --strip-components=1
    sudo chmod -R ugo+rX /var/www/html/cloudera-repos/keytrustee-kms
  2. Visit the Repository URL http://<web_server>/cloudera-repos/ in your browser and verify the files you downloaded are present. If you do not see anything, your web server may have been configured to not show indexes.

Creating a Temporary Internal Repository

You can quickly create a temporary remote repository to deploy packages on a one-time basis. Cloudera recommends using the same host that runs Cloudera Manager, or a gateway host. This example uses Python SimpleHTTPServer as the web server to host the /var/www/html directory, but you can use a different directory.

  1. Download the repository you need following the instructions in Downloading and Publishing the Package Repository.
  2. Determine a port that your system is not listening on. This example uses port 8900.
  3. Start a Python SimpleHTTPServer in the /var/www/html directory: 
    cd /var/www/html
python -m SimpleHTTPServer 8900
    Serving HTTP on 0.0.0.0 port 8900 ...
  4. Visit the Repository URL http://<web_server>:8900/cloudera-repos/ in your browser and verify the files you downloaded are present.

Configuring Hosts to Use the Internal Repository

Having established the repository, modify the client configuration to use it:
OS Procedure
RHEL compatible Create /etc/yum.repos.d/cloudera-repo.repo files on cluster hosts with the following content, where <web_server> is the hostname of the web server: 
[cloudera-repo]
name=cloudera-repo
baseurl=http://<web_server>/cm/5
enabled=1
gpgcheck=0
SLES Use the zypper utility to update client system repo information by issuing the following command: 
zypper addrepo http://<web_server>/cm <alias>
Ubuntu Create /etc/apt/sources.list.d/cloudera-repo.list files on all cluster hosts with the following content, where <web_server> is the hostname of the web server: 
deb http://<web_server>/cm <codename> <components>
You can find the <codename> and <components> variables in the ./conf/distributions file in the repository.
After creating the .list file, run the following command: 
sudo apt-get update