Cloudera Docs

Backing up Cloudera Data Engineering service using the docker image

You must run the docker image to take a backup of a Cloudera Data Engineering service. It takes backup of all the active virtual clusters in that Cloudera Data Engineering service. You can take backup of only one active Cloudera Data Engineering service at a time.

You must download the dex-upgrade-utils docker image and create the cde-upgrade-util.properties file before backing up jobs as described in the Prerequisites for upgrading Cloudera Data Engineering Service with endpoint stability section.
  1. Set the following environment variables in the ECS Server host terminal: 
    export PATH=$PATH:/opt/cloudera/parcels/ECS/installer/install/bin/linux/:/opt/cloudera/parcels/ECS/docker
export KUBECONFIG=~/kubeconfig
export BASE_WORK_DIR=/opt/backup-restore
export BACKUP_OUTPUT_DIR=/home/dex/backup
  2. Run the dex-upgrade-utils docker image on the host machine: 
    docker run \
-v [***KUBECONFIG_FILE_PATH***]:/home/dex/.kube/config:ro \
-v [***CDP_CREDENTIAL_FILE_PATH***]:/home/dex/.cdp/credentials:ro \
-v [***CDE-UPGRADE-UTIL.PROPERTIES_FILE_PATH***]:/opt/cde-backup-restore/scripts/backup-restore/cde-upgrade-util.properties:ro \
-v [***LOCAL_BACKUP_DIRECTORY***]:$BACKUP_OUTPUT_DIR \
-e KUBECONFIG=/home/dex/.kube/config \
[***DOCKER_IMAGE_NAME***]:[***DOCKER_IMAGE_VERSION***] prepare-for-upgrade -s [***CDE-CLUSTER-ID***] -o $BACKUP_OUTPUT_DIR

    Example:

    docker run \

-v $BASE_WORK_DIR/secrets/kubeconfig:/home/dex/.kube/config:ro \
-v $BASE_WORK_DIR/secrets/credentials:/home/dex/.cdp/credentials:ro \
-v $BASE_WORK_DIR/cde-upgrade-util.properties:/opt/cde-backup-restore/scripts/backup-restore/cde-upgrade-util.properties:ro \
-v $BASE_WORK_DIR/backup:$BACKUP_OUTPUT_DIR \
-e KUBECONFIG=/home/dex/.kube/config \
docker-private.infra.cloudera.com/cloudera/dex/dex-upgrade-utils:1.20.1 prepare-for-upgrade -s cluster-c2dhkp22 -o $BACKUP_OUTPUT_DIR
    To identify the CDE-CLUSTER-ID, do the following steps:
    1. In the Cloudera console, click the Data Engineering tile. The Cloudera Data Engineering home page displays.
    2. On the left navigation menu, click Administration.
    3. In the Services column, click for the Service for which you want to identify the Cluster ID.
    4. Note the CLUSTER ID displayed on the page.
You have now taken the Cloudera Data Engineering service backup as a ZIP file. You can make a note of the Zip file name from the logs to use it while restoring the Cloudera Data Engineering service.
You must now expand the resource pool, and then upgrade your Cloudera Data Platform (CDP) before you restore the Cloudera Data Engineering service. For information about configuring resource pool and capacity, see Managing cluster resources using Quota Management.