Reverse-Checkpointed Topics Subcommand

Learn how to use the reverse-checkpointed-topics subcommand of the srm-control command line tool.

The reverse-checkpointed topics subcommand is used to manipulate the reverse-checkpointed topic allowlist (whitelist) and denylist (blacklist). Through this you can control which topics are reverse checkpointed by Streams Replication Manager.

If you are modifying allow and denylists which target newly created topics, changes made with the srm-control tool may not be instantaneous. A topic needs to be discovered by Streams Replication Manager before it can be added to or removed from allow or denylists. New topic discovery happens every 10 minutes by default. As a result, you may need to wait up to 10 minutes until you can see the changes made.

Add topics to an allowlist:

srm-control reverse-checkpointed-topics --source [SOURCE_CLUSTER] --target [TARGET_CLUSTER] --add [TOPIC1],[TOPIC2]

Remove topics from an allowlist:

srm-control reverse-checkpointed-topics --source [SOURCE_CLUSTER] --target [TARGET_CLUSTER] --remove [TOPIC1],[TOPIC2]

Add topics to a denylist (blacklist):

srm-control reverse-checkpointed-topics --source [SOURCE_CLUSTER] --target [TARGET_CLUSTER] --add-blacklist [TOPIC1],[TOPIC2]

Remove topics from a denylist:

srm-control reverse-checkpointed-topics --source [SOURCE_CLUSTER] --target [TARGET_CLUSTER] --remove-blacklist [TOPIC1],[TOPIC2]

In addition to adding or removing items, you can also use the tool to look at the contents of a deny or allowlist.

srm-control reverse-checkpointed-topics --source [SOURCE_CLUSTER] --target [TARGET_CLUSTER] --list

Using Regular Expressions

Specifying reverse-checkpointed topics is also possible with regular expressions. This is done by specifying a regular expression instead of topic names in the --add, add-blackilist, or remove-blacklist options.

For example, assume that you want to reverse-checkpoint all replicated topics in a cluster. In a case like this, you must add all topics to the allowlist. This can be done with a single srm-control command using the .* regular expression.

srm-control reverse-checkpointed-topics --source [SOURCE_CLUSTER] --target [TARGET_CLUSTER] --add ".*"

You can also use regular expressions to specify a set of topics. For example, assume that you have many topics in your Kafka cluster, but only want to add the following topics to the allowlist.

testTopic1
testTopic2
testTopic3

In a case like this, you can use the testTopic.* or ^test.* regular expressions to add the topics.

srm-control reverse-checkpointed-topics --source [SOURCE_CLUSTER] --target [TARGET_CLUSTER] --add "testTopic.*"

srm-control reverse-checkpointed-topics --source [SOURCE_CLUSTER] --target [TARGET_CLUSTER] --add "^test.*"

Client Override Options

The reverse-checkpointed topics subcommand supports a number of client override options. Client override options allow users to temporarily specify or override configuration properties used for replication. These options also enable users to issue srm-control commands even if the configuration of Streams Replication Manager is not available on the host where the command is being issued. While it is possible to specify a range of properties with the client override options, and they can prove to be a powerful tool in certain scenarios, Cloudera recommends that you use Cloudera Manager to manage client configuration options.

The following client override options are available:

--bootstrap-servers: Specifies the bootstraps servers.
--producer-props: Specifies producer configuration properties.
--consumer-props: Specifies consumer configuration properties.
--props: Specifies client configuration properties.

A simple example of using client override options is when you want to change the bootstrap server. This can be done in two ways.

You can specify the bootstrap server with the --bootstrap-servers option.

srm-control --bootstrap-servers localhost:9092 reverse-checkpointed-topics --source [SOURCE_CLUSTER] --target [TARGET_CLUSTER] --list

Alternatively, you can also use the --props option together with the bootstrap.servers Kafka property to define the bootstrap server.

srm-control --props bootstrap.servers=localhost:9092 reverse-checkpointed-topics --source [SOURCE_CLUSTER] --list