Skip to content

punchplatform-kafka-topics.sh

NAME

punchplatform-kafka-topics.sh - Kafka debug commands

SYNOPSIS

punchplatform-kafka-topics.sh COMMAND [OPTIONS]

DESCRIPTION

Kafka debug commands.

By default, this command targets the Kafka cluster configured in your punchplatform.properties file. If more than one Kafka cluster is configured, the option --kafkaCluster <CLUSTER_NAME> is mandatory.

OPTIONS

COMMAND must include exactly one action: --list, --describe, --create, --alter, --add-config, --delete-config, --offsets or --delete.

  • --alter
    • Alter the number of partitions, replica assignment

Warning

If zookeeper is started with TLS encryption, use kafkactl alter-topic command (alpha version) :

kafkactl man page

  • --add-config PROP_NAME=VALUE

    • A topic configuration override for the topic being altered.The following is a list of valid configurations: cleanup.policy, compression.type, delete.retention.ms, file.delete.delay.ms, flush.messages, flush.ms, follower.replication.throttled.replicas, index.interval.bytes, leader.replication.throttled.replicas, max.message.bytes, message.format.version, message.timestamp.difference.max.ms, message.timestamp.type, min.cleanable.dirty.ratio, min.compaction.lag.ms, min.insync.replicas, preallocate, retention.bytes, retention.ms, segment.bytes, segment.index.bytes, segment.jitter.ms, segment.ms, unclean.leader.election.enable. See the Kafka documentation for full details on the topic configs.

  • --delete-config PROP_NAME

    • Removes a topic configuration override for the topic being or altered. The configuration property will take back the default value coming from the cluster brokers configuration. the Kafka documentation for full details on the topic configs.

  • --create

    • Create a new topic.

  • --delete

    • Delete a topic

  • --delete-config NAME

    • A topic configuration override to be removed for an existing topic (see the list of configurations under the --config option).

  • --describe

    • List details for the given topics.

  • --disable-rack-aware

    • Disable rack aware replica assignment.

  • --force

    • Suppress console prompts.

  • --help

    • Print usage information.

  • --if-exists

    • if set when altering or deleting topics, the action will only execute if the topic exists.

  • --if-not-exists

    • if set when creating topics, the action will only execute if the topic does not already exist.

  • --list

    • List all available topics.

  • --partitions NUMBER

    • The number of partitions for the topic being created or altered (WARNING: If partitions are increased for a topic that has a key, the partition logic or ordering of the messages will be affected.

  • --replica-assignment broker_id_for_part1_replica1:broker_ id_for_part1_replica2,broker_id_for_part2_replica1:broker_id_for_part2_replica2,...

    • A list of manual partition-to-broker assignments for the topic being created or altered.

  • --replication-factor FACTOR

    • The replication factor for each partition in the topic being created.

  • --topic TOPIC_NAME

    • The topic to be create, alter or describe. Can also accept a regular expression except for --create option.

  • --topics-with-overrides

    • if set when describing topics, only show topics that have overridden configs

  • --unavailable-partitions

    • if set when describing topics, only show partitions whose leader is not available

  • --under-replicated-partitions

    • if set when describing topics, only show under replicated partitions

  • --zookeeper URL

    • The connection string for the zookeeper connection in the form host:port. Multiple URLS can be given to allow fail-over.

  • --offsets

    • This subcommand displays topic earliest/latest offsets (requires --topic ) WARNING: this subcommand does not work when TLS is active at Kafka level.

FILES

  • punchplatform.properties:
    • This json file defines the punchplatform settings. It is used by punchplatform-kafka-topics.sh primarily to know the broker list.

EXAMPLE

  • Create a topic
punchplatform-kafka-topics.sh --create --topic test-topic --partitions 1 --replication-factor 1
  • Describe a topic
punchplatform-kafka-topics.sh --describe --topic test-topic
  • Add partition to an existing topic 
punchplatform-kafka-topics.sh --topic TOPIC_NAME --alter --partitions 4
  • Change retention settings
punchplatform-kafka-topics.sh --topic TOPIC_NAME --add-config retention.bytes=NNNNNNNNNNN
punchplatform-kafka-topics.sh --topic TOPIC_NAME --add-config retention.ms=NNNNNNNNNNN
  • Target another cluster
punchplatform-kafka-topics.sh --kafkaCluster CLUSTER --describe --topic TOPIC_NAME

SEE ALSO