Skip to content

KCP CLI

KCP is a CLI tool for planning and executing Apache Kafka® migrations to Confluent Cloud.

Note

KCP supports migrations from two source types:

  • AWS MSK (Managed Streaming for Apache Kafka) — full discovery via AWS APIs + Kafka Admin API.
  • Apache Kafka — direct scanning via Kafka Admin API.

The workflow differs slightly based on your source type. See the Command Reference for per-command specifics, or the Source Compatibility matrix for which commands support which source flavor (MSK Provisioned/Express, MSK Serverless, Apache Kafka).

Installation

Tip

Recommended: download the binary for your platform directly from the latest release on GitHub, make it executable, and place it on your PATH. Binaries are published for macOS, Linux (amd64/arm64), and Windows (amd64).

Run uname -m if you're unsure of your architecture: arm64 / aarch64 → arm64 build, x86_64 → amd64 build. Apple Silicon Macs are arm64; older Intel Macs are amd64.

The tabs below show the equivalent terminal flow for headless or scripted installs.

# Apple Silicon: PLATFORM=darwin_arm64. Intel: PLATFORM=darwin_amd64.
PLATFORM=darwin_arm64
LATEST_TAG=$(curl -s https://api.github.com/repos/confluentinc/kcp/releases/latest \
  | grep '"tag_name":' | sed -E 's/.*"([^"]+)".*/\1/')

curl -L -o kcp "https://github.com/confluentinc/kcp/releases/download/${LATEST_TAG}/kcp_${PLATFORM}"
chmod +x kcp
sudo mv kcp /usr/local/bin/kcp
kcp version

# PLATFORM=linux_amd64 or linux_arm64.
PLATFORM=linux_amd64
LATEST_TAG=$(curl -s https://api.github.com/repos/confluentinc/kcp/releases/latest \
  | grep '"tag_name":' | sed -E 's/.*"([^"]+)".*/\1/')

curl -L -o kcp "https://github.com/confluentinc/kcp/releases/download/${LATEST_TAG}/kcp_${PLATFORM}"
chmod +x kcp
sudo mv kcp /usr/local/bin/kcp
kcp version

Download kcp_windows_amd64.exe from the releases page, move it onto a folder on your PATH, and verify with kcp version.

Authentication

KCP uses the standard AWS credential chain for any command that calls AWS APIs. Supported auth methods:

  • Environment variables: AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, and (optionally) AWS_SESSION_TOKEN.
  • AWS credentials file: configure with aws configure.
  • AWS SSO / Identity Center: aws sso login.
  • IAM roles: assumed roles or EC2 instance profiles.
  • Credential helpers: any tool that writes to the standard AWS credential locations (e.g. granted).

Verify with:

aws sts get-caller-identity

Each command's per-command AWS IAM permission requirements are documented on its page in the Command Reference.

Note

Apache Kafka migrations do not require AWS authentication. Apache Kafka clusters are accessed directly via Kafka Admin API using the credentials you provide in apache-kafka-credentials.yaml. See kcp scan clusters for details.

Workflow

The typical migration flow:

  1. Discover / scan — kcp discover (MSK) or kcp scan clusters (MSK or Apache Kafka) to build kcp-state.json.
  2. Report — kcp report costs and kcp report metrics for cost and utilization analysis. Alternatively, use the kcp ui for fine-grained analysis.
  3. Generate migration assets for data migration — kcp create-asset target-infra, migration-infra, migrate-topics, migrate-schemas, migrate-acls, migrate-connectors.
  4. Initialize and execute client switchover — kcp migration init followed by kcp migration execute.

The Getting Started with Zero-Cut Migrations guide walks through the end-to-end migration reference, including how KCP fits with the Confluent Cloud Gateway.

Key infrastructure decisions

Before starting, decide on:

  1. Is your source Kafka cluster reachable over the public internet, or only from within a private VPC?
  2. If private, do you already have a bastion / jump host, or do you need one?
  3. What authentication methods are enabled on the source, and which will you use for the migration cluster link?

Only certain migration topologies are possible for a given combination — see kcp create-asset migration-infra for the type matrix.

Bastion host requirements

  • Public endpoints — you can run kcp commands directly from your local machine.
  • Private endpoints — kcp must run from inside the source VPC. Either:
  • Provision a new bastion with kcp create-asset bastion-host, or
  • Use an existing jump server and copy the kcp binary onto it.

Note

For private MSK, transfer the kcp binary to a host inside the same VPC before continuing.

Command reference

The full CLI reference is generated directly from the Cobra command definitions and lives under Command Reference. Entry points: