Skip to content

KCP CLI

KCP is a CLI tool for planning and executing migrations to Confluent Cloud.

Note

KCP supports migrations from two source types:

  • AWS MSK (Managed Streaming for Apache Kafka) — full discovery via AWS APIs + Kafka Admin API.
  • Apache Kafka (including Confluent Platform and other Kafka-API-compatible distributions) — direct scanning via Kafka Admin API.

The workflow differs slightly based on your source type. See the Command Reference for per-command specifics, or the Source Compatibility matrix for which commands support which source flavor (MSK Provisioned/Express, MSK Serverless, Apache Kafka).

Installation

Tip

Recommended: on macOS or Linux, install the latest stable release with the install script. It detects your OS/architecture, downloads the matching binary, verifies its checksum, and places it on your PATH:

curl -fsSL https://raw.githubusercontent.com/confluentinc/kcp/main/install.sh | sh

Pin a version with KCP_VERSION=v0.8.5 or change the target directory with KCP_INSTALL_DIR="$HOME/.local/bin".

Binaries are published for macOS, Linux (amd64/arm64), and Windows (amd64) with every release on GitHub. Each release includes these assets: kcp_darwin_amd64, kcp_darwin_arm64, kcp_linux_amd64, kcp_linux_arm64, and kcp_windows_amd64.exe.

Important

Always install a released binary. Building from the main branch produces an unreleased, in-development build — see CONTRIBUTING if you intend to build from source.

Run uname -m if you're unsure of your architecture: arm64 / aarch64 → arm64 build, x86_64 → amd64 build. Apple Silicon Macs are arm64; older Intel Macs are amd64.

To download manually instead of using the install script, the tabs below show the equivalent terminal flow for headless or scripted installs.

# Apple Silicon: PLATFORM=darwin_arm64. Intel: PLATFORM=darwin_amd64.
PLATFORM=darwin_arm64
LATEST_TAG=$(curl -s https://api.github.com/repos/confluentinc/kcp/releases/latest \
  | grep '"tag_name":' | sed -E 's/.*"([^"]+)".*/\1/')

curl -L -o kcp "https://github.com/confluentinc/kcp/releases/download/${LATEST_TAG}/kcp_${PLATFORM}"
chmod +x kcp
./kcp version

# Optional: put kcp on your PATH so you can run it from any directory
sudo mv kcp /usr/local/bin/kcp
kcp version

# PLATFORM=linux_amd64 or linux_arm64.
PLATFORM=linux_amd64
LATEST_TAG=$(curl -s https://api.github.com/repos/confluentinc/kcp/releases/latest \
  | grep '"tag_name":' | sed -E 's/.*"([^"]+)".*/\1/')

curl -L -o kcp "https://github.com/confluentinc/kcp/releases/download/${LATEST_TAG}/kcp_${PLATFORM}"
chmod +x kcp
./kcp version

# Optional: put kcp on your PATH so you can run it from any directory
sudo mv kcp /usr/local/bin/kcp
kcp version

Download kcp_windows_amd64.exe from the releases page, move it onto a folder on your PATH, and verify with kcp version.

Authentication

KCP uses the standard AWS credential chain for any command that calls AWS APIs. Supported auth methods:

  • Environment variables: AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, and (optionally) AWS_SESSION_TOKEN.
  • AWS credentials file: configure with aws configure.
  • AWS SSO / Identity Center: aws sso login.
  • IAM roles: assumed roles or EC2 instance profiles.
  • Credential helpers: any tool that writes to the standard AWS credential locations (e.g. granted).

Verify with:

aws sts get-caller-identity

Each command's per-command AWS IAM permission requirements are documented on its page in the Command Reference.

Note

Apache Kafka migrations do not require AWS authentication. Apache Kafka clusters are accessed directly via Kafka Admin API using the credentials you provide in apache-kafka-credentials.yaml. See kcp scan clusters for details.

High-Level Workflow

The typical migration flow:

  1. Discover / scan — kcp discover (MSK) or kcp scan clusters (MSK or Apache Kafka) to build kcp-state.json. See Getting started for the step-by-step runbook.
  2. Report — kcp report costs and kcp report metrics for cost and utilization analysis. Alternatively, use the kcp ui for fine-grained analysis.

  3. Generate migration assets for data migration:

  4. kcp iac <asset> generate for Terraform using target-infra, migration-infra, topics, schemas; kcp iac acls kafka|iam generate; kcp iac connectors msk|self-managed|connector-utility generate.

  5. kcp migration resources for direct creation on the destination cluster through the REST API for cluster links, topics and ACLs.

  6. Execute client switchover — kcp migration cutover execute (validates and runs the cutover in one command, continuing from the live state if an earlier run was interrupted; --dry-run validates only).

Key infrastructure decisions

Before starting, decide on:

  1. Is your source Kafka cluster reachable over the public internet, or only from within a private VPC?
  2. If private, do you already have a bastion / jump host, or do you need one?
  3. What authentication methods are enabled on the source, and which will you use for the migration cluster link?

Only certain migration topologies are possible for a given combination — see kcp iac migration-infra generate for the type matrix.

Bastion host requirements

  • Public endpoints — you can run kcp commands directly from your local machine.
  • Private endpoints — kcp must run from inside the source VPC. Either:
  • Provision a new bastion with kcp iac bastion-host generate, or
  • Use an existing jump server and copy the kcp binary onto it.

Note

For private MSK, transfer the kcp binary to a host inside the same VPC before continuing.

Command reference

The full CLI reference is generated directly from the Cobra command definitions and lives under Command Reference. Entry points:

  • kcp discover — scan AWS for MSK clusters
  • kcp scan — scan a Kafka cluster, S3 broker logs, or a schema registry
  • kcp report — generate cost and metrics reports
  • kcp iac — generate Terraform for target, migration, topic, schema, ACL and connector assets
  • kcp migration — cut clients over to Confluent Cloud through the Gateway (cutover) and create Kafka resources directly (resources)
  • kcp ui — launch the local web UI
  • kcp update / kcp version / kcp docs — housekeeping