Skip to content

kcp migration cutover

kcp migration cutover

Cut clients over to Confluent Cloud using Cluster Linking and the Confluent Gateway

Synopsis

Execute end-to-end Kafka migrations to Confluent Cloud using Cluster Linking and the Confluent Gateway.

The migration runs as a fixed sequence of steps:

  1. Reconcile and validate — every kcp migration cutover execute run reads which migration type the manifest declares (spec.staticMigration, spec.dynamicMigration or spec.dynamicToStaticConversion), checks it against the live route's mode, and validates the cluster link and gateway CRs live against the manifest and the current cluster state. --dry-run runs this validation alone and exits, touching no state.
  2. Check Lags — wait until every selected topic's replication lag (the sum of its partition lags) is at or below spec.<staticMigration|dynamicMigration>.policies.lagThreshold (default 0: fully caught up). Producers are still running at this point.
  3. Fence Gateway — fence the route in the gateway CR to block traffic during cutover, and wait for the gateway to confirm it. A static route is fenced as a whole; a topic-based (dynamic) route fences only the topics in this migration.
  4. Pause Offset Sync (static routes only) — with spec.staticMigration.consumerOffsetSync.pause, pause cluster-link consumer offset sync (passes through otherwise). Topic-based (dynamic) migrations skip this step — see the note below.
  5. Verify Fence — only when spec.<staticMigration|dynamicMigration>.policies.detectUnroutedProducersDuration is set (minimum 10s; the default, 0, skips this step): take two source offset snapshots that far apart. If any partition's offset advanced, a producer is bypassing the gateway and the run stops (see "Failure and rollback" below). This is the only check for unrouted producers before promote, and it only sees producers active during the window.
  6. Promote Topics — promote mirror topics at zero lag and wait for each to reach STOPPED. Promote is the point of no return.
  7. Switch Gateway — switch the route in the gateway CR to Confluent Cloud: a static route is switched as a whole, a dynamic route gets a routing condition for this migration's topics.
  8. Restore Offset Sync (static routes only) — with spec.staticMigration.consumerOffsetSync.pause, set cluster-link consumer offset sync back to the declared spec.staticMigration.consumerOffsetSync.baseline; a failure fails the run, and re-running retries it.

A topic-based migration (spec.dynamicMigration) has no consumerOffsetSync section: a dynamic route requires consumer offset sync to be disabled, and reconcile refuses a link that has it enabled.

Failure and rollback. If a step fails after the fence is up but before the promote step starts, kcp removes the fence and restores offset sync, so traffic returns to the source. Once promote starts, kcp never removes the fence, even if no topic was promoted yet — sending those clients back to the source would split them from the target. The same holds from the first step of a re-run that finds a topic already promoted or promoting. Fix the cause and re-run kcp migration cutover execute to roll forward. If removing the fence itself fails, the error is logged and the original failure is returned, so check the gateway CR.

Interrupting kcp (Ctrl-C) does not remove the fence: kcp has no signal handler, so the gateway stays fenced until you re-run kcp migration cutover execute.

If execution is interrupted at any step, re-running kcp migration cutover execute safely continues: every run re-derives the migration's state live from the manifest and the cluster, and re-applying an already-completed step is safe (idempotent). A resume re-applies the fence rather than skipping it, and repeats the unrouted-producer window if one is configured.

Before you run it. Every run, including --dry-run, connects to the source Kafka cluster, the destination Kafka cluster, the cluster-link REST API and the Kubernetes API, using the credentials in the manifest. A --dry-run only reads: on Kubernetes it reads the Gateway CR (and, for a static migration, checks the staged Secrets exist). The permissions to patch the Gateway, read its Deployment and pods, and write cluster-link configs are first used after the fence, so see the Permissions technical guide before a real run. All producers must go through the gateway for the fence to hold. If the reconcile plan is refused — for example a selected topic is not a mirror topic on the cluster link — nothing is changed and the whole run is refused; --dry-run prints the plan and the reasons.

The manifest these commands read is documented in the Migration manifest reference — every field, validation rules, and a fully-annotated example.

kcp migration cutover [flags]

Options

  -h, --help   help for cutover

Options inherited from parent commands

      --verbose   Enable verbose logging to console

SEE ALSO