kcp scan self-managed-connectors

kcp scan self-managed-connectors

Scan self-managed Kafka Connect cluster for connector information

Synopsis

Scan a self-managed Kafka Connect cluster using its REST API to discover connector configurations and status. Sensitive config values are redacted before being written to the state file.

kcp scan self-managed-connectors [flags]

Examples

  # Scan connectors for an MSK cluster (auto-detected from ARN format)
  kcp scan self-managed-connectors \
    --state-file kcp-state.json \
    --connect-rest-url http://connect:8083 \
    --cluster-id arn:aws:kafka:us-east-1:123456789012:cluster/my-cluster/abc-123 \
    --use-unauthenticated

  # Scan connectors for an OSK cluster (auto-detected from non-ARN format)
  kcp scan self-managed-connectors \
    --state-file kcp-state.json \
    --connect-rest-url https://connect.example.com:8083 \
    --cluster-id production-kafka \
    --use-basic-auth \
    --username admin \
    --password secret

  # Explicitly specify source type (overrides auto-detection)
  kcp scan self-managed-connectors \
    --state-file kcp-state.json \
    --connect-rest-url http://connect:8083 \
    --cluster-id my-cluster \
    --source-type osk \
    --use-unauthenticated

  # Scan with Jolokia metrics collection
  kcp scan self-managed-connectors \
    --state-file kcp-state.json \
    --connect-rest-url http://connect:8083 \
    --cluster-id my-cluster \
    --use-unauthenticated \
    --metrics jolokia --metrics-duration 5m --metrics-interval 10s \
    --credentials-file osk-credentials.yaml

Options

      --cluster-id string          The cluster identifier in the state file. Accepts both MSK ARNs (arn:aws:kafka:...) and OSK cluster IDs.
      --connect-rest-url string    The Kafka Connect REST API URL (e.g., http://localhost:8083).
      --credentials-file string    Path to the apache-kafka-credentials.yaml file providing Jolokia/Prometheus configuration. Required with --metrics.
  -h, --help                       help for self-managed-connectors
      --insecure-skip-tls-verify   Skip TLS certificate verification for the Connect REST endpoint. Usable with any auth method; test environments only.
      --metrics string             Collect Connect worker metrics: 'jolokia' or 'prometheus'. Requires --credentials-file. Endpoints/filters must target the Connect workers, not the Kafka brokers.
      --metrics-duration string    Duration to poll Jolokia metrics (e.g. 5m, 1h). Required with --metrics jolokia.
      --metrics-interval string    Polling interval for Jolokia metrics (e.g. 10s, 30s). Default: 10s. (default "10s")
      --metrics-range string       Day range to query from Prometheus (e.g. 7d, 30d). Required with --metrics prometheus.
      --password string            HTTP Basic password (required when using --use-basic-auth).
      --source-type string         Source type: 'msk' or 'osk'. If not specified, auto-detects from cluster-id format (ARN = MSK, non-ARN = OSK).
      --state-file string          The path to the kcp state file to update with connector information.
      --tls-ca-cert string         Path to a CA certificate that verifies the Connect REST server's TLS certificate. Optional, usable with ANY auth method (including --use-tls) when the endpoint is HTTPS behind a private/internal CA; omit for a public/system-trusted CA.
      --tls-client-cert string     Path to the client certificate presented for mutual TLS (required when using --use-tls).
      --tls-client-key string      Path to the client key presented for mutual TLS (required when using --use-tls).
      --use-basic-auth             Use HTTP Basic authentication for the Connect REST API (requires --username and --password).
      --use-tls                    Use mutual TLS authentication (requires --tls-client-cert and --tls-client-key; add --tls-ca-cert only for a private/internal server CA).
      --use-unauthenticated        Use no authentication.
      --username string            HTTP Basic username (required when using --use-basic-auth).

Options inherited from parent commands

      --verbose   Enable verbose logging to console

SEE ALSO

• kcp scan - Scan AWS resources for migration planning