kcp scan self-managed-connectors

kcp scan self-managed-connectors

Scan self-managed Kafka Connect cluster for connector information

Synopsis

Scan a self-managed Kafka Connect cluster using its REST API to discover connector configurations and status. Sensitive config values are redacted before being written to the state file.

kcp scan self-managed-connectors [flags]

Examples

  # Scan connectors for an MSK cluster (auto-detected from ARN format)
  kcp scan self-managed-connectors \
    --state-file kcp-state.json \
    --connect-rest-url http://connect:8083 \
    --cluster-id arn:aws:kafka:us-east-1:123456789012:cluster/my-cluster/abc-123 \
    --use-unauthenticated

  # Scan connectors for an OSK cluster (auto-detected from non-ARN format)
  kcp scan self-managed-connectors \
    --state-file kcp-state.json \
    --connect-rest-url https://connect.example.com:8083 \
    --cluster-id production-kafka \
    --use-sasl-scram \
    --sasl-scram-username admin \
    --sasl-scram-password secret

  # Explicitly specify source type (overrides auto-detection)
  kcp scan self-managed-connectors \
    --state-file kcp-state.json \
    --connect-rest-url http://connect:8083 \
    --cluster-id my-cluster \
    --source-type osk \
    --use-unauthenticated

  # Scan with Jolokia metrics collection
  kcp scan self-managed-connectors \
    --state-file kcp-state.json \
    --connect-rest-url http://connect:8083 \
    --cluster-id my-cluster \
    --use-unauthenticated \
    --metrics jolokia --metrics-duration 5m --metrics-interval 10s \
    --credentials-file osk-credentials.yaml

Options

      --cluster-id string            The cluster identifier in the state file. Accepts both MSK ARNs (arn:aws:kafka:...) and OSK cluster IDs.
      --connect-rest-url string      The Kafka Connect REST API URL (e.g., http://localhost:8083).
      --credentials-file string      Path to the apache-kafka-credentials.yaml file providing Jolokia/Prometheus configuration. Required with --metrics.
  -h, --help                         help for self-managed-connectors
      --metrics string               Collect Connect worker metrics: 'jolokia' or 'prometheus'. Requires --credentials-file. Endpoints/filters must target the Connect workers, not the Kafka brokers.
      --metrics-duration string      Duration to poll Jolokia metrics (e.g. 5m, 1h). Required with --metrics jolokia.
      --metrics-interval string      Polling interval for Jolokia metrics (e.g. 10s, 30s). Default: 10s. (default "10s")
      --metrics-range string         Day range to query from Prometheus (e.g. 7d, 30d). Required with --metrics prometheus.
      --sasl-scram-password string   SASL/SCRAM password (required when using --use-sasl-scram).
      --sasl-scram-username string   SASL/SCRAM username (required when using --use-sasl-scram).
      --source-type string           Source type: 'msk' or 'osk'. If not specified, auto-detects from cluster-id format (ARN = MSK, non-ARN = OSK).
      --state-file string            The path to the kcp state file to update with connector information.
      --tls-ca-cert string           Path to CA certificate file (required when using --use-tls).
      --tls-client-cert string       Path to client certificate file (required when using --use-tls).
      --tls-client-key string        Path to client key file (required when using --use-tls).
      --use-sasl-scram               Use SASL/SCRAM authentication (requires --sasl-scram-username and --sasl-scram-password).
      --use-tls                      Use TLS certificate authentication (requires --tls-ca-cert, --tls-client-cert, and --tls-client-key).
      --use-unauthenticated          Use no authentication.

Options inherited from parent commands

      --verbose   Enable verbose logging to console

SEE ALSO

• kcp scan - Scan AWS resources for migration planning