Metric labels
ClickPipes metrics use the standard service-level labels (clickhouse_org, clickhouse_service, clickhouse_service_name) plus the following ClickPipes-specific labels:
| Label | Description |
|---|---|
clickpipe_id | The unique identifier of the ClickPipe |
clickpipe_name | The name of the ClickPipe |
clickpipe_source | The source type (e.g., kafka, postgres, s3) |
clickpipe_state | The state of the ClickPipe (e.g., Stopped, Provisioning, Running) |
Sample response
The following shows what ClickPipes metrics look like in a Prometheus scrape response:# HELP ClickPipes_Info Always equal to 1. Label "clickpipe_state" contains the current state of the pipe: Stopped/Provisioning/Running/Paused/Failed
# TYPE ClickPipes_Info gauge
ClickPipes_Info{clickhouse_org="11dfa1ec-767d-43cb-bfad-618ce2aaf959",clickhouse_service="82b83b6a-5568-4a82-aa78-fed9239db83f",clickhouse_service_name="my service",clickpipe_id="642bb967-940b-459e-9f63-a2833f62ec44",clickpipe_name="my kafka pipe",clickpipe_source="confluent",clickpipe_state="Running"} 1
# HELP ClickPipes_FetchedEvents_Total Total number of records fetched from the source.
# TYPE ClickPipes_FetchedEvents_Total counter
ClickPipes_FetchedEvents_Total{clickhouse_org="11dfa1ec-767d-43cb-bfad-618ce2aaf959",clickhouse_service="82b83b6a-5568-4a82-aa78-fed9239db83f",clickhouse_service_name="my service",clickpipe_id="642bb967-940b-459e-9f63-a2833f62ec44",clickpipe_name="my kafka pipe",clickpipe_source="confluent"} 5535376
# HELP ClickPipes_SentEvents_Total Total number of records sent to ClickHouse
# TYPE ClickPipes_SentEvents_Total counter
ClickPipes_SentEvents_Total{clickhouse_org="11dfa1ec-767d-43cb-bfad-618ce2aaf959",clickhouse_service="82b83b6a-5568-4a82-aa78-fed9239db83f",clickhouse_service_name="my service",clickpipe_id="642bb967-940b-459e-9f63-a2833f62ec44",clickpipe_name="my kafka pipe",clickpipe_source="confluent"} 5534250
# HELP ClickPipes_FetchedBytes_Total Total uncompressed bytes fetched from the source.
# TYPE ClickPipes_FetchedBytes_Total counter
ClickPipes_FetchedBytes_Total{clickhouse_org="11dfa1ec-767d-43cb-bfad-618ce2aaf959",clickhouse_service="82b83b6a-5568-4a82-aa78-fed9239db83f",clickhouse_service_name="my service",clickpipe_id="642bb967-940b-459e-9f63-a2833f62ec44",clickpipe_name="my kafka pipe",clickpipe_source="confluent"} 873286202
# HELP ClickPipes_SentBytes_Total Total uncompressed bytes sent to ClickHouse.
# TYPE ClickPipes_SentBytes_Total counter
ClickPipes_SentBytes_Total{clickhouse_org="11dfa1ec-767d-43cb-bfad-618ce2aaf959",clickhouse_service="82b83b6a-5568-4a82-aa78-fed9239db83f",clickhouse_service_name="my service",clickpipe_id="642bb967-940b-459e-9f63-a2833f62ec44",clickpipe_name="my kafka pipe",clickpipe_source="confluent"} 477187967
# HELP ClickPipes_Errors_Total Total errors ingesting data.
# TYPE ClickPipes_Errors_Total counter
ClickPipes_Errors_Total{clickhouse_org="11dfa1ec-767d-43cb-bfad-618ce2aaf959",clickhouse_service="82b83b6a-5568-4a82-aa78-fed9239db83f",clickhouse_service_name="my service",clickpipe_id="642bb967-940b-459e-9f63-a2833f62ec44",clickpipe_name="my kafka pipe",clickpipe_source="confluent"} 0
# HELP ClickPipes_Latency Time in milliseconds between when the record was produced to when it was written to ClickHouse.
# TYPE ClickPipes_Latency gauge
ClickPipes_Latency{clickhouse_org="11dfa1ec-767d-43cb-bfad-618ce2aaf959",clickhouse_service="82b83b6a-5568-4a82-aa78-fed9239db83f",clickhouse_service_name="my service",clickpipe_id="642bb967-940b-459e-9f63-a2833f62ec44",clickpipe_name="my kafka pipe",clickpipe_source="confluent"} 340
Available metrics
Data transfer
| Metric | Type | Description |
|---|---|---|
ClickPipes_FetchedBytes_Total | Counter | Total uncompressed bytes fetched from the source. |
ClickPipes_FetchedBytesCompressed_Total | Counter | Total compressed bytes fetched from the source. If the source data is uncompressed, equivalent to ClickPipes_FetchedBytes_Total. |
ClickPipes_FetchedBytesInitialLoad_Total | Counter | CDC ClickPipes Total uncompressed bytes fetched during the initial load phase. |
ClickPipes_FetchedBytesResync_Total | Counter | CDC ClickPipes Total uncompressed bytes fetched during a resync operation. |
ClickPipes_SentBytes_Total | Counter | Total uncompressed bytes sent to ClickHouse. |
ClickPipes_SentBytesCompressed_Total | Counter | Total compressed bytes sent to ClickHouse. |
Events and records
| Metric | Type | Description |
|---|---|---|
ClickPipes_FetchedEvents_Total | Counter | Total number of records fetched from the source. |
ClickPipes_SentEvents_Total | Counter | Total number of records sent to ClickHouse. |
ClickPipes_FetchedEvents_PerTable | Gauge | CDC ClickPipes Number of records fetched per destination table and event type. Uses additional destination_table and event_type labels. |
Errors
| Metric | Type | Description |
|---|---|---|
ClickPipes_Errors_Total | Counter | Total errors encountered during ingestion. |
Latency
| Metric | Type | Description |
|---|---|---|
ClickPipes_Latency | Gauge (avg) | Time in milliseconds between when a record was produced at the source and when it was written to ClickHouse. |
ClickPipes_SourceReplicationLatency_MiB | Gauge (avg) | CDC ClickPipes Replication lag at the source in MiB. For Postgres ClickPipes, this reflects the replication slot lag. |
Resource usage
| Metric | Type | Description |
|---|---|---|
ClickPipes_Replica_CPUUsage | Gauge (avg) | Average CPU usage of the ingestion replicas, in millicores. |
ClickPipes_Replica_CPULimit | Gauge (last) | Configured CPU limit of the ingestion replicas, in millicores. |
ClickPipes_Replica_MemoryUsage | Gauge (avg) | Average memory usage of the ingestion replicas, in bytes. |
ClickPipes_Replica_MemoryLimit | Gauge (last) | Configured memory limit of the ingestion replicas, in bytes. |
State and progress
| Metric | Type | Description |
|---|---|---|
ClickPipes_Info | Gauge | Always equal to 1. Use the clickpipe_state label to filter or alert on pipe state. |
ClickPipes_LastFetchedBatchId | Gauge | ID of the last batch fetched from the source. |
ClickPipes_LastSentBatchId | Gauge | ID of the last batch sent to ClickHouse. |
Related pages
- Prometheus endpoint — Endpoint reference, authentication, and setup for external tools (e.g., Grafana, Datadog)
- Error reporting — Where record and system errors are stored