6.2. Working with Metrics #
This section describes the steps required to manage metrics.
6.2.1. Adding and Configuring the postgrespro Receiver #
To collect metrics from the database instance, add the postgrespro receiver to the receivers section and specify its configuration.
Required configuration:
Specify the database instance connection parameters.
Specify the list of the plugins for data collection.
Additional configuration:
Collection parameters: parallelism, delay, interval.
receivers:
postgrespro:
max_threads: 3
collection_interval: 60s
initial_delay: 1s
transport: tcp
endpoint: localhost:5432
database: postgres
username: postgres
password: ${env:POSTGRESQL_PASSWORD}
metrics: null
plugins:
activity:
enabled: true
bgwriter:
enabled: true
locks:
enabled: true
version:
enabled: true
wal:
enabled: true
cache:
enabled: true
Some plugins have additional configuration parameters. For example, the plugins for metrics collection from DBMS objects (tablespaces, databases, tables, indexes) can be configured in such a way that the collection will only take place in a specified number of objects. This allows controlling the load on the database instance and the amount of data sent through the pipeline to an exporter. The detailed description of configuration parameters for each plugin can be found in the /usr/share/doc/pgpro-otel-collector/examples directory.
The receiver can also use Unix sockets when the endpoint is defined as shown below.
receivers:
postgrespro:
...
transport: unix
endpoint: /tmp:5432 # Or 'tmp:5432'
...
6.2.2. Adding and Configuring the hostmetrics Receiver #
The hostmetrics receiver is an open-source component of the OpenTelemetry Collector and is used for collecting metrics from the operating system. For detailed information about this receiver, refer to the OpenTelemetry documentation.
To configure the hostmetrics receiver, it is sufficient to list the plugins (scrapers) for data collection. Collection parameters are also available: delay and interval.
Some plugins also have additional configuration parameters.
receivers:
hostmetrics:
collection_interval: 60s
initial_delay: 1s
scrapers:
cpu:
metrics:
system.cpu.utilization:
enabled: true
disk: null
load: null
memory: null
network: null
6.2.3. Adding and Configuring the prometheus Exporter #
The prometheus exporter is an open-source component of the OpenTelemetry Collector. For detailed information, refer to the OpenTelemetry documentation.
prometheus is the easiest to use — it does not require the external component configuration and can be enabled by default. To set it up, it is sufficient to specify the address to listen for incoming requests:
exporters:
prometheus:
endpoint: "1.2.3.4:8889"
send_timestamps: true
6.2.4. Adding and Configuring the otlphttp Exporter #
The otlphttp exporter is an open-source component of the OpenTelemetry Collector and is used for exporting collected logs to an OTLP-compatible storage or monitoring system that has to be predeployed and accessible. For more details, refer to the OpenTelemetry documentation.
To configure the otlphttp exporter, it is sufficient to specify the address of the target system where data should be sent:
exporters:
otlphttp:
endpoint: https://otlp.example.org
6.2.5. Adding and Configuring the kafka Exporter #
The kafka exporter is an open-source component of the OpenTelemetry Collector for sending metrics and logs to Apache Kafka. For more details, refer to the OpenTelemetry documentation.
Below is the example of setting it up for sending metrics.
receivers:
postgrespro:
max_threads: 3
collection_interval: 60s
initial_delay: 1s
transport: tcp
endpoint: localhost:5432
database: postgres
username: postgres
password: ${env:POSTGRESQL_PASSWORD}
metrics: null
plugins:
activity:
enabled: true
bgwriter:
enabled: true
locks:
enabled: true
version:
enabled: true
wal:
enabled: true
cache:
enabled: true
exporters:
kafka:
brokers:
- localhost:9092
protocol_version: 2.1.0
client_id: pgpro-otel-collector
metrics:
topic: otlp_metrics
encoding: otlp_json # proto supported
include_metadata_keys:
- service.name
- service.instance.id
tls:
insecure: true
timeout: 30s
producer:
max_message_bytes: 1000000
required_acks: 1
compression: none # gzip, snappy, lz4, and zstd;
processors:
batch/kafka:
send_batch_size: 1024
timeout: 1s
resource:
attributes:
- key: service.name
action: upsert
value: postgresql
- key: service.instance.id
action: upsert
value: address-of-postgres-instance:5432
service:
pipelines:
metrics/kafka:
receivers: [ postgrespro ]
processors: [ batch/kafka,resource ]
exporters: [ kafka ]
6.2.6. Setting up a Pipeline #
Once receivers and exporters are added and configured, they need to be combined into a pipeline. The pipeline is configured in the service section. The pipeline contents depend altogether on the previously added components (there is no default configuration).
The example below shows how to set up a pipeline for metric management. The data is collected by the postgrespro and hostmetrics receivers, processed by the batch processor and exported by the prometheus and otlphttp exporters.
Thus, all the components used in the pipeline should also be added in the configuration file and set up.
service:
extensions: []
pipelines:
metrics:
receivers:
- postgrespro
- hostmetrics
processors:
- batch
exporters:
- prometheus
- otlphttp