3.8. Integration with Postgres Pro Enterprise Manager #
Postgres Pro Enterprise Manager (PPEM) supports integration with pgpro-otel-collector and can receive metrics and logs.
3.8.1. Setting up Metrics #
To export metrics to PPEM, follow the steps below:
Enable and configure the required metric receivers (as per usual).
Enable and configure the
otlphttpexporter.Specify the PPEM connection URL.
In the header
X-Ppem-Source-Agent-Name, specify the name of the agent that controls the database instance (seeppem-agent.yml).In the header
X-Ppem-Source-Instance-Port, specify the port number of the required DBMS instance (seepostgresql.conf).Set up a pipeline (as per usual).
receivers:
hostmetrics:
# ...
postgrespro:
# ...
exporters:
otlphttp/ppem_metrics:
compression: ''
endpoint: "https://ppem.example.org"
headers:
X-Ppem-Source-Agent-Name: "dbserver01"
X-Ppem-Source-Instance-Port: "5432"
service:
extensions: []
pipelines:
metrics:
receivers:
- postgrespro
- hostmetrics
processors:
exporters:
- otlphttp/ppem_metrics
Make sure you have specified the correct values in the HTTP headers — these headers are used to connect metrics with instances in PPEM.
Note
To avoid excessive resource consumption caused by unused metrics, use the default section in your configuration, as described in Section 3.2.1.9.
3.8.2. Setting up Logs #
To export logs to PPEM, do the following:
Enable and configure the
filelogreceivers according to the database instance logging format (csvlogorjsonlog).Enable and configure the
attributes/convertprocessor according to the database instance logging format (csvlogorjsonlog).Enable and configure the
otlphttpexporter.Specify the PPEM connection URL.
Specify the agent name that controls the required database instance in the header
X-Ppem-Source-Agent-Name(seeppem-agent.yml).Specify the port number of the required DBMS instance in the header
X-Ppem-Source-Instance-Port(seepostgresql.conf).Set up a pipeline (as per usual).
receivers:
filelog:
# ...
processors:
attributes/convert:
# ...
exporters:
otlphttp/ppem_logs:
compression: ''
endpoint: "https://ppem.example.org"
headers:
X-Ppem-Source-Agent-Name: "dbserver01"
X-Ppem-Source-Instance-Port: '5432'
service:
extensions: []
pipelines:
logs:
receivers:
- filelog
processors:
- attributes/convert
exporters:
- otlphttp/ppem_logs
3.8.3. Configuration Example #
The example below shows how to set up the collector configuration for exporting metrics and logs (CSV) to PPEM.
receivers:
filelog:
include:
- /var/log/postgresql/*.csv
multiline:
line_start_pattern: ^[0-9]{4}-[0-9]{2}-[0-9]{2}
operators:
- header: timestamp,user,dbname,pid,connection_from,session_id,line_num,ps,session_start,vxid,txid,error_severity,state_code,message,detail,hint,internal_query,internal_position,context,statement,cursor_position,func_name,application_name,backend_type,leader_pid,query_id
timestamp:
layout: '%Y-%m-%d %H:%M:%S.%L %Z'
layout_type: strptime
parse_from: attributes.timestamp
type: csv_parser
- field: attributes.timestamp
type: remove
retry_on_failure:
enabled: true
initial_interval: 1s
max_elapsed_time: 5m
max_interval: 30s
start_at: end
hostmetrics:
collection_interval: 60s
initial_delay: 1s
scrapers:
cpu:
metrics:
system.cpu.utilization:
enabled: true
disk: null
load: null
memory: null
network: null
postgrespro:
max_threads: 3
initial_delay: 1s
collection_interval: 60s
transport: tcp
endpoint: localhost:5432
database: postgres
username: postgres
password: ${env:POSTGRESQL_PASSWORD}
metrics: null
plugins:
activity:
enabled: true
bgwriter:
enabled: true
locks:
enabled: true
versions:
enabled: true
wal:
enabled: true
cache:
enabled: true
processors:
attributes/convert:
actions:
- action: convert
converted_type: string
key: pid
- action: convert
converted_type: int
key: line_num
- action: convert
converted_type: int
key: txid
- action: convert
converted_type: int
key: remote_port
- action: convert
converted_type: int
key: cursor_position
- action: convert
converted_type: int
key: internal_position
- action: convert
converted_type: int
key: leader_pid
batch:
send_batch_size: 8192
timeout: 10s
resource:
attributes:
- action: upsert
key: service.name
value: postgresql
- action: upsert
key: service.instance.id
value: dbserver-01.example.org:5432
exporters:
otlphttp/ppem_logs:
endpoint: 'https://ppem.example.org'
headers:
X-Ppem-Source-Agent-Name: 'dbserver-01.example.org'
X-Ppem-Source-Instance-Port: '5432'
otlphttp/ppem_metrics:
endpoint: 'https://ppem.example.org'
headers:
X-Ppem-Source-Agent-Name: 'dbserver-01.example.org'
X-Ppem-Source-Instance-Port: '5432'
service:
extensions: []
pipelines:
logs:
receivers:
- filelog
processors:
- resource
- attributes/convert
- batch
exporters:
- otlphttp/ppem_logs
metrics:
receivers:
- postgrespro
- hostmetrics
processors:
- batch
exporters:
- otlphttp/ppem_metrics
An example of setting up journald for sending logs in the journald format to PPEM can be found in Section 3.3.1.2.