Request a Demo
24.3. Integration with VictoriaLogs
Prev UpChapter 24. Integration with External Data SourcesHome Next

24.3. Integration with VictoriaLogs #

24.3.1. Configuring pgpro-otel-collector for VictoriaLogs
24.3.2. Checking Logs in VictoriaLogs
24.3.3. Configuring a Log Data Source
24.3.4. Checking the Log Storage Operation

Integration with external VictoriaLogs data sources is used for reading logs written by pgpro-otel-collector.

The components below are required for integration.

pgpro-otel-collector #

The monitoring agent that provides the following functionality:

  • collects activity logs from Postgres Pro DBMS instances

  • sends activity logs to VictoriaLogs

VictoriaLogs #

The system for activity log management that provides the following functionality:

  • receives activity logs from monitoring agents

  • stores activity logs according to internal storage parameters

  • provides the interface for receiving activity logs

PPEM #

The Postgres Pro Enterprise Manager system that provides the following functionality:

  • accesses the VictoriaLogs system for receiving DBMS instance activity logs

  • provides the user with the monitoring interface in the form of text data based on activity logs

The integration process includes the following steps:

Additional configuration of the agent is not required.

24.3.1. Configuring pgpro-otel-collector for VictoriaLogs #

  1. Enable and configure the filelog receiver.

    Receiver configuration example for the scenario when PostgreSQL logs are generated in the JSON format: 

    receivers:
  filelog:
    include:
    - /var/log/postgresql/*.json
    operators:
    - parse_ints: true
      timestamp:
        layout: '%Y-%m-%d %H:%M:%S.%L %Z'
        layout_type: strptime
        parse_from: attributes.timestamp
      type: json_parser
    - field: attributes.timestamp
      type: remove
    retry_on_failure:
      enabled: true
      initial_interval: 1s
      max_elapsed_time: 5m
      max_interval: 30s
    start_at: end

  2. Configure processors: 

    processors:
  resource:
    attributes:
    - action: upsert
      key: service.name
      value: postgresql
    - action: upsert
      key: service.instance.id
      value: postgresql-01.example.org:5432

    Where:

    • service.name is the key for identifying Postgres Pro logs.

    • service.instance.id is the key for identifying the instance.

  3. Configure log sending using otlphttpexporter and the pipeline: 

    exporters:
  otlphttp/victorialogs:
    compression: gzip
    encoding: proto
    endpoint: https://victorialogs.example.org/insert/opentelemetry/v1/logs
    headers:
      VL-Stream-Fields: service.name,service.instance.id
    tls:
      insecure_skip_verify: false

service:
  extensions: []
  pipelines:
    logs:
      receivers:
      - filelog
      processors:
      - resource
      exporters:
      - otlphttp/victorialogs

  4. Start the collector and ensure that logs are published on its side: 

    # systemctl start pgpro-otel-collector

# systemctl status pgpro-otel-collector
● pgpro-otel-collector.service - PostgresPro OpenTelemetry Collector
    Loaded: loaded (/lib/systemd/system/pgpro-otel-collector.service; enabled; preset: enabled)
    Active: active (running) since Thu 2025-03-20 01:18:08 MSK; 4h 13min ago
  Main PID: 6991 (pgpro-otel-coll)
    Tasks: 8 (limit: 3512)
    Memory: 119.3M
      CPU: 2min 49.311s
    CGroup: /system.slice/pgpro-otel-collector.service
            └─6991 /usr/bin/pgpro-otel-collector --config /etc/pgpro-otel-collector/basic.yml

Mar 20 01:18:08 postgresql-01.example.org pgpro-otel-collector[6991]: {"level":"info","ts":1742422688.366656,"msg":"Setting up own telemetry..."}
Mar 20 01:18:08 postgresql-01.example.org pgpro-otel-collector[6991]: {"level":"info","ts":1742422688.367178,"msg":"Skipped telemetry setup."}
Mar 20 01:18:08 postgresql-01.example.org pgpro-otel-collector[6991]: {"level":"info","ts":1742422688.3679142,"msg":"Development component. May change in the future.","kind":"receiver","name":"postgrespro","data_type":"metrics"}
Mar 20 01:18:08 postgresql-01.example.org pgpro-otel-collector[6991]: {"level":"warn","ts":1742422688.3494158,"caller":"envprovider@v1.16.0/provider.go:59","msg":"Configuration references unset environment variable","name":"POSTGRESQL_P>
Mar 20 01:18:08 postgresql-01.example.org pgpro-otel-collector[6991]: {"level":"info","ts":1742422688.4481084,"msg":"Starting pgpro-otel-collector...","Version":"v0.5.0","NumCPU":1}
Mar 20 01:18:08 postgresql-01.example.org pgpro-otel-collector[6991]: {"level":"info","ts":1742422688.4481149,"msg":"Starting extensions..."}
Mar 20 01:18:08 postgresql-01.example.org pgpro-otel-collector[6991]: {"level":"warn","ts":1742422688.4483361,"msg":"Using the 0.0.0.0 address exposes this server to every network interface, which may facilitate Denial of Service attack>
Mar 20 01:18:08 postgresql-01.example.org pgpro-otel-collector[6991]: {"level":"info","ts":1742422688.4515307,"msg":"Starting stanza receiver","kind":"receiver","name":"filelog","data_type":"logs"}
Mar 20 01:18:08 postgresql-01.example.org pgpro-otel-collector[6991]: {"level":"info","ts":1742422688.451749,"msg":"Everything is ready. Begin running and processing data."}
Mar 20 01:18:08 postgresql-01.example.org pgpro-otel-collector[6991]: {"level":"info","ts":1742422688.6523068,"msg":"Started watching file","kind":"receiver","name":"filelog","data_type":"logs","component":"fileconsumer","path":"/var/log/postgresql/postgresql-2025-03-20.json"}

24.3.2. Checking Logs in VictoriaLogs #

After configuring log sending from pgpro-otel-collector, ensure that logs are received by VictoriaLogs.

For this check, you can execute a query to the storage using the curl utility.

Query example: 

curl https://victorialogs.example.org/select/logsql/query \
  -d query=service.name:postgresql \
  -d start=30m \
  -d end=now \
  -d limit=10

Where:

  • https://victorialogs.example.org is the URL of the log storage system.

  • query=service.name:postgresql is the search query to select all PostgreSQL logs.

  • start=30m and end=now search logs for the last 30 minutes.

  • limit=10 limits the number of returned logs.

Response example: 

{
    "_time":"2026-03-20T17:14:12.153Z",
    "log.file.name":"postgresql-2026-03-20.json",
    "service.instance.id":"postgresql-01.example.org:5432",
    "service.name":"postgresql",
    "backend_type":"pg_wait_sampling collector",
    "error_severity":"LOG",
    "message":"pg_wait_sampling collector started"
}

24.3.3. Configuring a Log Data Source #

  1. In the navigation panel, go to Infrastructure → Data sources → Message storages.

  2. In the top-right corner of the page, click Create storage.

  3. Specify the log storage parameters (parameters marked with an asterisk are required):

    • Message storage system: The type of the log storage system.

      Select VictoriaLogs.

    • Name: The unique name of the log storage. For example, VictoriaLogs.

    • URL: The network address for connecting to the log storage. For example, https://victorialogs.example.org/select/logsql/query.

    • User: The unique name of the user if authentication is used.

    • Password: The password of the user if authentication is used.

    • Description: The description of the log storage.

    • Make default datasource: Specifies whether the log storage is used by default for all queries requesting activity logs.

24.3.4. Checking the Log Storage Operation #

  • In the navigation panel, go to Monitoring → Logs.

    The table of logs will be displayed.

  • To reset filters, click Reset all above the table.

  • Check if new activity logs are listed in the table.

Prev Up Next
24.2. Integration with Elasticsearch and Elastic APM Home Chapter 25. Integration with OpenLDAP and Active Directory
epub pdf