1.1. Architecture #

pgpro-otel-collector is based on the OpenTelemetry Collector and consists of several components that work together in a pipeline (collection — processing — exporting):

  • Receivers — the primary components responsible for collecting data. pgpro-otel-collector includes the following receivers:

    • postgrespro — the central component of the collector, which contains the main implementation (built-in plugins) for collecting data from a database instance.

    • file_log — an auxiliary receiver responsible for collecting information from database logs.

      Important

      filelog has been renamed to file_log to comply with the component naming convention. The deprecated alias is still supported but produces warnings and will be completely removed in future releases. Update your configuration accordingly.

    • host_metrics — an auxiliary receiver that collects information from the operating system; may require root access.

      Important

      hostmetrics has been renamed to host_metrics to comply with the component naming convention. The deprecated alias is still supported but produces warnings and will be completely removed in future releases. Update your configuration accordingly.

    • journald — an auxiliary receiver that collects logs from the systemd journal.

    • otlp — a dedicated receiver for collecting data in the OTLP (OpenTelemetry Protocol) format. It is especially useful for working with Postgres ProGate.

    • sqlquery — an auxiliary receiver that collects metrics and/or logs from custom SQL queries.

      Warning

      The sqlquery receiver is currently experimental and is not recommended for production use.

  • Processors receive data from receivers and process it. pgpro-otel-collector includes the following processors:

    • attributes processes specific attributes of log records.

    • batch batches data and regulates the data transmission mechanism.

    • filter allows filtering metrics and logs based on specified conditions using OTTL (OpenTelemetry Transformation Language).

    • memory_limiter limits the pgpro-otel-collector memory consumption.

    • metrics_transform allows renaming metrics and managing labels, including scaling and aggregation.

      Important

      metricstransform has been renamed to metrics_transform to comply with the component naming convention. The deprecated alias is still supported but produces warnings and will be completely removed in future releases. Update your configuration accordingly.

    • resource adds extra log record attributes.

    • transform modifies the data using OTTL.

  • Exporters receive processed data from processors and deliver it to external consumers. pgpro-otel-collector supports the following exporters:

    • kafka sends metrics and logs to Apache Kafka — an open-source distributed event streaming platform.

    • otlp_http sends information over the OTLP protocol (the standard OpenTelemetry protocol used by many monitoring system vendors) with the help of the HTTP protocol.

      Important

      otlphttp has been renamed to otlp_http to comply with the component naming convention. The deprecated alias is still supported but produces warnings and will be completely removed in future releases. Update your configuration accordingly.

    • prometheus exports metrics over the built-in HTTP server and publishes them on the /metrics page in the format of the Prometheus monitoring system. prometheus is used by default because it does not require additional configuration.

    • prometheus_remote_write sends metrics via the remote write protocol to Prometheus-compatible systems, including Cortex, Mimir, Thanos, VictoriaMetrics.

    • zabbix sends metrics to Zabbix — an open-source monitoring solution.

Following the OpenTelemetry Collector architecture, the pgpro-otel-collector configuration allows using the same component multiple times. For example, you can use several copies of the same receiver to collect data from numerous DBMS instances. Similarly, the same exporter can be used for exporting data to various storages.

For detailed description of each component, see Section 1.2.