1.1. Architecture #
pgpro-otel-collector is based on the OpenTelemetry Collector and consists of several components that work together in a pipeline (collection — processing — exporting):
Receivers — the primary components responsible for collecting data. pgpro-otel-collector includes the following receivers:
postgrespro— the central component of the collector, which contains the main implementation (built-in plugins) for collecting data from a database instance.file_log— an auxiliary receiver responsible for collecting information from database logs.Important
fileloghas been renamed tofile_logto comply with the component naming convention. The deprecated alias is still supported but produces warnings and will be completely removed in future releases. Update your configuration accordingly.host_metrics— an auxiliary receiver that collects information from the operating system; may require root access.Important
hostmetricshas been renamed tohost_metricsto comply with the component naming convention. The deprecated alias is still supported but produces warnings and will be completely removed in future releases. Update your configuration accordingly.journald— an auxiliary receiver that collects logs from the systemd journal.otlp— a dedicated receiver for collecting data in the OTLP (OpenTelemetry Protocol) format. It is especially useful for working with Postgres ProGate.sqlquery— an auxiliary receiver that collects metrics and/or logs from custom SQL queries.Warning
The
sqlqueryreceiver is currently experimental and is not recommended for production use.
Processors receive data from receivers and process it. pgpro-otel-collector includes the following processors:
attributesprocesses specific attributes of log records.batchbatches data and regulates the data transmission mechanism.filterallows filtering metrics and logs based on specified conditions using OTTL (OpenTelemetry Transformation Language).memory_limiterlimits the pgpro-otel-collector memory consumption.metrics_transformallows renaming metrics and managing labels, including scaling and aggregation.Important
metricstransformhas been renamed tometrics_transformto comply with the component naming convention. The deprecated alias is still supported but produces warnings and will be completely removed in future releases. Update your configuration accordingly.resourceadds extra log record attributes.transformmodifies the data using OTTL.
Exporters receive processed data from processors and deliver it to external consumers. pgpro-otel-collector supports the following exporters:
kafkasends metrics and logs to Apache Kafka — an open-source distributed event streaming platform.otlp_httpsends information over the OTLP protocol (the standard OpenTelemetry protocol used by many monitoring system vendors) with the help of the HTTP protocol.Important
otlphttphas been renamed tootlp_httpto comply with the component naming convention. The deprecated alias is still supported but produces warnings and will be completely removed in future releases. Update your configuration accordingly.prometheusexports metrics over the built-in HTTP server and publishes them on the/metricspage in the format of the Prometheus monitoring system.prometheusis used by default because it does not require additional configuration.prometheus_remote_writesends metrics via the remote write protocol to Prometheus-compatible systems, including Cortex, Mimir, Thanos, VictoriaMetrics.zabbixsends metrics to Zabbix — an open-source monitoring solution.
Following the OpenTelemetry Collector architecture, the pgpro-otel-collector configuration allows using the same component multiple times. For example, you can use several copies of the same receiver to collect data from numerous DBMS instances. Similarly, the same exporter can be used for exporting data to various storages.
For detailed description of each component, see Section 1.2.