3.4. Monitoring Architecture #
To provide monitoring functions related to metrics and logs (hereinafter telemetry), PPEM uses pgpro-otel-collector from Postgres Pro.
Working with telemetry can be organized in two ways:
The repository databases are used to store telemetry: metrics are stored in the separate
monitoringschema, and logs are stored in thelogsschema.External data storages (in relation to PPEM) can be used to store telemetry.
In both cases pgpro-otel-collector is the required component. The collector gathers statistics and logs, generates telemetry data, and, depending on the selected mode, exports data to the manager or to an external storage system.
Note
Using the internal storage is not recommended in production environments, as it has limited scalability and potential performance issues if data volumes grow. Writing and reading large amounts of telemetry can negatively impact PPEM overall performance when accessing the repository database.
3.4.1. Internal Storage #
The diagram below illustrates an example of a monitoring architecture that uses an internal storage.
Figure 3.3. Monitoring Using Internal Storage
Where:
Users work with PPEM through the web application in a browser where they view graphs of metrics and logs.
In PPEM, metrics and logs are stored in the repository. The manager receives metrics and logs from the collector and stores them in repository databases. It also requests metrics and logs from the repository databases and returns them to the consumer via the REST API. The web application queries the manager for metrics and logs and then visualizes the received data.
In the operating system, pgpro-otel-collector connects to the DBMS instance to receive metrics, reads the DBMS log files, processes the received data, and sends it to the manager via the REST API. The agent is a standalone component that provides control functionality and does not collect metrics or logs.
3.4.2. External Storage #
The diagram below illustrates an example of a monitoring architecture that uses an external storage.
Figure 3.4. Monitoring Using External Storage
Where:
Users work with PPEM through the web application in a browser where they view graphs of metrics and logs.
In PPEM, the manager requests metrics and logs from external storages and returns them to the consumer via the REST API. The web application queries the manager for metrics and logs and then visualizes the received data.
In the operating system, pgpro-otel-collector connects to the DBMS instance to receive metrics, reads the DBMS log files, processes the received data, and sends it to external storages via the REST API. The agent is a standalone component that provides control functionality and does not collect metrics or logs.
External storages for metrics and logs:
A dedicated monitoring system can act as a metrics storage. Prometheus, Victoriametrics, and OTLP-compatible storages are supported.
A separate system with OTLP-compatible storage can serve as a log storage. Elasticsearch is supported.