1.2. pgpro-otel-collector Components #
This section contains description of the components included in pgpro-otel-collector.
Each component, depending on its type, should be added to the receivers, processors, or exporters section accordingly, and configured if necessary. The component configuration depends on the database instance setup and the log format used (see the logging_collector and log_destination parameters).
1.2.1. Receivers #
1.2.1.1. postgrespro #
The postgrespro receiver is the central component of the collector, which contains the main implementation (built-in plugins) for collecting data from a database instance. This receiver is a product of Postgres Pro internal development and is not part of the open-source OpenTelemetry ecosystem.
The receiver is designed with a modular approach, allowing control over the volume of collected data by enabling specific plugins. Some plugins support additional configuration.
Plugins are internal components of a receiver responsible for collecting various types of statistical data. Each plugin is designed to collect specific information, which is then converted into metric groups. All metric groups are listed in Chapter 4.
pgpro-otel-collector provides metric groups that describe various subsystems, events, and processes within the database management system (DBMS), such as table and index usage, background processes, resource usage by queries, and more.
The amount of statistical data may vary depending on the Postgres Pro version and edition. This adds up to variability and complexity of collection methods. Another important role of pgpro-otel-collector is to abstract these differences from monitoring systems while providing comprehensive information about the database.
1.2.1.1.1. postgrespro Plugins #
The postgrespro receiver includes the following plugins:
- activity #
Retrieves general activity in the instance from the
pg_stat_activityview.See the Activity Metrics section.
- archiver #
Retrieves Write-Ahead Logging (WAL) archiving activity from the
pg_stat_archiverview.See the WAL Archiving Metrics section.
- bgwriter #
Retrieves background writer and checkpointer process activity from the
pg_stat_bgwriterview.See the Background Record Metrics section.
- biha #
Retrieves BiHA cluster node statistics from the
biha.status_vandbiha.nodes_vviews.See the BiHA Metrics section.
- bloat_indexes #
Retrieves information about index bloat.
See the Table and Index Bloat Metrics section.
- bloat_tables #
Retrieves information about table bloat.
See the Table and Index Bloat Metrics section.
- buffercache #
Monitors shared cache usage (requires the pg_buffercache extension).
See the Shared Cache Usage Metrics section.
- cache #
Collects cache usage statistics from the
pg_stat_databaseview.See the Database Usage Metrics section.
- checkpointer #
Retrieves the cluster checkpointer process activity from the
pg_stat_checkpointerview.See the Background Record Metrics section.
- cfs #
Monitors CFS (Compressed File System) compression usage (available in Postgres Pro Enterprise only).
See the CFS Metrics section.
- databases #
Retrieves database usage statistics from the
pg_stat_databaseview.See the Database Usage Metrics section.
- functions #
Retrieves user function usage statistics from the
pg_stat_user_functionsview.See the User-Defined Function Usage Metrics section.
- health #
Retrieves health-related statistics, for example, uptime.
See the Other Metrics section.
- indexes #
Retrieves index usage statistics from the
pg_stat_user_indexesandpg_statio_user_indexesviews.See the Index Usage Metrics section.
- io #
Collects Input/Output (I/O) metrics from the
pg_stat_ioview.See the I/O Usage Metrics section.
- locks #
Retrieves lock and wait state information from the
pg_locksview.See the Lock Metrics section.
- logs #
Collects log statistics (for example, log directory size) using the
pg_ls_logdir()function.See the Other Metrics section.
- pg_stat_slru #
Collects SLRU (simple least-recently-used) cache statistics from the
pg_stat_slruview.See the Other Metrics section.
- pgpro_statements_generic #
Collects general statistics from the
pgpro_stats_statementsview (requires the pgpro_stats extension).See the SQL Query and Statement Metrics section.
- pgpro_statements_inval_msgs #
Gathers the number of cache invalidation messages from the
pgpro_stats_statementsview (requires the pgpro_stats extension).See the SQL Query and Statement Metrics section.
- pgpro_statements_jit #
Collects statistics on Just-In-Time (JIT) compiler usage by statements from the
pgpro_stats_statementsview (requires the pgpro_stats extension).See the SQL Query and Statement Metrics section.
- pgpro_statements_local_io #
Collects statistics on local cache usage by statements from the
pgpro_stats_statementsview (requires the pgpro_stats extension).See the SQL Query and Statement Metrics section.
- pgpro_statements_parallel #
Collects parallel statistics from the
pgpro_stats_statementsview (requires the pgpro_stats extension).See the SQL Query and Statement Metrics section.
- pgpro_statements_planning #
Collects statistics on statement planning time from the
pgpro_stats_statementsview (requires the pgpro_stats extension).See the SQL Query and Statement Metrics section.
- pgpro_statements_plans #
Gathers plan texts from the
pgpro_stats_statementsview (requires the pgpro_stats extension).See the SQL Query and Statement Metrics section.
- pgpro_statements_queries #
Retrieves information about statement texts from the
pgpro_stats_statementsview (requires the pgpro_stats extension).See the SQL Query and Statement Metrics section.
- pgpro_statements_rusage #
Gathers resource usage statistics of the statement planning and execution from the
pgpro_stats_statementsview (requires the pgpro_stats extension).See the SQL Query and Statement Metrics section.
- pgpro_statements_shared_io #
Collects statistics on shared buffer usage by statements from the
pgpro_stats_statementsview (requires the pgpro_stats extension).See the SQL Query and Statement Metrics section.
- pgpro_statements_temp_io #
Collects statistics on temporary file usage by statements from the
pgpro_stats_statementsview (requires the pgpro_stats extension).See the SQL Query and Statement Metrics section.
- pgpro_statements_timing #
Collects statistics on statement execution time from the
pgpro_stats_statementsview (requires the pgpro_stats extension).See the SQL Query and Statement Metrics section.
- pgpro_statements_wait_stats #
Gathers statistics on wait events for each execution of the query from the
pgpro_stats_statementsview (requires the pgpro_stats extension).See the SQL Query and Statement Metrics section.
- pgpro_statements_wal #
Collects statistics on the amount of WAL written by statements from the
pgpro_stats_statementsview (requires the pgpro_stats extension).See the SQL Query and Statement Metrics section.
- prepared_transactions #
Retrieves statistics on prepared transactions from the
pg_prepared_xactsview.See the Prepared Transaction Metrics section.
- recovery #
Collects recovery statistics from the
pg_stat_recovery_prefetchandpg_stat_database_conflictsviews.See the Recovery Metrics section.
- replication #
Retrieves metrics about standby nodes and streaming replication from the
pg_stat_replicationandpg_stat_wal_receiverviews.See the Replication Metrics section.
- replication_slots #
Retrieves replication slot usage statistics from the
pg_replication_slotsview.See the Replication Slot Metrics section.
- sequences #
Collects sequence usage statistics from the
pg_sequencesview.See the Sequence Metrics section.
- settings_info #
Collects run-time parameter statistics from the
pg_settingsview.See the Other Metrics section.
- statements_generic #
Collects general statistics about statements from the
pg_stat_statementsview.See the SQL Query and Statement Metrics section.
- statements_timing #
Collects statistics on statement execution time from the
pg_stat_statementsview.See the SQL Query and Statement Metrics section.
- statements_parallel #
Collects parallel statistics from the
pg_stat_statementsview.See the SQL Query and Statement Metrics section.
- statements_planning #
Collects statistics on statement planning time from the
pg_stat_statementsview.See the SQL Query and Statement Metrics section.
- statements_shared_io #
Collects statistics on shared buffer usage by statements from the
pg_stat_statementsview.See the SQL Query and Statement Metrics section.
- statements_local_io #
Collects statistics on local cache usage by statements from the
pg_stat_statementsview.See the SQL Query and Statement Metrics section.
- statements_temp_io #
Collects statistics on temporary file usage by statements from the
pg_stat_statementsview.See the SQL Query and Statement Metrics section.
- statements_wal #
Collects statistics on the amount of WAL written by statements from the
pg_stat_statementsview.See the SQL Query and Statement Metrics section.
- statements_jit #
Collects statistics on Just-In-Time (JIT) compiler usage by statements from the
pg_stat_statementsview.See the SQL Query and Statement Metrics section.
- statements_queries #
Retrieves information about statement texts from the
pg_stat_statementsview.See the SQL Query and Statement Metrics section.
- subscription #
Collects subscription statistics from the
pg_stat_subscriptionandpg_stat_subscription_statsviews.See the Subscription Metrics section.
- tables #
Retrieves table-level access and tuple statistics from the pg_stat_all_tables view.
See the Table Usage Metrics section.
- tables_io #
Retrieves table-level I/O statistics from the pg_statio_all_tables view.
See the Table Usage Metrics section.
- tables_size #
Retrieves table, heap, toast size statistics from the pg_tables view.
See the Table Usage Metrics section.
- tables_vacuum #
Retrieves table-level vacuum and analyze statistics from the pg_stat_all_tables view.
See the Table Usage Metrics section.
- tablespaces #
Retrieves tablespace usage statistics.
See the Tablespace Usage Metrics section.
- version #
Retrieves metrics with Postgres Pro version information.
See the Other Metrics section.
- wait_sampling #
Retrieves metrics based on wait event samples (requires the pg_wait_sampling extension).
See the Wait Event Metrics section.
- wal #
Retrieves statistics on the amount of WAL written by the instance from the pg_stat_wal view.
See the Tablespace Usage Metrics section.
Warning
Enabling extra plugins can lead to high overload on the DBMS instance and should be considered carefully.
The following plugins are disabled by default:
bloat_indexes and bloat_tables. Bloat estimation is a resource-intensive process, and, in the case of numerous tables, can create excessive load. It is recommended to limit data collection to specific databases, schemas, or even tables. Additionally, bloat data changes infrequently and may contain inaccuracies, so it is advisable to set a long interval between data collections — for example, 5 minutes or more.
indexes and tables. Tables and indexes contain a large amount of statistics. When there are many tables and indexes, a significant volume of metrics may be generated, potentially causing performance issues when writing to the metric storage. Additionally, not all of these metrics are valuable to the user. It is recommended to limit data collection by selecting only specific databases, schemas, or tables.
statements. Depending on the load on the DBMS instance, the volume of statistics on SQL statements can be huge. This may result in a significant number of metrics generated and increased write load on the metric storage. Due to the specific nature of statement statistics, they do not always provide sufficient user value. It is recommended to enable only those plugins that are truly useful for your specific use case.
statements_queries. Depending on the length of queries in the statement statistics, the metrics can incur significant overhead when writing to and storing in the metric storage. It is recommended to configure the collector to limit the maximum length of values in the metrics.
1.2.1.2. filelog #
The filelog receiver is an open-source component of the OpenTelemetry Collector for gathering logs from the DBMS instance. The collector supports log collection in the CSV and JSON formats.
1.2.1.3. hostmetrics #
The hostmetrics receiver is an open-source component of the OpenTelemetry Collector for retrieving metrics from the operating system (may require root access).
1.2.1.4. journald #
The journald receiver is an open-source component of the OpenTelemetry Collector for gathering logs from the systemd journal, which is particularly useful for Shardman and Postgres Pro Enterprise Manager (PPEM).
1.2.2. Processors #
1.2.2.1. attributes #
The attributes processor is an open-source component of the OpenTelemetry Collector for processing specific log record attributes.
1.2.2.2. resource #
The resource processor is an open-source component of the OpenTelemetry Collector for adding extra log record attributes.
1.2.2.3. batch #
The batch processor is an open-source component of the OpenTelemetry Collector for data batching and regulating the data transmission mechanism.
1.2.2.4. transform #
The transform processor is an open-source component of the OpenTelemetry Collector for modifying telemetry data using the OpenTelemetry Transformation Language (OTTL).
1.2.2.5. memory_limiter #
The memory_limiter processor is an open-source component of the OpenTelemetry Collector for preventing out-of-memory errors by limiting memory consumption.
1.2.2.6. filter #
The filter processor is an open-source component of the OpenTelemetry Collector that allows including or excluding telemetry data based on specified conditions.
1.2.2.7. metricstransform #
The metricstransform processor is an open-source component of the OpenTelemetry Collector that allows renaming metrics and managing (adding, renaming, deleting) labels, including scaling and aggregation operations.
1.2.3. Exporters #
1.2.3.1. prometheus #
The prometheus exporter is an open-source component of the OpenTelemetry Collector for exporting collected metrics over the HTTP server and publishing them on the /metrics page in the format of the Prometheus monitoring system. This exporter is the easiest to use — it does not require any external component configuration and can be enabled by default.
1.2.3.2. otlphttp #
The otlphttp exporter is an open-source component of the OpenTelemetry Collector for exporting collected metrics to an OTLP-compatible storage or monitoring system that has to be predeployed and accessible. otlphttp uses the HTTP protocol.
1.2.3.3. kafka #
The kafka exporter is an open-source component of the OpenTelemetry Collector for sending metrics and logs to Apache Kafka.
1.2.3.4. zabbix #
The zabbix exporter is a component of the collector for sending metrics to Zabbix. For usage examples, refer to Integration with Zabbix.