Chapter 34. Built-in Analytical Platform

Built-in analytical platform is a Postgres Pro solution designed to operate with OLAP (Online Analytical Processing) workloads. The key component of the platform is pgpro_duckdb, an extension that allows building a modern analytical platform within a Postgres Pro instance without any additional tools.

Modern analytical platforms usually include several independent components, which allows for greater flexibility and scalability. The main components of the system are the query execution engine, data storage, and catalog of analytical tables. The DuckDB pluggable engine supports vectorized query execution and columnar data formats. It can access both Postgres Pro tables and external storages.

The built-in analytical platform offers the following advantages:

  • Allows building a full-scale data warehouse using only Postgres Pro components and makes OLAP functionality available to any Postgres Pro instance.

  • Minimizes the number of components required for building a data warehouse and simplifies the development stack, deployment, and usage of OLAP features.

  • Significantly accelerates the execution of analytical queries when reading data from columnar data formats.

  • Allows building a full-scale ETL process supporting multiple popular file formats and data sources.

  • Provides real-time analytics for hybrid transactional/analytical processing (HTAP) scenarios in which analytical queries can be executed on views that combine Postgres Pro tables and OLAP data sources.

  • Reduces the total cost of ownership (TCO) for hardware. Typically, OLTP and OLAP peak workloads do not occur simultaneously, as OLTP workloads peak during business hours while resource-intensive OLAP operations are usually scheduled for nighttime.

  • Allows using Postgres Pro replicas for executing analytical queries to further improve resource utilization and reduce negative impact on OLTP workloads.

  • Allows building a data warehouse with unlimited scalability by integrating with Tengri Data Platform, an analytical solution by OpenLakeHouse.

  • Allows using the procopy utility for initial and incremental OLAP data loading to analytical storages. Consult the Postgres Pro support team for more information about the procopy utility.

Note

The pgpro_duckdb extension is currently in an experimental phase.