Chapter 20. Storages
Table of Contents
- 20.1. Creating a Storage (
metastore.add_storage)- 20.2. Deleting a Storage (
metastore.remove_storage)- 20.3. Configuring the Parquet File Cache
- 20.4. Reading Storage Data
- 20.2. Deleting a Storage (
- 20.4.1. Reading Parquet Files (
read_parquet)- 20.4.2. Reading CSV Files (
read_csv)- 20.4.3. Reading JSON Files (
read_json)- 20.4.4. Reading Iceberg Tables (
iceberg_scan)- 20.4.5. Reading Iceberg Table Metadata (
iceberg_metadata)- 20.4.6. Reading Iceberg Table Snapshots (
iceberg_snapshots)- 20.4.7. Reading Delta Datasets (
delta_scan) - 20.4.2. Reading CSV Files (
- 20.4.1. Reading Parquet Files (
Storages are physical locations of Parquet files and shared directories.
Postgres Pro AXE supports the following types of storages:
Local storages: Arrays of disks on servers with Postgres Pro AXE installed.
Network File System (NFS) storages.
S3 storages.
You can select the appropriate storage type based on the characteristics shown in the table below.
Local Storage | NFS Storage | S3 Storage | |
|---|---|---|---|
Throughput | High Determined by the number of NVMe disks in the array | Medium Limited by the capacity of the network interface of the server, network load, and NFS storage speed | Medium Limited by the capacity of the network interface of the server, network load, and S3 storage speed |
Data scalability | Medium Determined by the number and volume of NVMe disks | High | High |
Data distribution between servers | Not supported | Within the network of the organization | Global |
High availability provider | Postgres Pro AXE server administrator | NFS administrator | S3 provider |
Cost per terabyte and per one storage request | Low | Medium | Depends on the S3 provider |
Any directory structure can be used for storing the data, for example:
In a local or NFS storage:
root_path/db_name/schema_name/table_name
In an S3 storage:
s3://bucket/db_name/schema_name/table_name
The metadata of storages is stored in the pga_storage metadata table.