Postgres Pro AXE : Documentation: 1: 31.2. Adding a Parquet File to a Local Storage : Postgres Professional

31.2. Adding a Parquet File to a Local Storage
Prev	Up	Chapter 31. Local Storages	Home	Next

31.2. Adding a Parquet File to a Local Storage #

Consider a scenario where the data already exists as a Parquet file and must be loaded to a local storage for analytical queries.

To prepare the environment for this scenario:

Note

If you already have a Parquet file and a local storage configured, skip these steps.

Create directories for Parquet files and grant write access to the operating system user on behalf of whom the Postgres Pro DBMS runs.
These directories store Parquet files of analytical tables and are required to create a local storage.
For example:
```
  sudo -u postgres mkdir -p /tmp/my_storage /tmp/my_storage/tmp_mystorage /tmp/my_storage/data
  sudo chown -R postgres:postgres /tmp/my_storage
```

Create a local storage.

For example:

  SELECT metastore.add_storage(
      'my_storage',
      'file:///tmp/my_storage/',
      'file:///tmp/my_storage/tmp_mystorage/'
  ) ON CONFLICT DO NOTHING;

Place a Parquet file in the local storage or create a Parquet file by exporting the data from a heap table.
For example:
```
  COPY my_table TO '/tmp/my_storage/data/data.parquet' (FORMAT PARQUET) ON CONFLICT DO NOTHING;
```

To configure this scenario:

Create an analytical table from the Parquet file.

Postgres Pro AXE uses analytical tables to register the data in the metadata catalog.

For example:

  SELECT metastore.add_table(
      'analytic_table_from_parquet',
      'my_storage',
      'file:///tmp/my_storage/data/data.parquet',
      ''
  ) ON CONFLICT DO NOTHING;

Create a shared directory.
The shared directory specifies the path to the Parquet file to be added to the analytical table.
For example:
```
  SELECT metastore.add_folder(
      'src_folder',
      'my_storage',
      'data/'
  ) ON CONFLICT DO NOTHING;
```

Add the Parquet file to the analytical table.

When you add the file, it is registered in the metadata catalog.

For example:

  SELECT metastore.add_files(
      'analytic_table_from_parquet',
      'src_folder/data.parquet'
  ) ON CONFLICT DO NOTHING;

Create a view for the analytical table.

The data becomes available for analytical queries through this view.

For example:

  SELECT metastore.create_view('analytic_table_from_parquet') ON CONFLICT DO NOTHING;
  SELECT * FROM analytic_table_from_parquet;

Prev	Up	Next
31.1. Loading the Data from a Heap Table to a Local Storage as Parquet Files	Home	Part V. Appendixes

epub pdf