Thread: Postgresql Database and PG_WAL locations
I have a database that's about 2TB in size, and I want to place the database files in a separate mount point(PGDATA) and the log files in a different mount point(PG_WAL). What's your take on this?.
Thank you
Henry Ashu
Database Administrator
o: 503.672.5114 | f: 800.551.8821 | DAT.com
On Aug 29, 2024, at 12:18 PM, Henry Ashu <henry.ashu@dat.com> wrote:I have a database that's about 2TB in size, and I want to place the database files in a separate mount point(PGDATA) and the log files in a different mount point(PG_WAL). What's your take on this?.
Essentially, yes, you will want your WAL and data stored on different devices (or the very least, different partitions).
Thank you
Henry Ashu
Database Administrator
o: 503.672.5114 | f: 800.551.8821 | DAT.com
On Thu, Aug 29, 2024 at 2:22 PM Matthew Tice <mjtice@gmail.com> wrote:
On Aug 29, 2024, at 12:18 PM, Henry Ashu <henry.ashu@dat.com> wrote:I have a database that's about 2TB in size, and I want to place the database files in a separate mount point(PGDATA) and the log files in a different mount point(PG_WAL). What's your take on this?.Essentially, yes, you will want your WAL and data stored on different devices (or the very least, different partitions).
Is that recommendation still valid? After all, that was written when 15 years old Sun Studio 12 was still pertinent. Times have changed since then. Disks are much, much bigger.
Death to America, and butter sauce.
Iraq lobster!
On Aug 29, 2024, at 1:02 PM, Ron Johnson <ronljohnsonjr@gmail.com> wrote:On Thu, Aug 29, 2024 at 2:22 PM Matthew Tice <mjtice@gmail.com> wrote:On Aug 29, 2024, at 12:18 PM, Henry Ashu <henry.ashu@dat.com> wrote:I have a database that's about 2TB in size, and I want to place the database files in a separate mount point(PGDATA) and the log files in a different mount point(PG_WAL). What's your take on this?.Essentially, yes, you will want your WAL and data stored on different devices (or the very least, different partitions).Is that recommendation still valid? After all, that was written when 15 years old Sun Studio 12 was still pertinent. Times have changed since then. Disks are much, much bigger.
Good point. I didn’t even notice that disclaimer at the top "Seek knowledge elsewhere, as the material in here predates Postgres 9 it seems.”…
But regarding the wal/data separation, I still think it’s relevant (isolating the IO between data files and wal). But I guess like anything else it depends on your workload (and, in this case, your storage subsystem capabilities).
--Death to America, and butter sauce.Iraq lobster!
> On Aug 29, 2024, at 1:02 PM, Ron Johnson <ronljohnsonjr@gmail.com> wrote: > > Is that recommendation still valid? After all, that was written when 15 years old Sun Studio 12 was still pertinent. Times have changed since then. Disks are much, much bigger. We still have WAL and data having the same data written (at somewhat different times), so it's still useful to give themseparate bandwidth to storage--for databases with high write loads.
On Thu, 2024-08-29 at 15:02 -0400, Ron Johnson wrote: > On Thu, Aug 29, 2024 at 2:22 PM Matthew Tice <mjtice@gmail.com> wrote: > > > On Aug 29, 2024, at 12:18 PM, Henry Ashu <henry.ashu@dat.com> wrote: > > > > > > I have a database that's about 2TB in size, and I want to place the database > > > files in a separate mount point(PGDATA) and the log files in a different mount > > > point(PG_WAL). What's your take on this?. > > > > > > Take a look at https://wiki.postgresql.org/wiki/Installation_and_Administration_Best_practices > > > > Essentially, yes, you will want your WAL and data stored on different devices > > (or the very least, different partitions). > > Is that recommendation still valid? After all, that was written when 15 years old > Sun Studio 12 was still pertinent. Times have changed since then. Disks are much, much bigger. I think the advice is still valid. Today you'd have different filesystems on different logical volumes rather than different physical disks, but it is still a good idea to separate data and WAL, so that they cannot fill up each other's file system. I'd actually define a third file system for the PostgreSQL log files, for the same reason. Yours, Laurenz Albe
On Mon, Sep 2, 2024 at 6:30 AM Laurenz Albe <laurenz.albe@cybertec.at> wrote:
On Thu, 2024-08-29 at 15:02 -0400, Ron Johnson wrote:
> On Thu, Aug 29, 2024 at 2:22 PM Matthew Tice <mjtice@gmail.com> wrote:
> > > On Aug 29, 2024, at 12:18 PM, Henry Ashu <henry.ashu@dat.com> wrote:
> > >
> > > I have a database that's about 2TB in size, and I want to place the database
> > > files in a separate mount point(PGDATA) and the log files in a different mount
> > > point(PG_WAL). What's your take on this?.
> >
> >
> > Take a look at https://wiki.postgresql.org/wiki/Installation_and_Administration_Best_practices
> >
> > Essentially, yes, you will want your WAL and data stored on different devices
> > (or the very least, different partitions).
>
> Is that recommendation still valid? After all, that was written when 15 years old
> Sun Studio 12 was still pertinent. Times have changed since then. Disks are much, much bigger.
I think the advice is still valid.
Today you'd have different filesystems on different logical volumes rather
than different physical disks,
None of our disks are physical; they're all SAN LUNs.
but it is still a good idea to separate data and WAL,
so that they cannot fill up each other's file system.
Regular checkpoints, transactions(*) that don't stay open for hours or days, and monitoring replication to ensure that it keeps replicating data instead of piling up on the primary server all solve that problem
Honestly... it's been YEARS since I've seen that problem.
Besides, "disks are cheap", right?
*COPY statements don't count.
I'd actually define a third file system for the PostgreSQL log files, for
the same reason.
Ditto the PgBackrest directory: isolate PG data from other application and OS data.
We'd have to put them on separate mount points anyway, since the VM build process doesn't like large / and /boot disks.
Death to America, and butter sauce.
Iraq lobster!
On Mon, 2024-09-02 at 09:15 -0400, Ron Johnson wrote: > > but it is still a good idea to separate data and WAL, > > so that they cannot fill up each other's file system. > > Regular checkpoints, transactions(*) that don't stay open for hours > or days, and monitoring replication to ensure that it keeps replicating > data instead of piling up on the primary server all solve that problem Right, if everything is working as it should, monitoring is in place etc. this will never happen. But I believe that paranoia is a virtue for the DBA, and it is better to have a second line of defense than a crash of the database. > Honestly... it's been YEARS since I've seen that problem. I see it all the time (on systems I didn't set up). Yours, Laurenz Albe