Re: Large files for relations - Mailing list pgsql-hackers

From Stephen Frost
Subject Re: Large files for relations
Date
Msg-id ZG+WGPmLHCCVc29a@tamriel.snowman.net
Whole thread Raw
In response to Re: Large files for relations  (Peter Eisentraut <peter.eisentraut@enterprisedb.com>)
Responses Re: Large files for relations
List pgsql-hackers
Greetings,

* Peter Eisentraut (peter.eisentraut@enterprisedb.com) wrote:
> On 24.05.23 02:34, Thomas Munro wrote:
> > Thanks all for the feedback.  It was a nice idea and it *almost*
> > works, but it seems like we just can't drop segmented mode.  And the
> > automatic transition schemes I showed don't make much sense without
> > that goal.
> >
> > What I'm hearing is that something simple like this might be more acceptable:
> >
> > * initdb --rel-segsize (cf --wal-segsize), default unchanged
>
> makes sense

Agreed, this seems alright in general.  Having more initdb-time options
to help with certain use-cases rather than having things be compile-time
is definitely just generally speaking a good direction to be going in,
imv.

> > * pg_upgrade would convert if source and target don't match
>
> This would be good, but it could also be an optional or later feature.

Agreed.

> Maybe that should be a different mode, like --copy-and-adjust-as-necessary,
> so that users would have to opt into what would presumably be slower than
> plain --copy, rather than being surprised by it, if they unwittingly used
> incompatible initdb options.

I'm curious as to why it would be slower than a regular copy..?

> > I would probably also leave out those Windows file API changes, too.
> > --rel-segsize would simply refuse larger sizes until someone does the
> > work on that platform, to keep the initial proposal small.
>
> Those changes from off_t to pgoff_t?  Yes, it would be good to do without
> those.  Apart of the practical problems that have been brought up, this was
> a major annoyance with the proposed patch set IMO.
>
> > I would probably leave the experimental copy_on_write() ideas out too,
> > for separate discussion in a separate proposal.
>
> right

You mean copy_file_range() here, right?

Shouldn't we just add support for that today into pg_upgrade,
independently of this?  Seems like a worthwhile improvement even without
the benefit it would provide to changing segment sizes.

Thanks,

Stephen

Attachment

pgsql-hackers by date:

Previous
From: Tom Lane
Date:
Subject: Re: Why does pg_bsd_indent need to be installed?
Next
From: Heikki Linnakangas
Date:
Subject: Re: ResourceOwner refactoring