Thread: pg_upgrade and schema complexity...
Ran into this when upgrading from 13.11 to 15.3... The pg_restore phase failed with "ERROR: out of shared memory", and recommended that I increase max_locks_per_transaction. Doing so let the process run to completion. It took 12.5 minutes to upgrade a 13GB instance. Soon after, I upgraded a 78GB cluster, and it only took 3.1 minutes. (Both are VMs (same number of CPUs and RAM) connected to the same SAN.) A "pg_dump --schema-only" of the two systems shows that the small-but-slow schema is 5.9M lines. Is this to be expected of such a huge schema? -- Born in Arizona, moved to Babylonia.
On 6/2/23 17:44, Ron wrote: > Ran into this when upgrading from 13.11 to 15.3... > > The pg_restore phase failed with "ERROR: out of shared memory", and > recommended that I increase max_locks_per_transaction. Doing so let the > process run to completion. > > It took 12.5 minutes to upgrade a 13GB instance. Soon after, I upgraded > a 78GB cluster, and it only took 3.1 minutes. Where/how did you measure those sizes? > > (Both are VMs (same number of CPUs and RAM) connected to the same SAN.) > > A "pg_dump --schema-only" of the two systems shows that the > small-but-slow schema is 5.9M lines. Anything special you are doing in this cluster to create all those lines? What is the line count for the other instance? > > Is this to be expected of such a huge schema? > -- Adrian Klaver adrian.klaver@aklaver.com
On 6/2/23 19:58, Adrian Klaver wrote:
> On 6/2/23 17:44, Ron wrote:
>> Ran into this when upgrading from 13.11 to 15.3...
>>
>> The pg_restore phase failed with "ERROR: out of shared memory", and
>> recommended that I increase max_locks_per_transaction. Doing so let the
>> process run to completion.
>>
>> It took 12.5 minutes to upgrade a 13GB instance. Soon after, I upgraded
>> a 78GB cluster, and it only took 3.1 minutes.
>
> Where/how did you measure those sizes?
Does it really matter?
START_SECS=$(date +"%s")
pg_upgrade ...
FINISH_SECS=$(date +"%s")
ET=`echo "scale=2;(${FINISH_SECS} - ${START_SECS})/60" | bc`
date +"%F %T pg_upgrade finished. Elapsed time: ${ET} minutes."
(Text copied between air-gapped computers, so there might be errors.)
>>
>> (Both are VMs (same number of CPUs and RAM) connected to the same SAN.)
>>
>> A "pg_dump --schema-only" of the two systems shows that the
>> small-but-slow schema is 5.9M lines.
>
> Anything special you are doing in this cluster to create all those lines?
I do nothing; the schema is provided by the vendor.
>
> What is the line count for the other instance?
227K rows.
>>
>> Is this to be expected of such a huge schema?
>>
--
Born in Arizona, moved to Babylonia.
> On Jun 2, 2023, at 17:44, Ron <ronljohnsonjr@gmail.com> wrote: > Is this to be expected of such a huge schema? pg_upgrade time is pretty much proportional to the number of database objects in the schema, so a much larger schema takingmuch longer is to be expected.
On 6/2/23 18:06, Ron wrote:
> On 6/2/23 19:58, Adrian Klaver wrote:
>> On 6/2/23 17:44, Ron wrote:
>>> Ran into this when upgrading from 13.11 to 15.3...
>>>
>>> The pg_restore phase failed with "ERROR: out of shared memory", and
>>> recommended that I increase max_locks_per_transaction. Doing so let
>>> the process run to completion.
>>>
>>> It took 12.5 minutes to upgrade a 13GB instance. Soon after, I
>>> upgraded a 78GB cluster, and it only took 3.1 minutes.
>>
>> Where/how did you measure those sizes?
>
> Does it really matter?
>
> START_SECS=$(date +"%s")
> pg_upgrade ...
> FINISH_SECS=$(date +"%s")
> ET=`echo "scale=2;(${FINISH_SECS} - ${START_SECS})/60" | bc`
> date +"%F %T pg_upgrade finished. Elapsed time: ${ET} minutes."
Unless I'm not mistaken the above is how the elapsed time was measured.
I was looking for the procedure for determining the size.
>
> (Text copied between air-gapped computers, so there might be errors.)
>
>>>
>>> (Both are VMs (same number of CPUs and RAM) connected to the same SAN.)
>>>
>>> A "pg_dump --schema-only" of the two systems shows that the
>>> small-but-slow schema is 5.9M lines.
>>
>> Anything special you are doing in this cluster to create all those lines?
>
> I do nothing; the schema is provided by the vendor.
Alright so it is not your design, but you do have an idea of what is in
the database correct?
>
>>
>> What is the line count for the other instance?
>
> 227K rows.
>
>>>
>>> Is this to be expected of such a huge schema?
>>>
>
--
Adrian Klaver
adrian.klaver@aklaver.com
On 6/2/23 21:22, Adrian Klaver wrote:
> On 6/2/23 18:06, Ron wrote:
>> On 6/2/23 19:58, Adrian Klaver wrote:
>>> On 6/2/23 17:44, Ron wrote:
>>>> Ran into this when upgrading from 13.11 to 15.3...
>>>>
>>>> The pg_restore phase failed with "ERROR: out of shared memory", and
>>>> recommended that I increase max_locks_per_transaction. Doing so let the
>>>> process run to completion.
>>>>
>>>> It took 12.5 minutes to upgrade a 13GB instance. Soon after, I
>>>> upgraded a 78GB cluster, and it only took 3.1 minutes.
>>>
>>> Where/how did you measure those sizes?
>>
>> Does it really matter?
>>
>> START_SECS=$(date +"%s")
>> pg_upgrade ...
>> FINISH_SECS=$(date +"%s")
>> ET=`echo "scale=2;(${FINISH_SECS} - ${START_SECS})/60" | bc`
>> date +"%F %T pg_upgrade finished. Elapsed time: ${ET} minutes."
>
> Unless I'm not mistaken the above is how the elapsed time was measured. I
> was looking for the procedure for determining the size.
du -cm -d1 $DATA13
du -cm -d1 $DATA15
>
>>
>> (Text copied between air-gapped computers, so there might be errors.)
>>
>>>>
>>>> (Both are VMs (same number of CPUs and RAM) connected to the same SAN.)
>>>>
>>>> A "pg_dump --schema-only" of the two systems shows that the
>>>> small-but-slow schema is 5.9M lines.
>>>
>>> Anything special you are doing in this cluster to create all those lines?
>>
>> I do nothing; the schema is provided by the vendor.
>
> Alright so it is not your design, but you do have an idea of what is in
> the database correct?
"In" like the /purpose/ of the database? Sure.
"In" like the /design/ of the database? No; that's all on the vendor.
>
>
>>
>>>
>>> What is the line count for the other instance?
>>
>> 227K rows.
>
--
Born in Arizona, moved to Babylonia.