Re: Creating A GIN index on JSONB column (large database) - Mailing list pgsql-general

From Taylor Smith
Subject Re: Creating A GIN index on JSONB column (large database)
Date
Msg-id CAPLtOcni6VzF-ERXY4xNrci+au9G6TWX8cMn+miGzcmb74SX0Q@mail.gmail.com
Whole thread Raw
In response to Re: Creating A GIN index on JSONB column (large database)  (Rob Sargent <robjsargent@gmail.com>)
Responses Re: Creating A GIN index on JSONB column (large database)
List pgsql-general
Yes, the database was always going to be a minimum of 20TB however we build the indexes on the partition which at max is about 1.3TB, which in temp files will use about 800GB temporary files. Note it doesn't have the same effect when doing non-concurrently. I don't think rolling off would help as its likely largest partition will either be the current month or the previous month so the problem would still persist.



On Tue, Aug 9, 2022 at 3:50 PM Rob Sargent <robjsargent@gmail.com> wrote:


> On Aug 9, 2022, at 7:45 AM, Taylor Smith <taylorjrsmith@gmail.com> wrote:
>
> Thanks Rob,
>
> This is what I was thinking. Would you say it presents any risk then having a database unable to rebuild its own indexes if needed?
>
>

No I would not.  The scale-up + rebuild should tell you how much you actually need and if you’ve over done the scale up you can scale down to what you need.  Was the original spec expecting 20+T?  Any chance of rolling older months off to some other node?

pgsql-general by date:

Previous
From: Rob Sargent
Date:
Subject: Re: Creating A GIN index on JSONB column (large database)
Next
From: Ron
Date:
Subject: Re: Creating A GIN index on JSONB column (large database)