Thread: [PATCH] pgarchives: Fix database install procedure: remove redundant tables in schema.sql
[PATCH] pgarchives: Fix database install procedure: remove redundant tables in schema.sql
From
Célestin Matte
Date:
Hello, While installing pgarchives, I've encountered issues that are addressed in this patch. Django's model and schema.sql both contain tables that are necessary for the execution of the application, but are mutually exclusive. This is due to some fields in the "messages" table that cannot be defined in django, and definition of tables in schema.sql that are already created by django. Additionally, I have two questions: - Where is the "tsparser" parser defined? (See commit e05f813b of pgarchives). It is used in schema.sql, but I haven't found its definition in the pgarchives, pglister or pgweb repositories. Is it an alias to pg_catalog.pg_ts_parser? (I have been able to complete the install procedure by reverting e05f813b, but I have no idea what the consequences are for the application) - Same question for /usr/share/postgresql/12/tsearch_data/pg_dict.stop I've found pg_dict.syn in pgweb, but not this file. Cheers, -- Célestin Matte
Attachment
Re: [PATCH] pgarchives: Fix database install procedure: remove redundant tables in schema.sql
From
Célestin Matte
Date:
Hello, Apologies, patch sent in previous email of this thread is incorrect. Please find correct patch attached. In fact, the pgarchives repository is in a weird state, in which the migration adding two tables is missing (0005_apiclient_threadsubscription.pywhen running makemigrations). My guess is that this is because these tables use SERIALs,which django does not handle properly, and must by created using raw SQL. Am I right? Is it something that should be fixed? I would still need answers for the two questions asked in previous email: > Additionally, I have two questions: > - Where is the "tsparser" parser defined? (See commit e05f813b of pgarchives). > It is used in schema.sql, but I haven't found its definition in the > pgarchives, pglister or pgweb repositories. Is it an alias to > pg_catalog.pg_ts_parser? > (I have been able to complete the install procedure by reverting e05f813b, > but I have no idea what the consequences are for the application) > - Same question for /usr/share/postgresql/12/tsearch_data/pg_dict.stop > I've found pg_dict.syn in pgweb, but not this file. > > Cheers, > Cheers, -- Célestin Matte
Attachment
Re: [PATCH] pgarchives: Fix database install procedure: remove redundant tables in schema.sql
From
Magnus Hagander
Date:
On Mon, Oct 18, 2021 at 6:19 PM Célestin Matte <celestin.matte@cmatte.me> wrote:
Hello,
While installing pgarchives, I've encountered issues that are addressed in this patch.
Django's model and schema.sql both contain tables that are necessary for
the execution of the application, but are mutually exclusive.
This is due to some fields in the "messages" table that cannot be defined in
django, and definition of tables in schema.sql that are already created
by django.
Ugh, yeah that is clearly a mess. I thought we had cleaned that up at some point, but clearly we haven't, and tracking changse has been inconsistent.
I think the correct solution to the problem is to actually get rid of loader/sql/schema.sql and move it all into the django migrations. Trying to keep track of half in one place and half in another is just going to lead to more problems down the road.
So your removal of messages there is a good start, but adding some columns back after the fact in a different file is a big ugh. We should just make all that happen in 0001_initial.py.
Additionally, I have two questions:
- Where is the "tsparser" parser defined? (See commit e05f813b of pgarchives).
It is used in schema.sql, but I haven't found its definition in the
pgarchives, pglister or pgweb repositories. Is it an alias to
pg_catalog.pg_ts_parser?
It's this one over here: https://github.com/postgrespro/pg_tsparser
(I have been able to complete the install procedure by reverting e05f813b,
but I have no idea what the consequences are for the application)
- Same question for /usr/share/postgresql/12/tsearch_data/pg_dict.stop
I've found pg_dict.syn in pgweb, but not this file.
This is just a textfile with stop words. interestingly enough the prod version for the archives just has the word "sql" in it -- I think there was a great big plan to do somethign good with that, that turned out to never happen.
Re: [PATCH] pgarchives: Fix database install procedure: remove redundant tables in schema.sql
From
Magnus Hagander
Date:
On Fri, Oct 22, 2021 at 5:20 PM Célestin Matte <celestin.matte@cmatte.me> wrote:
Hello,
Apologies, patch sent in previous email of this thread is incorrect. Please find correct patch attached.
In fact, the pgarchives repository is in a weird state, in which the migration adding two tables is missing (0005_apiclient_threadsubscription.py when running makemigrations). My guess is that this is because these tables use SERIALs, which django does not handle properly, and must by created using raw SQL. Am I right?
Is it something that should be fixed?
Ugfh, yeah that one is really embarrassing on my part, and I thought I'd cleaned that one up but clearly not. The whole threadsubscription part wasn't finished, and was not supposed to be included in 4d159ca8b4efb0b1cf0893bca63d6e9489a9095f.
The correct solution there is to revert most of 4d159ca8b4efb0b1cf0893bca63d6e9489a9095f, to have it include only the parts that are supposed to be there. And then reapply them at some time in the future when the code is finished. I'll go clean that up -- thanks for bringing my attention back to that!