Thread: ERROR: insufficient columns in the PRIMARY KEY constraint definition

ERROR: insufficient columns in the PRIMARY KEY constraint definition

From
Nagaraj Raj
Date:

I recently upgraded the database from PostgreSQL v9.6 to v11.7. we have some partitioned table with `inherence` and planning to migrate them to the `declaration`.

Table DDL:


CREATE TABLE c_account_p
(
    billing_account_guid character varying(40)  NOT NULL,
    ingestion_process_id bigint NOT NULL DEFAULT '-1'::integer,
    load_dttm timestamp(6) without time zone NOT NULL,
    ban integer NOT NULL,
    CONSTRAINT billing_account_pkey PRIMARY KEY (billing_account_guid, ban)
) PARTITION by RANGE(load_dttm);



When I try the create table, it's throwing below error:

ERROR:  insufficient columns in the PRIMARY KEY constraint definition
DETAIL:  PRIMARY KEY constraint on table "l_billing_account_p" lacks column "load_dttm" which is part of the partition key.
SQL state: 0A000


Is it mandatory/necessary that the `partition column` should be a primary key? cause if I  include `load_dttm` as `PK` then its working fine.

If the partition column should be supposed to be a PK, it's challenging to create a partition by range with the date column, cause the load_dttm column chances to have duplicate if data loaded `COPY`.


Could some please help me to understand this scenario? 

Thanks.




Re: ERROR: insufficient columns in the PRIMARY KEY constraint definition

From
Nagaraj Raj
Date:
On Tuesday, September 29, 2020, 02:36:17 PM PDT, Nagaraj Raj <nagaraj.sf@yahoo.com> wrote:


I recently upgraded the database from PostgreSQL v9.6 to v11.7. we have some partitioned table with `inherence` and planning to migrate them to the `declaration`.

Table DDL:


CREATE TABLE c_account_p
(
    billing_account_guid character varying(40)  NOT NULL,
    ingestion_process_id bigint NOT NULL DEFAULT '-1'::integer,
    load_dttm timestamp(6) without time zone NOT NULL,
    ban integer NOT NULL,
    CONSTRAINT billing_account_pkey PRIMARY KEY (billing_account_guid, ban)
) PARTITION by RANGE(load_dttm);



When I try the create table, it's throwing below error:

ERROR:  insufficient columns in the PRIMARY KEY constraint definition
DETAIL:  PRIMARY KEY constraint on table "l_billing_account_p" lacks column "load_dttm" which is part of the partition key.
SQL state: 0A000


Is it mandatory/necessary that the `partition column` should be a primary key? cause if I  include `load_dttm` as `PK` then its working fine.

If the partition column should be supposed to be a PK, it's challenging to create a partition by range with the date column, cause the load_dttm column chances to have duplicate if data loaded `COPY`.


Could some please help me to understand this scenario? 

Thanks.




Re: ERROR: insufficient columns in the PRIMARY KEY constraint definition

From
David Rowley
Date:
On Wed, 30 Sep 2020 at 10:36, Nagaraj Raj <nagaraj.sf@yahoo.com> wrote:
> Is it mandatory/necessary that the `partition column` should be a primary key? cause if I  include `load_dttm` as
`PK`then its working fine.
 

Yes, this is required.  There is no concept of an index over all
partitions in PostgreSQL.  The requirement of having the partition key
a subset of the primary key allows us to implement primary keys by
just having individual unique indexes on each partition.  The fact
that it does not work is not a bug.

There's mention in [1] section 5.10.2.3. "Unique constraints on
partitioned tables must include all the partition key columns. This
limitation exists because PostgreSQL can only enforce uniqueness in
each partition individually.". That text likely should also mention
PRIMARY KEY constraints. That probably should be changed

David

[1] https://www.postgresql.org/docs/11/ddl-partitioning.html



Re: ERROR: insufficient columns in the PRIMARY KEY constraint definition

From
Tom Lane
Date:
David Rowley <dgrowleyml@gmail.com> writes:
> On Wed, 30 Sep 2020 at 10:36, Nagaraj Raj <nagaraj.sf@yahoo.com> wrote:
>> Is it mandatory/necessary that the `partition column` should be a primary key? cause if I  include `load_dttm` as
`PK`then its working fine. 

> Yes, this is required.

Indeed.  However, this complaint shows that the error message is not clear
enough.  I propose changing it to

ERROR: unique constraint on partitioned table must be a superset of the partitioning columns

or perhaps

ERROR: unique constraint on partitioned table must include all partitioning columns

The DETAIL seems fine as-is:

DETAIL:  PRIMARY KEY constraint on table "l_billing_account_p" lacks column "load_dttm" which is part of the partition
key.

> There's mention in [1] section 5.10.2.3. "Unique constraints on
> partitioned tables must include all the partition key columns. This
> limitation exists because PostgreSQL can only enforce uniqueness in
> each partition individually.". That text likely should also mention
> PRIMARY KEY constraints. That probably should be changed

Meh.  If you've read that bit you probably already understand that
pkeys are unique constraints.  I think the problem is with the error
text not the docs.

            regards, tom lane



Re: ERROR: insufficient columns in the PRIMARY KEY constraint definition

From
David Rowley
Date:
On Wed, 30 Sep 2020 at 11:22, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>
> David Rowley <dgrowleyml@gmail.com> writes:
> > On Wed, 30 Sep 2020 at 10:36, Nagaraj Raj <nagaraj.sf@yahoo.com> wrote:
> >> Is it mandatory/necessary that the `partition column` should be a primary key? cause if I  include `load_dttm` as
`PK`then its working fine.
 
>
> > Yes, this is required.
>
> Indeed.  However, this complaint shows that the error message is not clear
> enough.  I propose changing it to
>
> ERROR: unique constraint on partitioned table must be a superset of the partitioning columns
>
> or perhaps
>
> ERROR: unique constraint on partitioned table must include all partitioning columns

I prefer the former. Although I'd rather see the constraint type
mentioned explicitly rather than using the word "unique" regardless of
what the constraint type is.

> The DETAIL seems fine as-is:
>
> DETAIL:  PRIMARY KEY constraint on table "l_billing_account_p" lacks column "load_dttm" which is part of the
partitionkey.
 
>
> > There's mention in [1] section 5.10.2.3. "Unique constraints on
> > partitioned tables must include all the partition key columns. This
> > limitation exists because PostgreSQL can only enforce uniqueness in
> > each partition individually.". That text likely should also mention
> > PRIMARY KEY constraints. That probably should be changed
>
> Meh.  If you've read that bit you probably already understand that
> pkeys are unique constraints.  I think the problem is with the error
> text not the docs.

I think you're assuming too much.  If you don't think too hard about
it, it might seem reasonable that we can implement something for a
primary key constraint, because there can only be at most 1 per table,
but not a unique constraint there can be any number.

David



Re: ERROR: insufficient columns in the PRIMARY KEY constraint definition

From
Tom Lane
Date:
David Rowley <dgrowleyml@gmail.com> writes:
> On Wed, 30 Sep 2020 at 11:22, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>> Indeed.  However, this complaint shows that the error message is not clear
>> enough.  I propose changing it to
>> ERROR: unique constraint on partitioned table must be a superset of the partitioning columns
>> or perhaps
>> ERROR: unique constraint on partitioned table must include all partitioning columns

> I prefer the former. Although I'd rather see the constraint type
> mentioned explicitly rather than using the word "unique" regardless of
> what the constraint type is.

If the DETAIL has that, which it does, I'd rather have the primary text
lay out the general rule not a specific case.  Anybody who is not clear
that a pkey is a kind of unique constraint will get that from reading the
DETAIL.  Also, avoiding that insertion should allow more robust
translation of the primary message, which is something we should worry
about if we are concerned that users won't understand the message.

FWIW, after a few minutes' more thought I think I like the second wording;
it's just as precise and does not rely on two-dollar words.  I'm not dead
set on it though.

            regards, tom lane



Re: ERROR: insufficient columns in the PRIMARY KEY constraint definition

From
Alvaro Herrera
Date:
On 2020-Sep-29, Tom Lane wrote:

> Indeed.  However, this complaint shows that the error message is not clear
> enough.  I propose changing it to
> 
> ERROR: unique constraint on partitioned table must be a superset of the partitioning columns
> 
> or perhaps
> 
> ERROR: unique constraint on partitioned table must include all partitioning columns

Either of those work for me.  How common is the word "superset" in
English?  We don't seem seem to use it either in error messages, though
it does occur in docs.

> > There's mention in [1] section 5.10.2.3. "Unique constraints on
> > partitioned tables must include all the partition key columns. This
> > limitation exists because PostgreSQL can only enforce uniqueness in
> > each partition individually.". That text likely should also mention
> > PRIMARY KEY constraints. That probably should be changed
> 
> Meh.  If you've read that bit you probably already understand that
> pkeys are unique constraints.  I think the problem is with the error
> text not the docs.

Maybe mention PKs in parens:

"Unique constraints (and[, by extension,] primary key constraints) on
partitioned tables must include all the partition key columns. This
limitation exists because PostgreSQL can only enforce uniqueness in each
partition individually.". That text likely should also mention PRIMARY
KEY constraints. That probably should be changed

-- 
Álvaro Herrera                https://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services



Re: ERROR: insufficient columns in the PRIMARY KEY constraint definition

From
Tom Lane
Date:
Alvaro Herrera <alvherre@2ndquadrant.com> writes:
> On 2020-Sep-29, Tom Lane wrote:
>> Meh.  If you've read that bit you probably already understand that
>> pkeys are unique constraints.  I think the problem is with the error
>> text not the docs.

> Maybe mention PKs in parens:

> "Unique constraints (and[, by extension,] primary key constraints) on
> partitioned tables must include all the partition key columns. This
> limitation exists because PostgreSQL can only enforce uniqueness in each
> partition individually.".

I don't object to clarifying that (and that wording seems fine), but I
think fixing the error message is more important.  We'd not be having this
discussion if the OP had found that documentation.

            regards, tom lane



Re: ERROR: insufficient columns in the PRIMARY KEY constraint definition

From
David Rowley
Date:
On Wed, 30 Sep 2020 at 11:43, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>
> Alvaro Herrera <alvherre@2ndquadrant.com> writes:
> > On 2020-Sep-29, Tom Lane wrote:
> >> Meh.  If you've read that bit you probably already understand that
> >> pkeys are unique constraints.  I think the problem is with the error
> >> text not the docs.
>
> > Maybe mention PKs in parens:
>
> > "Unique constraints (and[, by extension,] primary key constraints) on
> > partitioned tables must include all the partition key columns. This
> > limitation exists because PostgreSQL can only enforce uniqueness in each
> > partition individually.".
>
> I don't object to clarifying that (and that wording seems fine), but I
> think fixing the error message is more important.  We'd not be having this
> discussion if the OP had found that documentation.

I do agree the error message is the first thing we should be changing.

I'll write a doc patch if you handle the error message.

David



Re: ERROR: insufficient columns in the PRIMARY KEY constraint definition

From
David Rowley
Date:
On Wed, 30 Sep 2020 at 12:08, David Rowley <dgrowleyml@gmail.com> wrote:
>
> On Wed, 30 Sep 2020 at 11:43, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> >
> > Alvaro Herrera <alvherre@2ndquadrant.com> writes:
> > > "Unique constraints (and[, by extension,] primary key constraints) on
> > > partitioned tables must include all the partition key columns. This
> > > limitation exists because PostgreSQL can only enforce uniqueness in each
> > > partition individually.".
> >
> > I don't object to clarifying that (and that wording seems fine), but I
> > think fixing the error message is more important.  We'd not be having this
> > discussion if the OP had found that documentation.
>
> I do agree the error message is the first thing we should be changing.
>
> I'll write a doc patch if you handle the error message.

I didn't go with the same wording.  The reason was that I didn't feel
the word "constraint" had to be mentioned twice.

I won't object if you or Alvaro want to keep Alvaro's suggestion though.

David

Attachment

Re: ERROR: insufficient columns in the PRIMARY KEY constraint definition

From
Tom Lane
Date:
David Rowley <dgrowleyml@gmail.com> writes:
> On Wed, 30 Sep 2020 at 12:08, David Rowley <dgrowleyml@gmail.com> wrote:
>> Alvaro Herrera <alvherre@2ndquadrant.com> writes:
>>> "Unique constraints (and[, by extension,] primary key constraints) on
>>> partitioned tables must include all the partition key columns.

> I didn't go with the same wording.  The reason was that I didn't feel
> the word "constraint" had to be mentioned twice.
> I won't object if you or Alvaro want to keep Alvaro's suggestion though.

I kind of like Alvaro's wording because it helps to reinforce the point
that pkeys are a type of unique constraint.  If you dislike repeating
"constraints", perhaps we could go with something like

Unique constraints (and hence primary keys) on partitioned tables ...

I'm not hugely against your wording though.

            regards, tom lane



Re: ERROR: insufficient columns in the PRIMARY KEY constraint definition

From
Alvaro Herrera
Date:
On 2020-Sep-30, David Rowley wrote:

> I didn't go with the same wording.  The reason was that I didn't feel
> the word "constraint" had to be mentioned twice.
> 
> I won't object if you or Alvaro want to keep Alvaro's suggestion though.

*Shrug* this seems good enough.  A purist could complain that it is
redundant, but in practice it's not important.

Here's the proposed error message fix, using the wording that saves
$1.99.  I agree that trying to cram the constraint type in the primary
message is uglier.

-- 
Álvaro Herrera                https://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Attachment

Re: ERROR: insufficient columns in the PRIMARY KEY constraint definition

From
David Rowley
Date:
On Wed, 30 Sep 2020 at 12:21, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>
> I kind of like Alvaro's wording because it helps to reinforce the point
> that pkeys are a type of unique constraint.  If you dislike repeating
> "constraints", perhaps we could go with something like
>
> Unique constraints (and hence primary keys) on partitioned tables ...

ok, cool. I'll go with that.  Thanks

David



Re: ERROR: insufficient columns in the PRIMARY KEY constraint definition

From
Tom Lane
Date:
Alvaro Herrera <alvherre@2ndquadrant.com> writes:
> Here's the proposed error message fix, using the wording that saves
> $1.99.  I agree that trying to cram the constraint type in the primary
> message is uglier.

WFM.

            regards, tom lane



Re: ERROR: insufficient columns in the PRIMARY KEY constraint definition

From
David Rowley
Date:
On Wed, 30 Sep 2020 at 13:06, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>
> Alvaro Herrera <alvherre@2ndquadrant.com> writes:
> > Here's the proposed error message fix, using the wording that saves
> > $1.99.  I agree that trying to cram the constraint type in the primary
> > message is uglier.
>
> WFM.

Looks good to me too.

David



Re: ERROR: insufficient columns in the PRIMARY KEY constraint definition

From
Nagaraj Raj
Date:
Noted, thank you all. 

On Tuesday, September 29, 2020, 05:12:40 PM PDT, David Rowley <dgrowleyml@gmail.com> wrote:


On Wed, 30 Sep 2020 at 13:06, Tom Lane <tgl@sss.pgh.pa.us> wrote:

>
> Alvaro Herrera <alvherre@2ndquadrant.com> writes:
> > Here's the proposed error message fix, using the wording that saves
> > $1.99.  I agree that trying to cram the constraint type in the primary
> > message is uglier.
>
> WFM.


Looks good to me too.

David



RE: [EXTERNAL] Re: ERROR: insufficient columns in the PRIMARY KEY constraint definition

From
"Godfrin, Philippe E"
Date:

I am curious why this is considered Class 0A, versus 42 (syntax error?)

 

From: Alvaro Herrera <alvherre@2ndquadrant.com>
Sent: Tuesday, September 29, 2020 6:22 PM
To: David Rowley <dgrowleyml@gmail.com>
Cc: Tom Lane <tgl@sss.pgh.pa.us>; Nagaraj Raj <nagaraj.sf@yahoo.com>; Pg Bugs <pgsql-bugs@postgresql.org>
Subject: [EXTERNAL] Re: ERROR: insufficient columns in the PRIMARY KEY constraint definition

 

On 2020-Sep-30, David Rowley wrote:

> I didn't go with the same wording. The reason was that I didn't feel
> the word "constraint" had to be mentioned twice.
>
> I won't object if you or Alvaro want to keep Alvaro's suggestion though.

*Shrug* this seems good enough. A purist could complain that it is
redundant, but in practice it's not important.

Here's the proposed error message fix, using the wording that saves
$1.99. I agree that trying to cram the constraint type in the primary
message is uglier.

--
Álvaro Herrera https://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

"Godfrin, Philippe E" <Philippe.Godfrin@nov.com> writes:
> I am curious why this is considered Class 0A, versus 42 (syntax error?)

FEATURE_NOT_SUPPORTED seems perfectly appropriate from here.
What you tried to do is not semantically nonsensical, it's just something
we can't do given the current architecture for partitioned tables.

            regards, tom lane



Re: ERROR: insufficient columns in the PRIMARY KEY constraint definition

From
Alvaro Herrera
Date:
On 2020-Sep-30, David Rowley wrote:

> On Wed, 30 Sep 2020 at 13:06, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> >
> > Alvaro Herrera <alvherre@2ndquadrant.com> writes:
> > > Here's the proposed error message fix, using the wording that saves
> > > $1.99.  I agree that trying to cram the constraint type in the primary
> > > message is uglier.
> >
> > WFM.
> 
> Looks good to me too.

Great, thanks, pushed now.