Thread: Does Postgres support bookmarks (tuples ids)?

Does Postgres support bookmarks (tuples ids)?

From
Konstantin Izmailov
Date:
Dear experts,
I've noticed that commercial databases (e.g. SQLServer) and some open source databases (e.g. Cubrid) support so called "bookmarks".
As far as I understood, a bookmark allows quickly jump to a row for retrieval or modification.

Here is scenario that I'm trying to deal with:
A BI/ETL application is querying the Postgres database. The queries return lots of rows (36 mil), each is about 1KB or larger.
So I'm using DECLARE CURSOR/FETCH to read the rows into a buffer (size is 10000 rows, and I'm freeing memory for oldest rows).
The application may alter or re-read some previously read rows by the row index.
Problem is: if a row is not in the buffer (freed) the application cannot resolve row index into row itself.
I considered using a unique key to located the row, but unfortunately some queries do no allows determining the most unique key.

I'm thinking, is it possible to retrieve/alter row by its index after a Postgres Cursor have read the row?

The application allows a customer to define DB Schema as well as the queries, so my code does not have a prior knowledge about DB and queries.
It is supposed to provide a certain API with functions based on row indexes. The API was initially designed for SQLServer, so the goal is to migrate the application from SQLServer to Postgres.

Would you recommend a solution?

Thank you
Konstantin

Re: Does Postgres support bookmarks (tuples ids)?

From
Dann Corbit
Date:

Perhaps you want to use the ctid.  You can query it like any other column:

http://www.postgresql.org/docs/9.0/static/ddl-system-columns.html

 

The ctid is not permanent.   An alternative is to create tables with OID values.

 

From: pgsql-general-owner@postgresql.org [mailto:pgsql-general-owner@postgresql.org] On Behalf Of Konstantin Izmailov
Sent: Tuesday, November 30, 2010 9:50 PM
To: pgsql-general@postgresql.org
Subject: [GENERAL] Does Postgres support bookmarks (tuples ids)?

 

Dear experts,
I've noticed that commercial databases (e.g. SQLServer) and some open source databases (e.g. Cubrid) support so called "bookmarks".
As far as I understood, a bookmark allows quickly jump to a row for retrieval or modification.

Here is scenario that I'm trying to deal with:
A BI/ETL application is querying the Postgres database. The queries return lots of rows (36 mil), each is about 1KB or larger.
So I'm using DECLARE CURSOR/FETCH to read the rows into a buffer (size is 10000 rows, and I'm freeing memory for oldest rows).
The application may alter or re-read some previously read rows by the row index.
Problem is: if a row is not in the buffer (freed) the application cannot resolve row index into row itself.
I considered using a unique key to located the row, but unfortunately some queries do no allows determining the most unique key.

I'm thinking, is it possible to retrieve/alter row by its index after a Postgres Cursor have read the row?

The application allows a customer to define DB Schema as well as the queries, so my code does not have a prior knowledge about DB and queries.
It is supposed to provide a certain API with functions based on row indexes. The API was initially designed for SQLServer, so the goal is to migrate the application from SQLServer to Postgres.

Would you recommend a solution?

Thank you
Konstantin

Re: Does Postgres support bookmarks (tuples ids)?

From
Dann Corbit
Date:

Is your application by chance using OLEDB?

If that is the case, then just get a PostgreSQL OLEDB provider that supports bookmarks.

 

From: pgsql-general-owner@postgresql.org [mailto:pgsql-general-owner@postgresql.org] On Behalf Of Konstantin Izmailov
Sent: Tuesday, November 30, 2010 9:50 PM
To: pgsql-general@postgresql.org
Subject: [GENERAL] Does Postgres support bookmarks (tuples ids)?

 

Dear experts,
I've noticed that commercial databases (e.g. SQLServer) and some open source databases (e.g. Cubrid) support so called "bookmarks".
As far as I understood, a bookmark allows quickly jump to a row for retrieval or modification.

Here is scenario that I'm trying to deal with:
A BI/ETL application is querying the Postgres database. The queries return lots of rows (36 mil), each is about 1KB or larger.
So I'm using DECLARE CURSOR/FETCH to read the rows into a buffer (size is 10000 rows, and I'm freeing memory for oldest rows).
The application may alter or re-read some previously read rows by the row index.
Problem is: if a row is not in the buffer (freed) the application cannot resolve row index into row itself.
I considered using a unique key to located the row, but unfortunately some queries do no allows determining the most unique key.

I'm thinking, is it possible to retrieve/alter row by its index after a Postgres Cursor have read the row?

The application allows a customer to define DB Schema as well as the queries, so my code does not have a prior knowledge about DB and queries.
It is supposed to provide a certain API with functions based on row indexes. The API was initially designed for SQLServer, so the goal is to migrate the application from SQLServer to Postgres.

Would you recommend a solution?

Thank you
Konstantin

Re: Does Postgres support bookmarks (tuples ids)?

From
Vick Khera
Date:
On Wed, Dec 1, 2010 at 12:57 AM, Dann Corbit <DCorbit@connx.com> wrote:
> The ctid is not permanent.   An alternative is to create tables with OID
> values.

Creating OIDs gives you zero benefit over having a PK.  They would be
roughly equivalent if you added a unique index on the OID column, but
that is by no means automatic.

Re: Does Postgres support bookmarks (tuples ids)?

From
"Joshua D. Drake"
Date:
On Wed, 2010-12-01 at 09:55 -0500, Vick Khera wrote:
> On Wed, Dec 1, 2010 at 12:57 AM, Dann Corbit <DCorbit@connx.com> wrote:
> > The ctid is not permanent.   An alternative is to create tables with OID
> > values.
>
> Creating OIDs gives you zero benefit over having a PK.  They would be
> roughly equivalent if you added a unique index on the OID column, but
> that is by no means automatic.

It is also a really bad idea considering the global nature of an OID.

JD

>

--
PostgreSQL.org Major Contributor
Command Prompt, Inc: http://www.commandprompt.com/ - 509.416.6579
Consulting, Training, Support, Custom Development, Engineering
http://twitter.com/cmdpromptinc | http://identi.ca/commandprompt


Re: Does Postgres support bookmarks (tuples ids)?

From
Konstantin Izmailov
Date:
Dann,
it is a very good suggestion! How does OLEDB implement the bookmarks? Is there a specific Postgres feature that is used for the bookmarks?

Thank you!
Konstantin

On Tue, Nov 30, 2010 at 11:13 PM, Dann Corbit <DCorbit@connx.com> wrote:

Is your application by chance using OLEDB?

If that is the case, then just get a PostgreSQL OLEDB provider that supports bookmarks.

 

From: pgsql-general-owner@postgresql.org [mailto:pgsql-general-owner@postgresql.org] On Behalf Of Konstantin Izmailov
Sent: Tuesday, November 30, 2010 9:50 PM
To: pgsql-general@postgresql.org
Subject: [GENERAL] Does Postgres support bookmarks (tuples ids)?

 

Dear experts,


I've noticed that commercial databases (e.g. SQLServer) and some open source databases (e.g. Cubrid) support so called "bookmarks".
As far as I understood, a bookmark allows quickly jump to a row for retrieval or modification.

Here is scenario that I'm trying to deal with:
A BI/ETL application is querying the Postgres database. The queries return lots of rows (36 mil), each is about 1KB or larger.
So I'm using DECLARE CURSOR/FETCH to read the rows into a buffer (size is 10000 rows, and I'm freeing memory for oldest rows).
The application may alter or re-read some previously read rows by the row index.
Problem is: if a row is not in the buffer (freed) the application cannot resolve row index into row itself.
I considered using a unique key to located the row, but unfortunately some queries do no allows determining the most unique key.

I'm thinking, is it possible to retrieve/alter row by its index after a Postgres Cursor have read the row?

The application allows a customer to define DB Schema as well as the queries, so my code does not have a prior knowledge about DB and queries.
It is supposed to provide a certain API with functions based on row indexes. The API was initially designed for SQLServer, so the goal is to migrate the application from SQLServer to Postgres.

Would you recommend a solution?

Thank you
Konstantin