Congratulations! and some thoughts about C code program architecture - Mailing list pgsql-odbc

It's really great that you did get the job and PostgreSQL will
get a new much better driver.

Congratulations for you and your team members!

Joshua, I wanted to support you also by writing about my thoughts
about ODBC driver architecture and about some improvements,
that I personally have seen a request for.

I hope, that the architecture thoughts about C coding are helpful for you.


I wrote the first message for one purpose:
We need to maintain this driver until the replacement is ready.
I have learned from others, that it is wise to remember to maintain
the current driver until the new one is ready.


So there is a need for ODBC supporters while you and your
team are making the new driver.


I haven't been able to improve the current driver,
but I don't have detailed ODBC specs or time either.

I have been thinkin about one year ago, that this driver
needs something like a rewrite: I wrote about higher
level ODBC objects and read/write functions from the objects into the
backend
messages for many different database backend versions
almost one year ago (it was in the psqlodbc feature request list).

With those objects, you could do something like:
- insert a bunch of rows into the backend and get a bunch of results.
- support different backends by isolating the backend differences
 into object read/write functions to the backend.
- support column conversion by using conversion functions selected at
  Prepare() time (if possible). Then read data from the backend one row
  at a time (the backend sends them continuously).
  Use the already selected conversion functions for the data conversion.
  This way you would avoid duplicated work.
- Do not store a large cache for the client, if you don't need to
(forward only cursor).
  This would give a steady rate of data rows from the backend.

So I miss higher level code on the ODBC driver.

Other ideas:
- The Client driver does the backend->local charset conversions.
  One reason is to move unnecessary work from the backend to the client.
  If the database and the client are on a different server, the
backend's CPU
  load decreases.


I wrote some object based code, but the task was overly large for me
even for an extremely small workable part of the driver with my lack of
time.

I understood also then, that my suggestion might have been very slow
because of many memory allocations ...

Personally I like about FILE * style object hiding:
- Header files contain only the function definitions. I know very little
about
  the struct FILE *. I am able to create, delete and use it only via
functions.
- C code files contain the structure definition and
    implementations of the functions, that use
- Documentation (info libc and manual pages) has the usage of the functions.

A sample:

env.h:
struct Env;
extern struct Env *env_New();

env.c:
/* This is the Env structure ... */
struct Env {
   ...
};
/* env_New() creates a new environment object ...  */
struct  Env *env_New()
{
   struct Env *env;
   env = (struct Env*)malloc(sizeof(struct Env));
   ...
  return env;
}

The above FILE * style object hiding is suitable for library objects,
that do not have lots of cross-dependencies.
One of the important tasks is then to decide the needed FILE * structures,
that create the core of the architecture, and their interdependencies.

For example some object pointers:
Env*
Connection*
Stmt*
BackendPipe*  (might be a socket or a TCP/IP connection object.)
DataRow*
RowDescriptors*
ColDescriptor*

(BackendPipe could be a socket, or a TCP/IP connection.)

By storing all the struct Env {...}; style definitions into their own C
files, you
are able to minimice the interdependencies of the objects,
and that simplifies the implementation and understandability a lot.
I don't know, if that is fast enough though.

It has taken me a lot time to learn, that the *.h files are the incorrect
place for the structure definitions in C code,
because of the interdependency problems!

This way the result might not be as fast, but it is very much
more maintainable.
You can use high enough abstraction level, and it helps a lot.

I hope my ideas about good coding in C are helpful for you.
(I know, that these ideas are applicable for general libraries.)

I haven't looked at the libpq code. It is very good, that you have
an excellent and stable code base to continue with.

(Maybe a spanish virus checker does not understand the pseudo email
addresses
 into this mailing list. The reply did get caught by that virus checker.
 Checker thought, that I have a Windows virus, although I'm running
Debian Linux.)

Sincerely with congratulations,

Marko Ristola


Joshua D. Drake wrote:
...