Thread: another question concerning TOAST

Hi,

after having created my own variable length data type I did not encounter problems... until now:

I have two structures:

typedef struct statistics {
float8 avg; /* average value of timeseries */
float8 min; /* min value */
float8 max; /* max value */
double sum; /* all values of timeseries added */
} STATISTICS;

typedef struct timeseries {
int32 size;         /* size of struct */
int4 nflts;         /* dimension of values[] */
int4 deltat;         /* delta t of values in sec */
STATISTICS stats; /* some statistical information */
float8 values[1];    /* array */
} TIMESERIES;

My declaration in postgres:

CREATE TYPE timeseries(
INPUT=timeseries_in,
OUTPUT=timeseries_out,
INTERNALLENGTH=VARIABLE,
STORAGE=EXTENDED
);

All of this was working pretty fine, until I now seem to try to create a timeseries which is too big.

My database:

CREATE TABLE zeitreihe(
zeit timestamp with time zone,
reihe timeseries
);

At the moment I store timeseries containing values for one day, that means the dimension of the values-array is 92, 96 or 100.
I now try to select multiple rows and create another longer timeseries.
Trying to do so, I sometimes lose the server connection:

server closed the connection unexpectedly
This probably means the server terminated abnormally
before or while processing the request.
The connection to the server was lost. Attempting reset: Failed.
!>

When I append just a small number of daily timeseries, my function is able to return the pointer normally.
But when the dimension of the array reaches 10.000, I lose the connection.

I'm not able to debug postgres with gdb / ddd because I'm a novice (although I'd love to), but the debugging information I log tells me, that there's a segmentation fault which kills the process. 

...
INFO:  repalloc(ts_res) war erfolgreich    //repalloc was successful and did not return a NULL pointer
NOTICE:  ### TS_RES #53 nflts: 5184 - size: 20780
NOTICE:  nach der for-Schleife        // that indicates my very end of the function, the next statement is PG_RETURN_TIMESERIES_P(ts_res)
DEBUG:  reaping dead processes
DEBUG:  child process (PID 6543) was terminated by signal 11
LOG:  server process (PID 6543) was terminated by signal 11
LOG:  terminating any other active server processes
LOG:  all server processes terminated; reinitializing
DEBUG:  shmem_exit(0)
DEBUG:  invoking IpcMemoryCreate(size=10436608)
LOG:  database system was interrupted at 2005-05-30 19:45:05 CEST
LOG:  checkpoint record is at 0/A8CE7B0
LOG:  redo record is at 0/A8CE7B0; undo record is at 0/0; shutdown TRUE
LOG:  next transaction ID: 73597; next OID: 1470196
LOG:  database system was not properly shut down; automatic recovery in progress
FATAL:  the database system is starting up
...

The window I execute the pg_ctl statement shows:
*** glibc detected *** free(): invalid next size (normal): 0x09b94680 ***

I don't know, where I made a mistake. I haven't been thinking of toasting / detoasting yet, although I now think, I have to.
When I got the literature I read right, I don't have to care about toasting, but about detoasting, right?

I wrote my own PG_GETARG and PG_RETURN macros, may I also write my own TIMESERIES_DETOAST macro? Will it work?
Do you think, that this will help me at all?

Any reply will be helpful to me!!!

Thank you very much in advance!

Jessica

PS: Postgres-Version: 7.4.5, OS: Fedora Core 3 (kernel 2.6.11-1.14)

Jessica Ditt <jessica.ditt@web.de> writes:
> I don't know, where I made a mistake. I haven't been thinking of
> toasting / detoasting yet, although I now think, I have to.

If you are going to mark the datatype as toastable then you have to
be prepared to detoast values received by your functions.

Usually it's possible to hide this in your GETARG macro for the type.

            regards, tom lane