Re: psql and bytea - Mailing list pgsql-general
From | Féliciano Matias |
---|---|
Subject | Re: psql and bytea |
Date | |
Msg-id | 1053060938.4573.53.camel@one.myworld Whole thread Raw |
In response to | Re: psql and bytea (Tom Lane <tgl@sss.pgh.pa.us>) |
Responses |
Re: psql and bytea
Re: psql and bytea Re: psql and bytea |
List | pgsql-general |
Le jeu 15/05/2003 à 16:47, Tom Lane a écrit : > =?ISO-8859-1?Q?F=E9liciano?= Matias <feliciano.matias@free.fr> writes: > > one=3D> -- this is not the original size (20M : 20971520). > > I don't have any problems with smaller files (10Mo is always fine). > > All of this is used with postgresql 7.3.2 shipped with Red Hat Linux 9 . > > Hmm. There used to be some off-by-one type bugs in psql's > variable-substitution code, but those were fixed long before 7.3.2. > In any case, it's hard to see why such a problem would only arise > when you got past 10Mb string lengths. > More then 10Mb string lengths : > > $ tobytea < data | wc > > 0 81574 74379444 (71 Mo) > I couldn't duplicate the problem here, It seems it's a bug in glibc-2.3.2-27.9.i686.rpm coming with RH9. > so I'm going to suggest that > maybe you have a hardware problem? Perhaps there's a flaky RAM chip in > an area of memory that doesn't get used until you push up the size > of psql quite a bit. It'd be worth running memtest86 for awhile to > check. The hardware is ok. I found the problem. It's bug in sprintf() =============================================== #include <stdio.h> #include <stdlib.h> #include <string.h> #define SIZE (1024*70000) int main(void) { char * s = malloc(SIZE) ; char * d = malloc(SIZE) ; memset(s, 'a', SIZE-1) ; d[SIZE-1] = '\0' ; sprintf(d,"%s",s) ; printf("%zi\n", strlen(d)) ; return 0 ; } =============================================== $ ./a.out 67108863 (2^26-1) My libc have a problem. I will fill a bug report to http://bugzilla.redhat.com/ . I apply this patch to the postgresql source : =============================================== diff -urN postgresql-7.3.2.orig/src/bin/psql/mainloop.c postgresql-7.3.2/src/bin/psql/mainloop.c --- postgresql-7.3.2.orig/src/bin/psql/mainloop.c 2002-10-13 01:09:34.000000000 +0200 +++ postgresql-7.3.2/src/bin/psql/mainloop.c 2003-05-16 04:46:03.000000000 +0200 @@ -389,8 +389,9 @@ exit(EXIT_FAILURE); } - sprintf(new, "%.*s%s%s", i, line, value, - &line[i + thislen + in_length]); + sprintf(new, "%.*s", i, line) ; + strcat(&new[i], value) ; + strcat(&new[i+out_length], &line[i + thislen + in_length]) ; free(line); line = new; =============================================== Also, i want to know if someone is interesting by tools such as (to|from)bytea. I can put this little toys (with some enhancement. "--help" :-) ) in contrib and update the man page of psql to show how to use this tools to copy the content of a file (or stream) into a field. Since Postgresql have binary string, it is interesting and better than \lo_* for little files (this use more memory than \lo_*). By the way, i can add an option to psql that toggle the output of carrier return at the end of each records. Perhaps this can be add to \t. > > regards, tom lane > -- Féliciano Matias <feliciano.matias@free.fr>
Attachment
pgsql-general by date: