Thread: question on new psql datatype
I am developing a new "image" datatype in postgres which contains a binary field for storing image data and some other fields for additional information about the image, like size, resolution, etc. I was hoping that the clients can saving their time by directly retrieving these information from the fields without retrieving the whole image, that is, the time for reading these fields should be constant with respect to the image size. However, the result shows that the time for direct retrieving of information from the fields of an image datatype is still increasing with the size of the image. My explanation is that in order to read the fields of a image datatype, the whole struct would still need to be loaded into memory, which includes the binary field containing the actual image data. Can anyone tell me is this true for a user- defined datatype (using C struct)? Thanks a lot. Yu
On Sun, Jan 23, 2005 at 04:34:44PM -0700, Yu Pan wrote: > I am developing a new "image" datatype in postgres which contains a binary > field for storing image data and some other fields for additional information > about the image, like size, resolution, etc. I was hoping that the clients can > saving their time by directly retrieving these information from the fields > without retrieving the whole image, that is, the time for reading these fields > should be constant with respect to the image size. However, the result shows > that the time for direct retrieving of information from the fields of an image > datatype is still increasing with the size of the image. My explanation is > that in order to read the fields of a image datatype, the whole struct would > still need to be loaded into memory, which includes the binary field > containing the actual image data. Can anyone tell me is this true for a user- > defined datatype (using C struct)? Thanks a lot. I'd think that if the datatype was stored compressed, then yes, the system needs to load the whole field before being able to access any member. You could try setting the storage type to EXTERNAL. See http://developer.postgresql.org/docs/postgres/storage-toast.html Also read the code related to substring in TEXT and BYTEA in non-compressed mode. HTH, -- Alvaro Herrera (<alvherre[@]dcc.uchile.cl>) "At least to kernel hackers, who really are human, despite occasional rumors to the contrary" (LWN.net)
Hello Yu Pu, Am 2005-01-23 16:34:44, schrieb Yu Pan: > I am developing a new "image" datatype in postgres which contains a binary > field for storing image data and some other fields for additional information > about the image, like size, resolution, etc. I was hoping that the clients can > saving their time by directly retrieving these information from the fields > without retrieving the whole image, that is, the time for reading these fields > should be constant with respect to the image size. However, the result shows Realy cool > that the time for direct retrieving of information from the fields of an image > datatype is still increasing with the size of the image. My explanation is > that in order to read the fields of a image datatype, the whole struct would > still need to be loaded into memory, which includes the binary field > containing the actual image data. Can anyone tell me is this true for a user- > defined datatype (using C struct)? Thanks a lot. No, thats not right. An Image is a Header (image type, width, height, rawsize, colortable) plus the Data In most cases the Header is between 20 and 300 Bytes But which data do you need exactly ? I have done this in Winword 6.0 under WfW 3.11 for 10 years :-) You need only the first Bytes not the whole Image. > Yu Greetings Michelle -- Linux-User #280138 with the Linux Counter, http://counter.li.org/ Michelle Konzack Apt. 917 ICQ #328449886 50, rue de Soultz MSM LinuxMichi 0033/3/88452356 67100 Strasbourg/France IRC #Debian (irc.icq.com)