Thread: Receiving data in binary format how is it encoded?
Hi all, while implementing the protocol 3.0 I could get the data in text mode setting the format to 0. (Thanks all for helping me out with flush message) Now, I want to test the binary format. While some primitive types are easy to realize, for example a int4 field will be encoded in 4 bytes, I don't know how other types, like date, time, timestamp, text will be encoded in binary format. I tried to find in the code how it would be encoded, but I end up in some dinamically called functions to do the output and I don't know where this functions are defined. Can you provide me some tips where the functions are or the binary format used to encode the types? While writing this mail, I saw the docs about binary types... So, my assumption about int4 be encoded in 4 bytes may be wrong? I mean, the return of an int4 value, for example 12345 in text would be the chars 12345, but in binary format would be the bytes of 1 2 3 4 5?? Instead of a 4 bytes int4 value? The first 4 bytes of bytea is its length? I think I saw something about this in the code. So in the example below I would get a first 4 bytes containing the length (5) of the bytes? Also, just question: If I have a field which has binary type and I use the extended protocol to set all result fields to text, will this field also be sent in text, regardless its type? Thanks in advance! -- Regards, Francisco Figueiredo Jr. ------ "My grandfather once told me that there are two kinds of people: those who work and those who take the credit. He told me to try to be in the first group; there was less competition there." - Indira Gandhi
Hello: > while implementing the protocol 3.0 I could get the data in text mode > setting the format to 0. (Thanks all for helping me out with flush message) > > > Now, I want to test the binary format. While some primitive types are > easy to realize, for example a int4 field will be encoded in 4 bytes, > I don't know how other types, like date, time, timestamp, text will be > encoded in binary format. Huummmmm i'm going to comment how i make in my C# implementation (stream is a binary reader ) for example for int4, date and time. For int4: int val = stream.ReadInt32(); int int4Value = IPAddress.HostToNetworkOrder(val); For date ( not sure if i'm making this correct way but values seems to be correct :) ) DateTime base = new DateTime(2000, 1, 1) int val = stream.ReadInt32(); int days = IPAddress.HostToNetworkOrder(val); DateTime date = base.AddDays(days); For Time: The same as for date but reading a double and adding it to base as seconds instead of days. The only basic data type i'm not handling now is numeric that seems to be encoded in 10 bytes but i have no seen the postgre source code yet. -- Best regards Carlos Guzmán Álvarez Vigo-Spain P.S.: Has you receive an email i sent to you abot little things i see on npgsql ?? :)
Carlos Guzman Alvarez wrote: > Hello: > >> while implementing the protocol 3.0 I could get the data in text mode >> setting the format to 0. (Thanks all for helping me out with flush >> message) >> >> >> Now, I want to test the binary format. While some primitive types are >> easy to realize, for example a int4 field will be encoded in 4 bytes, >> I don't know how other types, like date, time, timestamp, text will be >> encoded in binary format. > > > Huummmmm i'm going to comment how i make in my C# implementation (stream > is a binary reader ) for example for int4, date and time. > Thanks. > For int4: > > int val = stream.ReadInt32(); > int int4Value = IPAddress.HostToNetworkOrder(val); > Yeah, while waiting for the response, I added support for the int4 datatype. It is really encoded as a 4 bytes value. > > For date ( not sure if i'm making this correct way but values seems to > be correct :) ) > > DateTime base = new DateTime(2000, 1, 1) > int val = stream.ReadInt32(); > int days = IPAddress.HostToNetworkOrder(val); > > DateTime date = base.AddDays(days); > > Uhmmm, where did you take the 2000,1,1 as base from? If the date is before 2000, days will have a negative value, so date is calculated correctly backwards? > For Time: > > The same as for date but reading a double and adding it to base as > seconds instead of days. > > > > The only basic data type i'm not handling now is numeric that seems to > be encoded in 10 bytes but i have no seen the postgre source code yet. > > Thanks Carlos. do you know in what file are these values produced? -- Regards, Francisco Figueiredo Jr. ------ "My grandfather once told me that there are two kinds of people: those who work and those who take the credit. He told me to try to be in the first group; there was less competition there." - Indira Gandhi
Hello: > Yeah, while waiting for the response, I added support for the int4 > datatype. It is really encoded as a 4 bytes value. :) > Uhmmm, where did you take the 2000,1,1 as base from? > If the date is before 2000, days will have a negative value, so date is > calculated correctly backwards? In the little tests i do, yes. > Thanks Carlos. do you know in what file are these values produced? Not sure but i'm reviewing files in directory: src\backend\utils\adt This is what seems to be sent for numeric datatype i'm going to test it now :D: pq_sendint(&buf, x.ndigits, sizeof(int16));pq_sendint(&buf, x.weight, sizeof(int16));pq_sendint(&buf, x.sign, sizeof(int16));pq_sendint(&buf,x.dscale, sizeof(int16));for (i = 0; i < x.ndigits; i++) pq_sendint(&buf, x.digits[i],sizeof(NumericDigit)); -- Best regards Carlos Guzmán Álvarez Vigo-Spain
Carlos Guzman Alvarez wrote: > Hello: > Hello >> Yeah, while waiting for the response, I added support for the int4 >> datatype. It is really encoded as a 4 bytes value. > > > :) > >> Uhmmm, where did you take the 2000,1,1 as base from? >> If the date is before 2000, days will have a negative value, so date >> is calculated correctly backwards? > > > In the little tests i do, yes. Good. I think I found where you get 200,1,1 from: In datetime.c in the AST dir, there is a line which says: Assert(POSTGRES_EPOCH_JDATE == date2j(2000, 1, 1)); > > >> Thanks Carlos. do you know in what file are these values produced? > > > Not sure but i'm reviewing files in directory: > > src\backend\utils\adt > > This is what seems to be sent for numeric datatype i'm going to test it > now :D: > > pq_sendint(&buf, x.ndigits, sizeof(int16)); > pq_sendint(&buf, x.weight, sizeof(int16)); > pq_sendint(&buf, x.sign, sizeof(int16)); > pq_sendint(&buf, x.dscale, sizeof(int16)); > for (i = 0; i < x.ndigits; i++) > pq_sendint(&buf, x.digits[i], sizeof(NumericDigit)); > > Thanks. I will have a look in the types in the AST dir. -- Regards, Francisco Figueiredo Jr. ------ "My grandfather once told me that there are two kinds of people: those who work and those who take the credit. He told me to try to be in the first group; there was less competition there." - Indira Gandhi