fix for libpq choking on varlen fields - Mailing list pgsql-hackers

I broke it, I'm guilty :-(.  Here's the fix.

In view of recent discussions about eliminating the fixed limit
on tuple sizes, a better long-term solution would be to increase
the width of field size values from 2 bytes to 4 in the FE/BE
protocol.  I suspect we will end up doing that someday, so maybe
we should bite the bullet and do it now as part of the "2.0" FE/BE
protocol revision.  On the other hand, we did promise a few weeks ago
that we were done changing the FE/BE protocol for 6.4.  Comments?

            regards, tom lane


*** src/interfaces/libpq/fe-exec.c.orig    Wed Sep  9 22:01:38 1998
--- src/interfaces/libpq/fe-exec.c    Wed Sep  9 22:45:53 1998
***************
*** 541,546 ****
--- 541,556 ----
              PQclear(result);
              return EOF;
          }
+         /*
+          * Since pqGetInt treats 2-byte integers as unsigned, we need to
+          * coerce the special value "-1" to signed form.  (-1 is sent for
+          * variable-length fields.)  Formerly, libpq effectively did a
+          * sign-extension on the 2-byte value by storing it in a signed short.
+          * Now we only coerce the single value 65535 == -1; values
+          * 32768..65534 are taken as valid field lengths.
+          */
+         if (typlen == 0xFFFF)
+             typlen = -1;
          result->attDescs[i].name = strdup(typName);
          result->attDescs[i].typid = typid;
          result->attDescs[i].typlen = typlen;
***************
*** 1488,1494 ****
      if (strncmp(res->cmdStatus, "INSERT ", 7) != 0)
          return "";

!     /*
       * The cmdStatus string looks like
       *     INSERT oid count\0
       * In order to be able to return an ordinary C string without
--- 1498,1504 ----
      if (strncmp(res->cmdStatus, "INSERT ", 7) != 0)
          return "";

!     /*----------
       * The cmdStatus string looks like
       *     INSERT oid count\0
       * In order to be able to return an ordinary C string without
***************
*** 1498,1503 ****
--- 1508,1514 ----
       *     INSERT oid count\0oid\0
       *                       ^ our return value points here
       * Pretty klugy eh?  This routine should've just returned an Oid value.
+      *----------
       */

      slen = strlen(res->cmdStatus);