Thread: Regression tests

Regression tests

From
Magnus Hagander
Date:
Joachim Wieland attempted to post this patch, but it appears to be gone.
I tried a repost, and notivced it got rejected because it was >100kb.
Let me repeat previous objections that it really should be possible to
post a patch >100kb.
That said, here's a gzipped version.

Joachim, once it comes through, feel free to post whatever comments you
had in your original mail.

//Magnus

Attachment

Re: Regression tests

From
Tom Lane
Date:
Magnus Hagander <magnus@hagander.net> writes:
> Joachim Wieland attempted to post this patch, but it appears to be gone.

I trust the applied version will contain neither Windows newlines nor
non-English comments.

            regards, tom lane

Re: Regression tests

From
Magnus Hagander
Date:
Tom Lane wrote:
> Magnus Hagander <magnus@hagander.net> writes:
>> Joachim Wieland attempted to post this patch, but it appears to be gone.
>
> I trust the applied version will contain neither Windows newlines nor
> non-English comments.

Certainly would assume so. I haven't actually looked at the patch yet, I
just forwarded it.
(And I may well be responsible for the windows newlines - it may have
happened when I gzipped it on a windows box)

//Magnus

Re: Regression tests

From
Magnus Hagander
Date:
Magnus Hagander wrote:
> Joachim Wieland attempted to post this patch, but it appears to be gone.
> I tried a repost, and notivced it got rejected because it was >100kb.
> Let me repeat previous objections that it really should be possible to
> post a patch >100kb.
> That said, here's a gzipped version.
>
> Joachim, once it comes through, feel free to post whatever comments you
> had in your original mail.

Here's an updated version of this patch. I reversed some parts of where
files went, which made the patch a whole lot smaller and easier to read,
and changed a couple of smaller things.

I have not yet looked at the actual ecpg parts (except noticed that they
are working on msvc at least), just the changes to main regression
tests. But I wanted to send off my new version for others to look at as
well before I do that.

This is still WIP, but comments appreciated.

//Magnus

Index: src/interfaces/ecpg/test/Makefile
===================================================================
RCS file: /projects/cvsroot/pgsql/src/interfaces/ecpg/test/Makefile,v
retrieving revision 1.67
diff -c -r1.67 Makefile
*** src/interfaces/ecpg/test/Makefile    29 Mar 2007 12:02:24 -0000    1.67
--- src/interfaces/ecpg/test/Makefile    9 Jun 2007 13:12:35 -0000
***************
*** 8,13 ****
--- 8,16 ----
  # this is also defined in test/connect/Makefile
  TEMP_PORT = 5$(DEF_PGPORT)

+ # where to find psql for testing an existing installation
+ PSQLDIR = $(bindir)
+
  # default encoding
  MULTIBYTE = SQL_ASCII

***************
*** 26,31 ****
--- 29,43 ----
  abs_builddir := $(shell pwd -W)
  endif

+ # stuff to pass into build of pg_regress
+ EXTRADEFS = '-DHOST_TUPLE="$(host_tuple)"' \
+     '-DMAKEPROG="$(MAKE)"' \
+     '-DSHELLPROG="$(SHELL)"' \
+     '-DDLSUFFIX="$(DLSUFFIX)"'
+
+ REGRESSINCLUDES = "-I$(top_builddir)/src/test/regress"
+ REGRESSDRIVER = "$(top_builddir)/src/test/regress/pg_regress.o"
+
  all install installdirs uninstall distprep:
      $(MAKE) -C connect $@
      $(MAKE) -C expected $@
***************
*** 45,64 ****
      $(MAKE) -C compat_informix $@
      $(MAKE) -C thread $@
      rm -rf tmp_check results log
!     rm -f pg_regress regression.diffs

! all: pg_regress

! pg_regress: pg_regress.sh $(top_builddir)/src/Makefile.global
!     sed -e 's,@bindir@,$(bindir),g' \
!         -e 's,@libdir@,$(libdir),g' \
!         -e 's,@pkglibdir@,$(pkglibdir),g' \
!         -e 's,@datadir@,$(datadir),g' \
!         -e 's/@VERSION@/$(VERSION)/g' \
!         -e 's/@host_tuple@/$(host_tuple)/g' \
!         -e 's,@GMAKE@,$(MAKE),g' \
!         -e 's/@enable_shared@/$(enable_shared)/g' \
!       $< >$@

  # When doing a VPATH build, copy over the .pgc, .stdout and .stderr
  # files so that the driver script can find them.  We have to use an
--- 57,77 ----
      $(MAKE) -C compat_informix $@
      $(MAKE) -C thread $@
      rm -rf tmp_check results log
!     rm -f pg_regress regression.diffs pg_regress_ecpg.o
!
! # Build regression test driver
!
! all: pg_regress$(X)
!
! pg_regress$(X): pg_regress_ecpg.o
!     $(CC) $(CFLAGS) $^ $(REGRESSDRIVER) $(LDFLAGS) $(LIBS) -o $@

! # dependencies ensure that path changes propagate
! pg_regress_ecpg.o: pg_regress_ecpg.c $(top_builddir)/src/port/pg_config_paths.h
!     $(CC) $(CFLAGS) $(CPPFLAGS) -I$(top_builddir)/src/port $(REGRESSINCLUDES) $(EXTRADEFS) -c -o $@ $<

! $(top_builddir)/src/port/pg_config_paths.h: $(top_builddir)/src/Makefile.global
!     $(MAKE) -C $(top_builddir)/src/port pg_config_paths.h

  # When doing a VPATH build, copy over the .pgc, .stdout and .stderr
  # files so that the driver script can find them.  We have to use an
***************
*** 78,88 ****


  check: all
!     sh ./pg_regress  --dbname=regress1 --temp-install --top-builddir=$(top_builddir) --temp-port=$(TEMP_PORT)
--multibyte=$(MULTIBYTE)--load-language=plpgsql $(NOLOCALE) $(THREAD) 

  # the same options, but with --listen-on-tcp
  checktcp: all
!     sh ./pg_regress  --dbname=regress1 --temp-install --top-builddir=$(top_builddir) --temp-port=$(TEMP_PORT)
--multibyte=$(MULTIBYTE)--load-language=plpgsql $(NOLOCALE) --listen-on-tcp $(THREAD) 

  installcheck: all
!     sh ./pg_regress  --dbname=regress1 --top-builddir=$(top_builddir) --load-language=plpgsql $(NOLOCALE)
--- 91,101 ----


  check: all
!     ./pg_regress  --dbname=regress1,connectdb --top-builddir=$(top_builddir) --temp-install=./tmp_check
--temp-port=$(TEMP_PORT)--multibyte=$(MULTIBYTE) --load-language=plpgsql $(NOLOCALE) $(THREAD)
--schedule=$(srcdir)/ecpg_schedule--create-role=connectuser,connectdb 

  # the same options, but with --listen-on-tcp
  checktcp: all
!     ./pg_regress  --dbname=regress1,connectdb --top-builddir=$(top_builddir) --temp-install=./tmp_check
--temp-port=$(TEMP_PORT)--multibyte=$(MULTIBYTE) --load-language=plpgsql $(NOLOCALE) $(THREAD)
--schedule=$(srcdir)/ecpg_schedule_tcp--create-role=connectuser,connectdb --host=localhost 

  installcheck: all
!     ./pg_regress  --psqldir=$(PSQLDIR) --dbname=regress1,connectdb --top-builddir=$(top_builddir)
--multibyte=$(MULTIBYTE)--load-language=plpgsql $(NOLOCALE) $(THREAD) --schedule=$(srcdir)/ecpg_schedule
--create-role=connectuser,connectdb
Index: src/test/regress/GNUmakefile
===================================================================
RCS file: /projects/cvsroot/pgsql/src/test/regress/GNUmakefile,v
retrieving revision 1.67
diff -c -r1.67 GNUmakefile
*** src/test/regress/GNUmakefile    13 Mar 2007 22:56:48 -0000    1.67
--- src/test/regress/GNUmakefile    9 Jun 2007 11:36:13 -0000
***************
*** 51,57 ****

  all: submake-libpgport pg_regress$(X)

! pg_regress$(X): pg_regress.o
      $(CC) $(CFLAGS) $^ $(LDFLAGS) $(LIBS) -o $@

  # dependencies ensure that path changes propagate
--- 51,57 ----

  all: submake-libpgport pg_regress$(X)

! pg_regress$(X): pg_regress.o pg_regress_main.o
      $(CC) $(CFLAGS) $^ $(LDFLAGS) $(LIBS) -o $@

  # dependencies ensure that path changes propagate
***************
*** 165,171 ****
  # things built by `all' target
      rm -f $(NAME)$(DLSUFFIX) $(OBJS)
      $(MAKE) -C $(contribdir)/spi clean
!     rm -f $(output_files) $(input_files) pg_regress.o pg_regress$(X)
  # things created by various check targets
      rm -rf testtablespace
      rm -rf results tmp_check log
--- 165,171 ----
  # things built by `all' target
      rm -f $(NAME)$(DLSUFFIX) $(OBJS)
      $(MAKE) -C $(contribdir)/spi clean
!     rm -f $(output_files) $(input_files) pg_regress_main.o pg_regress.o pg_regress$(X)
  # things created by various check targets
      rm -rf testtablespace
      rm -rf results tmp_check log
Index: src/test/regress/pg_regress.c
===================================================================
RCS file: /projects/cvsroot/pgsql/src/test/regress/pg_regress.c,v
retrieving revision 1.32
diff -c -r1.32 pg_regress.c
*** src/test/regress/pg_regress.c    31 May 2007 15:13:06 -0000    1.32
--- src/test/regress/pg_regress.c    9 Jun 2007 17:58:10 -0000
***************
*** 16,22 ****
   *-------------------------------------------------------------------------
   */

! #include "postgres_fe.h"

  #include <ctype.h>
  #include <sys/stat.h>
--- 16,22 ----
   *-------------------------------------------------------------------------
   */

! #include "pg_regress.h"

  #include <ctype.h>
  #include <sys/stat.h>
***************
*** 32,57 ****
  #include "getopt_long.h"
  #include "pg_config_paths.h"

- #ifndef WIN32
- #define PID_TYPE pid_t
- #define INVALID_PID (-1)
- #else
- #define PID_TYPE HANDLE
- #define INVALID_PID INVALID_HANDLE_VALUE
- #endif
-
-
- /* simple list of strings */
- typedef struct _stringlist
- {
-     char       *str;
-     struct _stringlist *next;
- }    _stringlist;
-
  /* for resultmap we need a list of pairs of strings */
  typedef struct _resultmap
  {
      char       *test;
      char       *resultfile;
      struct _resultmap *next;
  }    _resultmap;
--- 32,42 ----
  #include "getopt_long.h"
  #include "pg_config_paths.h"

  /* for resultmap we need a list of pairs of strings */
  typedef struct _resultmap
  {
      char       *test;
+     char       *type;
      char       *resultfile;
      struct _resultmap *next;
  }    _resultmap;
***************
*** 63,72 ****
   * In non-temp_install mode, the only thing we need is the location of psql,
   * which we expect to find in psqldir, or in the PATH if psqldir isn't given.
   */
! static char *bindir = PGBINDIR;
! static char *libdir = LIBDIR;
! static char *datadir = PGSHAREDIR;
! static char *host_platform = HOST_TUPLE;
  #ifndef WIN32_ONLY_COMPILER
  static char *makeprog = MAKEPROG;
  #endif
--- 48,57 ----
   * In non-temp_install mode, the only thing we need is the location of psql,
   * which we expect to find in psqldir, or in the PATH if psqldir isn't given.
   */
! char *bindir = PGBINDIR;
! char *libdir = LIBDIR;
! char *datadir = PGSHAREDIR;
! char *host_platform = HOST_TUPLE;
  #ifndef WIN32_ONLY_COMPILER
  static char *makeprog = MAKEPROG;
  #endif
***************
*** 76,89 ****
  #endif

  /* currently we can use the same diff switches on all platforms */
! static const char *basic_diff_opts = "-w";
! static const char *pretty_diff_opts = "-w -C3";

  /* options settable from command line */
! static char *dbname = "regression";
! static bool debug = false;
! static char *inputdir = ".";
! static char *outputdir = ".";
  static _stringlist *loadlanguage = NULL;
  static int    max_connections = 0;
  static char *encoding = NULL;
--- 61,76 ----
  #endif

  /* currently we can use the same diff switches on all platforms */
! const char *basic_diff_opts = "-w";
! const char *pretty_diff_opts = "-w -C3";

  /* options settable from command line */
! _stringlist *dblist = NULL;
! bool debug = false;
! char *inputdir = ".";
! char *outputdir = ".";
! char *psqldir = NULL;
! char *ecpgdir = NULL;
  static _stringlist *loadlanguage = NULL;
  static int    max_connections = 0;
  static char *encoding = NULL;
***************
*** 93,103 ****
  static char *top_builddir = NULL;
  static int    temp_port = 65432;
  static bool nolocale = false;
- static char *psqldir = NULL;
  static char *hostname = NULL;
  static int    port = -1;
  static char *user = NULL;
  static char *srcdir = NULL;

  /* internal variables */
  static const char *progname;
--- 80,90 ----
  static char *top_builddir = NULL;
  static int    temp_port = 65432;
  static bool nolocale = false;
  static char *hostname = NULL;
  static int    port = -1;
  static char *user = NULL;
  static char *srcdir = NULL;
+ static _stringlist *extraroles = NULL;

  /* internal variables */
  static const char *progname;
***************
*** 170,176 ****
  /*
   * Add an item at the end of a stringlist.
   */
! static void
  add_stringlist_item(_stringlist ** listhead, const char *str)
  {
      _stringlist *newentry = malloc(sizeof(_stringlist));
--- 157,163 ----
  /*
   * Add an item at the end of a stringlist.
   */
! void
  add_stringlist_item(_stringlist ** listhead, const char *str)
  {
      _stringlist *newentry = malloc(sizeof(_stringlist));
***************
*** 189,194 ****
--- 176,212 ----
  }

  /*
+  * Free a stringlist.
+  */
+ static void
+ free_stringlist(_stringlist **listhead)
+ {
+     if (listhead == NULL || *listhead == NULL)
+         return;
+     if ((*listhead)->next != NULL)
+         free_stringlist(&((*listhead)->next));
+     free((*listhead)->str);
+     free(*listhead);
+     *listhead = NULL;
+ }
+
+ /*
+  * Split a delimited string into a stringlist
+  */
+ static void
+ split_to_stringlist(const char *s, const char *delim, _stringlist **listhead)
+ {
+     char *sc = strdup(s);
+     char *token = strtok(sc, delim);
+     while (token)
+     {
+         add_stringlist_item(listhead, token);
+         token = strtok(NULL, delim);
+     }
+     free(sc);
+ }
+
+ /*
   * Print a progress banner on stdout.
   */
  static void
***************
*** 265,271 ****
   * Always exit through here, not through plain exit(), to ensure we make
   * an effort to shut down a temp postmaster
   */
! static void
  exit_nicely(int code)
  {
      stop_postmaster();
--- 283,289 ----
   * Always exit through here, not through plain exit(), to ensure we make
   * an effort to shut down a temp postmaster
   */
! void
  exit_nicely(int code)
  {
      stop_postmaster();
***************
*** 349,355 ****
   * Replace all occurances of a string in a string with a different string.
   * NOTE: Assumes there is enough room in the target buffer!
   */
! static void
  replace_string(char *string, char *replace, char *replacement)
  {
      char *ptr;
--- 367,373 ----
   * Replace all occurances of a string in a string with a different string.
   * NOTE: Assumes there is enough room in the target buffer!
   */
! void
  replace_string(char *string, char *replace, char *replacement)
  {
      char *ptr;
***************
*** 537,542 ****
--- 555,561 ----
      while (fgets(buf, sizeof(buf), f))
      {
          char       *platform;
+         char       *file_type;
          char       *expected;
          int            i;

***************
*** 546,552 ****
              buf[--i] = '\0';

          /* parse out the line fields */
!         platform = strchr(buf, '/');
          if (!platform)
          {
              fprintf(stderr, _("incorrectly formatted resultmap entry: %s\n"),
--- 565,580 ----
              buf[--i] = '\0';

          /* parse out the line fields */
!         file_type = strchr(buf, ':');
!         if (!file_type)
!         {
!             fprintf(stderr, _("incorrectly formatted resultmap entry: %s\n"),
!                 buf);
!             exit_nicely(2);
!         }
!         *file_type++ = '\0';
!
!         platform = strchr(file_type, ':');
          if (!platform)
          {
              fprintf(stderr, _("incorrectly formatted resultmap entry: %s\n"),
***************
*** 574,579 ****
--- 602,608 ----
              _resultmap *entry = malloc(sizeof(_resultmap));

              entry->test = strdup(buf);
+             entry->type = strdup(file_type);
              entry->resultfile = strdup(expected);
              entry->next = resultmap;
              resultmap = entry;
***************
*** 583,588 ****
--- 612,646 ----
  }

  /*
+  * Check in resultmap if we should be looking at a different file
+  */
+ static
+ const char *get_expectfile(const char *testname, const char *file)
+ {
+     char *file_type;
+     _resultmap *rm;
+
+     /*
+      * Determine the file type from the file name. This is just what is
+      * following the last dot in the file name.
+      */
+     if (!file || !(file_type = strrchr(file, '.')))
+         return NULL;
+
+     file_type++;
+
+     for (rm = resultmap; rm != NULL; rm = rm->next)
+     {
+         if (strcmp(testname, rm->test) == 0 && strcmp(file_type, rm->type) == 0)
+         {
+             return rm->resultfile;
+         }
+     }
+
+     return NULL;
+ }
+
+ /*
   * Handy subroutine for setting an environment variable "var" to "val"
   */
  static void
***************
*** 704,709 ****
--- 762,770 ----

          /* psql will be installed into temp-install bindir */
          psqldir = bindir;
+
+         /* ecpg will be installed into temp-install bindir */
+         ecpgdir = bindir;

          /*
           * Set up shared library paths to include the temp install.
***************
*** 820,826 ****
   *
   * Returns the process ID (or HANDLE) so we can wait for it later
   */
! static PID_TYPE
  spawn_process(const char *cmdline)
  {
  #ifndef WIN32
--- 881,887 ----
   *
   * Returns the process ID (or HANDLE) so we can wait for it later
   */
! PID_TYPE
  spawn_process(const char *cmdline)
  {
  #ifndef WIN32
***************
*** 944,986 ****
  }

  /*
-  * start a psql test process for specified file (including redirection),
-  * and return process ID
-  */
- static PID_TYPE
- psql_start_test(const char *testname)
- {
-     PID_TYPE    pid;
-     char        infile[MAXPGPATH];
-     char        outfile[MAXPGPATH];
-     char        psql_cmd[MAXPGPATH * 3];
-
-     snprintf(infile, sizeof(infile), "%s/sql/%s.sql",
-              inputdir, testname);
-     snprintf(outfile, sizeof(outfile), "%s/results/%s.out",
-              outputdir, testname);
-
-     snprintf(psql_cmd, sizeof(psql_cmd),
-              SYSTEMQUOTE "\"%s%spsql\" -X -a -q -d \"%s\" < \"%s\" > \"%s\" 2>&1" SYSTEMQUOTE,
-              psqldir ? psqldir : "",
-              psqldir ? "/" : "",
-              dbname,
-              infile,
-              outfile);
-
-     pid = spawn_process(psql_cmd);
-
-     if (pid == INVALID_PID)
-     {
-         fprintf(stderr, _("could not start process for test %s\n"),
-                 testname);
-         exit_nicely(2);
-     }
-
-     return pid;
- }
-
- /*
   * Count bytes in file
   */
  static long
--- 1005,1010 ----
***************
*** 1062,1067 ****
--- 1086,1111 ----
  }

  /*
+  * In: filename.ext, Return: filename_i.ext, where 0 < i <= 9
+  */
+ static char *
+ get_alternative_expectfile(const char *expectfile, int i)
+ {
+     char *last_dot;
+     int ssize = strlen(expectfile) + 2 + 1;
+     char *tmp = (char *)malloc(ssize);
+     char *s = (char *)malloc(ssize);
+     strcpy(tmp, expectfile);
+     last_dot = strrchr(tmp,'.');
+     if (!last_dot)
+         return NULL;
+     *last_dot = '\0';
+     snprintf(s, ssize, "%s_%d.%s", tmp, i, last_dot+1);
+     free(tmp);
+     return s;
+ }
+
+ /*
   * Run a "diff" command and also check that it didn't crash
   */
  static int
***************
*** 1098,1139 ****
   * In the true case, the diff is appended to the diffs file.
   */
  static bool
! results_differ(const char *testname)
  {
-     const char *expectname;
-     char        resultsfile[MAXPGPATH];
      char        expectfile[MAXPGPATH];
      char        diff[MAXPGPATH];
      char        cmd[MAXPGPATH * 3];
      char        best_expect_file[MAXPGPATH];
-     _resultmap *rm;
      FILE       *difffile;
      int            best_line_count;
      int            i;
      int            l;

!     /* Check in resultmap if we should be looking at a different file */
!     expectname = testname;
!     for (rm = resultmap; rm != NULL; rm = rm->next)
      {
!         if (strcmp(testname, rm->test) == 0)
!         {
!             expectname = rm->resultfile;
!             break;
!         }
      }

-     /* Name of test results file */
-     snprintf(resultsfile, sizeof(resultsfile), "%s/results/%s.out",
-              outputdir, testname);
-
-     /* Name of expected-results file */
-     snprintf(expectfile, sizeof(expectfile), "%s/expected/%s.out",
-              inputdir, expectname);
-
      /* Name to use for temporary diff file */
!     snprintf(diff, sizeof(diff), "%s/results/%s.diff",
!              outputdir, testname);

      /* OK, run the diff */
      snprintf(cmd, sizeof(cmd),
--- 1142,1179 ----
   * In the true case, the diff is appended to the diffs file.
   */
  static bool
! results_differ(const char *testname, const char *resultsfile, const char *default_expectfile)
  {
      char        expectfile[MAXPGPATH];
      char        diff[MAXPGPATH];
      char        cmd[MAXPGPATH * 3];
      char        best_expect_file[MAXPGPATH];
      FILE       *difffile;
      int            best_line_count;
      int            i;
      int            l;
+     const char *platform_expectfile;

!     /*
!      * We can pass either the resultsfile or the expectfile, they should
!      * have the same type (filename.type) anyway.
!      */
!     platform_expectfile = get_expectfile(testname, resultsfile);
!
!     strcpy(expectfile, default_expectfile);
!     if (platform_expectfile)
      {
!         /*
!          * Replace everything afer the last slash in expectfile with what the
!          * platform_expectfile contains. XXX SEPARATOR
!          */
!         char *p = strrchr(expectfile, '/');
!         if (p)
!             strcpy(++p, platform_expectfile);
      }

      /* Name to use for temporary diff file */
!     snprintf(diff, sizeof(diff), "%s.diff", resultsfile);

      /* OK, run the diff */
      snprintf(cmd, sizeof(cmd),
***************
*** 1153,1166 ****

      for (i = 0; i <= 9; i++)
      {
!         snprintf(expectfile, sizeof(expectfile), "%s/expected/%s_%d.out",
!                  inputdir, expectname, i);
!         if (!file_exists(expectfile))
              continue;

          snprintf(cmd, sizeof(cmd),
                   SYSTEMQUOTE "diff %s \"%s\" \"%s\" > \"%s\"" SYSTEMQUOTE,
!                  basic_diff_opts, expectfile, resultsfile, diff);

          if (run_diff(cmd, diff) == 0)
          {
--- 1193,1207 ----

      for (i = 0; i <= 9; i++)
      {
!         char *alt_expectfile;
!
!         alt_expectfile = get_alternative_expectfile(expectfile, i);
!         if (!file_exists(alt_expectfile))
              continue;

          snprintf(cmd, sizeof(cmd),
                   SYSTEMQUOTE "diff %s \"%s\" \"%s\" > \"%s\"" SYSTEMQUOTE,
!                  basic_diff_opts, alt_expectfile, resultsfile, diff);

          if (run_diff(cmd, diff) == 0)
          {
***************
*** 1173,1180 ****
          {
              /* This diff was a better match than the last one */
              best_line_count = l;
!             strcpy(best_expect_file, expectfile);
          }
      }

      /*
--- 1214,1222 ----
          {
              /* This diff was a better match than the last one */
              best_line_count = l;
!             strcpy(best_expect_file, alt_expectfile);
          }
+         free(alt_expectfile);
      }

      /*
***************
*** 1182,1195 ****
       * haven't found a complete match yet.
       */

!     if (strcmp(expectname, testname) != 0)
      {
-         snprintf(expectfile, sizeof(expectfile), "%s/expected/%s.out",
-                  inputdir, testname);
-
          snprintf(cmd, sizeof(cmd),
                   SYSTEMQUOTE "diff %s \"%s\" \"%s\" > \"%s\"" SYSTEMQUOTE,
!                  basic_diff_opts, expectfile, resultsfile, diff);

          if (run_diff(cmd, diff) == 0)
          {
--- 1224,1234 ----
       * haven't found a complete match yet.
       */

!     if (platform_expectfile)
      {
          snprintf(cmd, sizeof(cmd),
                   SYSTEMQUOTE "diff %s \"%s\" \"%s\" > \"%s\"" SYSTEMQUOTE,
!                  basic_diff_opts, default_expectfile, resultsfile, diff);

          if (run_diff(cmd, diff) == 0)
          {
***************
*** 1203,1209 ****
          {
              /* This diff was a better match than the last one */
              best_line_count = l;
!             strcpy(best_expect_file, expectfile);
          }
      }

--- 1242,1248 ----
          {
              /* This diff was a better match than the last one */
              best_line_count = l;
!             strcpy(best_expect_file, default_expectfile);
          }
      }

***************
*** 1302,1317 ****
   * Run all the tests specified in one schedule file
   */
  static void
! run_schedule(const char *schedule)
  {
  #define MAX_PARALLEL_TESTS 100
      char       *tests[MAX_PARALLEL_TESTS];
      PID_TYPE    pids[MAX_PARALLEL_TESTS];
      _stringlist *ignorelist = NULL;
      char        scbuf[1024];
      FILE       *scf;
      int            line_num = 0;

      scf = fopen(schedule, "r");
      if (!scf)
      {
--- 1341,1363 ----
   * Run all the tests specified in one schedule file
   */
  static void
! run_schedule(const char *schedule, test_function tfunc)
  {
  #define MAX_PARALLEL_TESTS 100
      char       *tests[MAX_PARALLEL_TESTS];
+     _stringlist *resultfiles[MAX_PARALLEL_TESTS];
+     _stringlist *expectfiles[MAX_PARALLEL_TESTS];
+     _stringlist *tags[MAX_PARALLEL_TESTS];
      PID_TYPE    pids[MAX_PARALLEL_TESTS];
      _stringlist *ignorelist = NULL;
      char        scbuf[1024];
      FILE       *scf;
      int            line_num = 0;

+     memset(resultfiles,0,sizeof(_stringlist *) * MAX_PARALLEL_TESTS);
+     memset(expectfiles,0,sizeof(_stringlist *) * MAX_PARALLEL_TESTS);
+     memset(tags,0,sizeof(_stringlist *) * MAX_PARALLEL_TESTS);
+
      scf = fopen(schedule, "r");
      if (!scf)
      {
***************
*** 1330,1335 ****
--- 1376,1390 ----

          line_num++;

+         for (i = 0; i < MAX_PARALLEL_TESTS; i++)
+         {
+             if (resultfiles[i] == NULL)
+                 break;
+             free_stringlist(&resultfiles[i]);
+             free_stringlist(&expectfiles[i]);
+             free_stringlist(&tags[i]);
+         }
+
          /* strip trailing whitespace, especially the newline */
          i = strlen(scbuf);
          while (i > 0 && isspace((unsigned char) scbuf[i - 1]))
***************
*** 1394,1400 ****
          if (num_tests == 1)
          {
              status(_("test %-20s ... "), tests[0]);
!             pids[0] = psql_start_test(tests[0]);
              wait_for_tests(pids, NULL, 1);
              /* status line is finished below */
          }
--- 1449,1455 ----
          if (num_tests == 1)
          {
              status(_("test %-20s ... "), tests[0]);
!             pids[0] = (*tfunc)(tests[0], &resultfiles[0], &expectfiles[0], &tags[0]);
              wait_for_tests(pids, NULL, 1);
              /* status line is finished below */
          }
***************
*** 1411,1417 ****
                      wait_for_tests(pids + oldest, tests + oldest, i - oldest);
                      oldest = i;
                  }
!                 pids[i] = psql_start_test(tests[i]);
              }
              wait_for_tests(pids + oldest, tests + oldest, i - oldest);
              status_end();
--- 1466,1472 ----
                      wait_for_tests(pids + oldest, tests + oldest, i - oldest);
                      oldest = i;
                  }
!                 pids[i] = (tfunc)(tests[i], &resultfiles[i], &expectfiles[i], &tags[i]);
              }
              wait_for_tests(pids + oldest, tests + oldest, i - oldest);
              status_end();
***************
*** 1421,1427 ****
              status(_("parallel group (%d tests): "), num_tests);
              for (i = 0; i < num_tests; i++)
              {
!                 pids[i] = psql_start_test(tests[i]);
              }
              wait_for_tests(pids, tests, num_tests);
              status_end();
--- 1476,1482 ----
              status(_("parallel group (%d tests): "), num_tests);
              for (i = 0; i < num_tests; i++)
              {
!                 pids[i] = (tfunc)(tests[i], &resultfiles[i], &expectfiles[i], &tags[i]);
              }
              wait_for_tests(pids, tests, num_tests);
              status_end();
***************
*** 1430,1439 ****
          /* Check results for all tests */
          for (i = 0; i < num_tests; i++)
          {
              if (num_tests > 1)
                  status(_("     %-20s ... "), tests[i]);

!             if (results_differ(tests[i]))
              {
                  bool        ignore = false;
                  _stringlist *sl;
--- 1485,1520 ----
          /* Check results for all tests */
          for (i = 0; i < num_tests; i++)
          {
+             _stringlist *rl, *el, *tl;
+             bool differ = false;
+
              if (num_tests > 1)
                  status(_("     %-20s ... "), tests[i]);

!             /*
!              * Advance over all three lists simultaneously.
!              *
!              * Compare resultfiles[j] with expectfiles[j] always.
!              * Tags are optional but if there are tags, the tag list has the
!              * same length as the other two lists.
!              */
!             for (rl = resultfiles[i], el = expectfiles[i], tl = tags[i];
!                 rl != NULL; /* rl and el have the same length */
!                 rl = rl->next, el = el->next)
!             {
!                 bool newdiff;
!                 if (tl)
!                     tl = tl->next; /* tl has the same lengt has rl and el if it exists */
!
!                 newdiff = results_differ(tests[i], rl->str, el->str);
!                 if (newdiff && tl)
!                 {
!                     printf("%s ", tl->str);
!                 }
!                 differ |= newdiff;
!             }
!
!             if (differ)
              {
                  bool        ignore = false;
                  _stringlist *sl;
***************
*** 1474,1488 ****
   * Run a single test
   */
  static void
! run_single_test(const char *test)
  {
      PID_TYPE    pid;

      status(_("test %-20s ... "), test);
!     pid = psql_start_test(test);
      wait_for_tests(&pid, NULL, 1);

!     if (results_differ(test))
      {
          status(_("FAILED"));
          fail_count++;
--- 1555,1598 ----
   * Run a single test
   */
  static void
! run_single_test(const char *test, test_function tfunc)
  {
      PID_TYPE    pid;
+     _stringlist *resultfiles;
+     _stringlist *expectfiles;
+     _stringlist *tags;
+     _stringlist *rl, *el, *tl;
+     bool        differ;

      status(_("test %-20s ... "), test);
!     /* XXX: Initialize varialbes? */
!     pid = (tfunc)(test, &resultfiles, &expectfiles, &tags);
      wait_for_tests(&pid, NULL, 1);

!     /*
!      * Advance over all three lists simultaneously.
!      *
!      * Compare resultfiles[j] with expectfiles[j] always.
!      * Tags are optional but if there are tags, the tag list has the
!      * same length as the other two lists.
!      */
!     for (rl = resultfiles, el = expectfiles, tl = tags;
!         rl != NULL; /* rl and el have the same length */
!         rl = rl->next, el = el->next)
!     {
!         bool newdiff;
!         if (tl)
!             tl = tl->next; /* tl has the same lengt has rl and el if it exists */
!
!         newdiff = results_differ(test, rl->str, el->str);
!         if (newdiff && tl)
!         {
!             printf("%s ", tl->str);
!         }
!         differ |= newdiff;
!     }
!
!     if (differ)
      {
          status(_("FAILED"));
          fail_count++;
***************
*** 1535,1540 ****
--- 1645,1707 ----
  }

  static void
+ drop_database_if_exists(const char *dbname)
+ {
+     header(_("dropping database \"%s\""), dbname);
+     psql_command("postgres", "DROP DATABASE IF EXISTS \"%s\"", dbname);
+ }
+
+ static void
+ create_database(const char *dbname)
+ {
+     _stringlist *sl;
+     /*
+      * We use template0 so that any installation-local cruft in template1 will
+      * not mess up the tests.
+      */
+     header(_("creating database \"%s\""), dbname);
+     if (encoding)
+         psql_command("postgres", "CREATE DATABASE \"%s\" TEMPLATE=template0 ENCODING='%s'", dbname, encoding);
+     else
+         psql_command("postgres", "CREATE DATABASE \"%s\" TEMPLATE=template0", dbname);
+     psql_command(dbname,
+         "ALTER DATABASE \"%s\" SET lc_messages TO 'C';"
+         "ALTER DATABASE \"%s\" SET lc_monetary TO 'C';"
+         "ALTER DATABASE \"%s\" SET lc_numeric TO 'C';"
+         "ALTER DATABASE \"%s\" SET lc_time TO 'C';"
+         "ALTER DATABASE \"%s\" SET timezone_abbreviations TO 'Default';",
+         dbname, dbname, dbname, dbname, dbname);
+
+     /*
+      * Install any requested procedural languages
+      */
+     for (sl = loadlanguage; sl != NULL; sl = sl->next)
+     {
+         header(_("installing %s"), sl->str);
+         psql_command(dbname, "CREATE LANGUAGE \"%s\"", sl->str);
+     }
+ }
+
+ static void
+ drop_role_if_exists(const char *rolename)
+ {
+     header(_("dropping role \"%s\""), rolename);
+     psql_command("postgres", "DROP ROLE IF EXISTS \"%s\"", rolename);
+ }
+
+ static void
+ create_role(const char *rolename, const _stringlist *granted_dbs)
+ {
+     header(_("creating role \"%s\""), rolename);
+     psql_command("postgres", "CREATE ROLE \"%s\" WITH LOGIN", rolename);
+     for (; granted_dbs != NULL; granted_dbs = granted_dbs->next)
+     {
+         psql_command("postgres", "GRANT ALL ON DATABASE \"%s\" TO \"%s\"",
+             granted_dbs->str, rolename);
+     }
+ }
+
+ static void
  help(void)
  {
      printf(_("PostgreSQL regression test driver\n"));
***************
*** 1547,1552 ****
--- 1714,1720 ----
      printf(_("  --inputdir=DIR            take input files from DIR (default \".\")\n"));
      printf(_("  --load-language=lang      load the named language before running the\n"));
      printf(_("                            tests; can appear multiple times\n"));
+     printf(_("  --create-role=ROLE        create the specified role before testing\n"));
      printf(_("  --max-connections=N       maximum number of concurrent connections\n"));
      printf(_("                            (default is 0 meaning unlimited)\n"));
      printf(_("  --multibyte=ENCODING      use ENCODING as the multibyte encoding\n"));
***************
*** 1566,1571 ****
--- 1734,1740 ----
      printf(_("  --port=PORT               use postmaster running at PORT\n"));
      printf(_("  --user=USER               connect as USER\n"));
      printf(_("  --psqldir=DIR             use psql in DIR (default: find in PATH)\n"));
+     printf(_("  --ecpgdir=DIR             use ecpg in DIR (default: find in PATH)\n"));
      printf(_("\n"));
      printf(_("The exit status is 0 if all tests passed, 1 if some tests failed, and 2\n"));
      printf(_("if the tests could not be run for some reason.\n"));
***************
*** 1574,1580 ****
  }

  int
! main(int argc, char *argv[])
  {
      _stringlist *sl;
      int            c;
--- 1743,1749 ----
  }

  int
! regression_main(int argc, char *argv[], init_function ifunc, test_function tfunc)
  {
      _stringlist *sl;
      int            c;
***************
*** 1602,1607 ****
--- 1771,1778 ----
          {"user", required_argument, NULL, 15},
          {"psqldir", required_argument, NULL, 16},
          {"srcdir", required_argument, NULL, 17},
+         {"create-role", required_argument, NULL, 18},
+         {"ecpgdir", required_argument, NULL, 19},
          {NULL, 0, NULL, 0}
      };

***************
*** 1613,1618 ****
--- 1784,1795 ----
      hostname = "localhost";
  #endif

+     /*
+      * We call the initialization function here because that way we can set
+      * default parameters and let them be overwritten by the commandline.
+      */
+     ifunc();
+
      while ((c = getopt_long(argc, argv, "hV", long_options, &option_index)) != -1)
      {
          switch (c)
***************
*** 1624,1630 ****
                  printf("pg_regress (PostgreSQL %s)\n", PG_VERSION);
                  exit_nicely(0);
              case 1:
!                 dbname = strdup(optarg);
                  break;
              case 2:
                  debug = true;
--- 1801,1807 ----
                  printf("pg_regress (PostgreSQL %s)\n", PG_VERSION);
                  exit_nicely(0);
              case 1:
!                 split_to_stringlist(strdup(optarg), ", ", &dblist);
                  break;
              case 2:
                  debug = true;
***************
*** 1697,1702 ****
--- 1874,1887 ----
              case 17:
                  srcdir = strdup(optarg);
                  break;
+             case 18:
+                 split_to_stringlist(strdup(optarg), ", ", &extraroles);
+                 break;
+             case 19:
+                 /* "--ecpgdir=" should mean to use PATH */
+                 if (strlen(optarg))
+                     ecpgdir = strdup(optarg);
+                 break;
              default:
                  /* getopt_long already emitted a complaint */
                  fprintf(stderr, _("\nTry \"%s -h\" for more information.\n"),
***************
*** 1865,1909 ****
      {
          /*
           * Using an existing installation, so may need to get rid of
!          * pre-existing database.
           */
!         header(_("dropping database \"%s\""), dbname);
!         psql_command("postgres", "DROP DATABASE IF EXISTS \"%s\"", dbname);
      }

      /*
!      * Create the test database
!      *
!      * We use template0 so that any installation-local cruft in template1 will
!      * not mess up the tests.
       */
!     header(_("creating database \"%s\""), dbname);
!     if (encoding)
!         psql_command("postgres",
!                    "CREATE DATABASE \"%s\" TEMPLATE=template0 ENCODING='%s'",
!                      dbname, encoding);
!     else
!         /* use installation default */
!         psql_command("postgres",
!                      "CREATE DATABASE \"%s\" TEMPLATE=template0",
!                      dbname);
!
!     psql_command(dbname,
!                  "ALTER DATABASE \"%s\" SET lc_messages TO 'C';"
!                  "ALTER DATABASE \"%s\" SET lc_monetary TO 'C';"
!                  "ALTER DATABASE \"%s\" SET lc_numeric TO 'C';"
!                  "ALTER DATABASE \"%s\" SET lc_time TO 'C';"
!             "ALTER DATABASE \"%s\" SET timezone_abbreviations TO 'Default';",
!                  dbname, dbname, dbname, dbname, dbname);
!
!     /*
!      * Install any requested PL languages
!      */
!     for (sl = loadlanguage; sl != NULL; sl = sl->next)
!     {
!         header(_("installing %s"), sl->str);
!         psql_command(dbname, "CREATE LANGUAGE \"%s\"", sl->str);
!     }

      /*
       * Ready to run the tests
--- 2050,2070 ----
      {
          /*
           * Using an existing installation, so may need to get rid of
!          * pre-existing database(s) and role(s)
           */
!         for (sl = dblist; sl; sl = sl->next)
!             drop_database_if_exists(sl->str);
!         for (sl = extraroles; sl; sl = sl->next)
!             drop_role_if_exists(sl->str);
      }

      /*
!      * Create the test database(s) and role(s)
       */
!     for (sl = dblist; sl; sl = sl->next)
!         create_database(sl->str);
!     for (sl = extraroles; sl; sl = sl->next)
!         create_role(sl->str, dblist);

      /*
       * Ready to run the tests
***************
*** 1912,1923 ****

      for (sl = schedulelist; sl != NULL; sl = sl->next)
      {
!         run_schedule(sl->str);
      }

      for (sl = extra_tests; sl != NULL; sl = sl->next)
      {
!         run_single_test(sl->str);
      }

      /*
--- 2073,2084 ----

      for (sl = schedulelist; sl != NULL; sl = sl->next)
      {
!         run_schedule(sl->str, tfunc);
      }

      for (sl = extra_tests; sl != NULL; sl = sl->next)
      {
!         run_single_test(sl->str, tfunc);
      }

      /*
Index: src/test/regress/resultmap
===================================================================
RCS file: /projects/cvsroot/pgsql/src/test/regress/resultmap,v
retrieving revision 1.84
diff -c -r1.84 resultmap
*** src/test/regress/resultmap    8 Feb 2007 15:28:58 -0000    1.84
--- src/test/regress/resultmap    9 Jun 2007 11:14:35 -0000
***************
*** 1,11 ****
! float4/i.86-pc-mingw32=float4-exp-three-digits
! float4/i.86-pc-win32vc=float4-exp-three-digits
! float8/i.86-.*-freebsd=float8-small-is-zero
! float8/i.86-.*-openbsd=float8-small-is-zero
! float8/i.86-.*-netbsd=float8-small-is-zero
! float8/m68k-.*-netbsd=float8-small-is-zero
! float8/i.86-pc-mingw32=float8-exp-three-digits-win32
! float8/i.86-pc-win32vc=float8-exp-three-digits-win32
! float8/i.86-pc-cygwin=float8-small-is-zero
! int8/i.86-pc-mingw32=int8-exp-three-digits
! int8/i.86-pc-win32vc=int8-exp-three-digits
\ No newline at end of file
--- 1,11 ----
! float4:out:i.86-pc-mingw32=float4-exp-three-digits.out
! float4:out:i.86-pc-win32vc=float4-exp-three-digits.out
! float8:out:i.86-.*-freebsd=float8-small-is-zero.out
! float8:out:i.86-.*-openbsd=float8-small-is-zero.out
! float8:out:i.86-.*-netbsd=float8-small-is-zero.out
! float8:out:m68k-.*-netbsd=float8-small-is-zero.out
! float8:out:i.86-pc-mingw32=float8-exp-three-digits-win32.out
! float8:out:i.86-pc-win32vc=float8-exp-three-digits-win32.out
! float8:out:i.86-pc-cygwin=float8-small-is-zero.out
! int8:out:i.86-pc-mingw32=int8-exp-three-digits.out
! int8:out:i.86-pc-win32vc=int8-exp-three-digits.out
Index: src/tools/msvc/Install.pm
===================================================================
RCS file: /projects/cvsroot/pgsql/src/tools/msvc/Install.pm,v
retrieving revision 1.15
diff -c -r1.15 Install.pm
*** src/tools/msvc/Install.pm    13 May 2007 15:33:07 -0000    1.15
--- src/tools/msvc/Install.pm    9 Jun 2007 11:14:36 -0000
***************
*** 25,30 ****
--- 25,31 ----
      require 'config.pl';

      chdir("../../..") if (-f "../../../configure");
+     chdir("../../../..") if (-f "../../../../configure");
      my $conf = "";
      if (-d "debug")
      {
***************
*** 115,120 ****
--- 116,122 ----
      {
          chomp;
          next if /regress/; # Skip temporary install in regression subdir
+         next if /ecpg.test/; # Skip temporary install in regression subdir
          my $tgt = $target . basename($_);
          print ".";
          my $src = $norecurse?(dirname($spec) . '/' . $_):$_;
Index: src/tools/msvc/Mkvcbuild.pm
===================================================================
RCS file: /projects/cvsroot/pgsql/src/tools/msvc/Mkvcbuild.pm,v
retrieving revision 1.11
diff -c -r1.11 Mkvcbuild.pm
*** src/tools/msvc/Mkvcbuild.pm    27 Apr 2007 16:45:54 -0000    1.11
--- src/tools/msvc/Mkvcbuild.pm    9 Jun 2007 13:02:38 -0000
***************
*** 154,159 ****
--- 154,168 ----
      $ecpg->AddDefine('PATCHLEVEL=1');
      $ecpg->AddReference($libpgport);

+     my $pgregress_ecpg = $solution->AddProject('pg_regress_ecpg','exe','misc');
+     $pgregress_ecpg->AddFile('src\interfaces\ecpg\test\pg_regress_ecpg.c');
+     $pgregress_ecpg->AddFile('src\test\regress\pg_regress.c');
+     $pgregress_ecpg->AddIncludeDir('src\port');
+     $pgregress_ecpg->AddIncludeDir('src\test\regress');
+     $pgregress_ecpg->AddDefine('HOST_TUPLE="i686-pc-win32vc"');
+     $pgregress_ecpg->AddDefine('FRONTEND');
+     $pgregress_ecpg->AddReference($libpgport);
+
      # src/bin
      my $initdb = AddSimpleFrontend('initdb', 1);
      $initdb->AddLibrary('wsock32.lib ws2_32.lib');
***************
*** 315,320 ****
--- 324,330 ----

      my $pgregress = $solution->AddProject('pg_regress','exe','misc');
      $pgregress->AddFile('src\test\regress\pg_regress.c');
+     $pgregress->AddFile('src\test\regress\pg_regress_main.c');
      $pgregress->AddIncludeDir('src\port');
      $pgregress->AddDefine('HOST_TUPLE="i686-pc-win32vc"');
      $pgregress->AddDefine('FRONTEND');
Index: src/tools/msvc/vcregress.bat
===================================================================
RCS file: /projects/cvsroot/pgsql/src/tools/msvc/vcregress.bat,v
retrieving revision 1.11
diff -c -r1.11 vcregress.bat
*** src/tools/msvc/vcregress.bat    21 Apr 2007 20:58:05 -0000    1.11
--- src/tools/msvc/vcregress.bat    9 Jun 2007 13:01:10 -0000
***************
*** 11,16 ****
--- 11,17 ----
  if /I "%1"=="installcheck" SET what=INSTALLCHECK
  if /I "%1"=="plcheck" SET what=PLCHECK
  if /I "%1"=="contribcheck" SET what=CONTRIBCHECK
+ if /I "%1"=="ecpgcheck" SET what=ECPGCHECK
  if "%what%"=="" goto usage

  SET CONFIG=Debug
***************
*** 28,37 ****
  SET TEMPPORT=54321
  IF NOT "%2"=="" SET SCHEDULE=%2

! SET PERL5LIB=..\..\tools\msvc

  if "%what%"=="INSTALLCHECK" ..\..\..\%CONFIG%\pg_regress\pg_regress --psqldir="..\..\..\%CONFIG%\psql"
--schedule=%SCHEDULE%_schedule--multibyte=SQL_ASCII --load-language=plpgsql --no-locale 
  if "%what%"=="CHECK" ..\..\..\%CONFIG%\pg_regress\pg_regress --psqldir="..\..\..\%CONFIG%\psql"
--schedule=%SCHEDULE%_schedule--multibyte=SQL_ASCII --load-language=plpgsql --no-locale --temp-install=./tmp_check
--top-builddir="%TOPDIR%"--temp-port=%TEMPPORT% 
  if "%what%"=="PLCHECK" call :plcheck
  if "%what%"=="CONTRIBCHECK" call :contribcheck
  SET E=%ERRORLEVEL%
--- 29,45 ----
  SET TEMPPORT=54321
  IF NOT "%2"=="" SET SCHEDULE=%2

! IF "%what%"=="ECPGCHECK" (
!    cd "%TOPDIR%"
!    cd src\interfaces\ecpg\test
!    SET SCHEDULE=ecpg
! )
!
! SET PERL5LIB=%TOPDIR%\src\tools\msvc

  if "%what%"=="INSTALLCHECK" ..\..\..\%CONFIG%\pg_regress\pg_regress --psqldir="..\..\..\%CONFIG%\psql"
--schedule=%SCHEDULE%_schedule--multibyte=SQL_ASCII --load-language=plpgsql --no-locale 
  if "%what%"=="CHECK" ..\..\..\%CONFIG%\pg_regress\pg_regress --psqldir="..\..\..\%CONFIG%\psql"
--schedule=%SCHEDULE%_schedule--multibyte=SQL_ASCII --load-language=plpgsql --no-locale --temp-install=./tmp_check
--top-builddir="%TOPDIR%"--temp-port=%TEMPPORT% 
+ if "%what%"=="ECPGCHECK" ..\..\..\..\%CONFIG%\pg_regress_ecpg\pg_regress_ecpg --psqldir="..\..\..\%CONFIG%\psql"
--dbname=regress1,connectdb--create-role=connectuser,connectdb --schedule=%SCHEDULE%_schedule --multibyte=SQL_ASCII
--load-language=plpgsql--no-locale --temp-install=./tmp_check --top-builddir="%TOPDIR%" --temp-port=%TEMPPORT% 
  if "%what%"=="PLCHECK" call :plcheck
  if "%what%"=="CONTRIBCHECK" call :contribcheck
  SET E=%ERRORLEVEL%
***************
*** 40,46 ****
  exit /b %E%

  :usage
! echo "Usage: vcregress <check|installcheck> [schedule]"
  goto :eof


--- 48,54 ----
  exit /b %E%

  :usage
! echo "Usage: vcregress <check|installcheck|plcheck|contribcheck|ecpgcheck> [schedule]"
  goto :eof


test: compat_informix/dec_test
test: compat_informix/charfuncs
test: compat_informix/rfmtdate
test: compat_informix/rfmtlong
test: compat_informix/rnull
test: compat_informix/test_informix
test: compat_informix/test_informix2
test: connect/test2
test: connect/test3
test: connect/test4
test: connect/test5
test: pgtypeslib/dt_test
test: pgtypeslib/dt_test2
test: pgtypeslib/num_test
test: pgtypeslib/num_test2
test: preproc/comment
test: preproc/define
test: preproc/init
test: preproc/type
test: preproc/variable
test: preproc/whenever
test: sql/array
test: sql/binary
test: sql/code100
test: sql/copystdout
test: sql/define
test: sql/desc
test: sql/dynalloc
test: sql/dynalloc2
test: sql/dyntest
test: sql/execute
test: sql/fetch
test: sql/func
test: sql/indicators
test: sql/quote
test: sql/show
test: sql/insupd
test: sql/parser
# test: thread/thread
# test: thread/thread_implicit
/*-------------------------------------------------------------------------
 *
 * pg_regress_ecpg --- regression test driver for ecpg
 *
 * This is a C implementation of the previous shell script for running
 * the regression tests, and should be mostly compatible with it.
 * Initial author of C translation: Magnus Hagander
 *
 * This code is released under the terms of the PostgreSQL License.
 *
 * Portions Copyright (c) 1996-2007, PostgreSQL Global Development Group
 * Portions Copyright (c) 1994, Regents of the University of California
 *
 * $PostgreSQL$
 *
 *-------------------------------------------------------------------------
 */

#include "pg_regress.h"

static int
ecpg_compile(const char *inputdir, const char *pgcfile, const char *cfile, const char *binary,
             const char *ecpg_binary, const char* ecpg_flags,
             int isrelease)
{
    char        cwd[MAXPGPATH];
    const char *ecpgcmdfmt = SYSTEMQUOTE "\"%s\\ecpg\" --regression %s -o \"%s\" \"%s\"" SYSTEMQUOTE;
    const char *clcmdfmt = "cl.exe %s /TC /MD%s /I. /I..\\..\\include /I..\\..\\..\\libpq  /I..\\..\\..\\..\\include
/link/defaultlib:..\\..\\..\\..\\..\\%s\\libecpg\\libecpg.lib
/defaultlib:..\\..\\..\\..\\..\\%s\\libecpg_compat\\libecpg_compat.lib
/defaultlib:..\\..\\..\\..\\..\\%s\\libpgtypes\\libpgtypes.lib/out:\"%s\""; 
    int size = strlen(clcmdfmt) + 1024;
    char *s = malloc(size);
    char *p;

    /* XXX - check for ERANGE */
    getcwd(cwd, MAXPGPATH);
    chdir(inputdir);
    snprintf(s, size, ecpgcmdfmt, ecpg_binary,
                                  ecpg_flags,
                                  cfile,
                                  pgcfile);
    printf("executing %s\n", s);
    system(s);

    snprintf(s, size, clcmdfmt, cfile,
                              isrelease ? "" : "d",
                              isrelease ? "Release" : "debug",
                              isrelease ? "Release" : "debug",
                              isrelease ? "Release" : "debug",
                              binary);
    printf("executing %s\n", s);
    system(s);

    strcpy(s, binary);
    strcat(s, ".manifest");
    printf("deleting %s\n", s);
    //unlink(s);
    strcpy(s, binary);
    p = strstr(s, ".exe");
    *p = '\0';
    strcat(s, ".obj");
    printf("deleting %s\n", s);
    //unlink(s);
    strcpy(s, binary);
    p = strstr(s, ".exe");
    *p = '\0';
    strcat(s, ".suo");
    printf("deleting %s\n", s);
    unlink(s);
    chdir(cwd);

    return 1;
}

#define LINEBUFSIZE 300
static void
ecpg_filter(const char *sourcefile, const char *outfile)
{
    /*
     * Create a filtered copy of sourcefile, replacing
     * #line x "./../bla/foo.h"
     * with
     * #line x "foo.h"
     */
    FILE *s, *t;
    char linebuf[LINEBUFSIZE];

    s = fopen(sourcefile, "r");
    if (!s)
    {
        fprintf(stderr, "Could not open file %s for reading\n", sourcefile);
        exit_nicely(2);
    }
    t = fopen(outfile, "w");
    if (!t)
    {
        fprintf(stderr, "Could not open file %s for writing\n", outfile);
        exit_nicely(2);
    }

    while (fgets(linebuf, LINEBUFSIZE, s))
    {
        /* check for "#line " in the beginning */
        if (strstr(linebuf, "#line ") == linebuf)
        {
            char *p = strchr(linebuf, '"');
            char *n;
            int plen = 1;
            while (*p && (*(p + plen) == '.' || strchr(p + plen, '/') != NULL))
            {
                plen++;
            }
            /* plen is one more than the number of . and / characters */
            if (plen > 1)
            {
                n = (char *) malloc(plen);
                strncpy(n, p+1, plen - 1);
                n[plen-1] = '\0';
                replace_string(linebuf, n, "");
            }
        }
        fputs(linebuf, t);
    }
    fclose(s);
    fclose(t);
}

/*
 * start an ecpg test process for specified file (including redirection),
 * and return process ID
 */

static PID_TYPE
ecpg_start_test(const char *testname,
                _stringlist **resultfiles,
                _stringlist **expectfiles,
                _stringlist **tags)
{
    PID_TYPE    pid;
    char        inprg[MAXPGPATH];
    char        insource[MAXPGPATH];
    char        *outfile_stdout, expectfile_stdout[MAXPGPATH];
    char        *outfile_stderr, expectfile_stderr[MAXPGPATH];
    char        *outfile_source, expectfile_source[MAXPGPATH];
    char        cmd[MAXPGPATH * 3];
    char        *testname_dash;

    snprintf(inprg, sizeof(inprg), "%s/%s", inputdir, testname);

    testname_dash = strdup(testname);
    replace_string(testname_dash, "/", "-");
    snprintf(expectfile_stdout, sizeof(expectfile_stdout),
             "%s/expected/%s.stdout",
             outputdir, testname_dash);
    snprintf(expectfile_stderr, sizeof(expectfile_stderr),
             "%s/expected/%s.stderr",
             outputdir, testname_dash);
    snprintf(expectfile_source, sizeof(expectfile_source),
             "%s/expected/%s.c",
             outputdir, testname_dash);

    /*
     * We can use replace_string() here because the replacement string does
     * not occupy more space than the replaced one.
     */
    outfile_stdout = strdup(expectfile_stdout);
    replace_string(outfile_stdout, "/expected/", "/results/");
    outfile_stderr = strdup(expectfile_stderr);
    replace_string(outfile_stderr, "/expected/", "/results/");
    outfile_source = strdup(expectfile_source);
    replace_string(outfile_source, "/expected/", "/results/");

    add_stringlist_item(resultfiles, outfile_stdout);
    add_stringlist_item(expectfiles, expectfile_stdout);
    add_stringlist_item(tags, "stdout");

    add_stringlist_item(resultfiles, outfile_stderr);
    add_stringlist_item(expectfiles, expectfile_stderr);
    add_stringlist_item(tags, "stderr");

    add_stringlist_item(resultfiles, outfile_source);
    add_stringlist_item(expectfiles, expectfile_source);
    add_stringlist_item(tags, "source");

    /* XXX - move to tmp dir */
    snprintf(insource, sizeof(insource), "%s.c", testname);
    if (strstr(host_platform, "-win32vc"))
    {
        char    pgcfile[MAXPGPATH];
        char    cfile[MAXPGPATH];
        char    exefile[MAXPGPATH];
        char    testdir[MAXPGPATH];
        char   *p;
        char   *ecpg_flags = "";
        /* testname is like foo/bar, replace '/' with \0 */

        strcpy(testdir, testname);
        p = strchr(testdir, '/');
        if (!p) {
            /* XXX */
        }
        *p = '\0';
        strcpy(pgcfile, p+1);
        strcat(pgcfile, ".pgc");
        strcpy(exefile, p+1);
        strcat(exefile, ".exe");
        strcpy(cfile, p+1);
        strcat(cfile, ".c");
        if (strstr(testname, "compat_informix")) {
            if (strstr(testname,  "rnull"))
                ecpg_flags = "-C INFORMIX -r no_indicator";
            else
                ecpg_flags = "-C INFORMIX";
        } else
            ecpg_flags = "";

        /* XXX: detect release build */
        ecpg_compile(testdir, pgcfile, cfile, exefile, ecpgdir, ecpg_flags, 0);
    }
    ecpg_filter(insource, outfile_source);

    snprintf(inprg, sizeof(inprg), "%s/%s", inputdir, testname);

    snprintf(cmd, sizeof(cmd),
             SYSTEMQUOTE "\"%s\" >\"%s\" 2>\"%s\"" SYSTEMQUOTE,
             inprg,
             outfile_stdout,
             outfile_stderr);

    pid = spawn_process(cmd);

    if (pid == INVALID_PID)
    {
        fprintf(stderr, _("could not start process for test %s\n"),
                testname);
        exit_nicely(2);
    }

    free(outfile_stdout);
    free(outfile_stderr);
    free(outfile_source);

    return pid;
}

static void
ecpg_init(void)
{
    /* no reason to set -w for ecpg checks, except for when on windows */
    if (strstr(host_platform, "-win32"))
        basic_diff_opts = "-w";
    else
        basic_diff_opts = "";
    if (strstr(host_platform, "-win32"))
        pretty_diff_opts = "-C3 -w";
    else
        pretty_diff_opts = "-C3";
}

int
main(int argc, char *argv[])
{
    return regression_main(argc, argv, ecpg_init, ecpg_start_test);
}

/*-------------------------------------------------------------------------
 *
 * pg_regress_main --- regression test for the main backend
 *
 * This is a C implementation of the previous shell script for running
 * the regression tests, and should be mostly compatible with it.
 * Initial author of C translation: Magnus Hagander
 *
 * This code is released under the terms of the PostgreSQL License.
 *
 * Portions Copyright (c) 1996-2007, PostgreSQL Global Development Group
 * Portions Copyright (c) 1994, Regents of the University of California
 *
 * $PostgreSQL$
 *
 *-------------------------------------------------------------------------
 */

#include "pg_regress.h"

/*
 * start a psql test process for specified file (including redirection),
 * and return process ID
 */
static PID_TYPE
psql_start_test(const char *testname,
                _stringlist **resultfiles,
                _stringlist **expectfiles,
                _stringlist **tags)
{
    PID_TYPE    pid;
    char        infile[MAXPGPATH];
    char        outfile[MAXPGPATH];
    char        expectfile[MAXPGPATH];
    char        psql_cmd[MAXPGPATH * 3];

    snprintf(infile, sizeof(infile), "%s/sql/%s.sql",
             inputdir, testname);
    snprintf(outfile, sizeof(outfile), "%s/results/%s.out",
             outputdir, testname);
    snprintf(expectfile, sizeof(expectfile), "%s/expected/%s.out",
             inputdir, testname);

    add_stringlist_item(resultfiles, outfile);
    add_stringlist_item(expectfiles, expectfile);

    snprintf(psql_cmd, sizeof(psql_cmd),
             SYSTEMQUOTE "\"%s%spsql\" -X -a -q -d \"%s\" < \"%s\" > \"%s\" 2>&1" SYSTEMQUOTE,
             psqldir ? psqldir : "",
             psqldir ? "/" : "",
             dblist->str,
             infile,
             outfile);

    pid = spawn_process(psql_cmd);

    if (pid == INVALID_PID)
    {
        fprintf(stderr, _("could not start process for test %s\n"),
                testname);
        exit_nicely(2);
    }

    return pid;
}

static void
psql_init(void)
{
    /* set default regression database name */
    add_stringlist_item(&dblist, "regression");
}

int
main(int argc, char *argv[])
{
    return regression_main(argc, argv, psql_init, psql_start_test);
}
/*-------------------------------------------------------------------------
 * pg_regress.h --- regression test driver
 *
 * Portions Copyright (c) 1996-2007, PostgreSQL Global Development Group
 * Portions Copyright (c) 1994, Regents of the University of California
 *
 * $PostgreSQL$
 *-------------------------------------------------------------------------
 */

#include "postgres_fe.h"
#include <unistd.h>

#ifndef WIN32
#define PID_TYPE pid_t
#define INVALID_PID (-1)
#else
#define PID_TYPE HANDLE
#define INVALID_PID INVALID_HANDLE_VALUE
#endif

/* simple list of strings */
typedef struct _stringlist
{
    char       *str;
    struct _stringlist *next;
}    _stringlist;

typedef PID_TYPE (*test_function)(const char *,
                _stringlist **,
                _stringlist **,
                _stringlist **);
typedef void (*init_function)(void);

extern char *bindir;
extern char *libdir;
extern char *datadir;
extern char *host_platform;

extern _stringlist *dblist;
extern bool debug;
extern char *inputdir;
extern char *outputdir;
/*
 * This should not be global but every module should be able to read command
 * line parameters. ecpg tests don't need psql and psql doesn't need ecpg
 */
extern char *psqldir;
extern char *ecpgdir;

extern const char *basic_diff_opts;
extern const char *pretty_diff_opts;

int regression_main(int argc, char *argv[],
                    init_function ifunc, test_function tfunc);
void add_stringlist_item(_stringlist ** listhead, const char *str);
PID_TYPE spawn_process(const char *cmdline);
void exit_nicely(int code);
void replace_string(char *string, char *replace, char *replacement);


Re: Regression tests

From
Magnus Hagander
Date:
On Sat, Jun 09, 2007 at 10:55:55PM +0200, Magnus Hagander wrote:
> Magnus Hagander wrote:
> > Joachim Wieland attempted to post this patch, but it appears to be gone.
> > I tried a repost, and notivced it got rejected because it was >100kb.
> > Let me repeat previous objections that it really should be possible to
> > post a patch >100kb.
> > That said, here's a gzipped version.
> >
> > Joachim, once it comes through, feel free to post whatever comments you
> > had in your original mail.
>
> Here's an updated version of this patch. I reversed some parts of where
> files went, which made the patch a whole lot smaller and easier to read,
> and changed a couple of smaller things.
>
> I have not yet looked at the actual ecpg parts (except noticed that they
> are working on msvc at least), just the changes to main regression
> tests. But I wanted to send off my new version for others to look at as
> well before I do that.
>
> This is still WIP, but comments appreciated.

I have applied a fairly well reworked version of this. The big thing is
that I moved the building of the pgc code out of the regression test driver
and into the build system using msbuild.

I also did a couple of minor fixes to the threading tests and such.

//Magnus

Re: Regression tests

From
Michael Meskes
Date:
On Tue, Jun 12, 2007 at 01:16:32PM +0200, Magnus Hagander wrote:
> I have applied a fairly well reworked version of this. The big thing is
> that I moved the building of the pgc code out of the regression test driver
> and into the build system using msbuild.
>
> I also did a couple of minor fixes to the threading tests and such.

Thanks.

It seems you missed one file. make checktcp says:

pg_regress: could not open file "./ecpg_schedule_tcp" for reading

Michael
--
Michael Meskes
Email: Michael at Fam-Meskes dot De, Michael at Meskes dot (De|Com|Net|Org)
ICQ: 179140304, AIM/Yahoo: michaelmeskes, Jabber: meskes@jabber.org
Go SF 49ers! Go Rhein Fire! Use Debian GNU/Linux! Use PostgreSQL!

Re: Regression tests

From
Magnus Hagander
Date:
On Tue, Jun 12, 2007 at 02:25:24PM +0200, Michael Meskes wrote:
> On Tue, Jun 12, 2007 at 01:16:32PM +0200, Magnus Hagander wrote:
> > I have applied a fairly well reworked version of this. The big thing is
> > that I moved the building of the pgc code out of the regression test driver
> > and into the build system using msbuild.
> >
> > I also did a couple of minor fixes to the threading tests and such.
>
> Thanks.
>
> It seems you missed one file. make checktcp says:
>
> pg_regress: could not open file "./ecpg_schedule_tcp" for reading

Oops, search/replace error. Fixed.

//Magnus