Re: proposal: possibility to read dumped table's name from file - Mailing list pgsql-hackers

From Pavel Stehule
Subject Re: proposal: possibility to read dumped table's name from file
Date
Msg-id CAFj8pRCrbDPAWyHjwZauEX78jyGBowOoiaWySA0ibRYoWzZTMw@mail.gmail.com
Whole thread Raw
In response to Re: proposal: possibility to read dumped table's name from file  (Justin Pryzby <pryzby@telsasoft.com>)
Responses Re: proposal: possibility to read dumped table's name from file  (Pavel Stehule <pavel.stehule@gmail.com>)
List pgsql-hackers
Hi

ne 29. 11. 2020 v 0:49 odesílatel Justin Pryzby <pryzby@telsasoft.com> napsal:
On Sat, Nov 28, 2020 at 09:14:35PM +0100, Pavel Stehule wrote:
> Any short or long option can be read from this file in simple format - one
> option per line. Arguments inside double quotes can be multi lined. Row
> comments started by # and can be used everywhere.

here is updated patch


Does this support even funkier table names ?

This tests a large number and fraction of characters in dbname/username, so all
of pg_dump has to continue supporting that:
./src/bin/pg_dump/t/010_dump_connstr.pl

I tested and it seems to work with -t "fooå"
But it didn't work with -t "foo\nbar" (literal newline).  Fix attached.
If you send another patch, please consider including a test case for quoted
names in long and short options.

I implemented some basic backslash escaping. I will write more tests, when there will be good agreement on the main concept.
 

> +static char *optsfilename = NULL;

> + * It assign the values of options to related DumpOption fields or to
> + * some global values. It is called from twice. First, for processing
> + * the command line argumens. Second, for processing an options from
> + * options file.

This didn't support multiple config files, nor config files which include
config files, as Dean and I mentioned.  I think the argument parsers should
themselves call the config file parser, as need be, so the last option
specification should override previous ones.

For example pg_dump --config-file=./pg_dump.conf --blobs should have blobs even
if the config file says --no-blobs.  (Command-line arguments normally take
precedence over config files, certainly if the argument is specified "later").
I think it'd be ok if it's recursive.  I made a quick hack to do that.

I did it. I used a different design than you. Making "dopt" be a global variable looks too invasive. Almost all functions there expect "dopt" as an argument. But I think it is not necessary.

I implemented two iterations of argument's processing. 1. for options file (more options-file options are allowed, and nesting is allowed too), 2. all other arguments from the command line. Any options file is processed only once - second processing is ignored. So there is no problem with cycles.

The name of the new option - "config-file" or "options-file" ? I prefer "options-file". "config-file" is valid too, but "options-file" is more specific, more descriptive (it is self descriptive).

I merged your patch with a fix of typos.
 
Regards

Pavel


I doubt this will satisfy Stephen.  Personally, I would use this if it were a
plain and simple text config file (which for our purposes I would pass on
stdin), and I would almost certainly not use it if it were json.  But it'd be
swell if there were a standard config file format, that handled postgresql.conf
and maybe pg_hba.conf.

--
Justin
Attachment

pgsql-hackers by date:

Previous
From: Kasahara Tatsuhito
Date:
Subject: Re: autovac issue with large number of tables
Next
From: James Coleman
Date:
Subject: Re: Fix generate_useful_gather_paths for parallel unsafe pathkeys