Re: New "raw" COPY format - Mailing list pgsql-hackers

From jian he
Subject Re: New "raw" COPY format
Date
Msg-id CACJufxFt15StJcJ-4SgJJauOUB9dr=sCVWY=Xd2BmriNHS4K2g@mail.gmail.com
Whole thread Raw
List pgsql-hackers
On Sat, Oct 12, 2024 at 5:02 AM Joel Jacobson <joel@compiler.org> wrote:
>
> On Fri, Oct 11, 2024, at 22:29, Joel Jacobson wrote:
> > Hi hackers,
> >
> > This thread is about implementing a new "raw" COPY format.
> ...
> > The attached patch implements the above ideas.
> >
> > I think with these changes, it would be easier to hack on new and existing
> > copy options and formats.
> >
> > /Joel
> >
> > Attachments:
> > * v1-0001-Replace-binary-flags-binary-and-csv_mode-with-format.patch
> > * v1-0002-Reorganize-ProcessCopyOptions-for-clarity-and-consis.patch
>
> Ops, I see I failed to use the correct way to check if
> opts_out->force_notnull or opts_out->force_null
> have been set, that is using != NIL.
>
> However, thanks to not just blindly copy/pasting this code,
> I see I actually fixed a bug in HEAD, by also checking
> opts_out->force_notnull_all or opts_out->force_null_all,
> which HEAD currently fails to do:

git version 2.34.1
cannot do `git apply`

trying:
patch -p1 < patch -p1 <
$PATCHES/v2-0001-Replace-binary-flags-binary-and-csv_mode-with-format.patch
patch -p1 <  $PATCHES/v2-0002-Reorganize-ProcessCopyOptions-for-clarity-and-consis.patch
After that, I still cannot apply.


typedef enum CopyFormat
{
    COPY_FORMAT_TEXT,
    COPY_FORMAT_BINARY,
    COPY_FORMAT_CSV
} CopyFormat;
the last element should add a comma.

CopyFormat should add to
src/tools/pgindent/typedefs.list


+        /* Assert options have been set (defaults applied if not specified) */
+        Assert(opts_out->delim);
+        Assert(opts_out->null_print);
+
+        /* Don't allow the delimiter to appear in the NULL or DEFAULT
strings */
+
+        if (strchr(opts_out->null_print, opts_out->delim[0]) != NULL)

+        Assert(opts_out->delim);
+        Assert(opts_out->quote);
+        Assert(opts_out->null_print);
+
+        if (opts_out->delim[0] == opts_out->quote[0])
these Asserts, no need? Without it, if conditions are not met, it will
still segfault.


there is no sql example, like
copy the_table from :'filename' (format raw);

in the patch.
I thought you were going to implement something like that.



pgsql-hackers by date:

Previous
From: Masahiko Sawada
Date:
Subject: Re: Missing deconstruct_array_builtin usage
Next
From: Zhang Mingli
Date:
Subject: Re: [BUG FIX] Fix validation of COPY options FORCE_NOT_NULL/FORCE_NULL