Thread: Allowing to run a buildfarm animal under valgrind
Hi,
I'm setting up a buildfarm animal that runs under
valgrind. Unfortunately there's not really any good solution to force
make check et al. to start postgres wrapped in valgrind. For now I've
resorted to adding something like
sub replace_postgres
{ my $srcdir=$use_vpath ? "../pgsql/" : "."; my $builddir=abs_path("$pgsql"); $srcdir=abs_path("$pgsql/$srcdir");
chdir "$pgsql/src/backend/"; rename "postgres", "postgres.orig"; sysopen my $fh, "postgres",
O_CREAT|O_TRUNC|O_RDWR,0700 or die "Could not create postgres wrapper"; print $fh <<"END";
#!/bin/bash
~/src/valgrind/vg-in-place \\ --quiet \\ --error-exitcode=128 \\ --suppressions=$srcdir/src/tools/valgrind.supp
\\ --trace-children=yes --track-origins=yes --read-var-info=yes \\ --leak-check=no \\
$builddir/src/backend/postgres.orig\\ "\$@"
END close $fh; chdir $branch_root;
}
to the buildfarm client.
i.e. a script that replaces the postgres binary with a wrapper that
invokes postgres via valgrind.
That's obviously not a very good approach though. It's buildfarm
specific and thus can't be invoked by developers and it doesn't really
support being installed somewhere.
Does anybody have a better idea about how to do this?
Regards,
Andres
Hi, On 2016-03-07 17:39:30 -0800, Andres Freund wrote: > I'm setting up a buildfarm animal that runs under valgrind. Which is now running as 'skink'. The first failed due to a missing trick in the wrapper script, but the second one looks like it had a legit issue: http://pgbuildfarm.org/cgi-bin/show_log.pl?nm=skink&dt=2016-03-08%2004%3A22%3A00 ==1827== Invalid write of size 4 ==1827== at 0x14E35DD1: plperl_ref_from_pg_array (plperl.c:1459) ==1827== by 0x14E3608C: plperl_call_perl_func (plperl.c:2135) ==1827== by 0x14E3C24F: plperl_func_handler (plperl.c:2357) ==1827== by 0x14E3C24F: plperl_call_handler (plperl.c:1778) ==1827== by 0x5E0531: ExecMakeFunctionResultNoSets (execQual.c:2041) ==1827== by 0x5E641C: ExecTargetList (execQual.c:5367) ==1827== by 0x5E641C: ExecProject (execQual.c:5582) ==1827== by 0x5FC1C1: ExecResult (nodeResult.c:155) ==1827== by 0x5DF577: ExecProcNode (execProcnode.c:392) ==1827== by 0x5DB675: ExecutePlan (execMain.c:1566) ==1827== by 0x5DB675: standard_ExecutorRun (execMain.c:338) ==1827== by 0x6F4DD7: PortalRunSelect (pquery.c:942) ==1827== by 0x6F631D: PortalRun (pquery.c:786) ==1827== by 0x6F2FFA: exec_simple_query (postgres.c:1094) ==1827== by 0x6F2FFA: PostgresMain (postgres.c:4021) ==1827== by 0x46D33F: BackendRun (postmaster.c:4258) ==1827== by 0x46D33F: BackendStartup (postmaster.c:3932) ==1827== by 0x46D33F: ServerLoop (postmaster.c:1690) ==1827== Address 0x15803220 is 304 bytes inside a block of size 8,192 alloc'd ==1827== at 0x4C28BB5: malloc (vg_replace_malloc.c:299) ==1827== by 0x808A37: AllocSetAlloc (aset.c:864) ==1827== by 0x80A3B3: palloc (mcxt.c:907) ==1827== by 0x7E0802: get_func_signature (lsyscache.c:1483) ==1827== by 0x14E367D6: plperl_call_perl_func (plperl.c:2116) ==1827== by 0x14E3C24F: plperl_func_handler (plperl.c:2357) ==1827== by 0x14E3C24F: plperl_call_handler (plperl.c:1778) ==1827== by 0x5E0531: ExecMakeFunctionResultNoSets (execQual.c:2041) ==1827== by 0x5E641C: ExecTargetList (execQual.c:5367) ==1827== by 0x5E641C: ExecProject (execQual.c:5582) ==1827== by 0x5FC1C1: ExecResult (nodeResult.c:155) ==1827== by 0x5DF577: ExecProcNode (execProcnode.c:392) ==1827== by 0x5DB675: ExecutePlan (execMain.c:1566) ==1827== by 0x5DB675: standard_ExecutorRun (execMain.c:338) ==1827== by 0x6F4DD7: PortalRunSelect (pquery.c:942) ==1827== I've now added configuration to valgrind so it wraps VALGRINDERROR-BEGIN / VALGRINDERROR-END around errors, to make the logs easier to search. Greetings, Andres Freund
On 03/07/2016 08:39 PM, Andres Freund wrote:
> Hi,
>
> I'm setting up a buildfarm animal that runs under
> valgrind. Unfortunately there's not really any good solution to force
> make check et al. to start postgres wrapped in valgrind. For now I've
> resorted to adding something like
>
> sub replace_postgres
> {
> my $srcdir=$use_vpath ? "../pgsql/" : ".";
> my $builddir=abs_path("$pgsql");
> $srcdir=abs_path("$pgsql/$srcdir");
> chdir "$pgsql/src/backend/";
> rename "postgres", "postgres.orig";
> sysopen my $fh, "postgres", O_CREAT|O_TRUNC|O_RDWR, 0700
> or die "Could not create postgres wrapper";
> print $fh <<"END";
> #!/bin/bash
> ~/src/valgrind/vg-in-place \\
> --quiet \\
> --error-exitcode=128 \\
> --suppressions=$srcdir/src/tools/valgrind.supp \\
> --trace-children=yes --track-origins=yes --read-var-info=yes \\
> --leak-check=no \\
> $builddir/src/backend/postgres.orig \\
> "\$@"
> END
> close $fh;
> chdir $branch_root;
> }
> to the buildfarm client.
>
> i.e. a script that replaces the postgres binary with a wrapper that
> invokes postgres via valgrind.
>
> That's obviously not a very good approach though. It's buildfarm
> specific and thus can't be invoked by developers and it doesn't really
> support being installed somewhere.
>
> Does anybody have a better idea about how to do this?
>
Why not just create a make target which does this? It could be run after
'make' and before 'make check'. I would make it assume valgrind was
installed and in the path rather than using vg-in-place.
If that doesn't work and you want to do something with the buildfarm,
probably a buildfarm module would be the way to go. We might need to add
a new module hook to support it, to run right after make(), but that
would be a one line addition to run_build.pl.
cheers
andrew
On 2016-03-08 08:58:22 -0500, Andrew Dunstan wrote: > On 03/07/2016 08:39 PM, Andres Freund wrote: > >Does anybody have a better idea about how to do this? > > Why not just create a make target which does this? It could be run after > 'make' and before 'make check'. I would make it assume valgrind was > installed and in the path rather than using vg-in-place. I don't really see how that'd work. make check et al. start postgres via pg_ctl, so we need to invoke valgrind from there. Additionally I don't want to just be able to run make check via valgrind, but all contrib modules et al too, including eventually the replication regression tests and such. > If that doesn't work and you want to do something with the buildfarm, > probably a buildfarm module would be the way to go. We might need to add a > new module hook to support it, to run right after make(), but that would be > a one line addition to run_build.pl. Yea, I think that's what it probably has to be. What I'm decidedly unhappy with right now is that this seems to require moving make install up, or manually add a new file for installation. The reason for that is that if we replace the postgres binary with the wrapper, the original file obviously doesn't get installed anymore. So it's invoked at it's original location; which only works if share files are installed in the correct location. I do wonder if adding a PGCTLPOSTGRESWRAPPER or something to pg_ctl would be ok. That'd just supplant calling the postgres binary, making all this easier. Andres
Andres Freund wrote: > I do wonder if adding a PGCTLPOSTGRESWRAPPER or something to pg_ctl > would be ok. That'd just supplant calling the postgres binary, making > all this easier. This seems a reasonably principled way to go about this. Eventually we might plug other things in it ... -- Álvaro Herrera http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
On 2016-03-08 18:24:23 -0300, Alvaro Herrera wrote: > Andres Freund wrote: > > > I do wonder if adding a PGCTLPOSTGRESWRAPPER or something to pg_ctl > > would be ok. That'd just supplant calling the postgres binary, making > > all this easier. > > This seems a reasonably principled way to go about this. Eventually we > might plug other things in it ... It's not that easy to write such a wrapper though - you *have* to exec the final binary, because -w assumes that the child pid is going to appear in postmaster.pid... To be actually useful we kinda would have to backpatch this to 9.4 (where the valgrind hardening stuff started)... Andres