Thread: PostgreSQL stops when adding a breakpoint in CLion
I tried debugging PostgreSQL to better understand how it works. It worked fine a day ago, but for some reason I have issues with debugging now:
- If I put a breakpoint before I start the process then everything works fine
- But if I put/remove a breakpoint after it's fully initialized - the process just stops
And when reading the next command postgres.c, I see that input_message is empty. I assume CLion sends a signal which awakens PostgreSQL, but there's no data on the input? But should PostgreSQL quit in such a situation?
The way I build and start:
make clean
./configure --enable-cassert --enable-debug CFLAGS="-ggdb -O0 -g3 -fno-omit-frame-pointer"
make
make install
/usr/local/pgsql/bin/initdb -D /Users/stas/projects/postgres/data
Starting command:
/usr/local/pgsql/bin/postgres --single -D /Users/stas/projects/postgres/data postgres
On 1/3/22 16:54, Stanislav Bashkyrtsev wrote: > I tried debugging PostgreSQL to better understand how it works. It > worked fine a day ago, but for some reason I have issues with debugging now: > > - If I put a breakpoint before I start the process then everything works > fine > - But if I put/remove a breakpoint after it's fully initialized - the > process just stops > > And when reading the next command postgres.c, I see that > input_message is empty. I assume CLion sends a signal which awakens > PostgreSQL, but there's no data on the input? But should PostgreSQL quit > in such a situation? > Why do you think postgres quits? AFAIK CLion uses gdb or lldb for debugging, which are the debugger of choice for many (most?) hackers on this list. So that should work fine. > The way I build and start: > make clean > ./configure --enable-cassert --enable-debug CFLAGS="-ggdb -O0 -g3 > -fno-omit-frame-pointer" > make > make install > /usr/local/pgsql/bin/initdb -D /Users/stas/projects/postgres/data > > Starting command: > /usr/local/pgsql/bin/postgres --single -D > /Users/stas/projects/postgres/data postgres Now sure why you start it in single-user mode, but I don't think that should affect debugging. Try redirecting the output to a log file, maybe that'll tell you what happened. regards -- Tomas Vondra EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company
Tomas Vondra <tomas.vondra@enterprisedb.com> writes:
> On 1/3/22 16:54, Stanislav Bashkyrtsev wrote:
>> - If I put a breakpoint before I start the process then everything works
>> fine
>> - But if I put/remove a breakpoint after it's fully initialized - the
>> process just stops
> Why do you think postgres quits? AFAIK CLion uses gdb or lldb for
> debugging, which are the debugger of choice for many (most?) hackers on
> this list. So that should work fine.
FWIW, it's normal in gdb that if you attach to an existing process,
the process stops until you say "continue". I know nothing of CLion,
but it likely follows that convention too.
regards, tom lane
> Why do you think postgres quits?
The process was running and then it stopped. And in the console I see:
2022-01-03 23:23:29.495 MSK [76717] LOG: checkpoint starting: shutdown immediate
2022-01-03 23:23:29.498 MSK [76717] LOG: checkpoint complete: wrote 3 buffers (0.0%); 0 WAL file(s) added, 0 removed, 0 recycled; write=0.002 s, sync=0.001 s, total=0.005 s; sync files=2, longest=0.001 s, average=0.001 s; distance=0 kB, estimate=0 kB
2022-01-03 23:23:29.498 MSK [76717] LOG: checkpoint complete: wrote 3 buffers (0.0%); 0 WAL file(s) added, 0 removed, 0 recycled; write=0.002 s, sync=0.001 s, total=0.005 s; sync files=2, longest=0.001 s, average=0.001 s; distance=0 kB, estimate=0 kB
> AFAIK CLion uses gdb or lldb for
> debugging, which are the debugger of choice for many (most?) hackers on> this list. So that should work fine.
Yep, and it worked for me too.. Yesterday :) I see that CLion uses LLDB on MacOS by default.
> Now sure why you start it in single-user mode, but I don't think that
> should affect debugging.Well, --single seems convenient because CLion starts that process and attaches to it right away. I don't have to look for a way of attaching to the forks. Maybe it's a good point to mention that I'm not very familiar with developing in C/C++ and therefore have a vague understanding of how to set up an efficient dev environment. Moreover in multi-user mode CLion/LLDB keeps stopping in postmaster.c:
selres = select(nSockets, &rmask, NULL, NULL, &timeout);
>Try redirecting the output to a log file, maybe
> that'll tell you what happened.
I see all the output in the console, so not sure what redirecting to a file would achieve.
On Mon, Jan 3, 2022 at 10:08 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
Tomas Vondra <tomas.vondra@enterprisedb.com> writes:
> On 1/3/22 16:54, Stanislav Bashkyrtsev wrote:
>> - If I put a breakpoint before I start the process then everything works
>> fine
>> - But if I put/remove a breakpoint after it's fully initialized - the
>> process just stops
> Why do you think postgres quits? AFAIK CLion uses gdb or lldb for
> debugging, which are the debugger of choice for many (most?) hackers on
> this list. So that should work fine.
FWIW, it's normal in gdb that if you attach to an existing process,
the process stops until you say "continue". I know nothing of CLion,
but it likely follows that convention too.
regards, tom lane
Stanislav Bashkyrtsev <stanislav.bashkirtsev@gmail.com> writes:
>> Why do you think postgres quits?
> The process was running and then it stopped. And in the console I see:
> 2022-01-03 23:23:29.495 MSK [76717] LOG: checkpoint starting: shutdown
> immediate
In a standalone backend, I think there are only 3 ways to get to
normal shutdown:
* SIGTERM
* SIGQUIT
* EOF on stdin
It's not very clear which of those your setup is triggering.
In any case, debugging standalone mode is very very rarely
what you should be doing; it's only vaguely related to normal
operation, plus you lack all the creature comforts of psql.
The usual thing is to start a normal interactive session, find out
the PID of its connected backend process ("select pg_backend_pid()"
is a reliable way), and then attach to that process with GDB or your
debugger of choice.
regards, tom lane
> In a standalone backend, I think there are only 3 ways to get to
> normal shutdown:
> * SIGTERM
> * SIGQUIT
> * EOF on stdin
> normal shutdown:
> * SIGTERM
> * SIGQUIT
> * EOF on stdin
I debugged a bit more and I see that getc() returns with -1 in interactive_getc() which is interpreted as EOF:
c = getc(stdin);
I see that errno == EINTR when it happens. This is as much as I can figure out in C, so I'm leaving it at that. Your advice about debugging the backend process ("select pg_backend_pid()") instead of running in a single-user mode worked for me, thank you!
On Tue, Jan 4, 2022 at 1:02 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
Stanislav Bashkyrtsev <stanislav.bashkirtsev@gmail.com> writes:
>> Why do you think postgres quits?
> The process was running and then it stopped. And in the console I see:
> 2022-01-03 23:23:29.495 MSK [76717] LOG: checkpoint starting: shutdown
> immediate
In a standalone backend, I think there are only 3 ways to get to
normal shutdown:
* SIGTERM
* SIGQUIT
* EOF on stdin
It's not very clear which of those your setup is triggering.
In any case, debugging standalone mode is very very rarely
what you should be doing; it's only vaguely related to normal
operation, plus you lack all the creature comforts of psql.
The usual thing is to start a normal interactive session, find out
the PID of its connected backend process ("select pg_backend_pid()"
is a reliable way), and then attach to that process with GDB or your
debugger of choice.
regards, tom lane