Thread: There are many processes created and died frequently during PostgreSQl idle
Hello all:
I found that during postgresql running, there are so many processes being created and then died.
I am interested in the reason.
Here is the detail:
I installed from postgresql-9.2.1.tar.bz2.
I put some debug code in fd.c 's PathNameOpenFile function:
fprintf(stderr,"+++While Calling PathNameOpenFile pid=%d.....",getpid();
fprintf(stderr,"VfdCache Address is: %p \n\n",VfdCache);
When it run, I open two termianal using psql(pid=22109 and pid=22114), then I can got information like this:
[postgres@lex pgsql]$ ./bin/pg_ctl -D ./data start
server starting
[postgres@lex pgsql]$ LOG: database system was shut down at 2013-05-24 09:28:59 CST
LOG: autovacuum launcher started
+++While Calling PathNameOpenFile pid=22106.....VfdCache Address is: 0xbfb3320
LOG: database system is ready to accept connections
+++While Calling PathNameOpenFile pid=22109.....VfdCache Address is: 0xbfb2de0
+++While Calling PathNameOpenFile pid=22109.....VfdCache Address is: 0xbfb2de0
+++While Calling PathNameOpenFile pid=22109.....VfdCache Address is: 0xbfb2de0
+++While Calling PathNameOpenFile pid=22109.....VfdCache Address is: 0xbfb2de0
+++While Calling PathNameOpenFile pid=22109.....VfdCache Address is: 0xbfb2de0
+++While Calling PathNameOpenFile pid=22109.....VfdCache Address is: 0xbfb2de0
+++While Calling PathNameOpenFile pid=22109.....VfdCache Address is: 0xbfb2de0
+++While Calling PathNameOpenFile pid=22109.....VfdCache Address is: 0xbfb2de0
+++While Calling PathNameOpenFile pid=22109.....VfdCache Address is: 0xbfb2de0
+++While Calling PathNameOpenFile pid=22109.....VfdCache Address is: 0xbfb2de0
+++While Calling PathNameOpenFile pid=22109.....VfdCache Address is: 0xbfb2de0
+++While Calling PathNameOpenFile pid=22109.....VfdCache Address is: 0xbfb2de0
+++While Calling PathNameOpenFile pid=22109.....VfdCache Address is: 0xbfb2de0
+++While Calling PathNameOpenFile pid=22109.....VfdCache Address is: 0xbfb2de0
+++While Calling PathNameOpenFile pid=22109.....VfdCache Address is: 0xbfb2de0
+++While Calling PathNameOpenFile pid=22114.....VfdCache Address is: 0xbfba400
+++While Calling PathNameOpenFile pid=22114.....VfdCache Address is: 0xbfba400
+++While Calling PathNameOpenFile pid=22114.....VfdCache Address is: 0xbfba400
+++While Calling PathNameOpenFile pid=22114.....VfdCache Address is: 0xbfba400
+++While Calling PathNameOpenFile pid=22120.....VfdCache Address is: 0xbfe6700
+++While Calling PathNameOpenFile pid=22120.....VfdCache Address is: 0xbfe6700
+++While Calling PathNameOpenFile pid=22120.....VfdCache Address is: 0xbfe6700
+++While Calling PathNameOpenFile pid=22127.....VfdCache Address is: 0xbfe6700
+++While Calling PathNameOpenFile pid=22127.....VfdCache Address is: 0xbfe6700
+++While Calling PathNameOpenFile pid=22127.....VfdCache Address is: 0xbfe6700
+++While Calling PathNameOpenFile pid=22134.....VfdCache Address is: 0xbfe6700
+++While Calling PathNameOpenFile pid=22134.....VfdCache Address is: 0xbfe6700
+++While Calling PathNameOpenFile pid=22134.....VfdCache Address is: 0xbfe6700
+++While Calling PathNameOpenFile pid=22142.....VfdCache Address is: 0xbfe6700
+++While Calling PathNameOpenFile pid=22142.....VfdCache Address is: 0xbfe6700
+++While Calling PathNameOpenFile pid=22142.....VfdCache Address is: 0xbfe6700
+++While Calling PathNameOpenFile pid=22149.....VfdCache Address is: 0xbfe6700
+++While Calling PathNameOpenFile pid=22149.....VfdCache Address is: 0xbfe6700
+++While Calling PathNameOpenFile pid=22149.....VfdCache Address is: 0xbfe6700
+++While Calling PathNameOpenFile pid=22156.....VfdCache Address is: 0xbfe6700
+++While Calling PathNameOpenFile pid=22156.....VfdCache Address is: 0xbfe6700
+++While Calling PathNameOpenFile pid=22156.....VfdCache Address is: 0xbfe6700
+++While Calling PathNameOpenFile pid=22165.....VfdCache Address is: 0xbfe6700
+++While Calling PathNameOpenFile pid=22165.....VfdCache Address is: 0xbfe6700
+++While Calling PathNameOpenFile pid=22165.....VfdCache Address is: 0xbfe6700
+++While Calling PathNameOpenFile pid=22171.....VfdCache Address is: 0xbfe6700
+++While Calling PathNameOpenFile pid=22171.....VfdCache Address is: 0xbfe6700
+++While Calling PathNameOpenFile pid=22171.....VfdCache Address is: 0xbfe6700
+++While Calling PathNameOpenFile pid=22178.....VfdCache Address is: 0xbfe6700
+++While Calling PathNameOpenFile pid=22178.....VfdCache Address is: 0xbfe6700
+++While Calling PathNameOpenFile pid=22178.....VfdCache Address is: 0xbfe6700
But when I grep process information using ps command, I can't find process such like 22120,22127,22134,22142,22149,22156,22178.
And the output is growing longer and longer...
[root@lex ~]# ps -ef | grep post
root 4641 4621 0 08:28 pts/3 00:00:00 su - postgres
postgres 4642 4641 0 08:28 pts/3 00:00:00 -bash
root 4709 4688 0 08:28 pts/4 00:00:00 su - postgres
postgres 4710 4709 0 08:28 pts/4 00:00:00 -bash
root 4772 4751 0 08:29 pts/5 00:00:00 su - postgres
postgres 4773 4772 0 08:29 pts/5 00:00:00 -bash
postgres 22101 1 0 09:45 pts/3 00:00:00 /usr/local/pgsql/bin/postgres -D ./data
postgres 22103 22101 0 09:45 ? 00:00:00 postgres: checkpointer process
postgres 22104 22101 0 09:45 ? 00:00:00 postgres: writer process
postgres 22105 22101 0 09:45 ? 00:00:00 postgres: wal writer process
postgres 22106 22101 0 09:45 ? 00:00:00 postgres: autovacuum launcher process
postgres 22107 22101 0 09:45 ? 00:00:00 postgres: stats collector process
postgres 22108 4710 0 09:45 pts/4 00:00:00 ./psql
postgres 22109 22101 0 09:45 ? 00:00:00 postgres: postgres postgres [local] idle
postgres 22113 4773 0 09:46 pts/5 00:00:00 ./psql
postgres 22114 22101 0 09:46 ? 00:00:00 postgres: postgres postgres [local] idle
root 22187 6429 0 09:55 pts/6 00:00:00 grep post
[root@lex ~]#
So I think that some process is created and then quickly died . But would somebody can kindly say what are the purpose of those process?
How about using one worker process and keep it be alive?
the process you see may be autovacuum worker process,you can tune the log_autovacuum_min_duration to log the autovacuum activities and check if the pids match
2013/5/24 高健 <luckyjackgao@gmail.com>
Hello all:I found that during postgresql running, there are so many processes being created and then died.I am interested in the reason.Here is the detail:I installed from postgresql-9.2.1.tar.bz2.I put some debug code in fd.c 's PathNameOpenFile function:fprintf(stderr,"+++While Calling PathNameOpenFile pid=%d.....",getpid();fprintf(stderr,"VfdCache Address is: %p \n\n",VfdCache);When it run, I open two termianal using psql(pid=22109 and pid=22114), then I can got information like this:[postgres@lex pgsql]$ ./bin/pg_ctl -D ./data startserver starting[postgres@lex pgsql]$ LOG: database system was shut down at 2013-05-24 09:28:59 CSTLOG: autovacuum launcher started+++While Calling PathNameOpenFile pid=22106.....VfdCache Address is: 0xbfb3320LOG: database system is ready to accept connections+++While Calling PathNameOpenFile pid=22109.....VfdCache Address is: 0xbfb2de0+++While Calling PathNameOpenFile pid=22109.....VfdCache Address is: 0xbfb2de0+++While Calling PathNameOpenFile pid=22109.....VfdCache Address is: 0xbfb2de0+++While Calling PathNameOpenFile pid=22109.....VfdCache Address is: 0xbfb2de0+++While Calling PathNameOpenFile pid=22109.....VfdCache Address is: 0xbfb2de0+++While Calling PathNameOpenFile pid=22109.....VfdCache Address is: 0xbfb2de0+++While Calling PathNameOpenFile pid=22109.....VfdCache Address is: 0xbfb2de0+++While Calling PathNameOpenFile pid=22109.....VfdCache Address is: 0xbfb2de0+++While Calling PathNameOpenFile pid=22109.....VfdCache Address is: 0xbfb2de0+++While Calling PathNameOpenFile pid=22109.....VfdCache Address is: 0xbfb2de0+++While Calling PathNameOpenFile pid=22109.....VfdCache Address is: 0xbfb2de0+++While Calling PathNameOpenFile pid=22109.....VfdCache Address is: 0xbfb2de0+++While Calling PathNameOpenFile pid=22109.....VfdCache Address is: 0xbfb2de0+++While Calling PathNameOpenFile pid=22109.....VfdCache Address is: 0xbfb2de0+++While Calling PathNameOpenFile pid=22109.....VfdCache Address is: 0xbfb2de0+++While Calling PathNameOpenFile pid=22114.....VfdCache Address is: 0xbfba400+++While Calling PathNameOpenFile pid=22114.....VfdCache Address is: 0xbfba400+++While Calling PathNameOpenFile pid=22114.....VfdCache Address is: 0xbfba400+++While Calling PathNameOpenFile pid=22114.....VfdCache Address is: 0xbfba400+++While Calling PathNameOpenFile pid=22120.....VfdCache Address is: 0xbfe6700+++While Calling PathNameOpenFile pid=22120.....VfdCache Address is: 0xbfe6700+++While Calling PathNameOpenFile pid=22120.....VfdCache Address is: 0xbfe6700+++While Calling PathNameOpenFile pid=22127.....VfdCache Address is: 0xbfe6700+++While Calling PathNameOpenFile pid=22127.....VfdCache Address is: 0xbfe6700+++While Calling PathNameOpenFile pid=22127.....VfdCache Address is: 0xbfe6700+++While Calling PathNameOpenFile pid=22134.....VfdCache Address is: 0xbfe6700+++While Calling PathNameOpenFile pid=22134.....VfdCache Address is: 0xbfe6700+++While Calling PathNameOpenFile pid=22134.....VfdCache Address is: 0xbfe6700+++While Calling PathNameOpenFile pid=22142.....VfdCache Address is: 0xbfe6700+++While Calling PathNameOpenFile pid=22142.....VfdCache Address is: 0xbfe6700+++While Calling PathNameOpenFile pid=22142.....VfdCache Address is: 0xbfe6700+++While Calling PathNameOpenFile pid=22149.....VfdCache Address is: 0xbfe6700+++While Calling PathNameOpenFile pid=22149.....VfdCache Address is: 0xbfe6700+++While Calling PathNameOpenFile pid=22149.....VfdCache Address is: 0xbfe6700+++While Calling PathNameOpenFile pid=22156.....VfdCache Address is: 0xbfe6700+++While Calling PathNameOpenFile pid=22156.....VfdCache Address is: 0xbfe6700+++While Calling PathNameOpenFile pid=22156.....VfdCache Address is: 0xbfe6700+++While Calling PathNameOpenFile pid=22165.....VfdCache Address is: 0xbfe6700+++While Calling PathNameOpenFile pid=22165.....VfdCache Address is: 0xbfe6700+++While Calling PathNameOpenFile pid=22165.....VfdCache Address is: 0xbfe6700+++While Calling PathNameOpenFile pid=22171.....VfdCache Address is: 0xbfe6700+++While Calling PathNameOpenFile pid=22171.....VfdCache Address is: 0xbfe6700+++While Calling PathNameOpenFile pid=22171.....VfdCache Address is: 0xbfe6700+++While Calling PathNameOpenFile pid=22178.....VfdCache Address is: 0xbfe6700+++While Calling PathNameOpenFile pid=22178.....VfdCache Address is: 0xbfe6700+++While Calling PathNameOpenFile pid=22178.....VfdCache Address is: 0xbfe6700But when I grep process information using ps command, I can't find process such like 22120,22127,22134,22142,22149,22156,22178.And the output is growing longer and longer...[root@lex ~]# ps -ef | grep postroot 4641 4621 0 08:28 pts/3 00:00:00 su - postgrespostgres 4642 4641 0 08:28 pts/3 00:00:00 -bashroot 4709 4688 0 08:28 pts/4 00:00:00 su - postgrespostgres 4710 4709 0 08:28 pts/4 00:00:00 -bashroot 4772 4751 0 08:29 pts/5 00:00:00 su - postgrespostgres 4773 4772 0 08:29 pts/5 00:00:00 -bashpostgres 22101 1 0 09:45 pts/3 00:00:00 /usr/local/pgsql/bin/postgres -D ./datapostgres 22103 22101 0 09:45 ? 00:00:00 postgres: checkpointer processpostgres 22104 22101 0 09:45 ? 00:00:00 postgres: writer processpostgres 22105 22101 0 09:45 ? 00:00:00 postgres: wal writer processpostgres 22106 22101 0 09:45 ? 00:00:00 postgres: autovacuum launcher processpostgres 22107 22101 0 09:45 ? 00:00:00 postgres: stats collector processpostgres 22108 4710 0 09:45 pts/4 00:00:00 ./psqlpostgres 22109 22101 0 09:45 ? 00:00:00 postgres: postgres postgres [local] idlepostgres 22113 4773 0 09:46 pts/5 00:00:00 ./psqlpostgres 22114 22101 0 09:46 ? 00:00:00 postgres: postgres postgres [local] idleroot 22187 6429 0 09:55 pts/6 00:00:00 grep post[root@lex ~]#So I think that some process is created and then quickly died . But would somebody can kindly say what are the purpose of those process?How about using one worker process and keep it be alive?
Jov
blog: http:amutu.com/blog
Hi:
Thanks for Jov's reply.
I traced it again, and found they are really for autovacuum.
I found that
some will call proc_exit() from within AutoVacLauncherMain function,
some will call proc_exit() from within AutoVacWorkerMain function.
But I wonder why not using only a few daemon , instead of making those processes be short life.
2013/5/24 Jov <amutu@amutu.com>
the process you see may be autovacuum worker process,you can tune the log_autovacuum_min_duration to log the autovacuum activities and check if the pids match--2013/5/24 高健 <luckyjackgao@gmail.com>Hello all:I found that during postgresql running, there are so many processes being created and then died.I am interested in the reason.Here is the detail:I installed from postgresql-9.2.1.tar.bz2.I put some debug code in fd.c 's PathNameOpenFile function:fprintf(stderr,"+++While Calling PathNameOpenFile pid=%d.....",getpid();fprintf(stderr,"VfdCache Address is: %p \n\n",VfdCache);When it run, I open two termianal using psql(pid=22109 and pid=22114), then I can got information like this:[postgres@lex pgsql]$ ./bin/pg_ctl -D ./data startserver starting[postgres@lex pgsql]$ LOG: database system was shut down at 2013-05-24 09:28:59 CSTLOG: autovacuum launcher started+++While Calling PathNameOpenFile pid=22106.....VfdCache Address is: 0xbfb3320LOG: database system is ready to accept connections+++While Calling PathNameOpenFile pid=22109.....VfdCache Address is: 0xbfb2de0+++While Calling PathNameOpenFile pid=22109.....VfdCache Address is: 0xbfb2de0+++While Calling PathNameOpenFile pid=22109.....VfdCache Address is: 0xbfb2de0+++While Calling PathNameOpenFile pid=22109.....VfdCache Address is: 0xbfb2de0+++While Calling PathNameOpenFile pid=22109.....VfdCache Address is: 0xbfb2de0+++While Calling PathNameOpenFile pid=22109.....VfdCache Address is: 0xbfb2de0+++While Calling PathNameOpenFile pid=22109.....VfdCache Address is: 0xbfb2de0+++While Calling PathNameOpenFile pid=22109.....VfdCache Address is: 0xbfb2de0+++While Calling PathNameOpenFile pid=22109.....VfdCache Address is: 0xbfb2de0+++While Calling PathNameOpenFile pid=22109.....VfdCache Address is: 0xbfb2de0+++While Calling PathNameOpenFile pid=22109.....VfdCache Address is: 0xbfb2de0+++While Calling PathNameOpenFile pid=22109.....VfdCache Address is: 0xbfb2de0+++While Calling PathNameOpenFile pid=22109.....VfdCache Address is: 0xbfb2de0+++While Calling PathNameOpenFile pid=22109.....VfdCache Address is: 0xbfb2de0+++While Calling PathNameOpenFile pid=22109.....VfdCache Address is: 0xbfb2de0+++While Calling PathNameOpenFile pid=22114.....VfdCache Address is: 0xbfba400+++While Calling PathNameOpenFile pid=22114.....VfdCache Address is: 0xbfba400+++While Calling PathNameOpenFile pid=22114.....VfdCache Address is: 0xbfba400+++While Calling PathNameOpenFile pid=22114.....VfdCache Address is: 0xbfba400+++While Calling PathNameOpenFile pid=22120.....VfdCache Address is: 0xbfe6700+++While Calling PathNameOpenFile pid=22120.....VfdCache Address is: 0xbfe6700+++While Calling PathNameOpenFile pid=22120.....VfdCache Address is: 0xbfe6700+++While Calling PathNameOpenFile pid=22127.....VfdCache Address is: 0xbfe6700+++While Calling PathNameOpenFile pid=22127.....VfdCache Address is: 0xbfe6700+++While Calling PathNameOpenFile pid=22127.....VfdCache Address is: 0xbfe6700+++While Calling PathNameOpenFile pid=22134.....VfdCache Address is: 0xbfe6700+++While Calling PathNameOpenFile pid=22134.....VfdCache Address is: 0xbfe6700+++While Calling PathNameOpenFile pid=22134.....VfdCache Address is: 0xbfe6700+++While Calling PathNameOpenFile pid=22142.....VfdCache Address is: 0xbfe6700+++While Calling PathNameOpenFile pid=22142.....VfdCache Address is: 0xbfe6700+++While Calling PathNameOpenFile pid=22142.....VfdCache Address is: 0xbfe6700+++While Calling PathNameOpenFile pid=22149.....VfdCache Address is: 0xbfe6700+++While Calling PathNameOpenFile pid=22149.....VfdCache Address is: 0xbfe6700+++While Calling PathNameOpenFile pid=22149.....VfdCache Address is: 0xbfe6700+++While Calling PathNameOpenFile pid=22156.....VfdCache Address is: 0xbfe6700+++While Calling PathNameOpenFile pid=22156.....VfdCache Address is: 0xbfe6700+++While Calling PathNameOpenFile pid=22156.....VfdCache Address is: 0xbfe6700+++While Calling PathNameOpenFile pid=22165.....VfdCache Address is: 0xbfe6700+++While Calling PathNameOpenFile pid=22165.....VfdCache Address is: 0xbfe6700+++While Calling PathNameOpenFile pid=22165.....VfdCache Address is: 0xbfe6700+++While Calling PathNameOpenFile pid=22171.....VfdCache Address is: 0xbfe6700+++While Calling PathNameOpenFile pid=22171.....VfdCache Address is: 0xbfe6700+++While Calling PathNameOpenFile pid=22171.....VfdCache Address is: 0xbfe6700+++While Calling PathNameOpenFile pid=22178.....VfdCache Address is: 0xbfe6700+++While Calling PathNameOpenFile pid=22178.....VfdCache Address is: 0xbfe6700+++While Calling PathNameOpenFile pid=22178.....VfdCache Address is: 0xbfe6700But when I grep process information using ps command, I can't find process such like 22120,22127,22134,22142,22149,22156,22178.And the output is growing longer and longer...[root@lex ~]# ps -ef | grep postroot 4641 4621 0 08:28 pts/3 00:00:00 su - postgrespostgres 4642 4641 0 08:28 pts/3 00:00:00 -bashroot 4709 4688 0 08:28 pts/4 00:00:00 su - postgrespostgres 4710 4709 0 08:28 pts/4 00:00:00 -bashroot 4772 4751 0 08:29 pts/5 00:00:00 su - postgrespostgres 4773 4772 0 08:29 pts/5 00:00:00 -bashpostgres 22101 1 0 09:45 pts/3 00:00:00 /usr/local/pgsql/bin/postgres -D ./datapostgres 22103 22101 0 09:45 ? 00:00:00 postgres: checkpointer processpostgres 22104 22101 0 09:45 ? 00:00:00 postgres: writer processpostgres 22105 22101 0 09:45 ? 00:00:00 postgres: wal writer processpostgres 22106 22101 0 09:45 ? 00:00:00 postgres: autovacuum launcher processpostgres 22107 22101 0 09:45 ? 00:00:00 postgres: stats collector processpostgres 22108 4710 0 09:45 pts/4 00:00:00 ./psqlpostgres 22109 22101 0 09:45 ? 00:00:00 postgres: postgres postgres [local] idlepostgres 22113 4773 0 09:46 pts/5 00:00:00 ./psqlpostgres 22114 22101 0 09:46 ? 00:00:00 postgres: postgres postgres [local] idleroot 22187 6429 0 09:55 pts/6 00:00:00 grep post[root@lex ~]#So I think that some process is created and then quickly died . But would somebody can kindly say what are the purpose of those process?How about using one worker process and keep it be alive?
Jovblog: http:amutu.com/blog
Re: There are many processes created and died frequently during PostgreSQl idle
From
Amit Langote
Date:
On Mon, May 27, 2013 at 10:56 AM, 高健 <luckyjackgao@gmail.com> wrote: > Hi: > Thanks for Jov's reply. > I traced it again, and found they are really for autovacuum. > I found that > some will call proc_exit() from within AutoVacLauncherMain function, > some will call proc_exit() from within AutoVacWorkerMain function. > > But I wonder why not using only a few daemon , instead of making those > processes be short life. The short lived processes are autovacuum "worker" processes which need to exit as soon as their work is done. There does exist a daemon called autovacuum launcher, which periodically forks autovacuum worker processes to perform vacuum activity. You might want to read more about them here: http://www.postgresql.org/docs/9.2/static/routine-vacuuming.html#AUTOVACUUM -- Amit Langote