Thread: .pgpass
I'm running Debian unstable, and I keep getting a message from a cron job that wants to do.maintenance about no password being supplied. Password: psql: fe_sendauth: no password supplied Now, I gather in the past a person had to explicitly put a password in the cron job, but we are now supposed to use a file called ~/.pgpass (~ being /var/lib/postgres). Well, I do have such a file, so I am confused as to why I keep getting this error message from the cron job. Five colon separated fields host:port:*:user:password are in this file. Host is localhost, port is 5432, .... I must be missing something simple, but I don't see it. Gord
<ghaverla@freenet.edmonton.ab.ca> writes: > I'm running Debian unstable, and I keep getting a message from a > cron job that wants to do.maintenance about no password being > supplied. > Now, I gather in the past a person had to explicitly put a > password in the cron job, but we are now supposed to use a file > called ~/.pgpass (~ being /var/lib/postgres). That is the recommended solution, but I'll bet that the cron job is not running with the same value of $HOME that you think it is, and so it is failing to find the file. You might want to explicitly set $HOME in the script. Also, test the .pgpass file by manually running psql with it; if there is something wrong with the file contents you'll be able to prove it... regards, tom lane
On Thu, 1 Jul 2004, Tom Lane wrote: > <ghaverla@freenet.edmonton.ab.ca> writes: > > I'm running Debian unstable, and I keep getting a message from a > > cron job that wants to do.maintenance about no password being > > supplied. > > > Now, I gather in the past a person had to explicitly put a > > password in the cron job, but we are now supposed to use a file > > called ~/.pgpass (~ being /var/lib/postgres). > > That is the recommended solution, but I'll bet that the cron job is not > running with the same value of $HOME that you think it is, and so it is > failing to find the file. You might want to explicitly set $HOME in the > script. Well, neither the cron script nor /etc/postgresql/postgresql.env (which is sourced by the cron script) explicitly set $HOME. But the message from cron indicates that home is /var/lib/postgres and the user is postgres when the script runs. X-Cron-Env: <SHELL=/bin/sh> X-Cron-Env: <HOME=/var/lib/postgres> X-Cron-Env: <PATH=/usr/bin:/bin> X-Cron-Env: <LOGNAME=postgres> > Also, test the .pgpass file by manually running psql with it; if there > is something wrong with the file contents you'll be able to prove it... Well, if I 'su - postgres' from root, and then run 'psql template1' everything works fine. I suppose I could edit the .pgpass to contain a bad password, and see if psql fails. If I put a bad password in .pgpass, the attempt to try psql fails. So, psql is using .pgpass. Gord
<ghaverla@freenet.edmonton.ab.ca> writes: > the message from cron indicates that home is /var/lib/postgres > and the user is postgres when the script runs. > X-Cron-Env: <SHELL=/bin/sh> > X-Cron-Env: <HOME=/var/lib/postgres> > X-Cron-Env: <PATH=/usr/bin:/bin> > X-Cron-Env: <LOGNAME=postgres> Hm. Given that you seem to have eliminated other possibilities, I guess I'd question that message. Have you tried putting something like "env > logfile" into the cron script to directly verify what environment it gets? regards, tom lane
On Thu, 2004-07-01 at 18:26, ghaverla@freenet.edmonton.ab.ca wrote: > On Thu, 1 Jul 2004, Tom Lane wrote: > > <ghaverla@freenet.edmonton.ab.ca> writes: > > > > I'm running Debian unstable, and I keep getting a message from a > > > cron job that wants to do.maintenance about no password being > > > supplied. > Well, if I 'su - postgres' from root, and then run > 'psql template1' everything works fine. I suppose I could edit > the .pgpass to contain a bad password, and see if psql fails. If > I put a bad password in .pgpass, the attempt to try psql fails. > So, psql is using .pgpass. I suggest you edit /usr/lib/postgresql/bin/do.maintenance and add a line set -x at line 2. Then you will be able to see exactly what command it is trying to run. -- Oliver Elphick olly@lfix.co.uk Isle of Wight http://www.lfix.co.uk/oliver GPG: 1024D/A54310EA 92C8 39E7 280E 3631 3F0E 1EC0 5664 7A2F A543 10EA ======================================== "Therefore being justified by faith, we have peace with God through our Lord Jesus Christ." Romans 5:1
On Fri, 2 Jul 2004, Oliver Elphick wrote: > > > <ghaverla@freenet.edmonton.ab.ca> writes: > > > > > > I'm running Debian unstable, and I keep getting a message from a > > > > cron job that wants to do.maintenance about no password being > > > > supplied. > I suggest you edit /usr/lib/postgresql/bin/do.maintenance and add a line > set -x > at line 2. Then you will be able to see exactly what command it is > trying to run. ++ /usr/bin/psql -q -X -t -d template1 -P border=0 -c ' SELECT datname FROM pg_database WHERE datallowconn ORDER BY datname' Password: psql: fe_sendauth: no password supplied + dblist= + IFS= + /usr/bin/psql -d template1 -t -q -X Password: psql: fe_sendauth: no password supplied My guess is it is that -X (disallow reading of a .psqlrc file) is somehow turning off the reading of any .pgpass file. Gord
On Sun, 2004-07-04 at 20:41, ghaverla@freenet.edmonton.ab.ca wrote: > + /usr/bin/psql -d template1 -t -q -X > Password: > psql: fe_sendauth: no password supplied > > My guess is it is that -X (disallow reading of a .psqlrc file) is > somehow turning off the reading of any .pgpass file. I can't see how that could be. -X is a psql option, but .pgpass is read by the libpq library, which has no knowledge of the psql options. Can you check that do.maintenance is being run by the correct user? (Put the line "user identity = `id`" somewhere near the top.) Also, see if do.maintenance works when run directly rather than through cron. -- Oliver Elphick olly@lfix.co.uk Isle of Wight http://www.lfix.co.uk/oliver GPG: 1024D/A54310EA 92C8 39E7 280E 3631 3F0E 1EC0 5664 7A2F A543 10EA ======================================== "Therefore being justified by faith, we have peace with God through our Lord Jesus Christ." Romans 5:1
On Sun, 4 Jul 2004, Oliver Elphick wrote: > On Sun, 2004-07-04 at 20:41, ghaverla@freenet.edmonton.ab.ca wrote: > > + /usr/bin/psql -d template1 -t -q -X > > Password: > > psql: fe_sendauth: no password supplied > > > > My guess is it is that -X (disallow reading of a .psqlrc file) is > > somehow turning off the reading of any .pgpass file. > > I can't see how that could be. -X is a psql option, but .pgpass is read > by the libpq library, which has no knowledge of the psql options. Well, -X is one of the few switches common to both commands. Reading .pgpass seems to me, to be some kind of initialization thing. That is all I based me guess on. > Can you check that do.maintenance is being run by the correct user? (Put > the line "user identity = `id`" somewhere near the top.) Also, see if > do.maintenance works when run directly rather than through cron. I already have the output of env in the cron script. According to that, it is postgres that it is being run as. X-Cron-Env: <HOME=/var/lib/postgres> X-Cron-Env: <PATH=/usr/bin:/bin> X-Cron-Env: <LOGNAME=postgres> ... HOME=/var/lib/postgres SHLVL=2 LOGNAME=postgres ... But, we'll check that too. Gord
On Sun, 4 Jul 2004, Tom Lane wrote: > Oliver Elphick <olly@lfix.co.uk> writes: > > On Sun, 2004-07-04 at 20:41, ghaverla@freenet.edmonton.ab.ca wrote: > >> My guess is it is that -X (disallow reading of a .psqlrc file) is > >> somehow turning off the reading of any .pgpass file. > > > I can't see how that could be. > > I don't believe it either --- libpq will always look for a $HOME/.pgpass > file. > > A couple of possibilities that I don't think have been considered yet: > > 1. Old libpq? .pgpass is only supported in 7.3 and later. Even if > your psql is 7.3, I think it might be possible for the dynamic linker > to bind a 7.2 libpq.so to it, if your ldconfig search path is such > that the older libpq.so is found first. Well, I'm as up to date as debian/unstable is. Almost every day I do the apt-get update/upgrade thing. My libpq.so.3 is from 7.4.3-1 in terms of Debian. > 2. Bad permissions on .pgpass? libpq will silently ignore a .pgpass > file it cannot open for reading.. It will not-so-silently ignore > a .pgpass file that has any group or world permissions. I'm not > sure if you are capturing stderr from the psql run; if not, it's > possible the latter case is occurring and we're just not seeing > the bleat... Well, the permissions of .pgpass are 600 owned by postgres in group postgres. The permissions on /var/lib/postgres are 755 with it owned by postgres, in group postgres. And the cron script is telling us the user is postgres, so there should be no problems reading the file. What's next, modify the call to psql in do.maintenance by putting it in a strace? Gord
SELECT from two tables... one to many relationship... can postgresql offer anything unique?
From
"Alan T. Miller"
Date:
I know what I want to do has been done a thousand times before, however I am looking for suggestions on the best way to go about it. I have two tables, one holds products, the other holds photos. There is a one to many relationship between products and photographs. For example... TABLE 1 (products): products.id products.title products.description TABLE 2 (product photos) photos.id photos.id_product photos.filename photos.height photos.width photos.position the 'position' field is used to determine which is the main photo (photos are ordered by their position). IN other words the photo with position 1 would be the main thumbnail. There are two things I would like to be able to get from a SELECT statement. The first is a list of products with the thumbnail info for the product. I immediately see two ways of doing this but am looking for a better solution. SOLUTION 1: select all products, and then loop through that result set and do another select to get the photo information on each iteration of the loop. This would be slow, and create unnecesary overhead on the database. I do not think this is a good solution. SOLUTION 2: select all products and then use subselects to get the photograph information. I do not like this solution either, as there seems to be no clear way to get multiple fields such as filename and height and width etc without many subselects, or perhaps a creative concatenation routine. SOLUTION 3: I am open to suggestions... please! The other main task I would like to accomplish is to be able to select all the product information for a single product and get all the photographs for the product as well. However, aside from using an array to return all photos in that array in the select for products, I cannot think of an efficient way to do this without issuing two queries (one to get the product, and one to get the photo). I am very interested to hear how others have tacked simular situations like this. Any help is greatly appreciated. Alan
<ghaverla@freenet.edmonton.ab.ca> writes: > What's next, modify the call to psql in do.maintenance by putting > it in a strace? Great minds think alike ;-) --- I was just gonna suggest that. regards, tom lane
Oliver Elphick <olly@lfix.co.uk> writes: > On Sun, 2004-07-04 at 20:41, ghaverla@freenet.edmonton.ab.ca wrote: >> My guess is it is that -X (disallow reading of a .psqlrc file) is >> somehow turning off the reading of any .pgpass file. > I can't see how that could be. I don't believe it either --- libpq will always look for a $HOME/.pgpass file. A couple of possibilities that I don't think have been considered yet: 1. Old libpq? .pgpass is only supported in 7.3 and later. Even if your psql is 7.3, I think it might be possible for the dynamic linker to bind a 7.2 libpq.so to it, if your ldconfig search path is such that the older libpq.so is found first. 2. Bad permissions on .pgpass? libpq will silently ignore a .pgpass file it cannot open for reading. It will not-so-silently ignore a .pgpass file that has any group or world permissions. I'm not sure if you are capturing stderr from the psql run; if not, it's possible the latter case is occurring and we're just not seeing the bleat... regards, tom lane
Re: SELECT from two tables... one to many relationship... can postgresql offer anything unique?
From
Oliver Fromme
Date:
Alan T. Miller wrote: > [...] > TABLE 1 (products): > products.id > products.title > products.description > > TABLE 2 (product photos) > photos.id > photos.id_product > photos.filename > photos.height > photos.width > photos.position > > the 'position' field is used to determine which is the main photo (photos > are ordered by their position). IN other words the photo with position 1 > would be the main thumbnail. > > There are two things I would like to be able to get from a SELECT statement. > The first is a list of products with the thumbnail info for the product. I > immediately see two ways of doing this but am looking for a better solution. You basically want to JOIN the tables, I think. This is from the top of my head in the early morning without any coffee, so please excuse any stupid syntax errors ... SELECT pr.*, ph.* FROM products AS pr, photos AS ph WHERE pr.id = ph.id_product and ph.position = 1; That will give you one row for each product, along with the photo information for the first thumbnail. > [...] > The other main task I would like to accomplish is to be able to select all > the product information for a single product and get all the photographs for > the product as well. SELECT pr.*, ph.* FROM products AS pr, photos AS ph WHERE pr.id = ph.id_product and pr.id == your_desired_product_id; That will give you one row for each photo which belongs to your_desired_product_id, as well as that product's data (which the same for every row). Please note that the may well be more efficient ways to perform those joins (and I'm sure someone else will mention them). But basically the above should work. Best regards Oliver -- Oliver Fromme, secnetix GmbH & Co KG, Oettingenstr. 2, 80538 München Any opinions expressed in this message may be personal to the author and may not necessarily reflect the opinions of secnetix in any way. "That's what I love about GUIs: They make simple tasks easier, and complex tasks impossible." -- John William Chambless
Well, I captured a run using strace. 301 lines I believe, I've edited it down to 120 or so. It's included below. At about line 59, you can see where it opens the .pgpass file. It opens a socket (to postmaster?) and after that it tries to control the tty so that it can be put into noecho mode to manually retrieve a password. After some other stuff, it again opens .pgpass and talks to a socket, this time failing with the message cron sends me, and the strace ends. I guess the password isn't correct, although it does work for a shell as we seen earlier. The line from .pgpass in the strace is listed as 34 characters long, but only 32 characters are displayed. One character has been chopped from the password, so that and the newline make 34 which is what it is supposed to be. Gord ----- Removed: brk(), mmap(), munmap(), select(), getrlimit(), setrlimit(), rt_sigaction(), rt_sigprocmask(), non-postgres system library stuff (ld.so, libc, etc. except pam, krb), _sysctl(), ioctl(), duplicate getpid(), gettimeofday(), duplicate stat64(), duplicate open(), Plaintext password changed to PASSWORD execve("/usr/bin/psql", ["/usr/bin/psql", "-q", "-X", "-t", "-d", "template1", "-P", "border=0", "-c", " SELECT datname FROM pg_database WHERE datallowconn ORDER BY datname"], [/* 11 vars */]) = 0 uname({sys="Linux", node="newmain.materia", ...}) = 0 vfork() = 17976 --- SIGCHLD (Child exited) @ 0 (0) --- fstat64(3, {st_mode=S_IFIFO|0600, st_size=0, ...}) = 0 read(3, "PGLIB=/usr/lib/postgresql/lib\nPW"..., 4096) = 162 read(3, "", 4096) = 0 close(3) = 0 waitpid(17976, [{WIFEXITED(s) && WEXITSTATUS(s) == 0}], 0) = 17976 execve("/usr/lib/postgresql/bin/psql", ["/usr/lib/postgresql/bin/psql", "-q", "-X", "-t", "-P", "border=0", "-c", "\nSELECT datname\n FROM "..., "-d", "template1"], [/* 13 vars */]) = 0 uname({sys="Linux", node="newmain.materia", ...}) = 0 open("/usr/lib/libpq.so.3", O_RDONLY) = 3 read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0pX\0\000"..., 512) = 512 fstat64(3, {st_mode=S_IFREG|0644, st_size=127184, ...}) = 0 close(3) = 0 open("/lib/libpam.so.0", O_RDONLY) = 3 read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0`\25\0\000"..., 512) = 512 fstat64(3, {st_mode=S_IFREG|0644, st_size=30360, ...}) = 0 close(3) = 0 open("/usr/lib/libkrb5.so.3", O_RDONLY) = 3 read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0p\372\0"..., 512) = 512 fstat64(3, {st_mode=S_IFREG|0644, st_size=425800, ...}) = 0 close(3) = 0 open("/usr/lib/libk5crypto.so.3", O_RDONLY) = 3 read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\2205\0"..., 512) = 512 fstat64(3, {st_mode=S_IFREG|0644, st_size=138664, ...}) = 0 close(3) = 0 getpid() = 17973 getuid32() = 31 (postgres) stat64("/etc/krb5.conf", {st_mode=S_IFREG|0644, st_size=2602, ...}) = 0 open("/etc/krb5.conf", O_RDONLY) = 3 access("/etc/krb5.conf", W_OK) = -1 EACCES (Permission denied) fstat64(3, {st_mode=S_IFREG|0644, st_size=2602, ...}) = 0 read(3, "[libdefaults]\n\tdefault_realm = M"..., 4096) = 2602 read(3, "", 4096) = 0 close(3) = 0 stat64("/usr/etc/krb5.conf", 0xbfffd46c) = -1 ENOENT (No such file or directory) open("/dev/urandom", O_RDONLY) = 3 fstat64(3, {st_mode=S_IFCHR|0444, st_rdev=makedev(1, 9), ...}) = 0 read(3, "\271\254c\353\240\200\371$\310\377g\356lM\27\3316\337\243"..., 20) = 20 close(3) = 0 open("/tmp/krb5cc_31", O_RDONLY) = -1 ENOENT (No such file or directory) geteuid32() = 31 (postgres) socket(PF_FILE, SOCK_STREAM, 0) = 3 connect(3, {sa_family=AF_FILE, path="/var/run/.nscd_socket"}, 110) = 0 writev(3, [{"\2\0\0\0\1\0\0\0\3\0\0\0", 12}, {"31\0", 3}], 2) = 15 read(3, "\2\0\0\0\1\0\0\0\t\0\0\0\2\0\0\0\37\0\0\0 \0\0\0\t\0\0"..., 36) = 36 read(3, "postgres\0x\0postgres\0/var/lib/pos"..., 46) = 46 close(3) = 0 stat64("/var/lib/postgres/.pgpass", {st_mode=S_IFREG|0600, st_size=34, ...}) = 0 open("/var/lib/postgres/.pgpass", O_RDONLY|O_LARGEFILE) = 3 fstat64(3, {st_mode=S_IFREG|0600, st_size=34, ...}) = 0 read(3, "localhost:5432:*:postgres:PASSWORD"..., 4096) = 34 read(3, "", 4096) = 0 close(3) = 0 socket(PF_FILE, SOCK_STREAM, 0) = 3 fcntl64(3, F_SETFL, O_RDONLY|O_NONBLOCK) = 0 connect(3, {sa_family=AF_FILE, path="/var/run/postgresql/.s.PGSQL.5432"}, 110) = 0 getsockopt(3, SOL_SOCKET, SO_ERROR, [0], [4]) = 0 getsockname(3, {sa_family=AF_FILE, path=@}, [2]) = 0 poll([{fd=3, events=POLLOUT|POLLERR, revents=POLLOUT}], 1, -1) = 1 send(3, "\0\0\0*\0\3\0\0user\0postgres\0database\0t"..., 42, 0) = 42 poll([{fd=3, events=POLLIN|POLLERR, revents=POLLIN}], 1, -1) = 1 recv(3, "R\0\0\0\10\0\0\0\3", 16384, 0) = 9 close(3) = 0 open("/dev/tty", O_RDONLY|O_LARGEFILE) = -1 ENXIO (No such device or address) open("/dev/tty", O_WRONLY|O_CREAT|O_TRUNC|O_LARGEFILE, 0666) = -1 ENXIO (No such device or address) ioctl(0, SNDCTL_TMR_TIMEBASE or TCGETS, 0xbffffaf0) = -1 EINVAL (Invalid argument) ioctl(0, SNDCTL_TMR_TIMEBASE or TCGETS, 0xbffffac0) = -1 EINVAL (Invalid argument) ioctl(0, SNDCTL_TMR_CONTINUE or TCSETSF, {B50 -opost isig icanon -echo ...}) = -1 EINVAL (Invalid argument) write(2, "Password: ", 10) = 10 fstat64(0, {st_mode=S_IFIFO|0600, st_size=0, ...}) = 0 read(0, "", 4096) = 0 ioctl(0, SNDCTL_TMR_TIMEBASE or TCGETS, 0xbffffac0) = -1 EINVAL (Invalid argument) ioctl(0, SNDCTL_TMR_CONTINUE or TCSETSF, {B50 -opost isig icanon echo ...}) = -1 EINVAL (Invalid argument) write(2, "\n", 1) = 1 getuid32() = 31 (postgres) stat64("/etc/krb5.conf", {st_mode=S_IFREG|0644, st_size=2602, ...}) = 0 open("/etc/krb5.conf", O_RDONLY) = 3 access("/etc/krb5.conf", W_OK) = -1 EACCES (Permission denied) fstat64(3, {st_mode=S_IFREG|0644, st_size=2602, ...}) = 0 read(3, "[libdefaults]\n\tdefault_realm = M"..., 4096) = 2602 read(3, "", 4096) = 0 close(3) = 0 stat64("/usr/etc/krb5.conf", 0xbfffd46c) = -1 ENOENT (No such file or directory) open("/dev/urandom", O_RDONLY) = 3 fstat64(3, {st_mode=S_IFCHR|0444, st_rdev=makedev(1, 9), ...}) = 0 read(3, "\3120\303\255\376\17\326\377\347\356\265\215\22\35\355"..., 20) = 20 close(3) = 0 open("/tmp/krb5cc_31", O_RDONLY) = -1 ENOENT (No such file or directory) open("/tmp/krb5cc_31", O_RDONLY) = -1 ENOENT (No such file or directory) geteuid32() = 31 (postgres) socket(PF_FILE, SOCK_STREAM, 0) = 3 connect(3, {sa_family=AF_FILE, path="/var/run/.nscd_socket"}, 110) = 0 writev(3, [{"\2\0\0\0\1\0\0\0\3\0\0\0", 12}, {"31\0", 3}], 2) = 15 read(3, "\2\0\0\0\1\0\0\0\t\0\0\0\2\0\0\0\37\0\0\0 \0\0\0\t\0\0"..., 36) = 36 read(3, "postgres\0x\0postgres\0/var/lib/pos"..., 46) = 46 close(3) = 0 stat64("/var/lib/postgres/.pgpass", {st_mode=S_IFREG|0600, st_size=34, ...}) = 0 open("/var/lib/postgres/.pgpass", O_RDONLY|O_LARGEFILE) = 3 fstat64(3, {st_mode=S_IFREG|0600, st_size=34, ...}) = 0 read(3, "localhost:5432:*:postgres:PASSWORD"..., 4096) = 34 read(3, "", 4096) = 0 close(3) = 0 socket(PF_FILE, SOCK_STREAM, 0) = 3 fcntl64(3, F_SETFL, O_RDONLY|O_NONBLOCK) = 0 connect(3, {sa_family=AF_FILE, path="/var/run/postgresql/.s.PGSQL.5432"}, 110) = 0 getsockopt(3, SOL_SOCKET, SO_ERROR, [0], [4]) = 0 getsockname(3, {sa_family=AF_FILE, path=@}, [2]) = 0 poll([{fd=3, events=POLLOUT|POLLERR, revents=POLLOUT}], 1, -1) = 1 send(3, "\0\0\0*\0\3\0\0user\0postgres\0database\0t"..., 42, 0) = 42 poll([{fd=3, events=POLLIN|POLLERR, revents=POLLIN}], 1, -1) = 1 recv(3, "R\0\0\0\10\0\0\0\3", 16384, 0) = 9 write(2, "psql: fe_sendauth: no password s"..., 40) = 40 close(3) = 0 exit_group(2) = ?
<ghaverla@freenet.edmonton.ab.ca> writes: > Well, I captured a run using strace. 301 lines I believe, I've > edited it down to 120 or so. It's included below. Hm. There is nothing apparently wrong in the trace (though it's too bad we cannot see whether the password from the file is actually sent in the subsequent connection-request packet ... does your strace have an option to print more of the data involved in a send call?) > geteuid32() = 31 (postgres) > socket(PF_FILE, SOCK_STREAM, 0) = 3 > connect(3, {sa_family=AF_FILE, path="/var/run/.nscd_socket"}, 110) > = 0 > writev(3, [{"\2\0\0\0\1\0\0\0\3\0\0\0", 12}, {"31\0", 3}], 2) = 15 > read(3, "\2\0\0\0\1\0\0\0\t\0\0\0\2\0\0\0\37\0\0\0 > \0\0\0\t\0\0"..., 36) = 36 > read(3, "postgres\0x\0postgres\0/var/lib/pos"..., 46) = 46 > close(3) = 0 This part of the trace implies that you're using nscd for authentication, which raises a few red flags for me since I've seen bugs in nscd before. Can you try turning off nscd and see if the behavior changes? regards, tom lane