Thread: CVS corruption/mistagging?
Looking into recent buildfarm failures on the 7.4 branch: http://www.pgbuildfarm.org/cgi-bin/show_status.pl It looks like parts of the CVS repository have been mistagged as belonging to REL7_4_STABLE or have been corrupted somehow: http://developer.postgresql.org/cvsweb.cgi/pgsql/contrib/btree_gist/btree_bit.c?sortby=date;only_with_tag=REL7_4_STABLE I'm not sure what's going on here, but something has gone bad. Kris Jurka
Kris Jurka <books@ejurka.com> writes: > It looks like parts of the CVS repository have been mistagged as belonging > to REL7_4_STABLE or have been corrupted somehow: > http://developer.postgresql.org/cvsweb.cgi/pgsql/contrib/btree_gist/btree_bit.c?sortby=date;only_with_tag=REL7_4_STABLE Hmm ... btree_bit.c shouldn't be in 7.4 at all. I can't help thinking this has something to do with the recent CVS server move. Magnus, any thoughts? regards, tom lane
I wrote: > Kris Jurka <books@ejurka.com> writes: >> It looks like parts of the CVS repository have been mistagged as belonging >> to REL7_4_STABLE or have been corrupted somehow: >> http://developer.postgresql.org/cvsweb.cgi/pgsql/contrib/btree_gist/btree_bit.c?sortby=date;only_with_tag=REL7_4_STABLE > Hmm ... btree_bit.c shouldn't be in 7.4 at all. I did a fresh checkout of the 7.4 branch and diff'd against my local copy, and it seems clear that every file that was not in 7.4 at all has had its HEAD version tagged as REL7_4_STABLE. The files that did exist then are all right. That's throughout the whole tree, not just in contrib/btree_gist. I have no idea how you make CVS do that, but I'm sure there is some magic one-liner for it. Checking the files in contrib/btree_gist on the CVS server gives a pretty good fix on who changed it and when: > ls -la total 466 drwxrwxr-x 6 scrappy dev 1024 Aug 14 22:30 . drwxrwxr-x 90 scrappy dev 2048 Aug 14 22:27 .. drwxrwxr-x 2 scrappy dev 512 Apr 20 15:16 Attic -r--r--r-- 1 tgl dev 9555 Jun 26 22:05 Makefile,v -r--r--r-- 1 258 dev 5870 Apr 20 16:19 README.btree_gist,v -r--r--r-- 1 meskes dev 12052 Aug 12 09:50 btree_bit.c,v -r--r--r-- 1 meskes dev 9921 Aug 12 09:50 btree_bytea.c,v -r--r--r-- 1 meskes dev 10246 Aug 12 09:50 btree_cash.c,v -r--r--r-- 1 meskes dev 11433 Aug 12 09:50 btree_date.c,v -r--r--r-- 1 meskes dev 10230 Aug 12 09:50 btree_float4.c,v -r--r--r-- 1 meskes dev 10093 Aug 12 09:50 btree_float8.c,v -r--r--r-- 1 meskes dev 33935 Aug 12 09:50 btree_gist.c,v -r--r--r-- 1 258 dev 4325 Apr 20 15:16 btree_gist.h,v -r--r--r-- 1 258 dev 58744 Apr 20 16:19 btree_gist.sql.in,v -r--r--r-- 1 meskes dev 14736 Aug 12 09:50 btree_inet.c,v -r--r--r-- 1 meskes dev 10253 Aug 12 09:50 btree_int2.c,v -r--r--r-- 1 meskes dev 10240 Aug 12 09:50 btree_int4.c,v -r--r--r-- 1 meskes dev 10254 Aug 12 09:50 btree_int8.c,v -r--r--r-- 1 meskes dev 18111 Aug 12 09:50 btree_interval.c,v -r--r--r-- 1 meskes dev 12025 Aug 12 09:50 btree_macaddr.c,v -r--r--r-- 1 meskes dev 14671 Aug 12 09:50 btree_numeric.c,v -r--r--r-- 1 meskes dev 9796 Aug 12 09:50 btree_oid.c,v -r--r--r-- 1 meskes dev 18247 Aug 12 09:50 btree_text.c,v -r--r--r-- 1 meskes dev 26180 Aug 12 09:50 btree_time.c,v -r--r--r-- 1 258 dev 29712 Apr 20 15:16 btree_ts.c,v -r--r--r-- 1 meskes dev 17588 Aug 12 09:50 btree_utils_num.c,v -r--r--r-- 1 meskes dev 8959 Aug 12 09:50 btree_utils_num.h,v -r--r--r-- 1 meskes dev 48824 Aug 12 09:50 btree_utils_var.c,v -r--r--r-- 1 meskes dev 7099 Aug 12 09:50 btree_utils_var.h,v drwxrwxr-x 3 scrappy dev 1024 Aug 14 22:27 data drwxrwxr-x 3 scrappy dev 1024 Aug 14 22:27 expected drwxrwxr-x 3 scrappy dev 1024 Aug 14 22:27 sql -r--r--r-- 1 meskes dev 10021 Aug 12 09:50 uninstall_btree_gist.sql,v Michael, want to fess up to what you did? In the meantime, though, it's not quite clear why this would lead to a buildfarm failure --- it should just mean a lot of extraneous files appearing in a fresh checkout. (Looks a bit harder ... Oh, it looks like btree_gist has some files that used to be autogenerated and are now in CVS, so the bogusly new versions from CVS are suppressing the desired generation from the old btree_num.c file.) regards, tom lane
Tom Lane wrote: > In the meantime, though, it's not quite clear why this would lead to > a buildfarm failure --- it should just mean a lot of extraneous files > appearing in a fresh checkout. (Looks a bit harder ... Oh, it looks > like btree_gist has some files that used to be autogenerated and are > now in CVS, so the bogusly new versions from CVS are suppressing the > desired generation from the old btree_num.c file.) > > > Looking at the Committers mail, it looks like there have only been two very small commits since Michael's series of commits around 12 to 13.5 hours ago, and before that nothing since around 28 hours ago. Do we have a backup snapshot of the repo taken during that time that we can roll back to? That might be simpler than doing surgery on the repo. cheers andrew
Andrew Dunstan <andrew@dunslane.net> writes:
> Looking at the Committers mail, it looks like there have only been two
> very small commits since Michael's series of commits around 12 to 13.5
> hours ago, and before that nothing since around 28 hours ago. Do we have
> a backup snapshot of the repo taken during that time that we can roll
> back to? That might be simpler than doing surgery on the repo.
It looks to me like the mistake was unrelated to any commit, and was
made at Aug 12 09:50 UTC. (Aside from the file-timestamp evidence,
caracara's build failure at 2007-08-12 213845 shows that the problem
is more than 2 days old.) I count seven commits of my own and several
of Michael's since then. If I'm right that a simple "cvs rtag" will fix
it, I'd rather do that than try to reapply patches.
regards, tom lane
I wrote:
> I did a fresh checkout of the 7.4 branch and diff'd against my local
> copy, and it seems clear that every file that was not in 7.4 at all has
> had its HEAD version tagged as REL7_4_STABLE. The files that did exist
> then are all right. That's throughout the whole tree, not just in
> contrib/btree_gist.
As for getting out of it: it looks like the misapplied tags are revision
tags not branch tags, eg
RCS file: /cvsroot/pgsql/contrib/btree_gist/btree_bit.c,v
Working file: btree_bit.c
head: 1.7
branch:
locks: strict
access list:
symbolic names: REL7_4_STABLE: 1.7 <--- wrong REL8_0_13: 1.2 REL8_1_9: 1.6 REL8_2_4:
1.6 REL8_0_12: 1.2 REL8_1_8: 1.6 REL8_2_3: 1.6 REL8_0_11: 1.2 REL8_1_7: 1.6
REL8_2_2:1.6 REL8_0_10: 1.2 REL8_1_6: 1.6 REL8_2_1: 1.6 REL8_2_STABLE: 1.6.0.4
which I think means that there's a one-liner way out of it. The CVS
manual quoth:
To delete a tag, specify the `-d' option to either `cvs tag' or `cvs rtag'. For example:
cvs rtag -d rel-0-4 tc
deletes the non-branch tag `rel-0-4' from the module `tc'. In the event that branch tags are encountered within the
repositorywith the given name, a warning message will be issued and the branch tag will not be deleted. If you are
absolutelycertain you know what you are doing, the `-B' option may be specified to allow deletion of branch tags. In
thatcase, any non-branch tags encountered will trigger warnings and will not be deleted.
So I think that "cvs rtag -d REL7_4_STABLE pgsql" will fix it.
I'd like someone to double-check that though. Also maybe we should back
up the repository first?
regards, tom lane
Tom Lane wrote: > > So I think that "cvs rtag -d REL7_4_STABLE pgsql" will fix it. > I'd like someone to double-check that though. I will test on a copy of my mirror. > Also maybe we should back > up the repository first? > Amen. cheers andrew
Andrew Dunstan wrote: > > > Tom Lane wrote: >> >> So I think that "cvs rtag -d REL7_4_STABLE pgsql" will fix it. >> I'd like someone to double-check that though. > > I will test on a copy of my mirror. > I copied the mirror, did a checkout from it, ran the command above in the checked out version, then remove my checkout and checked out REL7_4_STABLE and did a diff against my copy from the postgres repo that hasn't been updated for some time. There were no significant differences, and no extra files. So I'd say your strategy looks good - backup and remove the phony tag. I'd also say we should probably be logging tag commands in taginfo. Presumably we mere mortal committers should not be doing any tagging whatsoever, and tags should only be applied in preparation for releases. Anyway, if we actually logged it we'd catch anything untoward early rather than possibly days later. cheers andrew
Andrew Dunstan <andrew@dunslane.net> writes:
> So I'd say your strategy looks good - backup and remove the phony tag.
> I'd also say we should probably be logging tag commands in taginfo.
> Presumably we mere mortal committers should not be doing any tagging
> whatsoever, and tags should only be applied in preparation for releases.
+1 ... we should at least log such commands, and maybe disallow to anyone
except Marc's "pgsql" account. Particularly since they don't get
reported in pgsql-committers messages (or is that something we
can/should change?)
Meanwhile, is there anyone around who can either (1) tar up the
repository directory tree as root, or (2) confirm that a tarball
made by a non-root committer is sufficient?
regards, tom lane
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 - --On Tuesday, August 14, 2007 22:20:16 -0400 Tom Lane <tgl@sss.pgh.pa.us> wrote: > +1 ... we should at least log such commands, and maybe disallow to anyone > except Marc's "pgsql" account. Particularly since they don't get > reported in pgsql-committers messages (or is that something we > can/should change?) > > Meanwhile, is there anyone around who can either (1) tar up the > repository directory tree as root, or (2) confirm that a tarball > made by a non-root committer is sufficient? I've just finished tar'ng up the cvs repository as root ... I believe one as a non-root would be sufficient also, since it *should* retain file ownership / permissions with the -p option ... cvs# ls -lt cvsroot.aug14_2007.tar.bz2 - -rw-r--r-- 1 root wheel 62555651 Aug 15 02:52 cvsroot.aug14_2007.tar.bz2 cvs# du -sk cvsroot 414406 cvsroot - ---- Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email . scrappy@hub.org MSN . scrappy@hub.org Yahoo . yscrappy Skype: hub.org ICQ . 7615664 -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.4 (FreeBSD) iD8DBQFGwmq14QvfyHIvDvMRAsyFAJ9AND3JTDmOMR48Hl3Fdse2lVCjSwCglsu1 Pxj1LPPPx311rpY2XFj+Kvo= =Gz6E -----END PGP SIGNATURE-----
"Marc G. Fournier" <scrappy@hub.org> writes:
> On Tuesday, August 14, 2007 22:20:16 -0400 Tom Lane <tgl@sss.pgh.pa.us>
> wrote:
>> Meanwhile, is there anyone around who can either (1) tar up the
>> repository directory tree as root, or (2) confirm that a tarball
>> made by a non-root committer is sufficient?
> I've just finished tar'ng up the cvs repository as root ...
Great --- launching cvs rtag command now.
> I believe one as a
> non-root would be sufficient also, since it *should* retain file ownership /
> permissions with the -p option ...
That's what I thought too, but a little extra paranoia doesn't hurt.
regards, tom lane
I wrote:
> Great --- launching cvs rtag command now.
Done, and I got a plausible-looking mix of messages like
cvs rtag: Not removing branch tag `REL7_4_STABLE' from `/cvsroot/pgsql/src/tutorial/funcs_new.c,v'.
and a fresh checkout of REL7_4_STABLE now matches what I had locally.
So I think we walked away from that one.
TODO items:
* restrict, or at least log, cvs tag/rtag commands. Maybe report them
to pgsql-committers.
* I notice that the repository contains some files owned by
no-longer-existent accounts, eg 258 in contrib/btree_gist:
> ls -la
total 460
drwxrwxr-x 6 scrappy dev 1024 Aug 15 03:00 .
drwxrwxr-x 90 scrappy dev 2048 Aug 15 03:00 ..
drwxrwxr-x 2 scrappy dev 512 Apr 20 15:16 Attic
-r--r--r-- 1 tgl dev 9555 Jun 26 22:05 Makefile,v
-r--r--r-- 1 258 dev 5870 Apr 20 16:19 README.btree_gist,v
-r--r--r-- 1 tgl dev 12033 Aug 15 03:00 btree_bit.c,v
-r--r--r-- 1 tgl dev 9902 Aug 15 03:00 btree_bytea.c,v
-r--r--r-- 1 tgl dev 10227 Aug 15 03:00 btree_cash.c,v
-r--r--r-- 1 tgl dev 11414 Aug 15 03:00 btree_date.c,v
-r--r--r-- 1 tgl dev 10211 Aug 15 03:00 btree_float4.c,v
-r--r--r-- 1 tgl dev 10074 Aug 15 03:00 btree_float8.c,v
-r--r--r-- 1 tgl dev 33915 Aug 15 03:00 btree_gist.c,v
-r--r--r-- 1 258 dev 4325 Apr 20 15:16 btree_gist.h,v
-r--r--r-- 1 258 dev 58744 Apr 20 16:19 btree_gist.sql.in,v
-r--r--r-- 1 tgl dev 14717 Aug 15 03:00 btree_inet.c,v
-r--r--r-- 1 tgl dev 10234 Aug 15 03:00 btree_int2.c,v
-r--r--r-- 1 tgl dev 10221 Aug 15 03:00 btree_int4.c,v
-r--r--r-- 1 tgl dev 10235 Aug 15 03:00 btree_int8.c,v
I suppose this is due to Magnus' cleanup of unused committers accounts.
That was a good thing, but for security's sake these files ought to be
chown'd to some existing committer's account.
regards, tom lane
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 - --On Tuesday, August 14, 2007 23:26:03 -0400 Tom Lane <tgl@sss.pgh.pa.us> wrote: > I wrote: >> Great --- launching cvs rtag command now. > > Done, and I got a plausible-looking mix of messages like > cvs rtag: Not removing branch tag `REL7_4_STABLE' from > `/cvsroot/pgsql/src/tutorial/funcs_new.c,v'. and a fresh checkout of > REL7_4_STABLE now matches what I had locally. So I think we walked away from > that one. > > TODO items: > > * restrict, or at least log, cvs tag/rtag commands. Maybe report them > to pgsql-committers. working on it ... have code for it, I've just spent so much time with CVSROOT = /cvsroot, I have to relearn remote CVS :) >> ls -la > total 460 > drwxrwxr-x 6 scrappy dev 1024 Aug 15 03:00 . > drwxrwxr-x 90 scrappy dev 2048 Aug 15 03:00 .. > drwxrwxr-x 2 scrappy dev 512 Apr 20 15:16 Attic > -r--r--r-- 1 tgl dev 9555 Jun 26 22:05 Makefile,v > -r--r--r-- 1 258 dev 5870 Apr 20 16:19 README.btree_gist,v > -r--r--r-- 1 tgl dev 12033 Aug 15 03:00 btree_bit.c,v > -r--r--r-- 1 tgl dev 9902 Aug 15 03:00 btree_bytea.c,v > -r--r--r-- 1 tgl dev 10227 Aug 15 03:00 btree_cash.c,v > -r--r--r-- 1 tgl dev 11414 Aug 15 03:00 btree_date.c,v > -r--r--r-- 1 tgl dev 10211 Aug 15 03:00 btree_float4.c,v > -r--r--r-- 1 tgl dev 10074 Aug 15 03:00 btree_float8.c,v > -r--r--r-- 1 tgl dev 33915 Aug 15 03:00 btree_gist.c,v > -r--r--r-- 1 258 dev 4325 Apr 20 15:16 btree_gist.h,v > -r--r--r-- 1 258 dev 58744 Apr 20 16:19 btree_gist.sql.in,v > -r--r--r-- 1 tgl dev 14717 Aug 15 03:00 btree_inet.c,v > -r--r--r-- 1 tgl dev 10234 Aug 15 03:00 btree_int2.c,v > -r--r--r-- 1 tgl dev 10221 Aug 15 03:00 btree_int4.c,v > -r--r--r-- 1 tgl dev 10235 Aug 15 03:00 btree_int8.c,v > > I suppose this is due to Magnus' cleanup of unused committers accounts. > That was a good thing, but for security's sake these files ought to be > chown'd to some existing committer's account. I can do a quick chown -R scrappy on the whole repository ... ok? - ---- Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email . scrappy@hub.org MSN . scrappy@hub.org Yahoo . yscrappy Skype: hub.org ICQ . 7615664 -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.4 (FreeBSD) iD8DBQFGwnf34QvfyHIvDvMRAnGTAJ43gELVZ5Q5SpV/jppR9rODB/ILaACeKCEN DE16fnY/PmKlrvdHFaJeIKI= =VCFu -----END PGP SIGNATURE-----
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 - --On Tuesday, August 14, 2007 23:26:03 -0400 Tom Lane <tgl@sss.pgh.pa.us> wrote: > * restrict, or at least log, cvs tag/rtag commands. Maybe report them > to pgsql-committers. It should be done ... if you try and create a tag, it should generate an error message ... not sure of the best way to test that it actually works, since, if it doesn't work, of course, you'll add on another tag, but do you want to try and create a 'fake tag' and then delete it afterwards? - ---- Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email . scrappy@hub.org MSN . scrappy@hub.org Yahoo . yscrappy Skype: hub.org ICQ . 7615664 -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.4 (FreeBSD) iD8DBQFGwnoO4QvfyHIvDvMRArHSAJ4qYuexUGn6Ff3/PkHU/DjJBlCQnwCgxyy9 Qc852wMVRRMYdwoD3+p7X3Y= =FCD9 -----END PGP SIGNATURE-----
"Marc G. Fournier" <scrappy@hub.org> writes:
>> That was a good thing, but for security's sake these files ought to be
>> chown'd to some existing committer's account.
> I can do a quick chown -R scrappy on the whole repository ... ok?
Seems close enough, but please keep that tarball around for awhile just
in case we need to look at the prior state.
regards, tom lane
"Marc G. Fournier" <scrappy@hub.org> writes:
> It should be done ... if you try and create a tag, it should generate
> an error message ...
Uh, nope:
$ cvs tag fooey README
T README
$ cvs log README | more
RCS file: /cvsroot/pgsql/src/backend/optimizer/README,v
Working file: README
head: 1.39
branch:
locks: strict
access list:
symbolic names: fooey: 1.39 REL7_3_19: 1.25 REL7_4_17: 1.29 REL8_0_13: 1.30 REL8_1_9:
1.33 REL8_2_4: 1.35.2.1 REL8_0_12: 1.30 REL8_1_8: 1.33 REL8_2_3: 1.35 REL7_3_18: 1.25
REL7_4_16: 1.29 REL8_0_11: 1.30 REL8_1_7: 1.33 REL8_2_2: 1.35 REL8_0_10: 1.30
cvs [log aborted]: received broken pipe signal
$ cvs tag -d fooey README
D README
$
regards, tom lane
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 'k, let me play with it ... - --On Wednesday, August 15, 2007 00:13:11 -0400 Tom Lane <tgl@sss.pgh.pa.us> wrote: > "Marc G. Fournier" <scrappy@hub.org> writes: >> It should be done ... if you try and create a tag, it should generate >> an error message ... > > Uh, nope: > > $ cvs tag fooey README > T README > $ cvs log README | more > > RCS file: /cvsroot/pgsql/src/backend/optimizer/README,v > Working file: README > head: 1.39 > branch: > locks: strict > access list: > symbolic names: > fooey: 1.39 > REL7_3_19: 1.25 > REL7_4_17: 1.29 > REL8_0_13: 1.30 > REL8_1_9: 1.33 > REL8_2_4: 1.35.2.1 > REL8_0_12: 1.30 > REL8_1_8: 1.33 > REL8_2_3: 1.35 > REL7_3_18: 1.25 > REL7_4_16: 1.29 > REL8_0_11: 1.30 > REL8_1_7: 1.33 > REL8_2_2: 1.35 > REL8_0_10: 1.30 > cvs [log aborted]: received broken pipe signal > $ cvs tag -d fooey README > D README > $ > > regards, tom lane > > ---------------------------(end of broadcast)--------------------------- > TIP 7: You can help support the PostgreSQL project by donating at > > http://www.postgresql.org/about/donate - ---- Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email . scrappy@hub.org MSN . scrappy@hub.org Yahoo . yscrappy Skype: hub.org ICQ . 7615664 -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.4 (FreeBSD) iD8DBQFGwn9q4QvfyHIvDvMRAuqOAJwPcIcsfQ74wBwdjBR478C72nr+bwCfd8PY EeH2sh+biTfoFbTF2KxqYVU= =iMwZ -----END PGP SIGNATURE-----
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
One better (and done now):
find . -nouser -exec chown scrappy {} \;
so, all files that had valid users owning them stay with those ...
- --On Wednesday, August 15, 2007 00:08:07 -0400 Tom Lane <tgl@sss.pgh.pa.us>
wrote:
> "Marc G. Fournier" <scrappy@hub.org> writes:
>>> That was a good thing, but for security's sake these files ought to be
>>> chown'd to some existing committer's account.
>
>> I can do a quick chown -R scrappy on the whole repository ... ok?
>
> Seems close enough, but please keep that tarball around for awhile just
> in case we need to look at the prior state.
>
> regards, tom lane
- ----
Marc G. Fournier Hub.Org Networking Services (http://www.hub.org)
Email . scrappy@hub.org MSN . scrappy@hub.org
Yahoo . yscrappy Skype: hub.org ICQ . 7615664
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.4 (FreeBSD)
iD8DBQFGwonj4QvfyHIvDvMRAhNBAJ9b6Y1o4eICLrtuWncRIqN8g07XggCgnnc8
ttJoOScGbFEY/y3qP3MZYWY=
=RUO5
-----END PGP SIGNATURE-----
"Marc G. Fournier" <scrappy@hub.org> writes:
> One better (and done now):
> find . -nouser -exec chown scrappy {} \;
[ eyeballs repository ... ] Check, that looks great from here.
regards, tom lane
Tom Lane wrote: > In the meantime, though, it's not quite clear why this would lead to > a buildfarm failure --- it should just mean a lot of extraneous files > appearing in a fresh checkout. (Looks a bit harder ... Oh, it looks > like btree_gist has some files that used to be autogenerated and are > now in CVS, so the bogusly new versions from CVS are suppressing the > desired generation from the old btree_num.c file.) Funny enoughy, that's exactly the problem I saw on win32 when I rsynced from a windows box and tried a local CVS checkout :-) Once I rsynced to a local linux instead and checked out from there, things worked again :) //Magnus
Tom Lane wrote: > So I think that "cvs rtag -d REL7_4_STABLE pgsql" will fix it. > I'd like someone to double-check that though. Also maybe we should back > up the repository first? Just for your info: all VMs on tribble, which includes cvs, are backed up at 02:30 every day, CEST (that's 00:30 UTC right now, but it follows the local time on tribble so it'll be 23:30 UTC when austra switches later this year. Perhaps we should switch the machine to GMT, but that's a different discussion) For something like this it's good to run a special backup before that of course, but it's probably good to know when the backups run anyway... //Magnus
Tom Lane wrote: > Andrew Dunstan <andrew@dunslane.net> writes: >> So I'd say your strategy looks good - backup and remove the phony tag. >> I'd also say we should probably be logging tag commands in taginfo. >> Presumably we mere mortal committers should not be doing any tagging >> whatsoever, and tags should only be applied in preparation for releases. > > +1 ... we should at least log such commands, and maybe disallow to anyone > except Marc's "pgsql" account. Particularly since they don't get > reported in pgsql-committers messages (or is that something we > can/should change?) > I think you can, by sticking a command in the taginfo command. It's the same script that Marc is using to try to restrict who can do it - once he has that working, it should be as simple as adding a mail command to the end of that script. And +1 on doing it. //Magnus
Magnus Hagander <magnus@hagander.net> writes:
> Tom Lane wrote:
>> I'd like someone to double-check that though. Also maybe we should back
>> up the repository first?
> Just for your info: all VMs on tribble, which includes cvs, are backed
> up at 02:30 every day, CEST
Good, but the salient followup questions to that are (1) backed up to
where exactly?, and (2) how many days' past backups could we get at,
if we had to?
regards, tom lane
Tom Lane wrote: > Magnus Hagander <magnus@hagander.net> writes: >> Tom Lane wrote: >>> I'd like someone to double-check that though. Also maybe we should back >>> up the repository first? > >> Just for your info: all VMs on tribble, which includes cvs, are backed >> up at 02:30 every day, CEST > > Good, but the salient followup questions to that are (1) backed up to > where exactly?, and (2) how many days' past backups could we get at, > if we had to? 1) The entire VM, with "dump" 2) We have today, yesterday and one weekly (copies the daily on monday, if I read it right) They are dumped to a NFS share on this schedule. That NFS share is dumped to tape by systems at Conova - I'll let Stefan fill in the details about that. //Magnus
Magnus Hagander <magnus@hagander.net> writes:
> Tom Lane wrote:
>> Good, but the salient followup questions to that are (1) backed up to
>> where exactly?, and (2) how many days' past backups could we get at,
>> if we had to?
> They are dumped to a NFS share on this schedule. That NFS share is
> dumped to tape by systems at Conova - I'll let Stefan fill in the
> details about that.
That's good as far as it goes, but seeing that PG is a worldwide
organization now, I wonder whether our primary CVS shouldn't have
backups on several continents. Pardon my paranoia ... but our
collective arses have been saved by offsite backups at least once
already ...
regards, tom lane
Tom Lane wrote: > Magnus Hagander <magnus@hagander.net> writes: >> Tom Lane wrote: >>> Good, but the salient followup questions to that are (1) backed up to >>> where exactly?, and (2) how many days' past backups could we get at, >>> if we had to? > >> They are dumped to a NFS share on this schedule. That NFS share is >> dumped to tape by systems at Conova - I'll let Stefan fill in the >> details about that. > > That's good as far as it goes, but seeing that PG is a worldwide > organization now, I wonder whether our primary CVS shouldn't have > backups on several continents. Pardon my paranoia ... but our > collective arses have been saved by offsite backups at least once > already ... The CVS repository is already synced almost-live back to both svr1 in panama and the anoncvs box in the US. It will be synced to the new VM marc is setting up as well. I assume these machines also have some kind of backup - Marc will have to fill in on that. Oh, and the plan is to have the entire VM synced up between austria and panama machine once / day again once Marc has fixed the issues with that box. //Magnus
Tom Lane wrote: > Magnus Hagander <magnus@hagander.net> writes: > > Tom Lane wrote: > >> Good, but the salient followup questions to that are (1) backed up to > >> where exactly?, and (2) how many days' past backups could we get at, > >> if we had to? > > > They are dumped to a NFS share on this schedule. That NFS share is > > dumped to tape by systems at Conova - I'll let Stefan fill in the > > details about that. > > That's good as far as it goes, but seeing that PG is a worldwide > organization now, I wonder whether our primary CVS shouldn't have > backups on several continents. Pardon my paranoia ... but our > collective arses have been saved by offsite backups at least once > already ... We already have that -- there's plenty of people who have rsync'ed copies of the repository. Now, I don't back it up, so if someone makes a mistake my copy will faithfully follow it. -- Alvaro Herrera http://www.CommandPrompt.com/ The PostgreSQL Company - Command Prompt, Inc.
On 8/15/07, Tom Lane <tgl@sss.pgh.pa.us> wrote: > I wrote: > > Kris Jurka <books@ejurka.com> writes: > >> It looks like parts of the CVS repository have been mistagged as belonging > >> to REL7_4_STABLE or have been corrupted somehow: > I have no idea how you make CVS do that, but I'm > sure there is some magic one-liner for it. Something like "cvs tag" without -F on the HEAD? -- marko
Magnus Hagander wrote: > Tom Lane wrote: >> Magnus Hagander <magnus@hagander.net> writes: >>> Tom Lane wrote: >>>> Good, but the salient followup questions to that are (1) backed up to >>>> where exactly?, and (2) how many days' past backups could we get at, >>>> if we had to? >>> They are dumped to a NFS share on this schedule. That NFS share is >>> dumped to tape by systems at Conova - I'll let Stefan fill in the >>> details about that. >> That's good as far as it goes, but seeing that PG is a worldwide >> organization now, I wonder whether our primary CVS shouldn't have >> backups on several continents. Pardon my paranoia ... but our >> collective arses have been saved by offsite backups at least once >> already ... > > The CVS repository is already synced almost-live back to both svr1 in > panama and the anoncvs box in the US. It will be synced to the new VM > marc is setting up as well. I assume these machines also have some kind > of backup - Marc will have to fill in on that. > > Oh, and the plan is to have the entire VM synced up between austria and > panama machine once / day again once Marc has fixed the issues with that > box. yeah that has been my understanding as well though I'm not sure if marc has set up the sync of the full VM already. Stefan
Magnus Hagander wrote: > Tom Lane wrote: >> Magnus Hagander <magnus@hagander.net> writes: >>> Tom Lane wrote: >>>> I'd like someone to double-check that though. Also maybe we should back >>>> up the repository first? >>> Just for your info: all VMs on tribble, which includes cvs, are backed >>> up at 02:30 every day, CEST >> Good, but the salient followup questions to that are (1) backed up to >> where exactly?, and (2) how many days' past backups could we get at, >> if we had to? > > 1) The entire VM, with "dump" > 2) We have today, yesterday and one weekly (copies the daily on monday, > if I read it right) > > They are dumped to a NFS share on this schedule. That NFS share is > dumped to tape by systems at Conova - I'll let Stefan fill in the > details about that. that's exactly what happens on the backup side here - but maybe I missed an earlier mail - is something broken on tribble or on the new CVS VM that requires a restore from backup ? Stefan
Stefan Kaltenbrunner wrote: > Magnus Hagander wrote: >> Tom Lane wrote: >>> Magnus Hagander <magnus@hagander.net> writes: >>>> Tom Lane wrote: >>>>> I'd like someone to double-check that though. Also maybe we should back >>>>> up the repository first? >>>> Just for your info: all VMs on tribble, which includes cvs, are backed >>>> up at 02:30 every day, CEST >>> Good, but the salient followup questions to that are (1) backed up to >>> where exactly?, and (2) how many days' past backups could we get at, >>> if we had to? >> 1) The entire VM, with "dump" >> 2) We have today, yesterday and one weekly (copies the daily on monday, >> if I read it right) >> >> They are dumped to a NFS share on this schedule. That NFS share is >> dumped to tape by systems at Conova - I'll let Stefan fill in the >> details about that. > > that's exactly what happens on the backup side here - but maybe I missed > an earlier mail - is something broken on tribble or on the new CVS VM > that requires a restore from backup ? erm -ENOTENOUGHCOFFEE Stefan
Am Mittwoch, 15. August 2007 04:20 schrieb Tom Lane: > we should at least log such commands, and maybe disallow to anyone > except Marc's "pgsql" account. I don't think we should disallow it. Or otherwise we might one day be stuck if we need to release while some specific person is on vacation. I never understood why tagging uses a special account anyway. It should be done as the person doing the tagging. -- Peter Eisentraut http://developer.postgresql.org/~petere/
Tom Lane wrote: > Magnus Hagander <magnus@hagander.net> writes: > >> Tom Lane wrote: >> >>> Good, but the salient followup questions to that are (1) backed up to >>> where exactly?, and (2) how many days' past backups could we get at, >>> if we had to? >>> > > >> They are dumped to a NFS share on this schedule. That NFS share is >> dumped to tape by systems at Conova - I'll let Stefan fill in the >> details about that. >> > > That's good as far as it goes, but seeing that PG is a worldwide > organization now, I wonder whether our primary CVS shouldn't have > backups on several continents. Pardon my paranoia ... but our > collective arses have been saved by offsite backups at least once > already ... > > > Yes, I think we could improve on that. Have we considered more sophisticated solutions that provide incremental backup on a more frequent basis? I'd be inclined to use Bacula or similar (and it uses Postgres for its metadata store :-) ). Ideally I think we'd like to be able fairly easily and quickly to roll the repo (or some portion of it) back to a fairly arbitrary and fairly precise (say within an hour or two) point in recent time. Meanwhile, those of us who rsync the entire repo could easily make rolling backup copies. Arguably this might be better done from a repo made using --numeric-ids. Tarred and compressed it's a shade under 90 Mb, which isn't huge. If people do this at various times of the day we'd get pretty good coverage :-) cheers andrew
Andrew Dunstan wrote: > > > Tom Lane wrote: >> Magnus Hagander <magnus@hagander.net> writes: >> >>> Tom Lane wrote: >>> >>>> Good, but the salient followup questions to that are (1) backed up to >>>> where exactly?, and (2) how many days' past backups could we get at, >>>> if we had to? >>>> >> >> >>> They are dumped to a NFS share on this schedule. That NFS share is >>> dumped to tape by systems at Conova - I'll let Stefan fill in the >>> details about that. >>> >> >> That's good as far as it goes, but seeing that PG is a worldwide >> organization now, I wonder whether our primary CVS shouldn't have >> backups on several continents. Pardon my paranoia ... but our >> collective arses have been saved by offsite backups at least once >> already ... >> >> >> > > Yes, I think we could improve on that. Have we considered more > sophisticated solutions that provide incremental backup on a more > frequent basis? I'd be inclined to use Bacula or similar (and it uses > Postgres for its metadata store :-) ). Ideally I think we'd like to be > able fairly easily and quickly to roll the repo (or some portion of it) > back to a fairly arbitrary and fairly precise (say within an hour or > two) point in recent time. well yeah - while I do think that something as complex like bacula is probably overkill for our needs we can certainly improve over the current state. One thing to consider is that we actually have two major scenarios to deal with: 1. simple repo corruption (accident,cvs software bug, admin error) this one might require us to restore the repo from an older backup in the case the corruption cannot be repaired easily. For this we already have myriads of copies of the trees out in the wild but i might be a good idea to keep a number snapshots of the main repo on the CVS-VPS itself (either done every few hours or made as part of the push to anoncvs and svr1). This way we could do a very simple "inplace" recovery on the same running VPS instance with fairly low inpact to all the commiters (and depending parts of teh infrastructure) 2. total loss of the main CVS-VPS (extended power failure, hardware error, OS bug, admin error, fire, some other catastropic event) - in this case we will have to fail over to one of the other project datacenters and for this we need to have full regular copies of the whole VM on external hosts. Stefan
Peter Eisentraut wrote: > Am Mittwoch, 15. August 2007 04:20 schrieb Tom Lane: >> we should at least log such commands, and maybe disallow to anyone >> except Marc's "pgsql" account. > > I don't think we should disallow it. Or otherwise we might one day be stuck > if we need to release while some specific person is on vacation. Is this really a problem in practise ? If such a need arises any commiter (or sysadmin) would probably be able to remove that restriction in a few minutes. I think the point here is to prevent such things done by accident(vs. on purpose) which a script like that seems like a fairly simple yet effective solution. Stefan
Stefan Kaltenbrunner <stefan@kaltenbrunner.cc> writes:
> Peter Eisentraut wrote:
>> Am Mittwoch, 15. August 2007 04:20 schrieb Tom Lane:
>>> we should at least log such commands, and maybe disallow to anyone
>>> except Marc's "pgsql" account.
>>
>> I don't think we should disallow it. Or otherwise we might one day be stuck
>> if we need to release while some specific person is on vacation.
> Is this really a problem in practise ?
I'd be happy if such commands got reported to the pgsql-committers
mailing list. The real problem with this mistake is not so much
that it was made as that we had to do detective work to find out.
regards, tom lane
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 - --On Wednesday, August 15, 2007 12:11:35 +0200 Peter Eisentraut <peter_e@gmx.net> wrote: > Am Mittwoch, 15. August 2007 04:20 schrieb Tom Lane: >> we should at least log such commands, and maybe disallow to anyone >> except Marc's "pgsql" account. > > I don't think we should disallow it. Or otherwise we might one day be stuck > if we need to release while some specific person is on vacation. This isn't a big issue ... note that the 'restriction' is easy to remove ... you checkout CVSROOT, modify taginfo and comment out the ALL l ine and check that in ... What this will prevent is an 'accidental tagging' ... you would have to consciously remove the restriction .. but its something anyone could do ... > I never understood why tagging uses a special account anyway. It should be > done as the person doing the tagging. Agreed, I'm going to start doing it as myself from now on ... I'm not even 100% certain *why* I started doing it as pgsql in the first place ... - ---- Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email . scrappy@hub.org MSN . scrappy@hub.org Yahoo . yscrappy Skype: hub.org ICQ . 7615664 -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.4 (FreeBSD) iD8DBQFGwye54QvfyHIvDvMRAiHEAKCHGtXA+r5PM1SoBewMJDo3An7BKACg0//z gM7eZNoTEU4sqwBIHprFK1k= =/Qga -----END PGP SIGNATURE-----
Marc G. Fournier wrote: > > > --On Wednesday, August 15, 2007 12:11:35 +0200 Peter Eisentraut > <peter_e@gmx.net> wrote: > >> Am Mittwoch, 15. August 2007 04:20 schrieb Tom Lane: >>> we should at least log such commands, and maybe disallow to anyone >>> except Marc's "pgsql" account. >> I don't think we should disallow it. Or otherwise we might one day be stuck >> if we need to release while some specific person is on vacation. > > This isn't a big issue ... note that the 'restriction' is easy to remove ... > you checkout CVSROOT, modify taginfo and comment out the ALL l ine and check > that in ... > > What this will prevent is an 'accidental tagging' ... you would have to > consciously remove the restriction .. but its something anyone could do ... > >> I never understood why tagging uses a special account anyway. It should be >> done as the person doing the tagging. > > Agreed, I'm going to start doing it as myself from now on ... I'm not even 100% > certain *why* I started doing it as pgsql in the first place ... If you're doing that, we should probably just delete the pgsql userid from the system? Or at least change it so it doesn't have 'dev' permissions. That way you can't do it wrong in that direction ;-) Seems reasonable? //Magnus
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 - --On Wednesday, August 15, 2007 20:57:14 +0200 Magnus Hagander <magnus@hagander.net> wrote: > Marc G. Fournier wrote: >> >> >> --On Wednesday, August 15, 2007 12:11:35 +0200 Peter Eisentraut >> <peter_e@gmx.net> wrote: >> >>> Am Mittwoch, 15. August 2007 04:20 schrieb Tom Lane: >>>> we should at least log such commands, and maybe disallow to anyone >>>> except Marc's "pgsql" account. >>> I don't think we should disallow it. Or otherwise we might one day be stuck >>> if we need to release while some specific person is on vacation. >> >> This isn't a big issue ... note that the 'restriction' is easy to remove >> ... you checkout CVSROOT, modify taginfo and comment out the ALL l ine and >> check that in ... >> >> What this will prevent is an 'accidental tagging' ... you would have to >> consciously remove the restriction .. but its something anyone could do ... >> >>> I never understood why tagging uses a special account anyway. It should be >>> done as the person doing the tagging. >> >> Agreed, I'm going to start doing it as myself from now on ... I'm not even >> 100% certain *why* I started doing it as pgsql in the first place ... > > If you're doing that, we should probably just delete the pgsql userid > from the system? Or at least change it so it doesn't have 'dev' > permissions. That way you can't do it wrong in that direction ;-) > Seems reasonable? there is no pgsql user *on* cvs.postgresql.org, at least there wasn't when I logged on last night: cvs# grep pgsql /etc/passwd cvs# - ---- Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email . scrappy@hub.org MSN . scrappy@hub.org Yahoo . yscrappy Skype: hub.org ICQ . 7615664 -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.4 (FreeBSD) iD8DBQFGw1AR4QvfyHIvDvMRArsSAJ9WzXyMu7Io6dP8kDnR5HHex1f1gQCg7iDq 3p2RrAzww4dCDPVCyMozAnM= =L+/M -----END PGP SIGNATURE-----
Marc G. Fournier wrote: >> If you're doing that, we should probably just delete the pgsql userid >> from the system? Or at least change it so it doesn't have 'dev' >> permissions. That way you can't do it wrong in that direction ;-) >> Seems reasonable? > > there is no pgsql user *on* cvs.postgresql.org, at least there wasn't when I > logged on last night: > > cvs# grep pgsql /etc/passwd > cvs# Ah, seems I was faster than I expected myself :-) Problem solved then. //Magnus