Thread: Several tags around PostgreSQL 7.1 broken
I have now managed to investigate why some conversions of the PostgreSQL CVS
repository to other formats are having trouble or are failing. Here, I am
looking at git-cvsimport in particular.
The problem appears to be that several tags around the time of PostgreSQL 7.1
are broken or inconsistent. For instance, here is a piece of output of
cvsps:
WARNING: Invalid PatchSet 9441, Tag REL7_1: src/pl/plpgsql/src/mklang.sql.in:1.6=after,
src/pl/plpgsql/src/INSTALL:1.2=before. Treated as 'before'
It turns out that src/pl/plpgsql/src/mklang.sql.in:1.6 and
src/pl/plpgsql/src/INSTALL:1.2 are from the same commit ("PatchSet"), as
determined by a common log message and timestamp, but the REL7_1 tag is on
src/pl/plpgsql/src/mklang.sql.in:1.5 and src/pl/plpgsql/src/INSTALL:1.2. So
a part of the commit is before the tag and part of it is after the tag.
(The commit in question was to remove mklang.sql.in and adjust the INSTALL
contents accordingly.)
In fact, if you check out the REL7_1 tag, you get the new INSTALL file but
still the mklang.sql.in. The released postgresql-7.1.tar.gz tarball is
correct.
I guess the cause of this is that the tag was done on a partially updated
checkout.
There are a few dozen inconsistencies like this in the tags REL7_1_BETA,
REL7_1_BETA2, REL7_1_BETA3, REL7_1. As a consequence of this, checkouts of
the tags don't match the respective released tarballs.
Here are more examples:
WARNING: Invalid PatchSet 9297, Tag REL7_1: src/backend/port/hpux/port-protos.h:1.10=after, COPYRIGHT:1.6=before.
Treated as 'before'
WARNING: Invalid PatchSet 8906, Tag REL7_1: doc/src/sgml/populate.sgml:2.4=after,
doc/src/sgml/filelist.sgml:1.3=before. Treated as 'before'
WARNING: Invalid PatchSet 9371, Tag REL7_1: doc/TODO.detail/subquery:1.3=after, doc/TODO:1.366=before. Treated
as 'before'
WARNING: Invalid PatchSet 8815, Tag REL7_1_BETA2: src/include/c.h:1.2=after, contrib/pgcrypto/md5.c:1.2=before.
Treated
as 'before'
etc.
I could get the conversion to run successfully if I remove the mentioned tags.
When I find more time, I'm going to try if I can move/fix the tags instead so
they correspond to the actual releases and the patch sets become consistent.
In the meantime, does anyone have more information about how this came about?
. * Peter Eisentraut <peter_e@gmx.net> [080401 12:01]: > I have now managed to investigate why some conversions of the PostgreSQL CVS > repository to other formats are having trouble or are failing. Here, I am > looking at git-cvsimport in particular. > > The problem appears to be that several tags around the time of PostgreSQL 7.1 > are broken or inconsistent. For instance, here is a piece of output of > cvsps: > > WARNING: Invalid PatchSet 9441, Tag REL7_1: > src/pl/plpgsql/src/mklang.sql.in:1.6=after, > src/pl/plpgsql/src/INSTALL:1.2=before. Treated as 'before' I talked about this here:http://mid.gmane.org/20080220225300.GE16099@yugib.highrise.ca Here's a "quick hack" to cvsps that I use to ignore the problematic tags. Of course, I've stuck with fromcvs for my PostgreSQL git mirror simply because even though I *can* use cvsps on PostgreSQL with that hack, it's still loads slower than fromcvs. Since cvsps/cvsimport is so slow, I didn't pursue making this patch usable, and moving forward with using cvsps/cvsimport. aidan@db1-dapper:~/build/cvsps.git$ git diffdiff --git a/cvsps.c b/cvsps.cindex 981cd78..d436591 100644--- a/cvsps.c+++ b/cvsps.c@@-2184,15 +2184,28 @@ static void parse_sym(CvsFile * file, char * sym) } } +const char* skip_symbols[] =+{+ "REL7_1_BETA",+ "REL7_1_BETA2",+ "REL7_1_BETA3",+ NULL+};+ voidcvs_file_add_symbol(CvsFile * file, const char * rev_str, const char * p_tag_str) { CvsFileRevision * rev; GlobalSymbol* sym; Tag * tag;+ int i; /* get a permanent storage string */ char * tag_str = get_string(p_tag_str); + for (i = 0; skip_symbols[i] != NULL; i++)+ if (strcmp(tag_str, skip_symbols[i]) == 0)+ return;+ debug(DEBUG_STATUS, "adding symbol to file: %s %s->%s", file->filename, tag_str, rev_str); rev = cvs_file_add_revision(file,rev_str); put_hash_object_ex(file->symbols, tag_str, rev, HT_NO_KEYCOPY, NULL, NULL); a. -- Aidan Van Dyk Create like a god, aidan@highrise.ca command like a king, http://www.highrise.ca/ work like a slave.
Peter Eisentraut <peter_e@gmx.net> writes:
> In the meantime, does anyone have more information about how this came about?
Marc's always done both the tagging and the tarball-making, so you'd
have to ask him about that. I believe he's made it more scripted over
the years, so this might reflect a manual foulup that (hopefully) is no
longer possible.
+1 for adjusting the tags in CVS to match what we actually shipped.
regards, tom lane
* Tom Lane <tgl@sss.pgh.pa.us> [080401 14:15]: > Peter Eisentraut <peter_e@gmx.net> writes: > > In the meantime, does anyone have more information about how this came about? > > Marc's always done both the tagging and the tarball-making, so you'd > have to ask him about that. I believe he's made it more scripted over > the years, so this might reflect a manual foulup that (hopefully) is no > longer possible. > > +1 for adjusting the tags in CVS to match what we actually shipped. > > regards, tom lane If somebody's going to be fudging around in $CVSROOT, is it possible to give us all a big warning, and hopefully "pause" the anoncvs/rsync sync stuff for the duration of the "history change", so we (those of us out here working off anoncvs/rsync) get an atomic swap of ancient history? a. -- Aidan Van Dyk Create like a god, aidan@highrise.ca command like a king, http://www.highrise.ca/ work like a slave.
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 - --On Tuesday, April 01, 2008 14:06:09 -0400 Tom Lane <tgl@sss.pgh.pa.us> wrote: > Peter Eisentraut <peter_e@gmx.net> writes: >> In the meantime, does anyone have more information about how this came about? > > Marc's always done both the tagging and the tarball-making, so you'd > have to ask him about that. I believe he's made it more scripted over > the years, so this might reflect a manual foulup that (hopefully) is no > longer possible. Ya, I'll go with that (considering 7.1 was back in 2001 ... ) ... but, from the way Peter describes it (taging partially checked out code), I'm not 100% how its possible to 'foul up' ... a tag operation is: cvs -q update -APd . cvs -q tag REL7_1 . unless its a sub-tagging, which would have: cvs -q update -rREL7_1_STABLE -Pd . cvs -q tag REL7_1_1 . And since I don't do the update until things are "quiet" (generally when Tom has finished his last commit before release), I'm not sure how I could have gotten a 'partial checkout' ... > +1 for adjusting the tags in CVS to match what we actually shipped. Agreed ... but, stupid question here ... if our tags are wrong in CVS, are the 7.1.x releases themselves wrong too? When I do a release tarball, I run: cvs -q export -rREL7_1_1 pgsql so, if the tags are wrong, then all of those releases are wrong too, since they are based on the tag ... - -- Marc G. Fournier Hub.Org Hosting Solutions S.A. (http://www.hub.org) Email . scrappy@hub.org MSN . scrappy@hub.org Yahoo . yscrappy Skype: hub.org ICQ . 7615664 -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.4 (FreeBSD) iD8DBQFH8tX24QvfyHIvDvMRAndoAJ9KA86BZl21zLb3rie9ynlmDL7BHQCfdtjB VrYLsml4H+ppnXvC26ywKTU= =RWHE -----END PGP SIGNATURE-----
Am Mittwoch, 2. April 2008 schrieb Marc G. Fournier: > Agreed ... but, stupid question here ... if our tags are wrong in CVS, are > the 7.1.x releases themselves wrong too? When I do a release tarball, I > run: > > cvs -q export -rREL7_1_1 pgsql > > so, if the tags are wrong, then all of those releases are wrong too, since > they are based on the tag ... I believe we moved to using cvs export many years after 7.1. Before that, the releases were made straight out of a cvs checkout. With cvs export it is of course nearly impossible to create such a mess. I sporadically tested the correspondence of tarball and tag for some other older releases, and they seem OK. At least to the extent that corresponding tags exist at all ... ;-)
Peter Eisentraut wrote: > Am Mittwoch, 2. April 2008 schrieb Marc G. Fournier: > > Agreed ... but, stupid question here ... if our tags are wrong in CVS, are > > the 7.1.x releases themselves wrong too? When I do a release tarball, I > > run: > > > > cvs -q export -rREL7_1_1 pgsql > > I believe we moved to using cvs export many years after 7.1. Before that, the > releases were made straight out of a cvs checkout. With cvs export it is of > course nearly impossible to create such a mess. Hmm, if we use a CVS export, why do we have, on "make distdir", the business to remove CVS files? -- Alvaro Herrera http://www.CommandPrompt.com/ The PostgreSQL Company - Command Prompt, Inc.
Alvaro Herrera <alvherre@commandprompt.com> writes:
> Peter Eisentraut wrote:
>> I believe we moved to using cvs export many years after 7.1. Before
>> that, the releases were made straight out of a cvs checkout. With
>> cvs export it is of course nearly impossible to create such a mess.
> Hmm, if we use a CVS export, why do we have, on "make distdir", the
> business to remove CVS files?
Leftover from the old way, no doubt.
I have a vague recollection that the rest of core pressed Marc to start
using "cvs export" precisely because we were worried about, or actually
had seen, inconsistencies between the tagging and the tarballs. It
might be that what Peter has noticed was exactly what prompted that,
and we never attempted to clean it up. But it was a long time ago
and I'm not sure about it.
regards, tom lane
Marc G. Fournier wrote: > - --On Tuesday, April 01, 2008 14:06:09 -0400 Tom Lane > <tgl@sss.pgh.pa.us> wrote: > > > Peter Eisentraut <peter_e@gmx.net> writes: > >> In the meantime, does anyone have more information about how this > >> came about? > > > > Marc's always done both the tagging and the tarball-making, so you'd > > have to ask him about that. I believe he's made it more scripted > > over the years, so this might reflect a manual foulup that > > (hopefully) is no longer possible. > > Ya, I'll go with that (considering 7.1 was back in 2001 ... ) ... > but, from the way Peter describes it (taging partially checked out > code), I'm not 100% how its possible to 'foul up' ... a tag operation > is: > > cvs -q update -APd . > cvs -q tag REL7_1 . > > unless its a sub-tagging, which would have: > > cvs -q update -rREL7_1_STABLE -Pd . > cvs -q tag REL7_1_1 . > > And since I don't do the update until things are "quiet" (generally > when Tom has finished his last commit before release), I'm not sure > how I could have gotten a 'partial checkout' ... Could it be that a commit was done while the tag operation was running? Given that neither is an atomic operation in cvs, and it used to be that large repo operations could take quite a long time? //Magnus
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 - --On Wednesday, April 02, 2008 10:10:59 +0200 Magnus Hagander <magnus@hagander.net> wrote: > Could it be that a commit was done while the tag operation was running? > Given that neither is an atomic operation in cvs, and it used to be > that large repo operations could take quite a long time? I doubt it, back in 7.1 days, we had something like 4 committers, I think, and all core members ... very little chance of overlap ... the weird thing is Peter is reporting its *all* on the 7.1 branch ... none of hte others affected ... - -- Marc G. Fournier Hub.Org Hosting Solutions S.A. (http://www.hub.org) Email . scrappy@hub.org MSN . scrappy@hub.org Yahoo . yscrappy Skype: hub.org ICQ . 7615664 -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.4 (FreeBSD) iD8DBQFH8/It4QvfyHIvDvMRAqnKAJ4h/+4elPLe3XEfW9lRU6IfAAiSWgCcCRCd pWs76n1JzLgiOExj6kGytBA= =XVkS -----END PGP SIGNATURE-----
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 - --On Wednesday, April 02, 2008 12:33:24 -0400 Alvaro Herrera <alvherre@commandprompt.com> wrote: > Peter Eisentraut wrote: >> Am Mittwoch, 2. April 2008 schrieb Marc G. Fournier: >> > Agreed ... but, stupid question here ... if our tags are wrong in CVS, are >> > the 7.1.x releases themselves wrong too? When I do a release tarball, I >> > run: >> > >> > cvs -q export -rREL7_1_1 pgsql >> >> I believe we moved to using cvs export many years after 7.1. Before that, >> the releases were made straight out of a cvs checkout. With cvs export it >> is of course nearly impossible to create such a mess. > > Hmm, if we use a CVS export, why do we have, on "make distdir", the > business to remove CVS files? Using export was a relatively recent change ... wasn't a command I knew about at the start, someone (possibly Peter) pointed it out to me and we changed the script ... never thought to change distdir though ... - -- Marc G. Fournier Hub.Org Hosting Solutions S.A. (http://www.hub.org) Email . scrappy@hub.org MSN . scrappy@hub.org Yahoo . yscrappy Skype: hub.org ICQ . 7615664 -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.4 (FreeBSD) iD8DBQFH8/vv4QvfyHIvDvMRAu2NAKDjiCnjJfJ3pXtUX9PFH8vZfSnr5ACfcWdg EiNTuC6bd2PFxlZ1tt8oA68= =VbGW -----END PGP SIGNATURE-----
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 - --On Wednesday, April 02, 2008 18:08:05 +0200 Peter Eisentraut <peter_e@gmx.net> wrote: > I believe we moved to using cvs export many years after 7.1. Before that, > the releases were made straight out of a cvs checkout. With cvs export it > is of course nearly impossible to create such a mess. 'k, you need to explain this one to me ... how does export affect what files are tag'd? export isn't used as part of the tagging process, only to create the tar balls themselves ... - -- Marc G. Fournier Hub.Org Hosting Solutions S.A. (http://www.hub.org) Email . scrappy@hub.org MSN . scrappy@hub.org Yahoo . yscrappy Skype: hub.org ICQ . 7615664 -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.4 (FreeBSD) iD8DBQFH8/w44QvfyHIvDvMRAmXeAKDc7V43uQmP8SdxuLBaIyvjntKlzgCfYF5N Tsn/rXOk2AH9RgjlOO9duKQ= =29dq -----END PGP SIGNATURE-----
"Marc G. Fournier" <scrappy@hub.org> writes:
> - --On Wednesday, April 02, 2008 18:08:05 +0200 Peter Eisentraut
> <peter_e@gmx.net> wrote:
>> I believe we moved to using cvs export many years after 7.1. Before that,
>> the releases were made straight out of a cvs checkout. With cvs export it
>> is of course nearly impossible to create such a mess.
> 'k, you need to explain this one to me ... how does export affect what files
> are tag'd?
It doesn't, of course. What it does do is guarantee that the tarball
matches the tag that has already been laid down in CVS.
But there must have been more to it than that. Peter is reporting
that the tag is on mutually inconsistent versions of some files;
which means that the problem was with the tagging operation more than
with the tarball-making.
regards, tom lane
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 - --On Wednesday, April 02, 2008 17:49:49 -0400 Tom Lane <tgl@sss.pgh.pa.us> wrote: > It doesn't, of course. What it does do is guarantee that the tarball > matches the tag that has already been laid down in CVS. 'k, that was my thought, so using export vs update to create the tarbal is irrelevant to this discussion ... > But there must have been more to it than that. Peter is reporting > that the tag is on mutually inconsistent versions of some files; > which means that the problem was with the tagging operation more than > with the tarball-making. Has anyone actually checked the tarballs themselves? If the tag's are wrong, then doesn't it follow that the tarballs themselves are all wrong too? - -- Marc G. Fournier Hub.Org Hosting Solutions S.A. (http://www.hub.org) Email . scrappy@hub.org MSN . scrappy@hub.org Yahoo . yscrappy Skype: hub.org ICQ . 7615664 -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.4 (FreeBSD) iD8DBQFH9BQr4QvfyHIvDvMRAjC9AJ9qBRom7aU7LWmZGnhfOFtbwv7zRQCgxPqx qv5B4ffClv4RRXc2FVg6LpI= =zjTy -----END PGP SIGNATURE-----
I had originally sent this mail in April, but it appears to have been blocked because the attachment was too large. While we are in the process of dealing with CVS-Git conversion, we should also apply this fixup to make the CVS repository more sane. Could someone with write access to the repository files apply this patch? Am Dienstag, 1. April 2008 schrieb Peter Eisentraut: > The problem appears to be that several tags around the time of PostgreSQL > 7.1 are broken or inconsistent. Here is a patch to bring the REL7_1 tag in line with the released tarball. The cvsps errors with the REL7_1_BETA* tags appear to be of a different nature. I haven't been able to fix those yet.
*PLEASE* I'll cry again, loudly... If somebody's doing this type of operation on the CVS replo, please warn is (I'll consider this warning) *AND STOP RSYNC/ANONCVS ACCESS* around the operation.... *PLEASE PLEASE PLEASE* Note that this change doesn't even "fix" the broken CVS history tags; the REL7_1_BETA[23] tags are the ones that are actually inconsistent. Note that I'm pausing the CVS->GIT conversion until I hear that the repository is stable again... a. * Peter Eisentraut <peter_e@gmx.net> [081229 09:50]: > I had originally sent this mail in April, but it appears to have been blocked > because the attachment was too large. > > While we are in the process of dealing with CVS-Git conversion, we should also > apply this fixup to make the CVS repository more sane. > > Could someone with write access to the repository files apply this patch? > > > Am Dienstag, 1. April 2008 schrieb Peter Eisentraut: > > The problem appears to be that several tags around the time of PostgreSQL > > 7.1 are broken or inconsistent. > > Here is a patch to bring the REL7_1 tag in line with the released tarball. > > The cvsps errors with the REL7_1_BETA* tags appear to be of a different > nature. I haven't been able to fix those yet. > > -- > Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) > To make changes to your subscription: > http://www.postgresql.org/mailpref/pgsql-hackers -- Aidan Van Dyk Create like a god, aidan@highrise.ca command like a king, http://www.highrise.ca/ work like a slave.
Has this gone anywhere? Is the CVS repo safe yet? a. > * Peter Eisentraut <peter_e@gmx.net> [081229 09:50]: > > I had originally sent this mail in April, but it appears to have been blocked > > because the attachment was too large. > > > > While we are in the process of dealing with CVS-Git conversion, we should also > > apply this fixup to make the CVS repository more sane. > > > > Could someone with write access to the repository files apply this patch? > > > > > > Am Dienstag, 1. April 2008 schrieb Peter Eisentraut: > > > The problem appears to be that several tags around the time of PostgreSQL > > > 7.1 are broken or inconsistent. > > > > Here is a patch to bring the REL7_1 tag in line with the released tarball. > > > > The cvsps errors with the REL7_1_BETA* tags appear to be of a different > > nature. I haven't been able to fix those yet. > > > > > > -- > > Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) > > To make changes to your subscription: > > http://www.postgresql.org/mailpref/pgsql-hackers -- Aidan Van Dyk Create like a god, aidan@highrise.ca command like a king, http://www.highrise.ca/ work like a slave.
On Friday 02 January 2009 23:33:34 Aidan Van Dyk wrote: > Has this gone anywhere? Is the CVS repo safe yet? Nothing has been done about this. > > a. > > > * Peter Eisentraut <peter_e@gmx.net> [081229 09:50]: > > > I had originally sent this mail in April, but it appears to have been > > > blocked because the attachment was too large. > > > > > > While we are in the process of dealing with CVS-Git conversion, we > > > should also apply this fixup to make the CVS repository more sane. > > > > > > Could someone with write access to the repository files apply this > > > patch? > > > > > > Am Dienstag, 1. April 2008 schrieb Peter Eisentraut: > > > > The problem appears to be that several tags around the time of > > > > PostgreSQL 7.1 are broken or inconsistent. > > > > > > Here is a patch to bring the REL7_1 tag in line with the released > > > tarball. > > > > > > The cvsps errors with the REL7_1_BETA* tags appear to be of a different > > > nature. I haven't been able to fix those yet. > > > > > > > > > > > > -- > > > Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) > > > To make changes to your subscription: > > > http://www.postgresql.org/mailpref/pgsql-hackers
* Peter Eisentraut <peter_e@gmx.net> [090102 18:33]: > On Friday 02 January 2009 23:33:34 Aidan Van Dyk wrote: > > Has this gone anywhere? Is the CVS repo safe yet? > > Nothing has been done about this. So, what's the concensus. Are we(you,everybody?) going to leave the CVS repo alone, or is someone going to muck around in it? If someone's planning to muck around in it, are we all going to get a day or two of warning? a. -- Aidan Van Dyk Create like a god, aidan@highrise.ca command like a king, http://www.highrise.ca/ work like a slave.