Thread: gitweb is no longer a real-time view
It used to be that http://git.postgresql.org/gitweb/?p=postgresql.git would update within a minute or so after somebody pushed something. This seems to have been broken for the last week or so; not only do commits not show up for a random amount of time, but the displayed age of the last commit is noticeably wrong. I recall somebody muttering about having installed caching on the web server. If so, could we get that fixed or reverted? regards, tom lane
Tom, * Tom Lane (tgl@sss.pgh.pa.us) wrote: > It used to be that http://git.postgresql.org/gitweb/?p=postgresql.git > would update within a minute or so after somebody pushed something. The cronjob which syncs the between the git master and git.postgresql.org appears to run once every 15 minutes (unchanged since mid-2011). Given that commits simply aren't that frequent, I do wonder if we could set that up as a trigger or similar instead of polling for updates. > This seems to have been broken for the last week or so; not only do > commits not show up for a random amount of time, but the displayed > age of the last commit is noticeably wrong. The recent change was the addition of varnish to do cacheing in front of git.postgresql.org to reduce load on the system. The cacheing was set to 30m/1h, which does seem a bit high. I've lowered it down to 15m/30m. That should help some, but obviously isn't going to give within-a-minute response time. I'll talk with the other admins and see what we can do to improve it further. Thanks, Stephen
Stephen Frost <sfrost@snowman.net> writes: > * Tom Lane (tgl@sss.pgh.pa.us) wrote: >> It used to be that http://git.postgresql.org/gitweb/?p=postgresql.git >> would update within a minute or so after somebody pushed something. > The cronjob which syncs the between the git master and > git.postgresql.org appears to run once every 15 minutes (unchanged since > mid-2011). cron job? I was under the impression there was some sort of push operation driven by a commit trigger. The web site has certainly updated nearly immediately for as long as we've been using git. Until this week, that is. regards, tom lane
* Tom Lane (tgl@sss.pgh.pa.us) wrote:
> cron job? I was under the impression there was some sort of push
> operation driven by a commit trigger. The web site has certainly
> updated nearly immediately for as long as we've been using git.
> Until this week, that is.
Curiously, there's two cron jobs, apparently. There's a 'push' one and
then another, independent, 'pull' one. I'll assume they're actually
doing different things, but I wonder if the pull isn't just a
hold-over.. In any case, the push-to-anon, which I hadn't seen
initially (looking at the pull side instead of the push side), does run
once a minute, though it looks like there's a hook mechanism which
would allow us to trigger the webserver to do a pull when a commit
happens and would still be better than a once-a-minute cronjob.
Even with that, however, the concern raised was that the gitweb perl
script is quite expensive to run for every request, hence the reason for
doing the cacheing. I've lowered the varnish cacheing to a 5m ttl and
15m grace and I'll keep an eye on it.
Thanks,
Stephen
On Mon, Mar 4, 2013 at 3:33 AM, Stephen Frost <sfrost@snowman.net> wrote: > * Tom Lane (tgl@sss.pgh.pa.us) wrote: >> cron job? I was under the impression there was some sort of push >> operation driven by a commit trigger. The web site has certainly >> updated nearly immediately for as long as we've been using git. >> Until this week, that is. > > Curiously, there's two cron jobs, apparently. There's a 'push' one and > then another, independent, 'pull' one. I'll assume they're actually > doing different things, but I wonder if the pull isn't just a > hold-over.. In any case, the push-to-anon, which I hadn't seen No, the pull one doesn't do the main repository - that one pulls "third party" repositories in as mirrors, such as e.g. Bucardo. That one was, btw, broken for several months and nobody noticed - so that's clearly a less important one :) > initially (looking at the pull side instead of the push side), does run > once a minute, though it looks like there's a hook mechanism which > would allow us to trigger the webserver to do a pull when a commit > happens and would still be better than a once-a-minute cronjob. The one that deals with the main repo mirroring runs once per minute. The commit trigger only drops a file in the filesystem so the cronjob knows there is something to do. So that we don't risk holding up the actual commit in case there is a problem with the anonymous mirror server. > Even with that, however, the concern raised was that the gitweb perl > script is quite expensive to run for every request, hence the reason for > doing the cacheing. I've lowered the varnish cacheing to a 5m ttl and > 15m grace and I'll keep an eye on it. That's fixing the wrong problem. The cache is supposed to be automatically purged whenever the push happens, to make sure that the data never gets more than maybe a second stale. So clearly something in that is not working - I didn't specifically test it with the postgres.git repository, but I tested it with a couple of other ones where it worked fine. But maybe we did something special with the main one... --Magnus HaganderMe: http://www.hagander.net/Work: http://www.redpill-linpro.com/
On Mon, Mar 4, 2013 at 8:16 AM, Magnus Hagander <magnus@hagander.net> wrote: > On Mon, Mar 4, 2013 at 3:33 AM, Stephen Frost <sfrost@snowman.net> wrote: >> * Tom Lane (tgl@sss.pgh.pa.us) wrote: >>> cron job? I was under the impression there was some sort of push >>> operation driven by a commit trigger. The web site has certainly >>> updated nearly immediately for as long as we've been using git. >>> Until this week, that is. >> >> Curiously, there's two cron jobs, apparently. There's a 'push' one and >> then another, independent, 'pull' one. I'll assume they're actually >> doing different things, but I wonder if the pull isn't just a >> hold-over.. In any case, the push-to-anon, which I hadn't seen > > No, the pull one doesn't do the main repository - that one pulls > "third party" repositories in as mirrors, such as e.g. Bucardo. That > one was, btw, broken for several months and nobody noticed - so that's > clearly a less important one :) > >> initially (looking at the pull side instead of the push side), does run >> once a minute, though it looks like there's a hook mechanism which >> would allow us to trigger the webserver to do a pull when a commit >> happens and would still be better than a once-a-minute cronjob. > > The one that deals with the main repo mirroring runs once per minute. > The commit trigger only drops a file in the filesystem so the cronjob > knows there is something to do. So that we don't risk holding up the > actual commit in case there is a problem with the anonymous mirror > server. > > >> Even with that, however, the concern raised was that the gitweb perl >> script is quite expensive to run for every request, hence the reason for >> doing the cacheing. I've lowered the varnish cacheing to a 5m ttl and >> 15m grace and I'll keep an eye on it. > > That's fixing the wrong problem. The cache is supposed to be > automatically purged whenever the push happens, to make sure that the > data never gets more than maybe a second stale. So clearly something > in that is not working - I didn't specifically test it with the > postgres.git repository, but I tested it with a couple of other ones > where it worked fine. But maybe we did something special with the main > one... Actually, looking closer, I'm seeing a failure when it actually tries the push: To ssh://git@git.postgresql.org/postgresql.git! [rejected] master -> master (non-fast-forward) So the problem might have nothing at all to do with the cacheing. AFAICT, the three missing commits are materialized views, accidentally committed .orig file and \l support. But. The *anonymous* repository also has: bc61878682051678ade5f59da7bfd90ab72ce13b Fix map_sql_value_to_xml_value() to treat domains like their base types. This patch is *not* in the master repository, it's only in anonymous. (The object is in the repository, but it's not part of any branch) How the hell did *that* happen? The master repo has: commit 5db5974c692b0fc68e7608dd85a6b4e6173a0f28 Author: Peter Eisentraut <peter_e@gmx.net> psql: Let \l accept a pattern commit d63977eea3ab18fdec05e370b633d10b9fd20179 Author: Kevin Grittner <kgrittn@postgresql.org> Remove accidentally-committed .orig file. commit 3bf3ab8c563699138be02f9dc305b7b77a724307 Author: Kevin Grittner <kgrittn@postgresql.org> Add a materialized view relations. commit b15a6da29217b14f02895af1d9271e84415a91ae Author: Tom Lane <tgl@sss.pgh.pa.us> Get rid of any toast table when converting a table to a view. And the anonymous one has: commit bc61878682051678ade5f59da7bfd90ab72ce13b Author: Tom Lane <tgl@sss.pgh.pa.us> Fix map_sql_value_to_xml_value() to treat domains like their base types. commit 3bf3ab8c563699138be02f9dc305b7b77a724307 Author: Kevin Grittner <kgrittn@postgresql.org> Add a materialized view relations. commit b15a6da29217b14f02895af1d9271e84415a91ae Author: Tom Lane <tgl@sss.pgh.pa.us> Get rid of any toast table when converting a table to a view. Does anybody have an explanation for that? Did someone do a force-push on the master repository, overwriting some old history? We don't explicitly forbid this on the master repo, since we expect committers to know how things work.. Maybe we need to do that, and manually turn it off in case someone actually *needs* to do a non fast forward push? But either way, it would be good to actually know how tihs happened... Copying this one to Kevin and Peter too, since they seem to be the other people who are involved in any of these pushes. Seems like we have a problem much worse than just the web... --Magnus HaganderMe: http://www.hagander.net/Work: http://www.redpill-linpro.com/
On 2013-03-04 08:32:35 +0100, Magnus Hagander wrote: > On Mon, Mar 4, 2013 at 8:16 AM, Magnus Hagander <magnus@hagander.net> wrote: > > On Mon, Mar 4, 2013 at 3:33 AM, Stephen Frost <sfrost@snowman.net> wrote: > >> * Tom Lane (tgl@sss.pgh.pa.us) wrote: > >>> cron job? I was under the impression there was some sort of push > >>> operation driven by a commit trigger. The web site has certainly > >>> updated nearly immediately for as long as we've been using git. > >>> Until this week, that is. > >> > >> Curiously, there's two cron jobs, apparently. There's a 'push' one and > >> then another, independent, 'pull' one. I'll assume they're actually > >> doing different things, but I wonder if the pull isn't just a > >> hold-over.. In any case, the push-to-anon, which I hadn't seen > > > > No, the pull one doesn't do the main repository - that one pulls > > "third party" repositories in as mirrors, such as e.g. Bucardo. That > > one was, btw, broken for several months and nobody noticed - so that's > > clearly a less important one :) > > > >> initially (looking at the pull side instead of the push side), does run > >> once a minute, though it looks like there's a hook mechanism which > >> would allow us to trigger the webserver to do a pull when a commit > >> happens and would still be better than a once-a-minute cronjob. > > > > The one that deals with the main repo mirroring runs once per minute. > > The commit trigger only drops a file in the filesystem so the cronjob > > knows there is something to do. So that we don't risk holding up the > > actual commit in case there is a problem with the anonymous mirror > > server. > > > > > >> Even with that, however, the concern raised was that the gitweb perl > >> script is quite expensive to run for every request, hence the reason for > >> doing the cacheing. I've lowered the varnish cacheing to a 5m ttl and > >> 15m grace and I'll keep an eye on it. > > > > That's fixing the wrong problem. The cache is supposed to be > > automatically purged whenever the push happens, to make sure that the > > data never gets more than maybe a second stale. So clearly something > > in that is not working - I didn't specifically test it with the > > postgres.git repository, but I tested it with a couple of other ones > > where it worked fine. But maybe we did something special with the main > > one... > > Actually, looking closer, I'm seeing a failure when it actually tries the push: > > To ssh://git@git.postgresql.org/postgresql.git > ! [rejected] master -> master (non-fast-forward) > > > So the problem might have nothing at all to do with the cacheing. > > AFAICT, the three missing commits are materialized views, > accidentally committed .orig file and \l support. > > But. The *anonymous* repository also has: > bc61878682051678ade5f59da7bfd90ab72ce13b Fix > map_sql_value_to_xml_value() to treat domains like their base types. > > This patch is *not* in the master repository, it's only in anonymous. > (The object is in the repository, but it's not part of any branch) > > How the hell did *that* happen? > > The master repo has: > commit 5db5974c692b0fc68e7608dd85a6b4e6173a0f28 > Author: Peter Eisentraut <peter_e@gmx.net> > > psql: Let \l accept a pattern > > commit d63977eea3ab18fdec05e370b633d10b9fd20179 > Author: Kevin Grittner <kgrittn@postgresql.org> > > Remove accidentally-committed .orig file. > > commit 3bf3ab8c563699138be02f9dc305b7b77a724307 > Author: Kevin Grittner <kgrittn@postgresql.org> > > Add a materialized view relations. > > commit b15a6da29217b14f02895af1d9271e84415a91ae > Author: Tom Lane <tgl@sss.pgh.pa.us> > > Get rid of any toast table when converting a table to a view. > > > And the anonymous one has: > commit bc61878682051678ade5f59da7bfd90ab72ce13b > Author: Tom Lane <tgl@sss.pgh.pa.us> > > Fix map_sql_value_to_xml_value() to treat domains like their base types. > > commit 3bf3ab8c563699138be02f9dc305b7b77a724307 > Author: Kevin Grittner <kgrittn@postgresql.org> > > Add a materialized view relations. > > commit b15a6da29217b14f02895af1d9271e84415a91ae > Author: Tom Lane <tgl@sss.pgh.pa.us> > > Get rid of any toast table when converting a table to a view. > > > > Does anybody have an explanation for that? Did someone do a force-push > on the master repository, overwriting some old history? This really looks like Kevin forced a push. Could you show git show --pretty=raw d63977eea3ab18fdec05e370b633d10b9fd20179 git show --pretty=raw bc61878682051678ade5f59da7bfd90ab72ce13b git show --pretty=raw 5db5974c692b0fc68e7608dd85a6b4e6173a0f28 ? I don't have all the commits here... The raw format will explicitly tell which parents are referenced in the commits... > We don't explicitly forbid this on the master repo, since we expect > committers to know how things work.. Maybe we need to do that, and > manually turn it off in case someone actually *needs* to do a non fast > forward push? But either way, it would be good to actually know how > tihs happened... If you allow it the mirroring script should always force a push, otherwise it will stop pushing in those cases. I think the most realistic way to resolve this is to push the current anongit state to master and cherry-pick bc61878682051678ade5f59da7bfd90ab72ce13b ontop it. That way only committers need to deal with rebasing their branches and not everyone else. Greetings, Andres Freund -- Andres Freund http://www.2ndQuadrant.com/PostgreSQL Development, 24x7 Support, Training & Services
Magnus Hagander <magnus@hagander.net> wrote: > Actually, looking closer, I'm seeing a failure when it actually tries the > push: > > To ssh://git@git.postgresql.org/postgresql.git > ! [rejected] master -> master (non-fast-forward) > > > So the problem might have nothing at all to do with the cacheing. > > AFAICT, the three missing commits are materialized views, > accidentally committed .orig file and \l support. > > But. The *anonymous* repository also has: > bc61878682051678ade5f59da7bfd90ab72ce13b Fix > map_sql_value_to_xml_value() to treat domains like their base types. > > This patch is *not* in the master repository, it's only in anonymous. > (The object is in the repository, but it's not part of any branch) > > How the hell did *that* happen? > > The master repo has: > commit 5db5974c692b0fc68e7608dd85a6b4e6173a0f28 > Author: Peter Eisentraut <peter_e@gmx.net> > > psql: Let \l accept a pattern > > commit d63977eea3ab18fdec05e370b633d10b9fd20179 > Author: Kevin Grittner <kgrittn@postgresql.org> > > Remove accidentally-committed .orig file. > > commit 3bf3ab8c563699138be02f9dc305b7b77a724307 > Author: Kevin Grittner <kgrittn@postgresql.org> > > Add a materialized view relations. > > commit b15a6da29217b14f02895af1d9271e84415a91ae > Author: Tom Lane <tgl@sss.pgh.pa.us> > > Get rid of any toast table when converting a table to a view. > > > And the anonymous one has: > commit bc61878682051678ade5f59da7bfd90ab72ce13b > Author: Tom Lane <tgl@sss.pgh.pa.us> > > Fix map_sql_value_to_xml_value() to treat domains like their base types. > > commit 3bf3ab8c563699138be02f9dc305b7b77a724307 > Author: Kevin Grittner <kgrittn@postgresql.org> > > Add a materialized view relations. > > commit b15a6da29217b14f02895af1d9271e84415a91ae > Author: Tom Lane <tgl@sss.pgh.pa.us> > > Get rid of any toast table when converting a table to a view. > > > > Does anybody have an explanation for that? Did someone do a force-push > on the master repository, overwriting some old history? > > We don't explicitly forbid this on the master repo, since we expect > committers to know how things work.. Maybe we need to do that, and > manually turn it off in case someone actually *needs* to do a non fast > forward push? But either way, it would be good to actually know how > tihs happened... > > Copying this one to Kevin and Peter too, since they seem to be the > other people who are involved in any of these pushes. Apologies if I contributed to this. At no point did I use a force option. I'm using this pattern: http://wiki.postgresql.org/wiki/Committing_with_Git#Dependent_Clone_per_Branch.2C_Pushing_and_Pulling_From_a_Local_Repository I still have the shell windows open that I used, so (redacting things which I'm sure are not significant, like git log commands) there is this: You will see a pull command used. Here's what that is: kgrittn@Kevin-Desktop:~/pg/master$ cat ~/bin/pull #!/bin/bash pushd ~/pg/postgresql.git >/dev/null && git fetch && popd >/dev/null && git pull Here's what I have in my (still open) shell windows: kgrittn@Kevin-Desktop:~/pg/master$ git push Counting objects: 301, done. Delta compression using up to 8 threads. Compressing objects: 100% (154/154), done. Writing objects: 100% (155/155), 40.94 KiB, done. Total 155 (delta 147), reused 6 (delta 1) To /home/kgrittn/pg/postgresql 2b78d10..3a21ef1 master -> master kgrittn@Kevin-Desktop:~/pg/master$ cd ../postgresql.git/ kgrittn@Kevin-Desktop:~/pg/postgresql.git$ git push --dry-run To ssh://git@gitmaster.postgresql.org/postgresql.git + b15a6da...3a21ef1 master -> master (forced update) + b15a6da...2b78d10 anonymous/master -> anonymous/master (forced update) kgrittn@Kevin-Desktop:~/pg/postgresql.git$ git fetch remote: Counting objects: 31, done. remote: Compressing objects: 100% (16/16), done. remote: Total 16 (delta 15), reused 0 (delta 0) Unpacking objects: 100% (16/16), done. From ssh://gitmaster.postgresql.org/postgresql + 3a21ef1...b15a6da master -> master (forced update) 2b78d10..b15a6da anonymous/master -> anonymous/master kgrittn@Kevin-Desktop:~/pg/postgresql.git$ git remote --verbose origin ssh://git@gitmaster.postgresql.org/postgresql.git (fetch) origin ssh://git@gitmaster.postgresql.org/postgresql.git (push) kgrittn@Kevin-Desktop:~/pg/postgresql.git$ cd ../master kgrittn@Kevin-Desktop:~/pg/master$ pull remote: Counting objects: 31, done. remote: Compressing objects: 100% (16/16), done. remote: Total 16 (delta 15), reused 0 (delta 0) Unpacking objects: 100% (16/16), done. From /home/kgrittn/pg/postgresql + 3a21ef1...b15a6da master -> origin/master (forced update) First, rewinding head to replay your work on top of it... kgrittn@Kevin-Desktop:~/pg/master$ git status # On branch master nothing to commit (working directory clean) kgrittn@Kevin-Desktop:~/pg/master$ git pull Current branch master is up to date. kgrittn@Kevin-Desktop:~/pg/master$ git status # On branch master nothing to commit (working directory clean) kgrittn@Kevin-Desktop:~/pg/master$ patch -p1 <~/patch/matview-v6.diff [some offsets in rewriteDefine.c which looked OK on review] kgrittn@Kevin-Desktop:~/pg/master$ git add --all kgrittn@Kevin-Desktop:~/pg/master$ git commit [master 3bf3ab8] Add a materialized view relations. 103 files changed, 4240 insertions(+), 438 deletions(-) create mode 100644 doc/src/sgml/ref/alter_materialized_view.sgml create mode 100644 doc/src/sgml/ref/create_materialized_view.sgml create mode 100644 doc/src/sgml/ref/drop_materialized_view.sgml create mode 100644 doc/src/sgml/ref/refresh_materialized_view.sgml create mode 100644 src/backend/commands/matview.c create mode 100644 src/backend/rewrite/rewriteDefine.c.orig create mode 100644 src/include/commands/matview.h create mode 100644 src/test/regress/expected/matview.out create mode 100644 src/test/regress/sql/matview.sql kgrittn@Kevin-Desktop:~/pg/master$ git push Counting objects: 301, done. Delta compression using up to 8 threads. Compressing objects: 100% (154/154), done. Writing objects: 100% (155/155), 40.95 KiB, done. Total 155 (delta 147), reused 5 (delta 1) To /home/kgrittn/pg/postgresql b15a6da..3bf3ab8 master -> master kgrittn@Kevin-Desktop:~/pg/master$ cd ../postgresql.git/ kgrittn@Kevin-Desktop:~/pg/postgresql.git$ git push --dry-run To ssh://git@gitmaster.postgresql.org/postgresql.git b15a6da..3bf3ab8 master -> master kgrittn@Kevin-Desktop:~/pg/postgresql.git$ git push Counting objects: 301, done. Delta compression using up to 8 threads. Compressing objects: 100% (8/8), done. Writing objects: 100% (155/155), 40.95 KiB, done. Total 155 (delta 147), reused 155 (delta 147) To ssh://git@gitmaster.postgresql.org/postgresql.git b15a6da..3bf3ab8 master -> master [on review, discover accidental commit of .orig file] kgrittn@Kevin-Desktop:~/pg/master$ git rm src/backend/rewrite/rewriteDefine.c.orig rm 'src/backend/rewrite/rewriteDefine.c.orig' kgrittn@Kevin-Desktop:~/pg/master$ git commit [master d63977e] Remove accidentally-committed .orig file. 1 file changed, 945 deletions(-) delete mode 100644 src/backend/rewrite/rewriteDefine.c.orig kgrittn@Kevin-Desktop:~/pg/master$ git push Counting objects: 9, done. Delta compression using up to 8 threads. Compressing objects: 100% (5/5), done. Writing objects: 100% (5/5), 440 bytes, done. Total 5 (delta 4), reused 0 (delta 0) Unpacking objects: 100% (5/5), done. To /home/kgrittn/pg/postgresql 3bf3ab8..d63977e master -> master kgrittn@Kevin-Desktop:~/pg/master$ cd ../postgresql.git/ kgrittn@Kevin-Desktop:~/pg/postgresql.git$ git push Counting objects: 15018, done. Delta compression using up to 8 threads. Compressing objects: 100% (4617/4617), done. Writing objects: 100% (13087/13087), 2.31 MiB | 632 KiB/s, done. Total 13087 (delta 11231), reused 9860 (delta 8431) To ssh://git@gitmaster.postgresql.org/postgresql.git + bc61878...d63977e master -> master (forced update) + bc61878...b15a6da anonymous/master -> anonymous/master (forced update) kgrittn@Kevin-Desktop:~/pg/postgresql.git$ cd ../master ; git diff --stat HEAD^ HEAD src/backend/rewrite/rewriteDefine.c.orig | 945 ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 1 file changed, 945 deletions(-) kgrittn@Kevin-Desktop:~/pg/master$ git status # On branch master nothing to commit (working directory clean) kgrittn@Kevin-Desktop:~/pg/master$ pull Current branch master is up to date. -- Kevin Grittner EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company
On Mon, Mar 4, 2013 at 2:11 PM, Kevin Grittner <kgrittn@ymail.com> wrote: > Magnus Hagander <magnus@hagander.net> wrote: > >> Actually, looking closer, I'm seeing a failure when it actually tries the >> push: >> >> To ssh://git@git.postgresql.org/postgresql.git >> ! [rejected] master -> master (non-fast-forward) >> >> >> So the problem might have nothing at all to do with the cacheing. >> >> AFAICT, the three missing commits are materialized views, >> accidentally committed .orig file and \l support. >> >> But. The *anonymous* repository also has: >> bc61878682051678ade5f59da7bfd90ab72ce13b Fix >> map_sql_value_to_xml_value() to treat domains like their base types. >> >> This patch is *not* in the master repository, it's only in anonymous. >> (The object is in the repository, but it's not part of any branch) >> >> How the hell did *that* happen? >> >> The master repo has: >> commit 5db5974c692b0fc68e7608dd85a6b4e6173a0f28 >> Author: Peter Eisentraut <peter_e@gmx.net> >> >> psql: Let \l accept a pattern >> >> commit d63977eea3ab18fdec05e370b633d10b9fd20179 >> Author: Kevin Grittner <kgrittn@postgresql.org> >> >> Remove accidentally-committed .orig file. >> >> commit 3bf3ab8c563699138be02f9dc305b7b77a724307 >> Author: Kevin Grittner <kgrittn@postgresql.org> >> >> Add a materialized view relations. >> >> commit b15a6da29217b14f02895af1d9271e84415a91ae >> Author: Tom Lane <tgl@sss.pgh.pa.us> >> >> Get rid of any toast table when converting a table to a view. >> >> >> And the anonymous one has: >> commit bc61878682051678ade5f59da7bfd90ab72ce13b >> Author: Tom Lane <tgl@sss.pgh.pa.us> >> >> Fix map_sql_value_to_xml_value() to treat domains like their base types. >> >> commit 3bf3ab8c563699138be02f9dc305b7b77a724307 >> Author: Kevin Grittner <kgrittn@postgresql.org> >> >> Add a materialized view relations. >> >> commit b15a6da29217b14f02895af1d9271e84415a91ae >> Author: Tom Lane <tgl@sss.pgh.pa.us> >> >> Get rid of any toast table when converting a table to a view. >> >> >> >> Does anybody have an explanation for that? Did someone do a force-push >> on the master repository, overwriting some old history? >> >> We don't explicitly forbid this on the master repo, since we expect >> committers to know how things work.. Maybe we need to do that, and >> manually turn it off in case someone actually *needs* to do a non fast >> forward push? But either way, it would be good to actually know how >> tihs happened... >> >> Copying this one to Kevin and Peter too, since they seem to be the >> other people who are involved in any of these pushes. > > Apologies if I contributed to this. At no point did I use a force option. > > I'm using this pattern: > > http://wiki.postgresql.org/wiki/Committing_with_Git#Dependent_Clone_per_Branch.2C_Pushing_and_Pulling_From_a_Local_Repository > > I still have the shell windows open that I used, so (redacting > things which I'm sure are not significant, like git log commands) > there is this: > > You will see a pull command used. Here's what that is: > > kgrittn@Kevin-Desktop:~/pg/master$ cat ~/bin/pull > #!/bin/bash > > pushd ~/pg/postgresql.git >/dev/null && git fetch && popd >/dev/null && git pull > > > Here's what I have in my (still open) shell windows: > > kgrittn@Kevin-Desktop:~/pg/master$ git push > Counting objects: 301, done. > Delta compression using up to 8 threads. > Compressing objects: 100% (154/154), done. > Writing objects: 100% (155/155), 40.94 KiB, done. > Total 155 (delta 147), reused 6 (delta 1) > To /home/kgrittn/pg/postgresql > 2b78d10..3a21ef1 master -> master > kgrittn@Kevin-Desktop:~/pg/master$ cd ../postgresql.git/ > kgrittn@Kevin-Desktop:~/pg/postgresql.git$ git push --dry-run > To ssh://git@gitmaster.postgresql.org/postgresql.git > + b15a6da...3a21ef1 master -> master (forced update) > + b15a6da...2b78d10 anonymous/master -> anonymous/master (forced update) > kgrittn@Kevin-Desktop:~/pg/postgresql.git$ git fetch > remote: Counting objects: 31, done. > remote: Compressing objects: 100% (16/16), done. > remote: Total 16 (delta 15), reused 0 (delta 0) > Unpacking objects: 100% (16/16), done. > From ssh://gitmaster.postgresql.org/postgresql > + 3a21ef1...b15a6da master -> master (forced update) > 2b78d10..b15a6da anonymous/master -> anonymous/master ^^ there it clearly says forced update. So - at least now we know what happened. Question still remains how then. Normally it should, AFAIK, refuse to do such a push without a --force. But admittedly, I don't know for sure exactly how those more complex setups work. Do you by any chance have any other git settings either in your local or in the repository config? Also, can someone who knows how that stuff is set up, explain why it's pushing anonymous/master in this case? Seems like it should just push the actual branch, no? Are those instructions for usage wrong? > kgrittn@Kevin-Desktop:~/pg/postgresql.git$ git remote --verbose > origin ssh://git@gitmaster.postgresql.org/postgresql.git (fetch) > origin ssh://git@gitmaster.postgresql.org/postgresql.git (push) > kgrittn@Kevin-Desktop:~/pg/postgresql.git$ cd ../master > kgrittn@Kevin-Desktop:~/pg/master$ pull > remote: Counting objects: 31, done. > remote: Compressing objects: 100% (16/16), done. > remote: Total 16 (delta 15), reused 0 (delta 0) > Unpacking objects: 100% (16/16), done. > From /home/kgrittn/pg/postgresql > + 3a21ef1...b15a6da master -> origin/master (forced update) ^^ there's another forced update. Which seems to indicate that your git is doing forced updates by default in both directions. I have no idea how it would do that by default - I've always received an error when accidentally doing something like that... --Magnus HaganderMe: http://www.hagander.net/Work: http://www.redpill-linpro.com/
On 2013-03-04 05:11:00 -0800, Kevin Grittner wrote: > Magnus Hagander <magnus@hagander.net> wrote: > > > Actually, looking closer, I'm seeing a failure when it actually tries the > > push: > > > > To ssh://git@git.postgresql.org/postgresql.git > > ! [rejected] master -> master (non-fast-forward) > > > > > > So the problem might have nothing at all to do with the cacheing. > > > > AFAICT, the three missing commits are materialized views, > > accidentally committed .orig file and \l support. > > > > But. The *anonymous* repository also has: > > bc61878682051678ade5f59da7bfd90ab72ce13b Fix > > map_sql_value_to_xml_value() to treat domains like their base types. > > > > This patch is *not* in the master repository, it's only in anonymous. > > (The object is in the repository, but it's not part of any branch) > > > > How the hell did *that* happen? > > > > The master repo has: > > commit 5db5974c692b0fc68e7608dd85a6b4e6173a0f28 > > Author: Peter Eisentraut <peter_e@gmx.net> > > > > psql: Let \l accept a pattern > > > > commit d63977eea3ab18fdec05e370b633d10b9fd20179 > > Author: Kevin Grittner <kgrittn@postgresql.org> > > > > Remove accidentally-committed .orig file. > > > > commit 3bf3ab8c563699138be02f9dc305b7b77a724307 > > Author: Kevin Grittner <kgrittn@postgresql.org> > > > > Add a materialized view relations. > > > > commit b15a6da29217b14f02895af1d9271e84415a91ae > > Author: Tom Lane <tgl@sss.pgh.pa.us> > > > > Get rid of any toast table when converting a table to a view. > > > > > > And the anonymous one has: > > commit bc61878682051678ade5f59da7bfd90ab72ce13b > > Author: Tom Lane <tgl@sss.pgh.pa.us> > > > > Fix map_sql_value_to_xml_value() to treat domains like their base types. > > > > commit 3bf3ab8c563699138be02f9dc305b7b77a724307 > > Author: Kevin Grittner <kgrittn@postgresql.org> > > > > Add a materialized view relations. > > > > commit b15a6da29217b14f02895af1d9271e84415a91ae > > Author: Tom Lane <tgl@sss.pgh.pa.us> > > > > Get rid of any toast table when converting a table to a view. > > > > > > > > Does anybody have an explanation for that? Did someone do a force-push > > on the master repository, overwriting some old history? > > > > We don't explicitly forbid this on the master repo, since we expect > > committers to know how things work.. Maybe we need to do that, and > > manually turn it off in case someone actually *needs* to do a non fast > > forward push? But either way, it would be good to actually know how > > tihs happened... > > > > Copying this one to Kevin and Peter too, since they seem to be the > > other people who are involved in any of these pushes. > > Apologies if I contributed to this. At no point did I use a force option. > > I'm using this pattern: > > http://wiki.postgresql.org/wiki/Committing_with_Git#Dependent_Clone_per_Branch.2C_Pushing_and_Pulling_From_a_Local_Repository > > I still have the shell windows open that I used, so (redacting > things which I'm sure are not significant, like git log commands) > there is this: > > You will see a pull command used. Here's what that is: > > kgrittn@Kevin-Desktop:~/pg/master$ cat ~/bin/pull > #!/bin/bash > > pushd ~/pg/postgresql.git >/dev/null && git fetch && popd >/dev/null && git pull Completely unrelated, but subshells are useful for that kind of thing, e.g. (cd ~/pg/postgresql.git && git fetch) && git pull would do the same. > Here's what I have in my (still open) shell windows: > > kgrittn@Kevin-Desktop:~/pg/master$ git push > Counting objects: 301, done. > Delta compression using up to 8 threads. > Compressing objects: 100% (154/154), done. > Writing objects: 100% (155/155), 40.94 KiB, done. > Total 155 (delta 147), reused 6 (delta 1) > To /home/kgrittn/pg/postgresql > 2b78d10..3a21ef1 master -> master > kgrittn@Kevin-Desktop:~/pg/master$ cd ../postgresql.git/ > kgrittn@Kevin-Desktop:~/pg/postgresql.git$ git push --dry-run > To ssh://git@gitmaster.postgresql.org/postgresql.git > + b15a6da...3a21ef1 master -> master (forced update) > + b15a6da...2b78d10 anonymous/master -> anonymous/master (forced update) > kgrittn@Kevin-Desktop:~/pg/postgresql.git$ git fetch > remote: Counting objects: 31, done. > remote: Compressing objects: 100% (16/16), done. > remote: Total 16 (delta 15), reused 0 (delta 0) > Unpacking objects: 100% (16/16), done. > From ssh://gitmaster.postgresql.org/postgresql > + 3a21ef1...b15a6da master -> master (forced update) > 2b78d10..b15a6da anonymous/master -> anonymous/master > kgrittn@Kevin-Desktop:~/pg/postgresql.git$ git remote --verbose > origin ssh://git@gitmaster.postgresql.org/postgresql.git (fetch) > origin ssh://git@gitmaster.postgresql.org/postgresql.git (push) > kgrittn@Kevin-Desktop:~/pg/postgresql.git$ cd ../master > kgrittn@Kevin-Desktop:~/pg/master$ pull > remote: Counting objects: 31, done. > remote: Compressing objects: 100% (16/16), done. > remote: Total 16 (delta 15), reused 0 (delta 0) > Unpacking objects: 100% (16/16), done. > From /home/kgrittn/pg/postgresql > + 3a21ef1...b15a6da master -> origin/master (forced update) > First, rewinding head to replay your work on top of it... > kgrittn@Kevin-Desktop:~/pg/master$ git status > # On branch master > nothing to commit (working directory clean) > kgrittn@Kevin-Desktop:~/pg/master$ git pull > Current branch master is up to date. > kgrittn@Kevin-Desktop:~/pg/master$ git status > # On branch master > nothing to commit (working directory clean) > kgrittn@Kevin-Desktop:~/pg/master$ patch -p1 <~/patch/matview-v6.diff > [some offsets in rewriteDefine.c which looked OK on review] > kgrittn@Kevin-Desktop:~/pg/master$ git add --all > kgrittn@Kevin-Desktop:~/pg/master$ git commit > [master 3bf3ab8] Add a materialized view relations. > 103 files changed, 4240 insertions(+), 438 deletions(-) > create mode 100644 doc/src/sgml/ref/alter_materialized_view.sgml > create mode 100644 doc/src/sgml/ref/create_materialized_view.sgml > create mode 100644 doc/src/sgml/ref/drop_materialized_view.sgml > create mode 100644 doc/src/sgml/ref/refresh_materialized_view.sgml > create mode 100644 src/backend/commands/matview.c > create mode 100644 src/backend/rewrite/rewriteDefine.c.orig > create mode 100644 src/include/commands/matview.h > create mode 100644 src/test/regress/expected/matview.out > create mode 100644 src/test/regress/sql/matview.sql > kgrittn@Kevin-Desktop:~/pg/master$ git push > Counting objects: 301, done. > Delta compression using up to 8 threads. > Compressing objects: 100% (154/154), done. > Writing objects: 100% (155/155), 40.95 KiB, done. > Total 155 (delta 147), reused 5 (delta 1) > To /home/kgrittn/pg/postgresql > b15a6da..3bf3ab8 master -> master > kgrittn@Kevin-Desktop:~/pg/master$ cd ../postgresql.git/ > kgrittn@Kevin-Desktop:~/pg/postgresql.git$ git push --dry-run > To ssh://git@gitmaster.postgresql.org/postgresql.git > b15a6da..3bf3ab8 master -> master > kgrittn@Kevin-Desktop:~/pg/postgresql.git$ git push > Counting objects: 301, done. > Delta compression using up to 8 threads. > Compressing objects: 100% (8/8), done. > Writing objects: 100% (155/155), 40.95 KiB, done. > Total 155 (delta 147), reused 155 (delta 147) > To ssh://git@gitmaster.postgresql.org/postgresql.git > b15a6da..3bf3ab8 master -> master > > [on review, discover accidental commit of .orig file] > > kgrittn@Kevin-Desktop:~/pg/master$ git rm src/backend/rewrite/rewriteDefine.c.orig > rm 'src/backend/rewrite/rewriteDefine.c.orig' > kgrittn@Kevin-Desktop:~/pg/master$ git commit > [master d63977e] Remove accidentally-committed .orig file. > 1 file changed, 945 deletions(-) > delete mode 100644 src/backend/rewrite/rewriteDefine.c.orig > kgrittn@Kevin-Desktop:~/pg/master$ git push > Counting objects: 9, done. > Delta compression using up to 8 threads. > Compressing objects: 100% (5/5), done. > Writing objects: 100% (5/5), 440 bytes, done. > Total 5 (delta 4), reused 0 (delta 0) > Unpacking objects: 100% (5/5), done. > To /home/kgrittn/pg/postgresql > 3bf3ab8..d63977e master -> master > kgrittn@Kevin-Desktop:~/pg/master$ cd ../postgresql.git/ > kgrittn@Kevin-Desktop:~/pg/postgresql.git$ git push > Counting objects: 15018, done. > Delta compression using up to 8 threads. > Compressing objects: 100% (4617/4617), done. > Writing objects: 100% (13087/13087), 2.31 MiB | 632 KiB/s, done. > Total 13087 (delta 11231), reused 9860 (delta 8431) > To ssh://git@gitmaster.postgresql.org/postgresql.git > + bc61878...d63977e master -> master (forced update) > + bc61878...b15a6da anonymous/master -> anonymous/master (forced update) So, here it pushed with force, overwriting the old HEAD? It moves master from bc61878 to d63977e which exactly explains the missing map_sql_value_to_xml_value() commit from Tom. Greetings, Andres Freund -- Andres Freund http://www.2ndQuadrant.com/PostgreSQL Development, 24x7 Support, Training & Services
On Mon, Mar 4, 2013 at 1:50 PM, Andres Freund <andres@2ndquadrant.com> wrote: > On 2013-03-04 08:32:35 +0100, Magnus Hagander wrote: >> On Mon, Mar 4, 2013 at 8:16 AM, Magnus Hagander <magnus@hagander.net> wrote: >> > On Mon, Mar 4, 2013 at 3:33 AM, Stephen Frost <sfrost@snowman.net> wrote: >> >> * Tom Lane (tgl@sss.pgh.pa.us) wrote: >> >>> cron job? I was under the impression there was some sort of push >> >>> operation driven by a commit trigger. The web site has certainly >> >>> updated nearly immediately for as long as we've been using git. >> >>> Until this week, that is. >> >> >> >> Curiously, there's two cron jobs, apparently. There's a 'push' one and >> >> then another, independent, 'pull' one. I'll assume they're actually >> >> doing different things, but I wonder if the pull isn't just a >> >> hold-over.. In any case, the push-to-anon, which I hadn't seen >> > >> > No, the pull one doesn't do the main repository - that one pulls >> > "third party" repositories in as mirrors, such as e.g. Bucardo. That >> > one was, btw, broken for several months and nobody noticed - so that's >> > clearly a less important one :) >> > >> >> initially (looking at the pull side instead of the push side), does run >> >> once a minute, though it looks like there's a hook mechanism which >> >> would allow us to trigger the webserver to do a pull when a commit >> >> happens and would still be better than a once-a-minute cronjob. >> > >> > The one that deals with the main repo mirroring runs once per minute. >> > The commit trigger only drops a file in the filesystem so the cronjob >> > knows there is something to do. So that we don't risk holding up the >> > actual commit in case there is a problem with the anonymous mirror >> > server. >> > >> > >> >> Even with that, however, the concern raised was that the gitweb perl >> >> script is quite expensive to run for every request, hence the reason for >> >> doing the cacheing. I've lowered the varnish cacheing to a 5m ttl and >> >> 15m grace and I'll keep an eye on it. >> > >> > That's fixing the wrong problem. The cache is supposed to be >> > automatically purged whenever the push happens, to make sure that the >> > data never gets more than maybe a second stale. So clearly something >> > in that is not working - I didn't specifically test it with the >> > postgres.git repository, but I tested it with a couple of other ones >> > where it worked fine. But maybe we did something special with the main >> > one... >> >> Actually, looking closer, I'm seeing a failure when it actually tries the push: >> >> To ssh://git@git.postgresql.org/postgresql.git >> ! [rejected] master -> master (non-fast-forward) >> >> >> So the problem might have nothing at all to do with the cacheing. >> >> AFAICT, the three missing commits are materialized views, >> accidentally committed .orig file and \l support. >> >> But. The *anonymous* repository also has: >> bc61878682051678ade5f59da7bfd90ab72ce13b Fix >> map_sql_value_to_xml_value() to treat domains like their base types. >> >> This patch is *not* in the master repository, it's only in anonymous. >> (The object is in the repository, but it's not part of any branch) >> >> How the hell did *that* happen? >> >> The master repo has: >> commit 5db5974c692b0fc68e7608dd85a6b4e6173a0f28 >> Author: Peter Eisentraut <peter_e@gmx.net> >> >> psql: Let \l accept a pattern >> >> commit d63977eea3ab18fdec05e370b633d10b9fd20179 >> Author: Kevin Grittner <kgrittn@postgresql.org> >> >> Remove accidentally-committed .orig file. >> >> commit 3bf3ab8c563699138be02f9dc305b7b77a724307 >> Author: Kevin Grittner <kgrittn@postgresql.org> >> >> Add a materialized view relations. >> >> commit b15a6da29217b14f02895af1d9271e84415a91ae >> Author: Tom Lane <tgl@sss.pgh.pa.us> >> >> Get rid of any toast table when converting a table to a view. >> >> >> And the anonymous one has: >> commit bc61878682051678ade5f59da7bfd90ab72ce13b >> Author: Tom Lane <tgl@sss.pgh.pa.us> >> >> Fix map_sql_value_to_xml_value() to treat domains like their base types. >> >> commit 3bf3ab8c563699138be02f9dc305b7b77a724307 >> Author: Kevin Grittner <kgrittn@postgresql.org> >> >> Add a materialized view relations. >> >> commit b15a6da29217b14f02895af1d9271e84415a91ae >> Author: Tom Lane <tgl@sss.pgh.pa.us> >> >> Get rid of any toast table when converting a table to a view. >> >> >> >> Does anybody have an explanation for that? Did someone do a force-push >> on the master repository, overwriting some old history? > > This really looks like Kevin forced a push. > > Could you show > git show --pretty=raw d63977eea3ab18fdec05e370b633d10b9fd20179 > git show --pretty=raw bc61878682051678ade5f59da7bfd90ab72ce13b > git show --pretty=raw 5db5974c692b0fc68e7608dd85a6b4e6173a0f28 In which of the repositories - master, anonymous, or both? > ? > I don't have all the commits here... > > The raw format will explicitly tell which parents are referenced in the > commits... > >> We don't explicitly forbid this on the master repo, since we expect >> committers to know how things work.. Maybe we need to do that, and >> manually turn it off in case someone actually *needs* to do a non fast >> forward push? But either way, it would be good to actually know how >> tihs happened... > > If you allow it the mirroring script should always force a push, > otherwise it will stop pushing in those cases. Yeah. But we shouldn't do it, so disallowing it seems like a good thing. > I think the most realistic way to resolve this is to push the current > anongit state to master and cherry-pick > bc61878682051678ade5f59da7bfd90ab72ce13b ontop it. That way only > committers need to deal with rebasing their branches and not everyone > else. Yeah, I agree that's probably the best way to do it - the number of committers is a lot more limited... Anyone else have an opinion on that part? -- Magnus HaganderMe: http://www.hagander.net/Work: http://www.redpill-linpro.com/
On 2013-03-04 14:18:09 +0100, Magnus Hagander wrote: > On Mon, Mar 4, 2013 at 2:11 PM, Kevin Grittner <kgrittn@ymail.com> wrote: > > Magnus Hagander <magnus@hagander.net> wrote: > > > >> Actually, looking closer, I'm seeing a failure when it actually tries the > >> push: > >> > >> To ssh://git@git.postgresql.org/postgresql.git > >> ! [rejected] master -> master (non-fast-forward) > >> > >> > >> So the problem might have nothing at all to do with the cacheing. > >> > >> AFAICT, the three missing commits are materialized views, > >> accidentally committed .orig file and \l support. > >> > >> But. The *anonymous* repository also has: > >> bc61878682051678ade5f59da7bfd90ab72ce13b Fix > >> map_sql_value_to_xml_value() to treat domains like their base types. > >> > >> This patch is *not* in the master repository, it's only in anonymous. > >> (The object is in the repository, but it's not part of any branch) > >> > >> How the hell did *that* happen? > >> > >> The master repo has: > >> commit 5db5974c692b0fc68e7608dd85a6b4e6173a0f28 > >> Author: Peter Eisentraut <peter_e@gmx.net> > >> > >> psql: Let \l accept a pattern > >> > >> commit d63977eea3ab18fdec05e370b633d10b9fd20179 > >> Author: Kevin Grittner <kgrittn@postgresql.org> > >> > >> Remove accidentally-committed .orig file. > >> > >> commit 3bf3ab8c563699138be02f9dc305b7b77a724307 > >> Author: Kevin Grittner <kgrittn@postgresql.org> > >> > >> Add a materialized view relations. > >> > >> commit b15a6da29217b14f02895af1d9271e84415a91ae > >> Author: Tom Lane <tgl@sss.pgh.pa.us> > >> > >> Get rid of any toast table when converting a table to a view. > >> > >> > >> And the anonymous one has: > >> commit bc61878682051678ade5f59da7bfd90ab72ce13b > >> Author: Tom Lane <tgl@sss.pgh.pa.us> > >> > >> Fix map_sql_value_to_xml_value() to treat domains like their base types. > >> > >> commit 3bf3ab8c563699138be02f9dc305b7b77a724307 > >> Author: Kevin Grittner <kgrittn@postgresql.org> > >> > >> Add a materialized view relations. > >> > >> commit b15a6da29217b14f02895af1d9271e84415a91ae > >> Author: Tom Lane <tgl@sss.pgh.pa.us> > >> > >> Get rid of any toast table when converting a table to a view. > >> > >> > >> > >> Does anybody have an explanation for that? Did someone do a force-push > >> on the master repository, overwriting some old history? > >> > >> We don't explicitly forbid this on the master repo, since we expect > >> committers to know how things work.. Maybe we need to do that, and > >> manually turn it off in case someone actually *needs* to do a non fast > >> forward push? But either way, it would be good to actually know how > >> tihs happened... > >> > >> Copying this one to Kevin and Peter too, since they seem to be the > >> other people who are involved in any of these pushes. > > > > Apologies if I contributed to this. At no point did I use a force option. > > > > I'm using this pattern: > > > > http://wiki.postgresql.org/wiki/Committing_with_Git#Dependent_Clone_per_Branch.2C_Pushing_and_Pulling_From_a_Local_Repository > > > > I still have the shell windows open that I used, so (redacting > > things which I'm sure are not significant, like git log commands) > > there is this: > > > > You will see a pull command used. Here's what that is: > > > > kgrittn@Kevin-Desktop:~/pg/master$ cat ~/bin/pull > > #!/bin/bash > > > > pushd ~/pg/postgresql.git >/dev/null && git fetch && popd >/dev/null && git pull > > > > > > Here's what I have in my (still open) shell windows: > > > > kgrittn@Kevin-Desktop:~/pg/master$ git push > > Counting objects: 301, done. > > Delta compression using up to 8 threads. > > Compressing objects: 100% (154/154), done. > > Writing objects: 100% (155/155), 40.94 KiB, done. > > Total 155 (delta 147), reused 6 (delta 1) > > To /home/kgrittn/pg/postgresql > > 2b78d10..3a21ef1 master -> master > > kgrittn@Kevin-Desktop:~/pg/master$ cd ../postgresql.git/ > > kgrittn@Kevin-Desktop:~/pg/postgresql.git$ git push --dry-run > > To ssh://git@gitmaster.postgresql.org/postgresql.git > > + b15a6da...3a21ef1 master -> master (forced update) > > + b15a6da...2b78d10 anonymous/master -> anonymous/master (forced update) > > kgrittn@Kevin-Desktop:~/pg/postgresql.git$ git fetch > > remote: Counting objects: 31, done. > > remote: Compressing objects: 100% (16/16), done. > > remote: Total 16 (delta 15), reused 0 (delta 0) > > Unpacking objects: 100% (16/16), done. > > From ssh://gitmaster.postgresql.org/postgresql > > + 3a21ef1...b15a6da master -> master (forced update) > > 2b78d10..b15a6da anonymous/master -> anonymous/master > > ^^ there it clearly says forced update. So - at least now we know what happened. Thats a fetch though. He pulls in Tom's commit. Which is unsurprisingly a forced update because he locally had committed something. > Question still remains how then. > > Normally it should, AFAIK, refuse to do such a push without a --force. > But admittedly, I don't know for sure exactly how those more complex > setups work. > > Do you by any chance have any other git settings either in your local > or in the repository config? > > Also, can someone who knows how that stuff is set up, explain why it's > pushing anonymous/master in this case? Seems like it should just push > the actual branch, no? Are those instructions for usage wrong? The defaults around this are afaik changing. Its push.default that configures that, I suggest setting it to the future default "simple" instead of "matching" which is the current value. > > kgrittn@Kevin-Desktop:~/pg/postgresql.git$ git remote --verbose > > origin ssh://git@gitmaster.postgresql.org/postgresql.git (fetch) > > origin ssh://git@gitmaster.postgresql.org/postgresql.git (push) > > kgrittn@Kevin-Desktop:~/pg/postgresql.git$ cd ../master > > kgrittn@Kevin-Desktop:~/pg/master$ pull > > remote: Counting objects: 31, done. > > remote: Compressing objects: 100% (16/16), done. > > remote: Total 16 (delta 15), reused 0 (delta 0) > > Unpacking objects: 100% (16/16), done. > > From /home/kgrittn/pg/postgresql > > + 3a21ef1...b15a6da master -> origin/master (forced update) > > ^^ there's another forced update. Which seems to indicate that your > git is doing forced updates by default in both directions. > > I have no idea how it would do that by default - I've always received > an error when accidentally doing something like that... Whats "git config -l" saying? Greetings, Andres Freund
On 2013-03-04 14:19:27 +0100, Magnus Hagander wrote: > > This really looks like Kevin forced a push. > > > > Could you show > > git show --pretty=raw d63977eea3ab18fdec05e370b633d10b9fd20179 > > git show --pretty=raw bc61878682051678ade5f59da7bfd90ab72ce13b > > git show --pretty=raw 5db5974c692b0fc68e7608dd85a6b4e6173a0f28 > > In which of the repositories - master, anonymous, or both? Master. The commit-ids should be unique. And I only see the second one in anon. But I guess its not really necessary anymore since we know its Kevin's second push that did the deed even we don't have a clue on why it did. Greetings, Andres Freund -- Andres Freund http://www.2ndQuadrant.com/PostgreSQL Development, 24x7 Support, Training & Services
Magnus Hagander <magnus@hagander.net> wrote: > Question still remains how then. > > Normally it should, AFAIK, refuse to do such a push without a --force. > But admittedly, I don't know for sure exactly how those more complex > setups work. > > Do you by any chance have any other git settings either in your local > or in the repository config? > > Also, can someone who knows how that stuff is set up, explain why it's > pushing anonymous/master in this case? Seems like it should just push > the actual branch, no? Are those instructions for usage wrong? > > >> kgrittn@Kevin-Desktop:~/pg/postgresql.git$ git remote --verbose >> origin ssh://git@gitmaster.postgresql.org/postgresql.git (fetch) >> origin ssh://git@gitmaster.postgresql.org/postgresql.git (push) >> kgrittn@Kevin-Desktop:~/pg/postgresql.git$ cd ../master >> kgrittn@Kevin-Desktop:~/pg/master$ pull >> remote: Counting objects: 31, done. >> remote: Compressing objects: 100% (16/16), done. >> remote: Total 16 (delta 15), reused 0 (delta 0) >> Unpacking objects: 100% (16/16), done. >> From /home/kgrittn/pg/postgresql >> + 3a21ef1...b15a6da master -> origin/master (forced update) > > ^^ there's another forced update. Which seems to indicate that your > git is doing forced updates by default in both directions. > > I have no idea how it would do that by default - I've always received > an error when accidentally doing something like that... Are there any files to look at besides these?: kgrittn@Kevin-Desktop:~/pg/master$ cat ~/.gitconfig [user] name = Kevin Grittner email = kgrittn@postgresql.org [core] excludesfile = /home/kgrittn/.gitexcludesfile pager = less -x4 editor = vim kgrittn@Kevin-Desktop:~/pg/master$ cat .git/config [core] repositoryformatversion = 0 filemode = true bare = false logallrefupdates = true [remote "origin"] fetch = +refs/heads/master:refs/remotes/origin/master url = /home/kgrittn/pg/postgresql [branch "master"] remote = origin merge = refs/heads/master rebase = true [remote "github"] url = git@github.com:kgrittn/postgres.git fetch = +refs/heads/*:refs/remotes/github/* kgrittn@Kevin-Desktop:~/pg/master$ cat ../postgresql.git/config [core] repositoryformatversion = 0 filemode = true bare = true [remote "origin"] fetch = +refs/*:refs/* mirror = true url = ssh://git@gitmaster.postgresql.org/postgresql.git The rebase option was something I did because the Wiki instructions told me to: http://wiki.postgresql.org/wiki/Committing_with_Git#Dependent_Clone_per_Branch.2C_Pushing_and_Pulling_From_a_Local_Repository I don't think there is anything there which wasn't done per instructions, but maybe the Wiki needs a fix? -- Kevin Grittner EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company
On Mon, Mar 4, 2013 at 2:52 PM, Kevin Grittner <kgrittn@ymail.com> wrote: > Magnus Hagander <magnus@hagander.net> wrote: > >> Question still remains how then. >> >> Normally it should, AFAIK, refuse to do such a push without a --force. >> But admittedly, I don't know for sure exactly how those more complex >> setups work. >> >> Do you by any chance have any other git settings either in your local >> or in the repository config? >> >> Also, can someone who knows how that stuff is set up, explain why it's >> pushing anonymous/master in this case? Seems like it should just push >> the actual branch, no? Are those instructions for usage wrong? >> >> >>> kgrittn@Kevin-Desktop:~/pg/postgresql.git$ git remote --verbose >>> origin ssh://git@gitmaster.postgresql.org/postgresql.git (fetch) >>> origin ssh://git@gitmaster.postgresql.org/postgresql.git (push) >>> kgrittn@Kevin-Desktop:~/pg/postgresql.git$ cd ../master >>> kgrittn@Kevin-Desktop:~/pg/master$ pull >>> remote: Counting objects: 31, done. >>> remote: Compressing objects: 100% (16/16), done. >>> remote: Total 16 (delta 15), reused 0 (delta 0) >>> Unpacking objects: 100% (16/16), done. >>> From /home/kgrittn/pg/postgresql >>> + 3a21ef1...b15a6da master -> origin/master (forced update) >> >> ^^ there's another forced update. Which seems to indicate that your >> git is doing forced updates by default in both directions. >> >> I have no idea how it would do that by default - I've always received >> an error when accidentally doing something like that... > > Are there any files to look at besides these?: > > kgrittn@Kevin-Desktop:~/pg/master$ cat ~/.gitconfig > [user] > name = Kevin Grittner > email = kgrittn@postgresql.org > [core] > excludesfile = /home/kgrittn/.gitexcludesfile > pager = less -x4 > editor = vim > kgrittn@Kevin-Desktop:~/pg/master$ cat .git/config > [core] > repositoryformatversion = 0 > filemode = true > bare = false > logallrefupdates = true > [remote "origin"] > fetch = +refs/heads/master:refs/remotes/origin/master > url = /home/kgrittn/pg/postgresql > [branch "master"] > remote = origin > merge = refs/heads/master > rebase = true > [remote "github"] > url = git@github.com:kgrittn/postgres.git > fetch = +refs/heads/*:refs/remotes/github/* > kgrittn@Kevin-Desktop:~/pg/master$ cat ../postgresql.git/config > [core] > repositoryformatversion = 0 > filemode = true > bare = true > [remote "origin"] > fetch = +refs/*:refs/* > mirror = true ^^ that's your problem. AFAICT, that one sets it to be force-always. > url = ssh://git@gitmaster.postgresql.org/postgresql.git > > The rebase option was something I did because the Wiki instructions > told me to: > > http://wiki.postgresql.org/wiki/Committing_with_Git#Dependent_Clone_per_Branch.2C_Pushing_and_Pulling_From_a_Local_Repository > > I don't think there is anything there which wasn't done per > instructions, but maybe the Wiki needs a fix? Yea, I'm pretty sure that --mirror thing is the cause of the problem. Of course, we shuld deploy protect on the server against this, but that's only going to cause the procedure to *fail* rather than owerwrite the repo. It still need to be fixed. -- Magnus HaganderMe: http://www.hagander.net/Work: http://www.redpill-linpro.com/
Andres Freund wrote: > I think the most realistic way to resolve this is to push the current > anongit state to master and cherry-pick > bc61878682051678ade5f59da7bfd90ab72ce13b ontop it. That way only > committers need to deal with rebasing their branches and not everyone > else. I agree this seems the sanest course of action, but I think it would be better to install defenses in the master repo so that forced pushes are no longer allowed *before* rebasing gitmaster; otherwise some other committer could clobber things again if they don't force-pull their trees. FWIW I also use a bare mirror (same setup as Kevin). -- Álvaro Herrera http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services
On 2013-03-04 11:33:49 -0300, Alvaro Herrera wrote: > Andres Freund wrote: > > > I think the most realistic way to resolve this is to push the current > > anongit state to master and cherry-pick > > bc61878682051678ade5f59da7bfd90ab72ce13b ontop it. That way only > > committers need to deal with rebasing their branches and not everyone > > else. > > I agree this seems the sanest course of action, but I think it would be > better to install defenses in the master repo so that forced pushes are > no longer allowed *before* rebasing gitmaster; otherwise some other > committer could clobber things again if they don't force-pull their > trees. Its just receive.denyNonFastForwards that needs to be set to true for that, so it should be simple enouh. > FWIW I also use a bare mirror (same setup as Kevin). I think that setup is fine as far as pulling goes, but you should push directly from the individual branches. Greetings, Andres Freund -- Andres Freund http://www.2ndQuadrant.com/PostgreSQL Development, 24x7 Support, Training & Services
Alvaro Herrera <alvherre@2ndquadrant.com> writes: > Andres Freund wrote: >> I think the most realistic way to resolve this is to push the current >> anongit state to master and cherry-pick >> bc61878682051678ade5f59da7bfd90ab72ce13b ontop it. That way only >> committers need to deal with rebasing their branches and not everyone >> else. > I agree this seems the sanest course of action, but I think it would be > better to install defenses in the master repo so that forced pushes are > no longer allowed *before* rebasing gitmaster; otherwise some other > committer could clobber things again if they don't force-pull their > trees. Agreed. I'm astonished that we had this configured this way in the first place. FWIW, I noticed something about a "forced update" when I did a git pull after Kevin's commits. I failed to pay enough attention though. Now that I look, the xml fix is not in my local history either, and I don't know what bc61878682051678ade5f59da7bfd90ab72ce13b is because none of the recent commits have that hash according to my repo: $ git status # On branch master nothing to commit (working directory clean) $ git log commit d63977eea3ab18fdec05e370b633d10b9fd20179 Author: Kevin Grittner <kgrittn@postgresql.org> Date: Sun Mar 3 18:38:44 2013 -0600 Remove accidentally-committed .orig file. commit 3bf3ab8c563699138be02f9dc305b7b77a724307 Author: Kevin Grittner <kgrittn@postgresql.org> Date: Sun Mar 3 18:23:31 2013 -0600 Add a materialized view relations. A materialized view has a rule just like a view and a heap and other physicalproperties like a table. The rule is only used to populate the table, references in queries refer to the materializeddata. This is a minimal implementation, but should still be useful in many cases. Currently data is onlypopulated "on demand" by the CREATE MATERIALIZED VIEW and REFRESH MATERIALIZED VIEW statements. It is expected thatfuture releases will add incremental updates with various timings, and that a more refined concept of defining whatis "fresh" data will be developed. At some point it may even be possible to have queries use a materialized in placeof references to underlying tables, but that requires the other above-mentioned features to be working first. Much of the documentation work by Robert Haas. Review by Noah Misch, Thom Brown, Robert Haas, Marko Tiikkaja Securityreview by KaiGai Kohei, with a decision on how best to implement sepgsql still pending. commit b15a6da29217b14f02895af1d9271e84415a91ae Author: Tom Lane <tgl@sss.pgh.pa.us> Date: Sun Mar 3 19:05:47 2013 -0500 Get rid of any toast table when converting a table to a view. Also make sure other fields of the view's pg_classentry are appropriate for a view; it shouldn't have relfrozenxid set for instance. This ancient omission isn'tbelieved to have any serious consequences in versions 8.4-9.2, so no backpatch. But let's fix it before it does bite us in some serious way. It's just luck that the case doesn't cause problems for autovacuum. (It did cause problemsin 8.3, but that's out of support.) Andres Freund commit 2b78d101d1d6b1d8533a7b7aeff4d82b10a915f8 Author: Tom Lane <tgl@sss.pgh.pa.us> Date: Sun Mar 3 17:39:58 2013 -0500 Fix SQL function execution to be safe with long-lived FmgrInfos. regards, tom lane
On 2013-03-04 09:40:38 -0500, Tom Lane wrote: > Alvaro Herrera <alvherre@2ndquadrant.com> writes: > > Andres Freund wrote: > >> I think the most realistic way to resolve this is to push the current > >> anongit state to master and cherry-pick > >> bc61878682051678ade5f59da7bfd90ab72ce13b ontop it. That way only > >> committers need to deal with rebasing their branches and not everyone > >> else. > > > I agree this seems the sanest course of action, but I think it would be > > better to install defenses in the master repo so that forced pushes are > > no longer allowed *before* rebasing gitmaster; otherwise some other > > committer could clobber things again if they don't force-pull their > > trees. > > Agreed. I'm astonished that we had this configured this way in the > first place. > > FWIW, I noticed something about a "forced update" when I did a git pull > after Kevin's commits. I failed to pay enough attention though. Now > that I look, the xml fix is not in my local history either, and I don't > know what bc61878682051678ade5f59da7bfd90ab72ce13b is because none of > the recent commits have that hash according to my repo: That is/was your map_sql_value_to_xml_value() commit which is now not in the normal history anymore because kevin pushed a HEAD that didn't include it. You should be able to see it with git show bc61878682051678ade5f59da7bfd90ab72ce13b Greetings, Andres Freund -- Andres Freund http://www.2ndQuadrant.com/PostgreSQL Development, 24x7 Support, Training & Services
Tom Lane wrote: > FWIW, I noticed something about a "forced update" when I did a git pull > after Kevin's commits. I failed to pay enough attention though. Now > that I look, the xml fix is not in my local history either, and I don't > know what bc61878682051678ade5f59da7bfd90ab72ce13b is because none of > the recent commits have that hash according to my repo: Maybe you can see it mentioned in a "git reflog"? -- Álvaro Herrera http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services
On Mon, Mar 4, 2013 at 1:50 PM, Andres Freund <andres@2ndquadrant.com> wrote: > I think the most realistic way to resolve this is to push the current > anongit state to master and cherry-pick > bc61878682051678ade5f59da7bfd90ab72ce13b ontop it. That way only > committers need to deal with rebasing their branches and not everyone > else. I satarted looking at doing this, but this is backwards, right? Commit bc61878682051678ade5f59da7bfd90ab72ce13b exists on anongit, and only there. So what we need is to reset master HEAD to that of anongit, and then cherrypick 3bf3ab8c563699138be02f9dc305b7b77a724307, d63977eea3ab18fdec05e370b633d10b9fd20179 and 5db5974c692b0fc68e7608dd85a6b4e6173a0f28 on top of that, no? --Magnus HaganderMe: http://www.hagander.net/Work: http://www.redpill-linpro.com/
On 2013-03-04 15:52:59 +0100, Magnus Hagander wrote: > On Mon, Mar 4, 2013 at 1:50 PM, Andres Freund <andres@2ndquadrant.com> wrote: > > > I think the most realistic way to resolve this is to push the current > > anongit state to master and cherry-pick > > bc61878682051678ade5f59da7bfd90ab72ce13b ontop it. That way only > > committers need to deal with rebasing their branches and not everyone > > else. > > I satarted looking at doing this, but this is backwards, right? Yes. Sorry. > Commit bc61878682051678ade5f59da7bfd90ab72ce13b exists on anongit, and > only there. > So what we need is to reset master HEAD to that of anongit, and then > cherrypick 3bf3ab8c563699138be02f9dc305b7b77a724307, Uhm, that is in both, isn't it? > d63977eea3ab18fdec05e370b633d10b9fd20179 and > 5db5974c692b0fc68e7608dd85a6b4e6173a0f28 on top of that, no? Yes. Greetings, Andres Freund -- Andres Freund http://www.2ndQuadrant.com/PostgreSQL Development, 24x7 Support, Training & Services
Andres Freund wrote: > On 2013-03-04 11:33:49 -0300, Alvaro Herrera wrote: > > Andres Freund wrote: > > FWIW I also use a bare mirror (same setup as Kevin). > > I think that setup is fine as far as pulling goes, but you should push > directly from the individual branches. From the individual repos you mean? I don't like that very much, because when I back-patch something, I prefer to push all commits locally to the bare mirror repo before pushing them out together to gitmaster. That way, if I notice a mistake at the last minute, I can still revoke everything and start afresh. One difference in my setup is that my bare mirror has two remotes, one from gitmaster and one from the anonymous mirror. The latter fetches from cron every 5 minutes. The other one only fetches when I invoke it manually (because it needs access to my passphrase-protected SSH key). -- Álvaro Herrera http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services
On 2013-03-04 11:58:42 -0300, Alvaro Herrera wrote: > Andres Freund wrote: > > On 2013-03-04 11:33:49 -0300, Alvaro Herrera wrote: > > > Andres Freund wrote: > > > > FWIW I also use a bare mirror (same setup as Kevin). > > > > I think that setup is fine as far as pulling goes, but you should push > > directly from the individual branches. > > From the individual repos you mean? I don't like that very much, > because when I back-patch something, I prefer to push all commits > locally to the bare mirror repo before pushing them out together to > gitmaster. That way, if I notice a mistake at the last minute, I can > still revoke everything and start afresh. > One difference in my setup is that my bare mirror has two remotes, one > from gitmaster and one from the anonymous mirror. The latter fetches > from cron every 5 minutes. The other one only fetches when I invoke it > manually (because it needs access to my passphrase-protected SSH key). Well, then you may not use --mirror/remote.xxx.mirror mode since that a) pushes all branches b) pushes new local branches c) forces updates. I guess in your setup you could set --mirror with remote.anongit.mirror so it only mirrors anongit but when pushing to ssh it doesnt... Greetings, Andres Freund -- Andres Freund http://www.2ndQuadrant.com/PostgreSQL Development, 24x7 Support, Training & Services
Magnus Hagander wrote: > On Mon, Mar 4, 2013 at 1:50 PM, Andres Freund <andres@2ndquadrant.com> wrote: > > > I think the most realistic way to resolve this is to push the current > > anongit state to master and cherry-pick > > bc61878682051678ade5f59da7bfd90ab72ce13b ontop it. That way only > > committers need to deal with rebasing their branches and not everyone > > else. > > I satarted looking at doing this, but this is backwards, right? > > Commit bc61878682051678ade5f59da7bfd90ab72ce13b exists on anongit, and > only there. > > So what we need is to reset master HEAD to that of anongit, and then > cherrypick 3bf3ab8c563699138be02f9dc305b7b77a724307, > d63977eea3ab18fdec05e370b633d10b9fd20179 and > 5db5974c692b0fc68e7608dd85a6b4e6173a0f28 on top of that, no? Uh, 3bf3ab8c5 is parent of bc6187868 so surely you don't need to cherry-pick that one? Only the other two. -- Álvaro Herrera http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services
Andres Freund <andres@2ndquadrant.com> writes:
> On 2013-03-04 09:40:38 -0500, Tom Lane wrote:
>> FWIW, I noticed something about a "forced update" when I did a git pull
>> after Kevin's commits. I failed to pay enough attention though. Now
>> that I look, the xml fix is not in my local history either, and I don't
>> know what bc61878682051678ade5f59da7bfd90ab72ce13b is because none of
>> the recent commits have that hash according to my repo:
> That is/was your map_sql_value_to_xml_value() commit which is now not in
> the normal history anymore because kevin pushed a HEAD that didn't
> include it. You should be able to see it with git show
> bc61878682051678ade5f59da7bfd90ab72ce13b
Ah, right, so it is still there but it's not in the chain leading to
HEAD. So "forced update" really means "I moved HEAD in a non fast
forward way"?
regards, tom lane
Andres Freund wrote: > On 2013-03-04 11:58:42 -0300, Alvaro Herrera wrote: > > Andres Freund wrote: > > > On 2013-03-04 11:33:49 -0300, Alvaro Herrera wrote: > > > > Andres Freund wrote: > > > > > > FWIW I also use a bare mirror (same setup as Kevin). > > > > > > I think that setup is fine as far as pulling goes, but you should push > > > directly from the individual branches. > > > > From the individual repos you mean? I don't like that very much, > > because when I back-patch something, I prefer to push all commits > > locally to the bare mirror repo before pushing them out together to > > gitmaster. That way, if I notice a mistake at the last minute, I can > > still revoke everything and start afresh. > > > One difference in my setup is that my bare mirror has two remotes, one > > from gitmaster and one from the anonymous mirror. The latter fetches > > from cron every 5 minutes. The other one only fetches when I invoke it > > manually (because it needs access to my passphrase-protected SSH key). > > Well, then you may not use --mirror/remote.xxx.mirror mode since that a) > pushes all branches b) pushes new local branches c) forces updates. Hm, there are no new local branches here; I fetch to the same branches as the other remote: git fetch -q anon-origin master:master REL9_2_STABLE:REL9_2_STABLE REL9_1_STABLE:REL9_1_STABLE REL9_0_STABLE:REL9_0_STABLEREL8_4_STABLE:REL8_4_STABLE REL8_3_STABLE:REL8_3_STABLE Note this fails (which is what I want) when it's not a FF pull. So my local repository hasn't had any corruption. In fact, to observe the breakage, I had to do a "git fetch anon-origin master:anon-master". I guess there clearly weren't enough protections in the gitmaster server to prevent me from causing major breakage (hopefully there are now), but then I'm so chicken that I triple-check these things before pushing, so no major harm has been done by me to date, even though my setup may be theoretically dangerous. -- Álvaro Herrera http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services
Tom Lane wrote: > Andres Freund <andres@2ndquadrant.com> writes: > > On 2013-03-04 09:40:38 -0500, Tom Lane wrote: > >> FWIW, I noticed something about a "forced update" when I did a git pull > >> after Kevin's commits. I failed to pay enough attention though. Now > >> that I look, the xml fix is not in my local history either, and I don't > >> know what bc61878682051678ade5f59da7bfd90ab72ce13b is because none of > >> the recent commits have that hash according to my repo: > > > That is/was your map_sql_value_to_xml_value() commit which is now not in > > the normal history anymore because kevin pushed a HEAD that didn't > > include it. You should be able to see it with git show > > bc61878682051678ade5f59da7bfd90ab72ce13b > > Ah, right, so it is still there but it's not in the chain leading to > HEAD. So "forced update" really means "I moved HEAD in a non fast > forward way"? Well, to git, "non fast forward" can also mean create a merge commit of two heads, which isn't very problematic. A forced update might mean (such as in this case) that one or more commits are left completely disconnected from history -- that's much worse. -- Álvaro Herrera http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services
On 2013-03-04 10:09:10 -0500, Tom Lane wrote: > Andres Freund <andres@2ndquadrant.com> writes: > > On 2013-03-04 09:40:38 -0500, Tom Lane wrote: > >> FWIW, I noticed something about a "forced update" when I did a git pull > >> after Kevin's commits. I failed to pay enough attention though. Now > >> that I look, the xml fix is not in my local history either, and I don't > >> know what bc61878682051678ade5f59da7bfd90ab72ce13b is because none of > >> the recent commits have that hash according to my repo: > > > That is/was your map_sql_value_to_xml_value() commit which is now not in > > the normal history anymore because kevin pushed a HEAD that didn't > > include it. You should be able to see it with git show > > bc61878682051678ade5f59da7bfd90ab72ce13b > > Ah, right, so it is still there but it's not in the chain leading to > HEAD. So "forced update" really means "I moved HEAD in a non fast > forward way"? Exactly. And normally you need to specify --force for that except in Kevin's setup the mirror mode used (and described in the wiki) also includes that... Greetings, Andres Freund -- Andres Freund http://www.2ndQuadrant.com/PostgreSQL Development, 24x7 Support, Training & Services
On 2013-03-04 12:15:26 -0300, Alvaro Herrera wrote: > Tom Lane wrote: > > Andres Freund <andres@2ndquadrant.com> writes: > > > On 2013-03-04 09:40:38 -0500, Tom Lane wrote: > > >> FWIW, I noticed something about a "forced update" when I did a git pull > > >> after Kevin's commits. I failed to pay enough attention though. Now > > >> that I look, the xml fix is not in my local history either, and I don't > > >> know what bc61878682051678ade5f59da7bfd90ab72ce13b is because none of > > >> the recent commits have that hash according to my repo: > > > > > That is/was your map_sql_value_to_xml_value() commit which is now not in > > > the normal history anymore because kevin pushed a HEAD that didn't > > > include it. You should be able to see it with git show > > > bc61878682051678ade5f59da7bfd90ab72ce13b > > > > Ah, right, so it is still there but it's not in the chain leading to > > HEAD. So "forced update" really means "I moved HEAD in a non fast > > forward way"? > > Well, to git, "non fast forward" can also mean create a merge commit of > two heads, which isn't very problematic. A forced update might mean > (such as in this case) that one or more commits are left completely > disconnected from history -- that's much worse. Well, but thats during pulling/merging and not during pushing. The latter doesn't do any merges. Greetings, Andres Freund -- Andres Freund http://www.2ndQuadrant.com/PostgreSQL Development, 24x7 Support, Training & Services
Alvaro Herrera <alvherre@2ndquadrant.com> writes:
> I guess there clearly weren't enough protections in the gitmaster server
> to prevent me from causing major breakage (hopefully there are now),
Are there? I concur with your recommendation that we should lock
this down *before* we set about repairing the master repo's contents.
It seems particularly important that we do so if we're going to ask
committers to rebase, because it seems like somebody who does that wrong
could easily screw things up again.
regards, tom lane
On Mon, Mar 4, 2013 at 4:17 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote: > Alvaro Herrera <alvherre@2ndquadrant.com> writes: >> I guess there clearly weren't enough protections in the gitmaster server >> to prevent me from causing major breakage (hopefully there are now), > > Are there? I concur with your recommendation that we should lock > this down *before* we set about repairing the master repo's contents. There are. I put them in, and I'm currently working on repairing the repo contents. > It seems particularly important that we do so if we're going to ask > committers to rebase, because it seems like somebody who does that wrong > could easily screw things up again. Yeah. -- Magnus HaganderMe: http://www.hagander.net/Work: http://www.redpill-linpro.com/
On Mon, Mar 4, 2013 at 4:22 PM, Magnus Hagander <magnus@hagander.net> wrote: > On Mon, Mar 4, 2013 at 4:17 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote: >> Alvaro Herrera <alvherre@2ndquadrant.com> writes: >>> I guess there clearly weren't enough protections in the gitmaster server >>> to prevent me from causing major breakage (hopefully there are now), >> >> Are there? I concur with your recommendation that we should lock >> this down *before* we set about repairing the master repo's contents. > > There are. I put them in, and I'm currently working on repairing the > repo contents. Ok, both repos *should* now be where they are supposed to be, per this discussion. AFAICT, pushing now works fine between primary and anonymous. And yes, the cache purging appears to be broken *as well*. The reason for that one seems to be that I forgot to update the port number for cache invalidation when I moved it into production. Oops. That should start working on the next commit that someone does. But git:// access to anonymous and ssh:// access to master should both work, and contain the complete and correct repositorires. -- Magnus HaganderMe: http://www.hagander.net/Work: http://www.redpill-linpro.com/
Magnus Hagander <magnus@hagander.net> writes:
> Ok, both repos *should* now be where they are supposed to be, per this
> discussion.
Being a paranoid sort, I'm doing a full new "git clone" to compare
to my old repo. But assuming that that's okay, what instructions
do you want to give committers about recovering? There should be
a followup with that info to the pgsql-hackers message you posted.
The other todo item was updating the wiki's instructions for committers.
regards, tom lane
On Mon, Mar 4, 2013 at 4:48 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote: > Magnus Hagander <magnus@hagander.net> writes: >> Ok, both repos *should* now be where they are supposed to be, per this >> discussion. > > Being a paranoid sort, I'm doing a full new "git clone" to compare > to my old repo. But assuming that that's okay, what instructions > do you want to give committers about recovering? There should be > a followup with that info to the pgsql-hackers message you posted. Yes, please do that comparison. Alvaro looked it over and found things to look right, but one more paranoid check would be very good. On my local clone it "just worked". The pull went through in non-force mode. So all that would be needed was a normal rebase. But I didn't have any active development branch against gitmaster - only against the anonymous one... So I'm not sure if others might run into issues with it. > The other todo item was updating the wiki's instructions for committers. Yeah, someone who actually knows those parts well should do that - I just use a separate repository and manual pull myself, so I don't know enough to do the instrucitons I think. -- Magnus HaganderMe: http://www.hagander.net/Work: http://www.redpill-linpro.com/
On 2013-03-04 10:48:59 -0500, Tom Lane wrote: > Magnus Hagander <magnus@hagander.net> writes: > > Ok, both repos *should* now be where they are supposed to be, per this > > discussion. > > Being a paranoid sort, I'm doing a full new "git clone" to compare > to my old repo. But assuming that that's okay, what instructions > do you want to give committers about recovering? There should be > a followup with that info to the pgsql-hackers message you posted. "git pull your-ssh-pg-upstream --rebase" should do the trick. Greetings, Andres Freund -- Andres Freund http://www.2ndQuadrant.com/PostgreSQL Development, 24x7 Support, Training & Services
Magnus Hagander <magnus@hagander.net> writes:
> On Mon, Mar 4, 2013 at 4:48 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>> Being a paranoid sort, I'm doing a full new "git clone" to compare
>> to my old repo. But assuming that that's okay, what instructions
>> do you want to give committers about recovering? There should be
>> a followup with that info to the pgsql-hackers message you posted.
> Yes, please do that comparison. Alvaro looked it over and found things
> to look right, but one more paranoid check would be very good.
AFAICT things are okay now.
One thing that would be interesting to try is to look for commits that
are unreachable from any of the branch heads, to verify that we've
not missed anything else. I imagine git has a command for that, but
I don't know it --- any ideas?
regards, tom lane
On 2013-03-04 12:11:31 -0500, Tom Lane wrote: > Magnus Hagander <magnus@hagander.net> writes: > > On Mon, Mar 4, 2013 at 4:48 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote: > >> Being a paranoid sort, I'm doing a full new "git clone" to compare > >> to my old repo. But assuming that that's okay, what instructions > >> do you want to give committers about recovering? There should be > >> a followup with that info to the pgsql-hackers message you posted. > > > Yes, please do that comparison. Alvaro looked it over and found things > > to look right, but one more paranoid check would be very good. > > AFAICT things are okay now. > > One thing that would be interesting to try is to look for commits that > are unreachable from any of the branch heads, to verify that we've > not missed anything else. I imagine git has a command for that, but > I don't know it --- any ideas? git fsck --unreachable|grep "^unreachable commit" might do it. Greetings, Andres Freund -- Andres Freund http://www.2ndQuadrant.com/PostgreSQL Development, 24x7 Support, Training & Services
On Mon, Mar 4, 2013 at 6:11 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote: > Magnus Hagander <magnus@hagander.net> writes: >> On Mon, Mar 4, 2013 at 4:48 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote: >>> Being a paranoid sort, I'm doing a full new "git clone" to compare >>> to my old repo. But assuming that that's okay, what instructions >>> do you want to give committers about recovering? There should be >>> a followup with that info to the pgsql-hackers message you posted. > >> Yes, please do that comparison. Alvaro looked it over and found things >> to look right, but one more paranoid check would be very good. > > AFAICT things are okay now. Great. I'll "release" in the other thread then. > One thing that would be interesting to try is to look for commits that > are unreachable from any of the branch heads, to verify that we've > not missed anything else. I imagine git has a command for that, but > I don't know it --- any ideas? I'm running the fsck command suggested by Andres. It's taking a long time to run, but I'll report if it finds something we shouldn't have. -- Magnus HaganderMe: http://www.hagander.net/Work: http://www.redpill-linpro.com/
Magnus Hagander <magnus@hagander.net> writes:
> And yes, the cache purging appears to be broken *as well*. The reason
> for that one seems to be that I forgot to update the port number for
> cache invalidation when I moved it into production. Oops. That should
> start working on the next commit that someone does.
Seemed to work when I pushed the tm2timestamp fix just now --- the
front page updated almost immediately after the next top-of-minute.
Thanks.
regards, tom lane
On Mon, Mar 4, 2013 at 9:18 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote: > Magnus Hagander <magnus@hagander.net> writes: >> And yes, the cache purging appears to be broken *as well*. The reason >> for that one seems to be that I forgot to update the port number for >> cache invalidation when I moved it into production. Oops. That should >> start working on the next commit that someone does. > > Seemed to work when I pushed the tm2timestamp fix just now --- the > front page updated almost immediately after the next top-of-minute. > Thanks. Great, thanks for confirming. Now I just need to get Dave to stop unintentionally purgning the cache every 5 minutes... -- Magnus HaganderMe: http://www.hagander.net/Work: http://www.redpill-linpro.com/
On Mon, Mar 4, 2013 at 8:31 PM, Magnus Hagander <magnus@hagander.net> wrote: > On Mon, Mar 4, 2013 at 9:18 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote: >> Magnus Hagander <magnus@hagander.net> writes: >>> And yes, the cache purging appears to be broken *as well*. The reason >>> for that one seems to be that I forgot to update the port number for >>> cache invalidation when I moved it into production. Oops. That should >>> start working on the next commit that someone does. >> >> Seemed to work when I pushed the tm2timestamp fix just now --- the >> front page updated almost immediately after the next top-of-minute. >> Thanks. > > Great, thanks for confirming. > > Now I just need to get Dave to stop unintentionally purgning the cache > every 5 minutes... Meh. It's on my list. -- Dave Page Blog: http://pgsnake.blogspot.com Twitter: @pgsnake EnterpriseDB UK: http://www.enterprisedb.com The Enterprise PostgreSQL Company
On Tue, Mar 5, 2013 at 10:02 AM, Dave Page <dpage@pgadmin.org> wrote: > On Mon, Mar 4, 2013 at 8:31 PM, Magnus Hagander <magnus@hagander.net> wrote: >> On Mon, Mar 4, 2013 at 9:18 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote: >>> Magnus Hagander <magnus@hagander.net> writes: >>>> And yes, the cache purging appears to be broken *as well*. The reason >>>> for that one seems to be that I forgot to update the port number for >>>> cache invalidation when I moved it into production. Oops. That should >>>> start working on the next commit that someone does. >>> >>> Seemed to work when I pushed the tm2timestamp fix just now --- the >>> front page updated almost immediately after the next top-of-minute. >>> Thanks. >> >> Great, thanks for confirming. >> >> Now I just need to get Dave to stop unintentionally purgning the cache >> every 5 minutes... > > Meh. It's on my list. Oh, I know it is. I'm working on the theory of that eventually getting that fixed will be a lower pain than listening to my whining about it :) -- Magnus HaganderMe: http://www.hagander.net/Work: http://www.redpill-linpro.com/