Thread: branches_of_interest.txt
This file on the buildfarm server is used to tell clients which branches we'd like built. When a new stable branch is created it's added manually to this file, and when one gets to EOL it's removed from the file. This is a rather cumbersome process, and it occurred to me that it could be streamlined by our keeping it in the core repo instead. Buildfarm clients would still need to get it from the buildfarm server, since they don't necessarily have a local repo at all at the stage when it's needed. But that doesn't matter much - the server *does* have a repo kept up to date fairly continuously (every 5 minutes).Essentially what we'd need is a script on the server called from cron which would do something along the lines of GIT_DIR=/path/to/local/clone git show master:path/to/branches_of_interest.txt > htdocs/branches_of_interest.txt We'd add a bit more sanity checking but essentially that would be its job. That way this would be under postgres committer control rather then requiring intervention by some buildfarm server admin. This became a bit of an issue when I was out of contact at the time a branch was created last year, IIRC. Thoughts? cheers andrew
Andrew Dunstan <andrew@dunslane.net> writes:
> This file on the buildfarm server is used to tell clients which branches
> we'd like built. When a new stable branch is created it's added manually
> to this file, and when one gets to EOL it's removed from the file. This
> is a rather cumbersome process, and it occurred to me that it could be
> streamlined by our keeping it in the core repo instead.
I can see the value of people other than you being able to change it,
but keeping it in the core repo seems like a kluge not a proper solution.
In particular, once it'd been around for awhile so that the master copy
had diverged from the back branches' copies, that would be pretty
confusing IMO.
Is it worth the trouble of having a separate repo for info that shouldn't
be tied to a particular PG development branch? I'm not quite sure what
else we would keep there besides this file, but perhaps other use-cases
will come up. Some of the stuff in src/tools/ has a bit of this flavor.
regards, tom lane
On 01.07.18 17:41, Tom Lane wrote: > I can see the value of people other than you being able to change it, > but keeping it in the core repo seems like a kluge not a proper solution. > In particular, once it'd been around for awhile so that the master copy > had diverged from the back branches' copies, that would be pretty > confusing IMO. Yeah, I'd find this kind of weird. The version control system contains the code, not the other way around. > Is it worth the trouble of having a separate repo for info that shouldn't > be tied to a particular PG development branch? I'm not quite sure what > else we would keep there besides this file, but perhaps other use-cases > will come up. Some of the stuff in src/tools/ has a bit of this flavor. The web site has some information about which versions are current, so maybe there is an opportunity to hook the buildfarm in there. (I also have to change some things on babel.postgresql.org every time a branch is made.) -- Peter Eisentraut http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
On Mon, Jul 2, 2018 at 10:39 AM, Peter Eisentraut <peter.eisentraut@2ndquadrant.com> wrote:
On 01.07.18 17:41, Tom Lane wrote:
> I can see the value of people other than you being able to change it,
> but keeping it in the core repo seems like a kluge not a proper solution.
> In particular, once it'd been around for awhile so that the master copy
> had diverged from the back branches' copies, that would be pretty
> confusing IMO.
Yeah, I'd find this kind of weird. The version control system contains
the code, not the other way around.
> Is it worth the trouble of having a separate repo for info that shouldn't
> be tied to a particular PG development branch? I'm not quite sure what
> else we would keep there besides this file, but perhaps other use-cases
> will come up. Some of the stuff in src/tools/ has a bit of this flavor.
The web site has some information about which versions are current, so
maybe there is an opportunity to hook the buildfarm in there.
(I also have to change some things on babel.postgresql.org every time a
branch is made.)
We could definitely do that on the website, if people think that's the easiest.
We could also set up a separate git repository with exactly the same set of committers as the main one (but only use the master branch there) as one option. The maintenance cost of doing so would be very close to zero as we have all the infrastructure for it already. so it'd be more about which one is more convenient to use.
On Mon, Jul 2, 2018 at 4:39 AM, Peter Eisentraut <peter.eisentraut@2ndquadrant.com> wrote: > On 01.07.18 17:41, Tom Lane wrote: >> I can see the value of people other than you being able to change it, >> but keeping it in the core repo seems like a kluge not a proper solution. >> In particular, once it'd been around for awhile so that the master copy >> had diverged from the back branches' copies, that would be pretty >> confusing IMO. > > Yeah, I'd find this kind of weird. The version control system contains > the code, not the other way around. +1. But I like the idea of using something to which there is broader access. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company
On Mon, Jul 2, 2018 at 4:45 AM, Magnus Hagander <magnus@hagander.net> wrote:
>
>
> On Mon, Jul 2, 2018 at 10:39 AM, Peter Eisentraut
> <peter.eisentraut@2ndquadrant.com> wrote:
>>
>> On 01.07.18 17:41, Tom Lane wrote:
>> > I can see the value of people other than you being able to change it,
>> > but keeping it in the core repo seems like a kluge not a proper
>> > solution.
>> > In particular, once it'd been around for awhile so that the master copy
>> > had diverged from the back branches' copies, that would be pretty
>> > confusing IMO.
>>
>> Yeah, I'd find this kind of weird. The version control system contains
>> the code, not the other way around.
>>
>> > Is it worth the trouble of having a separate repo for info that
>> > shouldn't
>> > be tied to a particular PG development branch? I'm not quite sure what
>> > else we would keep there besides this file, but perhaps other use-cases
>> > will come up. Some of the stuff in src/tools/ has a bit of this flavor.
>>
>> The web site has some information about which versions are current, so
>> maybe there is an opportunity to hook the buildfarm in there.
>>
>> (I also have to change some things on babel.postgresql.org every time a
>> branch is made.)
>
>
> We could definitely do that on the website, if people think that's the
> easiest.
>
> We could also set up a separate git repository with exactly the same set of
> committers as the main one (but only use the master branch there) as one
> option. The maintenance cost of doing so would be very close to zero as we
> have all the infrastructure for it already. so it'd be more about which one
> is more convenient to use.
>
Ideally this would be done as part of creating the new branch. Since
the web site doesn't have the same set of committers, a second metdata
repo like this seems sensible.
An alternative would be to create a special branch within the core
repo for such data, something like this (The first two lines are the
ones that are most important):
git checkout --orphan metadata
git rm --cached -r .
wget https://buildfarm.postgresql.org/branches_of_interest.txt
git add branches_of_interest.txt
git commit -m 'initial content' branches_of_interest.txt
git push origin HEAD
git checkout master
The new branch won't share any history or files with the existing branches.
cheers
andrew
--
Andrew Dunstan https://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
On Mon, Jul 2, 2018 at 8:18 AM, Andrew Dunstan <andrew.dunstan@2ndquadrant.com> wrote: > Ideally this would be done as part of creating the new branch. Since > the web site doesn't have the same set of committers, a second metdata > repo like this seems sensible. > An alternative would be to create a special branch within the core > repo for such data, something like this (The first two lines are the > ones that are most important): > > git checkout --orphan metadata > git rm --cached -r . > wget https://buildfarm.postgresql.org/branches_of_interest.txt > git add branches_of_interest.txt > git commit -m 'initial content' branches_of_interest.txt > git push origin HEAD > git checkout master > > The new branch won't share any history or files with the existing branches. Seems like too much magic to me. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company
On Mon, Jul 2, 2018 at 8:33 AM, Robert Haas <robertmhaas@gmail.com> wrote: > On Mon, Jul 2, 2018 at 8:18 AM, Andrew Dunstan > <andrew.dunstan@2ndquadrant.com> wrote: >> Ideally this would be done as part of creating the new branch. Since >> the web site doesn't have the same set of committers, a second metdata >> repo like this seems sensible. >> An alternative would be to create a special branch within the core >> repo for such data, something like this (The first two lines are the >> ones that are most important): >> >> git checkout --orphan metadata >> git rm --cached -r . >> wget https://buildfarm.postgresql.org/branches_of_interest.txt >> git add branches_of_interest.txt >> git commit -m 'initial content' branches_of_interest.txt >> git push origin HEAD >> git checkout master >> >> The new branch won't share any history or files with the existing branches. > > Seems like too much magic to me. > This is pretty much how GitHub's gh-pages docco mechanism works. It's not particularly deep magic. But if it makes people uncomfortable, let's go for a second repo. It's not worth having a huge argument over. cheers andrew -- Andrew Dunstan https://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Robert Haas <robertmhaas@gmail.com> writes:
> On Mon, Jul 2, 2018 at 8:18 AM, Andrew Dunstan
> <andrew.dunstan@2ndquadrant.com> wrote:
>> An alternative would be to create a special branch within the core
>> repo for such data, something like this (The first two lines are the
>> ones that are most important):
>> ...
>> The new branch won't share any history or files with the existing branches.
> Seems like too much magic to me.
Dunno, I was wondering yesterday whether something like that would be
possible. It'd be easier to maintain than a separate repo, for sure.
I wonder what that would look like in gitweb, though. If the website
treated it like a version branch, it'd likely be weird.
regards, tom lane
On 2018-07-01 11:41:07 -0400, Tom Lane wrote: > Andrew Dunstan <andrew@dunslane.net> writes: > > This file on the buildfarm server is used to tell clients which branches > > we'd like built. When a new stable branch is created it's added manually > > to this file, and when one gets to EOL it's removed from the file. This > > is a rather cumbersome process, and it occurred to me that it could be > > streamlined by our keeping it in the core repo instead. > > I can see the value of people other than you being able to change it, > but keeping it in the core repo seems like a kluge not a proper solution. > In particular, once it'd been around for awhile so that the master copy > had diverged from the back branches' copies, that would be pretty > confusing IMO. FWIW, I've a manually maintained version of this in the scripts I use to commit / backpatch things. I'd appreciate not having to manually maintain it, and be afraid to forget updating it ;) FWIW, I don't really see the problem of maintaining it in-tree, it has the advantage of guaranteeing the set of known-to-be-maintained branches is guaranteed to be current. Greetings, Andres Freund
Andres Freund <andres@anarazel.de> writes:
> On 2018-07-01 11:41:07 -0400, Tom Lane wrote:
>> I can see the value of people other than you being able to change it,
>> but keeping it in the core repo seems like a kluge not a proper solution.
> FWIW, I've a manually maintained version of this in the scripts I use to
> commit / backpatch things. I'd appreciate not having to manually
> maintain it, and be afraid to forget updating it ;)
Yeah, I have something similar as well, in fact a couple of them.
I thought about whether I'd change those scripts to use an in-tree
branches_of_interest list, and I concluded that most likely I would
not, because the instant at which I decide a given branch is no longer
interesting to me is not necessarily the same instant at which we
shut down the buildfarm support for it. It has even less to do with
the time that I next do a "git pull" after that shutdown. Contrariwise,
once a new branch has been created in the repo, I need to set up a workdir
for it before it can safely be added to the list of branches I auto
push/pull. So giving up control of the timing just isn't gonna work well.
If we were to keep this info in a separate repo requiring a separate
"git pull", it might be manageable since I could control when I update
that. But then you're right back to the situation of requiring a manual
step you might forget.
In any case, if we do put this into the tree, I want it to be named
and treated as something that *only* controls the buildfarm, not any
other stuff. We'll regret tying other stuff to that.
regards, tom lane