Thread: Git cvsserver serious issue

Git cvsserver serious issue

From
Magnus Hagander
Date:
So, I found (with some helpful hints from Robert who caught the final
nail in the coffin) a good reason why we really can't run a
git-cvsserver globally.

Any user can point their cvs client at the repository. And check out
an arbitrary branch, tag *or individual commit*. Doing so will create
a 50Mb sqlite database on the server with cache information about that
head.

That basically means that git-cvsserver is completely useless in a
public scenario as it stands. An easier way to DOS our server is hard
to find, really.

Now, if we can limit this by IP address, that would be ok. I assume we
can do this for the NLS stuff - peter?

As for buildfarm members needing CVS - is it workable to require that
the maintainers of these set up their own git clone with git cvsserver
(over ssh or pserver) and restrict it locally to the IP(s) of their
machines?

--
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/


Re: Git cvsserver serious issue

From
Tom Lane
Date:
Magnus Hagander <magnus@hagander.net> writes:
> Any user can point their cvs client at the repository. And check out
> an arbitrary branch, tag *or individual commit*. Doing so will create
> a 50Mb sqlite database on the server with cache information about that
> head.

> That basically means that git-cvsserver is completely useless in a
> public scenario as it stands. An easier way to DOS our server is hard
> to find, really.

Ugh.

> Now, if we can limit this by IP address, that would be ok. I assume we
> can do this for the NLS stuff - peter?

> As for buildfarm members needing CVS - is it workable to require that
> the maintainers of these set up their own git clone with git cvsserver
> (over ssh or pserver) and restrict it locally to the IP(s) of their
> machines?

If we're going to let people in by IP address, maybe we could let legacy
buildfarm members in by IP address.  It doesn't seem particularly
helpful to expect each buildfarm owner to solve this problem for
themselves.  I'd also note that if they could run git locally, they
wouldn't be needing cvsserver in the first place.

Also, couldn't we just set up the cvsserver on its own VM with a limited
amount of disk space, and not worry too much about any "DOS threat"?
If somebody does do this, block them and reinitialize that server.
        regards, tom lane


Re: Git cvsserver serious issue

From
Magnus Hagander
Date:
On Wed, Sep 22, 2010 at 16:23, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> Magnus Hagander <magnus@hagander.net> writes:
>> Any user can point their cvs client at the repository. And check out
>> an arbitrary branch, tag *or individual commit*. Doing so will create
>> a 50Mb sqlite database on the server with cache information about that
>> head.
>
>> That basically means that git-cvsserver is completely useless in a
>> public scenario as it stands. An easier way to DOS our server is hard
>> to find, really.
>
> Ugh.

Indeed.


>> Now, if we can limit this by IP address, that would be ok. I assume we
>> can do this for the NLS stuff - peter?
>
>> As for buildfarm members needing CVS - is it workable to require that
>> the maintainers of these set up their own git clone with git cvsserver
>> (over ssh or pserver) and restrict it locally to the IP(s) of their
>> machines?
>
> If we're going to let people in by IP address, maybe we could let legacy
> buildfarm members in by IP address.  It doesn't seem particularly
> helpful to expect each buildfarm owner to solve this problem for
> themselves.  I'd also note that if they could run git locally, they
> wouldn't be needing cvsserver in the first place.

We could. It's currently on a freebsd vm though and I don't think we
can set per-server IP filters on those. (I was thinking iptables). We
could move it though - it doesn't *have* to be on the anonymous git
VM. It's just some extra resources.

Well, the use-case I was thinking of was Stefan. While he can't run
git on each and every animal, he certainly has *some* machine(s) on
the correct side of whatever firewall there may be that can run git.



> Also, couldn't we just set up the cvsserver on its own VM with a limited
> amount of disk space, and not worry too much about any "DOS threat"?
> If somebody does do this, block them and reinitialize that server.

We could do that, but that could end up fighting a losing battle in
case some bot hits it.

I don't like deploying something with a known issue on it, sandboxed or not.

--
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/


Re: Git cvsserver serious issue

From
Andrew Dunstan
Date:

On 09/22/2010 10:23 AM, Tom Lane wrote:
> If we're going to let people in by IP address, maybe we could let legacy
> buildfarm members in by IP address.  It doesn't seem particularly
> helpful to expect each buildfarm owner to solve this problem for
> themselves.  I'd also note that if they could run git locally, they
> wouldn't be needing cvsserver in the first place.
>
> Also, couldn't we just set up the cvsserver on its own VM with a limited
> amount of disk space, and not worry too much about any "DOS threat"?
> If somebody does do this, block them and reinitialize that server.

I'm not convinced we need any such thing yet. 13 of the 38 animals that 
have reported in the last 5 days are using git already (OK, factoring 
out my animals that's 8 out of 33). I'm going to send out email in a few 
days prodding people to migrate. Let's see how far we get.

cheers

andrew


Re: Git cvsserver serious issue

From
Tom Lane
Date:
Andrew Dunstan <andrew@dunslane.net> writes:
> On 09/22/2010 10:23 AM, Tom Lane wrote:
>> If we're going to let people in by IP address, maybe we could let legacy
>> buildfarm members in by IP address.  It doesn't seem particularly
>> helpful to expect each buildfarm owner to solve this problem for
>> themselves.  I'd also note that if they could run git locally, they
>> wouldn't be needing cvsserver in the first place.

> I'm not convinced we need any such thing yet. 13 of the 38 animals that 
> have reported in the last 5 days are using git already (OK, factoring 
> out my animals that's 8 out of 33). I'm going to send out email in a few 
> days prodding people to migrate. Let's see how far we get.

Even if we get 100% compliance on the buildfarm side, Peter's already
stated that moving the NLS support over to git is going to take more
time than we have available right now.  We need a cvsserver for awhile
yet.  We can't just suddenly announce "CVS service is terminated as of
yesterday" and expect that that's not going to have any serious
consequences.

Is there anything we could do to patch the problem out of git-cvsserver?
Maybe hack it to only accept requests for the active branch tips?
        regards, tom lane


Re: Git cvsserver serious issue

From
Peter Eisentraut
Date:
On ons, 2010-09-22 at 16:03 +0200, Magnus Hagander wrote:
> That basically means that git-cvsserver is completely useless in a
> public scenario as it stands. An easier way to DOS our server is hard
> to find, really.
> 
> Now, if we can limit this by IP address, that would be ok. I assume we
> can do this for the NLS stuff - peter?

Well, let's see.  If someone can figure out the git equivalent of

if cvs -q update | egrep -q '^(U|P) '; then # ... something changed, so run the update ...
fi

(assuming, for simplicity, that the current directory has the
appropriate branch checked out already)

then I might be able to get this fixed.




Re: Git cvsserver serious issue

From
Robert Haas
Date:
On Wed, Sep 22, 2010 at 12:21 PM, Peter Eisentraut <peter_e@gmx.net> wrote:
> On ons, 2010-09-22 at 16:03 +0200, Magnus Hagander wrote:
>> That basically means that git-cvsserver is completely useless in a
>> public scenario as it stands. An easier way to DOS our server is hard
>> to find, really.
>>
>> Now, if we can limit this by IP address, that would be ok. I assume we
>> can do this for the NLS stuff - peter?
>
> Well, let's see.  If someone can figure out the git equivalent of
>
> if cvs -q update | egrep -q '^(U|P) '; then
>  # ... something changed, so run the update ...
> fi
>
> (assuming, for simplicity, that the current directory has the
> appropriate branch checked out already)
>
> then I might be able to get this fixed.

Can you just check whether the commit SHA of HEAD has changed?  e.g.

git show-ref --heads -s master
git log --format=format:%H -n 1 master

...and compare with previous results of same?

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise Postgres Company


Re: Git cvsserver serious issue

From
Alvaro Herrera
Date:
Excerpts from Peter Eisentraut's message of mié sep 22 12:21:45 -0400 2010:

> Well, let's see.  If someone can figure out the git equivalent of
> 
> if cvs -q update | egrep -q '^(U|P) '; then
>   # ... something changed, so run the update ...
> fi
> 
> (assuming, for simplicity, that the current directory has the
> appropriate branch checked out already)
> 
> then I might be able to get this fixed.

Would it work to save the previous commit hash in a file and compare to
the current one?

-- 
Álvaro Herrera <alvherre@commandprompt.com>
The PostgreSQL Company - Command Prompt, Inc.
PostgreSQL Replication, Consulting, Custom Development, 24x7 support


Re: Git cvsserver serious issue

From
Abhijit Menon-Sen
Date:
At 2010-09-22 19:21:45 +0300, peter_e@gmx.net wrote:
>
> Well, let's see.  If someone can figure out the git equivalent of
> 
> if cvs -q update | egrep -q '^(U|P) '; then
>   # ... something changed, so run the update ...
> fi

I think you want:
   git pull   if [ $(git rev-parse HEAD) != $(git rev-parse ORIG_HEAD) ]; then       # ... the pull changed something
...  fi
 

-- ams


Re: Git cvsserver serious issue

From
Andrew Dunstan
Date:

On 09/22/2010 10:26 AM, Magnus Hagander wrote:
> On Wed, Sep 22, 2010 at 16:23, Tom Lane<tgl@sss.pgh.pa.us>  wrote:
>> Magnus Hagander<magnus@hagander.net>  writes:
>>> Any user can point their cvs client at the repository. And check out
>>> an arbitrary branch, tag *or individual commit*. Doing so will create
>>> a 50Mb sqlite database on the server with cache information about that
>>> head.
>>> That basically means that git-cvsserver is completely useless in a
>>> public scenario as it stands. An easier way to DOS our server is hard
>>> to find, really.
>> Ugh.
> Indeed.
>
>
>>> Now, if we can limit this by IP address, that would be ok. I assume we
>>> can do this for the NLS stuff - peter?
>>> As for buildfarm members needing CVS - is it workable to require that
>>> the maintainers of these set up their own git clone with git cvsserver
>>> (over ssh or pserver) and restrict it locally to the IP(s) of their
>>> machines?
>> If we're going to let people in by IP address, maybe we could let legacy
>> buildfarm members in by IP address.  It doesn't seem particularly
>> helpful to expect each buildfarm owner to solve this problem for
>> themselves.  I'd also note that if they could run git locally, they
>> wouldn't be needing cvsserver in the first place.
> We could. It's currently on a freebsd vm though and I don't think we
> can set per-server IP filters on those. (I was thinking iptables). We
> could move it though - it doesn't *have* to be on the anonymous git
> VM. It's just some extra resources.
>
> Well, the use-case I was thinking of was Stefan. While he can't run
> git on each and every animal, he certainly has *some* machine(s) on
> the correct side of whatever firewall there may be that can run git.
>
>
>
>> Also, couldn't we just set up the cvsserver on its own VM with a limited
>> amount of disk space, and not worry too much about any "DOS threat"?
>> If somebody does do this, block them and reinitialize that server.
> We could do that, but that could end up fighting a losing battle in
> case some bot hits it.
>
> I don't like deploying something with a known issue on it, sandboxed or not.
>

Thinking about this some more, how about we do non-anonymous CVS over 
SSH access to the git-cvsserver for the few buildfarm members that can't 
currently handle using git (e.g. spoonbill)?

I'm not sure if that would handle other requirements, such as Peter's, 
but I hope the residual requirements for CVS support will be pretty rare.

cheers

andrew


Re: Git cvsserver serious issue

From
Magnus Hagander
Date:
On Thu, Sep 23, 2010 at 04:59, Andrew Dunstan <andrew@dunslane.net> wrote:
>>> Also, couldn't we just set up the cvsserver on its own VM with a limited
>>> amount of disk space, and not worry too much about any "DOS threat"?
>>> If somebody does do this, block them and reinitialize that server.
>>
>> We could do that, but that could end up fighting a losing battle in
>> case some bot hits it.
>>
>> I don't like deploying something with a known issue on it, sandboxed or
>> not.
>>
>
> Thinking about this some more, how about we do non-anonymous CVS over SSH
> access to the git-cvsserver for the few buildfarm members that can't
> currently handle using git (e.g. spoonbill)?

Well, if we do that centrally, we are back to a dedicated VM (hint:
we're most certainly not adding non-personal no-password accounts to
one of the VMs used for critical services - it's bad enough we have
Bruce's account there :P).

I assume most buildfarm clients are off static IPs (at least as seen
from the servers - they may be behind a NAT device, but that one
having static out)? If so, it seems simply easier to use pserver...


> I'm not sure if that would handle other requirements, such as Peter's, but I
> hope the residual requirements for CVS support will be pretty rare.

Just to be sure - do we have any other requirements for CVS *beyond*
buildfarm and NLS that we're not thinking of here?

--
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/


Re: Git cvsserver serious issue

From
Andrew Dunstan
Date:

On 09/23/2010 02:09 AM, Magnus Hagander wrote:
> On Thu, Sep 23, 2010 at 04:59, Andrew Dunstan<andrew@dunslane.net>  wrote:
>>>> Also, couldn't we just set up the cvsserver on its own VM with a limited
>>>> amount of disk space, and not worry too much about any "DOS threat"?
>>>> If somebody does do this, block them and reinitialize that server.
>>> We could do that, but that could end up fighting a losing battle in
>>> case some bot hits it.
>>>
>>> I don't like deploying something with a known issue on it, sandboxed or
>>> not.
>>>
>> Thinking about this some more, how about we do non-anonymous CVS over SSH
>> access to the git-cvsserver for the few buildfarm members that can't
>> currently handle using git (e.g. spoonbill)?
> Well, if we do that centrally, we are back to a dedicated VM (hint:
> we're most certainly not adding non-personal no-password accounts to
> one of the VMs used for critical services - it's bad enough we have
> Bruce's account there :P).
>
> I assume most buildfarm clients are off static IPs (at least as seen
> from the servers - they may be behind a NAT device, but that one
> having static out)? If so, it seems simply easier to use pserver...
>

Yes, I think we should have a VM. Is that so hard to do in these days of 
Xen etc? I'm surprised we can't run up a VM pretty much at the drop of a 
hat.

I was suggesting that the accounts would be protected using ssh keys. 
Password and IP address protection seem pretty weak to me. Passwords can 
be sniffed or attacked using brute force. IP addresses can be spoofed. 
But you're the SA, not me.

cheers

andrew


Re: Git cvsserver serious issue

From
Magnus Hagander
Date:
On Thu, Sep 23, 2010 at 11:27, Andrew Dunstan <andrew@dunslane.net> wrote:
>
>
> On 09/23/2010 02:09 AM, Magnus Hagander wrote:
>>
>> On Thu, Sep 23, 2010 at 04:59, Andrew Dunstan<andrew@dunslane.net>  wrote:
>>>>>
>>>>> Also, couldn't we just set up the cvsserver on its own VM with a
>>>>> limited
>>>>> amount of disk space, and not worry too much about any "DOS threat"?
>>>>> If somebody does do this, block them and reinitialize that server.
>>>>
>>>> We could do that, but that could end up fighting a losing battle in
>>>> case some bot hits it.
>>>>
>>>> I don't like deploying something with a known issue on it, sandboxed or
>>>> not.
>>>>
>>> Thinking about this some more, how about we do non-anonymous CVS over SSH
>>> access to the git-cvsserver for the few buildfarm members that can't
>>> currently handle using git (e.g. spoonbill)?
>>
>> Well, if we do that centrally, we are back to a dedicated VM (hint:
>> we're most certainly not adding non-personal no-password accounts to
>> one of the VMs used for critical services - it's bad enough we have
>> Bruce's account there :P).
>>
>> I assume most buildfarm clients are off static IPs (at least as seen
>> from the servers - they may be behind a NAT device, but that one
>> having static out)? If so, it seems simply easier to use pserver...
>>
>
> Yes, I think we should have a VM. Is that so hard to do in these days of Xen
> etc? I'm surprised we can't run up a VM pretty much at the drop of a hat.

In the new infrastructure, it is.

The main resource that's limited really is IP addresses ;)


> I was suggesting that the accounts would be protected using ssh keys.
> Password and IP address protection seem pretty weak to me. Passwords can be
> sniffed or attacked using brute force. IP addresses can be spoofed. But
> you're the SA, not me.

I prefer keys. But I don't want those users on the same VM as
important services, because passphrase-less keys are a lot more likely
to be compromised than the keys of say a regular committer...


--
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/


Re: Git cvsserver serious issue

From
Bruce Momjian
Date:
Magnus Hagander wrote:
> >> I assume most buildfarm clients are off static IPs (at least as seen
> >> from the servers - they may be behind a NAT device, but that one
> >> having static out)? If so, it seems simply easier to use pserver...
> >>
> >
> > Yes, I think we should have a VM. Is that so hard to do in these days of Xen
> > etc? I'm surprised we can't run up a VM pretty much at the drop of a hat.
> 
> In the new infrastructure, it is.
> 
> The main resource that's limited really is IP addresses ;)
> 
> 
> > I was suggesting that the accounts would be protected using ssh keys.
> > Password and IP address protection seem pretty weak to me. Passwords can be
> > sniffed or attacked using brute force. IP addresses can be spoofed. But
> > you're the SA, not me.
> 
> I prefer keys. But I don't want those users on the same VM as
> important services, because passphrase-less keys are a lot more likely
> to be compromised than the keys of say a regular committer...

Stupid question, but can't we just create a CVSROOT fed from git, and
use the normal CVS server to feed sites?

--  Bruce Momjian  <bruce@momjian.us>        http://momjian.us EnterpriseDB
http://enterprisedb.com
 + It's impossible for everything to be true. +


Re: Git cvsserver serious issue

From
Andrew Dunstan
Date:

On 09/23/2010 09:55 AM, Bruce Momjian wrote:

>   
> Stupid question, but can't we just create a CVSROOT fed from git, and
> use the normal CVS server to feed sites?
>

Where is it going to get the ,v files that CVS uses from? git-cvsserver 
emulates a CVS server from git. It doesn't create a CVS repository that 
CVS in server mode could use.

cheers

andrew


Re: Git cvsserver serious issue

From
Bruce Momjian
Date:
Andrew Dunstan wrote:
> 
> 
> On 09/23/2010 09:55 AM, Bruce Momjian wrote:
> 
> >   
> > Stupid question, but can't we just create a CVSROOT fed from git, and
> > use the normal CVS server to feed sites?
> >
> 
> Where is it going to get the ,v files that CVS uses from? git-cvsserver 
> emulates a CVS server from git. It doesn't create a CVS repository that 
> CVS in server mode could use.

I suppose there is no way to pull diffs from git continuously and commit
them to CVS.

--  Bruce Momjian  <bruce@momjian.us>        http://momjian.us EnterpriseDB
http://enterprisedb.com
 + It's impossible for everything to be true. +


Re: Git cvsserver serious issue

From
Magnus Hagander
Date:
On Thu, Sep 23, 2010 at 16:11, Bruce Momjian <bruce@momjian.us> wrote:
> Andrew Dunstan wrote:
>>
>>
>> On 09/23/2010 09:55 AM, Bruce Momjian wrote:
>>
>> >
>> > Stupid question, but can't we just create a CVSROOT fed from git, and
>> > use the normal CVS server to feed sites?
>> >
>>
>> Where is it going to get the ,v files that CVS uses from? git-cvsserver
>> emulates a CVS server from git. It doesn't create a CVS repository that
>> CVS in server mode could use.
>
> I suppose there is no way to pull diffs from git continuously and commit
> them to CVS.

You probably could. I'm very doubtful it's worth it...


--
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/


Re: Git cvsserver serious issue

From
David Fetter
Date:
On Thu, Sep 23, 2010 at 04:38:27PM +0200, Magnus Hagander wrote:
> On Thu, Sep 23, 2010 at 16:11, Bruce Momjian <bruce@momjian.us> wrote:
> > Andrew Dunstan wrote:
> >> On 09/23/2010 09:55 AM, Bruce Momjian wrote:
> >>
> >> > Stupid question, but can't we just create a CVSROOT fed from
> >> > git, and use the normal CVS server to feed sites?
> >>
> >> Where is it going to get the ,v files that CVS uses from?
> >> git-cvsserver emulates a CVS server from git. It doesn't create a
> >> CVS repository that CVS in server mode could use.
> >
> > I suppose there is no way to pull diffs from git continuously and
> > commit them to CVS.
> 
> You probably could. I'm very doubtful it's worth it...

Back to a question you asked earlier, what exactly still depends on
CVS right now, as in which buildfarm animals, what parts of the NLS
processes?  Also as you asked earlier, what else?

Cheers,
David.
-- 
David Fetter <david@fetter.org> http://fetter.org/
Phone: +1 415 235 3778  AIM: dfetter666  Yahoo!: dfetter
Skype: davidfetter      XMPP: david.fetter@gmail.com
iCal: webcal://www.tripit.com/feed/ical/people/david74/tripit.ics

Remember to vote!
Consider donating to Postgres: http://www.postgresql.org/about/donate


Re: Git cvsserver serious issue

From
Tom Lane
Date:
Magnus Hagander <magnus@hagander.net> writes:
> So, I found (with some helpful hints from Robert who caught the final
> nail in the coffin) a good reason why we really can't run a
> git-cvsserver globally.

> Any user can point their cvs client at the repository. And check out
> an arbitrary branch, tag *or individual commit*. Doing so will create
> a 50Mb sqlite database on the server with cache information about that
> head.

I'm still wondering why we don't simply lobotomize git-cvsserver to
refuse requests to check out anything except the active branch tips.
It's only a Perl script.  I could probably hack it in an hour,
there are those here who could do it in ten minutes.
        regards, tom lane


Re: Git cvsserver serious issue

From
Andrew Dunstan
Date:

On 09/23/2010 10:58 AM, David Fetter wrote:
> Back to a question you asked earlier, what exactly still depends on
> CVS right now, as in which buildfarm animals, what parts of the NLS
> processes?  Also as you asked earlier, what else?

At least one buildfarm member, spoonbill, is known to have issues with 
git. This machine, because of the flags it uses, has found numerous bugs 
for us in the past and is quite important for us to maintain (a direct 
counter argument to your suggestion of dropping platforms that don't 
support git).

cheers

andrew




Re: Git cvsserver serious issue

From
Magnus Hagander
Date:
On Thu, Sep 23, 2010 at 17:16, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> Magnus Hagander <magnus@hagander.net> writes:
>> So, I found (with some helpful hints from Robert who caught the final
>> nail in the coffin) a good reason why we really can't run a
>> git-cvsserver globally.
>
>> Any user can point their cvs client at the repository. And check out
>> an arbitrary branch, tag *or individual commit*. Doing so will create
>> a 50Mb sqlite database on the server with cache information about that
>> head.
>
> I'm still wondering why we don't simply lobotomize git-cvsserver to
> refuse requests to check out anything except the active branch tips.
> It's only a Perl script.  I could probably hack it in an hour,
> there are those here who could do it in ten minutes.

Yeah, that would not be a bad idea - if someone can do it who feels
comfortable doing it :-)

I could probably hack it up as well, but I wouldn't trust myself to
have convered all the bases.



--
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/


Re: Git cvsserver serious issue

From
David Fetter
Date:
On Thu, Sep 23, 2010 at 11:17:35AM -0400, Andrew Dunstan wrote:
> 
> 
> On 09/23/2010 10:58 AM, David Fetter wrote:
> >Back to a question you asked earlier, what exactly still depends on
> >CVS right now, as in which buildfarm animals, what parts of the NLS
> >processes?  Also as you asked earlier, what else?
> 
> At least one buildfarm member, spoonbill, is known to have issues
> with git.

Do those issues appear fixable?

Cheers,
David.
-- 
David Fetter <david@fetter.org> http://fetter.org/
Phone: +1 415 235 3778  AIM: dfetter666  Yahoo!: dfetter
Skype: davidfetter      XMPP: david.fetter@gmail.com
iCal: webcal://www.tripit.com/feed/ical/people/david74/tripit.ics

Remember to vote!
Consider donating to Postgres: http://www.postgresql.org/about/donate


Re: Git cvsserver serious issue

From
Andrew Dunstan
Date:

On 09/23/2010 11:18 AM, Magnus Hagander wrote:
> On Thu, Sep 23, 2010 at 17:16, Tom Lane<tgl@sss.pgh.pa.us>  wrote:
>> Magnus Hagander<magnus@hagander.net>  writes:
>>> So, I found (with some helpful hints from Robert who caught the final
>>> nail in the coffin) a good reason why we really can't run a
>>> git-cvsserver globally.
>>> Any user can point their cvs client at the repository. And check out
>>> an arbitrary branch, tag *or individual commit*. Doing so will create
>>> a 50Mb sqlite database on the server with cache information about that
>>> head.
>> I'm still wondering why we don't simply lobotomize git-cvsserver to
>> refuse requests to check out anything except the active branch tips.
>> It's only a Perl script.  I could probably hack it in an hour,
>> there are those here who could do it in ten minutes.
> Yeah, that would not be a bad idea - if someone can do it who feels
> comfortable doing it :-)
>
> I could probably hack it up as well, but I wouldn't trust myself to
> have convered all the bases.
>

Are we sure that's going to stop the DOS issue?

cheers

andrew


Re: Git cvsserver serious issue

From
Stefan Kaltenbrunner
Date:
David Fetter wrote:
> On Thu, Sep 23, 2010 at 11:17:35AM -0400, Andrew Dunstan wrote:
>>
>> On 09/23/2010 10:58 AM, David Fetter wrote:
>>> Back to a question you asked earlier, what exactly still depends on
>>> CVS right now, as in which buildfarm animals, what parts of the NLS
>>> processes?  Also as you asked earlier, what else?
>> At least one buildfarm member, spoonbill, is known to have issues
>> with git.
> 
> Do those issues appear fixable?

maybe but I have absolutely no time to investigate in the next few days.



Stefan


Re: Git cvsserver serious issue

From
Magnus Hagander
Date:
On Thu, Sep 23, 2010 at 17:32, Andrew Dunstan <andrew@dunslane.net> wrote:
>
>
> On 09/23/2010 11:18 AM, Magnus Hagander wrote:
>>
>> On Thu, Sep 23, 2010 at 17:16, Tom Lane<tgl@sss.pgh.pa.us>  wrote:
>>>
>>> Magnus Hagander<magnus@hagander.net>  writes:
>>>>
>>>> So, I found (with some helpful hints from Robert who caught the final
>>>> nail in the coffin) a good reason why we really can't run a
>>>> git-cvsserver globally.
>>>> Any user can point their cvs client at the repository. And check out
>>>> an arbitrary branch, tag *or individual commit*. Doing so will create
>>>> a 50Mb sqlite database on the server with cache information about that
>>>> head.
>>>
>>> I'm still wondering why we don't simply lobotomize git-cvsserver to
>>> refuse requests to check out anything except the active branch tips.
>>> It's only a Perl script.  I could probably hack it in an hour,
>>> there are those here who could do it in ten minutes.
>>
>> Yeah, that would not be a bad idea - if someone can do it who feels
>> comfortable doing it :-)
>>
>> I could probably hack it up as well, but I wouldn't trust myself to
>> have convered all the bases.
>>
>
> Are we sure that's going to stop the DOS issue?

As long as it's done right, I don't see how it wouldn't.


--
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/


Re: Git cvsserver serious issue

From
Tom Lane
Date:
Andrew Dunstan <andrew@dunslane.net> writes:
>> On Thu, Sep 23, 2010 at 17:16, Tom Lane<tgl@sss.pgh.pa.us>  wrote:
>>> I'm still wondering why we don't simply lobotomize git-cvsserver to
>>> refuse requests to check out anything except the active branch tips.

> Are we sure that's going to stop the DOS issue?

The claimed denial of service is that each checkout target requires a
separate SQLite database.  Limit the number of checkout targets accepted
and you're done.  Or at least, if you're not done, it behooves those
claiming there's a security problem to show what the problem is.  It's
not like this piece of software isn't used in production, so I doubt
it needs to be babied quite as much as this thread is assuming.
        regards, tom lane


Re: Git cvsserver serious issue

From
Tom Lane
Date:
Magnus Hagander <magnus@hagander.net> writes:
> On Thu, Sep 23, 2010 at 17:32, Andrew Dunstan <andrew@dunslane.net> wrote:
>> Are we sure that's going to stop the DOS issue?

> As long as it's done right, I don't see how it wouldn't.

There might be a cleaner way to do it, but after a moment's inspection
of the script, I'd be inclined to just hack GITCVS::updater->new() to
throw error if $module is neither "master" nor "REL\d_\d_STABLE".
Keep in mind of course that I'm a lousy Perl coder.
        regards, tom lane


Re: Git cvsserver serious issue

From
Aidan Van Dyk
Date:
On Thu, Sep 23, 2010 at 11:49 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> Magnus Hagander <magnus@hagander.net> writes:
>> On Thu, Sep 23, 2010 at 17:32, Andrew Dunstan <andrew@dunslane.net> wrote:
>>> Are we sure that's going to stop the DOS issue?
>
>> As long as it's done right, I don't see how it wouldn't.
>
> There might be a cleaner way to do it, but after a moment's inspection
> of the script, I'd be inclined to just hack GITCVS::updater->new() to
> throw error if $module is neither "master" nor "REL\d_\d_STABLE".
> Keep in mind of course that I'm a lousy Perl coder.

Here's a quick change that will allow you to specifig a "modules" in
the gitcvs section to export, like:   [gitcvs]        enabled=1        modules=master,REL9_0_STABLE,REL8_4_STABLE

--- git-cvsserver.orig  2010-09-23 12:03:06.000000000 -0400
+++ git-cvsserver       2010-09-23 13:16:53.000000000 -0400
@@ -2771,6 +2771,12 @@
    die "Git repo '$self->{git_path}' doesn't exist" unless ( -d
$self->{git_path} );

+    if (defined $cfg->{gitcvs}{modules})
+    {
+        $log->debug("Limitting modules: ". $cfg->{gitcvs}{modules});
+        die "Invalid module $module" unless map {/^ *$module$/}
split(',', $cfg->{gitcvs}{modules});
+    }
+    $self->{dbdriver} = $cfg->{gitcvs}{$state->{method}}{dbdriver} ||        $cfg->{gitcvs}{dbdriver} || "SQLite";
$self->{dbname}= $cfg->{gitcvs}{$state->{method}}{dbname} ||
 


Re: Git cvsserver serious issue

From
Andrew Dunstan
Date:

On 09/23/2010 01:18 PM, Aidan Van Dyk wrote:
> On Thu, Sep 23, 2010 at 11:49 AM, Tom Lane<tgl@sss.pgh.pa.us>  wrote:
>> Magnus Hagander<magnus@hagander.net>  writes:
>>> On Thu, Sep 23, 2010 at 17:32, Andrew Dunstan<andrew@dunslane.net>  wrote:
>>>> Are we sure that's going to stop the DOS issue?
>>> As long as it's done right, I don't see how it wouldn't.
>> There might be a cleaner way to do it, but after a moment's inspection
>> of the script, I'd be inclined to just hack GITCVS::updater->new() to
>> throw error if $module is neither "master" nor "REL\d_\d_STABLE".
>> Keep in mind of course that I'm a lousy Perl coder.
> Here's a quick change that will allow you to specifig a "modules" in
> the gitcvs section to export, like:
>      [gitcvs]
>           enabled=1
>           modules=master,REL9_0_STABLE,REL8_4_STABLE
>
> --- git-cvsserver.orig  2010-09-23 12:03:06.000000000 -0400
> +++ git-cvsserver       2010-09-23 13:16:53.000000000 -0400
> @@ -2771,6 +2771,12 @@
>
>       die "Git repo '$self->{git_path}' doesn't exist" unless ( -d
> $self->{git_path} );
>
> +    if (defined $cfg->{gitcvs}{modules})
> +    {
> +        $log->debug("Limitting modules: ". $cfg->{gitcvs}{modules});
> +        die "Invalid module $module" unless map {/^ *$module$/}
> split(',', $cfg->{gitcvs}{modules});
> +    }
> +
>       $self->{dbdriver} = $cfg->{gitcvs}{$state->{method}}{dbdriver} ||
>           $cfg->{gitcvs}{dbdriver} || "SQLite";
>       $self->{dbname} = $cfg->{gitcvs}{$state->{method}}{dbname} ||


OK, let's go with that. I was kinda hoping that we wouldn't have to do 
this at all, but Stefan has been having serious problems getting git to 
build and run on spoonbill, and I don't want to take up more of his time 
or be without it for very long.

cheers

andrew


Re: Git cvsserver serious issue

From
Magnus Hagander
Date:
On Thu, Oct 7, 2010 at 15:16, Andrew Dunstan <andrew@dunslane.net> wrote:
>
>
> On 09/23/2010 01:18 PM, Aidan Van Dyk wrote:
>>
>> On Thu, Sep 23, 2010 at 11:49 AM, Tom Lane<tgl@sss.pgh.pa.us>  wrote:
>>>
>>> Magnus Hagander<magnus@hagander.net>  writes:
>>>>
>>>> On Thu, Sep 23, 2010 at 17:32, Andrew Dunstan<andrew@dunslane.net>
>>>>  wrote:
>>>>>
>>>>> Are we sure that's going to stop the DOS issue?
>>>>
>>>> As long as it's done right, I don't see how it wouldn't.
>>>
>>> There might be a cleaner way to do it, but after a moment's inspection
>>> of the script, I'd be inclined to just hack GITCVS::updater->new() to
>>> throw error if $module is neither "master" nor "REL\d_\d_STABLE".
>>> Keep in mind of course that I'm a lousy Perl coder.
>>
>> Here's a quick change that will allow you to specifig a "modules" in
>> the gitcvs section to export, like:
>>     [gitcvs]
>>          enabled=1
>>          modules=master,REL9_0_STABLE,REL8_4_STABLE
>>
>> --- git-cvsserver.orig  2010-09-23 12:03:06.000000000 -0400
>> +++ git-cvsserver       2010-09-23 13:16:53.000000000 -0400
>> @@ -2771,6 +2771,12 @@
>>
>>      die "Git repo '$self->{git_path}' doesn't exist" unless ( -d
>> $self->{git_path} );
>>
>> +    if (defined $cfg->{gitcvs}{modules})
>> +    {
>> +        $log->debug("Limitting modules: ". $cfg->{gitcvs}{modules});
>> +        die "Invalid module $module" unless map {/^ *$module$/}
>> split(',', $cfg->{gitcvs}{modules});
>> +    }
>> +
>>      $self->{dbdriver} = $cfg->{gitcvs}{$state->{method}}{dbdriver} ||
>>          $cfg->{gitcvs}{dbdriver} || "SQLite";
>>      $self->{dbname} = $cfg->{gitcvs}{$state->{method}}{dbname} ||
>
>
> OK, let's go with that. I was kinda hoping that we wouldn't have to do this
> at all, but Stefan has been having serious problems getting git to build and
> run on spoonbill, and I don't want to take up more of his time or be without
> it for very long.

Are we doing this *just* for spoonbill? If so, it's a lot easier to
just filter-by-IP, so we don't have to maintain a patched version...

--
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/


Re: Git cvsserver serious issue

From
Andrew Dunstan
Date:

On 10/07/2010 09:44 AM, Magnus Hagander wrote:
> On Thu, Oct 7, 2010 at 15:16, Andrew Dunstan<andrew@dunslane.net>  wrote:
>>
>> On 09/23/2010 01:18 PM, Aidan Van Dyk wrote:
>>> On Thu, Sep 23, 2010 at 11:49 AM, Tom Lane<tgl@sss.pgh.pa.us>    wrote:
>>>> Magnus Hagander<magnus@hagander.net>    writes:
>>>>> On Thu, Sep 23, 2010 at 17:32, Andrew Dunstan<andrew@dunslane.net>
>>>>>   wrote:
>>>>>> Are we sure that's going to stop the DOS issue?
>>>>> As long as it's done right, I don't see how it wouldn't.
>>>> There might be a cleaner way to do it, but after a moment's inspection
>>>> of the script, I'd be inclined to just hack GITCVS::updater->new() to
>>>> throw error if $module is neither "master" nor "REL\d_\d_STABLE".
>>>> Keep in mind of course that I'm a lousy Perl coder.
>>> Here's a quick change that will allow you to specifig a "modules" in
>>> the gitcvs section to export, like:
>>>      [gitcvs]
>>>           enabled=1
>>>           modules=master,REL9_0_STABLE,REL8_4_STABLE
>>>
>>> --- git-cvsserver.orig  2010-09-23 12:03:06.000000000 -0400
>>> +++ git-cvsserver       2010-09-23 13:16:53.000000000 -0400
>>> @@ -2771,6 +2771,12 @@
>>>
>>>       die "Git repo '$self->{git_path}' doesn't exist" unless ( -d
>>> $self->{git_path} );
>>>
>>> +    if (defined $cfg->{gitcvs}{modules})
>>> +    {
>>> +        $log->debug("Limitting modules: ". $cfg->{gitcvs}{modules});
>>> +        die "Invalid module $module" unless map {/^ *$module$/}
>>> split(',', $cfg->{gitcvs}{modules});
>>> +    }
>>> +
>>>       $self->{dbdriver} = $cfg->{gitcvs}{$state->{method}}{dbdriver} ||
>>>           $cfg->{gitcvs}{dbdriver} || "SQLite";
>>>       $self->{dbname} = $cfg->{gitcvs}{$state->{method}}{dbname} ||
>>
>> OK, let's go with that. I was kinda hoping that we wouldn't have to do this
>> at all, but Stefan has been having serious problems getting git to build and
>> run on spoonbill, and I don't want to take up more of his time or be without
>> it for very long.
> Are we doing this *just* for spoonbill? If so, it's a lot easier to
> just filter-by-IP, so we don't have to maintain a patched version...
>

That's the only one I know of - there could certainly be others - but if 
we're going to support continued CVS use I want to be able to test it, 
and I don't have a static IP address. Can you filter by name lookup? I 
have a dyndns name.

OTOH, this patch seems pretty small and simple to maintain.

cheers

andrew


Re: Git cvsserver serious issue

From
Magnus Hagander
Date:
On Thu, Oct 7, 2010 at 16:07, Andrew Dunstan <andrew@dunslane.net> wrote:
>
> On 10/07/2010 09:44 AM, Magnus Hagander wrote:
>>
>> On Thu, Oct 7, 2010 at 15:16, Andrew Dunstan<andrew@dunslane.net>  wrote:
>>>
>>> On 09/23/2010 01:18 PM, Aidan Van Dyk wrote:
>>>>
>>>> On Thu, Sep 23, 2010 at 11:49 AM, Tom Lane<tgl@sss.pgh.pa.us>    wrote:
>>>>>
>>>>> Magnus Hagander<magnus@hagander.net>    writes:
>>>>>>
>>>>>> On Thu, Sep 23, 2010 at 17:32, Andrew Dunstan<andrew@dunslane.net>
>>>>>>  wrote:
>>>>>>>
>>>>>>> Are we sure that's going to stop the DOS issue?
>>>>>>
>>>>>> As long as it's done right, I don't see how it wouldn't.
>>>>>
>>>>> There might be a cleaner way to do it, but after a moment's inspection
>>>>> of the script, I'd be inclined to just hack GITCVS::updater->new() to
>>>>> throw error if $module is neither "master" nor "REL\d_\d_STABLE".
>>>>> Keep in mind of course that I'm a lousy Perl coder.
>>>>
>>>> Here's a quick change that will allow you to specifig a "modules" in
>>>> the gitcvs section to export, like:
>>>>     [gitcvs]
>>>>          enabled=1
>>>>          modules=master,REL9_0_STABLE,REL8_4_STABLE
>>>>
>>>> --- git-cvsserver.orig  2010-09-23 12:03:06.000000000 -0400
>>>> +++ git-cvsserver       2010-09-23 13:16:53.000000000 -0400
>>>> @@ -2771,6 +2771,12 @@
>>>>
>>>>      die "Git repo '$self->{git_path}' doesn't exist" unless ( -d
>>>> $self->{git_path} );
>>>>
>>>> +    if (defined $cfg->{gitcvs}{modules})
>>>> +    {
>>>> +        $log->debug("Limitting modules: ". $cfg->{gitcvs}{modules});
>>>> +        die "Invalid module $module" unless map {/^ *$module$/}
>>>> split(',', $cfg->{gitcvs}{modules});
>>>> +    }
>>>> +
>>>>      $self->{dbdriver} = $cfg->{gitcvs}{$state->{method}}{dbdriver} ||
>>>>          $cfg->{gitcvs}{dbdriver} || "SQLite";
>>>>      $self->{dbname} = $cfg->{gitcvs}{$state->{method}}{dbname} ||
>>>
>>> OK, let's go with that. I was kinda hoping that we wouldn't have to do
>>> this
>>> at all, but Stefan has been having serious problems getting git to build
>>> and
>>> run on spoonbill, and I don't want to take up more of his time or be
>>> without
>>> it for very long.
>>
>> Are we doing this *just* for spoonbill? If so, it's a lot easier to
>> just filter-by-IP, so we don't have to maintain a patched version...
>>
>
> That's the only one I know of - there could certainly be others - but if
> we're going to support continued CVS use I want to be able to test it, and I
> don't have a static IP address. Can you filter by name lookup? I have a
> dyndns name.

No, filtering is by IP. ssh forwarding might work.


> OTOH, this patch seems pretty small and simple to maintain.

True, it is rather small.

Does anybody know if there's an automated way to maintain that on
freebsd ports, and if so, how that works? I want to be *sure* we can't
accidentally upgrade git-cvsserver *without* the patch, since that is
a security issue.

--
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/


Re: Git cvsserver serious issue

From
Andrew Dunstan
Date:

On 10/07/2010 10:11 AM, Magnus Hagander wrote:
>
>> OTOH, this patch seems pretty small and simple to maintain.
> True, it is rather small.
>
> Does anybody know if there's an automated way to maintain that on
> freebsd ports, and if so, how that works? I want to be *sure* we can't
> accidentally upgrade git-cvsserver *without* the patch, since that is
> a security issue.
>

Why not just make a local copy somewhere else and patch and run that? 
It's just a Perl script, no?

cheers

andrew


Re: Git cvsserver serious issue

From
Magnus Hagander
Date:
On Thu, Oct 7, 2010 at 21:31, Andrew Dunstan <andrew@dunslane.net> wrote:
>
>
> On 10/07/2010 10:11 AM, Magnus Hagander wrote:
>>
>>> OTOH, this patch seems pretty small and simple to maintain.
>>
>> True, it is rather small.
>>
>> Does anybody know if there's an automated way to maintain that on
>> freebsd ports, and if so, how that works? I want to be *sure* we can't
>> accidentally upgrade git-cvsserver *without* the patch, since that is
>> a security issue.
>>
>
> Why not just make a local copy somewhere else and patch and run that? It's
> just a Perl script, no?

Yeah, but then we have to remember to manually patch that one when
somebody *else* finds/fixes a security issue. We have automatic
monitoring on the ports stuff to detect when that happens..

--
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/


Re: Git cvsserver serious issue

From
Andrew Dunstan
Date:

On 10/07/2010 03:37 PM, Magnus Hagander wrote:
> On Thu, Oct 7, 2010 at 21:31, Andrew Dunstan<andrew@dunslane.net>  wrote:
>>
>> On 10/07/2010 10:11 AM, Magnus Hagander wrote:
>>>> OTOH, this patch seems pretty small and simple to maintain.
>>> True, it is rather small.
>>>
>>> Does anybody know if there's an automated way to maintain that on
>>> freebsd ports, and if so, how that works? I want to be *sure* we can't
>>> accidentally upgrade git-cvsserver *without* the patch, since that is
>>> a security issue.
>>>
>> Why not just make a local copy somewhere else and patch and run that? It's
>> just a Perl script, no?
> Yeah, but then we have to remember to manually patch that one when
> somebody *else* finds/fixes a security issue. We have automatic
> monitoring on the ports stuff to detect when that happens..

There's a simpler solution which I have just tested. Instead of 
patching, use the Pg driver instead of SQLite. Set the dbname to %m. If 
the database doesn't exist the cvs checkout will fail. So we just set up 
databases for the modules we want to export (master and RELn_m_STABLE 
for the live branches).

cheers

andrew


Re: Git cvsserver serious issue

From
Andrew Dunstan
Date:

On 10/07/2010 09:52 PM, Andrew Dunstan wrote:
>
>
> On 10/07/2010 03:37 PM, Magnus Hagander wrote:
>> On Thu, Oct 7, 2010 at 21:31, Andrew Dunstan<andrew@dunslane.net>  
>> wrote:
>>>
>>> On 10/07/2010 10:11 AM, Magnus Hagander wrote:
>>>>> OTOH, this patch seems pretty small and simple to maintain.
>>>> True, it is rather small.
>>>>
>>>> Does anybody know if there's an automated way to maintain that on
>>>> freebsd ports, and if so, how that works? I want to be *sure* we can't
>>>> accidentally upgrade git-cvsserver *without* the patch, since that is
>>>> a security issue.
>>>>
>>> Why not just make a local copy somewhere else and patch and run 
>>> that? It's
>>> just a Perl script, no?
>> Yeah, but then we have to remember to manually patch that one when
>> somebody *else* finds/fixes a security issue. We have automatic
>> monitoring on the ports stuff to detect when that happens..
>
> There's a simpler solution which I have just tested. Instead of 
> patching, use the Pg driver instead of SQLite. Set the dbname to %m. 
> If the database doesn't exist the cvs checkout will fail. So we just 
> set up databases for the modules we want to export (master and 
> RELn_m_STABLE for the live branches).
>
>

BTW, because git-cvsserver treats a branch as a module, there needs to 
be a small change in the buildfarm client to allow us to use it. I'm 
working on that.

cheers

andrew


Re: Git cvsserver serious issue

From
Magnus Hagander
Date:
On Fri, Oct 8, 2010 at 03:52, Andrew Dunstan <andrew@dunslane.net> wrote:
>
>
> On 10/07/2010 03:37 PM, Magnus Hagander wrote:
>>
>> On Thu, Oct 7, 2010 at 21:31, Andrew Dunstan<andrew@dunslane.net>  wrote:
>>>
>>> On 10/07/2010 10:11 AM, Magnus Hagander wrote:
>>>>><
>>>>> OTOH, this patch seems pretty small and simple to maintain.
>>>>
>>>> True, it is rather small.
>>>>
>>>> Does anybody know if there's an automated way to maintain that on
>>>> freebsd ports, and if so, how that works? I want to be *sure* we can't
>>>> accidentally upgrade git-cvsserver *without* the patch, since that is
>>>> a security issue.
>>>>
>>> Why not just make a local copy somewhere else and patch and run that?
>>> It's
>>> just a Perl script, no?
>>
>> Yeah, but then we have to remember to manually patch that one when
>> somebody *else* finds/fixes a security issue. We have automatic
>> monitoring on the ports stuff to detect when that happens..
>
> There's a simpler solution which I have just tested. Instead of patching,
> use the Pg driver instead of SQLite. Set the dbname to %m. If the database
> doesn't exist the cvs checkout will fail. So we just set up databases for
> the modules we want to export (master and RELn_m_STABLE for the live
> branches).

A database per branch seems like a horrible idea in general, but if it
works us around the bug, it seems like a doable idea.. As long as
we'll never have a branch called "postgres" or "git" (already in use
on that box).

I'll look into it.

--
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/


Re: Git cvsserver serious issue

From
Andrew Dunstan
Date:

On 10/08/2010 02:09 AM, Magnus Hagander wrote:
> On Fri, Oct 8, 2010 at 03:52, Andrew Dunstan<andrew@dunslane.net>  wrote:
>>
>> There's a simpler solution which I have just tested. Instead of patching,
>> use the Pg driver instead of SQLite. Set the dbname to %m. If the database
>> doesn't exist the cvs checkout will fail. So we just set up databases for
>> the modules we want to export (master and RELn_m_STABLE for the live
>> branches).
> A database per branch seems like a horrible idea in general, but if it
> works us around the bug, it seems like a doable idea.. As long as
> we'll never have a branch called "postgres" or "git" (already in use
> on that box).
>
> I'll look into it.

That's what the default SQLite setup does anyway. The only difference 
here is that we are leveraging the fact that with the Pg driver the 
database must first exist. Of course, with Pg the database can live on a 
different host or in a server run on a different port, if you need to 
avoid naming conflicts.

cheers

andrew


Re: Git cvsserver serious issue

From
Marko Kreen
Date:
On Fri, Oct 8, 2010 at 3:13 PM, Andrew Dunstan <andrew@dunslane.net> wrote:
> On 10/08/2010 02:09 AM, Magnus Hagander wrote:
>> On Fri, Oct 8, 2010 at 03:52, Andrew Dunstan<andrew@dunslane.net>  wrote:
>>> There's a simpler solution which I have just tested. Instead of patching,
>>> use the Pg driver instead of SQLite. Set the dbname to %m. If the
>>> database
>>> doesn't exist the cvs checkout will fail. So we just set up databases for
>>> the modules we want to export (master and RELn_m_STABLE for the live
>>> branches).

Wouldn't it be simpler be to generate hourly tarball on some host and wget it?
It can be generated even more often, as no history need to be kept.

Considering the state of cvsserver, can you be certain that whatever
is coming from it is really the most recent code?

--
marko


Re: Git cvsserver serious issue

From
Andrew Dunstan
Date:

On 10/08/2010 09:15 AM, Marko Kreen wrote:
> On Fri, Oct 8, 2010 at 3:13 PM, Andrew Dunstan<andrew@dunslane.net>  wrote:
>> On 10/08/2010 02:09 AM, Magnus Hagander wrote:
>>> On Fri, Oct 8, 2010 at 03:52, Andrew Dunstan<andrew@dunslane.net>    wrote:
>>>> There's a simpler solution which I have just tested. Instead of patching,
>>>> use the Pg driver instead of SQLite. Set the dbname to %m. If the
>>>> database
>>>> doesn't exist the cvs checkout will fail. So we just set up databases for
>>>> the modules we want to export (master and RELn_m_STABLE for the live
>>>> branches).
> Wouldn't it be simpler be to generate hourly tarball on some host and wget it?
> It can be generated even more often, as no history need to be kept.
>
> Considering the state of cvsserver, can you be certain that whatever
> is coming from it is really the most recent code?

Sure you can, why not? It will be coming from the same git repo that 
servers git requests. git-cvsserver doesn't create a new CVS repo, it 
emulates CVS from a git repo.

cheers

andrew


Re: Git cvsserver serious issue

From
Aidan Van Dyk
Date:
On Fri, Oct 8, 2010 at 8:13 AM, Andrew Dunstan <andrew@dunslane.net> wrote:

> That's what the default SQLite setup does anyway. The only difference here
> is that we are leveraging the fact that with the Pg driver the database must
> first exist. Of course, with Pg the database can live on a different host or
> in a server run on a different port, if you need to avoid naming conflicts.

That can be done in SQLite to, just set the "database name" to a path
where there is no create access.  Pre-create the database, and don't
let SQLite "create" new ones every time they are accessed.  Standard
unix permissins should easily allow that setup.  chmod -w on the
directory the database files go in.

a.

--
Aidan Van Dyk                                             Create like a god,
aidan@highrise.ca                                       command like a king,
http://www.highrise.ca/                                   work like a slave.


Re: Git cvsserver serious issue

From
Andrew Dunstan
Date:

On 10/08/2010 10:53 AM, Aidan Van Dyk wrote:
> On Fri, Oct 8, 2010 at 8:13 AM, Andrew Dunstan<andrew@dunslane.net>  wrote:
>
>> That's what the default SQLite setup does anyway. The only difference here
>> is that we are leveraging the fact that with the Pg driver the database must
>> first exist. Of course, with Pg the database can live on a different host or
>> in a server run on a different port, if you need to avoid naming conflicts.
> That can be done in SQLite to, just set the "database name" to a path
> where there is no create access.  Pre-create the database, and don't
> let SQLite "create" new ones every time they are accessed.  Standard
> unix permissins should easily allow that setup.  chmod -w on the
> directory the database files go in.

*shrug*. We are the PostgreSQL project after all :-) But whatever Magnus 
wants to do is OK with me.

cheers

andrew



Re: Git cvsserver serious issue

From
Magnus Hagander
Date:
On Fri, Oct 8, 2010 at 08:09, Magnus Hagander <magnus@hagander.net> wrote:
> On Fri, Oct 8, 2010 at 03:52, Andrew Dunstan <andrew@dunslane.net> wrote:
>>
>>
>> On 10/07/2010 03:37 PM, Magnus Hagander wrote:
>>>
>>> On Thu, Oct 7, 2010 at 21:31, Andrew Dunstan<andrew@dunslane.net>  wrote:
>>>>
>>>> On 10/07/2010 10:11 AM, Magnus Hagander wrote:
>>>>>><
>>>>>> OTOH, this patch seems pretty small and simple to maintain.
>>>>>
>>>>> True, it is rather small.
>>>>>
>>>>> Does anybody know if there's an automated way to maintain that on
>>>>> freebsd ports, and if so, how that works? I want to be *sure* we can't
>>>>> accidentally upgrade git-cvsserver *without* the patch, since that is
>>>>> a security issue.
>>>>>
>>>> Why not just make a local copy somewhere else and patch and run that?
>>>> It's
>>>> just a Perl script, no?
>>>
>>> Yeah, but then we have to remember to manually patch that one when
>>> somebody *else* finds/fixes a security issue. We have automatic
>>> monitoring on the ports stuff to detect when that happens..
>>
>> There's a simpler solution which I have just tested. Instead of patching,
>> use the Pg driver instead of SQLite. Set the dbname to %m. If the database
>> doesn't exist the cvs checkout will fail. So we just set up databases for
>> the modules we want to export (master and RELn_m_STABLE for the live
>> branches).
>
> A database per branch seems like a horrible idea in general, but if it
> works us around the bug, it seems like a doable idea.. As long as
> we'll never have a branch called "postgres" or "git" (already in use
> on that box).
>
> I'll look into it.

Should be up and working now. master branch is ready, working on
prepping rel9_0_stable.

CVSROOT is :pserver:anonymous@git.postgresql.org:/postgresql.git

module name is master or REL9_0_STABLE. master available now,
rel_9_0_stable sohuld be available in about half an hour.

--
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/


Re: Git cvsserver serious issue

From
Andrew Dunstan
Date:

On 10/12/2010 03:57 PM, Magnus Hagander wrote:
> There's a simpler solution which I have just tested. Instead of patching,
> use the Pg driver instead of SQLite. Set the dbname to %m. If the database
> doesn't exist the cvs checkout will fail. So we just set up databases for
> the modules we want to export (master and RELn_m_STABLE for the live
> branches).
>>
>> A database per branch seems like a horrible idea in general, but if it
>> works us around the bug, it seems like a doable idea.. As long as
>> we'll never have a branch called "postgres" or "git" (already in use
>> on that box).
>>
>> I'll look into it.
> Should be up and working now. master branch is ready, working on
> prepping rel9_0_stable.
>
> CVSROOT is :pserver:anonymous@git.postgresql.org:/postgresql.git
>
> module name is master or REL9_0_STABLE. master available now,
> rel_9_0_stable sohuld be available in about half an hour.


Thanks for this.

I have tested it (see 
<http://www.pgbuildfarm.org/cgi-bin/show_log.pl?nm=quoll&dt=2010-10-12%2019:55:03>) 
and there is a new Release of the buildfarm client to support its use: 
see <http://pgfoundry.org/frs/?group_id=1000040>

cheers

andrew