Thread: 8.0 beta 1 on linux-mipsel R5900

8.0 beta 1 on linux-mipsel R5900

From
Chris
Date:
Hi,

I've compiled 8.0 beta 1 on a R5900 V3.1 (a playstation 2) running
Linux (PS2 Linux 1) for portability testing.

Here is what make check gave:

Without --disable-spinlocks
compilation *appears* to be fine, but the backends core dumps
on the spinlock assembly in s_lock.c (signal 4 -- illegal
instruction) on first run.

With --disable-spinlocks
all is well except the two floating point tests float4 and float8
out of 96 tests fail. This is likely due to the R5900 not being
fully IEEE-compliant(?) (regression.diffs attached).

--

I'm wondering: would it be hard to fix the assembly spinlock code
for the R5900?
If that's not worth the trouble, would it be a good idea to have
configure disable spinlocks automagically on unsupported platforms?
Or is it to hard to autodetect this?

Bye, Chris.


PS: I've compiled using gcc 2.95.2 and with CFLAGS -O0 -g.






Attachment

Re: 8.0 beta 1 on linux-mipsel R5900

From
Tom Lane
Date:
Chris <list@1006.org> writes:
> I've compiled 8.0 beta 1 on a R5900 V3.1 (a playstation 2) running
> Linux (PS2 Linux 1) for portability testing.
> ...
> I'm wondering: would it be hard to fix the assembly spinlock code
> for the R5900?

According to the previous port report from Red Hat, the PS2 chip simply
doesn't have any user-space TAS instruction, so you're pretty much stuck.
If you can find something that works, let us know.

> If that's not worth the trouble, would it be a good idea to have
> configure disable spinlocks automagically on unsupported platforms?

It used to do that.  We deliberately changed this in 8.0 so that people
would be aware that they were losing performance, and would be motivated
to look into possible solutions.
        regards, tom lane


Re: 8.0 beta 1 on linux-mipsel R5900

From
Greg Stark
Date:
Tom Lane <tgl@sss.pgh.pa.us> writes:

> Chris <list@1006.org> writes:
> > I've compiled 8.0 beta 1 on a R5900 V3.1 (a playstation 2) running
> > Linux (PS2 Linux 1) for portability testing.
> > ...
> > I'm wondering: would it be hard to fix the assembly spinlock code
> > for the R5900?
> 
> According to the previous port report from Red Hat, the PS2 chip simply
> doesn't have any user-space TAS instruction, so you're pretty much stuck.
> If you can find something that works, let us know.

Out of curiosity. If it lacks a tas instruction, is there really any smp
implementation that runs on it? Why would postgres want spinlocks at all with
only one processor?


-- 
greg



Re: 8.0 beta 1 on linux-mipsel R5900

From
Tom Lane
Date:
Greg Stark <gsstark@mit.edu> writes:
> Tom Lane <tgl@sss.pgh.pa.us> writes:
>> According to the previous port report from Red Hat, the PS2 chip simply
>> doesn't have any user-space TAS instruction, so you're pretty much stuck.

> Out of curiosity. If it lacks a tas instruction, is there really any smp
> implementation that runs on it?

My recollection is that it has TAS capability but the designers made it
a privileged instruction (in a fit of brain death :-()

> Why would postgres want spinlocks at all with only one processor?

Think harder... one processor != one process...
        regards, tom lane


Re: 8.0 beta 1 on linux-mipsel R5900

From
Gavin Sherry
Date:
On Tue, 24 Aug 2004, Greg Stark wrote:

>
> Tom Lane <tgl@sss.pgh.pa.us> writes:
>
> > Chris <list@1006.org> writes:
> > > I've compiled 8.0 beta 1 on a R5900 V3.1 (a playstation 2) running
> > > Linux (PS2 Linux 1) for portability testing.
> > > ...
> > > I'm wondering: would it be hard to fix the assembly spinlock code
> > > for the R5900?
> >
> > According to the previous port report from Red Hat, the PS2 chip simply
> > doesn't have any user-space TAS instruction, so you're pretty much stuck.
> > If you can find something that works, let us know.
>
> Out of curiosity. If it lacks a tas instruction, is there really any smp
> implementation that runs on it? Why would postgres want spinlocks at all with
> only one processor?

Errm. Even if we have only one CPU it doesn't mean instructions which
read/write to shared resources in a *multiprocess* environment will be
serialized in the way we want. In fact, its highly unlikely that they
will. If you want to test, make TAS() a noop and see how quickly
things are corrupted in shared memory :-)

Gavin


Re: 8.0 beta 1 on linux-mipsel R5900

From
Peter Eisentraut
Date:
Tom Lane wrote:
> > If that's not worth the trouble, would it be a good idea to have
> > configure disable spinlocks automagically on unsupported platforms?
>
> It used to do that.  We deliberately changed this in 8.0 so that
> people would be aware that they were losing performance, and would be
> motivated to look into possible solutions.

Actually, the previous port report for 7.4 already required explicit 
disabling:
   needs new config.guess, --disable-spinlocks 

(We fixed that first point anyway.)

-- 
Peter Eisentraut
http://developer.postgresql.org/~petere/



Re: 8.0 beta 1 on linux-mipsel R5900

From
Tom Lane
Date:
Peter Eisentraut <peter_e@gmx.net> writes:
> Actually, the previous port report for 7.4 already required explicit 
> disabling:
>     needs new config.guess, --disable-spinlocks 

Yeah, because *normal* ARM processors have a TAS instruction,
and configure only recognizes that you have an ARM.

Apparently Sony bought the 10-cents-off-special without TAS ...
        regards, tom lane


Re: 8.0 beta 1 on linux-mipsel R5900

From
Greg Stark
Date:
Tom Lane <tgl@sss.pgh.pa.us> writes:

> Greg Stark <gsstark@mit.edu> writes:
> > Tom Lane <tgl@sss.pgh.pa.us> writes:
> >> According to the previous port report from Red Hat, the PS2 chip simply
> >> doesn't have any user-space TAS instruction, so you're pretty much stuck.
> 
> > Out of curiosity. If it lacks a tas instruction, is there really any smp
> > implementation that runs on it?

Ah, that would be poor.

> My recollection is that it has TAS capability but the designers made it
> a privileged instruction (in a fit of brain death :-()
> 
> > Why would postgres want spinlocks at all with only one processor?
> 
> Think harder... one processor != one process...

Well sure, but you don't want a spinlock in that case. You may as well yield
immediately rather than spinning waiting for the inevitable context switch.
Your tas is never going to succeed until the context switch happens anyways.

But I think I'm on the wrong track. Without tas, you don't even get to handle
the common case of an uncontested lock without some sort of kernel call to
block context switches, so you still get a performance hit.

-- 
greg



Re: 8.0 beta 1 on linux-mipsel R5900

From
Tom Lane
Date:
Greg Stark <gsstark@mit.edu> writes:
> Tom Lane <tgl@sss.pgh.pa.us> writes:
>> Think harder... one processor != one process...

> Well sure, but you don't want a spinlock in that case.

Actually you do, when the normal case is that you don't have to block.
You want it to fall through as quickly as possible in the success case
(the blocking case is going to suck no matter what).  Given the present
set of available/portable technologies, spinlocks win.

If you've got a better alternative, educate us ...
        regards, tom lane


Re: 8.0 beta 1 on linux-mipsel R5900

From
"Zeugswetter Andreas SB SD"
Date:
> >> Think harder... one processor != one process...
>
> > Well sure, but you don't want a spinlock in that case.
>
> Actually you do, when the normal case is that you don't have to block.
> You want it to fall through as quickly as possible in the success case
> (the blocking case is going to suck no matter what).  Given the present
> set of available/portable technologies, spinlocks win.

I guess it could still save some CPU cycles in the single CPU case,
if you yield() instead of tight loop around TAS in the failure case.
Problem is yield and detecting single CPU is not portable.

Andreas


Re: 8.0 beta 1 on linux-mipsel R5900

From
Tom Lane
Date:
"Zeugswetter Andreas SB SD" <ZeugswetterA@spardat.at> writes:
> I guess it could still save some CPU cycles in the single CPU case,
> if you yield() instead of tight loop around TAS in the failure case.
> Problem is yield and detecting single CPU is not portable.

Sure, but that's still a spinlock --- you're just tuning the backoff
behavior for local conditions.

On some architectures (Alpha at least) the TAS instruction can "fail"
even though the lock is free, if an interrupt happens to get in the way.
So I'd be inclined to loop a few times even on a single-CPU machine.

But yes, a yield primitive would be nice, and so would knowing the
number of CPUs.
        regards, tom lane


Re: 8.0 beta 1 on linux-mipsel R5900

From
Neil Conway
Date:
Tom Lane wrote:
> But yes, a yield primitive would be nice, and so would knowing the
> number of CPUs.

I'm planning on submitting a patch shortly that determines the number of 
CPUs in the machine and adjusts the spinlock behavior accordingly (there 
are various APIs for getting the # of CPUs, like sysctl() on many Unixen 
and GetSystemInfo() on Win32).

Speaking of improving spinlock behavior, there's a Solaris API that I 
think might be worth using: schedctl_start() asks the scheduler to not 
pre-empt the current process, and schedctl_stop() cancels the request. 
The idea the first extremely short periods of time that we're holding a 
spinlock, we don't want to get preempted, since if the process was 
allowed to run for just a little bit longer it would probably give up 
the spinlock. The docs are here:

http://docs.sun.com/db/doc/816-3324/6m9k4vdu1?a=view

-Neil



Re: 8.0 beta 1 on linux-mipsel R5900

From
Tom Lane
Date:
Neil Conway <neilc@samurai.com> writes:
> Speaking of improving spinlock behavior, there's a Solaris API that I 
> think might be worth using: schedctl_start() asks the scheduler to not 
> pre-empt the current process, and schedctl_stop() cancels the request. 
> The idea the first extremely short periods of time that we're holding a 
> spinlock, we don't want to get preempted, since if the process was 
> allowed to run for just a little bit longer it would probably give up 
> the spinlock.

Associating such a thing with spinlocks seems certain to be a dead loss,
as the amount of time we normally hold a spinlock is much less than the
time to make one kernel call, let alone two.

Associating it with LWLocks is slightly more plausible.  There are some
LWLocks that are held for lengths of time that would make it reasonable
to do this (but not all of them are used that way, so some thought is
still needed).

On the count-the-number-of-CPUs patch, what sort of coverage are you
expecting to get?  If we could be certain that all SMP-capable platforms
are covered, then we could default to assuming 1 CPU on platforms that
don't have any of those APIs, which would be a win.
        regards, tom lane


Re: 8.0 beta 1 on linux-mipsel R5900

From
Neil Conway
Date:
Tom Lane wrote:
> Associating such a thing with spinlocks seems certain to be a dead loss,
> as the amount of time we normally hold a spinlock is much less than the
> time to make one kernel call, let alone two.

Yeah, I was thinking about this. ISTM the only way that Sun would bother 
to provide an API like this is if it had significantly less overhead 
than a standard system call. Anyway, I'll take a closer look.

> On the count-the-number-of-CPUs patch, what sort of coverage are you
> expecting to get?

I haven't yet seen a platform that doesn't provide some means to get the 
# of CPUs, but I suppose there might be one...

-Neil




Re: 8.0 beta 1 on linux-mipsel R5900

From
Tom Lane
Date:
Neil Conway <neilc@samurai.com> writes:
> Tom Lane wrote:
>> On the count-the-number-of-CPUs patch, what sort of coverage are you
>> expecting to get?

> I haven't yet seen a platform that doesn't provide some means to get the 
> # of CPUs, but I suppose there might be one...

It might be worth exposing the CPU count as a GUC variable.  This would
* make it easy to check on the results of the counting patch;
* make it easy to override the patch, if it's wrong on some platform;
* make it easy to experiment to see whether the spinlock behavioral change actually matters ;-)

But this may be overkill.
        regards, tom lane


Re: 8.0 beta 1 on linux-mipsel R5900

From
Greg Stark
Date:
Neil Conway <neilc@samurai.com> writes:

> Tom Lane wrote:
> > Associating such a thing with spinlocks seems certain to be a dead loss,
> > as the amount of time we normally hold a spinlock is much less than the
> > time to make one kernel call, let alone two.
> 
> Yeah, I was thinking about this. ISTM the only way that Sun would bother to
> provide an API like this is if it had significantly less overhead than a
> standard system call. Anyway, I'll take a closer look.

There are ways they could have done this too. It doesn't really need a syscall
since the kernel doesn't need the information until it tries to do a context
switch. The function could merely place the information in a shared memory
page for the kernel to consult when the timer interrupt goes off.

> > On the count-the-number-of-CPUs patch, what sort of coverage are you
> > expecting to get?
> 
> I haven't yet seen a platform that doesn't provide some means to get the # of
> CPUs, but I suppose there might be one...

As Tom mentions, it would be nice to be able to override it. One reason I can
think of is if you're on a machine with many processors but have used admin
tools to bind postgres to just a subset of the processors or even just a
single processor. You would want postgres to behave as if it's a machine with
only those processors.

-- 
greg



Re: 8.0 beta 1 on linux-mipsel R5900

From
Chris Mair
Date:

> > I've compiled 8.0 beta 1 on a R5900 V3.1 (a playstation 2) running
> > Linux (PS2 Linux 1) for portability testing.
> > ...
> > I'm wondering: would it be hard to fix the assembly spinlock code
> > for the R5900?
> 
> According to the previous port report from Red Hat, the PS2 chip simply
> doesn't have any user-space TAS instruction, so you're pretty much stuck.
> If you can find something that works, let us know.

I've asked about this issue on the ps2 linux forums and somebody
came up with this document:
http://lc.linux.or.jp/lc2001/papers/tas-ps2-paper.pdf

See also this releated thread on the linux-mips list:
http://www.linux-mips.org/archives/linux-mips/2002-01/msg00278.html

I'm not fluent in Japanese or MIPS assembly, so I won't comment
on these. Just posting the links :)

Bye, Chris.