Re: Read/Write block sizes

From: Ron
Subject: Re: Read/Write block sizes
Date: ,
Msg-id: 6.2.3.4.0.20050825185441.05e63ef0@pop.earthlink.net
(view: Whole thread, Raw)
In response to: Re: Read/Write block sizes  (Chris Browne)
List: pgsql-performance

Tree view

Re: Read/Write block sizes (Was: Caching by Postgres)  (Jignesh Shah, )
 Re: Read/Write block sizes  (Chris Browne, )
  Re: Read/Write block sizes  (Michael Stone, )
  Re: Read/Write block sizes  ("Jim C. Nasby", )
   Re: Read/Write block sizes  ("Jignesh K. Shah", )
    Re: Read/Write block sizes  (Bruce Momjian, )
  Re: Read/Write block sizes  (Steve Poe, )
   Re: Read/Write block sizes  (Josh Berkus, )
    Re: Read/Write block sizes  (Alan Stange, )
    Re: Read/Write block sizes  ("Jeffrey W. Baker", )
     Re: Read/Write block sizes  (Tom Lane, )
      Re: Read/Write block sizes  (Josh Berkus, )
     Re: Read/Write block sizes  (Guy Thornley, )
      Re: Read/Write block sizes  ("Jeffrey W. Baker", )
       Re: Read/Write block sizes  (Tom Lane, )
        Re: Read/Write block sizes  ("Jeffrey W. Baker", )
         Re: Read/Write block sizes  (Josh Berkus, )
          Re: Read/Write block sizes  (Ron, )
        Re: Read/Write block sizes  (PFC, )
 Re: Read/Write block sizes (Was: Caching by Postgres)  (Michael Stone, )
  Re: Read/Write block sizes (Was: Caching by Postgres)  ("Jeffrey W. Baker", )
 Re: Read/Write block sizes  (Chris Browne, )
  Re: Read/Write block sizes  ("Jim C. Nasby", )
 Re: Read/Write block sizes  (Chris Browne, )
  Re: Read/Write block sizes  (Ron, )

At 04:49 PM 8/25/2005, Chris Browne wrote:
> (Ron) writes:
> > At 03:45 PM 8/25/2005, Josh Berkus wrote:
> >> > Ask me sometime about my replacement for GNU sort. Â It uses the
> >> > same sorting algorithm, but it's an order of magnitude faster due
> >> > to better I/O strategy. Â Someday, in my infinite spare time, I
> >> > hope to demonstrate that kind of improvement with a patch to pg.
> >>
> >>Since we desperately need some improvements in sort performance, I
> >>do hope you follow up on this.
> >
> > I'll generalize that.  IMO we desperately need any and all
> > improvements in IO performance.  Even more so than we need
> > improvements in sorting or sorting IO performance.
>
>That's frankly a step backwards.  Feel free to "specialise" that instead.

We can agree to disagree, I'm cool with that.

I'm well aware that a Systems Approach to SW
Architecture is not always popular in the Open
Source world.  Nonetheless, my POV is that if we
want to be taken seriously and beat "the big
boys", we have to do everything smarter and
faster, as well as cheaper, than they do.  You
are not likely to be able to do that consistently
without using some of the "icky" stuff one is
required to study as part of formal training in
the Comp Sci and SW Engineering fields.


>A patch that improves some specific aspect of
>performance is a thousand times better than any
>sort of "desperate desire for any and
>all improvements in I/O performance."

minor twisting of my words: substituting "desire"
for "need".  The need is provable.  Just put "the
big 5" (SQL Server, Oracle, DB2, mySQL, and
PostgreSQL) into some realistic benches to see that.

Major twisting of my words: the apparent
implication by you that I don't appreciate
improvements in the IO behavior of specific
things like sorting as much as I'd appreciate
more "general" IO performance
improvements.  Performance optimization is best
done as an iterative improvement process that
starts with measuring where the need is greatest,
then improving that greatest need by the most you
can, then repeating the whole cycle.  _Every_
improvement in such a process is a specific
improvement, even if the improvement is a
decision to re-architect the entire product to
solve the current biggest issue.  Improving
sorting IO is cool.  OTOH, if pg's biggest IO
problems are elsewhere, then the amount of
overall benefit we will get from improving
sorting IO is going to be minimized until we
improve the bigger problem(s).  Amdahl's Law.


>The "specialized patch" is also pointedly better
>in that a *confidently submitted* patch is
>likely to be way better than any sort of
>"desperate clutching at whatever may come to hand."

Another distortion of my statement and POV.  I
never suggested nor implied any sort of
"desperate clutching...".  We have _measurable_
IO issues that need to be addressed in order for
pg to be a better competitor in the
marketplace.  Just as we do with sorting performance.


>Far too often, I see people trying to address
>performance problems via the "desperate
>clutching at whatever seems near to hand," and that
>generally turns out very badly as a particular
>result of the whole "desperate clutching" part.
>
>If you can get a sort improvement submitted, that's a concrete improvement...

As I said, I'm all in favor of concrete,
measurable improvement.  I do not think I ever
stated I was in favor of anything else.

You evidently are mildly ranting because you've
seen some examples of poor SW Engineering
Discipline/Practice by people with perhaps
inadequate skills for the issues they were trying
to address.  We all have. "90% of everything is
Jreck (eg of too low a quality)."

OTOH, I do not think I've given you any reason to
think I lack such Clue, nor do I think my post was advocating such thrashing.

My post was intended to say that we need an
Overall Systems Approach to pg optimization
rather than just applying what compiler writer's
call "peephole optimizations" to pg.  No more, no less.

I apologize if I somehow misled you,
Ron Peacetree




pgsql-performance by date:

From: Mark Kirkwood
Date:
Subject: Re: Limit + group + join
From: "Chun Yit(Chronos)"
Date:
Subject: postmaster memory keep going up????