Re: Re: pgsql: Add URLs for : * Speed WAL recovery by allowing more than one - Mailing list pgsql-committers

From Bruce Momjian
Subject Re: Re: pgsql: Add URLs for : * Speed WAL recovery by allowing more than one
Date
Msg-id 200803182335.m2INZgP03117@momjian.us
Whole thread Raw
In response to Re: Re: pgsql: Add URLs for : * Speed WAL recovery by allowing more than one  ("Heikki Linnakangas" <heikki@enterprisedb.com>)
List pgsql-committers
Heikki Linnakangas wrote:
> How about:
>
> * Speed WAL recovery by allowing more than one page to be prefetched
>
> This should be done utilizing the same infrastructure used for
> prefetching in general, to avoid introducing complex, error-prone code
> to WAL replay codepath, which doesn't get much testing compared to the
> rest of the system.

Updated to:

    * Speed WAL recovery by allowing more than one page to be prefetched

      This should be done utilizing the same infrastructure used for
      prefetching in general to avoid introducing complex error-prone code
      in WAL replay.

> There's already this TODO item:
>
> > Experiment with multi-threaded backend better resource utilization
> >
> > This would allow a single query to make use of multiple CPU's or multiple I/O channels simultaneously. One idea is
tocreate a background reader that can pre-fetch sequential and index scan pages needed by other backends. This could be
expandedto allow concurrent reads from multiple devices in a partitioned table.  
>
> This should probably be split into two. Using multiple CPUs for
> satisfying one query is quite different from implementing some kind of a
> pre-fetching mechanism using posix_fadvise(), libaio, or background
> reader processes.

Good idea, items split:

    * Experiment with multi-threaded backend better I/O utilization

      This would allow a single query to make use of multiple I/O channels
      simultaneously.  One idea is to create a background reader that can
      pre-fetch sequential and index scan pages needed by other backends.
      This could be expanded to allow concurrent reads from multiple devices
      in a partitioned table.

    * Experiment with multi-threaded backend better CPU utilization

      This would allow several CPUs to be used for a single query, such as
      for sorting or query execution.

--
  Bruce Momjian  <bruce@momjian.us>        http://momjian.us
  EnterpriseDB                             http://postgres.enterprisedb.com

  + If your life is a hard drive, Christ can be your backup. +

pgsql-committers by date:

Previous
From: momjian@postgresql.org (Bruce Momjian)
Date:
Subject: pgsql: Spit items: * Experiment with multi-threaded backend better I/O
Next
From: Alvaro Herrera
Date:
Subject: Re: Re: pgsql: Add URLs for : * Speed WAL recovery by allowing more than one