Re: [HACKERS] Re: Faster CREATE DATABASE by delaying fsync (was 8.4.1 ubuntu karmic slow createdb)

From: Andres Freund
Subject: Re: [HACKERS] Re: Faster CREATE DATABASE by delaying fsync (was 8.4.1 ubuntu karmic slow createdb)
Date: ,
Msg-id: 4B698605.50506@anarazel.de
(view: Whole thread, Raw)
In response to: Re: [HACKERS] Re: Faster CREATE DATABASE by delaying fsync (was 8.4.1 ubuntu karmic slow createdb)  (Robert Haas)
Responses: Re: [HACKERS] Re: Faster CREATE DATABASE by delaying fsync (was 8.4.1 ubuntu karmic slow createdb)  (Greg Smith)
List: pgsql-performance

Tree view

8.4.1 ubuntu karmic slow createdb  (Michael Clemmons, )
 Re: 8.4.1 ubuntu karmic slow createdb  (Andres Freund, )
  Re: 8.4.1 ubuntu karmic slow createdb  (Michael Clemmons, )
   Re: 8.4.1 ubuntu karmic slow createdb  (Andres Freund, )
    Re: 8.4.1 ubuntu karmic slow createdb  (Nikolas Everett, )
     Re: 8.4.1 ubuntu karmic slow createdb  ("Joshua D. Drake", )
      Re: 8.4.1 ubuntu karmic slow createdb  (Nikolas Everett, )
       Re: 8.4.1 ubuntu karmic slow createdb  ("Joshua D. Drake", )
        Re: 8.4.1 ubuntu karmic slow createdb  (Nikolas Everett, )
         Re: 8.4.1 ubuntu karmic slow createdb  (Scott Marlowe, )
          Re: 8.4.1 ubuntu karmic slow createdb  (Scott Carey, )
           Re: 8.4.1 ubuntu karmic slow createdb  (Scott Marlowe, )
         Re: 8.4.1 ubuntu karmic slow createdb  (Scott Mead, )
          Re: 8.4.1 ubuntu karmic slow createdb  (Scott Marlowe, )
           Re: 8.4.1 ubuntu karmic slow createdb  (Robert Haas, )
          Re: 8.4.1 ubuntu karmic slow createdb  (Greg Smith, )
           Re: 8.4.1 ubuntu karmic slow createdb  (Michael Clemmons, )
            Re: 8.4.1 ubuntu karmic slow createdb  (Scott Marlowe, )
             Re: 8.4.1 ubuntu karmic slow createdb  (Andres Freund, )
            Re: 8.4.1 ubuntu karmic slow createdb  (Michael Clemmons, )
             Re: 8.4.1 ubuntu karmic slow createdb  (Andres Freund, )
              Faster CREATE DATABASE by delaying fsync (was 8.4.1 ubuntu karmic slow createdb)  (Andres Freund, )
               Re: [HACKERS] Faster CREATE DATABASE by delaying fsync (was 8.4.1 ubuntu karmic slow createdb)  (Andres Freund, )
                Re: [HACKERS] Faster CREATE DATABASE by delaying fsync (was 8.4.1 ubuntu karmic slow createdb)  (Andres Freund, )
                 Re: Faster CREATE DATABASE by delaying fsync (was 8.4.1 ubuntu karmic slow createdb)  (Greg Stark, )
                  Re: Faster CREATE DATABASE by delaying fsync (was 8.4.1 ubuntu karmic slow createdb)  (Greg Stark, )
                   Re: Faster CREATE DATABASE by delaying fsync (was 8.4.1 ubuntu karmic slow createdb)  (Greg Stark, )
                    Re: [HACKERS] Re: Faster CREATE DATABASE by delaying fsync (was 8.4.1 ubuntu karmic slow createdb)  (Andres Freund, )
                    Re: Re: Faster CREATE DATABASE by delaying fsync (was 8.4.1 ubuntu karmic slow createdb)  (Greg Smith, )
                    Re: [HACKERS] Re: Faster CREATE DATABASE by delaying fsync (was 8.4.1 ubuntu karmic slow createdb)  (Greg Smith, )
                   Re: [HACKERS] Re: Faster CREATE DATABASE by delaying fsync (was 8.4.1 ubuntu karmic slow createdb)  (Andres Freund, )
                   Re: [HACKERS] Re: Faster CREATE DATABASE by delaying fsync (was 8.4.1 ubuntu karmic slow createdb)  (Tom Lane, )
                    Re: Faster CREATE DATABASE by delaying fsync (was 8.4.1 ubuntu karmic slow createdb)  (Greg Stark, )
                     Re: [HACKERS] Re: Faster CREATE DATABASE by delaying fsync (was 8.4.1 ubuntu karmic slow createdb)  (Tom Lane, )
                      Re: [HACKERS] Re: Faster CREATE DATABASE by delaying fsync (was 8.4.1 ubuntu karmic slow createdb)  (Robert Haas, )
                       Re: [HACKERS] Re: Faster CREATE DATABASE by delaying fsync (was 8.4.1 ubuntu karmic slow createdb)  (Andres Freund, )
                        Re: [HACKERS] Re: Faster CREATE DATABASE by delaying fsync (was 8.4.1 ubuntu karmic slow createdb)  (Robert Haas, )
                         Re: [HACKERS] Re: Faster CREATE DATABASE by delaying fsync (was 8.4.1 ubuntu karmic slow createdb)  (Andres Freund, )
                         Re: [HACKERS] Re: Faster CREATE DATABASE by delaying fsync (was 8.4.1 ubuntu karmic slow createdb)  (Tom Lane, )
                          Re: [HACKERS] Re: Faster CREATE DATABASE by delaying fsync (was 8.4.1 ubuntu karmic slow createdb)  (Robert Haas, )
                           Re: [HACKERS] Re: Faster CREATE DATABASE by delaying fsync (was 8.4.1 ubuntu karmic slow createdb)  (Greg Stark, )
                            Re: [HACKERS] Re: Faster CREATE DATABASE by delaying fsync (was 8.4.1 ubuntu karmic slow createdb)  (Andres Freund, )
                            Re: [HACKERS] Re: Faster CREATE DATABASE by delaying fsync (was 8.4.1 ubuntu karmic slow createdb)  (Robert Haas, )
                             Re: [HACKERS] Re: Faster CREATE DATABASE by delaying fsync (was 8.4.1 ubuntu karmic slow createdb)  (Andres Freund, )
                              Re: [HACKERS] Re: Faster CREATE DATABASE by delaying fsync (was 8.4.1 ubuntu karmic slow createdb)  (Greg Smith, )
                               Re: [HACKERS] Re: Faster CREATE DATABASE by delaying fsync (was 8.4.1 ubuntu karmic slow createdb)  (Andres Freund, )
                                Re: [HACKERS] Re: Faster CREATE DATABASE by delaying fsync (was 8.4.1 ubuntu karmic slow createdb)  (Robert Haas, )
                                 Re: [HACKERS] Re: Faster CREATE DATABASE by delaying fsync (was 8.4.1 ubuntu karmic slow createdb)  (Greg Smith, )
                                  Re: [HACKERS] Re: Faster CREATE DATABASE by delaying fsync (was 8.4.1 ubuntu karmic slow createdb)  (Tom Lane, )
                                   Re: [HACKERS] Re: Faster CREATE DATABASE by delaying fsync (was 8.4.1 ubuntu karmic slow createdb)  (Robert Haas, )
                                    Re: [HACKERS] Re: Faster CREATE DATABASE by delaying fsync (was 8.4.1 ubuntu karmic slow createdb)  (Andres Freund, )
                                     Re: [HACKERS] Re: Faster CREATE DATABASE by delaying fsync (was 8.4.1 ubuntu karmic slow createdb)  (Andres Freund, )
                                      Re: [HACKERS] Re: Faster CREATE DATABASE by delaying fsync (was 8.4.1 ubuntu karmic slow createdb)  (Alvaro Herrera, )
                                       Re: [HACKERS] Re: Faster CREATE DATABASE by delaying fsync (was 8.4.1 ubuntu karmic slow createdb)  (Robert Haas, )
                                        Re: [HACKERS] Re: Faster CREATE DATABASE by delaying fsync (was 8.4.1 ubuntu karmic slow createdb)  (Andres Freund, )
                                        Re: [HACKERS] Re: Faster CREATE DATABASE by delaying fsync (was 8.4.1 ubuntu karmic slow createdb)  (Greg Stark, )
                                         Re: [HACKERS] Re: Faster CREATE DATABASE by delaying fsync (was 8.4.1 ubuntu karmic slow createdb)  (Andres Freund, )
                                        Re: [HACKERS] Re: Faster CREATE DATABASE by delaying fsync (was 8.4.1 ubuntu karmic slow createdb)  (Andres Freund, )
                     Re: [HACKERS] Re: Faster CREATE DATABASE by delaying fsync (was 8.4.1 ubuntu karmic slow createdb)  (Andres Freund, )
                     Re: [HACKERS] Re: Faster CREATE DATABASE by delaying fsync (was 8.4.1 ubuntu karmic slow createdb)  (Robert Haas, )
                   Re: Faster CREATE DATABASE by delaying fsync (was 8.4.1 ubuntu karmic slow createdb)  (Andres Freund, )
                  Re: Re: Faster CREATE DATABASE by delaying fsync (was 8.4.1 ubuntu karmic slow createdb)  (Andres Freund, )
               Re: [HACKERS] Faster CREATE DATABASE by delaying fsync (was 8.4.1 ubuntu karmic slow createdb)  (Tom Lane, )
                Re: [HACKERS] Faster CREATE DATABASE by delaying fsync (was 8.4.1 ubuntu karmic slow createdb)  (Andres Freund, )
                 Re: [HACKERS] Faster CREATE DATABASE by delaying fsync (was 8.4.1 ubuntu karmic slow createdb)  (Greg Smith, )
                  Re: [HACKERS] Faster CREATE DATABASE by delaying fsync (was 8.4.1 ubuntu karmic slow createdb)  (Andres Freund, )
                   Re: [HACKERS] Faster CREATE DATABASE by delaying fsync (was 8.4.1 ubuntu karmic slow createdb)  (Michael Clemmons, )
                    Re: [HACKERS] Faster CREATE DATABASE by delaying fsync (was 8.4.1 ubuntu karmic slow createdb)  (Andres Freund, )
                     Re: [HACKERS] Faster CREATE DATABASE by delaying fsync (was 8.4.1 ubuntu karmic slow createdb)  (Michael Clemmons, )
                      Re: [HACKERS] Faster CREATE DATABASE by delaying fsync (was 8.4.1 ubuntu karmic slow createdb)  (Andres Freund, )
                   Re: [HACKERS] Faster CREATE DATABASE by delaying fsync (was 8.4.1 ubuntu karmic slow createdb)  (Greg Stark, )
                    Re: [HACKERS] Faster CREATE DATABASE by delaying fsync (was 8.4.1 ubuntu karmic slow createdb)  (Andres Freund, )
                Re: [HACKERS] Faster CREATE DATABASE by delaying fsync (was 8.4.1 ubuntu karmic slow createdb)  (Andres Freund, )
               Re: Faster CREATE DATABASE by delaying fsync (was 8.4.1 ubuntu karmic slow createdb)  (Greg Stark, )
                Re: [HACKERS] Re: Faster CREATE DATABASE by delaying fsync (was 8.4.1 ubuntu karmic slow createdb)  (Andres Freund, )
                Re: [PERFORM] Re: Faster CREATE DATABASE by delaying fsync (was 8.4.1ubuntu karmic slow createdb)  (Andres Freund, )
                 Re: Re: Faster CREATE DATABASE by delaying fsync (was 8.4.1 ubuntu karmic slow createdb)  (, )
                Re: Re: Faster CREATE DATABASE by delaying fsync (was 8.4.1 ubuntu karmic slow createdb)  (, )
            Re: 8.4.1 ubuntu karmic slow createdb  (Thomas Kellerer, )
       Re: 8.4.1 ubuntu karmic slow createdb  ("Joshua D. Drake", )
     Re: 8.4.1 ubuntu karmic slow createdb  ("Joshua D. Drake", )

On 02/03/10 14:42, Robert Haas wrote:
> On Wed, Feb 3, 2010 at 6:53 AM, Greg Stark<>  wrote:
>> On Tue, Feb 2, 2010 at 7:45 PM, Robert Haas<>  wrote:
>>> I think you're probably right, but it's not clear what the new name
>>> should be until we have a comment explaining what the function is
>>> responsible for.
>>
>> So I wrote some comments but wasn't going to repost the patch with the
>> unchanged name without explanation... But I think you're right though
>> I was looking at it the other way around. I want to have an API for a
>> two-stage sync and of course if I do that I'll comment it to explain
>> that clearly.
>>
>> The gist of the comments was that the function is preparing to fsync
>> to initiate the i/o early and allow the later fsync to fast -- but
>> also at the same time have the beneficial side-effect of avoiding
>> cache poisoning. It's not clear that the two are necessarily linked
>> though. Perhaps we need two separate apis, though it'll be hard to
>> keep them separate on all platforms.
>
> Well, maybe we should start with a discussion of what kernel calls
> you're aware of on different platforms and then we could try to put an
> API around it.
In linux there is sync_file_range. On newer Posixish systems one can
emulate that with mmap() and msync() (in batches obviously).

No idea about windows.

Andres


pgsql-performance by date:

From: Mridula Mahadevan
Date:
Subject: Re: Queries within a function
From: Robert Haas
Date:
Subject: Re: System overload / context switching / oom, 8.3