Re: sustained update load of 1-2k/sec

From: PFC
Subject: Re: sustained update load of 1-2k/sec
Date: ,
Msg-id: op.svrot4mxth1vuj@localhost
(view: Whole thread, Raw)
In response to: Re: sustained update load of 1-2k/sec  (Tom Lane)
Responses: Re: sustained update load of 1-2k/sec  (Mark Cotner)
List: pgsql-performance

Tree view

sustained update load of 1-2k/sec  (Mark Cotner, )
 Re: sustained update load of 1-2k/sec  (Bob Ippolito, )
  Re: sustained update load of 1-2k/sec  (Mark Cotner, )
   Re: sustained update load of 1-2k/sec  (Bob Ippolito, )
  Re: sustained update load of 1-2k/sec  (Tom Lane, )
   Re: sustained update load of 1-2k/sec  (Andreas Pflug, )
    Re: sustained update load of 1-2k/sec  (Tom Lane, )
    Re: sustained update load of 1-2k/sec  (Ron, )
   Re: sustained update load of 1-2k/sec  (PFC, )
    Re: sustained update load of 1-2k/sec  (Mark Cotner, )
 Re: sustained update load of 1-2k/sec  (Alex Turner, )
  Re: sustained update load of 1-2k/sec  (Ron, )
   Re: sustained update load of 1-2k/sec  ("Jeffrey W. Baker", )
    Re: sustained update load of 1-2k/sec  (Ron, )
   Re: sustained update load of 1-2k/sec  (Alex Turner, )
    Re: sustained update load of 1-2k/sec  (Ron, )
 Re: sustained update load of 1-2k/sec  ("J. Andrew Rogers", )
  Re: sustained update load of 1-2k/sec  (Mark Cotner, )

> While I agree that hundreds of threads seems like overkill, I think the
> above advice might be going too far in the other direction.  The problem
> with single-threaded operation is that any delay affects the whole
> system --- eg, if you're blocked waiting for disk I/O, the CPU doesn't

    You use UDP which is a connectionless protocol... then why use threads ?

    I'd advise this :

    Use asynchronous network code (one thread) to do your network stuff. This
will lower the CPU used by this code immensely.
    Every minute, dump a file contianing everything to insert into the table.
    Use another thread to COPY it into the DB, in a temporary table if you
wish, and then INSERT INTO ... SELECT.
    This should be well adapted to your requirements.


pgsql-performance by date:

From: Mark Cotner
Date:
Subject: Re: sustained update load of 1-2k/sec
From: Dan Harris
Date:
Subject: Re: Query plan looks OK, but slow I/O - settings advice?