Re: Using Postgres to store high volume streams of sensor readings - Mailing list pgsql-general

From Alvaro Herrera
Subject Re: Using Postgres to store high volume streams of sensor readings
Date
Msg-id 20081122222627.GD3813@alvh.no-ip.org
Whole thread Raw
In response to Re: Using Postgres to store high volume streams of sensor readings  ("Ciprian Dorin Craciun" <ciprian.craciun@gmail.com>)
Responses Re: Using Postgres to store high volume streams of sensor readings  (Tom Lane <tgl@sss.pgh.pa.us>)
Re: Using Postgres to store high volume streams of sensor readings  ("Ciprian Dorin Craciun" <ciprian.craciun@gmail.com>)
List pgsql-general
Ciprian Dorin Craciun escribió:

>     I've tested also Sqlite3 and it has the same behavior as
> Postgres... Meaning at beginning it goes really nice 20k inserts,
> drops to about 10k inserts, but after a few million records, the HDD
> led starts to blink non-stop, and then it drops to unde 1k....

The problem is, most likely, on updating the indexes.  Heap inserts
should always take more or less the same time, but index insertion
requires walking down the index struct for each insert, and the path to
walk gets larger the more data you have.

Postgres does not have bulk index insert, which could perhaps get you a
huge performance improvement.

--
Alvaro Herrera                                http://www.CommandPrompt.com/
The PostgreSQL Company - Command Prompt, Inc.

pgsql-general by date:

Previous
From: "David Wilson"
Date:
Subject: Re: Using Postgres to store high volume streams of sensor readings
Next
From: Alvaro Herrera
Date:
Subject: Re: Using Postgres to store high volume streams of sensor readings