Re: Acclerating INSERT/UPDATE using UPS - Mailing list pgsql-hackers

Joshua,

Thanks for your comments !

Right. As you pointed out, Sigres cannot recover if UPS fails.
Therefore, I think currently Sigres cannot be used for mission critical
applications.
Sigres keeps data integrity *only when* UPS works.

On the other hand, some users such as researchers of sensor networks
would like store
and analyze 10^6 Hz sensor data insertions in real-time, and obviously
research usages are not mission critical. Thus Sigres may be accepted
for researchers, but I have no prospect now since I have just started to
distributing Sigres for research institutes in Japan.

BTW, Joshua, could you please let me know or give me any pointers for
the reason why fsync=off option exists on PostgreSQL although PostgreSQL
developers does not allow sacrificing data integrity ?
If the reason is famous and clear in the community, I am sorry for
bothering you.


-- Hideyuki

Joshua D. Drake wrote:
> Hideyuki Kawashima wrote:
>   
>> Hello PostgreSQL Hackers,
>>
>> I have made a modification of PostgreSQL which accelerates INSERT/UPDATE using UPS. The name of the software is
"Sigres",and the philosophy is considering a battery supplied memory as a persistent device instead of a disk. You can
downloadSigres from http://sourceforge.jp/projects/sigres/ .
 
>>
>> In the maximum case, Sigres is 7 times faster than PostgreSQL default (fsync=on) in my environment (CoreDuo 2.66GHz,
UDMA/133),and it is also 10% faster than PostgreSQL without fsync (fsync=off).
 
>>     
>
> Interesting and what happens when the UPS fails? My main concern is that
> one of the purposes of PostgreSQL is data integrity. I am all for every
> performance enhancement we can achieve, that does *not* sacrifice that.
>
> Sincerely,
>
> Joshua D. Drake
>
>   
>> The magic lies in usually skipping XLogWrite() and ignoring WALWriteLock. The exceptions are XLogWrite() calls from
AdvanceXLInsertBuffer().In addition, in XLogFileClose() issue_xlog_fsync() before close(). (In this point, Sigres is
differentfrom just simply setting fsync=off.)
 
>>
>> Although I think Sigres can be considered as one of the future directions of PostgreSQL, I do not know whether this
softwarecan be accepted or not. Could you please give me some comments ?
 
>>
>> Best Regards,
>>
>> -- Hideyuki Kawashima 
>> Assistant Professor, University of Tsukuba
>>
>>
>>
>> ---------------------------(end of broadcast)---------------------------
>> TIP 1: if posting/reading through Usenet, please send an appropriate
>>        subscribe-nomail command to majordomo@postgresql.org so that your
>>        message can get through to the mailing list cleanly
>>
>>     
>
>
>