Thread: Write cache
Because I've lost a lot of data using postgresql (and I know for sure this should'nt happen) I've gone a bit further reading documentations on my disks and... I have FUJITSU MAP3367NC 36G 10000 rpm disks, those disk have a write cache of 8Mb, if someone could tell me hox to turn it off... Could'nt find it in the docs and this could very well explain why a busy database crashes every time Thanks for your help -- Olivier PRENANT Tel: +33-5-61-50-97-00 (Work) 6, Chemin d'Harraud Turrou +33-5-61-50-97-01 (Fax) 31190 AUTERIVE +33-6-07-63-80-64 (GSM) FRANCE Email: ohp@pyrenet.fr ------------------------------------------------------------------------------ Make your life a dream, make your dream a reality. (St Exupery)
IDE or SCSI? Why do you think the WC is screwing you? Which driver(s)? LER --On Tuesday, January 27, 2004 22:34:46 +0100 ohp@pyrenet.fr wrote: > Because I've lost a lot of data using postgresql (and I know for sure this > should'nt happen) I've gone a bit further reading documentations on my > disks and... > > I have FUJITSU MAP3367NC 36G 10000 rpm disks, those disk have a write > cache of 8Mb, if someone could tell me hox to turn it off... Could'nt find > it in the docs and this could very well explain why a busy database > crashes every time > > Thanks for your help > > -- > Olivier PRENANT Tel: +33-5-61-50-97-00 (Work) > 6, Chemin d'Harraud Turrou +33-5-61-50-97-01 (Fax) > 31190 AUTERIVE +33-6-07-63-80-64 (GSM) > FRANCE Email: ohp@pyrenet.fr > ------------------------------------------------------------------------- > ----- Make your life a dream, make your dream a reality. (St Exupery) -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 972-414-9812 E-Mail: ler@lerctr.org US Mail: 1905 Steamboat Springs Drive, Garland, TX 75044-6749
On Tue, 27 Jan 2004, Larry Rosenman wrote: > Date: Tue, 27 Jan 2004 15:38:30 -0600 > From: Larry Rosenman <ler@lerctr.org> > To: ohp@pyrenet.fr, pgsql-hackers list <pgsql-hackers@postgresql.org> > Subject: Re: Write cache > > IDE or SCSI? SCSI > > Why do you think the WC is screwing you? > Because after a sys crash, the most used filesystems (databases) are screwed > Which driver(s)? > Guess... ADPU320 > LER > > > --On Tuesday, January 27, 2004 22:34:46 +0100 ohp@pyrenet.fr wrote: > > > Because I've lost a lot of data using postgresql (and I know for sure this > > should'nt happen) I've gone a bit further reading documentations on my > > disks and... > > > > I have FUJITSU MAP3367NC 36G 10000 rpm disks, those disk have a write > > cache of 8Mb, if someone could tell me hox to turn it off... Could'nt find > > it in the docs and this could very well explain why a busy database > > crashes every time > > > > Thanks for your help > > > > -- > > Olivier PRENANT Tel: +33-5-61-50-97-00 (Work) > > 6, Chemin d'Harraud Turrou +33-5-61-50-97-01 (Fax) > > 31190 AUTERIVE +33-6-07-63-80-64 (GSM) > > FRANCE Email: ohp@pyrenet.fr > > ------------------------------------------------------------------------- > > ----- Make your life a dream, make your dream a reality. (St Exupery) > > > > -- Olivier PRENANT Tel: +33-5-61-50-97-00 (Work) 6, Chemin d'Harraud Turrou +33-5-61-50-97-01 (Fax) 31190 AUTERIVE +33-6-07-63-80-64 (GSM) FRANCE Email: ohp@pyrenet.fr ------------------------------------------------------------------------------ Make your life a dream, make your dream a reality. (St Exupery)
--On Tuesday, January 27, 2004 22:43:34 +0100 ohp@pyrenet.fr wrote: > On Tue, 27 Jan 2004, Larry Rosenman wrote: > >> Date: Tue, 27 Jan 2004 15:38:30 -0600 >> From: Larry Rosenman <ler@lerctr.org> >> To: ohp@pyrenet.fr, pgsql-hackers list <pgsql-hackers@postgresql.org> >> Subject: Re: Write cache >> >> IDE or SCSI? > SCSI >> >> Why do you think the WC is screwing you? >> > Because after a sys crash, the most used filesystems (databases) are > screwed what VxFS mount options are you using? >> Which driver(s)? >> > Guess... ADPU320 Hrm. >> LER >> >> >> --On Tuesday, January 27, 2004 22:34:46 +0100 ohp@pyrenet.fr wrote: >> >> > Because I've lost a lot of data using postgresql (and I know for sure >> > this should'nt happen) I've gone a bit further reading documentations >> > on my disks and... >> > >> > I have FUJITSU MAP3367NC 36G 10000 rpm disks, those disk have a write >> > cache of 8Mb, if someone could tell me hox to turn it off... Could'nt >> > find it in the docs and this could very well explain why a busy >> > database crashes every time >> > >> > Thanks for your help >> > >> > -- >> > Olivier PRENANT Tel: +33-5-61-50-97-00 (Work) >> > 6, Chemin d'Harraud Turrou +33-5-61-50-97-01 (Fax) >> > 31190 AUTERIVE +33-6-07-63-80-64 (GSM) >> > FRANCE Email: ohp@pyrenet.fr >> > ---------------------------------------------------------------------- >> > --- ----- Make your life a dream, make your dream a reality. (St >> > Exupery) >> >> >> >> > > -- > Olivier PRENANT Tel: +33-5-61-50-97-00 (Work) > 6, Chemin d'Harraud Turrou +33-5-61-50-97-01 (Fax) > 31190 AUTERIVE +33-6-07-63-80-64 (GSM) > FRANCE Email: ohp@pyrenet.fr > ------------------------------------------------------------------------- > ----- Make your life a dream, make your dream a reality. (St Exupery) -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 972-414-9812 E-Mail: ler@lerctr.org US Mail: 1905 Steamboat Springs Drive, Garland, TX 75044-6749
On Tue, 27 Jan 2004, Larry Rosenman wrote: > Date: Tue, 27 Jan 2004 15:45:20 -0600 > From: Larry Rosenman <ler@lerctr.org> > To: ohp@pyrenet.fr > Cc: pgsql-hackers list <pgsql-hackers@postgresql.org> > Subject: Re: Write cache > > > > --On Tuesday, January 27, 2004 22:43:34 +0100 ohp@pyrenet.fr wrote: > > > On Tue, 27 Jan 2004, Larry Rosenman wrote: > > > >> Date: Tue, 27 Jan 2004 15:38:30 -0600 > >> From: Larry Rosenman <ler@lerctr.org> > >> To: ohp@pyrenet.fr, pgsql-hackers list <pgsql-hackers@postgresql.org> > >> Subject: Re: Write cache > >> > >> IDE or SCSI? > > SCSI > >> > >> Why do you think the WC is screwing you? > >> > > Because after a sys crash, the most used filesystems (databases) are > > screwed > what VxFS mount options are you using? Nothing special... And any option I could use would'nt change a thing: the cache is on the disk itself... I did'nt look physicaly yet but according to the docs there's no way to disable it lik I always did on IBM... I was forced to buy those disk (more expensive, not better)> > > >> Which driver(s)? > >> > > Guess... ADPU320 > Hrm. > > >> LER > >> > >> > >> --On Tuesday, January 27, 2004 22:34:46 +0100 ohp@pyrenet.fr wrote: > >> > >> > Because I've lost a lot of data using postgresql (and I know for sure > >> > this should'nt happen) I've gone a bit further reading documentations > >> > on my disks and... > >> > > >> > I have FUJITSU MAP3367NC 36G 10000 rpm disks, those disk have a write > >> > cache of 8Mb, if someone could tell me hox to turn it off... Could'nt > >> > find it in the docs and this could very well explain why a busy > >> > database crashes every time > >> > > >> > Thanks for your help > >> > > >> > -- > >> > Olivier PRENANT Tel: +33-5-61-50-97-00 (Work) > >> > 6, Chemin d'Harraud Turrou +33-5-61-50-97-01 (Fax) > >> > 31190 AUTERIVE +33-6-07-63-80-64 (GSM) > >> > FRANCE Email: ohp@pyrenet.fr > >> > ---------------------------------------------------------------------- > >> > --- ----- Make your life a dream, make your dream a reality. (St > >> > Exupery) > >> > >> > >> > >> > > > > -- > > Olivier PRENANT Tel: +33-5-61-50-97-00 (Work) > > 6, Chemin d'Harraud Turrou +33-5-61-50-97-01 (Fax) > > 31190 AUTERIVE +33-6-07-63-80-64 (GSM) > > FRANCE Email: ohp@pyrenet.fr > > ------------------------------------------------------------------------- > > ----- Make your life a dream, make your dream a reality. (St Exupery) > > > > -- Olivier PRENANT Tel: +33-5-61-50-97-00 (Work) 6, Chemin d'Harraud Turrou +33-5-61-50-97-01 (Fax) 31190 AUTERIVE +33-6-07-63-80-64 (GSM) FRANCE Email: ohp@pyrenet.fr ------------------------------------------------------------------------------ Make your life a dream, make your dream a reality. (St Exupery)
--On Tuesday, January 27, 2004 22:48:42 +0100 ohp@pyrenet.fr wrote: > On Tue, 27 Jan 2004, Larry Rosenman wrote: > >> Date: Tue, 27 Jan 2004 15:45:20 -0600 >> From: Larry Rosenman <ler@lerctr.org> >> To: ohp@pyrenet.fr >> Cc: pgsql-hackers list <pgsql-hackers@postgresql.org> >> Subject: Re: Write cache >> >> >> >> --On Tuesday, January 27, 2004 22:43:34 +0100 ohp@pyrenet.fr wrote: >> >> > On Tue, 27 Jan 2004, Larry Rosenman wrote: >> > >> >> Date: Tue, 27 Jan 2004 15:38:30 -0600 >> >> From: Larry Rosenman <ler@lerctr.org> >> >> To: ohp@pyrenet.fr, pgsql-hackers list <pgsql-hackers@postgresql.org> >> >> Subject: Re: Write cache >> >> >> >> IDE or SCSI? >> > SCSI >> >> >> >> Why do you think the WC is screwing you? >> >> >> > Because after a sys crash, the most used filesystems (databases) are >> > screwed >> what VxFS mount options are you using? > Nothing special... And any option I could use would'nt change a thing: the > cache is on the disk itself... I did'nt look physicaly yet but according > to the docs there's no way to disable it lik I always did on IBM... > > I was forced to buy those disk (more expensive, not better) define not special? ISTM that the driver should force it out to the disk, unless the disk is lying to the driver, or the driver is buggy. > > >> >> >> Which driver(s)? >> >> >> > Guess... ADPU320 >> Hrm. >> >> >> LER >> >> >> >> >> >> --On Tuesday, January 27, 2004 22:34:46 +0100 ohp@pyrenet.fr wrote: >> >> >> >> > Because I've lost a lot of data using postgresql (and I know for >> >> > sure this should'nt happen) I've gone a bit further reading >> >> > documentations on my disks and... >> >> > >> >> > I have FUJITSU MAP3367NC 36G 10000 rpm disks, those disk have a >> >> > write cache of 8Mb, if someone could tell me hox to turn it off... >> >> > Could'nt find it in the docs and this could very well explain why a >> >> > busy database crashes every time >> >> > >> >> > Thanks for your help >> >> > >> >> > -- >> >> > Olivier PRENANT Tel: +33-5-61-50-97-00 (Work) >> >> > 6, Chemin d'Harraud Turrou +33-5-61-50-97-01 (Fax) >> >> > 31190 AUTERIVE +33-6-07-63-80-64 (GSM) >> >> > FRANCE Email: ohp@pyrenet.fr >> >> > ------------------------------------------------------------------- >> >> > --- --- ----- Make your life a dream, make your dream a reality. (St >> >> > Exupery) >> >> >> >> >> >> >> >> >> > >> > -- >> > Olivier PRENANT Tel: +33-5-61-50-97-00 (Work) >> > 6, Chemin d'Harraud Turrou +33-5-61-50-97-01 (Fax) >> > 31190 AUTERIVE +33-6-07-63-80-64 (GSM) >> > FRANCE Email: ohp@pyrenet.fr >> > ---------------------------------------------------------------------- >> > --- ----- Make your life a dream, make your dream a reality. (St >> > Exupery) >> >> >> >> > > -- > Olivier PRENANT Tel: +33-5-61-50-97-00 (Work) > 6, Chemin d'Harraud Turrou +33-5-61-50-97-01 (Fax) > 31190 AUTERIVE +33-6-07-63-80-64 (GSM) > FRANCE Email: ohp@pyrenet.fr > ------------------------------------------------------------------------- > ----- Make your life a dream, make your dream a reality. (St Exupery) -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 972-414-9812 E-Mail: ler@lerctr.org US Mail: 1905 Steamboat Springs Drive, Garland, TX 75044-6749
On Tue, 27 Jan 2004, Larry Rosenman wrote: > Date: Tue, 27 Jan 2004 15:55:49 -0600 > From: Larry Rosenman <ler@lerctr.org> > To: ohp@pyrenet.fr > Cc: pgsql-hackers list <pgsql-hackers@postgresql.org> > Subject: Re: Write cache > > > > --On Tuesday, January 27, 2004 22:48:42 +0100 ohp@pyrenet.fr wrote: > > > On Tue, 27 Jan 2004, Larry Rosenman wrote: > > > >> Date: Tue, 27 Jan 2004 15:45:20 -0600 > >> From: Larry Rosenman <ler@lerctr.org> > >> To: ohp@pyrenet.fr > >> Cc: pgsql-hackers list <pgsql-hackers@postgresql.org> > >> Subject: Re: Write cache > >> > >> > >> > >> --On Tuesday, January 27, 2004 22:43:34 +0100 ohp@pyrenet.fr wrote: > >> > >> > On Tue, 27 Jan 2004, Larry Rosenman wrote: > >> > > >> >> Date: Tue, 27 Jan 2004 15:38:30 -0600 > >> >> From: Larry Rosenman <ler@lerctr.org> > >> >> To: ohp@pyrenet.fr, pgsql-hackers list <pgsql-hackers@postgresql.org> > >> >> Subject: Re: Write cache > >> >> > >> >> IDE or SCSI? > >> > SCSI > >> >> > >> >> Why do you think the WC is screwing you? > >> >> > >> > Because after a sys crash, the most used filesystems (databases) are > >> > screwed > >> what VxFS mount options are you using? > > Nothing special... And any option I could use would'nt change a thing: the > > cache is on the disk itself... I did'nt look physicaly yet but according > > to the docs there's no way to disable it lik I always did on IBM... > > > > I was forced to buy those disk (more expensive, not better) > define not special? > > ISTM that the driver should force it out to the disk, unless the disk is > lying to the driver, or the driver is buggy. That's exactly what I think: the DISK has a write cache so it's lying to the controler saying data is written when it's only in the disk cache. That also would explain the fantastic write performance I have compared to read... > > > > > > >> > >> >> Which driver(s)? > >> >> > >> > Guess... ADPU320 > >> Hrm. > >> > >> >> LER > >> >> > >> >> > >> >> --On Tuesday, January 27, 2004 22:34:46 +0100 ohp@pyrenet.fr wrote: > >> >> > >> >> > Because I've lost a lot of data using postgresql (and I know for > >> >> > sure this should'nt happen) I've gone a bit further reading > >> >> > documentations on my disks and... > >> >> > > >> >> > I have FUJITSU MAP3367NC 36G 10000 rpm disks, those disk have a > >> >> > write cache of 8Mb, if someone could tell me hox to turn it off... > >> >> > Could'nt find it in the docs and this could very well explain why a > >> >> > busy database crashes every time > >> >> > > >> >> > Thanks for your help > >> >> > > >> >> > -- > >> >> > Olivier PRENANT Tel: +33-5-61-50-97-00 (Work) > >> >> > 6, Chemin d'Harraud Turrou +33-5-61-50-97-01 (Fax) > >> >> > 31190 AUTERIVE +33-6-07-63-80-64 (GSM) > >> >> > FRANCE Email: ohp@pyrenet.fr > >> >> > ------------------------------------------------------------------- > >> >> > --- --- ----- Make your life a dream, make your dream a reality. (St > >> >> > Exupery) > >> >> > >> >> > >> >> > >> >> > >> > > >> > -- > >> > Olivier PRENANT Tel: +33-5-61-50-97-00 (Work) > >> > 6, Chemin d'Harraud Turrou +33-5-61-50-97-01 (Fax) > >> > 31190 AUTERIVE +33-6-07-63-80-64 (GSM) > >> > FRANCE Email: ohp@pyrenet.fr > >> > ---------------------------------------------------------------------- > >> > --- ----- Make your life a dream, make your dream a reality. (St > >> > Exupery) > >> > >> > >> > >> > > > > -- > > Olivier PRENANT Tel: +33-5-61-50-97-00 (Work) > > 6, Chemin d'Harraud Turrou +33-5-61-50-97-01 (Fax) > > 31190 AUTERIVE +33-6-07-63-80-64 (GSM) > > FRANCE Email: ohp@pyrenet.fr > > ------------------------------------------------------------------------- > > ----- Make your life a dream, make your dream a reality. (St Exupery) > > > > -- Olivier PRENANT Tel: +33-5-61-50-97-00 (Work) 6, Chemin d'Harraud Turrou +33-5-61-50-97-01 (Fax) 31190 AUTERIVE +33-6-07-63-80-64 (GSM) FRANCE Email: ohp@pyrenet.fr ------------------------------------------------------------------------------ Make your life a dream, make your dream a reality. (St Exupery)
--On Tuesday, January 27, 2004 23:01:45 +0100 ohp@pyrenet.fr wrote: > On Tue, 27 Jan 2004, Larry Rosenman wrote: > >> Date: Tue, 27 Jan 2004 15:55:49 -0600 >> From: Larry Rosenman <ler@lerctr.org> >> To: ohp@pyrenet.fr >> Cc: pgsql-hackers list <pgsql-hackers@postgresql.org> >> Subject: Re: Write cache >> >> >> >> --On Tuesday, January 27, 2004 22:48:42 +0100 ohp@pyrenet.fr wrote: >> >> > On Tue, 27 Jan 2004, Larry Rosenman wrote: >> > >> >> Date: Tue, 27 Jan 2004 15:45:20 -0600 >> >> From: Larry Rosenman <ler@lerctr.org> >> >> To: ohp@pyrenet.fr >> >> Cc: pgsql-hackers list <pgsql-hackers@postgresql.org> >> >> Subject: Re: Write cache >> >> >> >> >> >> >> >> --On Tuesday, January 27, 2004 22:43:34 +0100 ohp@pyrenet.fr wrote: >> >> >> >> > On Tue, 27 Jan 2004, Larry Rosenman wrote: >> >> > >> >> >> Date: Tue, 27 Jan 2004 15:38:30 -0600 >> >> >> From: Larry Rosenman <ler@lerctr.org> >> >> >> To: ohp@pyrenet.fr, pgsql-hackers list >> >> >> <pgsql-hackers@postgresql.org> Subject: Re: Write cache >> >> >> >> >> >> IDE or SCSI? >> >> > SCSI >> >> >> >> >> >> Why do you think the WC is screwing you? >> >> >> >> >> > Because after a sys crash, the most used filesystems (databases) are >> >> > screwed >> >> what VxFS mount options are you using? >> > Nothing special... And any option I could use would'nt change a thing: >> > the cache is on the disk itself... I did'nt look physicaly yet but >> > according to the docs there's no way to disable it lik I always did on >> > IBM... >> > >> > I was forced to buy those disk (more expensive, not better) >> define not special? >> >> ISTM that the driver should force it out to the disk, unless the disk is >> lying to the driver, or the driver is buggy. > That's exactly what I think: the DISK has a write cache so it's lying to > the controler saying data is written when it's only in the disk cache. > > That also would explain the fantastic write performance I have compared to > read... SO, I consider these disks buggy or not acceptable for use. LER >> >> >> > > >> >> >> >> >> Which driver(s)? >> >> >> >> >> > Guess... ADPU320 >> >> Hrm. >> >> >> >> >> LER >> >> >> >> >> >> >> >> >> --On Tuesday, January 27, 2004 22:34:46 +0100 ohp@pyrenet.fr wrote: >> >> >> >> >> >> > Because I've lost a lot of data using postgresql (and I know for >> >> >> > sure this should'nt happen) I've gone a bit further reading >> >> >> > documentations on my disks and... >> >> >> > >> >> >> > I have FUJITSU MAP3367NC 36G 10000 rpm disks, those disk have a >> >> >> > write cache of 8Mb, if someone could tell me hox to turn it >> >> >> > off... Could'nt find it in the docs and this could very well >> >> >> > explain why a busy database crashes every time >> >> >> > >> >> >> > Thanks for your help >> >> >> > >> >> >> > -- >> >> >> > Olivier PRENANT Tel: +33-5-61-50-97-00 (Work) >> >> >> > 6, Chemin d'Harraud Turrou +33-5-61-50-97-01 (Fax) >> >> >> > 31190 AUTERIVE +33-6-07-63-80-64 (GSM) >> >> >> > FRANCE Email: ohp@pyrenet.fr >> >> >> > ---------------------------------------------------------------- >> >> >> > --- --- --- ----- Make your life a dream, make your dream a >> >> >> > reality. (St Exupery) >> >> >> >> >> >> >> >> >> >> >> >> >> >> > >> >> > -- >> >> > Olivier PRENANT Tel: +33-5-61-50-97-00 (Work) >> >> > 6, Chemin d'Harraud Turrou +33-5-61-50-97-01 (Fax) >> >> > 31190 AUTERIVE +33-6-07-63-80-64 (GSM) >> >> > FRANCE Email: ohp@pyrenet.fr >> >> > ------------------------------------------------------------------- >> >> > --- --- ----- Make your life a dream, make your dream a reality. (St >> >> > Exupery) >> >> >> >> >> >> >> >> >> > >> > -- >> > Olivier PRENANT Tel: +33-5-61-50-97-00 (Work) >> > 6, Chemin d'Harraud Turrou +33-5-61-50-97-01 (Fax) >> > 31190 AUTERIVE +33-6-07-63-80-64 (GSM) >> > FRANCE Email: ohp@pyrenet.fr >> > ---------------------------------------------------------------------- >> > --- ----- Make your life a dream, make your dream a reality. (St >> > Exupery) >> >> >> >> > > -- > Olivier PRENANT Tel: +33-5-61-50-97-00 (Work) > 6, Chemin d'Harraud Turrou +33-5-61-50-97-01 (Fax) > 31190 AUTERIVE +33-6-07-63-80-64 (GSM) > FRANCE Email: ohp@pyrenet.fr > ------------------------------------------------------------------------- > ----- Make your life a dream, make your dream a reality. (St Exupery) -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 972-414-9812 E-Mail: ler@lerctr.org US Mail: 1905 Steamboat Springs Drive, Garland, TX 75044-6749
On Tue, 27 Jan 2004, Larry Rosenman wrote: > Date: Tue, 27 Jan 2004 16:02:40 -0600 > From: Larry Rosenman <ler@lerctr.org> > To: ohp@pyrenet.fr > Cc: pgsql-hackers list <pgsql-hackers@postgresql.org> > Subject: Re: Write cache > > > > --On Tuesday, January 27, 2004 23:01:45 +0100 ohp@pyrenet.fr wrote: > > > On Tue, 27 Jan 2004, Larry Rosenman wrote: > > > >> Date: Tue, 27 Jan 2004 15:55:49 -0600 > >> From: Larry Rosenman <ler@lerctr.org> > >> To: ohp@pyrenet.fr > >> Cc: pgsql-hackers list <pgsql-hackers@postgresql.org> > >> Subject: Re: Write cache > >> > >> > >> > >> --On Tuesday, January 27, 2004 22:48:42 +0100 ohp@pyrenet.fr wrote: > >> > >> > On Tue, 27 Jan 2004, Larry Rosenman wrote: > >> > > >> >> Date: Tue, 27 Jan 2004 15:45:20 -0600 > >> >> From: Larry Rosenman <ler@lerctr.org> > >> >> To: ohp@pyrenet.fr > >> >> Cc: pgsql-hackers list <pgsql-hackers@postgresql.org> > >> >> Subject: Re: Write cache > >> >> > >> >> > >> >> > >> >> --On Tuesday, January 27, 2004 22:43:34 +0100 ohp@pyrenet.fr wrote: > >> >> > >> >> > On Tue, 27 Jan 2004, Larry Rosenman wrote: > >> >> > > >> >> >> Date: Tue, 27 Jan 2004 15:38:30 -0600 > >> >> >> From: Larry Rosenman <ler@lerctr.org> > >> >> >> To: ohp@pyrenet.fr, pgsql-hackers list > >> >> >> <pgsql-hackers@postgresql.org> Subject: Re: Write cache > >> >> >> > >> >> >> IDE or SCSI? > >> >> > SCSI > >> >> >> > >> >> >> Why do you think the WC is screwing you? > >> >> >> > >> >> > Because after a sys crash, the most used filesystems (databases) are > >> >> > screwed > >> >> what VxFS mount options are you using? > >> > Nothing special... And any option I could use would'nt change a thing: > >> > the cache is on the disk itself... I did'nt look physicaly yet but > >> > according to the docs there's no way to disable it lik I always did on > >> > IBM... > >> > > >> > I was forced to buy those disk (more expensive, not better) > >> define not special? > >> > >> ISTM that the driver should force it out to the disk, unless the disk is > >> lying to the driver, or the driver is buggy. > > That's exactly what I think: the DISK has a write cache so it's lying to > > the controler saying data is written when it's only in the disk cache. > > > > That also would explain the fantastic write performance I have compared to > > read... > SO, I consider these disks buggy or not acceptable for use. So do I... I have 6 disks... 250$ each... > > LER > > >> > >> > >> > > > >> >> > >> >> >> Which driver(s)? > >> >> >> > >> >> > Guess... ADPU320 > >> >> Hrm. > >> >> > >> >> >> LER > >> >> >> > >> >> >> > >> >> >> --On Tuesday, January 27, 2004 22:34:46 +0100 ohp@pyrenet.fr wrote: > >> >> >> > >> >> >> > Because I've lost a lot of data using postgresql (and I know for > >> >> >> > sure this should'nt happen) I've gone a bit further reading > >> >> >> > documentations on my disks and... > >> >> >> > > >> >> >> > I have FUJITSU MAP3367NC 36G 10000 rpm disks, those disk have a > >> >> >> > write cache of 8Mb, if someone could tell me hox to turn it > >> >> >> > off... Could'nt find it in the docs and this could very well > >> >> >> > explain why a busy database crashes every time > >> >> >> > > >> >> >> > Thanks for your help > >> >> >> > > >> >> >> > -- > >> >> >> > Olivier PRENANT Tel: +33-5-61-50-97-00 (Work) > >> >> >> > 6, Chemin d'Harraud Turrou +33-5-61-50-97-01 (Fax) > >> >> >> > 31190 AUTERIVE +33-6-07-63-80-64 (GSM) > >> >> >> > FRANCE Email: ohp@pyrenet.fr > >> >> >> > ---------------------------------------------------------------- > >> >> >> > --- --- --- ----- Make your life a dream, make your dream a > >> >> >> > reality. (St Exupery) > >> >> >> > >> >> >> > >> >> >> > >> >> >> > >> >> > > >> >> > -- > >> >> > Olivier PRENANT Tel: +33-5-61-50-97-00 (Work) > >> >> > 6, Chemin d'Harraud Turrou +33-5-61-50-97-01 (Fax) > >> >> > 31190 AUTERIVE +33-6-07-63-80-64 (GSM) > >> >> > FRANCE Email: ohp@pyrenet.fr > >> >> > ------------------------------------------------------------------- > >> >> > --- --- ----- Make your life a dream, make your dream a reality. (St > >> >> > Exupery) > >> >> > >> >> > >> >> > >> >> > >> > > >> > -- > >> > Olivier PRENANT Tel: +33-5-61-50-97-00 (Work) > >> > 6, Chemin d'Harraud Turrou +33-5-61-50-97-01 (Fax) > >> > 31190 AUTERIVE +33-6-07-63-80-64 (GSM) > >> > FRANCE Email: ohp@pyrenet.fr > >> > ---------------------------------------------------------------------- > >> > --- ----- Make your life a dream, make your dream a reality. (St > >> > Exupery) > >> > >> > >> > >> > > > > -- > > Olivier PRENANT Tel: +33-5-61-50-97-00 (Work) > > 6, Chemin d'Harraud Turrou +33-5-61-50-97-01 (Fax) > > 31190 AUTERIVE +33-6-07-63-80-64 (GSM) > > FRANCE Email: ohp@pyrenet.fr > > ------------------------------------------------------------------------- > > ----- Make your life a dream, make your dream a reality. (St Exupery) > > > > -- Olivier PRENANT Tel: +33-5-61-50-97-00 (Work) 6, Chemin d'Harraud Turrou +33-5-61-50-97-01 (Fax) 31190 AUTERIVE +33-6-07-63-80-64 (GSM) FRANCE Email: ohp@pyrenet.fr ------------------------------------------------------------------------------ Make your life a dream, make your dream a reality. (St Exupery)
--On Tuesday, January 27, 2004 23:03:56 +0100 ohp@pyrenet.fr wrote: > On Tue, 27 Jan 2004, Larry Rosenman wrote: > >> Date: Tue, 27 Jan 2004 16:02:40 -0600 >> From: Larry Rosenman <ler@lerctr.org> >> To: ohp@pyrenet.fr >> Cc: pgsql-hackers list <pgsql-hackers@postgresql.org> >> Subject: Re: Write cache >> >> >> >> --On Tuesday, January 27, 2004 23:01:45 +0100 ohp@pyrenet.fr wrote: >> >> > On Tue, 27 Jan 2004, Larry Rosenman wrote: >> > >> >> Date: Tue, 27 Jan 2004 15:55:49 -0600 >> >> From: Larry Rosenman <ler@lerctr.org> >> >> To: ohp@pyrenet.fr >> >> Cc: pgsql-hackers list <pgsql-hackers@postgresql.org> >> >> Subject: Re: Write cache >> >> >> >> >> >> >> >> --On Tuesday, January 27, 2004 22:48:42 +0100 ohp@pyrenet.fr wrote: >> >> >> >> > On Tue, 27 Jan 2004, Larry Rosenman wrote: >> >> > >> >> >> Date: Tue, 27 Jan 2004 15:45:20 -0600 >> >> >> From: Larry Rosenman <ler@lerctr.org> >> >> >> To: ohp@pyrenet.fr >> >> >> Cc: pgsql-hackers list <pgsql-hackers@postgresql.org> >> >> >> Subject: Re: Write cache >> >> >> >> >> >> >> >> >> >> >> >> --On Tuesday, January 27, 2004 22:43:34 +0100 ohp@pyrenet.fr wrote: >> >> >> >> >> >> > On Tue, 27 Jan 2004, Larry Rosenman wrote: >> >> >> > >> >> >> >> Date: Tue, 27 Jan 2004 15:38:30 -0600 >> >> >> >> From: Larry Rosenman <ler@lerctr.org> >> >> >> >> To: ohp@pyrenet.fr, pgsql-hackers list >> >> >> >> <pgsql-hackers@postgresql.org> Subject: Re: Write cache >> >> >> >> >> >> >> >> IDE or SCSI? >> >> >> > SCSI >> >> >> >> >> >> >> >> Why do you think the WC is screwing you? >> >> >> >> >> >> >> > Because after a sys crash, the most used filesystems (databases) >> >> >> > are screwed >> >> >> what VxFS mount options are you using? >> >> > Nothing special... And any option I could use would'nt change a >> >> > thing: the cache is on the disk itself... I did'nt look physicaly >> >> > yet but according to the docs there's no way to disable it lik I >> >> > always did on IBM... >> >> > >> >> > I was forced to buy those disk (more expensive, not better) >> >> define not special? >> >> >> >> ISTM that the driver should force it out to the disk, unless the disk >> >> is lying to the driver, or the driver is buggy. >> > That's exactly what I think: the DISK has a write cache so it's lying >> > to the controler saying data is written when it's only in the disk >> > cache. >> > >> > That also would explain the fantastic write performance I have >> > compared to read... >> SO, I consider these disks buggy or not acceptable for use. > So do I... I have 6 disks... 250$ each... What kind of warranty? How Old? >> >> LER >> >> >> >> >> >> >> > > >> >> >> >> >> >> >> Which driver(s)? >> >> >> >> >> >> >> > Guess... ADPU320 >> >> >> Hrm. >> >> >> >> >> >> >> LER >> >> >> >> >> >> >> >> >> >> >> >> --On Tuesday, January 27, 2004 22:34:46 +0100 ohp@pyrenet.fr >> >> >> >> wrote: >> >> >> >> >> >> >> >> > Because I've lost a lot of data using postgresql (and I know >> >> >> >> > for sure this should'nt happen) I've gone a bit further >> >> >> >> > reading documentations on my disks and... >> >> >> >> > >> >> >> >> > I have FUJITSU MAP3367NC 36G 10000 rpm disks, those disk have >> >> >> >> > a write cache of 8Mb, if someone could tell me hox to turn it >> >> >> >> > off... Could'nt find it in the docs and this could very well >> >> >> >> > explain why a busy database crashes every time >> >> >> >> > >> >> >> >> > Thanks for your help >> >> >> >> > >> >> >> >> > -- >> >> >> >> > Olivier PRENANT Tel: +33-5-61-50-97-00 (Work) >> >> >> >> > 6, Chemin d'Harraud Turrou +33-5-61-50-97-01 (Fax) >> >> >> >> > 31190 AUTERIVE +33-6-07-63-80-64 (GSM) >> >> >> >> > FRANCE Email: ohp@pyrenet.fr >> >> >> >> > ------------------------------------------------------------- >> >> >> >> > --- --- --- --- ----- Make your life a dream, make your dream >> >> >> >> > a reality. (St Exupery) >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> > >> >> >> > -- >> >> >> > Olivier PRENANT Tel: +33-5-61-50-97-00 (Work) >> >> >> > 6, Chemin d'Harraud Turrou +33-5-61-50-97-01 (Fax) >> >> >> > 31190 AUTERIVE +33-6-07-63-80-64 (GSM) >> >> >> > FRANCE Email: ohp@pyrenet.fr >> >> >> > ---------------------------------------------------------------- >> >> >> > --- --- --- ----- Make your life a dream, make your dream a >> >> >> > reality. (St Exupery) >> >> >> >> >> >> >> >> >> >> >> >> >> >> > >> >> > -- >> >> > Olivier PRENANT Tel: +33-5-61-50-97-00 (Work) >> >> > 6, Chemin d'Harraud Turrou +33-5-61-50-97-01 (Fax) >> >> > 31190 AUTERIVE +33-6-07-63-80-64 (GSM) >> >> > FRANCE Email: ohp@pyrenet.fr >> >> > ------------------------------------------------------------------- >> >> > --- --- ----- Make your life a dream, make your dream a reality. (St >> >> > Exupery) >> >> >> >> >> >> >> >> >> > >> > -- >> > Olivier PRENANT Tel: +33-5-61-50-97-00 (Work) >> > 6, Chemin d'Harraud Turrou +33-5-61-50-97-01 (Fax) >> > 31190 AUTERIVE +33-6-07-63-80-64 (GSM) >> > FRANCE Email: ohp@pyrenet.fr >> > ---------------------------------------------------------------------- >> > --- ----- Make your life a dream, make your dream a reality. (St >> > Exupery) >> >> >> >> > > -- > Olivier PRENANT Tel: +33-5-61-50-97-00 (Work) > 6, Chemin d'Harraud Turrou +33-5-61-50-97-01 (Fax) > 31190 AUTERIVE +33-6-07-63-80-64 (GSM) > FRANCE Email: ohp@pyrenet.fr > ------------------------------------------------------------------------- > ----- Make your life a dream, make your dream a reality. (St Exupery) -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 972-414-9812 E-Mail: ler@lerctr.org US Mail: 1905 Steamboat Springs Drive, Garland, TX 75044-6749
ohp@pyrenet.fr wrote: >>>>>> >>>>>Nothing special... And any option I could use would'nt change a thing: >>>>>the cache is on the disk itself... I did'nt look physicaly yet but >>>>>according to the docs there's no way to disable it lik I always did on >>>>>IBM... >>>>> >>>>>I was forced to buy those disk (more expensive, not better) >>>>> >>>>> >>>>define not special? >>>> >>>>ISTM that the driver should force it out to the disk, unless the disk is >>>>lying to the driver, or the driver is buggy. >>>> >>>> >>>That's exactly what I think: the DISK has a write cache so it's lying to >>>the controler saying data is written when it's only in the disk cache. >>> >>> >>> This would be the first time a SCSI disk lies about its write caching. There are plenty of low-cost (i.e. IDE) disks out there having a hidden write cache, but AFAIK a generic SCSI tool is usable to enable/disable the write cache. I'd be quite surprised if your disks wouldn't allow disabling write caching, because SCSI disks are usually targeted towards professional usage. Regards, Andreas
Andreas Pflug <pgadmin@pse-consulting.de> writes:
> This would be the first time a SCSI disk lies about its write caching.
> There are plenty of low-cost (i.e. IDE) disks out there having a hidden
> write cache, but AFAIK a generic SCSI tool is usable to enable/disable
> the write cache.
A SCSI disk shouldn't lie about write completion in any case; there's no
need to because the interface spec is inherently multi-threaded (unlike
IDE). See past discussions.
regards, tom lane
Hi Simon, Sorry I couldn't answer sooner. Hope your daughter is OK by now. On Wed, 28 Jan 2004, Simon Riggs wrote: > Date: Wed, 28 Jan 2004 14:56:40 -0000 > From: Simon Riggs <simon@2ndquadrant.com> > To: ohp@pyrenet.fr, 'pgsql-hackers list' <pgsql-hackers@postgresql.org> > Subject: RE: [HACKERS] Write cache > > > Olivier PRENANT writes... > > > > Because I've lost a lot of data using postgresql (and I know for sure > this > > should'nt happen) I've gone a bit further reading documentations on my > > disks and... > > > > The bottom line here is that Olivier has lost some data and I'm sure we > all want to know if there is a bug in PostgreSQL, or he has a hardware > problem. However, PostgreSQL is partially implicated only because it > discovered the error, but hasn't in any other way been associated yet > with the fatal crash itself. I agree I MAY have an hardware problem. What happens is more a system freeze than a system crash (there's no panic, no nothing, just freezes, no disk activity, not network) What bothers me is that the fs itself was badly hurt, although fsck did repair errors, postgresql complained that it could'nt read a file (relation) that obviously had a wrong block number somewhere. Now, what puzzle me is that my fs are all vxfs, with an intent log. Fairly like postgres. In that case, how can I loose data with it? Also I have mysql on the same filesystem (although VERY quiet) and it did'nt suffer. Postgresql is doing a LOT of job here, and since I host this very busy database I experience data loose in case of crash. This is NOT intended to start a war, I love postgres and I'm very confident in it, but I may have a configuration where ch.. happens. (like the 32 WAL buffers I have) Likewise, I'd like to understand that statistic buffer full condition> > My intuition tells me that this is hardware related. We've discussed > some probable causes, but nobody has come up with a diagnostic test to > evaluate the disks accuracy. This might be because this forum isn't the > most appropriate place to discuss disk storage or linux device drivers? > > Olivier: if your disks are supported or under warranty, then my advice > would be to contact these people and ask for details of a suitable > diagnostic test, or go via their support forums to research this. > Expensive disks are usually fairly well supported, especially if they > smell an upgrade. :) > According to my vendor, there is NO write cache, and the system freeze is the heart of the problem > My experience with other RDBMS vendor's support teams is that they give > out this advice regularly when faced with RDBMS-reported data corruption > errors: "check your disks are working"; I think it is reasonable to do > the same here. Data corruption by the dbms does occur, but my experience > is that this is frequent than hardware-related causes. In the past, I > have used the dd command to squirt data at the disk, then read it back > again - but there may be reasons I don't know why a success on that test > might not be conclusive, so I personally would be happy to defer to > someone that does. I've seen errors like this come from soon-to-fail > disks, poor device drivers, failing non-volatile RAM, cabinet backplane > noise, poorly wired cabling and intermittently used shared SCSI... The problem is that while the system is up and running, I have no log of any error, it goes very fast does it's job correctly. > > Best of luck, Simon Riggs > > Many thanks to all for your help Regards -- Olivier PRENANT Tel: +33-5-61-50-97-00 (Work) 6, Chemin d'Harraud Turrou +33-5-61-50-97-01 (Fax) 31190 AUTERIVE +33-6-07-63-80-64 (GSM) FRANCE Email: ohp@pyrenet.fr ------------------------------------------------------------------------------ Make your life a dream, make your dream a reality. (St Exupery)
On Wed, 28 Jan 2004 ohp@pyrenet.fr wrote: > I agree I MAY have an hardware problem. What happens is more a system > freeze than a system crash (there's no panic, no nothing, just freezes, no > disk activity, not network) I would suspect either bad hardware,a flakey SCSI driver, or a possible kernel bug. If your system is freezing hard, it is NOT postgresql's fault. It simply doesn't have the access to the kind of system resources needed to freeze a machine. Is there a different SCSI driver / card you can try in there? We've (and many others have too) had good luck with the LSI/MegaRAID cards and both the older 1.18 seris and new 2.x series drivers. No freezes, no crashes, no hangs on the boxes with those in them.
> Olivier PRENANT writes... > > Because I've lost a lot of data using postgresql (and I know for sure this > should'nt happen) I've gone a bit further reading documentations on my > disks and... > The bottom line here is that Olivier has lost some data and I'm sure we all want to know if there is a bug in PostgreSQL, or he has a hardware problem. However, PostgreSQL is partially implicated only because it discovered the error, but hasn't in any other way been associated yet with the fatal crash itself. My intuition tells me that this is hardware related. We've discussed some probable causes, but nobody has come up with a diagnostic test to evaluate the disks accuracy. This might be because this forum isn't the most appropriate place to discuss disk storage or linux device drivers? Olivier: if your disks are supported or under warranty, then my advice would be to contact these people and ask for details of a suitable diagnostic test, or go via their support forums to research this. Expensive disks are usually fairly well supported, especially if they smell an upgrade. :) My experience with other RDBMS vendor's support teams is that they give out this advice regularly when faced with RDBMS-reported data corruption errors: "check your disks are working"; I think it is reasonable to do the same here. Data corruption by the dbms does occur, but my experience is that this is frequent than hardware-related causes. In the past, I have used the dd command to squirt data at the disk, then read it back again - but there may be reasons I don't know why a success on that test might not be conclusive, so I personally would be happy to defer to someone that does. I've seen errors like this come from soon-to-fail disks, poor device drivers, failing non-volatile RAM, cabinet backplane noise, poorly wired cabling and intermittently used shared SCSI... Best of luck, Simon Riggs
"Simon Riggs" <simon@2ndquadrant.com> writes: > In the past, I have used the dd command to squirt data at the disk, then > read it back again - but there may be reasons I don't know why a success on > that test might not be conclusive, so I personally would be happy to defer > to someone that does. Well that's an interesting tangent. 1) I've seen bad memory on a scsi controller once. Caused one-bit errors in data read back after being written. a dd mightor might not find that depending on the buffer usage pattern, and depending on the pattern being written and read.Memory errors are notoriously fickle and can sometimes be triggered only by particular bit patterns in adjacent memoryaddresses in rapid succession. badblocks does try to address this by writing four different complementary patterns. but I'm not convinced it's reallyconclusive either. It's certainly not as sophisticated as memtest86 and can't really since it can't directly controlthe placement of data in the disk's buffers. 2) The disk could be finding lots of bad blocks during the dd run and remapping them. It gives no information to the OSthrough the regular interfaces. A low level diagnostic program can inquire about how many blocks have been remapped andhow many spare blocks are available. I know Maxtor is hot to have you run their PowerMax(tm) program whenever you call tech support. I think it just runs something similar to badblocks and asks the disk firmware if it's detected any low level problems. In theory it can check things like the drive having trouble syncing to tracks due to environmental factors like noise, vibrations, and heat. I don't know if it does or not though. -- greg
--On Tuesday, January 27, 2004 23:15:09 +0100 ohp@pyrenet.fr wrote: > On Tue, 27 Jan 2004, Larry Rosenman wrote: >> >> >> >> >> >> ISTM that the driver should force it out to the disk, unless the >> >> >> disk is lying to the driver, or the driver is buggy. >> >> > That's exactly what I think: the DISK has a write cache so it's >> >> > lying to the controler saying data is written when it's only in the >> >> > disk cache. >> >> > >> >> > That also would explain the fantastic write performance I have >> >> > compared to read... >> >> SO, I consider these disks buggy or not acceptable for use. >> > So do I... I have 6 disks... 250$ each... >> What kind of warranty? > 3 years 24 hours on site (the computer altogather) >> >> How Old? >> > 3 months > I'd tell them to get you disks that don't lie. and be FIRM about it. LER -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 972-414-9812 E-Mail: ler@lerctr.org US Mail: 1905 Steamboat Springs Drive, Garland, TX 75044-6749
On Tue, 27 Jan 2004, Larry Rosenman wrote: > >> >> > >> >> ISTM that the driver should force it out to the disk, unless the disk > >> >> is lying to the driver, or the driver is buggy. > >> > That's exactly what I think: the DISK has a write cache so it's lying > >> > to the controler saying data is written when it's only in the disk > >> > cache. > >> > > >> > That also would explain the fantastic write performance I have > >> > compared to read... > >> SO, I consider these disks buggy or not acceptable for use. > > So do I... I have 6 disks... 250$ each... > What kind of warranty? 3 years 24 hours on site (the computer altogather) > > How Old? > 3 months -- Olivier PRENANT Tel: +33-5-61-50-97-00 (Work) 6, Chemin d'Harraud Turrou +33-5-61-50-97-01 (Fax) 31190 AUTERIVE +33-6-07-63-80-64 (GSM) FRANCE Email: ohp@pyrenet.fr ------------------------------------------------------------------------------ Make your life a dream, make your dream a reality. (St Exupery)
Tom Lane wrote: >Andreas Pflug <pgadmin@pse-consulting.de> writes: > > >>This would be the first time a SCSI disk lies about its write caching. >>There are plenty of low-cost (i.e. IDE) disks out there having a hidden >>write cache, but AFAIK a generic SCSI tool is usable to enable/disable >>the write cache. >> >> > >A SCSI disk shouldn't lie about write completion in any case; there's no >need to because the interface spec is inherently multi-threaded (unlike >IDE). See past discussions. > > > Unless I am reading it wrong, http://www.fcpa.com/download/download/hard-drives/map10krpm-manual.pdf appears to suggest (page 5-21) that the write cache can be disabled. How I am not sure - maybe using one of the tools Andreas mentioned. cheers andrew
Andrew Dunstan wrote: > > > Tom Lane wrote: > > >Andreas Pflug <pgadmin@pse-consulting.de> writes: > > > > > >>This would be the first time a SCSI disk lies about its write caching. > >>There are plenty of low-cost (i.e. IDE) disks out there having a hidden > >>write cache, but AFAIK a generic SCSI tool is usable to enable/disable > >>the write cache. > >> > >> > > > >A SCSI disk shouldn't lie about write completion in any case; there's no > >need to because the interface spec is inherently multi-threaded (unlike > >IDE). See past discussions. > > > > > > > Unless I am reading it wrong, > > http://www.fcpa.com/download/download/hard-drives/map10krpm-manual.pdf > appears to suggest (page 5-21) that the write cache can be disabled. How > I am not sure - maybe using one of the tools Andreas mentioned. My guess is to look for scsicmd or something like that. The man page usually talks about how to dump the device parametes. Do that and you should see something clear. -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001+ If your life is a hard drive, | 13 Roberts Road + Christ can be your backup. | Newtown Square, Pennsylvania19073