Thread: BufFileWrite across MAX_PHYSICAL_FILESIZE boundary
Just noticed buffile.c:292 doesn't look quite right where BufFileDumpBuffer calls FileWrite: bytestowrite = FileWrite(thisfile, file->buffer, bytestowrite); It looks as though it would write the same data each time the loop is iterated. Would this be better? bytestowrite = FileWrite(thisfile, file->buffer + wpos, bytestowrite);
"Kurt Harriman" <kharriman@greenplum.com> writes:
> Just noticed buffile.c:292 doesn't look quite right where
> BufFileDumpBuffer calls FileWrite:
> bytestowrite = FileWrite(thisfile, file->buffer, bytestowrite);
> It looks as though it would write the same data each time the
> loop is iterated. Would this be better?
> bytestowrite = FileWrite(thisfile, file->buffer + wpos, bytestowrite);
Yeah, I think you're right. We've probably not seen this because in
most usages the buffer is always block-aligned, but it looks possible
for tuplestore.c to get out of alignment if its concurrent write/read
feature is exercised just so. Do you want to try demonstrating the bug
with a smaller-than-normal setting of MAX_PHYSICAL_FILESIZE?
regards, tom lane
This has been saved for the 8.4 release: http://momjian.postgresql.org/cgi-bin/pgpatches_hold --------------------------------------------------------------------------- Tom Lane wrote: > "Kurt Harriman" <kharriman@greenplum.com> writes: > > Just noticed buffile.c:292 doesn't look quite right where > > BufFileDumpBuffer calls FileWrite: > > bytestowrite = FileWrite(thisfile, file->buffer, bytestowrite); > > It looks as though it would write the same data each time the > > loop is iterated. Would this be better? > > bytestowrite = FileWrite(thisfile, file->buffer + wpos, bytestowrite); > > Yeah, I think you're right. We've probably not seen this because in > most usages the buffer is always block-aligned, but it looks possible > for tuplestore.c to get out of alignment if its concurrent write/read > feature is exercised just so. Do you want to try demonstrating the bug > with a smaller-than-normal setting of MAX_PHYSICAL_FILESIZE? > > regards, tom lane > > ---------------------------(end of broadcast)--------------------------- > TIP 5: don't forget to increase your free space map settings -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. +
Bruce Momjian wrote: > > This has been saved for the 8.4 release: > > http://momjian.postgresql.org/cgi-bin/pgpatches_hold Huh, no, this is a bug and should be fixed right away. > --------------------------------------------------------------------------- > > Tom Lane wrote: > > "Kurt Harriman" <kharriman@greenplum.com> writes: > > > Just noticed buffile.c:292 doesn't look quite right where > > > BufFileDumpBuffer calls FileWrite: > > > bytestowrite = FileWrite(thisfile, file->buffer, bytestowrite); > > > It looks as though it would write the same data each time the > > > loop is iterated. Would this be better? > > > bytestowrite = FileWrite(thisfile, file->buffer + wpos, bytestowrite); > > > > Yeah, I think you're right. We've probably not seen this because in > > most usages the buffer is always block-aligned, but it looks possible > > for tuplestore.c to get out of alignment if its concurrent write/read > > feature is exercised just so. Do you want to try demonstrating the bug > > with a smaller-than-normal setting of MAX_PHYSICAL_FILESIZE? -- Alvaro Herrera http://www.CommandPrompt.com/ The PostgreSQL Company - Command Prompt, Inc.
Alvaro Herrera wrote: > Bruce Momjian wrote: > > > > This has been saved for the 8.4 release: > > > > http://momjian.postgresql.org/cgi-bin/pgpatches_hold > > Huh, no, this is a bug and should be fixed right away. OK, moved to the 8.3 patch queue. --------------------------------------------------------------------------- > > > --------------------------------------------------------------------------- > > > > Tom Lane wrote: > > > "Kurt Harriman" <kharriman@greenplum.com> writes: > > > > Just noticed buffile.c:292 doesn't look quite right where > > > > BufFileDumpBuffer calls FileWrite: > > > > bytestowrite = FileWrite(thisfile, file->buffer, bytestowrite); > > > > It looks as though it would write the same data each time the > > > > loop is iterated. Would this be better? > > > > bytestowrite = FileWrite(thisfile, file->buffer + wpos, bytestowrite); > > > > > > Yeah, I think you're right. We've probably not seen this because in > > > most usages the buffer is always block-aligned, but it looks possible > > > for tuplestore.c to get out of alignment if its concurrent write/read > > > feature is exercised just so. Do you want to try demonstrating the bug > > > with a smaller-than-normal setting of MAX_PHYSICAL_FILESIZE? > > > -- > Alvaro Herrera http://www.CommandPrompt.com/ > The PostgreSQL Company - Command Prompt, Inc. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. +
I wrote:
> "Kurt Harriman" <kharriman@greenplum.com> writes:
>> Just noticed buffile.c:292 doesn't look quite right where
>> BufFileDumpBuffer calls FileWrite:
>> bytestowrite = FileWrite(thisfile, file->buffer, bytestowrite);
>> It looks as though it would write the same data each time the
>> loop is iterated. Would this be better?
>> bytestowrite = FileWrite(thisfile, file->buffer + wpos, bytestowrite);
> Yeah, I think you're right.
FYI, I was able to exercise the bug by changing RELSEG_SIZE to 2 within
buffile.c and then running this script in the regression database:
set work_mem = '64kB';
begin;
declare c scroll cursor for select * from tenk1 a join tenk1 b using(two);
fetch 1 from c;
fetch 100 from c;
fetch backward 10 from c;
fetch 100 from c;
fetch backward 10 from c;
fetch 100 from c;
fetch backward 100 from c;
commit;
It doesn't crash with the fix applied. Thanks for spotting it.
regards, tom lane