Thread: Point in time recovery 20020822_01_pitr.patch
Here's a patch to do point-in-time recovery against 7.3 head. Still to
be resolved in this patch:
* CREATE/DROP database are still unlogged. dbcommands.c needs a
re-write to make that possible, since all the file and directory
operations will need to be logged, as will the data.
* Is it really safe to do block-aligned reads to copy the database
files once (if) we remove backup blocks from the log? Should support be
added to go through the buffer manager?
New Files:
./doc/FAQ_PITR
./src/backend/commands/recoverycmds.c
./src/include/commands/recoverycmds.h
--
J. R. Nield
jrnield@usol.com
Attachment
Patch applied. Thanks. Ha. Ha. Just messing with you guys. I didn't apply anything, but this does give us something to talk about. Thanks. --------------------------------------------------------------------------- J. R. Nield wrote: > Here's a patch to do point-in-time recovery against 7.3 head. Still to > be resolved in this patch: > > * CREATE/DROP database are still unlogged. dbcommands.c needs a > re-write to make that possible, since all the file and directory > operations will need to be logged, as will the data. > > * Is it really safe to do block-aligned reads to copy the database > files once (if) we remove backup blocks from the log? Should support be > added to go through the buffer manager? > > > New Files: > ./doc/FAQ_PITR > ./src/backend/commands/recoverycmds.c > ./src/include/commands/recoverycmds.h > > > -- > J. R. Nield > jrnield@usol.com > > [ Attachment, skipping... ] -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup. | Newtown Square, Pennsylvania 19073
"J. R. Nield" <jrnield@usol.com> writes:
> * CREATE/DROP database are still unlogged. dbcommands.c needs a
> re-write to make that possible, since all the file and directory
> operations will need to be logged, as will the data.
Why don't you log the operations symbolically, viz "create database foo
with template bar"? I can't see any reason to insist on a finer level
of detail than that.
Just so you know: my feeling is this is already too late for 7.3.
regards, tom lane
J. R. Nield wrote: > Here's a patch to do point-in-time recovery against 7.3 head. Still to > be resolved in this patch: > > * CREATE/DROP database are still unlogged. dbcommands.c needs a > re-write to make that possible, since all the file and directory > operations will need to be logged, as will the data. > > * Is it really safe to do block-aligned reads to copy the database > files once (if) we remove backup blocks from the log? Should support be > added to go through the buffer manager? > > > New Files: > ./doc/FAQ_PITR > ./src/backend/commands/recoverycmds.c > ./src/include/commands/recoverycmds.h Here is quick overview of the patch. First, it starts with documentation about how the user interacts with the PITR system. Then it goes into a large chunk of code that logs and recovers from a variety of database changes that we don't currently log because they aren't needed for crash recovery but are required for PITR recovery. Next is a section that does the actual file backups for PITR, and finally, there is grammar code to support the new PITR commands. -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup. | Newtown Square, Pennsylvania 19073
Tom Lane wrote:
> "J. R. Nield" <jrnield@usol.com> writes:
> > * CREATE/DROP database are still unlogged. dbcommands.c needs a
> > re-write to make that possible, since all the file and directory
> > operations will need to be logged, as will the data.
>
> Why don't you log the operations symbolically, viz "create database foo
> with template bar"? I can't see any reason to insist on a finer level
> of detail than that.
>
>
> Just so you know: my feeling is this is already too late for 7.3.
I understand Tom's concerns. We have already talked about this on the
phone because we anticipated this may happen.
We are one week from beta. I think what we should do is to give
everyone a day to look over the patch and submit their ideas about it to
the group. We can then review the possible scenarios we can take:
apply and contine to September 1 beta
apply and delay beta until October 1
create a patch file against 7.3 for distribution
wait for 7.4 and do not distribute
--
Bruce Momjian | http://candle.pha.pa.us
pgman@candle.pha.pa.us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073
> I understand Tom's concerns. We have already talked about this on the > phone because we anticipated this may happen. > > We are one week from beta. I think what we should do is to give > everyone a day to look over the patch and submit their ideas about it to > the group. We can then review the possible scenarios we can take: > > apply and contine to September 1 beta > apply and delay beta until October 1 > create a patch file against 7.3 for distribution > wait for 7.4 and do not distribute I too understand Tom's concerns. It is a lot of changes in various important parts of code and it won't have been tested enough, etc., etc. I say leave it for 7.4. Distributing a patch against 7.3 will be very difficult to keep up-to-date as the CVS tree changes. Maybe we could look to aim to have win32 native and PITR for 7.4 and so as soon as they're in, working and tested we just release 7.4 - why not? I guess maybe the wire protocol might change as well. And what about neil's PREPARE? Chris
"Christopher Kings-Lynne" <chriskl@familyhealth.com.au> writes:
> And what about neil's PREPARE?
PREPARE I think we will have reviewed and in there for beta; I just need
to find a couple of hours for it. (I already looked at it once but
want to take another look at the revised patch.)
PREPARE is hardly of the magnitude of PITR anyway: the worst-case
scenario with PREPARE is that it's broken and people can't safely
use it in 7.3. It can't possibly hurt you if you don't use it.
In contrast, the worst-case scenario with PITR is, um, dire.
regards, tom lane
the fact that JR acknowledges issues that are yet to be resolved, and that we are in the final week before beta ... I agree with Tom that this should not go into v7.3, and be left for v7.4 ... On Fri, 23 Aug 2002, Bruce Momjian wrote: > J. R. Nield wrote: > > Here's a patch to do point-in-time recovery against 7.3 head. Still to > > be resolved in this patch: > > > > * CREATE/DROP database are still unlogged. dbcommands.c needs a > > re-write to make that possible, since all the file and directory > > operations will need to be logged, as will the data. > > > > * Is it really safe to do block-aligned reads to copy the database > > files once (if) we remove backup blocks from the log? Should support be > > added to go through the buffer manager? > > > > > > New Files: > > ./doc/FAQ_PITR > > ./src/backend/commands/recoverycmds.c > > ./src/include/commands/recoverycmds.h > > Here is quick overview of the patch. First, it starts with > documentation about how the user interacts with the PITR system. Then > it goes into a large chunk of code that logs and recovers from a variety > of database changes that we don't currently log because they aren't > needed for crash recovery but are required for PITR recovery. Next is a > section that does the actual file backups for PITR, and finally, there > is grammar code to support the new PITR commands. > > -- > Bruce Momjian | http://candle.pha.pa.us > pgman@candle.pha.pa.us | (610) 359-1001 > + If your life is a hard drive, | 13 Roberts Road > + Christ can be your backup. | Newtown Square, Pennsylvania 19073 > > ---------------------------(end of broadcast)--------------------------- > TIP 1: subscribe and unsubscribe commands go to majordomo@postgresql.org >
On Fri, 2002-08-23 at 00:29, Tom Lane wrote: > "J. R. Nield" <jrnield@usol.com> writes: > > * CREATE/DROP database are still unlogged. dbcommands.c needs a > > re-write to make that possible, since all the file and directory > > operations will need to be logged, as will the data. > > Why don't you log the operations symbolically, viz "create database foo > with template bar"? I can't see any reason to insist on a finer level > of detail than that. If you do symbolic logging like that, it forecloses any chance of adding individual relation recovery, because the template might be ahead of the log. That's main reason I avoided doing this with index builds. I don't know if we will want that feature, but I didn't want to prevent us from adding it. > > > Just so you know: my feeling is this is already too late for 7.3. > I'm not in a position to answer this. I'll do whatever I can to address concerns as they arise, and I'll continue to review my own work. Clearly this is not a good area to find bugs "in the field". > regards, tom lane > -- J. R. Nield jrnield@usol.com
"J. R. Nield" <jrnield@usol.com> writes:
> On Fri, 2002-08-23 at 00:29, Tom Lane wrote:
>> Why don't you log the operations symbolically, viz "create database foo
>> with template bar"? I can't see any reason to insist on a finer level
>> of detail than that.
> If you do symbolic logging like that, it forecloses any chance of adding
> individual relation recovery, because the template might be ahead of the
> log.
We are already assuming that the template database is stable while it's
being copied. While there are obvious risks in that assumption, I don't
think that you need to eliminate the assumption as an essential
component of PITR ... and I think that there are a number of issues
that'd have to be solved that are completely unrelated to PITR (eg, how
do you lock a table that's in a database you aren't in, and indeed that
you don't even know exists, because you can't read the pg_class table
that describes it?) My advice: this is a job not to tackle for version
1, and maybe not for version 2, 3, or 4...
regards, tom lane
The sooner the better on this. I really want to get this functionality
into the jdbc driver in 7.3. It is going to be a big performance win
for my applications.
thanks,
--Barry
Tom Lane wrote:
>"Christopher Kings-Lynne" <chriskl@familyhealth.com.au> writes:
>
>
>>And what about neil's PREPARE?
>>
>>
>
>PREPARE I think we will have reviewed and in there for beta; I just need
>to find a couple of hours for it. (I already looked at it once but
>want to take another look at the revised patch.)
>
>PREPARE is hardly of the magnitude of PITR anyway: the worst-case
>scenario with PREPARE is that it's broken and people can't safely
>use it in 7.3. It can't possibly hurt you if you don't use it.
>In contrast, the worst-case scenario with PITR is, um, dire.
>
>
regards, tom lane
>
>---------------------------(end of broadcast)---------------------------
>TIP 3: if posting/reading through Usenet, please send an appropriate
>subscribe-nomail command to majordomo@postgresql.org so that your
>message can get through to the mailing list cleanly
>
>
>