Thread: PITR Dead horse?

PITR Dead horse?

From
Austin Gonyou
Date:
Has this been beaten to death now? Just curious if PITR was in Dev tree
yet. Been out of the loop. TIA.
-- 
Austin Gonyou <austin@coremetrics.com>
Coremetrics, Inc.


Re: PITR Dead horse?

From
Tom Lane
Date:
Austin Gonyou <austin@coremetrics.com> writes:
> Has this been beaten to death now? Just curious if PITR was in Dev tree
> yet. Been out of the loop. TIA.

Nope... I've got some patches from Patrick Macdonald and JR Nield that I
need to integrate, but I believe those only cover some low-level changes
to the WAL log contents.  There's a lot of management code yet to be
written.
        regards, tom lane


Re: PITR Dead horse?

From
Tatsuo Ishii
Date:
> Has this been beaten to death now? Just curious if PITR was in Dev tree
> yet. Been out of the loop. TIA.

I and my co workers are very interested in implementing PITR. We will
tackle this for 7.5 if no one objects.
--
Tatsuo Ishii


Re: PITR Dead horse?

From
Satoshi Nagayasu
Date:
I and some other developers are also interested in.
Do you think we can work together?

Tatsuo Ishii <t-ishii@sra.co.jp> wrote:
> > Has this been beaten to death now? Just curious if PITR was in Dev tree
> > yet. Been out of the loop. TIA.
> 
> I and my co workers are very interested in implementing PITR. We will
> tackle this for 7.5 if no one objects.
> --
> Tatsuo Ishii
> 
> ---------------------------(end of broadcast)---------------------------
> TIP 2: you can get off all lists at once with the unregister command
>     (send "unregister YourEmailAddressHere" to majordomo@postgresql.org)
> 


-- 
NAGAYASU Satoshi <snaga@snaga.org>


Re: PITR Dead horse?

From
Tom Lane
Date:
Tatsuo Ishii <t-ishii@sra.co.jp> writes:
> I and my co workers are very interested in implementing PITR. We will
> tackle this for 7.5 if no one objects.

Sounds good.  I'll try to push in the work that Patrick and JR did
within the next day or two, and then you can take it from there...
        regards, tom lane


Re: PITR Dead horse?

From
Tatsuo Ishii
Date:
> I and some other developers are also interested in.
> Do you think we can work together?

Sure. Why not. I think it would be practical to decide who is the
leader of this project, though.
--
Tatsuo Ishii


Re: PITR Dead horse?

From
Koichi Suzuki
Date:
Hi, This is Suzuki from NTT DATA Intellilink.

I told Bruce Momjan that I and my colleagues are interested in 
implementing PITR in BOF in NY LW2004.  NTT's laboratory is very 
interested in this issue and I'm planning to work with them.  I hope we 
could cooperate.

Tatsuo Ishii wrote:

>>Has this been beaten to death now? Just curious if PITR was in Dev tree
>>yet. Been out of the loop. TIA.
> 
> 
> I and my co workers are very interested in implementing PITR. We will
> tackle this for 7.5 if no one objects.
> --
> Tatsuo Ishii
> 
> ---------------------------(end of broadcast)---------------------------
> TIP 2: you can get off all lists at once with the unregister command
>     (send "unregister YourEmailAddressHere" to majordomo@postgresql.org)
> 




Re: PITR Dead horse?

From
"Nicolai Tufar"
Date:
I would like to join this effort too.
I was afraid that people at RedHat are already
halfway though and were to release their work 
shortly. But it does not seem to be the case.

Regards,
Nicolai

> -----Original Message-----
> From: pgsql-hackers-owner@postgresql.org [mailto:pgsql-hackers-
> owner@postgresql.org] On Behalf Of Koichi Suzuki
> Sent: Wednesday, February 04, 2004 11:25 AM
> To: Tatsuo Ishii
> Cc: austin@coremetrics.com; pgsql-hackers@postgresql.org
> Subject: Re: [HACKERS] PITR Dead horse?
> 
> Hi, This is Suzuki from NTT DATA Intellilink.
> 
> I told Bruce Momjan that I and my colleagues are interested in
> implementing PITR in BOF in NY LW2004.  NTT's laboratory is very
> interested in this issue and I'm planning to work with them.  I hope
we
> could cooperate.
> 
> Tatsuo Ishii wrote:
> 
> >>Has this been beaten to death now? Just curious if PITR was in Dev
tree
> >>yet. Been out of the loop. TIA.
> >
> >
> > I and my co workers are very interested in implementing PITR. We
will
> > tackle this for 7.5 if no one objects.
> > --
> > Tatsuo Ishii
> >
> > ---------------------------(end of
broadcast)---------------------------
> > TIP 2: you can get off all lists at once with the unregister command
> >     (send "unregister YourEmailAddressHere" to
majordomo@postgresql.org)
> >
> 
> 
> 
> ---------------------------(end of
broadcast)---------------------------
> TIP 1: subscribe and unsubscribe commands go to
majordomo@postgresql.org



Re: PITR Dead horse?

From
"Marc G. Fournier"
Date:
On Wed, 4 Feb 2004, Tatsuo Ishii wrote:

> > I and some other developers are also interested in.
> > Do you think we can work together?
>
> Sure. Why not. I think it would be practical to decide who is the
> leader of this project, though.

Is this something large enough, like the win32 stuff, that having a side
list for discussions is worth setting up?

----
Marc G. Fournier           Hub.Org Networking Services (http://www.hub.org)
Email: scrappy@hub.org           Yahoo!: yscrappy              ICQ: 7615664


Re: PITR Dead horse?

From
Tom Lane
Date:
"Marc G. Fournier" <scrappy@postgresql.org> writes:
> Is this something large enough, like the win32 stuff, that having a side
> list for discussions is worth setting up?

In terms of the amount of code to be written, I expect it's larger than
the win32 porting effort.  And it should be mostly pretty separate from
hacking the core backend, since most of what remains to do is writing
external management utilities (I think).

I've been dissatisfied with having the separate pgsql-hackers-win32
list; I feel it just fragments the discussion, and people tend to end up
crossposting to -hackers anyway.  But a separate list for PITR work
might be a good idea despite that experience, since it seems like it'd
be a more separable project.

Any other opinions out there?
        regards, tom lane


Re: PITR Dead horse?

From
"Simon Riggs"
Date:
>Tom Lane wrote
> "Marc G. Fournier" <scrappy@postgresql.org> writes:
> > Is this something large enough, like the win32 stuff, that having a
side
> > list for discussions is worth setting up?
> 
> In terms of the amount of code to be written, I expect it's larger
than
> the win32 porting effort.  And it should be mostly pretty separate
from
> hacking the core backend, since most of what remains to do is writing
> external management utilities (I think).

Yes it is! I'd like to start the discussion about PITR and try to go
through some functional requirements and how those might be implemented.
The Win32 port has a self-evident set of functional requirements; I'm
not sure that the PITR stuff is as clear - so I couldn't pass any
judgement at all (even if I did know the code well enough) on how big a
coding task that is, but I can see that the analysis and discussion is
large indeed.

> I've been dissatisfied with having the separate pgsql-hackers-win32
> list; I feel it just fragments the discussion, and people tend to end
up
> crossposting to -hackers anyway.  But a separate list for PITR work
> might be a good idea despite that experience, since it seems like it'd
> be a more separable project.

I'd vote for a new list dedicated to discussing "Robustness" issues,
such as PITR and the fsync/sync issues. IMHO, PostgreSQL has the
Functionality and Performance, it just needs some rock-solid analysis of
where-things-can-go-wrong with it, so that the business data centre
people will be able to use it with absolute confidence...even if the
answer is "we've got every base covered". For me, the issues about
robustness are as much to do with risk reduction and confidence building
as they are about specific features in that area. [Wow, I expect some
flames on those comments!]

The list probably would remain clearly differentiated, in the same way
[Performance] covers lots of areas not discussed in [Hackers].

Not hung up on the name either, just something that indicates
breadth-of-scope, e.g. Availability or Data Protection or Resilience
etc..; maybe the Advocates would like to name it? It might even be a
press-release: "PostgreSQL community focuses new efforts towards
Robustness features for its next major release".

Best Regards, Simon Riggs



Re: PITR Dead horse?

From
Tom Lane
Date:
"Simon Riggs" <simon@2ndquadrant.com> writes:
> I'd vote for a new list dedicated to discussing "Robustness" issues,
> such as PITR and the fsync/sync issues. IMHO, PostgreSQL has the
> Functionality and Performance, it just needs some rock-solid analysis of
> where-things-can-go-wrong with it, so that the business data centre
> people will be able to use it with absolute confidence...even if the
> answer is "we've got every base covered". For me, the issues about
> robustness are as much to do with risk reduction and confidence building
> as they are about specific features in that area. [Wow, I expect some
> flames on those comments!]

You're right.  Exactly where do you expect to find the expertise and
interest to do such an analysis?  On pghackers, that's where.  There's
no reason to invent a new mailing list for what should be a continuing
topic in pghackers.  And to the extent that you were to move such a
discussion somewhere else, you'd just risk losing the attention of the
pair of eyeballs that might notice a hole in your analysis.

> Not hung up on the name either, just something that indicates
> breadth-of-scope, e.g. Availability or Data Protection or Resilience
> etc..; maybe the Advocates would like to name it? It might even be a
> press-release: "PostgreSQL community focuses new efforts towards
> Robustness features for its next major release".

I think such a press release would be counterproductive, as it would
immediately make people question whether we have reliability problems.
        regards, tom lane


Re: PITR Dead horse?

From
"Nicolai Tufar"
Date:
Totally agree. Robustness and rock-solidness are the only
things missing for PostgreSQL to become the killer of certain
commercial enterprise databases out there. And the only thing 
that is missing in this respect is PITR. PITR should be there
INGRES had it in '84 and some people as why PostgreSQL does 
not have it.

I am well versed in the internals of "PITR" features of a certain
leading enterprise-class database out there. And would like to
contribute (write code) to this effort  as much as I can.

Best regards,
Nicolai Tufar



> -----Original Message-----
> From: pgsql-hackers-owner@postgresql.org [mailto:pgsql-hackers-
> owner@postgresql.org] On Behalf Of Simon Riggs
> Sent: Thursday, February 05, 2004 1:33 AM
> To: 'Tom Lane'; 'Marc G. Fournier'
> Cc: 'Tatsuo Ishii'; snaga@snaga.org; austin@coremetrics.com; pgsql-
> hackers@postgresql.org
> Subject: Re: [HACKERS] PITR Dead horse?
> 
> >Tom Lane wrote
> > "Marc G. Fournier" <scrappy@postgresql.org> writes:
> > > Is this something large enough, like the win32 stuff, that having
a
> side
> > > list for discussions is worth setting up?
> >
> > In terms of the amount of code to be written, I expect it's larger
> than
> > the win32 porting effort.  And it should be mostly pretty separate
> from
> > hacking the core backend, since most of what remains to do is
writing
> > external management utilities (I think).
> 
> Yes it is! I'd like to start the discussion about PITR and try to go
> through some functional requirements and how those might be
implemented.
> The Win32 port has a self-evident set of functional requirements; I'm
> not sure that the PITR stuff is as clear - so I couldn't pass any
> judgement at all (even if I did know the code well enough) on how big
a
> coding task that is, but I can see that the analysis and discussion is
> large indeed.
> 
> > I've been dissatisfied with having the separate pgsql-hackers-win32
> > list; I feel it just fragments the discussion, and people tend to
end
> up
> > crossposting to -hackers anyway.  But a separate list for PITR work
> > might be a good idea despite that experience, since it seems like
it'd
> > be a more separable project.
> 
> I'd vote for a new list dedicated to discussing "Robustness" issues,
> such as PITR and the fsync/sync issues. IMHO, PostgreSQL has the
> Functionality and Performance, it just needs some rock-solid analysis
of
> where-things-can-go-wrong with it, so that the business data centre
> people will be able to use it with absolute confidence...even if the
> answer is "we've got every base covered". For me, the issues about
> robustness are as much to do with risk reduction and confidence
building
> as they are about specific features in that area. [Wow, I expect some
> flames on those comments!]
> 
> The list probably would remain clearly differentiated, in the same way
> [Performance] covers lots of areas not discussed in [Hackers].
> 
> Not hung up on the name either, just something that indicates
> breadth-of-scope, e.g. Availability or Data Protection or Resilience
> etc..; maybe the Advocates would like to name it? It might even be a
> press-release: "PostgreSQL community focuses new efforts towards
> Robustness features for its next major release".
> 
> Best Regards, Simon Riggs
> 
> 
> ---------------------------(end of
broadcast)---------------------------
> TIP 9: the planner will ignore your desire to choose an index scan if
your
>       joining column's datatypes do not match



Re: PITR Dead horse?

From
Josh Berkus
Date:
Simon,

> I'd vote for a new list dedicated to discussing "Robustness" issues,
> such as PITR and the fsync/sync issues.<snip>
> The list probably would remain clearly differentiated, in the same way
> [Performance] covers lots of areas not discussed in [Hackers].

Actually, Simon, you're welcome to bring this discussion over to PERFORMANCE.  
We discuss scalability and HA on Performance frequently, and I don't feel 
that the discussion you refer to would be out of place.

But Tom is right that you need the feedback of a lot of the people on Hackers 
once you start discussing a code solution, so there's not much point in 
starting a new mailing list that all the same people need to subscribe to.  
Certainly Jan had enough trouble getting meaningful feedback on the sync 
issue here; on his own list he'd still be talking to himself.

As far as promoting an image of reliability, that belongs on Advocacy.   The 
image and the reality don't sync much; we're already about 500% more reliable 
than MS SQL Server but ask any ten CIOs what they think?   That's just 
marketing.

-- 
-Josh BerkusAglio Database SolutionsSan Francisco



Re: PITR Dead horse?

From
"Dave Page"
Date:

> -----Original Message-----
> From: Nicolai Tufar [mailto:ntufar@pisem.net]
> Sent: 05 February 2004 00:01
> To: pgsql-hackers@postgresql.org
> Subject: Re: [HACKERS] PITR Dead horse?
>
> Totally agree. Robustness and rock-solidness are the only
> things missing for PostgreSQL to become the killer of certain
> commercial enterprise databases out there.

Well I've only been using PostgreSQL since 1997 and the *only* release I
ever had problems with was 6.3.2. We also use(d) Informix SE, DB2,
Unidata and SQL Server and only Informix and Unidata come close to the
robustness of PostgreSQL - and they're not the ones we need to worry
about.

Now I'm not saying we shouldn't be continually looking to improve
things, but I don't think this is quite the problem you imply.

Regards, Dave.


Re: PITR Dead horse?

From
"Dave Page"
Date:

> -----Original Message-----
> From: Nicolai Tufar [mailto:ntufar@pisem.net]
> Sent: 05 February 2004 08:15
> To: Dave Page
> Subject: RE: [HACKERS] PITR Dead horse?
>
> > -----Original Message-----
> > From: Dave Page [mailto:dpage@vale-housing.co.uk] Well I've
> only been
> > using PostgreSQL since 1997 and the *only* release
> I
> > ever had problems with was 6.3.2. We also use(d) Informix SE, DB2,
> > Unidata and SQL Server and only Informix and Unidata come
> close to the
> > robustness of PostgreSQL - and they're not the ones we need
> to worry
> > about.
>
> Don't know. But apparently different users will have
> different demands From a database.

Of course, but I would argue that my claim that PostgreSQL is reliable
is backed up by the lack of people posting messages like 'we had a
powercut and now my DB is hosed'.

> > Now I'm not saying we shouldn't be continually looking to improve
> > things, but I don't think this is quite the problem you imply.
>
> For the customers I am dealing with it is quite a problem, believe me.

Do they have specific problems with the reliability of PostgreSQL then?
Perhaps you could post details of how things have gone wrong for them
(assuming you haven't already - I don't recall anything on -hackers
recently).

Regards, Dave


Re: PITR Dead horse?

From
Bruce Momjian
Date:
Dave Page wrote:
>  
> 
> > -----Original Message-----
> > From: Nicolai Tufar [mailto:ntufar@pisem.net] 
> > Sent: 05 February 2004 00:01
> > To: pgsql-hackers@postgresql.org
> > Subject: Re: [HACKERS] PITR Dead horse? 
> > 
> > Totally agree. Robustness and rock-solidness are the only 
> > things missing for PostgreSQL to become the killer of certain 
> > commercial enterprise databases out there.
> 
> Well I've only been using PostgreSQL since 1997 and the *only* release I
> ever had problems with was 6.3.2. We also use(d) Informix SE, DB2,
> Unidata and SQL Server and only Informix and Unidata come close to the
> robustness of PostgreSQL - and they're not the ones we need to worry
> about.
> 
> Now I'm not saying we shouldn't be continually looking to improve
> things, but I don't think this is quite the problem you imply.

I assume he was talking about the lack of data recovery in cases of hard
drive failure --- we now require you restore from backup or use a
replicated machine/drive setup.

--  Bruce Momjian                        |  http://candle.pha.pa.us pgman@candle.pha.pa.us               |  (610)
359-1001+  If your life is a hard drive,     |  13 Roberts Road +  Christ can be your backup.        |  Newtown Square,
Pennsylvania19073
 


Re: PITR Dead horse?

From
Bruce Momjian
Date:
Tatsuo Ishii wrote:
> > Has this been beaten to death now? Just curious if PITR was in Dev tree
> > yet. Been out of the loop. TIA.
> 
> I and my co workers are very interested in implementing PITR. We will
> tackle this for 7.5 if no one objects.

I have put up a PITR project page:
http://momjian.postgresql.org/main/writings/pgsql/project

--  Bruce Momjian                        |  http://candle.pha.pa.us pgman@candle.pha.pa.us               |  (610)
359-1001+  If your life is a hard drive,     |  13 Roberts Road +  Christ can be your backup.        |  Newtown Square,
Pennsylvania19073
 


Re: PITR Dead horse?

From
Bruce Momjian
Date:
Koichi Suzuki wrote:
> Hi, This is Suzuki from NTT DATA Intellilink.
> 
> I told Bruce Momjan that I and my colleagues are interested in 
> implementing PITR in BOF in NY LW2004.  NTT's laboratory is very 
> interested in this issue and I'm planning to work with them.  I hope we 
> could cooperate.

Yes, I am going to focus on this next week when I return.  With Win32
moving along, PITR is my next big target.  I want to get things moving.
The first step is for Tom to get the PITR WAL patches in.  Then we need
to discuss what else we need and get those on the PITR project page:
http://momjian.postgresql.org/main/writings/pgsql/project

--  Bruce Momjian                        |  http://candle.pha.pa.us pgman@candle.pha.pa.us               |  (610)
359-1001+  If your life is a hard drive,     |  13 Roberts Road +  Christ can be your backup.        |  Newtown Square,
Pennsylvania19073
 


Re: PITR Dead horse?

From
Bruce Momjian
Date:
Nicolai Tufar wrote:
> I would like to join this effort too.
> I was afraid that people at RedHat are already
> halfway though and were to release their work 
> shortly. But it does not seem to be the case.

We are a long way away from completion:
http://momjian.postgresql.org/main/writings/pgsql/project


--  Bruce Momjian                        |  http://candle.pha.pa.us pgman@candle.pha.pa.us               |  (610)
359-1001+  If your life is a hard drive,     |  13 Roberts Road +  Christ can be your backup.        |  Newtown Square,
Pennsylvania19073
 


Re: PITR Dead horse?

From
Bruce Momjian
Date:
Tom Lane wrote:
> "Marc G. Fournier" <scrappy@postgresql.org> writes:
> > Is this something large enough, like the win32 stuff, that having a side
> > list for discussions is worth setting up?
> 
> In terms of the amount of code to be written, I expect it's larger than
> the win32 porting effort.  And it should be mostly pretty separate from
> hacking the core backend, since most of what remains to do is writing
> external management utilities (I think).
> 
> I've been dissatisfied with having the separate pgsql-hackers-win32
> list; I feel it just fragments the discussion, and people tend to end up
> crossposting to -hackers anyway.  But a separate list for PITR work
> might be a good idea despite that experience, since it seems like it'd
> be a more separable project.

I think the win32 email list has worked well.  What is has allowed is
people who want to track only win32 to get only those emails.  It
doesn't help people already on hackers because hacker input is needed.

There are currently 102 Win32 subscribers, and most are not on the
hackers list.

--  Bruce Momjian                        |  http://candle.pha.pa.us pgman@candle.pha.pa.us               |  (610)
359-1001+  If your life is a hard drive,     |  13 Roberts Road +  Christ can be your backup.        |  Newtown Square,
Pennsylvania19073
 


Re: PITR Dead horse?

From
Bruce Momjian
Date:
Marc G. Fournier wrote:
> On Wed, 4 Feb 2004, Tatsuo Ishii wrote:
> 
> > > I and some other developers are also interested in.
> > > Do you think we can work together?
> >
> > Sure. Why not. I think it would be practical to decide who is the
> > leader of this project, though.
> 
> Is this something large enough, like the win32 stuff, that having a side
> list for discussions is worth setting up?

Yes, I would like to have such a list, and will advertize it on the PITR
project page:
http://momjian.postgresql.org/main/writings/pgsql/project

--  Bruce Momjian                        |  http://candle.pha.pa.us pgman@candle.pha.pa.us               |  (610)
359-1001+  If your life is a hard drive,     |  13 Roberts Road +  Christ can be your backup.        |  Newtown Square,
Pennsylvania19073
 


Re: PITR Dead horse?

From
Bruce Momjian
Date:
Bruce Momjian wrote:
> Dave Page wrote:
> >  
> > 
> > > -----Original Message-----
> > > From: Nicolai Tufar [mailto:ntufar@pisem.net] 
> > > Sent: 05 February 2004 00:01
> > > To: pgsql-hackers@postgresql.org
> > > Subject: Re: [HACKERS] PITR Dead horse? 
> > > 
> > > Totally agree. Robustness and rock-solidness are the only 
> > > things missing for PostgreSQL to become the killer of certain 
> > > commercial enterprise databases out there.
> > 
> > Well I've only been using PostgreSQL since 1997 and the *only* release I
> > ever had problems with was 6.3.2. We also use(d) Informix SE, DB2,
> > Unidata and SQL Server and only Informix and Unidata come close to the
> > robustness of PostgreSQL - and they're not the ones we need to worry
> > about.
> > 
> > Now I'm not saying we shouldn't be continually looking to improve
> > things, but I don't think this is quite the problem you imply.
> 
> I assume he was talking about the lack of data recovery in cases of hard
> drive failure --- we now require you restore from backup or use a
> replicated machine/drive setup.

I retract this email.  He clearly was talking about PostgreSQL
reliability, and Dave is right, it is pretty much a non-issue, though
maybe mindshare needs some help.

--  Bruce Momjian                        |  http://candle.pha.pa.us pgman@candle.pha.pa.us               |  (610)
359-1001+  If your life is a hard drive,     |  13 Roberts Road +  Christ can be your backup.        |  Newtown Square,
Pennsylvania19073
 


Re: PITR Dead horse?

From
"Marc G. Fournier"
Date:
pgsql-hackers-pitr@postgresql.org

I set myself as owner, since I didn't figure it was something you really
needed added to your plate? :)  Just means you don't have to go through
and do the Approvals for postings when they need it, I'll just do it as my
normal stuff ...

On Thu, 5 Feb 2004, Bruce Momjian wrote:

> Marc G. Fournier wrote:
> > On Wed, 4 Feb 2004, Tatsuo Ishii wrote:
> >
> > > > I and some other developers are also interested in.
> > > > Do you think we can work together?
> > >
> > > Sure. Why not. I think it would be practical to decide who is the
> > > leader of this project, though.
> >
> > Is this something large enough, like the win32 stuff, that having a side
> > list for discussions is worth setting up?
>
> Yes, I would like to have such a list, and will advertize it on the PITR
> project page:
>
>     http://momjian.postgresql.org/main/writings/pgsql/project
>
>
> --
>   Bruce Momjian                        |  http://candle.pha.pa.us
>   pgman@candle.pha.pa.us               |  (610) 359-1001
>   +  If your life is a hard drive,     |  13 Roberts Road
>   +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073
>

----
Marc G. Fournier           Hub.Org Networking Services (http://www.hub.org)
Email: scrappy@hub.org           Yahoo!: yscrappy              ICQ: 7615664


Re: PITR Dead horse?

From
Austin Gonyou
Date:
Wow. What a wonderful response. Thanks all!


On Thu, 2004-02-05 at 08:57, Bruce Momjian wrote:
> Tatsuo Ishii wrote:
> > > Has this been beaten to death now? Just curious if PITR was in Dev tree
> > > yet. Been out of the loop. TIA.
> > 
> > I and my co workers are very interested in implementing PITR. We will
> > tackle this for 7.5 if no one objects.
> 
> I have put up a PITR project page:
> 
>     http://momjian.postgresql.org/main/writings/pgsql/project
-- 
Austin Gonyou <austin@coremetrics.com>
Coremetrics, Inc.


Re: PITR Dead horse?

From
"Nicolai Tufar"
Date:
> -----Original Message-----
> From: Dave Page [mailto:dpage@vale-housing.co.uk]
> Sent: Thursday, February 05, 2004 11:02 AM
> To: ntufar@pisem.net; pgsql-hackers@postgresql.org
> Subject: RE: [HACKERS] PITR Dead horse?
> Of course, but I would argue that my claim that PostgreSQL is reliable
> is backed up by the lack of people posting messages like 'we had a
> powercut and now my DB is hosed'.

It's not like that. It's more like 'what will happen if we had a
powercut/
disk failure/cpu failure/memory failure, etc, etc.' and that answer I
have
to give is 'why, there is PITR of course!'. No other answer will pass in
enterprise world. Those people are not open-minded, they'd rather be
safe
than sorry.

> 
> Do they have specific problems with the reliability of PostgreSQL
then?
> Perhaps you could post details of how things have gone wrong for them
> (assuming you haven't already - I don't recall anything on -hackers
> recently).

Nothing remarkable. PostgreSQL just works. Bu as I said before,
In enterprise world, good sleep at night is treasured above all.

> Regards, Dave



Re: PITR Dead horse?

From
"Dave Page"
Date:

> -----Original Message-----
> From: Nicolai Tufar [mailto:ntufar@pisem.net]
> Sent: 05 February 2004 17:35
> To: Dave Page; pgsql-hackers@postgresql.org
> Subject: RE: [HACKERS] PITR Dead horse?
>
> > -----Original Message-----
> > From: Dave Page [mailto:dpage@vale-housing.co.uk]
> > Sent: Thursday, February 05, 2004 11:02 AM
> > To: ntufar@pisem.net; pgsql-hackers@postgresql.org
> > Subject: RE: [HACKERS] PITR Dead horse?
> > Of course, but I would argue that my claim that PostgreSQL
> is reliable
> > is backed up by the lack of people posting messages like 'we had a
> > powercut and now my DB is hosed'.
>
> It's not like that. It's more like 'what will happen if we
> had a powercut/ disk failure/cpu failure/memory failure, etc,
> etc.' and that answer I have to give is 'why, there is PITR
> of course!'. No other answer will pass in enterprise world.
> Those people are not open-minded, they'd rather be safe than sorry.

Ahh, that's not quite what I thought you meant. It sounded like you were
questioning the reliability of PostgreSQL, not it's ability to be
recovered to point of failure.

> > Do they have specific problems with the reliability of PostgreSQL
> then?
> > Perhaps you could post details of how things have gone
> wrong for them
> > (assuming you haven't already - I don't recall anything on -hackers
> > recently).
>
> Nothing remarkable. PostgreSQL just works. Bu as I said
> before, In enterprise world, good sleep at night is treasured
> above all.

My SQL2K servers give me far more sleepless nights than PostgreSQL ever
did!

Regards, Dave.


Re: PITR Dead horse?

From
Tom Lane
Date:
"Dave Page" <dpage@vale-housing.co.uk> writes:
> Ahh, that's not quite what I thought you meant. It sounded like you were
> questioning the reliability of PostgreSQL, not it's ability to be
> recovered to point of failure.

I think the waters got muddied a bit by the suggestion elsewhere in the
thread (not from Nicolai, IIRC) that we needed a mailing list to talk
about reliability issues in general.  We know we need PITR to help us
become a more credible enterprise-grade database; so that discussion is
short and sweet.  What people were confused about was whether there was
enough other issues to need ongoing discussion.
        regards, tom lane


Re: PITR Dead horse?

From
"Nicolai Tufar"
Date:
> -----Original Message-----
> From: Dave Page [mailto:dpage@vale-housing.co.uk]
> My SQL2K servers give me far more sleepless nights than PostgreSQL
ever
> did!

You bet! I totally agree with you.
Technicians like you, me and most people on this list
Already know that PostgreSQL is stable and reliable.
It is management that needs to be convinced, and for this
we need to have PITR in feature list.

> Regards, Dave.

Regards,
Nicolai



Re: PITR Dead horse?

From
Austin Gonyou
Date:
On Thu, 2004-02-05 at 14:00, Nicolai Tufar wrote:
> > -----Original Message-----
> > From: Dave Page [mailto:dpage@vale-housing.co.uk]
> > My SQL2K servers give me far more sleepless nights than PostgreSQL
> ever
> > did!
> 
> You bet! I totally agree with you.
> Technicians like you, me and most people on this list
> Already know that PostgreSQL is stable and reliable.
> It is management that needs to be convinced, and for this
> we need to have PITR in feature list.
> 
> > Regards, Dave.


As previously stated by Bruce I believe, the mindshare department needs
some work. For this, the PITR is a necessity, but also when comparing
features with other DBs that people and businesses are currently
familiar with.

-- 
Austin Gonyou <austin@coremetrics.com>
Coremetrics, Inc.


Re: PITR Dead horse?

From
Chester Kustarz
Date:
I do not see the win32 list on http://www.postgresql.org/lists.html
How would I find out about it and join? I probably did not subscribe
to hackers when it started.

On Thu, 5 Feb 2004, Bruce Momjian wrote:
> I think the win32 email list has worked well.  What is has allowed is
> people who want to track only win32 to get only those emails.  It
> doesn't help people already on hackers because hacker input is needed.



Re: PITR Dead horse?

From
Satoshi Nagayasu
Date:
I and some other developers are also interested in.
Do you think we can work together?

Tatsuo Ishii <t-ishii@sra.co.jp> wrote:
> > Has this been beaten to death now? Just curious if PITR was in Dev tree
> > yet. Been out of the loop. TIA.
> 
> I and my co workers are very interested in implementing PITR. We will
> tackle this for 7.5 if no one objects.
> --
> Tatsuo Ishii
> 
> ---------------------------(end of broadcast)---------------------------
> TIP 2: you can get off all lists at once with the unregister command
>     (send "unregister YourEmailAddressHere" to majordomo@postgresql.org)
> 


-- 
NAGAYASU Satoshi <snaga@snaga.org>


Re: PITR Dead horse?

From
Rod Taylor
Date:
> > Don't know. But apparently different users will have
> > different demands From a database.
>
> Of course, but I would argue that my claim that PostgreSQL is reliable
> is backed up by the lack of people posting messages like 'we had a
> powercut and now my DB is hosed'.

One thing we could use (and I have no idea how to do it) is a "This
hardware is not appropriate for a database" test kit.

Something to detect lying disks, battery backed write cache that isn't
so battery backed, etc.

--
Rod Taylor <rbt [at] rbt [dot] ca>

Build A Brighter Lamp :: Linux Apache {middleware} PostgreSQL
PGP Key: http://www.rbt.ca/rbtpub.asc

Re: PITR Dead horse?

From
Bruce Momjian
Date:
Austin Gonyou wrote:
> On Thu, 2004-02-05 at 14:00, Nicolai Tufar wrote:
> > > -----Original Message-----
> > > From: Dave Page [mailto:dpage@vale-housing.co.uk]
> > > My SQL2K servers give me far more sleepless nights than PostgreSQL
> > ever
> > > did!
> > 
> > You bet! I totally agree with you.
> > Technicians like you, me and most people on this list
> > Already know that PostgreSQL is stable and reliable.
> > It is management that needs to be convinced, and for this
> > we need to have PITR in feature list.
> > 
> > > Regards, Dave.
> 
> 
> As previously stated by Bruce I believe, the mindshare department needs
> some work. For this, the PITR is a necessity, but also when comparing
> features with other DBs that people and businesses are currently
> familiar with.

PITR is required to recover all data after total hardware failure.  It
isn't just a mindshare issue.

--  Bruce Momjian                        |  http://candle.pha.pa.us pgman@candle.pha.pa.us               |  (610)
359-1001+  If your life is a hard drive,     |  13 Roberts Road +  Christ can be your backup.        |  Newtown Square,
Pennsylvania19073
 


Re: PITR Dead horse?

From
Bruce Momjian
Date:
Marc G. Fournier wrote:
> 
> pgsql-hackers-pitr@postgresql.org
> 
> I set myself as owner, since I didn't figure it was something you really
> needed added to your plate? :)  Just means you don't have to go through
> and do the Approvals for postings when they need it, I'll just do it as my
> normal stuff ...

OK, I have added the mailing list to the web page:
http://momjian.postgresql.org/main/writings/pgsql/project

and have subscribed myself.

--  Bruce Momjian                        |  http://candle.pha.pa.us pgman@candle.pha.pa.us               |  (610)
359-1001+  If your life is a hard drive,     |  13 Roberts Road +  Christ can be your backup.        |  Newtown Square,
Pennsylvania19073
 


Re: PITR Dead horse?

From
Christopher Browne
Date:
A long time ago, in a galaxy far, far away, pgman@candle.pha.pa.us (Bruce Momjian) wrote:
> Austin Gonyou wrote:
>> As previously stated by Bruce I believe, the mindshare department needs
>> some work. For this, the PITR is a necessity, but also when comparing
>> features with other DBs that people and businesses are currently
>> familiar with.
>
> PITR is required to recover all data after total hardware failure.  It
> isn't just a mindshare issue.

One of the valuable "use cases" of PITR is in replication, and
correspondingly, one of the valuable "use cases" of replication is in
doing major version upgrades.

As a result, a _really valuable thing_ would be for the "PITR reader"
process to be able to read data from "more elderly" versions of
PostgreSQL.  

That may not prove practical, but the more flexible it is, the more
useful it certainly is...
-- 
"cbbrowne","@","cbbrowne.com"
http://www.ntlug.org/~cbbrowne/wp.html
Space Corps Directive #997: Work done  by an officer's doppleganger in
a parallel universe cannot be claimed as overtime.  -- Red Dwarf


Re: PITR Dead horse?

From
"scott.marlowe"
Date:
On Thu, 5 Feb 2004, Rod Taylor wrote:

> > > Don't know. But apparently different users will have 
> > > different demands From a database.
> > 
> > Of course, but I would argue that my claim that PostgreSQL is reliable
> > is backed up by the lack of people posting messages like 'we had a
> > powercut and now my DB is hosed'.
> 
> One thing we could use (and I have no idea how to do it) is a "This
> hardware is not appropriate for a database" test kit.
> 
> Something to detect lying disks, battery backed write cache that isn't
> so battery backed, etc.

but I'm not sure you can test that without power off tests...  so, it 
would have to be a test that kinda started up then told you to pull the 
plug on the box.  Even a kernel panic wouldn't detect it because the drive 
would still be powered up.

Or, you could have a test that checked what kind of drive it was (IDE 
versus SCSI) and maybe had a table of drives that are known to lie, 
possibly even by version, should drives of the same model stop lying half 
way through production due to fixes in their firmware.

I'd guess it the table would still have to be built the old fashioned way, 
by doing power off tests.



Re: PITR Dead horse?

From
Alvaro Herrera
Date:
On Mon, Feb 09, 2004 at 10:04:56AM -0700, scott.marlowe wrote:
> On Thu, 5 Feb 2004, Rod Taylor wrote:

> > One thing we could use (and I have no idea how to do it) is a "This
> > hardware is not appropriate for a database" test kit.
> > 
> > Something to detect lying disks, battery backed write cache that isn't
> > so battery backed, etc.
> 
> but I'm not sure you can test that without power off tests...  so, it 
> would have to be a test that kinda started up then told you to pull the 
> plug on the box.  Even a kernel panic wouldn't detect it because the drive 
> would still be powered up.

Try UMLSIM, umlsim.sourceforge.net

-- 
Alvaro Herrera (<alvherre[a]dcc.uchile.cl>)
"El destino baraja y nosotros jugamos" (A. Schopenhauer)


Re: PITR Dead horse?

From
Bruce Momjian
Date:
Koichi Suzuki wrote:
> Hi, This is Suzuki from NTT DATA Intellilink.
> 
> I told Bruce Momjan that I and my colleagues are interested in 
> implementing PITR in BOF in NY LW2004.  NTT's laboratory is very 
> interested in this issue and I'm planning to work with them.  I hope we 
> could cooperate.

I assume everyone is on the PITR mailing list so you can get involved in
discussions when they start:
http://momjian.postgresql.org/main/writings/pgsql/project

--  Bruce Momjian                        |  http://candle.pha.pa.us pgman@candle.pha.pa.us               |  (610)
359-1001+  If your life is a hard drive,     |  13 Roberts Road +  Christ can be your backup.        |  Newtown Square,
Pennsylvania19073
 


Re: PITR Dead horse?

From
Greg Stark
Date:
"scott.marlowe" <scott.marlowe@ihs.com> writes:

> but I'm not sure you can test that without power off tests...  

Well the approach that's been taken manually on the list is to look at the
timing results and conclude they're just physically impossible.

Doing this automatically could be interesting. If the tool were given a
partition to act on directly it would be able to intentionally write to blocks
in reverse order doing an fsync between each block and testing whether the
bandwidth is low enough to conclude a full rotation between each write had
been completed.

Doing the same on the filesystem would be less reliable but might also be an
interesting test since the OS might make fsync lie directly, or might have
some additional intelligence in the filesystem that forces the drive to sync
to the platters before fsync returns.

-- 
greg