Thread: disk format changes

disk format changes

From
"Craig Isdahl"
Date:
Hello to Everyone,

First, I'm new to postgres and fairly new to Linux (redhat).  Really
love what I've seen of postgres, and this list so far.  It is
obvious there is a lot of experience on this list.

In the weeks I've been following this list I've only seen one
reference to my question, and I wasn't able find anything in the
docs.

Why does the on disk format of postgres change between major (not
sure if that's the right terminology) releases?  Benefits?
Drawbacks?

As someone new to the "scene" the thought of "dumping" databases,
running initdb, then restoring the databases is... umm... (looking
for the right word)... daunting/scary/silly.  Especially for
production dbs.

However, I've also noticed that many on this list are running some
"serious" postgres installations and this apparently isn't a
concern, so I thought I'd ask.

Thanks!
Craig
craig@dashsystems.com


Re: disk format changes

From
Martijn van Oosterhout
Date:
On Thu, Apr 25, 2002 at 02:27:42AM -0400, Craig Isdahl wrote:
> Why does the on disk format of postgres change between major (not
> sure if that's the right terminology) releases?  Benefits?
> Drawbacks?

Actually, it's not the on-disk format that keeps changing. In fact, the same
basic format was used for at least 6.5-7.1. What changed is the system
schema. Since the structure of the schema on disk is reflected in the
binary, these are bound (you could argue this is an optimisation).

For example, 7.0 added support for foreign keys. This required some new
tables and a few columns in pg_class. 7.1 added TOAST support, which also
required some new fields. 7.2 changed the names of the files on disk so that
DROP TABLE could be safely rolled back.

They're not changing the format on a whim, but they are required to support
new feature. Implementing backward compatability here would be quite
difficult. Note also that they are very careful about preserving structure
within minor releases.

> As someone new to the "scene" the thought of "dumping" databases,
> running initdb, then restoring the databases is... umm... (looking
> for the right word)... daunting/scary/silly.  Especially for
> production dbs.

Which is why production databases sometimes lag a version behind. _Working_
is more important than _latest_. Not to mention that you can do it with no
risk to data. And everyone has backups, right?

> However, I've also noticed that many on this list are running some
> "serious" postgres installations and this apparently isn't a
> concern, so I thought I'd ask.

You are free to implement a backward compatability layer if you wish. Though
I think you underestimate the amount of work involved.

HTH,
--
Martijn van Oosterhout   <kleptog@svana.org>   http://svana.org/kleptog/
> Canada, Mexico, and Australia form the Axis of Nations That
> Are Actually Quite Nice But Secretly Have Nasty Thoughts About America

Re: disk format changes

From
"Nigel J. Andrews"
Date:
On Thu, 25 Apr 2002, Craig Isdahl wrote:

> Hello to Everyone,

What ho

> Why does the on disk format of postgres change between major (not
> sure if that's the right terminology) releases?  Benefits?
> Drawbacks?

To implement new features and make improvements to existing ones I would have
thought, although I've only just started looking at the development side.

> As someone new to the "scene" the thought of "dumping" databases,
> running initdb, then restoring the databases is... umm... (looking
> for the right word)... daunting/scary/silly.  Especially for
> production dbs.

Some people have had problems but on the whole I wouldn't be afraid. It might
be daunting; just like me reloading a large table with indexes numerous times
recently, daunting but not scary. Plus you can always run up a second DB in
parallel with your older production one for the loading until switch over time,
although to be honest here I haven't worked out why my development snapshot
tries to install the perl modules into the standard place and aborts the
install at that point (pleased I wasn't running with root priviledges at the
time). This also calls into question how one does install a parallel system
when what may be essential items for production and development testing need to
be in the same place.

However, if it's at all worrying and it's not broken I often subscribe to the
don't fix it point of view. I have a FreeBSD box running a way out of date
system but it does it's job. I also have a 486 box with a linux 1.0.26 kernel
but I don't fire that up so much these days. :)

Just my 2 cents.


--
Nigel J. Andrews
Director

---
Logictree Systems Limited
Computer Consultants


Re: disk format changes

From
Bruce Momjian
Date:
Craig Isdahl wrote:
> Hello to Everyone,
>
> First, I'm new to postgres and fairly new to Linux (redhat).  Really
> love what I've seen of postgres, and this list so far.  It is
> obvious there is a lot of experience on this list.
>
> In the weeks I've been following this list I've only seen one
> reference to my question, and I wasn't able find anything in the
> docs.
>
> Why does the on disk format of postgres change between major (not
> sure if that's the right terminology) releases?  Benefits?
> Drawbacks?
>
> As someone new to the "scene" the thought of "dumping" databases,
> running initdb, then restoring the databases is... umm... (looking
> for the right word)... daunting/scary/silly.  Especially for
> production dbs.
>
> However, I've also noticed that many on this list are running some
> "serious" postgres installations and this apparently isn't a
> concern, so I thought I'd ask.

I did write a pg_upgrade utility for 7.2 in /contrib that allows
upgrades without dump/reload.  Unfortunately, the changes for the
compressed transaction log cause it to fail, and I didn't have enough
time to get that part working.  Hopefully it will work for 7.2->7.3
changes if I can make sure it transfers over all the data correctly.

So, we do know this type of utility is needed.  I just hope we can pull
it off, because each release has different changes.

--
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026