Proposal: In-Place upgrade concept - Mailing list pgsql-hackers

I attach In-Place upgrade project concept. Any technical details about
implementation of each part will be sent later (after concept acceptance).

    Please, let me know your comments.

        thanks Zdenek
In-place Upgrade project
---------------

Overview
--------
PostgreSQL community releases new version of PostgreSQL at least once a year but the released version is not able to
runwith old data files. For a user who wants to upgrade his database server is a nightmare to upgrade bigger database
andit is impossible to upgrade a very large database. This project will try to solve this issue and provide a
functionalityfor PostgreSQL which will cover all user's requirements. 


Current status
--------------
1) Data upgrade is supported only by exporting and importing data using pg_dump
   - slow and causes a long downtime
   - extra disk space needed
   - require both version (old&new)
2) pg_migrator
   - faster then 1) but it still has a long downtime when on disk structure has changed
   - previous version of PostgreSQL is needed.
   - no downgrade
   - require both version (old&new)


Project Goals
-----
1) Minimize the database downtime.
2) Do not require extra disk space.
3) Do not require an old version of PostgreSQL to be installed.
4) Allow a fallback to original PostgreSQL version if something would go wrong.
5) Support all data types, including user defined types
6) Should be easy to use.


Functional specification
----------------
There might be more different approaches on how to perform upgrade. We assume that we need to convert a very big
database.If we will upgrade the data files offline the database downtime might be critical. We will better  try to
convertdata online. This will bring a requirement that new major version of PostgreSQL must understand the old data
structuresand it must be able to work with them. 

The next important requirement is an ability to fallback (downgrade) to the previous PostgreSQL version. When new major
versiondoes not work well for the user, he needs to be able to go back to the previous version with minimal downtime.
Thiswill bring a requirement that PostgreSQL should also be able to write data in the old format. However, we might
expectsome performance penalty. 

The above requirements lead to PostgreSQL running in three modes:

compatibility mode (compat) - a new version is running on the old data files. It is possible to read and write data as
ina previous version. Features of a new version might not be available. Fallback to the previous version is possible. 

upgrade mode - a conversion of data files is running in the background. PostgreSQL have to be able to work with both
thenew data file format and the old data file format. Features of a new version might not be available. PostgreSQL have
tobe able to recover and continue in the upgrade mode in case of a power/system failure. Fallback to the previous
versionis not possible. 

normal mode - all required data files were upgraded to the new format. Features of a new version are available.
Fallbackto the previous version is not possible. 

Project Roadmap
-------------
1) Create a detailed design for each affected part
2) Design a testing environment/process
3) Implementation
4) Change development/release processes - patch author have to be responsible for providing the required upgrade
operations/functions.Some more development related to upgrade might be done also after feature freeze. In-place upgrade
shouldbe available since x.y.1 version. 


List of affected parts
----------------------

1) Data Page Structure
1.1) Page Layout
1.2) Tuple header and structure
1.3) Data encoding - data type changes
2) Control File
3) WAL
4) Catalog
5) Configuration files
7) Others (Procedure language, Communication protocol ...)