Thread: Development Plans

Development Plans

From
Simon Riggs
Date:
I'm giving a talk next week on PostgreSQL 8, so I would like some input
from the community on a few issues, so that my answers are as close to
majority opinion as possible.

One of the most frequent set of questions I get asked is around the
development vision and release strategy of PostgreSQL.

- When is the next release due?
- What will be in release 8.1?
- What are you working towards? Performance? Stability? X?

These are all good questions, you'll note. They mostly indicate that the
person asking the question has already got the more basic messages, and
are preparing themselves to fully accept PostgreSQL as the way forward
*for them*.

I think I've come to understand the answers to many of these questions,
but these answers are not written down. When I do answer them, I try to
make it clear that I present a personal opinion only - but that always
gets strange looks. People really do not understand why there is no
official answer, and take that as a black mark.
Other projects such as Ubuntu, Fedora and OpenOffice have much of this
type of information easily available - certainly commercial software
vendors spend a good deal of time on providing this information.
Could we find a way of expressing the project philosophy in writing, so
I can convey that message out to the world, exactly as intended, without
any Riggs filtering?

My own understandings would be...

- When is the next release due?
I'm happy with the Zen approach of "there is no answer, the code comes
when it is time" and "HACKERS list IS the process".
Many people take the lack of a planned release date as clear indication
that there is no strictly controlled release process, however-much I
state that there really is one. In the absence of release dates, could
we write down some indication of what the release process is, so
everybody understands there is one.
My understanding is:
- new release forked in code repository
- feature freeze, beta phase starts
- string freeze, to allow translations
- release candidate process
- release
Right now, I have zero idea which quarter, let alone which month feature
freeze for 8.1 is in. I think it will be in 2005, but I'm not sure.
[That makes it fairly difficult to get sponsorship for release of new
features, since I cannot guarantee which year they'll be in.]

- what will be in release 8.1?
The TODO list contains a partial mechanism for recording what is being
worked upon by various people. Could that process by beefed up somewhat,
so there is a clear list of Features in Next Release, as part of the
TODO list on the main web site? A caveat could easily warn that this is
a provisional list only and offers no guarantees of inclusion.
I'm happy to make certain commitments to particular features already on
the TODO list. I'm sure others are too.

- What are you working towards? Performance? Stability? X?
When I explain that the pace of development has increased, people
immediately ask which direction things are going in.
In the run-up to r8.0, PITR was listed as an URGENT feature. This gave
some indication of the direction that Core wished the code base to
travel in and gave many a strong indication that they understood the
momentum and trajectory of development.
That section has been removed now.
This is a more difficult one to answer, especially since it really
covers what the major sponsors want.

I'm fairly clear about my own directions: Enterprise features to enhance
robustness, performance and scalability, plus Data Warehousing features
- nearly all of which come from clients or interested parties.
Does anybody else wish to share theirs?

Just so it is clear, there is not a single criticism on this page from
me; I am merely passing on a pattern that I think I see emerging from a
variety of conversations with clients, course attendees and
press/exhibition contacts.

Many thanks,

Best Regards, Simon Riggs



Re: Development Plans

From
"Marc G. Fournier"
Date:
On Fri, 25 Feb 2005, Simon Riggs wrote:

> - When is the next release due?

Based on past discussions on a 12 to 18 month dev cycle for 8.1, and based
on past track record, I'd say closer to the 18 month, so figure on June
'06, with freeze in January of '06 (12 month dev, 6 month beta) ...
subject to change, but I wouldn't expect a freeze until Jan '06, beta
period might be shorter though ...


----
Marc G. Fournier           Hub.Org Networking Services (http://www.hub.org)
Email: scrappy@hub.org           Yahoo!: yscrappy              ICQ: 7615664

Re: Development Plans

From
Bruce Momjian
Date:
Marc G. Fournier wrote:
> On Fri, 25 Feb 2005, Simon Riggs wrote:
>
> > - When is the next release due?
>
> Based on past discussions on a 12 to 18 month dev cycle for 8.1, and based
> on past track record, I'd say closer to the 18 month, so figure on June
> '06, with freeze in January of '06 (12 month dev, 6 month beta) ...
> subject to change, but I wouldn't expect a freeze until Jan '06, beta
> period might be shorter though ...

Agreed.  That is good target, but might change, as Marc said.

--
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Re: [HACKERS] Development Plans

From
Bruce Momjian
Date:
Simon Riggs wrote:
> I'm giving a talk next week on PostgreSQL 8, so I would like some input
> from the community on a few issues, so that my answers are as close to
> majority opinion as possible.
>
> One of the most frequent set of questions I get asked is around the
> development vision and release strategy of PostgreSQL.
>
> - When is the next release due?
> - What will be in release 8.1?
> - What are you working towards? Performance? Stability? X?
>
> These are all good questions, you'll note. They mostly indicate that the
> person asking the question has already got the more basic messages, and
> are preparing themselves to fully accept PostgreSQL as the way forward
> *for them*.
>
> I think I've come to understand the answers to many of these questions,
> but these answers are not written down. When I do answer them, I try to
> make it clear that I present a personal opinion only - but that always
> gets strange looks. People really do not understand why there is no
> official answer, and take that as a black mark.
> Other projects such as Ubuntu, Fedora and OpenOffice have much of this
> type of information easily available - certainly commercial software
> vendors spend a good deal of time on providing this information.
> Could we find a way of expressing the project philosophy in writing, so
> I can convey that message out to the world, exactly as intended, without
> any Riggs filtering?

I have no idea how to predict what will be in 8.1.  I couldn't predict
what would be in 8.0 until just before feature freeze, so the idea that
we would have any clue about 8.1 is unrealistic.

How do other open source projects predict these things? The most visible
project I know that did that was Mozilla, and it was very unpredictive,
and they had a higher percentage of paid folks than we do.

--
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Re: Development Plans

From
Robert Treat
Date:
On Fri, 2005-02-25 at 05:34, Simon Riggs wrote:
> I'm giving a talk next week on PostgreSQL 8, so I would like some input
> from the community on a few issues, so that my answers are as close to
> majority opinion as possible.
>
> One of the most frequent set of questions I get asked is around the
> development vision and release strategy of PostgreSQL.
>
> - When is the next release due?
> - What will be in release 8.1?
> - What are you working towards? Performance? Stability? X?
>
> These are all good questions, you'll note. They mostly indicate that the
> person asking the question has already got the more basic messages, and
> are preparing themselves to fully accept PostgreSQL as the way forward
> *for them*.
>
> I think I've come to understand the answers to many of these questions,
> but these answers are not written down. When I do answer them, I try to
> make it clear that I present a personal opinion only - but that always
> gets strange looks. People really do not understand why there is no
> official answer, and take that as a black mark.
> Other projects such as Ubuntu, Fedora and OpenOffice have much of this
> type of information easily available - certainly commercial software
> vendors spend a good deal of time on providing this information.
> Could we find a way of expressing the project philosophy in writing, so
> I can convey that message out to the world, exactly as intended, without
> any Riggs filtering?
>
> My own understandings would be...
>
> - When is the next release due?
> I'm happy with the Zen approach of "there is no answer, the code comes
> when it is time" and "HACKERS list IS the process".
> Many people take the lack of a planned release date as clear indication
> that there is no strictly controlled release process, however-much I
> state that there really is one. In the absence of release dates, could
> we write down some indication of what the release process is, so
> everybody understands there is one.
> My understanding is:
> - new release forked in code repository
> - feature freeze, beta phase starts
> - string freeze, to allow translations
> - release candidate process
> - release
> Right now, I have zero idea which quarter, let alone which month feature
> freeze for 8.1 is in. I think it will be in 2005, but I'm not sure.
> [That makes it fairly difficult to get sponsorship for release of new
> features, since I cannot guarantee which year they'll be in.]
>

As Marc stated, core has decided on a 12 month development cycle, so
that might look like January of next year.  If I were speaking to
someone looking to sponsor development, I would tell them they would
need to being work in Q4 of 2005 at the very latest to have a chance of
something going in. As far as when they can expect to see it in a
released branch, Q2 of 2006 would probably be a reasonable estimate.
(I'm hopeful that beta wont take as long this time around since win32
should be more stable and the buildfarm should also be of some help, but
I can't imagine we'd get through it in less than 3 months)

--- update ---

Was just about to hit "send" on this email when I saw Tom's post come up
on -hackers talking about a 6 month cycle... so maybe all of the above
is off...


> - what will be in release 8.1?
> The TODO list contains a partial mechanism for recording what is being
> worked upon by various people. Could that process by beefed up somewhat,
> so there is a clear list of Features in Next Release, as part of the
> TODO list on the main web site? A caveat could easily warn that this is
> a provisional list only and offers no guarantees of inclusion.
> I'm happy to make certain commitments to particular features already on
> the TODO list. I'm sure others are too.
>

Actually the TODO list is really only a good indicator of things already
in the code... to get an idea of items being discussed you really just
have to hang out on the lists to see what people are working on.  A list
of some of what are probably the "big things" being worked on for 8.1
include:

* Improved Resource Managment (this encapsulates the buffer manager
changes and arc replacement and similar work)

* Integrated pg_autovacuum

* Stored procedures (with transaction control and multiple result sets
among other features, not to be confused with function support)

* SQL compatible recursive WITH statements

* 2 Phase Commit

As always none of that stuff is guaranteed but those items have someone
who has stated they want to "make it happen" for 8.1 (and by make it
happen I mean they plan to code it)  There are a couple of other big
things floating out there too (bitmapped indexes are a good example),
but I think those are the most concrete. There are also some things that
are cool but probably not big things (OS/2 support for instance)

> - What are you working towards? Performance? Stability? X?
> When I explain that the pace of development has increased, people
> immediately ask which direction things are going in.
> In the run-up to r8.0, PITR was listed as an URGENT feature. This gave
> some indication of the direction that Core wished the code base to
> travel in and gave many a strong indication that they understood the
> momentum and trajectory of development.
> That section has been removed now.
> This is a more difficult one to answer, especially since it really
> covers what the major sponsors want.
>

I would agree that since we have a number of commercial developers who
do have intentions of working on specific items in 8.1, it would be nice
to list those items somewhere (the urgent section of the TODO seems
fine), but it is up to those developers to speak up.


Robert Treat
--
Build A Brighter Lamp :: Linux Apache {middleware} PostgreSQL


Re: [HACKERS] Development Plans

From
Christopher Kings-Lynne
Date:
> I would agree that since we have a number of commercial developers who
> do have intentions of working on specific items in 8.1, it would be nice
> to list those items somewhere (the urgent section of the TODO seems
> fine), but it is up to those developers to speak up.

I'd like to bundle pg_dump and pg_dumpall into a single binary, allowing
pg_dumpall in custom (binary) format.  Dunno if I'll get the time though :)

I'd also like to backport 8.0's pg_dump to 7.4.x since I think it has
'bug fixes'...

Chris

Re: [HACKERS] Development Plans

From
Tom Lane
Date:
"Matthew T. O'Connor" <matthew@zeut.net> writes:
> Are there any big projects are people
> working on to get into 8.1?

I'm privately hoping to get bitmap index operations into 8.1 (that is,
build a bitmap of tuple locations from an index, possibly AND or OR the
results of multiple indexes, and finally visit the heap in CTID order).
This is not as big as say the Windows port, but it's easily a solid
month or two of effort.

            regards, tom lane

Re: [HACKERS] Development Plans

From
Peter Eisentraut
Date:
Am Freitag, 25. Februar 2005 16:49 schrieb Bruce Momjian:
> I have no idea how to predict what will be in 8.1.  I couldn't predict
> what would be in 8.0 until just before feature freeze, so the idea that
> we would have any clue about 8.1 is unrealistic.

A good guess for the features contained in PostgreSQL version X are those that
were first mentioned as targeted for PostgreSQL version X-2.  Seriously.

--
Peter Eisentraut
http://developer.postgresql.org/~petere/

Re: [HACKERS] Development Plans

From
Tom Lane
Date:
Simon Riggs <simon@2ndquadrant.com> writes:
> I'm happy with the Zen approach of "there is no answer, the code comes
> when it is time" and "HACKERS list IS the process".
> Many people take the lack of a planned release date as clear indication
> that there is no strictly controlled release process, however-much I
> state that there really is one. In the absence of release dates, could
> we write down some indication of what the release process is, so
> everybody understands there is one.

What we do not have is a marketing-driven development process ;-).
We let the decisions be dictated by the maturity of the code, not by
an arbitrary predetermined release date.  We do try to set a feature
freeze date in advance, but this has always been a pretty sloppy
cutoff --- the reason we set it in advance is just so that individual
developers can make their own plans about how much they think they can
get done in time for the next release.  (Awhile back we didn't even do
that, but we found that we were forever slipping a release because first
one feature then another was "almost ready".  We have learned that we
must be willing to say "too late for this release".)  Once we are past
feature freeze, the beta schedule and release schedule are determined
entirely by Core's judgment about the state of the code.

I think most of the developers consider our process a strength, not
a weakness.  It's certainly conditioned by the realities of an open-
source project in which most of the workers are part-time volunteers,
but it works well for us.

> Right now, I have zero idea which quarter, let alone which month feature
> freeze for 8.1 is in. I think it will be in 2005, but I'm not sure.

I'm sorry about the flux around the 8.1 schedule.  It's been caused by
what is hopefully a one-time problem, namely uncertainty about how
we ought to deal with the ARC patent problem.  Normally we would have
set a fairly definite feature freeze date before now.

> - what will be in release 8.1?

Whatever people get done.  In a project where the work is done by
volunteers, it's just about pointless to imagine that we can predict it.
I can say what I'm hoping to work on personally, but how much of it will
get done I don't really know.  Multiply that across a few dozen people
and what you have is pretty squishy.

I wouldn't mind seeing people be a little more vocal on the hackers list
about what they plan to be doing, just so that there's not duplication
of effort.  But trying to assemble that into some sort of published
Master Plan sounds to me like just a recipe for making ourselves look
foolish when the Plan ends up having little relationship to reality.

> - What are you working towards? Performance? Stability? X?

Yes.

Bruce used to try to describe our releases as being oriented towards some
particular goal, but I always thought that these were after-the-fact
descriptions that had nothing to do with the real development process.
The truth is that individual developers work on what they feel like,
or find interesting, or in some cases get paid to do.  But just as
there's not a master plan, there's not really some guiding vision that
in a particular release we are going to focus on X or Y or Z.

Note: the above is just my two cents and shouldn't be read as speaking
for Core.  I think if you asked the other members of core you'd get
roughly similar answers, but each with his own spin.

            regards, tom lane

Re: [HACKERS] Development Plans

From
Christopher Kings-Lynne
Date:
> I wouldn't mind seeing people be a little more vocal on the hackers list
> about what they plan to be doing, just so that there's not duplication
> of effort.  But trying to assemble that into some sort of published
> Master Plan sounds to me like just a recipe for making ourselves look
> foolish when the Plan ends up having little relationship to reality.

The things I would _like_ to do:

* Combine pg_dump and pg_dumpall into a single binary with common
options, allowing a full cluster custom format dump.

* Make psql backwards compatible like pg_dump is

* Work with a few of the IRC guys on a pgsql version of SQL*Loader.
This would be a rad high speed data loader, with overflow files of rows
that couldn't be loaded, etc.

Chris

Re: Development Plans

From
Josh Berkus
Date:
Simon,

Welcome back!   Ready to get to work?   I need to talk to you about some
stuff ....

> - When is the next release due?

Each of our previous 4 releases has taken between 11 and 14 months.   So,
early 2006 would not be unlikely; however, we have set no dates yet.

> - What will be in release 8.1?

Can't answer that.    Let me give you the text I've used with reporters:

"Currently there are developers working on bitmap indexes, database roles,
two-phase commit, faster GiST and R-tree indexes, integrated autovacuum, SQL
standard compliant procedures, and further improvements in memory usage.
However, it is still quite early in the development cycle, and it is very
likely some of these features won't be ready, or won't be good enough, for
8.1, and is equally likely that we will receive and accept four or five other
features that I don't know about yet."

> - What are you working towards? Performance? Stability? X?

X, definitely X.

We're working toward PostgreSQL being indisputably the very best SQL RDBMS in
the world.

> I think I've come to understand the answers to many of these questions,
> but these answers are not written down. When I do answer them, I try to
> make it clear that I present a personal opinion only - but that always
> gets strange looks. People really do not understand why there is no
> official answer, and take that as a black mark.

Well, they're used to dealing with private companies and company-sponsored
projects.   These things have marketing-driven agendas.   We are a
non-commercial, all-volunteer OSS project.    You will need to educate people
on this.

> Other projects such as Ubuntu, Fedora and OpenOffice have much of this
> type of information easily available

OpenOffice.org and Fedora are both single-company-sponsored projects, with
marketing-driven goals.  I don't know about Ubuntu.

> - certainly commercial software
> vendors spend a good deal of time on providing this information.

Yep.  And commercial vendors ship releases whether or not that release is
stable or actually contains the features advertised.

> Could we find a way of expressing the project philosophy in writing, so
> I can convey that message out to the world, exactly as intended, without
> any Riggs filtering?

That's not a small order, if we want to do it right.    Why don't you prepare
a Faq-ish page that covers these issues based on the responses you've
received on this thread?  I can add it to the Press FAQ.

> Right now, I have zero idea which quarter, let alone which month feature
> freeze for 8.1 is in. I think it will be in 2005, but I'm not sure.
> [That makes it fairly difficult to get sponsorship for release of new
> features, since I cannot guarantee which year they'll be in.]

Think early 2006.   Tentatively.  After all, we haven't discussed feature
freeze date yet.

> The TODO list contains a partial mechanism for recording what is being
> worked upon by various people. Could that process by beefed up somewhat,
> so there is a clear list of Features in Next Release, as part of the
> TODO list on the main web site? A caveat could easily warn that this is
> a provisional list only and offers no guarantees of inclusion.
> I'm happy to make certain commitments to particular features already on
> the TODO list. I'm sure others are too.

I've been in favor of converting the TODO list into a pgFoundry-based Task
List, so that TODOs can be claimed by people, for some time.

> I'm fairly clear about my own directions: Enterprise features to enhance
> robustness, performance and scalability, plus Data Warehousing features
> - nearly all of which come from clients or interested parties.
> Does anybody else wish to share theirs?

Well, I'll be working on Data Warehousing too.

--
Josh Berkus
Aglio Database Solutions
San Francisco

Re: [HACKERS] Development Plans

From
Bruce Momjian
Date:
Tom Lane wrote:
> Bruce used to try to describe our releases as being oriented towards some
> particular goal, but I always thought that these were after-the-fact
> descriptions that had nothing to do with the real development process.
> The truth is that individual developers work on what they feel like,
> or find interesting, or in some cases get paid to do.  But just as
> there's not a master plan, there's not really some guiding vision that
> in a particular release we are going to focus on X or Y or Z.

Right, my analysis was always after-the-fact in an attempt to put an
understandable story around the release features, nothing more.

--
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Re: Development Plans

From
Chris Travers
Date:
Hi Simon;

Here are my observations regarding being a user of the software for the
last five years.  Members of the Core should feel free to correct me if
they feel like it.

Simon Riggs wrote:

>
>One of the most frequent set of questions I get asked is around the
>development vision and release strategy of PostgreSQL.
>
>- When is the next release due?
>- What will be in release 8.1?
>- What are you working towards? Performance? Stability? X?
>
>
>
Ok, lets get back to development vision and release strategy.  In this
case, it doesn't really break down into your three questions.

You have already gotten your answers about release schedule and the
strategy of timing.  So lets look at development vision the strategy
regarding what is included in the released version.

The strategy is that it is better to focus on robustness and the correct
way to do things than it is to focus on getting new features in early.
So as Tom (I think) said, this is not a marketing-driven approach.
Instead we have a market-driven approach where features are developed
based on what developers in the market are willing to add either on
their own time or for hire.  For this reason it is very difficult to
predict what will be in the release.  I know a lot of us were surprised
regarding the inclusion of several (great) features in 8.0 which only
happened because of corporate sponsorship.

The market is not a monolithic entity.  It doesn't just focus on one
thing.  So I can say with relative certainty that 8.1 will likely have
better performance, be easier to manage, and scale better than our
current version.  It will probably also conform more closely to the ANSI
SQL 99 standards and have useful other enterprise features.  However
unlike products such as MS SQL Server (or MySQL), our strategy will
likely be guided by what people actually feel the need to fix rather
than what the marketing department things people want to see in the next
version.

I hope that this clarifies the other answers you have received.

Best Wishes
Chris Travers

Attachment

Re: [HACKERS] Development Plans

From
"Jim C. Nasby"
Date:
On Fri, Feb 25, 2005 at 10:49:59AM -0500, Bruce Momjian wrote:
> I have no idea how to predict what will be in 8.1.  I couldn't predict
> what would be in 8.0 until just before feature freeze, so the idea that
> we would have any clue about 8.1 is unrealistic.
>
> How do other open source projects predict these things? The most visible
> project I know that did that was Mozilla, and it was very unpredictive,
> and they had a higher percentage of paid folks than we do.

Not to sound like a broken FreeBSD drum, but they manage to do it, and
afaik a pretty good job of it.
http://www.freebsd.org/releases/5.4R/todo.html is an example.
http://www.freebsd.org/releases/5.4R/schedule.html is also interesting.

I suspect a big part of why/how they can do this is they have a much
larger developer pool than PostgreSQL; I believe there's over 2000
people with commit access, and there are probably 100-200 people who are
actively developing code for FBSD. But it's been some time since I
followed the details of FBSD development, so I could be way off on these
WAGs.
--
Jim C. Nasby, Database Consultant               decibel@decibel.org
Give your computer some brain candy! www.distributed.net Team #1828

Windows: "Where do you want to go today?"
Linux: "Where do you want to go tomorrow?"
FreeBSD: "Are you guys coming, or what?"

Re: [HACKERS] Development Plans

From
"Matthew T. O'Connor"
Date:
Bruce Momjian wrote:

>I have no idea how to predict what will be in 8.1.  I couldn't predict
>what would be in 8.0 until just before feature freeze, so the idea that
>we would have any clue about 8.1 is unrealistic.
>
>

I agree there is no way to accurately predict such things, however there
was a long list of large features that people were trying to get in for
8.0.  So while you can't be sure about what will make it for 8.1, I'm
not even aware of any large projects that anyone is working on.  BTW
when I say large project I'm talking about things like the Windows port,
PITR, nested Xacts etc....   Are there any big projects are people
working on to get into 8.1?


Re: [HACKERS] Development Plans

From
Neil Conway
Date:
Tom Lane wrote:
> I wouldn't mind seeing people be a little more vocal on the hackers list
> about what they plan to be doing, just so that there's not duplication
> of effort.

Stuff I have done in some form that I need to finish up and submit:

- GiST improvements: sane memory management, 10% scan perf. improvement
(not sure if I'll get to WAL and page-level locking for 8.1)

- CREATE TABLE AS overhaul & SQL 2003 compliance

- pl/pgsql dead code checking (only for trivially-dead code)

- default_with_oids=true by default

- use # of CPUs at runtime to adjust spinlock behavior for UP systems

Interested in looking at for 8.1 but no code yet:

- PREPARE planning improvements (at the least, do the planning when we
see the first EXECUTE, as in the fe/be protocol-level prepared statements)

- various planner improvements; haven't really decided what specifically
to do, yet

- logical column ordering, and possibly repacking of physical order of
columns to optimize disk space consumption by reducing alignment/padding
requirements

- O_DIRECT for WAL

- UNIQUE predicate per SQL2003

Investigated, probably not worth pursuing:

- GCC PGO support. At least in GCC 3.4, PGO is sufficiently flaky it
isn't really worth adding support for it. Maybe I'll take another look
when GCC 4.0 is out.

- futexes in PG spinlocks. Didn't solve the CS problem, perf.
improvement possibly (?) not worth the portability headaches.

(Of course, absolutely no guarantees that I actually get around to
implementing any of this stuff, this is just what's on my mind at the
moment...)

-Neil

Re: Development Plans

From
Josh Berkus
Date:
Simon,

> > We're working toward PostgreSQL being indisputably the very best SQL
> > RDBMS in the world.
>
> Hmmm, interestingly, that is not my objective. *Most-used* is both a
> sufficient and eventually realisable goal for me to contribute towards.
> Best seems like a niche, and not always a cost-effective one.

Well, we have different goals then; I personally think that best quality and
most-used may be mutually exclusive.   Examine, for instance, the broad
popularity of Microsoft Access ...

> > Yep.  And commercial vendors ship releases whether or not that release is
> > stable or actually contains the features advertised.
>
> Hmmm. Well, companies differ, lets say that.

Yeah, I should have said *some* companies.

> > That's not a small order, if we want to do it right.    Why don't you
> > prepare a Faq-ish page that covers these issues based on the responses
> > you've received on this thread?  I can add it to the Press FAQ.
>
> OK, well I was hoping that one-of-Core, not necessarily yourself, would
> be the one to put pen to paper on such an issue.
>
> ...but, I guess I'll have a stab at it.

<grin> you're still thinking of Core as "Authority" in the project.   We're
only authoritative when it comes to releases.   Otherwise, you are just as
qualified to write the thing as me ... and more likely to get it done sooner.
Face it, Simon, you are a "member" now.

--
Josh Berkus
Aglio Database Solutions
San Francisco

Re: [HACKERS] Development Plans

From
Simone Brunozzi
Date:
I'd like to add my personal opinion on development plans,
not regarding the pure coding, but the contour instead.
(I'm not a real PG expert, so I'm limited to this).

As a university teacher, I feel the lack of introductory and explanatory
material for postgresql.
I think it would be really good if somebody really expert can prepare
videos and slides and share it on the web site. Possibly in english, but
with some sort of subtitles in the case of videos.

This is not just to cover the normal topics of Postgresql, but instead
also something regarding migration issues, development plannings, etc...
I mean, some sort of documentation that is not just barely how to use
it, but something derived from the direct experience in real world
applications.

I don't know if these considerations fit in the subject of this thread,
but I think that having a development path can be very valuable if that
path is shared and simply explained to "nobrainers" or normal students
and users.

Thanks

Simone

Neil Conway wrote:
> Tom Lane wrote:
>
>> I wouldn't mind seeing people be a little more vocal on the hackers list
>> about what they plan to be doing, just so that there's not duplication
>> of effort.
>
>
> Stuff I have done in some form that I need to finish up and submit:
>
> - GiST improvements: sane memory management, 10% scan perf. improvement
> (not sure if I'll get to WAL and page-level locking for 8.1)
>
> - CREATE TABLE AS overhaul & SQL 2003 compliance
>
> - pl/pgsql dead code checking (only for trivially-dead code)
>
> - default_with_oids=true by default
>
> - use # of CPUs at runtime to adjust spinlock behavior for UP systems
>
> Interested in looking at for 8.1 but no code yet:
>
> - PREPARE planning improvements (at the least, do the planning when we
> see the first EXECUTE, as in the fe/be protocol-level prepared statements)
>
> - various planner improvements; haven't really decided what specifically
> to do, yet
>
> - logical column ordering, and possibly repacking of physical order of
> columns to optimize disk space consumption by reducing alignment/padding
> requirements
>
> - O_DIRECT for WAL
>
> - UNIQUE predicate per SQL2003
>
> Investigated, probably not worth pursuing:
>
> - GCC PGO support. At least in GCC 3.4, PGO is sufficiently flaky it
> isn't really worth adding support for it. Maybe I'll take another look
> when GCC 4.0 is out.
>
> - futexes in PG spinlocks. Didn't solve the CS problem, perf.
> improvement possibly (?) not worth the portability headaches.
>
> (Of course, absolutely no guarantees that I actually get around to
> implementing any of this stuff, this is just what's on my mind at the
> moment...)
>
> -Neil
>
> ---------------------------(end of broadcast)---------------------------
> TIP 5: Have you checked our extensive FAQ?
>
>               http://www.postgresql.org/docs/faq
>
>

--

Ing. iunior Simone Brunozzi

WEDOIT s.a.s. - Soluzioni informatiche

Via protomartiri Francescani, 26
06088 Assisi (PG) - ITALY

www.wedoit.us

Tel. +39 075-8041195
Cell. +39 340-5768488
---------------------------------------

Re: Development Plans

From
Simon Riggs
Date:
Thanks to all replies on this thread over last few days, many good point
and useful contributions, thank you.
[Please excuse many non-replies, since I've been ill.]

On Fri, 2005-02-25 at 09:41 -0800, Josh Berkus wrote:
> > - What are you working towards? Performance? Stability? X?
>
> X, definitely X.

Thats clear then. ;-)

> We're working toward PostgreSQL being indisputably the very best SQL RDBMS in
> the world.

Hmmm, interestingly, that is not my objective. *Most-used* is both a
sufficient and eventually realisable goal for me to contribute towards.
Best seems like a niche, and not always a cost-effective one.

> > I think I've come to understand the answers to many of these questions,
> > but these answers are not written down. When I do answer them, I try to
> > make it clear that I present a personal opinion only - but that always
> > gets strange looks. People really do not understand why there is no
> > official answer, and take that as a black mark.
>
> Well, they're used to dealing with private companies and company-sponsored
> projects.   These things have marketing-driven agendas.   We are a
> non-commercial, all-volunteer OSS project.    You will need to educate people
> on this.

Well....I'm trying, thats why I wanted all of those things written down.

> > Other projects such as Ubuntu, Fedora and OpenOffice have much of this
> > type of information easily available
>
> OpenOffice.org and Fedora are both single-company-sponsored projects, with
> marketing-driven goals.  I don't know about Ubuntu.

OK, thats a good point.

Ubuntu is single company too, and worse, it doesn't work on my laptop.

> > - certainly commercial software
> > vendors spend a good deal of time on providing this information.
>
> Yep.  And commercial vendors ship releases whether or not that release is
> stable or actually contains the features advertised.

Hmmm. Well, companies differ, lets say that.

> > Could we find a way of expressing the project philosophy in writing, so
> > I can convey that message out to the world, exactly as intended, without
> > any Riggs filtering?
>
> That's not a small order, if we want to do it right.    Why don't you prepare
> a Faq-ish page that covers these issues based on the responses you've
> received on this thread?  I can add it to the Press FAQ.

OK, well I was hoping that one-of-Core, not necessarily yourself, would
be the one to put pen to paper on such an issue.

...but, I guess I'll have a stab at it.

Best Regards, Simon Riggs


Re: Development Plans

From
"Greg Sabino Mullane"
Date:
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1


> What will be in release 8.1?

Three things I am working on and hope to get in 8.1:

* Synchronizing all the psql \d commands so that \dt
shows your tables, \df shows your functions, etc. The
majority of this is done, but the tab-completion part
has slowed me down a bit. Still, it should be in 8.1

* Support for "error catching" in psql via savepoints. This
one is pretty tricky, as we'll need to build our own mini-SQL
parser to catch savepoint creation/destruction, or wait for
someone to implement a hook in libpq to get at the current stack
of savepoints. If I get everything else on my list done, I'll
attack the hook, but would prefer if someone else would do it, as
that's still a bit over my head. The rest of it is pretty much written
though.

* Allowing triggers to be column-based. Someone at one point claimed
they were working on this, but we never heard back from them, so I
am assuming it is still an open item. I have not started on this, I want
to wait to see if anyone else is working on it and/or has existing work
I can build off of.

- --
Greg Sabino Mullane greg@turnstep.com
PGP Key: 0x14964AC8 200502281832
http://biglumber.com/x/web?pk=2529DF6AB8F79407E94445B4BC9B906714964AC8

-----BEGIN PGP SIGNATURE-----

iD8DBQFCI6rLvJuQZxSWSsgRAuntAKDU6ROIjrI6Mc2ZMabEBj/+/bdYTACeIV8G
w9kyTLUklM20MYC58APQkfw=
=ooGo
-----END PGP SIGNATURE-----



Re: Development Plans

From
Josh Berkus
Date:
Greg,

> * Synchronizing all the psql \d commands so that \dt
> shows your tables, \df shows your functions, etc. The
> majority of this is done, but the tab-completion part
> has slowed me down a bit. Still, it should be in 8.1

Is there a file of the queries that back the \d commands somewhere?  They'd
help us for the new system views.

--Josh

--
__Aglio Database Solutions_______________
Josh Berkus               Consultant
josh@agliodbs.com     www.agliodbs.com
Ph: 415-752-2500    Fax: 415-752-2387
2166 Hayes Suite 200    San Francisco, CA


Re: Development Plans

From
"Joshua D. Drake"
Date:
Josh Berkus wrote:

>Greg,
>
>
>
>>* Synchronizing all the psql \d commands so that \dt
>>shows your tables, \df shows your functions, etc. The
>>majority of this is done, but the tab-completion part
>>has slowed me down a bit. Still, it should be in 8.1
>>
>>
>
>Is there a file of the queries that back the \d commands somewhere?  They'd
>help us for the new system views.
>
>
You could use psql -E :)

J



>--Josh
>
>
>


--
Command Prompt, Inc., home of Mammoth PostgreSQL - S/ODBC and S/JDBC
Postgresql support, programming shared hosting and dedicated hosting.
+1-503-667-4564 - jd@commandprompt.com - http://www.commandprompt.com
PostgreSQL Replicator -- production quality replication for PostgreSQL


Attachment

Re: Development Plans

From
Robert Treat
Date:
On Monday 28 February 2005 18:33, Greg Sabino Mullane wrote:
> * Allowing triggers to be column-based. Someone at one point claimed
> they were working on this, but we never heard back from them, so I
> am assuming it is still an open item. I have not started on this, I want
> to wait to see if anyone else is working on it and/or has existing work
> I can build off of.

He might deny it, but I think that was Neil, who said he was thinking about
doing it after he did statement level triggers.  Enough time has passed now
that you could probably start on that without stepping on anyone's toes.

--
Robert Treat
Build A Brighter Lamp :: Linux Apache {middleware} PostgreSQL

Re: Development Plans

From
"Greg Sabino Mullane"
Date:
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1


> Is there a file of the queries that back the \d commands somewhere?
> They'd help us for the new system views.

Most of them are dynamically generated, and some require multiple queries,
but they are located in describe.c:

src/bin/psql/describe.c

As Joshua pointed out though, it's probably much easier to simply
use the -E option and not have to decipher all the C code.

- --
Greg Sabino Mullane greg@turnstep.com
PGP Key: 0x14964AC8 200502282148
http://biglumber.com/x/web?pk=2529DF6AB8F79407E94445B4BC9B906714964AC8

-----BEGIN PGP SIGNATURE-----

iD8DBQFCI9gtvJuQZxSWSsgRAk1ZAKDSIc0cZ8wUxQ9hpJARToc78bVY9wCeMlbk
QVKST5pQlbNQEMbgh7na3F0=
=+v1M
-----END PGP SIGNATURE-----



Re: [HACKERS] Development Plans

From
Bernd Helmle
Date:
--On Freitag, Februar 25, 2005 12:02:47 -0500 Tom Lane <tgl@sss.pgh.pa.us>
wrote:

> I wouldn't mind seeing people be a little more vocal on the hackers list
> about what they plan to be doing, just so that there's not duplication
> of effort.

Jaime Casanova and me are still working on view update rules.

--

  Bernd