Thread: WIP: About CMake v2

Hello Hackers

How would you react if I provided a patch which introduces a CMake build 
system?

Old thread:
http://www.postgresql.org/message-id/200812291325.13354.peter_e@gmx.net

The main argument against the "it's too hard". I'm right?

Thanks!
-- 
YUriy Zhuravlev
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company

YUriy Zhuravlev wrote:
> Hello Hackers
> 
> How would you react if I provided a patch which introduces a CMake build 
> system?

What's your motivation for doing so?

-- 
Álvaro Herrera                http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

On 2015-08-28 12:32:45 -0300, Alvaro Herrera wrote:
> YUriy Zhuravlev wrote:
> > Hello Hackers
> > 
> > How would you react if I provided a patch which introduces a CMake build 
> > system?
> 
> What's your motivation for doing so?

I definitely can see some advantages. Non-broken dependencies around
recursive make being a major one. But I'm also afraid it's a rather
large undertaking. There's a fair number of special kind of rules, and
we're probably not going to want to break pgxs for extensions.

I also have some doubts around the portability of cmake and it's
generated makefiles. We do support some odd platforms.

Greetings,

Andres Freund

Andres Freund wrote:
> On 2015-08-28 12:32:45 -0300, Alvaro Herrera wrote:
> > YUriy Zhuravlev wrote:
> > > Hello Hackers
> > > 
> > > How would you react if I provided a patch which introduces a CMake build 
> > > system?
> > 
> > What's your motivation for doing so?
> 
> I definitely can see some advantages. Non-broken dependencies around
> recursive make being a major one. But I'm also afraid it's a rather
> large undertaking. There's a fair number of special kind of rules, and
> we're probably not going to want to break pgxs for extensions.
> 
> I also have some doubts around the portability of cmake and it's
> generated makefiles. We do support some odd platforms.

If it allows us to get rid of our custom MSVC scripts, it's a huge
benefit, for sure -- that has been a huge pain in the neck since day
one.

Like you, I am also afraid it's a huge undertaking, but if Uri wants to
tackle it, we have that part covered.  Experimentation until we get it
all correct is going to waste some of everybody's time, too, I'm sure.

I wonder about two other things: one is speed of the build (not that
currently it's all that great, given all the mess with recursive make
invocations, but perhaps it can be even worse); the other is how ugly
the generated files are going to be, and are we going to carry them in
our repo -- right now we only have configure, but are we going to keep
extra files to cope with builds in systems that don't have cmake
installed (as we cope with missing bison and flex)?

I sure would oppose something that looks as ugly as Automake makefiles.

-- 
Álvaro Herrera                http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

On 2015-08-28 13:28:49 -0300, Alvaro Herrera wrote:
> the other is how ugly the generated files are going to be, and are we
> going to carry them in our repo -- right now we only have configure,
> but are we going to keep extra files to cope with builds in systems
> that don't have cmake installed (as we cope with missing bison and
> flex)?

Apparently it's still unsupported to ship generated files - they contain
absolute paths and such. Personally I think that might be the
death-knell - I doubt that we want to have a full dependency on cmake on
every platform?

Alvaro Herrera <alvherre@2ndquadrant.com> writes:
> I wonder about two other things: one is speed of the build (not that
> currently it's all that great, given all the mess with recursive make
> invocations, but perhaps it can be even worse); the other is how ugly
> the generated files are going to be, and are we going to carry them in
> our repo -- right now we only have configure, but are we going to keep
> extra files to cope with builds in systems that don't have cmake
> installed (as we cope with missing bison and flex)?

As near as I can tell, the generated files are platform-specific.
(They're certainly different for Unix and Windows; the overview
I'm looking at doesn't say in so many words whether they can vary
at a finer grain, but I bet they do.)  So I'm afraid cmake would
likely become a build requirement, even for tarball users.  That
is probably not a show-stopper, but it's a point against the idea.

I have no idea whether switching to cmake would be a good thing or not.
It's possible that it'd end up being even uglier than our current
autoconf+gmake+msvc-scripts mess ... although when phrased that way,
that sounds like a pretty low bar to clear.  Anyway, if YUriy is willing
to do the preliminary investigation, let's see what he comes up with.
        regards, tom lane

<p dir="ltr">It's broadly interesting, but since it bakes in a build dependency on CMake, there is some risk that the
dependenciesbecome an insurmountable problem.<p dir="ltr">(Does CMake run on a VAX 11/780??  :-))<p dir="ltr">It is
probablyworth a try, to see what improvements arise, albeit with the need to accept some risk of refusal of the
change.<pdir="ltr">The experiment is most likely necessary: we won't know the benefits without trying.<p dir="ltr">If
theresults represent little improvement, there will be little or no appetite to jump through the dependency hoops
neededto get the change accepted.<p dir="ltr">On the other hand, if there are big gains, that encourages pushing thru
thedependency issues.<div class="gmail_quote">On Aug 28, 2015 10:45, "YUriy Zhuravlev" <<a
href="mailto:u.zhuravlev@postgrespro.ru">u.zhuravlev@postgrespro.ru</a>>wrote:<br type="attribution" /><blockquote
class="gmail_quote"style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hello Hackers<br /><br /> How
wouldyou react if I provided a patch which introduces a CMake build<br /> system?<br /><br /> Old thread:<br /><a
href="http://www.postgresql.org/message-id/200812291325.13354.peter_e@gmx.net"rel="noreferrer"
target="_blank">http://www.postgresql.org/message-id/200812291325.13354.peter_e@gmx.net</a><br/><br /> The main
argumentagainst the "it's too hard". I'm right?<br /><br /> Thanks!<br /> --<br /> YUriy Zhuravlev<br /> Postgres
Professional:<a href="http://www.postgrespro.com" rel="noreferrer" target="_blank">http://www.postgrespro.com</a><br />
TheRussian Postgres Company<br /><br /><br /> --<br /> Sent via pgsql-hackers mailing list (<a
href="mailto:pgsql-hackers@postgresql.org">pgsql-hackers@postgresql.org</a>)<br/> To make changes to your
subscription:<br/><a href="http://www.postgresql.org/mailpref/pgsql-hackers" rel="noreferrer"
target="_blank">http://www.postgresql.org/mailpref/pgsql-hackers</a><br/></blockquote></div> 

Christopher Browne <cbbrowne@gmail.com> writes:
> (Does CMake run on a VAX 11/780??  :-))

Yeah.  I see the two major risks as being:

1. We limit ourselves to platforms that cmake works on.

2. We lose the ability to handle weird special-case tests that are
possible (if not necessarily pleasant) with autoconf/gmake.

I might be overly worried about #2 --- but the impression I have of cmake
is that they've spent lots of time on "make easy cases easy" and maybe not
enough on "make hard cases possible".

Anyway, we won't know unless somebody tries it.  There will certainly be
some pluses and some minuses, and we'll have to decide whether the pluses
outweigh the minuses.  I don't think we can tell that without a fairly
detailed attempt at making it work.
        regards, tom lane

On Friday 28 August 2015 13:28:49 Alvaro Herrera wrote:
> Andres Freund wrote:
> > On 2015-08-28 12:32:45 -0300, Alvaro Herrera wrote:
> > > YUriy Zhuravlev wrote:
> > > > Hello Hackers
> > > > 
> > > > How would you react if I provided a patch which introduces a CMake
> > > > build
> > > > system?
> > > 
> > > What's your motivation for doing so?
> > 
> > I definitely can see some advantages. Non-broken dependencies around
> > recursive make being a major one. But I'm also afraid it's a rather
> > large undertaking. There's a fair number of special kind of rules, and
> > we're probably not going to want to break pgxs for extensions.
> > 
> > I also have some doubts around the portability of cmake and it's
> > generated makefiles. We do support some odd platforms.
> 
> If it allows us to get rid of our custom MSVC scripts, it's a huge
> benefit, for sure -- that has been a huge pain in the neck since day
> one.

As a user, I suffer a bit with the current build system when I try to build 
libpq in MinGW.  So, if moving to CMake means the build is more 'decoupled', 
i.e. less inter-dependent modules, it would be great.

On Friday 28 August 2015 13:51:30 you wrote:
> It's broadly interesting, but since it bakes in a build dependency on
> CMake, there is some risk that the dependencies become an insurmountable
> problem.
> 
> (Does CMake run on a VAX 11/780??  :-))

http://public.kitware.com/Bug/view.php?id=13605 you about this? 


> 
> It is probably worth a try, to see what improvements arise, albeit with the
> need to accept some risk of refusal of the change.
> 
> The experiment is most likely necessary: we won't know the benefits without
> trying.

You right.

> 
> If the results represent little improvement, there will be little or no
> appetite to jump through the dependency hoops needed to get the change
> accepted.
> 
> On the other hand, if there are big gains, that encourages pushing thru the
> dependency issues.
> 
> On Aug 28, 2015 10:45, "YUriy Zhuravlev" <u.zhuravlev@postgrespro.ru> wrote:
> > Hello Hackers
> > 
> > How would you react if I provided a patch which introduces a CMake build
> > system?
> > 
> > Old thread:
> > http://www.postgresql.org/message-id/200812291325.13354.peter_e@gmx.net
> > 
> > The main argument against the "it's too hard". I'm right?
> > 
> > Thanks!
> > --
> > YUriy Zhuravlev
> > Postgres Professional: http://www.postgrespro.com
> > The Russian Postgres Company
> > 
> > 
> > --
> > Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
> > To make changes to your subscription:
> > http://www.postgresql.org/mailpref/pgsql-hackers

-- 
YUriy Zhuravlev
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company

YUriy Zhuravlev wrote:
> On Friday 28 August 2015 13:51:30 you wrote:
> > It's broadly interesting, but since it bakes in a build dependency on
> > CMake, there is some risk that the dependencies become an insurmountable
> > problem.
> > 
> > (Does CMake run on a VAX 11/780??  :-))
> 
> http://public.kitware.com/Bug/view.php?id=13605 you about this? 

Hmm, what this closed bug report says to me is that 1) cmake developers
take portability seriously, 2) all common and even some not-so-common
platforms are likely to be in hand already.  Therefore, either we will
not find any portability problems, or fixing upstream those we do find
will not be terribly difficult.

-- 
Álvaro Herrera                http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

On 2015-08-29 17:53:26 -0300, Alvaro Herrera wrote:
> Therefore, either we will not find any portability problems, or fixing
> upstream those we do find will not be terribly difficult.

Well, the difference to know is that we can't resolve that relatively
quickly ourselves, but that it'd rather require $whoever installing a
new version of cmake...

Andres Freund <andres@anarazel.de> writes:
> On 2015-08-29 17:53:26 -0300, Alvaro Herrera wrote:
>> Therefore, either we will not find any portability problems, or fixing
>> upstream those we do find will not be terribly difficult.

> Well, the difference to know is that we can't resolve that relatively
> quickly ourselves, but that it'd rather require $whoever installing a
> new version of cmake...

Yup.  So one of the things we'd want to establish is what's the minimum
version of cmake we're going to require, and which (hopefully old)
platforms will we be leaving out in the cold with that?

I suspect it's premature to try to decide that now, but it's certainly
a question to keep in mind.
        regards, tom lane

Tom Lane wrote:
> Andres Freund <andres@anarazel.de> writes:
> > On 2015-08-29 17:53:26 -0300, Alvaro Herrera wrote:
> >> Therefore, either we will not find any portability problems, or fixing
> >> upstream those we do find will not be terribly difficult.
> 
> > Well, the difference to know is that we can't resolve that relatively
> > quickly ourselves, but that it'd rather require $whoever installing a
> > new version of cmake...
> 
> Yup.  So one of the things we'd want to establish is what's the minimum
> version of cmake we're going to require, and which (hopefully old)
> platforms will we be leaving out in the cold with that?

Just for the record, in Debian land the versions shipped are thus:

oldoldstable (squeeze): 2.8.2 (has 2.8.9 available in squeeze-backports)
oldstable (wheezy): 2.8.9 (has 2.8.11 available in wheezy-backports)
stable (jessie): 3.0.2

VAX support was added in 2.8.11 (bug fixed in Oct. 2012).

I imagine that esoteric platforms are not going to have cmake at all and
are going to need their own installation anyway.  Not sure if that's
going to be more onerous than the requirement to install GNU make.

-- 
Álvaro Herrera                http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

On 8/28/15 10:14 AM, YUriy Zhuravlev wrote:
> Hello Hackers
>
> How would you react if I provided a patch which introduces a CMake build
> system?

I would say, "Yay!".

I had that on my todo list, but I won't mind if someone else does it. 
I've certainly "had it" with the current build system. ;-)

Alvaro Herrera <alvherre@2ndquadrant.com> writes:
> I imagine that esoteric platforms are not going to have cmake at all and
> are going to need their own installation anyway.  Not sure if that's
> going to be more onerous than the requirement to install GNU make.

If we get to the point where this is starting to look like a real
proposal, I'll try to install cmake from source on my buildfarm critters,
none of which have it ATM.  And we'll need to get other buildfarm owners
to do likewise.  So we'll be able to acquire data points on whether that's
a big problem or not.
        regards, tom lane

Thanks all hackers.
I have not heard of fundamental problems and continue its development. :)



-- 
YUriy Zhuravlev
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company

On Fri, Aug 28, 2015 at 01:28:49PM -0300, Alvaro Herrera wrote:
> If it allows us to get rid of our custom MSVC scripts, it's a huge
> benefit, for sure -- that has been a huge pain in the neck since day
> one.

Moreover, I suggest beginning with a patch that replaces the src/tools/msvc
build system with CMake.  One can test almost all Windows configurations using
two machines, so author and reviewer can test most everything.  src/tools/msvc
has more defects than the GNU make build system, so the minimum-quality
acceptable replacement is easier to deliver.  With CMake driving all MSVC
builds, it would be much easier to gradually extend to other platforms.

Thanks,
nm

Noah Misch <noah@leadboat.com> writes:
> On Fri, Aug 28, 2015 at 01:28:49PM -0300, Alvaro Herrera wrote:
>> If it allows us to get rid of our custom MSVC scripts, it's a huge
>> benefit, for sure -- that has been a huge pain in the neck since day
>> one.

> Moreover, I suggest beginning with a patch that replaces the src/tools/msvc
> build system with CMake.  One can test almost all Windows configurations using
> two machines, so author and reviewer can test most everything.  src/tools/msvc
> has more defects than the GNU make build system, so the minimum-quality
> acceptable replacement is easier to deliver.  With CMake driving all MSVC
> builds, it would be much easier to gradually extend to other platforms.

mmm ... dunno about that plan.  In the first place, you seem to be
recommending that YUriy incorporate all the deficiencies of src/tools/msvc
into his first draft and then hope somehow to fix them later.  In the
second place, I think that the core problem here involves being sure
we can still cover all the portability issues that have been laboriously
hacked into the autoconf/gmake build system over nigh two decades.
Starting from a build system that was designed to cover exactly one target
platform, and has never been asked to do anything more than that, doesn't
sound like a recipe for success there.

Maybe you're right about how to approach this, but I wouldn't bet on it.
        regards, tom lane

On Fri, Aug 28, 2015 at 11:39 AM, Andres Freund <andres@anarazel.de> wrote:
> I definitely can see some advantages. Non-broken dependencies around
> recursive make being a major one. But I'm also afraid it's a rather
> large undertaking. There's a fair number of special kind of rules, and
> we're probably not going to want to break pgxs for extensions.

Maybe we should merge all of the makefiles for subdirectories of
src/backend into a single makefile.  The major disadvantage would be
that you couldn't rebuild a subdirectory any more by typing make -C
src/backend/executor or whatever.  And I do do that sometimes, so
maybe it would be annoying, but presumably it would make the
dependency issues a lot easier to deal with.

I guess I'm a bit skeptical about the idea of porting to a new build
system.  There's a good chance of replacing the problems we know about
with new problems that are no less serious, but merely unknown to us.
But I'm not going to stand here and hold my breath if everyone else
thinks it's a good idea.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

Robert Haas <robertmhaas@gmail.com> writes:
> I guess I'm a bit skeptical about the idea of porting to a new build
> system.  There's a good chance of replacing the problems we know about
> with new problems that are no less serious, but merely unknown to us.
> But I'm not going to stand here and hold my breath if everyone else
> thinks it's a good idea.

FWIW, I'm also skeptical that this experiment will succeed.  I think it
is worth trying ... but I'm by no means promising to buy into the result.

There is precedent for this sort of thing: mysql converted from
autoconf+gmake to cmake a few years back, and AFAIK they've not
regretted it.  So that's a data point that a project of complexity
more or less similar to ours can make the change.

Where I do *not* want to end up is maintaining *three* build systems.
So I'll definitely resist any proposals to commit a partly-done cmake
conversion (which I fear might seem attractive at some point).
        regards, tom lane

On 2015-09-01 10:05:44 -0400, Tom Lane wrote:
> Where I do *not* want to end up is maintaining *three* build systems.
> So I'll definitely resist any proposals to commit a partly-done cmake
> conversion (which I fear might seem attractive at some point).

Enthusiastically seconded.

On Tue, Sep 01, 2015 at 12:49:55AM -0400, Tom Lane wrote:
> Noah Misch <noah@leadboat.com> writes:
> > On Fri, Aug 28, 2015 at 01:28:49PM -0300, Alvaro Herrera wrote:
> >> If it allows us to get rid of our custom MSVC scripts, it's a huge
> >> benefit, for sure -- that has been a huge pain in the neck since day
> >> one.
> 
> > Moreover, I suggest beginning with a patch that replaces the src/tools/msvc
> > build system with CMake.  One can test almost all Windows configurations using
> > two machines, so author and reviewer can test most everything.  src/tools/msvc
> > has more defects than the GNU make build system, so the minimum-quality
> > acceptable replacement is easier to deliver.  With CMake driving all MSVC
> > builds, it would be much easier to gradually extend to other platforms.
> 
> mmm ... dunno about that plan.  In the first place, you seem to be
> recommending that YUriy incorporate all the deficiencies of src/tools/msvc
> into his first draft and then hope somehow to fix them later.

No, src/tools/msvc is a poor source of implementation ideas.  Implement the
CMake build in idiomatic CMake style.  Write an initial patch sufficient so we
could delete src/tools/msvc in the same patch, and make little or no attempt
to cover other cases.  That's my suggestion.

> In the
> second place, I think that the core problem here involves being sure
> we can still cover all the portability issues that have been laboriously
> hacked into the autoconf/gmake build system over nigh two decades.

I agree that the first patch should demonstrate the CMake programming
environment's sufficiency for open-ended portability adaptations.  I doubt
demonstrating that requires actual porting to more than a platform or two.

> Starting from a build system that was designed to cover exactly one target
> platform, and has never been asked to do anything more than that, doesn't
> sound like a recipe for success there.

Fair point.  Perhaps replace src/tools/msvc completely and build+install on at
least one reasonable GNU/Linux configuration?

A monolithic patch replacing the GNU make build system with a CMake build
system sounds far too hard to write and review; we should expect to maintain
those two build systems in parallel for awhile.  Reconciling the MSVC and GNU
make build systems is a chronic problem today, so it would be a big plus to
have no more than two of MSVC, GNU Make and CMake build systems to maintain.

On 2015-09-01 10:32:39 -0400, Noah Misch wrote:
> A monolithic patch replacing the GNU make build system with a CMake build
> system sounds far too hard to write and review; we should expect to maintain
> those two build systems in parallel for awhile.

I don't buy this. I don't think we're going to decide to switch to cmake
before seing how the gmake conversion will look like. Possibly the gmake
part doesn't have to be immediately committable when we replace msvc
with cmake, but I'll definitely want to see the benefit (i.e. see a
one-source-of-truth tree) before paying.

Greetings,

Andres Freund

Andres Freund <andres@anarazel.de> writes:
> On 2015-09-01 10:32:39 -0400, Noah Misch wrote:
>> A monolithic patch replacing the GNU make build system with a CMake build
>> system sounds far too hard to write and review; we should expect to maintain
>> those two build systems in parallel for awhile.

> I don't buy this.

Me either.  I think the odds would be better than 50-50 that we'd never
get past that stage, and we'd be left with a situation where we've still
got two build systems, only one of them is cmake not hand-rolled perl.

Now, maybe that's still an improvement over what we've got, but the
argument is significantly harder to make.  We'd have bought into all of
cmake's disadvantages (whatever they prove to be) but we'd be missing
the main claimed advantage.  Also, while src/tools/msvc certainly has
got limitations, it does get a fair amount of its info out of the
Makefiles; having to change it when you add a file or whatever is the
exception not the rule.  A separate cmake build system would certainly
require maintenance *every* time we touch the Makefiles.

I would actually suggest that the cmake conversion would be better off
to ignore src/tools/msvc altogether to begin with.  Build something that
can handle, say, the Linux build, and then stop and evaluate.  If that
looks good then start extending to other platforms.  If cmake is as nifty
as some folks claim, you might already have an 80%-working Windows build
at that point.
        regards, tom lane

On Tuesday 01 September 2015 11:46:05 you wrote:
> I would actually suggest that the cmake conversion would be better off
> to ignore src/tools/msvc altogether to begin with.

Build postgres by MSVC is important task for me. But Linux first of course. 

> A separate cmake build system would certainly
> require maintenance *every* time we touch the Makefiles.

I can support the cmake build a separate git branch on github. 
When we (and users) see that all is well then we will think when and how to 
change GNUMake to CMake.
-- 
YUriy Zhuravlev
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company

On Tue, Sep 1, 2015 at 2:41 PM, Robert Haas <robertmhaas@gmail.com> wrote:
> Maybe we should merge all of the makefiles for subdirectories of
> src/backend into a single makefile.  The major disadvantage would be
> that you couldn't rebuild a subdirectory any more by typing make -C
> src/backend/executor or whatever.  And I do do that sometimes, so
> maybe it would be annoying, but presumably it would make the
> dependency issues a lot easier to deal with.

Actually I think it's possible to cater to that use case though I've
never tried.

What I think does become difficult in this scenario are directories
that are built more than once, eg, for client and server side builds.
I guess you could still do it but it would require doubling up every
rule and creating lots of extra targets representing the two builds
since you can't just call make with different variables.

It is tempting and I've been wanting to evalangize this approach ever
since read http://aegis.sourceforge.net/auug97.pdf but I've never
actually had practical experience with it and iirc it's always this
scenario of wanting to compile submodules multiple times that blocked
using it.

-- 
greg

Greg Stark wrote:

> It is tempting and I've been wanting to evalangize this approach ever
> since read http://aegis.sourceforge.net/auug97.pdf but I've never
> actually had practical experience with it and iirc it's always this
> scenario of wanting to compile submodules multiple times that blocked
> using it.

I actually played with that a bit awhile back, after seeing that paper
referenced the fourth or fifth time (meh).  I converted a few of our
current subdir makefiles into "module.mk" files that were included
rather than recursed into; for instance optimizer/util/module.mk would
do this:

+OBJS += optimizer/util/clauses.o optimizer/util/joininfo.o optimizer/util/orclauses.o optimizer/util/pathnode.o
optimizer/util/placeholder.o\
 
+       optimizer/util/plancat.o optimizer/util/predtest.o optimizer/util/relnode.o optimizer/util/restrictinfo.o
optimizer/util/tlist.ooptimizer/util/var.o
 

then the backend-level makefile would do this:

+include $(patsubst %,%/module.mk,$(NEWSUBDIRS))

where NEWSUBDIRS were the subdirs I had converted, so

NEWSUBDIRS = access optimizer

In turn, optimizer/module.mk would be this:

NEWSUBDIRS     = geqo path plan prep util

include $(patsubst %,optimizer/%/module.mk,$(NEWSUBDIRS))

It seemed to work fine for the small subset that I changed; then I got
lazy and never got around to writing a macro to do the subdir makefile
conversion (and of course I wasn't going to do it by hand).  Of course,
some things needs special treatment (postgres.bki, gram.c etc) which I
also never got around to even trying.

-- 
Álvaro Herrera                http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

On Tue, Sep 01, 2015 at 11:46:05AM -0400, Tom Lane wrote:
> Andres Freund <andres@anarazel.de> writes:
> > On 2015-09-01 10:32:39 -0400, Noah Misch wrote:
> >> A monolithic patch replacing the GNU make build system with a CMake build
> >> system sounds far too hard to write and review; we should expect to maintain
> >> those two build systems in parallel for awhile.
> 
> > I don't buy this.
> 
> Me either.

Okay.  If the author, reviewer(s) and committer deliver a correct monolithic
patch, I won't have anything to complain about.  For the PostgreSQL hackers
not involved in this work, that outcome is better than my suggestion.

Hello hackers.

News about CMake:
I built postgres, initdb, createdb, psql, pg_ctl using CMake.
After make install you can run initdb after run postgres after createdb and 
use it by psql. Only for Linux now and realy bugy (and the code is very dirt) 
but it work!
If someone wants to test or to help:
https://github.com/stalkerg/postgres_cmake

Thanks.

PS All define for pg_config.h generate and testing truly
-- 
YUriy Zhuravlev
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company

<div dir="ltr"><br /><div class="gmail_extra"><br /><div class="gmail_quote">On Wed, Nov 25, 2015 at 1:35 AM, YUriy
Zhuravlev<span dir="ltr"><<a href="mailto:u.zhuravlev@postgrespro.ru"
target="_blank">u.zhuravlev@postgrespro.ru</a>></span>wrote:<br /><blockquote class="gmail_quote" style="margin:0 0
0.8ex;border-left:1px #ccc solid;padding-left:1ex">News about CMake:<br /> I built postgres, initdb, createdb, psql,
pg_ctlusing CMake.<br /> After make install you can run initdb after run postgres after createdb and<br /> use it by
psql.Only for Linux now and realy bugy (and the code is very dirt)<br /> but it work!<br /> If someone wants to test or
tohelp:<br /><a href="https://github.com/stalkerg/postgres_cmake" rel="noreferrer"
target="_blank">https://github.com/stalkerg/postgres_cmake</a><br/><br /> PS All define for pg_config.h generate and
testingtruly<br /></blockquote></div><br /></div><div class="gmail_extra">Thanks for sharing the progress! That's
reallysomething that would pay long-term.<br />-- <br /><div class="gmail_signature">Michael<br /></div></div></div> 

On Wednesday 25 November 2015 13:27:33 you wrote:
> That was years ago, mind, but I
> haven't seen CMake exactly taking over the world since then.
KDE, MariaDB, neovim and etc. New projects usually selected CMake. 

> Many prewritten CMake modules fail to follow basic practices that make them
> work anywhere but where the author wanted/needed.
Now everything is much better. And it is worth split "make" and configuration. 
CMake better GNU Make and is similar to the m4 for configuring. But M4 older 
and elaborated.

These are my feelings on the current work.

Thanks.
-- 
YUriy Zhuravlev
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company

On 24-11-2015 13:35, YUriy Zhuravlev wrote:
> News about CMake:
> I built postgres, initdb, createdb, psql, pg_ctl using CMake.
> After make install you can run initdb after run postgres after createdb and 
> use it by psql. Only for Linux now and realy bugy (and the code is very dirt) 
> but it work!
> If someone wants to test or to help:
> https://github.com/stalkerg/postgres_cmake
> 
I give it a try. Nice WIP. IMHO you should try to support cmake version
that are available in the stable releases. Looking at [1], I think the
best choice is 2.8.11 (because it will cover Red Hat based distros and
also Debian based ones). Are you using a new feature from 3.1? I mean,
it should be nice to cover old stable releases, if it is possible.


[1] https://cmake.org/Wiki/CMake_Life_Cycle_Considerations


--   Euler Taveira                   Timbira - http://www.timbira.com.br/  PostgreSQL: Consultoria, Desenvolvimento,
Suporte24x7 e Treinamento
 

On Thursday 26 November 2015 01:29:37 Euler Taveira wrote:
> I give it a try. Nice WIP. IMHO you should try to support cmake version
> that are available in the stable releases. Looking at [1], I think the
> best choice is 2.8.11 (because it will cover Red Hat based distros and
> also Debian based ones). Are you using a new feature from 3.1? I mean,
> it should be nice to cover old stable releases, if it is possible.
Maybe you are right. But by the time I finish my work I think 3.0 will become 
a standard. CMake is developing rapidly and soon will have version 3.4.1
And one more thing: a normal documentation came with 3.0. :)
But I try to check my code for 2.8.11, now I have 3.4.0 (latest for Gentoo).
Thanks. 

-- 
YUriy Zhuravlev
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company

On 26-11-2015 07:33, YUriy Zhuravlev wrote:
> On Thursday 26 November 2015 01:29:37 Euler Taveira wrote:
>> I give it a try. Nice WIP. IMHO you should try to support cmake version
>> that are available in the stable releases. Looking at [1], I think the
>> best choice is 2.8.11 (because it will cover Red Hat based distros and
>> also Debian based ones). Are you using a new feature from 3.1? I mean,
>> it should be nice to cover old stable releases, if it is possible.
> Maybe you are right. But by the time I finish my work I think 3.0 will become 
> a standard. CMake is developing rapidly and soon will have version 3.4.1
> And one more thing: a normal documentation came with 3.0. :)
> But I try to check my code for 2.8.11, now I have 3.4.0 (latest for Gentoo).
> 
Have in mind that stable distros have a long cycle and are not released
soon. If you are planning your cmake work for 9.6 or even 9.7, it is
prudent to suport Red Hat 7 or Debian 8 because it will be a pain in the
neck to install a new cmake version just to compile postgres.


--   Euler Taveira                   Timbira - http://www.timbira.com.br/  PostgreSQL: Consultoria, Desenvolvimento,
Suporte24x7 e Treinamento
 

Euler Taveira <euler@timbira.com.br> writes:
> On 26-11-2015 07:33, YUriy Zhuravlev wrote:
>> Maybe you are right. But by the time I finish my work I think 3.0 will become 
>> a standard. CMake is developing rapidly and soon will have version 3.4.1
>> And one more thing: a normal documentation came with 3.0. :)
>> But I try to check my code for 2.8.11, now I have 3.4.0 (latest for Gentoo).

> Have in mind that stable distros have a long cycle and are not released
> soon. If you are planning your cmake work for 9.6 or even 9.7, it is
> prudent to suport Red Hat 7 or Debian 8 because it will be a pain in the
> neck to install a new cmake version just to compile postgres.

Not working with the cmake version shipped in current distributions would
almost certainly cause us to reject this patch.  Adding a new build
dependency is bad enough; adding one that isn't easily available is a
show-stopper.  You'd better think in terms of what's provided with RHEL6,
not RHEL7, as the minimum baseline on the Red Hat side.  I'm not sure what
the oldest active LTS distribution is in the Debian world, but I'm pretty
sure it won't have cmake 3.

(FWIW, RHEL6 seems to be carrying 2.8.12 currently.)
        regards, tom lane

On Thursday 26 November 2015 11:10:36 you wrote:
> Adding a new build
> dependency is bad enough; adding one that isn't easily available is a
> show-stopper.

If someone decided to compile from source Postgres rather than install from 
RPM then it will not be a problem as to build CMake.
On the one hand it is good that the GNU Make a stable and have not changed, 
but it can not last forever.

It is possible to make support for CMake 2.6 but now is not the time when you 
have to think about it. I'll come back to this when all the functionality will 
work.

Thanks. 
-- 
YUriy Zhuravlev
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company

On 2015-11-26 19:22:23 +0300, YUriy Zhuravlev wrote:
> On Thursday 26 November 2015 11:10:36 you wrote:
> > Adding a new build
> > dependency is bad enough; adding one that isn't easily available is a
> > show-stopper.
> 
> If someone decided to compile from source Postgres rather than install from 
> RPM then it will not be a problem as to build CMake.

Packages have to be built from something, and usually it's desirable
that all dependencies are from within the distribution.

I don't think you'll have much success pushing back on this.

> It is possible to make support for CMake 2.6 but now is not the time when you 
> have to think about it. I'll come back to this when all the functionality will 
> work.

No point in doing any work if you don't agree with the basic prerequisites.

Andres Freund <andres@anarazel.de> writes:
> On 2015-11-26 19:22:23 +0300, YUriy Zhuravlev wrote:
>> On Thursday 26 November 2015 11:10:36 you wrote:
>>> Adding a new build dependency is bad enough; adding one that isn't
>>> easily available is a show-stopper.

>> If someone decided to compile from source Postgres rather than install from 
>> RPM then it will not be a problem as to build CMake.

> Packages have to be built from something, and usually it's desirable
> that all dependencies are from within the distribution.

Yeah.  In particular, RPMs don't magically appear out of nowhere; somebody
has to build them, and more often than not, the somebody is subject to
distro packaging policy that says what build dependencies are allowed.

Red Hat certainly isn't ever going to ship a package that requires some
version of cmake that isn't what they ship in the particular distribution,
for example.
        regards, tom lane

On Thursday 26 November 2015 17:42:16 you wrote:
> No point in doing any work if you don't agree with the basic prerequisites.
I meant that support for older versions of CMake I'll do when will implement 
other functions.
-- 
YUriy Zhuravlev
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company

Tom Lane wrote:

> Not working with the cmake version shipped in current distributions would
> almost certainly cause us to reject this patch.  Adding a new build
> dependency is bad enough; adding one that isn't easily available is a
> show-stopper.  You'd better think in terms of what's provided with RHEL6,
> not RHEL7, as the minimum baseline on the Red Hat side.  I'm not sure what
> the oldest active LTS distribution is in the Debian world, but I'm pretty
> sure it won't have cmake 3.

Current Debian Stable (wheezy) has cmake 3.0.2.  Oldstable (jessie) has
2.8.4.

As for Ubuntu, according to https://wiki.ubuntu.com/LTS the oldest still
supported is 12.04 (Precise Pangolin) and they have 2.8.7:
http://packages.ubuntu.com/precise/cmake

For these systems, the CMake 2.8 functionality would be enough.  No need
to go back to 2.6.

-- 
Álvaro Herrera                http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

On 26-11-2015 14:06, YUriy Zhuravlev wrote:
> On Thursday 26 November 2015 17:42:16 you wrote:
>> No point in doing any work if you don't agree with the basic prerequisites.
> I meant that support for older versions of CMake I'll do when will implement 
> other functions.
> 
I think you don't understand the point: start with the *right* cmake
version because you could have to redo (a lot of) your work or have your
patch rejected because you don't follow our advice.


--   Euler Taveira                   Timbira - http://www.timbira.com.br/  PostgreSQL: Consultoria, Desenvolvimento,
Suporte24x7 e Treinamento
 

Euler Taveira <euler@timbira.com.br> writes:
> On 26-11-2015 14:06, YUriy Zhuravlev wrote:
>> I meant that support for older versions of CMake I'll do when will implement 
>> other functions.

> I think you don't understand the point: start with the *right* cmake
> version because you could have to redo (a lot of) your work or have your
> patch rejected because you don't follow our advice.

The way Yuriy wants to do it is not necessarily wrong.  He might end up
with cleaner code if he starts by making a cmake-3 implementation and then
figures out how to back-port to cmake-2, rather than starting with the
more limited language to begin with.

Or maybe not.  But I doubt it's open and shut.
        regards, tom lane

Craig Ringer <craig@2ndquadrant.com> writes:
> One thing to consider: I can't imagine backporting this to all supported
> back branches, it'd be a switch for the next release. Right?

Agreed.

> That means he doesn't have to worry about what RH / Debian policy for their
> old versions is. RH isn't going to release PostgreSQL 9.7 or whatever for
> RHEL6, Debian isn't going to release it for Wheezy, etc.

Well, they won't if we make it impossible for them to do so.

More generally, I do not buy this argument for one second as a reason why
we can demand latest-and-greatest cmake, rather than something that's
likely to be readily available on a wide variety of platforms.  Devrim is
not the only person in the world who will be needing to build PG on RHEL6,
or even older platforms.

If you take a close look at our build requirements, you will notice a
general distaste for insisting on latest anything.  cmake is not going
to escape that project bias.  Do you really think a project that still
works with C89, Perl 5.8.something, Python 2.3, bison 1.875, yadda yadda
is readily going to accept a patch that requires this year's cmake?
It would take a fairly impressive technical argument why working with
older cmakes is impossible/impractical before that will happen.
        regards, tom lane

On Fri, Nov 27, 2015 at 2:50 PM, Craig Ringer <craig@2ndquadrant.com> wrote:
> On 27 November 2015 at 12:39, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>>
>> Euler Taveira <euler@timbira.com.br> writes:
>> > On 26-11-2015 14:06, YUriy Zhuravlev wrote:
>> >> I meant that support for older versions of CMake I'll do when will
>> >> implement
>> >> other functions.
>>
>> > I think you don't understand the point: start with the *right* cmake
>> > version because you could have to redo (a lot of) your work or have your
>> > patch rejected because you don't follow our advice.
>>
>> The way Yuriy wants to do it is not necessarily wrong.  He might end up
>> with cleaner code if he starts by making a cmake-3 implementation and then
>> figures out how to back-port to cmake-2, rather than starting with the
>> more limited language to begin with.
>>
>> Or maybe not.  But I doubt it's open and shut.
>
> One thing to consider: I can't imagine backporting this to all supported
> back branches, it'd be a switch for the next release. Right?
>
> That means he doesn't have to worry about what RH / Debian policy for their
> old versions is. RH isn't going to release PostgreSQL 9.7 or whatever for
> RHEL6, Debian isn't going to release it for Wheezy, etc.
>
> We are. Or rather, the people within the community who perform the thankless
> task of packaging are.

That's not only a problem related to packaging by pgdg maintainers or
the main Linux distributions. I think that you are forgetting people
who actually work on or compile the Postgres code on those platforms,
say because they for example need to perform performance test or
measurements particularly on such platforms based on customer
requirements, one possibility being to test different configure
options like bigger relation file blocks for example. It does not seem
acceptable to me to choose a minimal version of cmake without this
criteria taken into account.
-- 
Michael

On Thursday 26 November 2015 19:28:15 you wrote:
> I think you don't understand the point: start with the *right* cmake
> version because you could have to redo (a lot of) your work

Between versions CMake there is no fundamental difference.
On my laptop or desktop is already 3.4.0 (new KDE requires). At friends is 
generally have the same all over 3.0.0. It is simply not convenient right now 
(develop under 2.8). 
But I try not to use 3.0 features. Later simply check will need to be 2.8.

Importantly do not forget CMake != GNU Make. CMake == autotools+gnumake.
New versions of CMake most fixes modules like FindBISON and etc. This 
different philosophy. 

Thanks.
-- 
YUriy Zhuravlev
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company

Hello Hackers.

I decided to talk about the current state of the project:
1. Merge with 9.6 master.
2. plpython2, plpython3, plperl, pltcl, plsql all work correctly (all tests
pass).
3. Works done for all contrib modules.
4. You can use gettext, .po->.mo will have converted by CMake.
5. All test pass under some Linux, FreeBSD, Solaris10 (on Sparc), Windows
MSVC 2015. MacOSX I think not big trouble too.
6. Prototype for PGXS (with MSVC support) done.

I think is very big progress but I came across one problem.
I not have access to many OS and platforms. For each platform need tests
and small fixes. I can't develop and give guarantee without it.

I think this is very important work which makes it easier further support
Postgres but I can not do everything himself. It's physically impossible.

I think without community support I can't do significantly more.

Current version you can get here:
https://github.com/stalkerg/postgres_cmake

Thanks!
--
Yury Zhuravlev
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company

Yury Zhuravlev wrote:
> Hello Hackers.
> 
> I decided to talk about the current state of the project:
> 1. Merge with 9.6 master. 2. plpython2, plpython3, plperl, pltcl, plsql all
> work correctly (all tests pass).
> 3. Works done for all contrib modules. 4. You can use gettext, .po->.mo will
> have converted by CMake.  5. All test pass under some Linux, FreeBSD,
> Solaris10 (on Sparc), Windows MSVC 2015. MacOSX I think not big trouble too.
> 6. Prototype for PGXS (with MSVC support) done.

I just tried a standard build in Debian Jessie and it works nicely.
Well done!  I think we should seriously consider this once 10 opens for
development.

-- 
Álvaro Herrera                http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

On Thu, Jun 30, 2016 at 5:10 AM, Alvaro Herrera
<alvherre@2ndquadrant.com> wrote:
> Yury Zhuravlev wrote:
>> Hello Hackers.
>>
>> I decided to talk about the current state of the project:
>> 1. Merge with 9.6 master. 2. plpython2, plpython3, plperl, pltcl, plsql all
>> work correctly (all tests pass).
>> 3. Works done for all contrib modules. 4. You can use gettext, .po->.mo will
>> have converted by CMake.  5. All test pass under some Linux, FreeBSD,
>> Solaris10 (on Sparc), Windows MSVC 2015. MacOSX I think not big trouble too.
>> 6. Prototype for PGXS (with MSVC support) done.

Yeah! 6 would be really cool to have. Now that's doable with the perl
classes at disposition, but that's not really user-friendly.

> I just tried a standard build in Debian Jessie and it works nicely.
> Well done!  I think we should seriously consider this once 10 opens for
> development.

Yes! Heartfully agreed.

You have not tested with macOS, and so did I. With 10.11 with cmake
from brew I am getting this error at the first step:
-- Check flexible array support - yes with
CMake Error at cmake/RegressCheck.cmake:12 (elif): Unknown CMake command "elif".
Call Stack (most recent call first): CMakeLists.txt:782 (include)
$ cmake --version
cmake version 3.5.2
CMake suite maintained and supported by Kitware (kitware.com/cmake)

It would be nice to come as well with simpler steps than all this
mkdir build, etc stanza. Perhaps with a wrapper of some kind, in perl
that may be a good idea, though perl is not a hard requirement in
source tarballs, and on Windows/MSVC it is.
Regards,
-- 
Michael

Michael Paquier wrote:
> You have not tested with macOS, and so did I.

Thanks! I fixed this typo. But for XCode I see more corrections.
I will can fix it today or maybe tomorrow.

>It would be nice to come as well with simpler steps than all this
>mkdir build, etc stanza. Perhaps with a wrapper of some kind, in perl
>that may be a good idea, though perl is not a hard requirement in
>source tarballs, and on Windows/MSVC it is.

Only one extra phase (mkdir build). I think it's not so important.
For windows and macosx I used cmake GUI it's so easy.

--
Yury Zhuravlev
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company

On Wed, Jun 29, 2016 at 7:23 PM, Yury Zhuravlev
<u.zhuravlev@postgrespro.ru> wrote:
> Hello Hackers.
>
> I decided to talk about the current state of the project:
> 1. Merge with 9.6 master. 2. plpython2, plpython3, plperl, pltcl, plsql all
> work correctly (all tests pass).
> 3. Works done for all contrib modules. 4. You can use gettext, .po->.mo will
> have converted by CMake.  5. All test pass under some Linux, FreeBSD,
> Solaris10 (on Sparc), Windows MSVC 2015. MacOSX I think not big trouble too.
> 6. Prototype for PGXS (with MSVC support) done.
> I think is very big progress but I came across one problem.
> I not have access to many OS and platforms. For each platform need tests and
> small fixes. I can't develop and give guarantee without it.

can we use our buildfarm infrastructure for this, Andrew ?

>
> I think this is very important work which makes it easier further support
> Postgres but I can not do everything himself. It's physically impossible.
>
> I think without community support I can't do significantly more.
>
> Current version you can get here: https://github.com/stalkerg/postgres_cmake
>
> Thanks!
> --
> Yury Zhuravlev
>
> Postgres Professional: http://www.postgrespro.com
> The Russian Postgres Company
>
>
> --
> Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-hackers

On 06/30/2016 06:26 AM, Oleg Bartunov wrote:
> On Wed, Jun 29, 2016 at 7:23 PM, Yury Zhuravlev
> <u.zhuravlev@postgrespro.ru> wrote:
>> Hello Hackers.
>>
>> I decided to talk about the current state of the project:
>> 1. Merge with 9.6 master. 2. plpython2, plpython3, plperl, pltcl, plsql all
>> work correctly (all tests pass).
>> 3. Works done for all contrib modules. 4. You can use gettext, .po->.mo will
>> have converted by CMake.  5. All test pass under some Linux, FreeBSD,
>> Solaris10 (on Sparc), Windows MSVC 2015. MacOSX I think not big trouble too.
>> 6. Prototype for PGXS (with MSVC support) done.
>> I think is very big progress but I came across one problem.
>> I not have access to many OS and platforms. For each platform need tests and
>> small fixes. I can't develop and give guarantee without it.
> can we use our buildfarm infrastructure for this, Andrew ?


When I get some time I intend to do some work with this to see what 
might be involved in enabling this in the buildfarm. I imagine it will 
probably involve something similar to the effort that moving to git 
entailed.

cheers

andrew

On 06/30/2016 03:13 AM, Yury Zhuravlev wrote:
> Michael Paquier wrote:
>> You have not tested with macOS, and so did I.
>
> Thanks! I fixed this typo. But for XCode I see more corrections. I 
> will can fix it today or maybe tomorrow.
>
>> It would be nice to come as well with simpler steps than all this
>> mkdir build, etc stanza. Perhaps with a wrapper of some kind, in perl
>> that may be a good idea, though perl is not a hard requirement in
>> source tarballs, and on Windows/MSVC it is.
>
> Only one extra phase (mkdir build). I think it's not so important. For 
> windows and macosx I used cmake GUI it's so easy.


We need this to be scriptable, not using a GUI.

cheers

andrew

On Fri, Jul 1, 2016 at 5:19 AM, Andrew Dunstan <andrew@dunslane.net> wrote:
> On 06/30/2016 03:13 AM, Yury Zhuravlev wrote:
>> Only one extra phase (mkdir build). I think it's not so important. For
>> windows and macosx I used cmake GUI it's so easy.
>
> We need this to be scriptable, not using a GUI.

Definitely agreed! Personally I am allergic to any kind of UIs for
development, and I am sure not to be the only one on this list.
-- 
Michael

Michael Paquier wrote:
> Personally I am allergic to any kind of UIs for
> development, and I am sure not to be the only one on this list.

Andrew Dunstan:
>We need this to be scriptable, not using a GUI.

GUI is strong optional feature. Helpful for some tasks.

--
Yury Zhuravlev
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company

Hi!

I think, we need simple configure script generator for backward 
compatibility and easy using this build system.
Try playing with cmake build system under Win2008+MinGW. I plan to write 
perl script for automatic build this
with depends.

On 29.06.2016 19:23, Yury Zhuravlev wrote:
> Hello Hackers.
>
> I decided to talk about the current state of the project:
> 1. Merge with 9.6 master. 2. plpython2, plpython3, plperl, pltcl, 
> plsql all work correctly (all tests pass).
> 3. Works done for all contrib modules. 4. You can use gettext, 
> .po->.mo will have converted by CMake.  5. All test pass under some 
> Linux, FreeBSD, Solaris10 (on Sparc), Windows MSVC 2015. MacOSX I 
> think not big trouble too.  6. Prototype for PGXS (with MSVC support) 
> done.
> I think is very big progress but I came across one problem.
> I not have access to many OS and platforms. For each platform need 
> tests and small fixes. I can't develop and give guarantee without it.
>
> I think this is very important work which makes it easier further 
> support Postgres but I can not do everything himself. It's physically 
> impossible.
>
> I think without community support I can't do significantly more.
>
> Current version you can get here: 
> https://github.com/stalkerg/postgres_cmake
>
> Thanks!

Hi Yury,

I'm interested in helping with your CMake effort.  I don't have any
experience contributing to PG, but I do have some free time at the
moment.  Please let me know if I can help.

I have an intermediate level of experience with CMake, Python, and
Bash scripting.  My native environment is Mint 17.3 (i.e., Ubuntu
15.10), but I'd be happy to create additional VM's as needed.  I'm
also happy to look into bugs on other systems (VMS, etc.) if I can get
SSH access.

Kind regards,
Christian Convey


On Fri, Jul 1, 2016 at 5:26 AM, Dmitry Maslyuk <d.masluk@postgrespro.ru> wrote:
> Hi!
>
> I think, we need simple configure script generator for backward
> compatibility and easy using this build system.
> Try playing with cmake build system under Win2008+MinGW. I plan to write
> perl script for automatic build this
> with depends.
>
> On 29.06.2016 19:23, Yury Zhuravlev wrote:
>>
>> Hello Hackers.
>>
>> I decided to talk about the current state of the project:
>> 1. Merge with 9.6 master. 2. plpython2, plpython3, plperl, pltcl, plsql
>> all work correctly (all tests pass).
>> 3. Works done for all contrib modules. 4. You can use gettext, .po->.mo
>> will have converted by CMake.  5. All test pass under some Linux, FreeBSD,
>> Solaris10 (on Sparc), Windows MSVC 2015. MacOSX I think not big trouble too.
>> 6. Prototype for PGXS (with MSVC support) done.
>> I think is very big progress but I came across one problem.
>> I not have access to many OS and platforms. For each platform need tests
>> and small fixes. I can't develop and give guarantee without it.
>>
>> I think this is very important work which makes it easier further support
>> Postgres but I can not do everything himself. It's physically impossible.
>>
>> I think without community support I can't do significantly more.
>>
>> Current version you can get here:
>> https://github.com/stalkerg/postgres_cmake
>>
>> Thanks!
>
>
>
>
> --
> Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-hackers

On 06/29/2016 06:23 PM, Yury Zhuravlev wrote:
> Hello Hackers.
> 
> I decided to talk about the current state of the project:
> 1. Merge with 9.6 master. 2. plpython2, plpython3, plperl, pltcl, plsql
> all work correctly (all tests pass).
> 3. Works done for all contrib modules. 4. You can use gettext, .po->.mo
> will have converted by CMake.  5. All test pass under some Linux,
> FreeBSD, Solaris10 (on Sparc), Windows MSVC 2015. MacOSX I think not big
> trouble too.  6. Prototype for PGXS (with MSVC support) done.
> I think is very big progress but I came across one problem.
> I not have access to many OS and platforms. For each platform need tests
> and small fixes. I can't develop and give guarantee without it.
> 
> I think this is very important work which makes it easier further
> support Postgres but I can not do everything himself. It's physically
> impossible.
> 
> I think without community support I can't do significantly more.
> 
> Current version you can get here:
> https://github.com/stalkerg/postgres_cmake

hmm how do you actually want reports on how it works?

I just played with it on spoonbill (OpenBSD 5.3/sparc64) and got this:

CMake Error at CMakeLists.txt:1250 (file): file does not recognize sub-command GENERATE


CMake Error at src/port/CMakeLists.txt:156 (target_compile_definitions): Unknown CMake command
"target_compile_definitions".


-- Configuring incomplete, errors occurred!


there is also a ton of stuff like:


Make Error: Internal CMake error, TryCompile generation of cmake failed
-- Looking for opterr - not found
-- Looking for optreset
CMake Error at CMakeLists.txt:10 (ADD_EXECUTABLE): Target "cmTryCompileExec3458204847" links to item " m" which has
leading or trailing whitespace.  This is now an error according to policy CMP0004.


CMake Error: Internal CMake error, TryCompile generation of cmake failed
-- Looking for optreset - not found
-- Looking for fseeko
CMake Error at CMakeLists.txt:10 (ADD_EXECUTABLE): Target "cmTryCompileExec2628321539" links to item " m" which has
leading or trailing whitespace.  This is now an error according to policy CMP0004.


CMake Error: Internal CMake error, TryCompile generation of cmake failed


which I have no idea whether they are an actual problem or just
"configure" noise





Stefan

Hi Stefan,

I think I've seen similar errors when a project's CMake files assumed
a newer version of CMake than the one being run.

Which version of CMake gave you those errors?  (Sorry if you provided
that detail and I'm just missing it.)

Kind regards,
Christian

On Thu, Aug 18, 2016 at 2:45 PM, Stefan Kaltenbrunner
<stefan@kaltenbrunner.cc> wrote:
> On 06/29/2016 06:23 PM, Yury Zhuravlev wrote:
>> Hello Hackers.
>>
>> I decided to talk about the current state of the project:
>> 1. Merge with 9.6 master. 2. plpython2, plpython3, plperl, pltcl, plsql
>> all work correctly (all tests pass).
>> 3. Works done for all contrib modules. 4. You can use gettext, .po->.mo
>> will have converted by CMake.  5. All test pass under some Linux,
>> FreeBSD, Solaris10 (on Sparc), Windows MSVC 2015. MacOSX I think not big
>> trouble too.  6. Prototype for PGXS (with MSVC support) done.
>> I think is very big progress but I came across one problem.
>> I not have access to many OS and platforms. For each platform need tests
>> and small fixes. I can't develop and give guarantee without it.
>>
>> I think this is very important work which makes it easier further
>> support Postgres but I can not do everything himself. It's physically
>> impossible.
>>
>> I think without community support I can't do significantly more.
>>
>> Current version you can get here:
>> https://github.com/stalkerg/postgres_cmake
>
> hmm how do you actually want reports on how it works?
>
> I just played with it on spoonbill (OpenBSD 5.3/sparc64) and got this:
>
> CMake Error at CMakeLists.txt:1250 (file):
>   file does not recognize sub-command GENERATE
>
>
> CMake Error at src/port/CMakeLists.txt:156 (target_compile_definitions):
>   Unknown CMake command "target_compile_definitions".
>
>
> -- Configuring incomplete, errors occurred!
>
>
> there is also a ton of stuff like:
>
>
> Make Error: Internal CMake error, TryCompile generation of cmake failed
> -- Looking for opterr - not found
> -- Looking for optreset
> CMake Error at CMakeLists.txt:10 (ADD_EXECUTABLE):
>   Target "cmTryCompileExec3458204847" links to item " m" which has
> leading or
>   trailing whitespace.  This is now an error according to policy CMP0004.
>
>
> CMake Error: Internal CMake error, TryCompile generation of cmake failed
> -- Looking for optreset - not found
> -- Looking for fseeko
> CMake Error at CMakeLists.txt:10 (ADD_EXECUTABLE):
>   Target "cmTryCompileExec2628321539" links to item " m" which has
> leading or
>   trailing whitespace.  This is now an error according to policy CMP0004.
>
>
> CMake Error: Internal CMake error, TryCompile generation of cmake failed
>
>
> which I have no idea whether they are an actual problem or just
> "configure" noise
>
>
>
>
>
> Stefan
>
>
> --
> Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-hackers

On 08/18/2016 08:57 PM, Christian Convey wrote:
> Hi Stefan,
> 
> I think I've seen similar errors when a project's CMake files assumed
> a newer version of CMake than the one being run.
> 
> Which version of CMake gave you those errors?  (Sorry if you provided
> that detail and I'm just missing it.)


% cmake --version
cmake version 2.8.10.2

a quick look in the docs does not seem to reveal any kind of "minimum"
cmake version required - and the above is what is packaged as part of
openbsd 5.3 (which is outdated and unsupported upstream but it is
currently perfectly fine building all postgresql versions including
-HEAD and serving as a buildfarm member for years)



Stefan

Stefan Kaltenbrunner <stefan@kaltenbrunner.cc> writes:
> On 08/18/2016 08:57 PM, Christian Convey wrote:
>> Which version of CMake gave you those errors?  (Sorry if you provided
>> that detail and I'm just missing it.)

> % cmake --version
> cmake version 2.8.10.2

> a quick look in the docs does not seem to reveal any kind of "minimum"
> cmake version required - and the above is what is packaged as part of
> openbsd 5.3 (which is outdated and unsupported upstream but it is
> currently perfectly fine building all postgresql versions including
> -HEAD and serving as a buildfarm member for years)

As an additional comment, I don't see us accepting a build system
that doesn't run on pretty old cmakes.  We have never had a policy
of "latest and greatest is required" for *any* build support tool,
and cmake isn't likely to be given an exception.

BTW, I just noticed that cmake doesn't seem to be supplied as part of
Apple's dev tools, at least not up to current (El Capitan) releases.
That's going to be a rather large minus to be taken into account
whenever we make the go/no-go decision on this.
        regards, tom lane

Hi Karl,

I'll need to let Yury answer your original question regarding the best
way to report CMake-related bugs.

Regarding the errors you're getting...  I just looked at CMake's
online documentation regarding your "target_compile_definitions"
error.

From what I can tell, the "target_compile_definition" property was
introduced in CMake 2.8.11.  It sounds like your version of CMake is
just a little too old.

Regarding how one can know the required CMake version: My modus
operandi for CMake projects in general is (1) read the project's
how-to-build docs, and if that's not heplful, (2) hope that the
project's CMake files contain a "cmake_minimum_required" call to give
me a clear error message.  I didn't find any such indication in Yuri's
files, although perhaps I missed it.


Yury: Would it make sense to add a call to "cmake_minimum_required" in
one or more of your CMakeLists.txt files?

Kind regards,
Christian

On Thu, Aug 18, 2016 at 3:08 PM, Stefan Kaltenbrunner
<stefan@kaltenbrunner.cc> wrote:
> On 08/18/2016 08:57 PM, Christian Convey wrote:
>> Hi Stefan,
>>
>> I think I've seen similar errors when a project's CMake files assumed
>> a newer version of CMake than the one being run.
>>
>> Which version of CMake gave you those errors?  (Sorry if you provided
>> that detail and I'm just missing it.)
>
>
> % cmake --version
> cmake version 2.8.10.2
>
> a quick look in the docs does not seem to reveal any kind of "minimum"
> cmake version required - and the above is what is packaged as part of
> openbsd 5.3 (which is outdated and unsupported upstream but it is
> currently perfectly fine building all postgresql versions including
> -HEAD and serving as a buildfarm member for years)
>
>
>
> Stefan

On 08/18/2016 09:30 PM, Christian Convey wrote:
> Hi Karl,
> 
> I'll need to let Yury answer your original question regarding the best
> way to report CMake-related bugs.
> 
> Regarding the errors you're getting...  I just looked at CMake's
> online documentation regarding your "target_compile_definitions"
> error.
> 
> From what I can tell, the "target_compile_definition" property was
> introduced in CMake 2.8.11.  It sounds like your version of CMake is
> just a little too old.

Well - "too old" is a relative term - cmake 2.8.10 was released in only
october 2012 and cmake 2.8.11 in may 2013 so it is not even 4 years old,
the oldest currently supported (though for not much longer) postgresql
release 9.1 was released in september 2011 and 9.2 was also released
before october 2012.
So while Cmake compat might only make it for v10, I dont think that we
can depend on bleeding edge version like that for our buildtools...


> 
> Regarding how one can know the required CMake version: My modus
> operandi for CMake projects in general is (1) read the project's
> how-to-build docs, and if that's not heplful, (2) hope that the
> project's CMake files contain a "cmake_minimum_required" call to give
> me a clear error message.  I didn't find any such indication in Yuri's
> files, although perhaps I missed it.
> 
> 
> Yury: Would it make sense to add a call to "cmake_minimum_required" in
> one or more of your CMakeLists.txt files?

it would make sense nevertheless but I dont think that 2.8.11 is old
enough - looking at the release information and the feature compatibily
matrix it would seems we should more aim at something like 2.8.0 or 2.8.3...


Stefan

Hi Tom,

Thanks for that information.

Is there some document I can read that explains which platform
versions (e.g., OpenBSD 5.3) are considered strongly supported?

I ask because I'm curious if/how someone in Yury's situation could
predict which minimum version of CMake must be supported in order for
his patch to be accepted.  (And if he accepts my offer to pitch in,
I'll actually need that particular detail.)

Kind regards,
Christian


> As an additional comment, I don't see us accepting a build system
> that doesn't run on pretty old cmakes.  We have never had a policy
> of "latest and greatest is required" for *any* build support tool,
> and cmake isn't likely to be given an exception.
>
> BTW, I just noticed that cmake doesn't seem to be supplied as part of
> Apple's dev tools, at least not up to current (El Capitan) releases.
> That's going to be a rather large minus to be taken into account
> whenever we make the go/no-go decision on this.
>
>                         regards, tom lane

Stefan Kaltenbrunner wrote:
> On 08/18/2016 09:30 PM, Christian Convey wrote:

> > Yury: Would it make sense to add a call to "cmake_minimum_required" in
> > one or more of your CMakeLists.txt files?
> 
> it would make sense nevertheless but I dont think that 2.8.11 is old
> enough - looking at the release information and the feature compatibily
> matrix it would seems we should more aim at something like 2.8.0 or 2.8.3...

Last year I checked versions installable in Debian:
https://www.postgresql.org/message-id/20150829213631.GI2912@alvherre.pgsql
From that I would say that the maximum minimum is 2.8.2.  Not sure if
there's any platform where 2.8.0 (released in 2009) or older would be
necessary.

For the record, IIRC the oldest Perl we support is something like 5.8.3
(released in 2004, bugfix release after 5.8.0 released in 2002)

-- 
Álvaro Herrera                http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

On 08/18/2016 09:42 PM, Christian Convey wrote:
> Hi Tom,
> 
> Thanks for that information.
> 
> Is there some document I can read that explains which platform
> versions (e.g., OpenBSD 5.3) are considered strongly supported?

well not sure we have very clear document on that - I would say that the
buildfarm is the most authoritative answer to that. So I think skimming
the buildfarm for the oldest and strangest platforms would be a good start.

> 
> I ask because I'm curious if/how someone in Yury's situation could
> predict which minimum version of CMake must be supported in order for
> his patch to be accepted.  (And if he accepts my offer to pitch in,
> I'll actually need that particular detail.)

well I personally think the level to meet would be that all the systems
on the buildfarm that can build -HEAD at the time the patch is proposed
for a commit should be able to build using the new system with whatever
cmake version is available in those by default (if it is at all).


Stefan

Hi Stefan,

>> Yury: Would it make sense to add a call to "cmake_minimum_required" in
>> one or more of your CMakeLists.txt files?
>
> it would make sense nevertheless but I dont think that 2.8.11 is old
> enough - looking at the release information and the feature compatibily
> matrix it would seems we should more aim at something like 2.8.0 or 2.8.3...

I'm new to PG development, so I don't know what ideas the community is
open to.  But I wonder if there's any merit to the following
approach...

* Allow the CMake-based build system to assume a fairly modern version
of CMake.  (Maybe 2.8.12, or 3.0.)

* For systems where the minimum CMake version isn't readily available,
have an alternative build system which is just a simplistic Bash
script that naively performs a full build every time it's invoked.
The idea being that PG contributors are mostly the people who want
efficient rebuilds, and most/all of them could easily install that
minimal CMake version.

*IF* it proved possible to write a clear, maintainable Bash script for
that, perhaps that would eliminate most concerns about CMake not being
well-supported on uncommon platforms / platform versions.

- Christian

On 08/18/2016 09:52 PM, Alvaro Herrera wrote:
> Stefan Kaltenbrunner wrote:
>> On 08/18/2016 09:30 PM, Christian Convey wrote:
> 
>>> Yury: Would it make sense to add a call to "cmake_minimum_required" in
>>> one or more of your CMakeLists.txt files?
>>
>> it would make sense nevertheless but I dont think that 2.8.11 is old
>> enough - looking at the release information and the feature compatibily
>> matrix it would seems we should more aim at something like 2.8.0 or 2.8.3...
> 
> Last year I checked versions installable in Debian:
> https://www.postgresql.org/message-id/20150829213631.GI2912@alvherre.pgsql
> From that I would say that the maximum minimum is 2.8.2.  Not sure if
> there's any platform where 2.8.0 (released in 2009) or older would be
> necessary.

well we have for example a NetBSD 5.1 boxe (coypu) on the buildfarm that
have a software stack that is basically 2008/2009ish...
So 2.8.0-2.8.3 seems like a realistic target to me still



Stefan

Hi Tom,

>> I ask because I'm curious if/how someone in Yury's situation could
>> predict which minimum version of CMake must be supported in order for
>> his patch to be accepted.  (And if he accepts my offer to pitch in,
>> I'll actually need that particular detail.)
>
> well I personally think the level to meet would be that all the systems
> on the buildfarm that can build -HEAD at the time the patch is proposed
> for a commit should be able to build using the new system with whatever
> cmake version is available in those by default (if it is at all).

I see.  In other projects I've worked on, the configuration of a build
farm has been driven by some list of platforms that were considered
important to support.

Is that the case here as well?  I.e., is the build-farm population
just a convenient proxy for some other source of information regarding
what platforms are important?

Apologies if my questions are so basic that I can find the answers
elsewhere.  I'll happily follow any RTFM links.

Thanks again,
Christian

Christian Convey wrote:

> * Allow the CMake-based build system to assume a fairly modern version
> of CMake.  (Maybe 2.8.12, or 3.0.)
> 
> * For systems where the minimum CMake version isn't readily available,
> have an alternative build system which is just a simplistic Bash
> script that naively performs a full build every time it's invoked.
> The idea being that PG contributors are mostly the people who want
> efficient rebuilds, and most/all of them could easily install that
> minimal CMake version.
> 
> *IF* it proved possible to write a clear, maintainable Bash script for
> that, perhaps that would eliminate most concerns about CMake not being
> well-supported on uncommon platforms / platform versions.

I think this "simplistic Bash script" is called "configure" and is
generated by autoconf from our configure.in.  I don't think the word
"simplistic" describes it very well (nor does "maintainable").

I don't think we're interested in maintaining more build systems than we
already are.  If cmake can replace the current MSVC tooling and
autoconf, all in one, my impression is that we're in.  If we're
replacing two tools we've hammered pretty well over the years with two
tools that we haven't, I doubt we're interested.

-- 
Álvaro Herrera                http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

>
> I don't think we're interested in maintaining more build systems than we
> already are.  If cmake can replace the current MSVC tooling and
> autoconf, all in one, my impression is that we're in.  If we're
> replacing two tools we've hammered pretty well over the years with two
> tools that we haven't, I doubt we're interested.

Good to know, thanks.

- Christian

Christian Convey wrote:

> I see.  In other projects I've worked on, the configuration of a build
> farm has been driven by some list of platforms that were considered
> important to support.

Build farm members are systems that somebody has seen as interesting
enough that they deserve enough effort to set up a buildfarm member for.
So the buildfarm roster *is* that list.

-- 
Álvaro Herrera                http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

On 2016-08-18 15:55:20 -0400, Christian Convey wrote:
> * Allow the CMake-based build system to assume a fairly modern version
> of CMake.  (Maybe 2.8.12, or 3.0.)
> 
> * For systems where the minimum CMake version isn't readily available,
> have an alternative build system which is just a simplistic Bash
> script that naively performs a full build every time it's invoked.
> The idea being that PG contributors are mostly the people who want
> efficient rebuilds, and most/all of them could easily install that
> minimal CMake version.

The benefit cmake brings to the table, from my pov, is that it allows to
get rid of somewhat a parallel buildsystem (msvc / windows, which
sources most of its information from the makefiles). If we continue to
have two, especially if they're entirely separate, I see little benefit
in this whole endeavor.

Hi Stefan,

>> I ask because I'm curious if/how someone in Yury's situation could
>> predict which minimum version of CMake must be supported in order for
>> his patch to be accepted.  (And if he accepts my offer to pitch in,
>> I'll actually need that particular detail.)
>
> well I personally think the level to meet would be that all the systems
> on the buildfarm that can build -HEAD at the time the patch is proposed
> for a commit should be able to build using the new system with whatever
> cmake version is available in those by default (if it is at all).

What standard would you suggest for those platforms which don't have
an obvious default version of CMake?

I assume this includes all versions of Windows and of OS X, but
perhaps I'm misinformed.

Thanks,
Christian

Christian Convey <christian.convey@gmail.com> writes:
>> well I personally think the level to meet would be that all the systems
>> on the buildfarm that can build -HEAD at the time the patch is proposed
>> for a commit should be able to build using the new system with whatever
>> cmake version is available in those by default (if it is at all).

> I see.  In other projects I've worked on, the configuration of a build
> farm has been driven by some list of platforms that were considered
> important to support.

> Is that the case here as well?  I.e., is the build-farm population
> just a convenient proxy for some other source of information regarding
> what platforms are important?

To a large extent, the buildfarm population is interesting because those
machines represent platforms for which someone is willing to put their
money where their mouth is, ie go to the effort of maintaining a buildfarm
member.  For those cases, we're likely to hear complaints if we break it.

There are also buildfarm members that are really only there as coal mine
canaries, that is, we don't want to break compatibility by accident ---
but if there were a good reason for it, we might be willing to throw those
platforms overboard.  My HPPA/HPUX dinosaur is certainly in that category,
for example.

I do not think we have a project policy about where the dividing line
is, though.

BTW, Stefan's formulation supposes that on platforms where the vendor
didn't supply any version of cmake, we can tell people to install whatever
version we want.  But that's assuming something not in evidence, namely
that cmake works on those platforms at all, in any version.  We'll have
a lot of legwork to do to find out what platforms we are risking losing.
        regards, tom lane

On 8/18/16 4:23 PM, Christian Convey wrote:
> What standard would you suggest for those platforms which don't have
> an obvious default version of CMake?

In the olden days, when many platforms we supported didn't come with GNU
make or other GNU tools or even a compiler, we collected a lot of
practical information about where to get these things.  You can see most
of that at
<https://www.postgresql.org/docs/devel/static/installation-platform-notes.html>.

That's at least the spirit of things.  I wouldn't worry about this so
much right now.  Get it working first.

-- 
Peter Eisentraut              http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

On Fri, Aug 19, 2016 at 4:25 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> BTW, I just noticed that cmake doesn't seem to be supplied as part of
> Apple's dev tools, at least not up to current (El Capitan) releases.
> That's going to be a rather large minus to be taken into account
> whenever we make the go/no-go decision on this.

Indeed. Now in order to test the patch on macos, one can depend on
brew's cmake. That's what I am doing.

I recall that a couple of releases before El Capitan (10.11), Lion and
Mountain Lion had a server edition that bundled Postgres natively.
Does this still exist for El Capitan? If yes, moving to cmake may
actually give an argument to those folks to begin to touch cmake and
integrate it natively... But as Apple is a fortress of secrecy that's
hard to tell.
-- 
Michael

Stefan Kaltenbrunner wrote:
> hmm how do you actually want reports on how it works?
>
> I just played with it on spoonbill (OpenBSD 5.3/sparc64) and got this:
>
> CMake Error at CMakeLists.txt:1250 (file):
>   file does not recognize sub-command GENERATE
>
>
> CMake Error at src/port/CMakeLists.txt:156 (target_compile_definitions):
>   Unknown CMake command "target_compile_definitions".
>
>
> -- Configuring incomplete, errors occurred!
>
>
> there is also a ton of stuff like:
>
>
> Make Error: Internal CMake error, TryCompile generation of cmake failed
> -- Looking for opterr - not found
> -- Looking for optreset
> CMake Error at CMakeLists.txt:10 (ADD_EXECUTABLE):
>   Target "cmTryCompileExec3458204847" links to item " m" which has
> leading or
>   trailing whitespace.  This is now an error according to policy CMP0004.
>
>
> CMake Error: Internal CMake error, TryCompile generation of cmake failed
> -- Looking for optreset - not found
> -- Looking for fseeko
> CMake Error at CMakeLists.txt:10 (ADD_EXECUTABLE):
>   Target "cmTryCompileExec2628321539" links to item " m" which has
> leading or
>   trailing whitespace.  This is now an error according to policy CMP0004.
>
>
> CMake Error: Internal CMake error, TryCompile generation of cmake failed
>
>
> which I have no idea whether they are an actual problem or just
> "configure" noise

Can you send issue into github?
https://github.com/stalkerg/postgres_cmake/issues

--
Yury Zhuravlev
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company

Christian Convey wrote:
> I'm interested in helping with your CMake effort.  I don't have any
> experience contributing to PG, but I do have some free time at the
> moment.  Please let me know if I can help.
I glad to hear it. I suppose you can just try build postgres and send all
problems to github tracker.
https://github.com/stalkerg/postgres_cmake/issues

Big thanks!

--
Yury Zhuravlev
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company

Stefan Kaltenbrunner wrote:
> On 08/18/2016 09:30 PM, Christian Convey wrote:
>> Hi Karl,
>>
>> I'll need to let Yury answer your original question regarding the best
>> way to report CMake-related bugs.
>>
>> Regarding the errors you're getting...  I just looked at CMake's
>> online documentation regarding your "target_compile_definitions" ...
>
> Well - "too old" is a relative term - cmake 2.8.10 was released in only
> october 2012 and cmake 2.8.11 in may 2013 so it is not even 4 years old,
> the oldest currently supported (though for not much longer) postgresql
> release 9.1 was released in september 2011 and 9.2 was also released
> before october 2012.
> So while Cmake compat might only make it for v10, I dont think that we
> can depend on bleeding edge version like that for our buildtools...

1. I don't know but nobody objections if we install latest CMake under
Windows or MacOSX but many if under Linux/*BSD. I have builded CMake
recently under old Solaris without any problems.

2. I use "graceful degradation" method and currently target is 3.0 version
BUT time to time I testing and backporting the code. I hope I can make all
features for 2.8.4 .

3. Please do not compare cmake and GNU Make. CMake it's like Autotools with
detect library scripts out the box. Every new version brings not only new
functions and syntax construction. Most fix in find_library scripts.
Besides you can't build project without gnu make, make, nmake, ninja and
etc because CMake generate files for another make systems. On this basis,
to CMake can not meet the same requirements for version as GNU Make. It is
quite another thing.

PS Special for Postgres in CMake 3.6 I implemented features for replace
gen_def.pl script under MSVC.
https://gitlab.kitware.com/cmake/cmake/merge_requests/29

--
Yury Zhuravlev
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company

Andres Freund wrote:
> The benefit cmake brings to the table, from my pov, is that it allows to
> get rid of somewhat a parallel buildsystem (msvc / windows, which
> sources most of its information from the makefiles). If we continue to
> have two, especially if they're entirely separate, I see little benefit
> in this whole endeavor.

I fully agree with this. Although I still leave the probability that for a
while (couple of months) CMake will coexist with autoconf solely for
testing and verification.

--
Yury Zhuravlev
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company

Stefan Kaltenbrunner wrote:
> well we have for example a NetBSD 5.1 boxe (coypu) on the buildfarm that
> have a software stack that is basically 2008/2009ish...
> So 2.8.0-2.8.3 seems like a realistic target to me still
>

You can install fresh CMake to NetBSD without big problems from source.

--
Yury Zhuravlev
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company

Hi Yury,


On Fri, Aug 19, 2016 at 9:46 AM, Yury Zhuravlev
<u.zhuravlev@postgrespro.ru> wrote:
> Christian Convey wrote:
>>
>> I'm interested in helping with your CMake effort.  I don't have any
>> experience contributing to PG, but I do have some free time at the
>> moment.  Please let me know if I can help.
>
> I glad to hear it. I suppose you can just try build postgres and send all
> problems to github tracker.
> https://github.com/stalkerg/postgres_cmake/issues

Thanks, I'll be happy to do that.  There's been a lot of discussion on
this thread regarding the minimum required CMake version.  If you'd
like me to test with a particular version (or versions) of CMake,
please let me know.

- Christian

Hi Yury,

>> I glad to hear it. I suppose you can just try build postgres and send all
>> problems to github tracker.
>> https://github.com/stalkerg/postgres_cmake/issues

FYI, I had success using your "postgres_cmake" repo.  I tested it up
through "make check" and "make install".

Here are the details:

* postgres_cmake commit e7e75160d4533cd8caa9f3f0dd7b485dbd4e7bdf
* compiler = cc (Ubuntu 4.8.4-2ubuntu1~14.04.3) 4.8.4
* cmake version = 3.5.3

Kind regards,
Christian

Yury,

On Mon, Aug 22, 2016 at 7:48 AM, Christian Convey
<christian.convey@gmail.com> wrote:
>>> I glad to hear it. I suppose you can just try build postgres and send all
>>> problems to github tracker.
>>> https://github.com/stalkerg/postgres_cmake/issues
>
> FYI, I had success using your "postgres_cmake" repo.  I tested it up
> through "make check" and "make install".
>
> Here are the details:
>
> * postgres_cmake commit e7e75160d4533cd8caa9f3f0dd7b485dbd4e7bdf
> * compiler = cc (Ubuntu 4.8.4-2ubuntu1~14.04.3) 4.8.4
> * cmake version = 3.5.3

Could it be possible to get a refreshed patch on this thread for at
least the sake of the archives? I'd really like to see somehting
happening here and do some progress for this CF.
-- 
Michael

Michael Paquier wrote:
> Could it be possible to get a refreshed patch on this thread for at
> least the sake of the archives? I'd really like to see somehting
> happening here and do some progress for this CF.

Sure, I will do it on Friday.
Today I finished mingw+msys support. (mingw without msys has not passed
some tests)
--
Yury Zhuravlev
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company

On Fri, Sep 16, 2016 at 4:38 AM, Yury Zhuravlev
<u.zhuravlev@postgrespro.ru> wrote:
> Michael Paquier wrote:
>>
>> Could it be possible to get a refreshed patch on this thread for at
>> least the sake of the archives? I'd really like to see somehting
>> happening here and do some progress for this CF.
>
>
> Sure, I will do it on Friday.
> Today I finished mingw+msys support. (mingw without msys has not passed some
> tests)

Okay. That sounds good to me. I don't recall what your patch is
exactly doing but could you still keep the vanilla Makefiles around?
This will reduce the diff of the patch, and we'd need anyway to keep
the former Makefile methods around until the buildfarm scripts are
updated, the animals do the switch and then get green. So a period
where both live in parallel is unavoidable.

I heard as well that MySQL is using cmake... It may be interesting to
see how they are integrating with it as a large project and avoid
their past mistakes.
-- 
Michael

On Fri, Sep 16, 2016 at 9:58 AM, Michael Paquier
<michael.paquier@gmail.com> wrote:
> Okay. That sounds good to me. I donas well a look around, and cmake really looks like a robust alternative to
./configure.Now I am aware of the fact that your patch recommends to build 't recall what your patch is
 
> exactly doing but could you still keep the vanilla Makefiles around?
> This will reduce the diff of the patch, and we'd need anyway to keep
> the former Makefile methods around until the buildfarm scripts are
> updated, the animals do the switch and then get green. So a period
> where both live in parallel is unavoidable.
>
> I heard as well that MySQL is using cmake... It may be interesting to
> see how they are integrating with it as a large project and avoid
> their past mistakes.

I could not resist so I just had a look at your patch. I had as well a
look around, and cmake really looks like a robust alternative to
./configure. In short, people doing now that:
./configure
make
make install
Would just do that:
cmake .
make
make install

I am no cmake guru yet, but VPATH builds are supported out of the box,
which is cool.

Your patch recommends to build with cmake after creating build/, now I
would expect most users to run cmake from the root folder. However
this causes all the Makefiles to get overwritten. As supporting all
platforms at once with cmake is going to be uncommitable, we are going
to need both methods able to live together for a while. Well, they can
coexist with this patch as long as cmake is not run from the root of
the code tree, which is acceptable for me as long as the switch is not
completed. However others may think differently on the matter.

Instead of getting support for all existing platforms, I would
recommend as well focusing only on one platform for the time being,
Linux, and get the work done correctly for that first. Once there is
something committed, we will be able to patch the buildfarm, and get
machines to switch to cmake one by one. After those are switched, we
could extend that. Another point of contention is support for
extensions. How long should we keep support for the existing PGXS? How
external extensions would compile with the new thing infrastructure?

Which brings me to another point, your patch is focused on features,
meaning that per-OS checks are all grouped by feature, but it may be a
good idea to split checks by OS if necessary, with for example
per-platform files and scripts in cmake/os/. And we could have just
something for Linux now.

A couple of issues I have noticed with your patch after a first set of tests:
- root's .gitignore needs to add entries for CMakeFiles and
cmake_install.cmake. You need as well to ignore CMakeCache
- A couple of headers are generated, like cubeparse.h (?)
- Currently a lot of users do things like that:
cd src/test/regress/ && make check
But this patch breaks that, and that's not cool. Recovery tests in
src/test/regress won't run either.

That's all I have for now. Looking forward to seeing some progress here.
-- 
Michael

Michael Paquier wrote:
> Your patch recommends to build with cmake after creating build/, now I
> would expect most users to run cmake from the root folder. However
> this causes all the Makefiles to get overwritten. As supporting all
> platforms at once with cmake is going to be uncommitable, we are going
> to need both methods able to live together for a while. Well, they can
> coexist with this patch as long as cmake is not run from the root of
> the code tree, which is acceptable for me as long as the switch is not
> completed. However others may think differently on the matter.

It's really a good point. Forbidden run cmake from root it is better
decision for me (of course for start).

>Instead of getting support for all existing platforms, I would
>recommend as well focusing only on one platform for the time being,
>Linux, and get the work done correctly for that first. Once there is
>something committed, we will be able to patch the buildfarm, and get
>machines to switch to cmake one by one. After those are switched, we
>could extend that.

You mean in first version of patch I can focus on Linux systems?

>Another point of contention is support for
>extensions. How long should we keep support for the existing PGXS? How
>external extensions would compile with the new thing infrastructure?

As long as possible. I hope I can make PGXS Makefiles generator.

>Which brings me to another point, your patch is focused on features,
>meaning that per-OS checks are all grouped by feature, but it may be a
>good idea to split checks by OS if necessary, with for example
>per-platform files and scripts in cmake/os/. And we could have just
>something for Linux now.

Currently I do not have a lot OS specific tests. All checks are doing in
same manner.

>- root's .gitignore needs to add entries for CMakeFiles and
>cmake_install.cmake. You need as well to ignore CMakeCache

Thanks I done this in last commit.

>- A couple of headers are generated, like cubeparse.h (?)

Because BISON generate header by default. I suppose author of cube launched
bison by hand but I made it automatic.

>- Currently a lot of users do things like that:
>cd src/test/regress/ && make check
>But this patch breaks that, and that's not cool. Recovery tests in
>src/test/regress won't run either.

It seems restriction by design because in CMake you have only one enter
point.

>That's all I have for now. Looking forward to seeing some progress here.

I merged master to my branch and I spent time to porting all changes. I
hope send patch in the weekend without terrible flaws.

--
Yury Zhuravlev
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company

On Sat, Sep 17, 2016 at 1:40 AM, Yury Zhuravlev
<u.zhuravlev@postgrespro.ru> wrote:
> Michael Paquier wrote:
> I merged master to my branch and I spent time to porting all changes. I hope
> send patch in the weekend without terrible flaws.

By the way, I noticed that you did not register this patch in the current CF..
-- 
Michael

Hi,

On 2016-09-16 22:33:50 +0900, Michael Paquier wrote:
> As supporting all platforms at once with cmake is going to be
> uncommitable, we are going to need both methods able to live together
> for a while.

I very strongly disagree with this.  It's way too likely that we end up
releasing with multiple buildsystems that way.  I think we'll have to
require support for the most common OSs, and then fix up the fallout
afterwards.

Andres

On Sat, Sep 17, 2016 at 8:11 AM, Andres Freund <andres@anarazel.de> wrote:
> Hi,
>
> On 2016-09-16 22:33:50 +0900, Michael Paquier wrote:
>> As supporting all platforms at once with cmake is going to be
>> uncommitable, we are going to need both methods able to live together
>> for a while.
>
> I very strongly disagree with this.  It's way too likely that we end up
> releasing with multiple buildsystems that way.  I think we'll have to
> require support for the most common OSs, and then fix up the fallout
> afterwards.

Ok, then if we go through the hard method the switch patch had better
remove as well all the Makefiles in the tree and recommend cmake run
from base root as the default build method. By the way, there will be
a strong need for docs in the patch sooner or later...
-- 
Michael

Michael Paquier wrote:
> On Sat, Sep 17, 2016 at 1:40 AM, Yury Zhuravlev
> <u.zhuravlev@postgrespro.ru> wrote:
>> Michael Paquier wrote:
>> I merged master to my branch and I spent time to porting all
>> changes. I hope
>> send patch in the weekend without terrible flaws.
>
> By the way, I noticed that you did not register this patch in
> the current CF..

Now, I published the first version of the patch.
This patch not ready for commit yet and all current task you can read here:
https://github.com/stalkerg/postgres_cmake/issues

I hope we realy close to end.

In this patch I forbade in-source build for avoid overwriting current
Makefiles.
We will can remove all Makefiles only after shall see in CMake. You don't
need support two system. During commitfests CMake build system will be
supported by me.

I need help with buildfarm because my knowledge of Perl is very bad
(thought about rewrite buildfarm to Python).


I hope for your support.

--
Yury Zhuravlev
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company

Andres Freund wrote:
> It's way too likely that we end up
> releasing with multiple buildsystems that way.

If we fail in the CMake integration we can easily remove all CMake files
before code freeze.
I hope step by step CMake integration better way for current situation.
But I do not insist, just I trying to find the easiest way.

--
Yury Zhuravlev
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company

On 9/16/16 9:33 AM, Michael Paquier wrote:
> I am no cmake guru yet, but VPATH builds are supported out of the box,
> which is cool.
> 
> Your patch recommends to build with cmake after creating build/, now I
> would expect most users to run cmake from the root folder.

My understanding is that cmake strongly recommends building in a
separate directory, which is usually a subdirectory named something like
"build".  So the above is likely going to be the standard workflow.

-- 
Peter Eisentraut              http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

> On 17 Sep 2016, at 20:21, Yury Zhuravlev <u.zhuravlev@postgrespro.ru> wrote:
>
> Michael Paquier wrote:
>> On Sat, Sep 17, 2016 at 1:40 AM, Yury Zhuravlev
>> <u.zhuravlev@postgrespro.ru> wrote:
>>> Michael Paquier wrote:
>>> I merged master to my branch and I spent time to porting all changes. I hope
>>> send patch in the weekend without terrible flaws.
>>
>> By the way, I noticed that you did not register this patch in the current CF..
>
> Now, I published the first version of the patch. This patch not ready for commit yet and all current task you can
readhere: 
> https://github.com/stalkerg/postgres_cmake/issues
>
> I hope we realy close to end.
> In this patch I forbade in-source build for avoid overwriting current Makefiles. We will can remove all Makefiles
onlyafter shall see in CMake. You don't need support two system. During commitfests CMake build system will be
supportedby me.   
> I need help with buildfarm because my knowledge of Perl is very bad (thought about rewrite buildfarm to Python).
>
> I hope for your support.


Tried to generate Xcode project out of cmake, build fails on genbki.pl: can't locate Catalog.pm (which itself lives in
src/backend/catalog/Catalog.pm)

--
Stas Kelvich
Postgres Professional: http://www.postgrespro.com
Russian Postgres Company

Stas Kelvich wrote:
>> On 17 Sep 2016, at 20:21, Yury Zhuravlev
>> <u.zhuravlev@postgrespro.ru> wrote:
>>
>> Michael Paquier wrote: ...
>
>
> Tried to generate Xcode project out of cmake, build fails on
> genbki.pl: can't locate Catalog.pm (which itself lives in
> src/backend/catalog/Catalog.pm)
>

Can you try again? On my Xcode 7.3 / ElCapitan everything is working
correctly.
--
Yury Zhuravlev
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company

Stas Kelvich wrote:
> Tried to generate Xcode project out of cmake, build fails on
> genbki.pl: can't locate Catalog.pm (which itself lives in
> src/backend/catalog/Catalog.pm)

Thanks again! I have corrected your issue.
--
Yury Zhuravlev
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company

On 9/17/16 1:21 PM, Yury Zhuravlev wrote:
> Now, I published the first version of the patch. 

I tried this out.  Because of some file moves in initdb and
pg_basebackup, the build fails:

[ 74%] Linking C executable initdbUndefined symbols for architecture x86_64:  "_fsync_pgdata", referenced from:
_mainin initdb.c.old: symbol(s) not found for architecture x86_64collect2: error: ld returned 1 exit statusmake[2]: ***
[src/bin/initdb/CMakeFiles/initdb.dir/build.make:177:
src/bin/initdb/initdb] Error 1make[1]: *** [CMakeFiles/Makefile2:2893:
src/bin/initdb/CMakeFiles/initdb.dir/all] Error 2make: *** [Makefile:128: all] Error 2

Please submit an updated patch.

I suggest you use git format-patch to produce patches.  This is easier
to apply, especially when there are a lot of new files involved.  Also
use the git facilities to check for whitespace errors.

Please supply some documentation, such as

- what are the basic commands
- how to set include/library paths, choose configure options
- how to set CFLAGS
- how to see raw build commands
- what are the targets for all/world/check/docs etc.
- explain directory structure

I suggest for now you could put this into a README.cmake file in your
patch.  We don't need to commit it that way, but it would help in the
meantime.

When I run cmake without options, it seems to do opportunistic feature
checking.  For example, it turns on OpenSSL or Python support if it can
find it, otherwise it turns it off.  We need this to be deterministic.
Without options, choose the basic feature set, require all other
features to be turned on explicitly, fail if they can't be found.
Whatever the Autoconf-based build does now has been fairly deliberately
tuned, so there should be very little reason to deviate from that.

The Python check appears to be picking up pieces from two different
Python installations:
-- Found PythonInterp: /usr/local/bin/python (found version "2.7.12")-- Found PythonLibs: /usr/lib/libpython2.7.dylib
(foundversion "2.7.10")
 

The check results otherwise look OK, but I'm a bit confused about the
order.  It checks for some functions before all the header files are
checked for.  Is that intentional?

There are a number of changes in .[ch] and .pl files that are unclear
and not explained.  Please explain them.  You can also submit separate
preliminary patches if you need to do some refactoring.  Ultimately, I
would expect this patch not to require C code changes.

-- 
Peter Eisentraut              http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Hello.

Peter Eisentraut wrote:
> I tried this out.  Because of some file moves in initdb and
> pg_basebackup, the build fails:
Yes, I need rebase patch to latest master. I do it as soon as possible.
Also I have some new achievements.

>I suggest you use git format-patch to produce patches.  This is easier
>to apply, especially when there are a lot of new files involved.  Also
>use the git facilities to check for whitespace errors.
I agree.

>I suggest for now you could put this into a README.cmake file in your
>patch.  We don't need to commit it that way, but it would help in the
>meantime.
Good idea, currently I write all documentation on github. I will try to
move it to patch.

>When I run cmake without options, it seems to do opportunistic feature
>checking.  For example, it turns on OpenSSL or Python support if it can
>find it, otherwise it turns it off.  We need this to be deterministic.
>Without options, choose the basic feature set, require all other
>features to be turned on explicitly, fail if they can't be found.
>Whatever the Autoconf-based build does now has been fairly deliberately
>tuned, so there should be very little reason to deviate from that.
This approach I see only in Postgres project and not fully understood.
Can you explain me more what reasons led to this approach?
Not big deal to make behavior like postgres Autoconf but I think it's
important clear view here.

>The Python check appears to be picking up pieces from two different
>Python installations:
Ooops you right. For Python detecting I use standard CMake module and
his behavior depending on CMake version. We should really careful here.

>The check results otherwise look OK, but I'm a bit confused about the
>order.  It checks for some functions before all the header files are
>checked for.  Is that intentional?
I have plans to change order checks.

>There are a number of changes in .[ch] and .pl files that are unclear
>and not explained.  Please explain them.  You can also submit separate
>preliminary patches if you need to do some refactoring.  Ultimately, I
>would expect this patch not to require C code changes.
I suppose separate patches with comments will be best way. I will do it.
I think we can talks about details after that.

Big thanks for your remarks it's very important for me and this "small"
project.
I will try to do all tasks quickly.

Best regards.

--
Yury Zhuravlev
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company

<p dir="ltr"><p dir="ltr">On 9 Nov. 2016 06:37, "Yury Zhuravlev" <<a
href="mailto:u.zhuravlev@postgrespro.ru">u.zhuravlev@postgrespro.ru</a>>wrote:<br /> ><br /> >> When I run
cmakewithout options, it seems to do opportunistic feature<br /> >> checking.  For example, it turns on OpenSSL
orPython support if it can<br /> >> find it, otherwise it turns it off.  We need this to be deterministic.<br />
>>Without options, choose the basic feature set, require all other<br /> >> features to be turned on
explicitly,fail if they can't be found.<br /> >> Whatever the Autoconf-based build does now has been fairly
deliberately<br/> >> tuned, so there should be very little reason to deviate from that.<br /> ><br /> >
Thisapproach I see only in Postgres project and not fully understood.<br /> > Can you explain me more what reasons
ledto this approach?<p dir="ltr">It's predictable. The default has the same result for everyone. I quite like it
myself.<br/> 

On Wed, Nov 9, 2016 at 7:54 AM, Craig Ringer
<craig.ringer@2ndquadrant.com> wrote:
> On 9 Nov. 2016 06:37, "Yury Zhuravlev" <u.zhuravlev@postgrespro.ru> wrote:
>> This approach I see only in Postgres project and not fully understood.
>> Can you explain me more what reasons led to this approach?
>
> It's predictable. The default has the same result for everyone. I quite like
> it myself.

+1. Let's tell to the system what we want him to do and not let him
guess what we'd like to be done or it will get harder to test and
develop code for all kind of code paths with #ifdef's. That's one step
away from Skynet.
-- 
Michael

On 9 November 2016 at 10:12, Michael Paquier <michael.paquier@gmail.com> wrote:
> On Wed, Nov 9, 2016 at 7:54 AM, Craig Ringer
> <craig.ringer@2ndquadrant.com> wrote:
>> On 9 Nov. 2016 06:37, "Yury Zhuravlev" <u.zhuravlev@postgrespro.ru> wrote:
>>> This approach I see only in Postgres project and not fully understood.
>>> Can you explain me more what reasons led to this approach?
>>
>> It's predictable. The default has the same result for everyone. I quite like
>> it myself.
>
> +1. Let's tell to the system what we want him to do and not let him
> guess what we'd like to be done or it will get harder to test and
> develop code for all kind of code paths with #ifdef's. That's one step
> away from Skynet.

Er... ok then. (Backs away slowly).

More seriously, I like it for development where a stable and
predictable default is great.

For users it slightly sucks, as most users will want us to find
whatever is on the system without being manually told to enable each
feature. "Of course I want SSL, I have openssl installed don't I?"
It's not like we require users to specify --enable-largefile
--enable-atomics --enable-getopt --enable-ipv6  .... we do detect a
lot automatically.

So personally I think it'd be fine if a cmake build defaulted to
finding and using what it could, but offered a --minimal mode or
whatever that gets us just core postgres + whatever we enable
explicitly. But our current behaviour is OK too.

-- Craig Ringer                   http://www.2ndQuadrant.com/PostgreSQL Development, 24x7 Support, Remote DBA, Training
&Services
 

Craig Ringer <craig.ringer@2ndquadrant.com> writes:
> On 9 Nov. 2016 06:37, "Yury Zhuravlev" <u.zhuravlev@postgrespro.ru> wrote:
>> This approach I see only in Postgres project and not fully understood.
>> Can you explain me more what reasons led to this approach?

> It's predictable. The default has the same result for everyone. I quite
> like it myself.

It's advantageous from a packaging standpoint, precisely because it's
predictable: either you get the set of features you expect, or you get
a build failure.  Years ago we were more on the "opportunistic" side
of things, and we had problems with packages sometimes silently losing
features.  A pretty-recent example is that OpenSSL changed their APIs
in 1.1.0 so that our configure probe failed.  With an opportunistic
approach, that would have meant builds silently going from "ssl yes"
to "ssl no".  That's not good.

So this is really not open for negotiation.  As Peter said upthread,
what we are looking for in a CMake reimplementation is that it behaves
exactly like the Autoconf version does.  To the extent that you are unable
or unwilling to duplicate that behavior, you increase the odds that
we'll reject this work.
        regards, tom lane

Tom Lane wrote:
> So this is really not open for negotiation.  As Peter said upthread,
> what we are looking for in a CMake reimplementation is that it behaves
> exactly like the Autoconf version does.  To the extent that you are unable
> or unwilling to duplicate that behavior, you increase the odds that
> we'll reject this work.
Who asking about negotiation? I just wanted an explanation for the clear
understanding and nothing more.
Now I know about reasons. Thanks.

regards
--
Yury Zhuravlev
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company

Craig Ringer wrote:
> So personally I think it'd be fine if a cmake build defaulted to
> finding and using what it could, but offered a --minimal mode or
> whatever that gets us just core postgres + whatever we enable
> explicitly. But our current behaviour is OK too.

To me it's best way. But I'm not sure what Tom Lane will accept this.


--
Yury Zhuravlev
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company

Craig Ringer wrote:
> So personally I think it'd be fine if a cmake build defaulted to
> finding and using what it could, but offered a --minimal mode or
> whatever that gets us just core postgres + whatever we enable
> explicitly. But our current behaviour is OK too.

To me it's best way. But I'm not sure what Tom Lane will accept this.


--
Yury Zhuravlev
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company

On 11/11/16 08:15, Yury Zhuravlev wrote:

> Craig Ringer wrote:
>> So personally I think it'd be fine if a cmake build defaulted to
>> finding and using what it could, but offered a --minimal mode or
>> whatever that gets us just core postgres + whatever we enable
>> explicitly. But our current behaviour is OK too.
>
> To me it's best way. But I'm not sure what Tom Lane will accept this.
>
>

I just had a play with this patch - nice! (ok so it needs a fix so that 
the compile completes as mentioned prev).

I would recommend making it behave as Tom suggested. *Then* add an 
--autodetect or similar option that makes
behave in the 'finding and using what it could' manner as a 2nd patch.

regards

Mark

Mark Kirkwood <mark.kirkwood@catalyst.net.nz> writes:
> I would recommend making it behave as Tom suggested. *Then* add an 
> --autodetect or similar option that makes
> behave in the 'finding and using what it could' manner as a 2nd patch.

An "--autodetect" switch would be fine with me.  I just don't want it
to behave that way by default, because then you get into the unexpected
results problem.
        regards, tom lane

I had a bit a play around to see if I could get this building properly
again with v10 (i.e master). I've attached a minimal *incremental* patch
that needs to be applied after v1. This gets everything under the src
tree building (added the new file_utils.c and reordered some link libs
and removed some duplicates).


I haven't made any changes with respect to it trying to detect and build
everything. One added nit I see is that unless I'm missing something
there appears to be no way to stop it trying to build all the
contribs...so an option to enable/disable their build is needed.


To make it display what options there are I use:

$ mkdir build; cd build ; cmake .. -LH


And to do a build that works:

$ cmake .. -DCMAKE_INSTALL_PREFIX=/usr/local/pgsql/10 -DWITH_PERL=OFF
-DWITH_OPENSSL=OFF -DWITH_PYTHON=OFF


With reference to Tom's comments, I'm thinking that these should all
default to 'OFF' plus an additional variable WITH_CONTRIB (or similar)
should default to OFF too.


regards


Mark



On 09/11/16 08:37, Peter Eisentraut wrote:
> On 9/17/16 1:21 PM, Yury Zhuravlev wrote:
>> Now, I published the first version of the patch.
> I tried this out.  Because of some file moves in initdb and
> pg_basebackup, the build fails:
>
> [ 74%] Linking C executable initdb
>   Undefined symbols for architecture x86_64:
>     "_fsync_pgdata", referenced from:
>         _main in initdb.c.o
>   ld: symbol(s) not found for architecture x86_64
>   collect2: error: ld returned 1 exit status
>   make[2]: *** [src/bin/initdb/CMakeFiles/initdb.dir/build.make:177:
> src/bin/initdb/initdb] Error 1
>   make[1]: *** [CMakeFiles/Makefile2:2893:
> src/bin/initdb/CMakeFiles/initdb.dir/all] Error 2
>   make: *** [Makefile:128: all] Error 2
>
> Please submit an updated patch.
>
> I suggest you use git format-patch to produce patches.  This is easier
> to apply, especially when there are a lot of new files involved.  Also
> use the git facilities to check for whitespace errors.
>
> Please supply some documentation, such as
>
> - what are the basic commands
> - how to set include/library paths, choose configure options
> - how to set CFLAGS
> - how to see raw build commands
> - what are the targets for all/world/check/docs etc.
> - explain directory structure
>
> I suggest for now you could put this into a README.cmake file in your
> patch.  We don't need to commit it that way, but it would help in the
> meantime.
>
> When I run cmake without options, it seems to do opportunistic feature
> checking.  For example, it turns on OpenSSL or Python support if it can
> find it, otherwise it turns it off.  We need this to be deterministic.
> Without options, choose the basic feature set, require all other
> features to be turned on explicitly, fail if they can't be found.
> Whatever the Autoconf-based build does now has been fairly deliberately
> tuned, so there should be very little reason to deviate from that.
>
> The Python check appears to be picking up pieces from two different
> Python installations:
>
>   -- Found PythonInterp: /usr/local/bin/python (found version "2.7.12")
>   -- Found PythonLibs: /usr/lib/libpython2.7.dylib (found version "2.7.10")
>
> The check results otherwise look OK, but I'm a bit confused about the
> order.  It checks for some functions before all the header files are
> checked for.  Is that intentional?
>
> There are a number of changes in .[ch] and .pl files that are unclear
> and not explained.  Please explain them.  You can also submit separate
> preliminary patches if you need to do some refactoring.  Ultimately, I
> would expect this patch not to require C code changes.
>

Yeah, there seems to be a lot of these. Looking through them almost all 
concern the addition of piece of code to wrap putenv. e.g:

--- a/src/bin/psql/command.c
+++ b/src/bin/psql/command.c
@@ -1350,7 +1350,7 @@ exec_command(const char *cmd,                        char       *newval;
                        newval = psprintf("%s=%s", envvar, envval);
-                       putenv(newval);
+                       pg_putenv_proxy(newval);                        success = true;
                        /*

Where pg_putenv_proxy either calls putenv or pgwin32_putenv (the latter 
on windows I'd guess). I wonder if this could have been avoided, since 
the original code handles this sort of thing. There are also some minor 
- and not immediately obvious - changes to a number of macros in various 
includes...If I'm feeling keen I'll experiment to see how far I can get 
without any source changes at all.


regards


Mark


On 09/11/16 08:37, Peter Eisentraut wrote:
>
> There are a number of changes in .[ch] and .pl files that are unclear
> and not explained.  Please explain them.  You can also submit separate
> preliminary patches if you need to do some refactoring.  Ultimately, I
> would expect this patch not to require C code changes.
>

Actually, it was not that tricky to separate out the cmake only changes,
and test this on unmodified sources. It appears to work fine for me -
passes 'make check' (needs the v1_1 incremental patch applied of
course). The Patch is attached. I wonder if the original had some
changes for building under latest Windows...(I'm using Ubuntu 16.10,
with cmake 3.5).


regards


Mark


On 16/11/16 16:00, Mark Kirkwood wrote:
> Yeah, there seems to be a lot of these. Looking through them almost
> all concern the addition of piece of code to wrap putenv. e.g:
>
> --- a/src/bin/psql/command.c
> +++ b/src/bin/psql/command.c
> @@ -1350,7 +1350,7 @@ exec_command(const char *cmd,
>                         char       *newval;
>
>                         newval = psprintf("%s=%s", envvar, envval);
> -                       putenv(newval);
> +                       pg_putenv_proxy(newval);
>                         success = true;
>
>                         /*
>
> Where pg_putenv_proxy either calls putenv or pgwin32_putenv (the
> latter on windows I'd guess). I wonder if this could have been
> avoided, since the original code handles this sort of thing. There are
> also some minor - and not immediately obvious - changes to a number of
> macros in various includes...If I'm feeling keen I'll experiment to
> see how far I can get without any source changes at all.
>
>
> regards
>
>
> Mark
>
>
> On 09/11/16 08:37, Peter Eisentraut wrote:
>>
>> There are a number of changes in .[ch] and .pl files that are unclear
>> and not explained.  Please explain them.  You can also submit separate
>> preliminary patches if you need to do some refactoring. Ultimately, I
>> would expect this patch not to require C code changes.
>>
>

Mark Kirkwood wrote:
> Actually, it was not that tricky to separate out the cmake only
> changes, and test this on unmodified sources. It appears to work
> fine for me - passes 'make check' (needs the v1_1 incremental
> patch applied of course). The Patch is attached. I wonder if the
> original had some changes for building under latest
> Windows...(I'm using Ubuntu 16.10, with cmake 3.5).
>
Thanks for all your works! Can you make push request here:
https://github.com/stalkerg/postgres_cmake

I have rebased (merge) to master and make other important fix.

--
Yury Zhuravlev
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company

Mark Kirkwood wrote:
> Yeah, there seems to be a lot of these. Looking through them
> almost all concern the addition of piece of code to wrap putenv.
> e.g:
I made this small wrapper special for MSVC 2015 without Service Packs
because postgres macross were in conflict with MS internal functions.
After some time and some updates for MSVC Michael Paquier could not
reproduce  my problem but I keep this patch to avoid problems in the
future.
I can check old behavior again and revert all changes if needed and
ofcourse I have plans to make separate patch for this changes.

Thanks!
--
Yury Zhuravlev
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company

On Tue, Nov 8, 2016 at 9:12 PM, Michael Paquier
<michael.paquier@gmail.com> wrote:
> On Wed, Nov 9, 2016 at 7:54 AM, Craig Ringer
> <craig.ringer@2ndquadrant.com> wrote:
>> On 9 Nov. 2016 06:37, "Yury Zhuravlev" <u.zhuravlev@postgrespro.ru> wrote:
>>> This approach I see only in Postgres project and not fully understood.
>>> Can you explain me more what reasons led to this approach?
>>
>> It's predictable. The default has the same result for everyone. I quite like
>> it myself.
>
> +1. Let's tell to the system what we want him to do and not let him
> guess what we'd like to be done or it will get harder to test and
> develop code for all kind of code paths with #ifdef's. That's one step
> away from Skynet.

Exaggerate much?

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

I see there are some patches for the putenv issue with Visual studio 
2013 in progress on this list - it is probably best to work with the 
author to see if 2015 has any new issues and keep all changes for that 
*out* of the cmake patches.


regards


Mark


On 16/11/16 21:22, Yury Zhuravlev wrote:
> I made this small wrapper special for MSVC 2015 without Service Packs 
> because postgres macross were in conflict with MS internal functions. 
> After some time and some updates for MSVC Michael Paquier could not 
> reproduce  my problem but I keep this patch to avoid problems in the 
> future. I can check old behavior again and revert all changes if 
> needed and ofcourse I have plans to make separate patch for this changes.
> Thanks!

On Wed, Nov 16, 2016 at 2:14 PM, Mark Kirkwood
<mark.kirkwood@catalyst.net.nz> wrote:
> I see there are some patches for the putenv issue with Visual studio 2013 in
> progress on this list - it is probably best to work with the author to see
> if 2015 has any new issues and keep all changes for that *out* of the cmake
> patches.

I don't recall all the details here, but no wrappers should be needed,
particularly on Windows where we already do that:
src/include/port/win32.h:#define putenv(x) pgwin32_putenv(x)
-- 
Michael

Michael Paquier wrote:
> src/include/port/win32.h:#define putenv(x) pgwin32_putenv(x)
and my MSVC2015 drop down here because pgwin32_putenv has wrong signature.
I hope it is not true now.

--
Yury Zhuravlev
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company

On 12/30/16 9:10 AM, Yuriy Zhuravlev wrote:
> cmake_v2_2_c_define.patch
> 
> Small chages in c.h . At first it is “#pragma fenv_access (off)” it is
> necessary if we use /fp:strict for MSVC compiler. Without this pragma we
> can’t calc floats for const variables in compiller time (2 * M_PI for
> example). Strict mode important if we want to be close with ieee754
> float format on MSVC (1.0 / 0.0 = inf for example).  Detail info here:
> https://msdn.microsoft.com/en-us/library/e7s85ffb.aspx

I don't understand what this has to do with cmake.  If this is a
worthwhile improvement for the Windows build, then please explain why,
with a "before" and "after" output and a patch for the existing build
system as well.

> Second change is because I find and set HAVE_INT128 directly from CMake.
> PG_INT128_TYPE used only for autoconfig scripts.

It might also be worth refactoring the existing Autoconf code here to
make this consistent.

(My assumption is that if we were to move forward with cmake or any
other build system change, we would have to keep the old one alongside
at least for a little while.  So any changes to the C code would need to
be side-ported.)

> cmake_v2_3_rijndael.patch
> 
> First I added special wraparound because here CMake have circular
> dependency (cmake very smart here). Second I removed rijndael.tbl
> because it generated during build process every time.

Please explain what the circular dependency is.  If there is one, we
should also side-port this change.

> cmake_v2_4_uuid.patch
> 
> Another small patch. Right place for uuid.h I find by CMake and not
> necessary this ifdef hell.

This patch removes the uuid.h include but doesn't add it anywhere else.
How does it work?

> Questions for discussion:
> 
> In generated project by CMake we always have only one enter point. Also
> INSTALL macross support only including to “all” targets. It follows that
> it is impossible build contrib modules separately only with “all”
> target. Here write about this behavior:
> https://cmake.org/cmake/help/v3.7/prop_tgt/EXCLUDE_FROM_ALL.html

Yeah, I think this is how the MSVC stuff effectively works right now as
well.

-- 
Peter Eisentraut              http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

On Tue, Jan 3, 2017 at 11:11 PM, Peter Eisentraut
<peter.eisentraut@2ndquadrant.com> wrote:
> On 12/30/16 9:10 AM, Yuriy Zhuravlev wrote:
>> cmake_v2_2_c_define.patch
>>
>> Small chages in c.h . At first it is “#pragma fenv_access (off)” it is
>> necessary if we use /fp:strict for MSVC compiler. Without this pragma we
>> can’t calc floats for const variables in compiller time (2 * M_PI for
>> example). Strict mode important if we want to be close with ieee754
>> float format on MSVC (1.0 / 0.0 = inf for example).  Detail info here:
>> https://msdn.microsoft.com/en-us/library/e7s85ffb.aspx
>
> I don't understand what this has to do with cmake.  If this is a
> worthwhile improvement for the Windows build, then please explain why,
> with a "before" and "after" output and a patch for the existing build
> system as well.
>
>> Second change is because I find and set HAVE_INT128 directly from CMake.
>> PG_INT128_TYPE used only for autoconfig scripts.
>
> It might also be worth refactoring the existing Autoconf code here to
> make this consistent.
>
> (My assumption is that if we were to move forward with cmake or any
> other build system change, we would have to keep the old one alongside
> at least for a little while.  So any changes to the C code would need to
> be side-ported.)
>
>> cmake_v2_3_rijndael.patch
>>
>> First I added special wraparound because here CMake have circular
>> dependency (cmake very smart here). Second I removed rijndael.tbl
>> because it generated during build process every time.
>
> Please explain what the circular dependency is.  If there is one, we
> should also side-port this change.
>
>> cmake_v2_4_uuid.patch
>>
>> Another small patch. Right place for uuid.h I find by CMake and not
>> necessary this ifdef hell.
>
> This patch removes the uuid.h include but doesn't add it anywhere else.
> How does it work?
>
>> Questions for discussion:
>>
>> In generated project by CMake we always have only one enter point. Also
>> INSTALL macross support only including to “all” targets. It follows that
>> it is impossible build contrib modules separately only with “all”
>> target. Here write about this behavior:
>> https://cmake.org/cmake/help/v3.7/prop_tgt/EXCLUDE_FROM_ALL.html
>
> Yeah, I think this is how the MSVC stuff effectively works right now as
> well.

I am marking this patch as "returned with feedback". That's quite a
heavy change and it looks to be too late in the development cycle of
PG10 to consider it. Peter's commit bits, who is also the reviewer,
are beginning to smoke as well after everything that has happened for
the logical replication changes.
--
Michael

On 2017-01-24 15:50:47 +0900, Michael Paquier wrote:
> I am marking this patch as "returned with feedback". That's quite a
> heavy change and it looks to be too late in the development cycle of
> PG10 to consider it. Peter's commit bits, who is also the reviewer,
> are beginning to smoke as well after everything that has happened for
> the logical replication changes.

I'm doubtful about this being ready in time too, but it seems a might
heavyhanded to make that call on your own. Including the judgement about
Peter's capability to handle more.

Greetings,

Andres Freund

On Tue, Jan 24, 2017 at 3:58 PM, Andres Freund <andres@anarazel.de> wrote:
> On 2017-01-24 15:50:47 +0900, Michael Paquier wrote:
>> I am marking this patch as "returned with feedback". That's quite a
>> heavy change and it looks to be too late in the development cycle of
>> PG10 to consider it. Peter's commit bits, who is also the reviewer,
>> are beginning to smoke as well after everything that has happened for
>> the logical replication changes.
>
> I'm doubtful about this being ready in time too, but it seems a might
> heavyhanded to make that call on your own. Including the judgement about
> Peter's capability to handle more.

Sure, sorry for that. I am switching back the patch, let's see what
happens by the end of the CF.
-- 
Michael

Craig Ringer <craig.ringer@2ndquadrant.com> writes:
> Personally I think we should aim to have this in as a non default build
> mode in pg10 if it can be made ready, and aim to make it default in pg11 at
> least for Windows.

AFAIK we haven't committed to accepting this at all, let alone trying
to do so on a tight schedule.  And I believe there was general agreement
that we would not accept it as something to maintain in parallel with
the existing makefiles.  If we have to maintain two build systems, we
have that already.
        regards, tom lane

On 1/24/17 8:37 AM, Tom Lane wrote:
> Craig Ringer <craig.ringer@2ndquadrant.com> writes:
>> Personally I think we should aim to have this in as a non default build
>> mode in pg10 if it can be made ready, and aim to make it default in pg11 at
>> least for Windows.
> 
> AFAIK we haven't committed to accepting this at all, let alone trying
> to do so on a tight schedule.  And I believe there was general agreement
> that we would not accept it as something to maintain in parallel with
> the existing makefiles.  If we have to maintain two build systems, we
> have that already.

My preferred scenario would be to replace the Windows build system by
this first, then refine it, then get rid of Autoconf.

The ideal timeline would be to have a ready patch to commit early in a
development cycle, then get rid of the Windows build system by the end
of it.  Naturally, this would need buy-in from Windows developers.

I don't foresee replacing the Autoconf build system by this immediately.

Right now, however, the patch isn't moving at all, and I don't see it
going into PG10, so I'm fine with returning with feedback.

-- 
Peter Eisentraut              http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Peter Eisentraut wrote:

> Right now, however, the patch isn't moving at all, and I don't see it
> going into PG10, so I'm fine with returning with feedback.

There are a bunch of side patches that we should apply separately, such
as the pgcrypto fix.  I don't understand why they are part of this patch
series, really, given that they seem largely independent prerequisites.

-- 
Álvaro Herrera                https://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

On Fri, Jan 27, 2017 at 11:09 PM, Peter Eisentraut
<peter.eisentraut@2ndquadrant.com> wrote:
> On 1/24/17 8:37 AM, Tom Lane wrote:
>> Craig Ringer <craig.ringer@2ndquadrant.com> writes:
>>> Personally I think we should aim to have this in as a non default build
>>> mode in pg10 if it can be made ready, and aim to make it default in pg11 at
>>> least for Windows.
>>
>> AFAIK we haven't committed to accepting this at all, let alone trying
>> to do so on a tight schedule.  And I believe there was general agreement
>> that we would not accept it as something to maintain in parallel with
>> the existing makefiles.  If we have to maintain two build systems, we
>> have that already.
>
> My preferred scenario would be to replace the Windows build system by
> this first, then refine it, then get rid of Autoconf.
>
> The ideal timeline would be to have a ready patch to commit early in a
> development cycle, then get rid of the Windows build system by the end
> of it.  Naturally, this would need buy-in from Windows developers.

This looks like a really good plan to me.
-- 
Michael

On 2017-01-27 09:09:36 -0500, Peter Eisentraut wrote:
> My preferred scenario would be to replace the Windows build system by
> this first, then refine it, then get rid of Autoconf.
> 
> The ideal timeline would be to have a ready patch to commit early in a
> development cycle, then get rid of the Windows build system by the end
> of it.  Naturally, this would need buy-in from Windows developers.
> 
> I don't foresee replacing the Autoconf build system by this immediately.

I'm very strongly against this path, it seems way too likely that we'll
end up with yet another fragile thing that nobody from the *nix side
will be able to test.

- Andres

On 1/27/17 6:11 PM, Andres Freund wrote:
> On 2017-01-27 09:09:36 -0500, Peter Eisentraut wrote:
>> My preferred scenario would be to replace the Windows build system by
>> this first, then refine it, then get rid of Autoconf.
>>
>> The ideal timeline would be to have a ready patch to commit early in a
>> development cycle, then get rid of the Windows build system by the end
>> of it.  Naturally, this would need buy-in from Windows developers.
>>
>> I don't foresee replacing the Autoconf build system by this immediately.
> 
> I'm very strongly against this path, it seems way too likely that we'll
> end up with yet another fragile thing that nobody from the *nix side
> will be able to test.

That's a fair concern, but at least with CMake, someone from the *nix
side *can* test it, whereas right now it's completely separate.

What kind of strategy do you have in mind?

-- 
Peter Eisentraut              http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

On Sat, Jan 28, 2017 at 12:20 PM, Peter Eisentraut
<peter.eisentraut@2ndquadrant.com> wrote:
> On 1/27/17 6:11 PM, Andres Freund wrote:
>> On 2017-01-27 09:09:36 -0500, Peter Eisentraut wrote:
>>> My preferred scenario would be to replace the Windows build system by
>>> this first, then refine it, then get rid of Autoconf.
>>>
>>> The ideal timeline would be to have a ready patch to commit early in a
>>> development cycle, then get rid of the Windows build system by the end
>>> of it.  Naturally, this would need buy-in from Windows developers.
>>>
>>> I don't foresee replacing the Autoconf build system by this immediately.
>>
>> I'm very strongly against this path, it seems way too likely that we'll
>> end up with yet another fragile thing that nobody from the *nix side
>> will be able to test.
>
> That's a fair concern, but at least with CMake, someone from the *nix
> side *can* test it, whereas right now it's completely separate.

And people complain all the time that the MSVC build scripts are hacky
and complicated.. So by beginning from there we switch from one build
to the other, not increasing the number of builds that need to be
maintained. Based on that Peter's strategy looks appealing to me. By
the way, I am marking the patch as returned with feedback for this CF.
-- 
Michael

On 2017-01-27 22:20:41 -0500, Peter Eisentraut wrote:
> On 1/27/17 6:11 PM, Andres Freund wrote:
> > On 2017-01-27 09:09:36 -0500, Peter Eisentraut wrote:
> >> My preferred scenario would be to replace the Windows build system by
> >> this first, then refine it, then get rid of Autoconf.
> >>
> >> The ideal timeline would be to have a ready patch to commit early in a
> >> development cycle, then get rid of the Windows build system by the end
> >> of it.  Naturally, this would need buy-in from Windows developers.
> >>
> >> I don't foresee replacing the Autoconf build system by this immediately.
> > 
> > I'm very strongly against this path, it seems way too likely that we'll
> > end up with yet another fragile thing that nobody from the *nix side
> > will be able to test.
> 
> That's a fair concern, but at least with CMake, someone from the *nix
> side *can* test it, whereas right now it's completely separate.

Given that fact, I just don't buy why it's a good idea to not also
replace autoconf initially.  Either we're able to properly test it -
i.e. it runs all tests - on *nix or we're not.  There's not a a whole of
effort between those if you also want to do the windows side of things
properly.


> What kind of strategy do you have in mind?

Do all of it. I'm unconvinced that a windows only version buys us
meaningful savings, and I think the dangers of adding more duplication
(msvc stuff after all gets some information from the makefiles) and
long-term coexistence are quite severe.


Greetings,

Andres Freund

On 1/30/17 1:28 AM, Andres Freund wrote:
> Given that fact, I just don't buy why it's a good idea to not also
> replace autoconf initially.

Well, I find it a bit scary.  If you do the big switch all at once, then
you will have to dedicate the following 3 months to fixing complaints
from developers and build farmers.

> Either we're able to properly test it - i.e. it runs all tests - on *nix or we're not.

That would work if there were a single entry point into the build
system.  But in practice there are many, and every one of them is
someone's favorite.  It's unlikely that we will be able to enumerate all
of them during patch review.

-- 
Peter Eisentraut              http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

On 2/8/17 6:21 AM, Yuriy Zhuravlev wrote:
> Support two build systems it's not big deal really. I have been working
> on this past year without any big troubles. 
> Also we have second perl build system...

The perl/msvc build system pulls in information from the makefiles.  So
when you add a file or something basic like that, you don't have to
update it.  So it's really more like 1.5 build systems.

-- 
Peter Eisentraut              http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Hi,

On 2017-01-30 10:26:18 -0500, Peter Eisentraut wrote:
> On 1/30/17 1:28 AM, Andres Freund wrote:
> > Given that fact, I just don't buy why it's a good idea to not also
> > replace autoconf initially.
> 
> Well, I find it a bit scary.  If you do the big switch all at once, then
> you will have to dedicate the following 3 months to fixing complaints
> from developers and build farmers.

That'll be the case just as well if we spread it out over two cycles,
except that we'll have it in multiple phases, and we run into the danger
of a half-done conversion.

I'd rather not change systems at all than run into the danger of that.


> > Either we're able to properly test it - i.e. it runs all tests - on *nix or we're not.
> 
> That would work if there were a single entry point into the build
> system.  But in practice there are many, and every one of them is
> someone's favorite.  It's unlikely that we will be able to enumerate all
> of them during patch review.

Not sure what you mean with "entry point"?


- Andres

Peter Eisentraut <peter.eisentraut@2ndquadrant.com> writes:
> On 2/8/17 6:21 AM, Yuriy Zhuravlev wrote:
>> Support two build systems it's not big deal really. I have been working
>> on this past year without any big troubles. 
>> Also we have second perl build system...

> The perl/msvc build system pulls in information from the makefiles.  So
> when you add a file or something basic like that, you don't have to
> update it.  So it's really more like 1.5 build systems.

Really it's more like 1.1 build systems, in that the MSVC scripts do that
just well enough that you *usually* don't have to think about them.  But
then when they fail, and you have to figure out why, it can be a pain.

For my own purposes, the only thing that I find seriously annoying about
the status quo is the amount of time required to run "configure".  For
me, that step is usually comparable to or even more than the time to
do the build proper, because (a) ccache and (b) multiple CPUs.
configure isn't parallelizable, and there's probably nothing that
can be done about that.  If CMake offers a substantial improvement
in configuration time then that would be attractive.  Otherwise I'd
just as soon see us put the effort into making the MSVC scripts more
robust and able to pull more data from the makefiles.
        regards, tom lane

On 2/8/17 3:52 PM, Tom Lane wrote:
> For my own purposes, the only thing that I find seriously annoying about
> the status quo is the amount of time required to run "configure".  For
> me, that step is usually comparable to or even more than the time to
> do the build proper, because (a) ccache and (b) multiple CPUs.
> configure isn't parallelizable, and there's probably nothing that
> can be done about that.  If CMake offers a substantial improvement
> in configuration time then that would be attractive.  Otherwise I'd
> just as soon see us put the effort into making the MSVC scripts more
> robust and able to pull more data from the makefiles.

FWIW, I've had good luck adding -C to configure to cache the output. I'd 
guess it's at least 10x faster on my laptop.

Obviously doesn't help if you're doing where you're testing something 
that alters config output. In those cases I'll either edit config.cache 
and delete the relevant lines or just temporarily move it out of the way 
(or just nuke it...).
-- 
Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX
Experts in Analytics, Data Architecture and PostgreSQL
Data in Trouble? Get it in Treble! http://BlueTreble.com
855-TREBLE2 (855-873-2532)

Hi,

On 2017-02-08 16:52:19 -0500, Tom Lane wrote:
> For my own purposes, the only thing that I find seriously annoying about
> the status quo is the amount of time required to run "configure".  For
> me, that step is usually comparable to or even more than the time to
> do the build proper, because (a) ccache and (b) multiple CPUs.
> configure isn't parallelizable, and there's probably nothing that
> can be done about that.

I use autoconf caching feature to make that a bit less painful (plus
some scripting about when to scrap the cache file...).  I find that
seriously annoying too.


> If CMake offers a substantial improvement
> in configuration time then that would be attractive.  Otherwise I'd
> just as soon see us put the effort into making the MSVC scripts more
> robust and able to pull more data from the makefiles.

Some of the build-tooling in cmake is quite nice, I have to admit. I've
e.g. grown to like using ninja instead of make to build the resulting
files. Primarily in projects that take longer than pg to compile - a
clean build in llvm with ninja takes like 0.1 seconds.

Being more able to rely on things working on windows when doing them on
linux does seem like an advantage to me - fiddlin with Mkvcbuild.pm is
quite annoying.

- Andres

On Mon, Jan 30, 2017 at 10:26 AM, Peter Eisentraut
<peter.eisentraut@2ndquadrant.com> wrote:
> On 1/30/17 1:28 AM, Andres Freund wrote:
>> Given that fact, I just don't buy why it's a good idea to not also
>> replace autoconf initially.
>
> Well, I find it a bit scary.  If you do the big switch all at once, then
> you will have to dedicate the following 3 months to fixing complaints
> from developers and build farmers.

I agree with that.  I think replacing the Windows build system first
and then the non-Windows build system later is a better plan than
replacing both at the same time.

But also, I'm not really sure whether this conversion makes sense.  I
mean, any build system is going to require some work, and accordingly
our present build systems require some work.  cmake will require
different work, but not necessarily less.  The current system has a
long history; we pretty much know it works.  Switching will inevitably
break some things.  Maybe we'll end up better off in the long term,
but maybe we won't.  Things are pretty good now, so it seems like it
would be easy for them to get worse rather than better.  If nothing
else, everybody who has to learn the new system either to use it for
development or because they are doing packaging will have to do some
amount of extra work as a result of any switch.

I do agree that - in theory - one build system is better than two.
But two well-tested, reliable build systems could easily be better
than one system with a bunch of problems.  And the points downthread
about our two existing systems being not entirely separate are on
point, too.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On 2017-02-10 12:07:15 -0500, Robert Haas wrote:
> On Mon, Jan 30, 2017 at 10:26 AM, Peter Eisentraut
> <peter.eisentraut@2ndquadrant.com> wrote:
> > On 1/30/17 1:28 AM, Andres Freund wrote:
> >> Given that fact, I just don't buy why it's a good idea to not also
> >> replace autoconf initially.
> >
> > Well, I find it a bit scary.  If you do the big switch all at once, then
> > you will have to dedicate the following 3 months to fixing complaints
> > from developers and build farmers.
> 
> I agree with that.  I think replacing the Windows build system first
> and then the non-Windows build system later is a better plan than
> replacing both at the same time.

Obviously I disagree ;)


But I'm more replying because of:

> But also, I'm not really sure whether this conversion makes sense.  I
> mean, any build system is going to require some work, and accordingly
> our present build systems require some work.  cmake will require
> different work, but not necessarily less.  The current system has a
> long history; we pretty much know it works.  Switching will inevitably
> break some things.  Maybe we'll end up better off in the long term,
> but maybe we won't.  Things are pretty good now, so it seems like it
> would be easy for them to get worse rather than better.

I do think it's kinda ok for people who've dealt with this for some
time.  But for the first few times having to write autoconf macros is
quite intimidating. A somewhat less obscure way to deal with that is
worth something.  As is better file dependency management, across
different environments.  As is the IDE integration cmake is more and
more getting.  As is properly builtin working caching of configure tests
(llvm first cmake run, 7.7s, second 2.54s). As is the fact that a lot of
big projects (llvm, kde, qt, and a lot of others) migrated to it.

For me personally those benefits aren't worth that much.  I've learned
autoconf stuff.  I've scripting around autoconf caching.  But for newer
people that's not true.

Greetings,

Andres Freund

On 02/10/2017 08:27 PM, Magnus Hagander wrote:
> For me a killer feature would be if/when we can get to a point where we can
> have something pgxs-style on cmake that also works on windows.
>
> Our homemade Windows build system works OK for postgres, and while ugly it
> is as you say well tested by now. But it doesn't do *anything* to help
> people build extensions on Windows.

Do we need to build PostgreSQL itself using cmake, to achieve that? 
Could we write something like pgxs for cmake, only for extensions?

- Heikki

On 2017-02-10 19:27:55 +0100, Magnus Hagander wrote:
> For me a killer feature would be if/when we can get to a point where we can
> have something pgxs-style on cmake that also works on windows.

That should be quite possible.  The ugliest part will be to determine
where to include a cmake file from (since that'll be copied in every
such project), but after that it should be nearly trivial.

- Andres

On 2017-02-10 20:31:12 +0200, Heikki Linnakangas wrote:
> On 02/10/2017 08:27 PM, Magnus Hagander wrote:
> > For me a killer feature would be if/when we can get to a point where we can
> > have something pgxs-style on cmake that also works on windows.
> > 
> > Our homemade Windows build system works OK for postgres, and while ugly it
> > is as you say well tested by now. But it doesn't do *anything* to help
> > people build extensions on Windows.
> 
> Do we need to build PostgreSQL itself using cmake, to achieve that? Could we
> write something like pgxs for cmake, only for extensions?

I don't see why it'd need to be done together.  The minimal version
would be a simple cmake file that just sets a bunch of variables from
pg_config, provides a few rules for specifying the current pgxs stuff,
and an example stanza how to include that file.  We'd have to duplicate
some of the pgxs specific logic, but that's probably not too bad.

- Andres

On 2017-02-10 19:33:18 +0100, Magnus Hagander wrote:
> I guess we wouldn't, but we'd still need the "replacement for autoconf"
> part. So then we're back to maintaining multiple buildsystems.

Hm? Do we really need that?  Most of the things in an extension you do
*not* want to determine separately from the backend.  It's not like pgxs
atm really allows to differ wildly from autoconf's results. And most of
the relevant determinations made by autoconf are available in headers
and/or we can generate a cmake include file with the results of
autoconf.

- Andres

On 02/10/2017 01:27 PM, Magnus Hagander wrote:
> On Fri, Feb 10, 2017 at 6:07 PM, Robert Haas <robertmhaas@gmail.com
> <mailto:robertmhaas@gmail.com>> wrote:
>
>     On Mon, Jan 30, 2017 at 10:26 AM, Peter Eisentraut
>     <peter.eisentraut@2ndquadrant.com
>     <mailto:peter.eisentraut@2ndquadrant.com>> wrote:
>     > On 1/30/17 1:28 AM, Andres Freund wrote:
>     >> Given that fact, I just don't buy why it's a good idea to not also
>     >> replace autoconf initially.
>     >
>     > Well, I find it a bit scary.  If you do the big switch all at
>     once, then
>     > you will have to dedicate the following 3 months to fixing
>     complaints
>     > from developers and build farmers.
>
>     I agree with that.  I think replacing the Windows build system first
>     and then the non-Windows build system later is a better plan than
>     replacing both at the same time.
>
>     But also, I'm not really sure whether this conversion makes sense.  I
>     mean, any build system is going to require some work, and accordingly
>     our present build systems require some work.  cmake will require
>     different work, but not necessarily less.  The current system has a
>     long history; we pretty much know it works.  Switching will inevitably
>     break some things.  Maybe we'll end up better off in the long term,
>     but maybe we won't.  Things are pretty good now, so it seems like it
>     would be easy for them to get worse rather than better.  If nothing
>     else, everybody who has to learn the new system either to use it for
>     development or because they are doing packaging will have to do some
>     amount of extra work as a result of any switch.
>
>
> For me a killer feature would be if/when we can get to a point where
> we can have something pgxs-style on cmake that also works on windows.
>
> Our homemade Windows build system works OK for postgres, and while
> ugly it is as you say well tested by now. But it doesn't do *anything*
> to help people build extensions on Windows. 
>
>


Watch this space. There is work being done on building extensions out of
tree using CMake on Windows. It's pretty neat, and I'm hoping to demo it
publicly soon. But it's not reliant on CMake in core.

cheers

andrew

-- 
Andrew Dunstan                https://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

On 2017-02-10 10:22:34 -0800, Andres Freund wrote:
> On 2017-02-10 12:07:15 -0500, Robert Haas wrote:
> > But also, I'm not really sure whether this conversion makes sense.  I
> > mean, any build system is going to require some work, and accordingly
> > our present build systems require some work.  cmake will require
> > different work, but not necessarily less.  The current system has a
> > long history; we pretty much know it works.  Switching will inevitably
> > break some things.  Maybe we'll end up better off in the long term,
> > but maybe we won't.  Things are pretty good now, so it seems like it
> > would be easy for them to get worse rather than better.
> 
> I do think it's kinda ok for people who've dealt with this for some
> time.  But for the first few times having to write autoconf macros is
> quite intimidating. A somewhat less obscure way to deal with that is
> worth something.  As is better file dependency management, across
> different environments.  As is the IDE integration cmake is more and
> more getting.  As is properly builtin working caching of configure tests
> (llvm first cmake run, 7.7s, second 2.54s). As is the fact that a lot of
> big projects (llvm, kde, qt, and a lot of others) migrated to it.
> 
> For me personally those benefits aren't worth that much.  I've learned
> autoconf stuff.  I've scripting around autoconf caching.  But for newer
> people that's not true.

Craig's email just now reminded me of another advantage: Using cmake
across the board, would mean we'd use the same ./configure alike logic
on both windows and linux.  To me that seems quite and advantage over
managing pg_config.h.win32/config_default.pl manually/separately - we
obviously have screwed up quite badly there in the past
(cf376a4adc0805b0).

On 2/8/17 4:44 PM, Andres Freund wrote:
>> Well, I find it a bit scary.  If you do the big switch all at once, then
>> you will have to dedicate the following 3 months to fixing complaints
>> from developers and build farmers.
> 
> That'll be the case just as well if we spread it out over two cycles,
> except that we'll have it in multiple phases, and we run into the danger
> of a half-done conversion.

They key term is "dedicated".  I don't think anyone wants to spend 3
months doing nothing but fixing the build system.  If we spread it over
several releases and bring more people up to speed along the way, that
looks more manageable.

>> That would work if there were a single entry point into the build
>> system.  But in practice there are many, and every one of them is
>> someone's favorite.  It's unlikely that we will be able to enumerate all
>> of them during patch review.
> 
> Not sure what you mean with "entry point"?

People have all kinds of expectations on how the build system behaves.
It's not just ./configure; make; make install.  All kinds of niche and
edge cases have evolved over the years.  Variables you can set,
overrides, targets in some subdirectories, building in subdirectories
and having it build another subdirectory first, testing this way and
testing that way, testing with a custom locale or a custom database
name, building before testing or not, testing without reinstalling, and
so on and so on.  You'd have to make sure at least some of that
continues to work or be able to explain it away.  And most of it is not
really documented.

-- 
Peter Eisentraut              http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

On Mon, Feb 13, 2017 at 10:52 AM, Peter Eisentraut
<peter.eisentraut@2ndquadrant.com> wrote:
> People have all kinds of expectations on how the build system behaves.
> It's not just ./configure; make; make install.  All kinds of niche and
> edge cases have evolved over the years.  Variables you can set,
> overrides, targets in some subdirectories, building in subdirectories
> and having it build another subdirectory first, testing this way and
> testing that way, testing with a custom locale or a custom database
> name, building before testing or not, testing without reinstalling, and
> so on and so on.  You'd have to make sure at least some of that
> continues to work or be able to explain it away.  And most of it is not
> really documented.

...which is another argument for just not changing anything.  I mean,
again, the current Windows build system is unbeautiful and
occasionally takes some work, but if we switch to cmake, that has to
get enough better to make up for all of the things that get worse.
And it's far from obvious than this will be so, especially when you
consider the fact that many people have detailed knowledge of how to
tweak the current system to do what they want.  As you point out here,
a lot of that stuff may stop working if we replace the system
wholesale.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On Wed, Feb  8, 2017 at 04:52:19PM -0500, Tom Lane wrote:
> Peter Eisentraut <peter.eisentraut@2ndquadrant.com> writes:
> > On 2/8/17 6:21 AM, Yuriy Zhuravlev wrote:
> >> Support two build systems it's not big deal really. I have been working
> >> on this past year without any big troubles. 
> >> Also we have second perl build system...
> 
> > The perl/msvc build system pulls in information from the makefiles.  So
> > when you add a file or something basic like that, you don't have to
> > update it.  So it's really more like 1.5 build systems.
> 
> Really it's more like 1.1 build systems, in that the MSVC scripts do that
> just well enough that you *usually* don't have to think about them.  But
> then when they fail, and you have to figure out why, it can be a pain.

If cmake isn't going to be able to query the Makefiles and adjust to
changes we make there, changing our Windows build system from MSVC to
cmake takes us from maintaining 1.1 build systems to two build systems,
and I don't think anyone wants that.

If we go to cmake, I think we need to agree we will eventually _only_
use cmake.  I don't think having two build systems we have to maintain
is better than 1.1 build systems where we can mostly ignore 0.1 of that.

--  Bruce Momjian  <bruce@momjian.us>        http://momjian.us EnterpriseDB
http://enterprisedb.com

+ As you are, so once was I.  As I am, so you will be. +
+                      Ancient Roman grave inscription +