Thread: Documentation for building with meson

Documentation for building with meson

From
samay sharma
Date:
Hi,

Creating a new thread focussed on adding docs for building Postgres with meson. This is a spinoff from the original thread [1] and I've attempted to address all the feedback provided there in the attached patch.

Please let me know your thoughts.


Regards,
Samay
Attachment

Re: Documentation for building with meson

From
Justin Pryzby
Date:
On Wed, Oct 19, 2022 at 11:35:10AM -0700, samay sharma wrote:
> Creating a new thread focussed on adding docs for building Postgres with
> meson. This is a spinoff from the original thread [1] and I've attempted to
> address all the feedback provided there in the attached patch.
> 
> Please let me know your thoughts.

It's easier to review rendered documentation.
I made a rendered copy available here:
https://api.cirrus-ci.com/v1/artifact/task/6084297781149696/html_docs/html_docs/install-meson.html

+      <application>Flex</application> and <application>Bison</application>
+      are needed to build <productname>PostgreSQL</productname> using
+      <application>meson</application>. Be sure to get
+      <application>Flex</application> 2.5.31 or later and
+      <application>Bison</application> 1.875 or later from your package manager.
+      Other <application>lex</application> and <application>yacc</application>
+      programs cannot be used.

These versions need to be updated, see also: 57bab3330:
  - b086a47a270 "Bump minimum version of Bison to 2.3"
  - 8b878bffa8d "Bump minimum version of Flex to 2.5.35"

+         will be enabled automatically if the required packages are found.

should refer to files/libraries/headers/dependencies rather than
"packages" ?

+         default is false that is to use <application>Readline</application>.

"that is to use" should be parenthesized or separate with commas, like
| default is false, that is to use <application>Readline</application>.

zlib is mentioned twice, the first being "strongly recommended".
Is that intended?  Also, basebackup can use zlib, too.

+         If you have the binaries for certain by programs required to build

remove "by" ?

+         Postgres (with or without optional flags) stored at non-standard
+         paths, you could specify them manually to meson configure. The complete
+         list of programs for whom this is supported can be found by running

for *which

Actually, I suggest to say:
|If a program required to build Postgres (with or without optional flags)
|is stored in a non-standard path, ...

+     a build with a different value of these options.

.. with different values ..

+     the server, it is recommended to use atleast the <option>--buildtype=debug</option>

at least

+    and it's options in the meson documentation.

its

Maybe other things should have <productname> ?

 Git
 Libxml2
 libxslt
 visual studio
 DTrace
 ninja

+      <application>Flex</application> and <application>Bison</application>

Maybe these should use <productname> ?

+      be installed with <literal>pip</literal>.

Should be <application> ?

This part is redundant with prior text:
" To use this option, you will need an implementation of the Gettext API. "

+ Enabls use of the Zlib library

typo: Enabls

+ This option is set to true by default and setting it to false will 

change "and" to ";" for spinlocks and atomics?

+ Debug info is generated but the result is not optimized. 

Maybe say the "code" is not optimized ?

+ the tests can slow down the server significantly

remove "can"

+ You can override the amount of parallel processes used with

s/amount/number/

+ If you'd like to build with a backend other that ninja

other *than

+      the <acronym>GNU</acronym> C library then you will additionally

library comma

+    argument. If no <literal>srcdir</literal> is given Meson will deduce the

given comma

+     It should be noted that after the initial configure step

step comma

+    After the installation you can free disk space by removing the built

installation comma

+    To learn more about running the tests and how to interpret the results
+    you can refer to the documentation for interpreting test results.

interpret the results comma
"for interpreting test results" seems redundant.

+ ninja install should work for most cases but if you'd like to use more options

cases comma

Starting with "Developer Options", this intermixes postgres
project-specific options like cassert and tap-tests with meson's stuff
like buildtype and werror.  IMO there's too much detail about meson's
options, which I think is better left to that project's own
documentation, and postgres docs should include only a brief mention and
a reference to their docs.

+ Ninja will automatically detect the number of CPUs in your computer and
+ parallelize itself accordingly. You can override the amount of parallel
+ processes used with the command line argument -j. 

too much detail for my taste..

-- 
Justin



Re: Documentation for building with meson

From
samay sharma
Date:
Hi,

On Wed, Oct 19, 2022 at 7:43 PM Justin Pryzby <pryzby@telsasoft.com> wrote:
On Wed, Oct 19, 2022 at 11:35:10AM -0700, samay sharma wrote:
> Creating a new thread focussed on adding docs for building Postgres with
> meson. This is a spinoff from the original thread [1] and I've attempted to
> address all the feedback provided there in the attached patch.
>
> Please let me know your thoughts.

It's easier to review rendered documentation.
I made a rendered copy available here:
https://api.cirrus-ci.com/v1/artifact/task/6084297781149696/html_docs/html_docs/install-meson.html

Thanks for your for review. Attached v2 of the patch here.
 


+      <application>Flex</application> and <application>Bison</application>
+      are needed to build <productname>PostgreSQL</productname> using
+      <application>meson</application>. Be sure to get
+      <application>Flex</application> 2.5.31 or later and
+      <application>Bison</application> 1.875 or later from your package manager.
+      Other <application>lex</application> and <application>yacc</application>
+      programs cannot be used.

These versions need to be updated, see also: 57bab3330:
  - b086a47a270 "Bump minimum version of Bison to 2.3"
  - 8b878bffa8d "Bump minimum version of Flex to 2.5.35"

Changed
 

+         will be enabled automatically if the required packages are found.

should refer to files/libraries/headers/dependencies rather than
"packages" ?

Changed to dependencies
 

+         default is false that is to use <application>Readline</application>.

"that is to use" should be parenthesized or separate with commas, like
| default is false, that is to use <application>Readline</application>.

zlib is mentioned twice, the first being "strongly recommended".
Is that intended?  Also, basebackup can use zlib, too.

Yes, the first is in the requirements section where we just list packages required / recommended. The other mention is in the list of configure options. This is similar to how the documentation looks today for make / autoconf. Added pg_basebackup as a use case too.
 

+         If you have the binaries for certain by programs required to build

remove "by" ?

Done
 

+         Postgres (with or without optional flags) stored at non-standard
+         paths, you could specify them manually to meson configure. The complete
+         list of programs for whom this is supported can be found by running

for *which

Actually, I suggest to say:
|If a program required to build Postgres (with or without optional flags)
|is stored in a non-standard path, ...

Liked this framing better. Changed. 

+     a build with a different value of these options.

.. with different values ..

Done 

+     the server, it is recommended to use atleast the <option>--buildtype=debug</option>

at least
Done 

+    and it's options in the meson documentation.

its
Done 

Maybe other things should have <productname> ?

 Git
 Libxml2
 libxslt
 visual studio
 DTrace
 ninja

+      <application>Flex</application> and <application>Bison</application>

Maybe these should use <productname> ?

I'm unsure of the right protocol for this. I tried to follow the precedent set in the make / autoconf part of the documentation, which uses <productname> at certain places and <application> at others. Is there a reference or guidance on which to use where or is it mostly a case by case decision?


+      be installed with <literal>pip</literal>.

Should be <application> ?

Changed. 

This part is redundant with prior text:
" To use this option, you will need an implementation of the Gettext API. "

Modified. 

+ Enabls use of the Zlib library

typo: Enabls

Fixed. 

+ This option is set to true by default and setting it to false will

change "and" to ";" for spinlocks and atomics?

Done 

+ Debug info is generated but the result is not optimized.

Maybe say the "code" is not optimized ?

Changed 

+ the tests can slow down the server significantly

remove "can"

Done.
 

+ You can override the amount of parallel processes used with

s/amount/number/

Done 

+ If you'd like to build with a backend other that ninja

other *than

Fixed. 

+      the <acronym>GNU</acronym> C library then you will additionally

library comma

Added 

+    argument. If no <literal>srcdir</literal> is given Meson will deduce the

given comma

Added 

+     It should be noted that after the initial configure step

step comma

Added 

+    After the installation you can free disk space by removing the built

installation comma

Added 

+    To learn more about running the tests and how to interpret the results
+    you can refer to the documentation for interpreting test results.

interpret the results comma
"for interpreting test results" seems redundant.

Changed. 

+ ninja install should work for most cases but if you'd like to use more options

cases comma

Added 

Starting with "Developer Options", this intermixes postgres
project-specific options like cassert and tap-tests with meson's stuff
like buildtype and werror.  IMO there's too much detail about meson's
options, which I think is better left to that project's own
documentation, and postgres docs should include only a brief mention and
a reference to their docs.

The meson specific options I've chosen to document are: auto_features, backend, c_args, c_link_args, buildtype, optimization, werror, errorlogs and b_coverage as I felt they might be used often and are useful to know. But, it's very possible that some of them might be obvious and others may not be as useful as I thought. Are there specific ones you'd suggest we can remove? Also, if you're curious, this is the list I picked from: https://mesonbuild.com/Commands.html#configure.

In terms of detail about individual options, I think the descriptions about most of them are brief but buildtype was pretty verbose. I have shortened it.
 

+ Ninja will automatically detect the number of CPUs in your computer and
+ parallelize itself accordingly. You can override the amount of parallel
+ processes used with the command line argument -j.

too much detail for my taste..

I added this as make / autoconf doesn't do something like this. So, it might be useful to know for people switching over.

Regards,
Samay
 

--
Justin
Attachment

Re: Documentation for building with meson

From
John Naylor
Date:
+# Run the main pg_regress and isolation tests
+<userinput>meson test --suite main</userinput>

This does not work for me in a fresh install until running

meson test --suite setup

In fact, we see in 

https://wiki.postgresql.org/wiki/Meson

meson test --suite setup --suite main

That was just an eyeball check from a naive user -- it would be good to try running everything documented here.

--
John Naylor
EDB: http://www.enterprisedb.com

Re: Documentation for building with meson

From
Jacob Champion
Date:
On Thu, Oct 27, 2022 at 1:04 AM John Naylor
<john.naylor@enterprisedb.com> wrote:
> This does not work for me in a fresh install until running
>
> meson test --suite setup
>
> In fact, we see in
>
> https://wiki.postgresql.org/wiki/Meson
>
> meson test --suite setup --suite main

(Is there a way to declare a dependency on the setup suite in Meson,
so that we don't have to specify it manually? I was bitten by this
recently; if you make a code change and forget to run setup, it'll
recompile locally but then skip reinstallation, giving false test
results.)

--Jacob



Re: Documentation for building with meson

From
Andres Freund
Date:
Hi,

On 2022-10-27 14:15:32 -0700, Jacob Champion wrote:
> On Thu, Oct 27, 2022 at 1:04 AM John Naylor
> <john.naylor@enterprisedb.com> wrote:
> > This does not work for me in a fresh install until running
> >
> > meson test --suite setup
> >
> > In fact, we see in
> >
> > https://wiki.postgresql.org/wiki/Meson
> >
> > meson test --suite setup --suite main
> 
> (Is there a way to declare a dependency on the setup suite in Meson,
> so that we don't have to specify it manually? I was bitten by this
> recently; if you make a code change and forget to run setup, it'll
> recompile locally but then skip reinstallation, giving false test
> results.)

Tests can have dependencies, and they're correctly built. The problem however
is that, for historical reasons if I understand correctly, dependencies of
tests are automatically included in the default 'all' target. Which means if
you just type in 'ninja', it'd automatically create the test installation -
which is probably not what we want, given that that's not a fast step on some
platforms.

Greetings,

Andres Freund



Re: Documentation for building with meson

From
Jacob Champion
Date:
On Thu, Oct 27, 2022 at 4:03 PM Andres Freund <andres@anarazel.de> wrote:
> Tests can have dependencies, and they're correctly built. The problem however
> is that, for historical reasons if I understand correctly, dependencies of
> tests are automatically included in the default 'all' target. Which means if
> you just type in 'ninja', it'd automatically create the test installation -
> which is probably not what we want, given that that's not a fast step on some
> platforms.

And I see that between-suite dependencies were rejected as a feature
[1]. Ah well, `--suite setup` is not so bad once you learn it.

Thanks!
--Jacob

[1] https://github.com/mesonbuild/meson/issues/2740



Re: Documentation for building with meson

From
samay sharma
Date:
Hi,

On Thu, Oct 27, 2022 at 1:04 AM John Naylor <john.naylor@enterprisedb.com> wrote:
+# Run the main pg_regress and isolation tests
+<userinput>meson test --suite main</userinput>

This does not work for me in a fresh install until running

meson test --suite setup

In fact, we see in 

https://wiki.postgresql.org/wiki/Meson

meson test --suite setup --suite main

You are right that this will be needed for a new install. I've added --suite setup in the testing section in the v3 of the patch (attached).


That was just an eyeball check from a naive user -- it would be good to try running everything documented here.

I retried all the instructions as suggested and they work for me.

Regards,
Samay
 

--
John Naylor
EDB: http://www.enterprisedb.com
Attachment

Re: Documentation for building with meson

From
Andres Freund
Date:
Hi,

On 2022-10-30 20:51:59 -0700, samay sharma wrote:
> +# setup and enter build directory (done only first time)
> +meson setup build src --prefix=$PWD/install

This command won't work on windows, I think.


> + <sect2 id="configure-meson">
> +  <title>Configuring the build</title>
> +
> +   <para>
> +    The first step of the installation procedure is to configure the
> +    source tree for your system and choose the options you would like. To

s/source tree/build tree/?


> +    create and configure the build directory, you can start with the
> +    <literal>meson setup</literal> command.
> +   </para>
> +
> +<screen>
> +<userinput>meson setup build</userinput>
> +</screen>
> +
> +   <para>
> +    The setup command takes a <literal>builddir</literal> and a <literal>srcdir</literal>
> +    argument. If no <literal>srcdir</literal> is given, Meson will deduce the
> +    <literal>srcdir</literal> based on the current directory and the location
> +    of <literal>meson.build</literal>. The <literal>builddir</literal> is mandatory.
> +   </para>
> +
> +   <para>
> +    Meson then loads the build configuration file and sets up the build directory.
> +    Additionally, the invocation can pass options to Meson. The list of commonly
> +    used options is in subsequent sections. A few examples of specifying different
> +    build options are:

Somehow the "tone" is descriptive in a distanced way, rather than instructing
what to do.


> +   <sect3 id="configure-install-locations">
> +    <title>Installation Locations</title>
> +
> +     <para>
> +      These options control where <literal>ninja install (or meson install)</literal> will put
> +      the files.  The <option>--prefix</option> option is sufficient for
> +      most cases.

Perhaps the short version use of prefix could be a link here? Not sure if
that's a good idea.



> +     <variablelist>
> +      <varlistentry>
> +       <term><option>--prefix=<replaceable>PREFIX</replaceable></option></term>
> +       <listitem>
> +        <para>
> +         Install all files under the directory <replaceable>PREFIX</replaceable>
> +         instead of <filename>/usr/local/pgsql</filename>. The actual
> +         files will be installed into various subdirectories; no files
> +         will ever be installed directly into the
> +         <replaceable>PREFIX</replaceable> directory.
> +        </para>
> +       </listitem>
> +      </varlistentry>

Hm, need to mention windows here likely. By default the installation will go
to <current drive letter>:/usr/local/pgsql.


> +      <varlistentry>
> +       <term><option>--bindir=<replaceable>DIRECTORY</replaceable></option></term>
> +       <listitem>
> +        <para>
> +         Specifies the directory for executable programs. The default
> +         is <filename><replaceable>PREFIX</replaceable>/bin</filename>, which
> +         normally means <filename>/usr/local/pgsql/bin</filename>.
> +        </para>
> +       </listitem>
> +      </varlistentry>

Hm, do we really want the "which normally means" part? That'll make the OS
stuff more complicated.


> +      <varlistentry>
> +       <term><option>--sysconfdir=<replaceable>DIRECTORY</replaceable></option></term>
> +       <listitem>
> +        <para>
> +         Sets the directory for various configuration files,
> +         <filename><replaceable>PREFIX</replaceable>/etc</filename> by default.
> +        </para>
> +       </listitem>
> +      </varlistentry>

Need to check what windows does here.


> +      <varlistentry>
> +       <term><option>-Dnls=<replaceable>auto/enabled/disabled</replaceable></option></term>

Wonder if we should define a entity for
<replaceable>auto/enabled/disabled</replaceable>? There's a lot of repetitions
of it.


> +       <listitem>
> +        <para>
> +         Enables or disables Native Language Support (<acronym>NLS</acronym>),
> +         that is, the ability to display a program's messages in a
> +         language other than English. It defaults to auto, meaning that it
> +         will be enabled automatically if an implementation of the
> +         <application>Gettext API</application> is found.
> +        </para>

Do we really want to repeat the "It defaults to auto, meaning that it will be
enabled automatically if ..." for each of these? Perhaps we could say
'defaults to <xref ...>auto</xref>'?


> +        <para>
> +         By default,
> +         <productname>pkg-config</productname><indexterm><primary>pkg-config</primary></indexterm>
> +         will be used to find the required compilation options.  This is
> +         supported for <productname>ICU4C</productname> version 4.6 and later.
> +         <!-- Add description for older ICU4C versions and when pkg-config isn't available-->
> +        </para>

I'd just remove this paragraph. Right now the meson build will just use solely
use pkg-config config files for icu.  I don't think we need to care about 4.6
(from 2010) anymore.


> +      <varlistentry id="configure-with-llvm-meson">
> +       <term><option>-Dllvm=<replaceable>auto/enabled/disabled</replaceable></option></term>
> +       <listitem>
> +        <para>
> +         Build with support for <productname>LLVM</productname> based
> +         <acronym>JIT</acronym> compilation<phrase
> +         condition="standalone-ignore"> (see <xref
> +         linkend="jit"/>)</phrase>.  This
> +         requires the <productname>LLVM</productname> library to be installed.
> +         The minimum required version of <productname>LLVM</productname> is
> +         currently 3.9. It is set to disabled by default.
> +        </para>
> +        <para>
> +         <command>llvm-config</command><indexterm><primary>llvm-config</primary></indexterm>
> +         will be used to find the required compilation options.
> +         <command>llvm-config</command>, and then
> +         <command>llvm-config-$major-$minor</command> for all supported
> +         versions, will be searched for in your <envar>PATH</envar>.
> +         <!--Add substitute fo LLVM_CONFIG when llvm-config is not in PATH-->
> +        </para>

Probably a link to the docs for meson native files suffices here for
now. Since the autoconf docs have been written there's only
llvm-config-$version, llvm stopped having separate major/minor versions
somewhere around llvm 4. I think it'd suffice to say llvm-config-$version?


> +        <para>
> +         <productname>LLVM</productname> support requires a compatible
> +         <command>clang</command> compiler (specified, if necessary, using the
> +         <envar>CLANG</envar> environment variable), and a working C++
> +         compiler (specified, if necessary, using the <envar>CXX</envar>
> +         environment variable).
> +        </para>

For clang we don't look for CLANG anymore, we use for the clang compiler
belonging to the llvm installation of llvm-config.


> +       <listitem>
> +        <para>
> +         Build with support for <acronym>SSL</acronym> (encrypted)
> +         connections. The only <replaceable>LIBRARY</replaceable>
> +         supported is <option>openssl</option>. This requires the
> +         <productname>OpenSSL</productname> package to be installed.
> +         <filename>configure</filename> will check for the required

The <filename>configure</filename> reference is out of date.

> +         header files and libraries to make sure that your
> +         <productname>OpenSSL</productname> installation is sufficient
> +         before proceeding. The default for this option is none.
> +        </para>
> +       </listitem>
> +      </varlistentry>
>

> +      <varlistentry>
> +       <term><option>-Dgssapi=<replaceable>auto/enabled/disabled</replaceable></option></term>
> +       <listitem>
> +        <para>
> +         Build with support for GSSAPI authentication. On many systems, the
> +         GSSAPI system (usually a part of the Kerberos installation) is not
> +         installed in a location
> +         that is searched by default (e.g., <filename>/usr/include</filename>,
> +         <filename>/usr/lib</filename>), so you must use the options
> +         <option>-Dextra_include_dirs</option> and <option>-Dextra_lib_dirs</option> in
> +         addition to this option.  <filename>meson configure</filename> will check
> +         for the required header files and libraries to make sure that
> +         your GSSAPI installation is sufficient before proceeding.
> +         It defaults to auto, meaning that it will be enabled automatically if the
> +         required dependencies are found.
> +        </para>
> +       </listitem>
> +      </varlistentry>

Right now we only work for gssapi installations providing a pkg-config config
file. We could change that if we encounter a system where that's insufficient.


> +      <varlistentry>
> +       <term><option>-Duuid=<replaceable>LIBRARY</replaceable></option></term>
> +       <listitem>
> +        <para>
> +         Build the <xref linkend="uuid-ossp"/> module
> +         (which provides functions to generate UUIDs), using the specified
> +         UUID library.<indexterm><primary>UUID</primary></indexterm>
> +         <replaceable>LIBRARY</replaceable> must be one of:
> +        </para>
> +        <itemizedlist>
> +         <listitem>
> +          <para>
> +           <option>none</option> to not build the ussp module. This is the default.
> +          </para>
> +         </listitem>

s/ussp/uuid/?



> +        <para>
> +         To detect the required compiler and linker options, PostgreSQL will
> +         query <command>pkg-config</command>, if that is installed and knows
> +         about libxml2.  Otherwise the program <command>xml2-config</command>,
> +         which is installed by libxml2, will be used if it is found.  Use
> +         of <command>pkg-config</command> is preferred, because it can deal
> +         with multi-architecture installations better.
> +        </para>

Right now only pkg-config is supported with meson.


> +
> +      <varlistentry>
> +       <term><option>-Dspinlocks=<replaceable>true/false</replaceable></option></term>
> +       <listitem>
> +        <para>
> +         This option is set to true by default; setting it to false will
> +         allow the build to succeed even if <productname>PostgreSQL</productname>
> +         has no CPU spinlock support for the platform.  The lack of
> +         spinlock support will result in very poor performance; therefore,
> +         this option should only be changed if the build aborts and
> +         informs you that the platform lacks spinlock support. If setting this
> +         option to false is required to build <productname>PostgreSQL</productname> on
> +         your platform, please report the problem to the
> +         <productname>PostgreSQL</productname> developers.
> +        </para>
> +       </listitem>
> +      </varlistentry>
> +
> +      <varlistentry>
> +       <term><option>-Datomics=<replaceable>true/false</replaceable></option></term>
> +       <listitem>
> +        <para>
> +         This option is set to true by default; setting it to false will
> +         disable use of CPU atomic operations.  The option does nothing on
> +         platforms that lack such operations.  On platforms that do have
> +         them, disabling atomics will result in poor performance.  Changing
> +         this option is only useful for debugging or making performance comparisons.
> +        </para>
> +       </listitem>
> +      </varlistentry>
> +    </variablelist>

I think these should rather be in the developer section? They're not
dependencies and as you noted, they're not normally useful.


> +      <varlistentry>
> +       <term><option>-Dextra_include_dirs=<replaceable>DIRECTORIES</replaceable></option></term>
> +       <listitem>
> +        <para>
> +         <replaceable>DIRECTORIES</replaceable> is a colon-separated list of
> +         directories that will be added to the list the compiler
> +         searches for header files. If you have optional packages
> +         (such as GNU <application>Readline</application>) installed in a non-standard
> +         location,
> +         you have to use this option and probably also the corresponding
> +         <option>-Dextra_lib_dirs</option> option.
> +        </para>
> +        <para>
> +         Example: <literal>-Dextra_include_dirs=/opt/gnu/include:/usr/sup/include</literal>.
> +        </para>
> +       </listitem>
> +      </varlistentry>

The separator for meson is a comma, rather than a :

> +      <varlistentry>
> +       <term><option>-Dextra_lib_dirs=<replaceable>DIRECTORIES</replaceable></option></term>
> +       <listitem>
> +        <para>
> +         <replaceable>DIRECTORIES</replaceable> is a colon-separated list of
> +         directories to search for libraries. You will probably have
> +         to use this option (and the corresponding
> +         <option>-Dextra_include_dirs</option> option) if you have packages
> +         installed in non-standard locations.
> +        </para>
> +        <para>
> +         Example: <literal>-Dextra_lib_dirs=/opt/gnu/lib:/usr/sup/lib</literal>.
> +        </para>
> +       </listitem>
> +      </varlistentry>

Dito.


> +    <variablelist>
> +
> +      <varlistentry>
> +       <term><option>-Dsegsize=<replaceable>SEGSIZE</replaceable></option></term>
> +       <listitem>
> +        <para>
> +         Set the <firstterm>segment size</firstterm>, in gigabytes.  Large tables are
> +         divided into multiple operating-system files, each of size equal
> +         to the segment size.  This avoids problems with file size limits
> +         that exist on many platforms.  The default segment size, 1 gigabyte,
> +         is safe on all supported platforms.  If your operating system has
> +         <quote>largefile</quote> support (which most do, nowadays), you can use
> +         a larger segment size.  This can be helpful to reduce the number of
> +         file descriptors consumed when working with very large tables.
> +         But be careful not to select a value larger than is supported
> +         by your platform and the file systems you intend to use.  Other
> +         tools you might wish to use, such as <application>tar</application>, could
> +         also set limits on the usable file size.
> +         It is recommended, though not absolutely required, that this value
> +         be a power of 2.
> +        </para>
> +       </listitem>
> +      </varlistentry>
> +
> +      <varlistentry>
> +       <term><option>-Dblocksize=<replaceable>BLOCKSIZE</replaceable></option></term>
> +       <listitem>
> +        <para>
> +         Set the <firstterm>block size</firstterm>, in kilobytes.  This is the unit
> +         of storage and I/O within tables.  The default, 8 kilobytes,
> +         is suitable for most situations; but other values may be useful
> +         in special cases.
> +         The value must be a power of 2 between 1 and 32 (kilobytes).
> +        </para>
> +       </listitem>
> +      </varlistentry>
> +
> +      <varlistentry>
> +       <term><option>-Dwal_blocksize=<replaceable>BLOCKSIZE</replaceable></option></term>
> +       <listitem>
> +        <para>
> +         Set the <firstterm>WAL block size</firstterm>, in kilobytes.  This is the unit
> +         of storage and I/O within the WAL log.  The default, 8 kilobytes,
> +         is suitable for most situations; but other values may be useful
> +         in special cases.
> +         The value must be a power of 2 between 1 and 64 (kilobytes).
> +        </para>
> +       </listitem>
> +      </varlistentry>
> +
> +    </variablelist>

The order of the list entries seems a bit random? Perhaps just go for
alphabetical?


> +   </sect3>
> +
> +   <sect3 id="configure-devel">
> +    <title>Developer Options</title>
> +
> +    <para>
> +     Most of the options in this section are only of interest for
> +     developing or debugging <productname>PostgreSQL</productname>.
> +     They are not recommended for production builds, except
> +     for <option>--debug</option>, which can be useful to enable
> +     detailed bug reports in the unlucky event that you encounter a bug.
> +     On platforms supporting DTrace, <option>-Ddtrace</option>
> +     may also be reasonable to use in production.
> +    </para>
> +
> +    <para>
> +     When building an installation that will be used to develop code inside
> +     the server, it is recommended to use at least the <option>--buildtype=debug</option>
> +     and <option>-Dcassert</option> options.
> +    </para>
> +
> +     <variablelist>
> +      <varlistentry>
> +       <term><option>--buildtype=<replaceable>BUILDTYPE</replaceable></option></term>
> +       <listitem>
> +        <para>
> +         This option can be used to specify the buildtype to use; defaults
> +         to release. If you'd like finer control on the debug symbols
> +         and optimization levels than what this option provides, you can
> +         refer to the --debug and --optimization flags.
> +
> +         The following build types are generally used: plain, debug, debugoptimized
> +         and release. More information about them can be found in the
> +         <ulink url="https://mesonbuild.com/Running-Meson.html#configuring-the-build-directory">
> +         meson documentation</ulink>.
> +        </para>
> +       </listitem>
> +      </varlistentry>
> +
> +      <varlistentry>
> +       <term><option>--debug</option></term>
> +       <listitem>
> +        <para>
> +         Compiles all programs and libraries with debugging symbols.
> +         This means that you can run the programs in a debugger
> +         to analyze problems. This enlarges the size of the installed
> +         executables considerably, and on non-GCC compilers it usually
> +         also disables compiler optimization, causing slowdowns. However,
> +         having the symbols available is extremely helpful for dealing
> +         with any problems that might arise.  Currently, this option is
> +         recommended for production installations only if you use GCC.
> +         But you should always have it on if you are doing development work
> +         or running a beta version.
> +        </para>
> +       </listitem>
> +      </varlistentry>
> +
> +      <varlistentry>
> +       <term><option>--optimization</option>=<replaceable>LEVEL</replaceable></term>
> +       <listitem>
> +        <para>
> +         Specify the optimization level. LEVEL can be set to any of {0,g,1,2,3,s}.
> +        </para>
> +       </listitem>
> +      </varlistentry>

Wonder if we should just document optimization and debug, rather than
buildtype. The fact that debug/optimization override buildtype is a bit
confusing.


> +      <varlistentry>
> +       <term><option>-Dtap-tests</option></term>
> +       <listitem>
> +        <para>
> +         Enable tests using the Perl TAP tools.  This requires a Perl
> +         installation and the Perl module <literal>IPC::Run</literal>.
> +         <phrase condition="standalone-ignore">See <xref linkend="regress-tap"/> for more information.</phrase>
> +        </para>
> +       </listitem>
> +      </varlistentry>

This is an auto option as well.


> +      <varlistentry>
> +       <term><option>--errorlogs</option></term>
> +       <listitem>
> +        <para>
> +        This option can be used to print the logs from the failing tests
> +        making debugging easier.
> +        </para>
> +       </listitem>
> +      </varlistentry>

I don't think it's worth documenting this, it defaults to true anyway.


> +        <para>
> +         To point to the <command>dtrace</command> program, the
> +         environment variable <envar>DTRACE</envar> can be set.  This
> +         will often be necessary because <command>dtrace</command> is
> +         typically installed under <filename>/usr/sbin</filename>,
> +         which might not be in your <envar>PATH</envar>.
> +        </para>

We don't read the DTRACE environment variable, but the DTRACE option.



> +   <para>
> +    <literal>ninja install</literal> should work for most cases,
> +    but if you'd like to use more options, you could also use
> +    <literal>meson install</literal> instead. You can learn more about
> +    <ulink url="https://mesonbuild.com/Commands.html#install">meson install</ulink>
> +    and its options in the meson documentation.
> +   </para>

Maybe we should mention meson --quiet here? The verbosity of ninja install is
a bit annoying.


> +# Run the main pg_regress and isolation tests
> +<userinput>meson test --suite setup --suite main</userinput>

Since yesterday the main suite is no more. There's 'regress' and 'isolation'
now.


Greetings,

Andres Freund



Re: Documentation for building with meson

From
samay sharma
Date:
Hi,

On Sat, Nov 5, 2022 at 2:39 PM Andres Freund <andres@anarazel.de> wrote:
Hi,

On 2022-10-30 20:51:59 -0700, samay sharma wrote:
> +# setup and enter build directory (done only first time)
> +meson setup build src --prefix=$PWD/install

This command won't work on windows, I think.

I'll submit another version after testing on windows and seeing what else we need to fix. I've addressed all the other feedback in the attached v4.


> + <sect2 id="configure-meson">
> +  <title>Configuring the build</title>
> +
> +   <para>
> +    The first step of the installation procedure is to configure the
> +    source tree for your system and choose the options you would like. To

s/source tree/build tree/?

Done 

> +    create and configure the build directory, you can start with the
> +    <literal>meson setup</literal> command.
> +   </para>
> +
> +<screen>
> +<userinput>meson setup build</userinput>
> +</screen>
> +
> +   <para>
> +    The setup command takes a <literal>builddir</literal> and a <literal>srcdir</literal>
> +    argument. If no <literal>srcdir</literal> is given, Meson will deduce the
> +    <literal>srcdir</literal> based on the current directory and the location
> +    of <literal>meson.build</literal>. The <literal>builddir</literal> is mandatory.
> +   </para>
> +
> +   <para>
> +    Meson then loads the build configuration file and sets up the build directory.
> +    Additionally, the invocation can pass options to Meson. The list of commonly
> +    used options is in subsequent sections. A few examples of specifying different
> +    build options are:

Somehow the "tone" is descriptive in a distanced way, rather than instructing
what to do.

This was mostly copy pasted from meson docs. Rewrote it to make briefer and changed the tone to be more conversational. 


> +   <sect3 id="configure-install-locations">
> +    <title>Installation Locations</title>
> +
> +     <para>
> +      These options control where <literal>ninja install (or meson install)</literal> will put
> +      the files.  The <option>--prefix</option> option is sufficient for
> +      most cases.

Perhaps the short version use of prefix could be a link here? Not sure if
that's a good idea.

Added as an example 



> +     <variablelist>
> +      <varlistentry>
> +       <term><option>--prefix=<replaceable>PREFIX</replaceable></option></term>
> +       <listitem>
> +        <para>
> +         Install all files under the directory <replaceable>PREFIX</replaceable>
> +         instead of <filename>/usr/local/pgsql</filename>. The actual
> +         files will be installed into various subdirectories; no files
> +         will ever be installed directly into the
> +         <replaceable>PREFIX</replaceable> directory.
> +        </para>
> +       </listitem>
> +      </varlistentry>

Hm, need to mention windows here likely. By default the installation will go
to <current drive letter>:/usr/local/pgsql.


> +      <varlistentry>
> +       <term><option>--bindir=<replaceable>DIRECTORY</replaceable></option></term>
> +       <listitem>
> +        <para>
> +         Specifies the directory for executable programs. The default
> +         is <filename><replaceable>PREFIX</replaceable>/bin</filename>, which
> +         normally means <filename>/usr/local/pgsql/bin</filename>.
> +        </para>
> +       </listitem>
> +      </varlistentry>

Hm, do we really want the "which normally means" part? That'll make the OS
stuff more complicated.

Removed. We mention what the default is in the description of PREFIX, so it shouldn't be needed anyway.


> +      <varlistentry>
> +       <term><option>--sysconfdir=<replaceable>DIRECTORY</replaceable></option></term>
> +       <listitem>
> +        <para>
> +         Sets the directory for various configuration files,
> +         <filename><replaceable>PREFIX</replaceable>/etc</filename> by default.
> +        </para>
> +       </listitem>
> +      </varlistentry>

Need to check what windows does here.


> +      <varlistentry>
> +       <term><option>-Dnls=<replaceable>auto/enabled/disabled</replaceable></option></term>

Wonder if we should define a entity for
<replaceable>auto/enabled/disabled</replaceable>? There's a lot of repetitions
of it.

I couldn't come up with a good entity name which is significantly shorter. I think it's probably fine to have this as it clearly tells you the possible values you can set it to. I'll remove repetitive descriptions of what they mean.


> +       <listitem>
> +        <para>
> +         Enables or disables Native Language Support (<acronym>NLS</acronym>),
> +         that is, the ability to display a program's messages in a
> +         language other than English. It defaults to auto, meaning that it
> +         will be enabled automatically if an implementation of the
> +         <application>Gettext API</application> is found.
> +        </para>

Do we really want to repeat the "It defaults to auto, meaning that it will be
enabled automatically if ..." for each of these? Perhaps we could say
'defaults to <xref ...>auto</xref>'?

I added a description to the beginning of the Postgres features section and removed the repetitive "enabled automatically if ....". 


> +        <para>
> +         By default,
> +         <productname>pkg-config</productname><indexterm><primary>pkg-config</primary></indexterm>
> +         will be used to find the required compilation options.  This is
> +         supported for <productname>ICU4C</productname> version 4.6 and later.
> +         <!-- Add description for older ICU4C versions and when pkg-config isn't available-->
> +        </para>

I'd just remove this paragraph. Right now the meson build will just use solely
use pkg-config config files for icu.  I don't think we need to care about 4.6
(from 2010) anymore.

Removed 


> +      <varlistentry id="configure-with-llvm-meson">
> +       <term><option>-Dllvm=<replaceable>auto/enabled/disabled</replaceable></option></term>
> +       <listitem>
> +        <para>
> +         Build with support for <productname>LLVM</productname> based
> +         <acronym>JIT</acronym> compilation<phrase
> +         condition="standalone-ignore"> (see <xref
> +         linkend="jit"/>)</phrase>.  This
> +         requires the <productname>LLVM</productname> library to be installed.
> +         The minimum required version of <productname>LLVM</productname> is
> +         currently 3.9. It is set to disabled by default.
> +        </para>
> +        <para>
> +         <command>llvm-config</command><indexterm><primary>llvm-config</primary></indexterm>
> +         will be used to find the required compilation options.
> +         <command>llvm-config</command>, and then
> +         <command>llvm-config-$major-$minor</command> for all supported
> +         versions, will be searched for in your <envar>PATH</envar>.
> +         <!--Add substitute fo LLVM_CONFIG when llvm-config is not in PATH-->
> +        </para>

Probably a link to the docs for meson native files suffices here for
now. Since the autoconf docs have been written there's only
llvm-config-$version, llvm stopped having separate major/minor versions
somewhere around llvm 4. I think it'd suffice to say llvm-config-$version?

LLVM_CONFIG is now supported by newer versions of meson https://github.com/mesonbuild/meson/pull/10757. So, will just ask users to use that?

Changed to llvm-config-$version


> +        <para>
> +         <productname>LLVM</productname> support requires a compatible
> +         <command>clang</command> compiler (specified, if necessary, using the
> +         <envar>CLANG</envar> environment variable), and a working C++
> +         compiler (specified, if necessary, using the <envar>CXX</envar>
> +         environment variable).
> +        </para>

For clang we don't look for CLANG anymore, we use for the clang compiler
belonging to the llvm installation of llvm-config.

Removed the paragraph. 


> +       <listitem>
> +        <para>
> +         Build with support for <acronym>SSL</acronym> (encrypted)
> +         connections. The only <replaceable>LIBRARY</replaceable>
> +         supported is <option>openssl</option>. This requires the
> +         <productname>OpenSSL</productname> package to be installed.
> +         <filename>configure</filename> will check for the required

The <filename>configure</filename> reference is out of date.

Removed. 

> +         header files and libraries to make sure that your
> +         <productname>OpenSSL</productname> installation is sufficient
> +         before proceeding. The default for this option is none.
> +        </para>
> +       </listitem>
> +      </varlistentry>
>

> +      <varlistentry>
> +       <term><option>-Dgssapi=<replaceable>auto/enabled/disabled</replaceable></option></term>
> +       <listitem>
> +        <para>
> +         Build with support for GSSAPI authentication. On many systems, the
> +         GSSAPI system (usually a part of the Kerberos installation) is not
> +         installed in a location
> +         that is searched by default (e.g., <filename>/usr/include</filename>,
> +         <filename>/usr/lib</filename>), so you must use the options
> +         <option>-Dextra_include_dirs</option> and <option>-Dextra_lib_dirs</option> in
> +         addition to this option.  <filename>meson configure</filename> will check
> +         for the required header files and libraries to make sure that
> +         your GSSAPI installation is sufficient before proceeding.
> +         It defaults to auto, meaning that it will be enabled automatically if the
> +         required dependencies are found.
> +        </para>
> +       </listitem>
> +      </varlistentry>

Right now we only work for gssapi installations providing a pkg-config config
file. We could change that if we encounter a system where that's insufficient.

Changed to use pkg-config for non-standard paths. 


> +      <varlistentry>
> +       <term><option>-Duuid=<replaceable>LIBRARY</replaceable></option></term>
> +       <listitem>
> +        <para>
> +         Build the <xref linkend="uuid-ossp"/> module
> +         (which provides functions to generate UUIDs), using the specified
> +         UUID library.<indexterm><primary>UUID</primary></indexterm>
> +         <replaceable>LIBRARY</replaceable> must be one of:
> +        </para>
> +        <itemizedlist>
> +         <listitem>
> +          <para>
> +           <option>none</option> to not build the ussp module. This is the default.
> +          </para>
> +         </listitem>

s/ussp/uuid/?

Changed. 



> +        <para>
> +         To detect the required compiler and linker options, PostgreSQL will
> +         query <command>pkg-config</command>, if that is installed and knows
> +         about libxml2.  Otherwise the program <command>xml2-config</command>,
> +         which is installed by libxml2, will be used if it is found.  Use
> +         of <command>pkg-config</command> is preferred, because it can deal
> +         with multi-architecture installations better.
> +        </para>

Right now only pkg-config is supported with meson.

Removed the paragraph and only left " To use a libxml2 installation that is in an unusual location, you can set <command>pkg-config</command>-related environment variables (see its documentation)."


> +
> +      <varlistentry>
> +       <term><option>-Dspinlocks=<replaceable>true/false</replaceable></option></term>
> +       <listitem>
> +        <para>
> +         This option is set to true by default; setting it to false will
> +         allow the build to succeed even if <productname>PostgreSQL</productname>
> +         has no CPU spinlock support for the platform.  The lack of
> +         spinlock support will result in very poor performance; therefore,
> +         this option should only be changed if the build aborts and
> +         informs you that the platform lacks spinlock support. If setting this
> +         option to false is required to build <productname>PostgreSQL</productname> on
> +         your platform, please report the problem to the
> +         <productname>PostgreSQL</productname> developers.
> +        </para>
> +       </listitem>
> +      </varlistentry>
> +
> +      <varlistentry>
> +       <term><option>-Datomics=<replaceable>true/false</replaceable></option></term>
> +       <listitem>
> +        <para>
> +         This option is set to true by default; setting it to false will
> +         disable use of CPU atomic operations.  The option does nothing on
> +         platforms that lack such operations.  On platforms that do have
> +         them, disabling atomics will result in poor performance.  Changing
> +         this option is only useful for debugging or making performance comparisons.
> +        </para>
> +       </listitem>
> +      </varlistentry>
> +    </variablelist>

I think these should rather be in the developer section? They're not
dependencies and as you noted, they're not normally useful.

Makes sense. Moved. 


> +      <varlistentry>
> +       <term><option>-Dextra_include_dirs=<replaceable>DIRECTORIES</replaceable></option></term>
> +       <listitem>
> +        <para>
> +         <replaceable>DIRECTORIES</replaceable> is a colon-separated list of
> +         directories that will be added to the list the compiler
> +         searches for header files. If you have optional packages
> +         (such as GNU <application>Readline</application>) installed in a non-standard
> +         location,
> +         you have to use this option and probably also the corresponding
> +         <option>-Dextra_lib_dirs</option> option.
> +        </para>
> +        <para>
> +         Example: <literal>-Dextra_include_dirs=/opt/gnu/include:/usr/sup/include</literal>.
> +        </para>
> +       </listitem>
> +      </varlistentry>

The separator for meson is a comma, rather than a :

Changed. 

> +      <varlistentry>
> +       <term><option>-Dextra_lib_dirs=<replaceable>DIRECTORIES</replaceable></option></term>
> +       <listitem>
> +        <para>
> +         <replaceable>DIRECTORIES</replaceable> is a colon-separated list of
> +         directories to search for libraries. You will probably have
> +         to use this option (and the corresponding
> +         <option>-Dextra_include_dirs</option> option) if you have packages
> +         installed in non-standard locations.
> +        </para>
> +        <para>
> +         Example: <literal>-Dextra_lib_dirs=/opt/gnu/lib:/usr/sup/lib</literal>.
> +        </para>
> +       </listitem>
> +      </varlistentry>

Dito.

Changed. 


> +    <variablelist>
> +
> +      <varlistentry>
> +       <term><option>-Dsegsize=<replaceable>SEGSIZE</replaceable></option></term>
> +       <listitem>
> +        <para>
> +         Set the <firstterm>segment size</firstterm>, in gigabytes.  Large tables are
> +         divided into multiple operating-system files, each of size equal
> +         to the segment size.  This avoids problems with file size limits
> +         that exist on many platforms.  The default segment size, 1 gigabyte,
> +         is safe on all supported platforms.  If your operating system has
> +         <quote>largefile</quote> support (which most do, nowadays), you can use
> +         a larger segment size.  This can be helpful to reduce the number of
> +         file descriptors consumed when working with very large tables.
> +         But be careful not to select a value larger than is supported
> +         by your platform and the file systems you intend to use.  Other
> +         tools you might wish to use, such as <application>tar</application>, could
> +         also set limits on the usable file size.
> +         It is recommended, though not absolutely required, that this value
> +         be a power of 2.
> +        </para>
> +       </listitem>
> +      </varlistentry>
> +
> +      <varlistentry>
> +       <term><option>-Dblocksize=<replaceable>BLOCKSIZE</replaceable></option></term>
> +       <listitem>
> +        <para>
> +         Set the <firstterm>block size</firstterm>, in kilobytes.  This is the unit
> +         of storage and I/O within tables.  The default, 8 kilobytes,
> +         is suitable for most situations; but other values may be useful
> +         in special cases.
> +         The value must be a power of 2 between 1 and 32 (kilobytes).
> +        </para>
> +       </listitem>
> +      </varlistentry>
> +
> +      <varlistentry>
> +       <term><option>-Dwal_blocksize=<replaceable>BLOCKSIZE</replaceable></option></term>
> +       <listitem>
> +        <para>
> +         Set the <firstterm>WAL block size</firstterm>, in kilobytes.  This is the unit
> +         of storage and I/O within the WAL log.  The default, 8 kilobytes,
> +         is suitable for most situations; but other values may be useful
> +         in special cases.
> +         The value must be a power of 2 between 1 and 64 (kilobytes).
> +        </para>
> +       </listitem>
> +      </varlistentry>
> +
> +    </variablelist>

The order of the list entries seems a bit random? Perhaps just go for
alphabetical?

Done 


> +   </sect3>
> +
> +   <sect3 id="configure-devel">
> +    <title>Developer Options</title>
> +
> +    <para>
> +     Most of the options in this section are only of interest for
> +     developing or debugging <productname>PostgreSQL</productname>.
> +     They are not recommended for production builds, except
> +     for <option>--debug</option>, which can be useful to enable
> +     detailed bug reports in the unlucky event that you encounter a bug.
> +     On platforms supporting DTrace, <option>-Ddtrace</option>
> +     may also be reasonable to use in production.
> +    </para>
> +
> +    <para>
> +     When building an installation that will be used to develop code inside
> +     the server, it is recommended to use at least the <option>--buildtype=debug</option>
> +     and <option>-Dcassert</option> options.
> +    </para>
> +
> +     <variablelist>
> +      <varlistentry>
> +       <term><option>--buildtype=<replaceable>BUILDTYPE</replaceable></option></term>
> +       <listitem>
> +        <para>
> +         This option can be used to specify the buildtype to use; defaults
> +         to release. If you'd like finer control on the debug symbols
> +         and optimization levels than what this option provides, you can
> +         refer to the --debug and --optimization flags.
> +
> +         The following build types are generally used: plain, debug, debugoptimized
> +         and release. More information about them can be found in the
> +         <ulink url="https://mesonbuild.com/Running-Meson.html#configuring-the-build-directory">
> +         meson documentation</ulink>.
> +        </para>
> +       </listitem>
> +      </varlistentry>
> +
> +      <varlistentry>
> +       <term><option>--debug</option></term>
> +       <listitem>
> +        <para>
> +         Compiles all programs and libraries with debugging symbols.
> +         This means that you can run the programs in a debugger
> +         to analyze problems. This enlarges the size of the installed
> +         executables considerably, and on non-GCC compilers it usually
> +         also disables compiler optimization, causing slowdowns. However,
> +         having the symbols available is extremely helpful for dealing
> +         with any problems that might arise.  Currently, this option is
> +         recommended for production installations only if you use GCC.
> +         But you should always have it on if you are doing development work
> +         or running a beta version.
> +        </para>
> +       </listitem>
> +      </varlistentry>
> +
> +      <varlistentry>
> +       <term><option>--optimization</option>=<replaceable>LEVEL</replaceable></term>
> +       <listitem>
> +        <para>
> +         Specify the optimization level. LEVEL can be set to any of {0,g,1,2,3,s}.
> +        </para>
> +       </listitem>
> +      </varlistentry>

Wonder if we should just document optimization and debug, rather than
buildtype. The fact that debug/optimization override buildtype is a bit
confusing.

Yes, it was a bit confusing which is why I ended up documenting them as well. Not sure about doing just the debug / optimization as buildtype is likely a useful shorthand. I've kept as is for now but if you feel strongly about documenting only one of the two, I can remove. 


> +      <varlistentry>
> +       <term><option>-Dtap-tests</option></term>
> +       <listitem>
> +        <para>
> +         Enable tests using the Perl TAP tools.  This requires a Perl
> +         installation and the Perl module <literal>IPC::Run</literal>.
> +         <phrase condition="standalone-ignore">See <xref linkend="regress-tap"/> for more information.</phrase>
> +        </para>
> +       </listitem>
> +      </varlistentry>

This is an auto option as well.

Fixed. 


> +      <varlistentry>
> +       <term><option>--errorlogs</option></term>
> +       <listitem>
> +        <para>
> +        This option can be used to print the logs from the failing tests
> +        making debugging easier.
> +        </para>
> +       </listitem>
> +      </varlistentry>

I don't think it's worth documenting this, it defaults to true anyway.

Makes sense. Removed. 


> +        <para>
> +         To point to the <command>dtrace</command> program, the
> +         environment variable <envar>DTRACE</envar> can be set.  This
> +         will often be necessary because <command>dtrace</command> is
> +         typically installed under <filename>/usr/sbin</filename>,
> +         which might not be in your <envar>PATH</envar>.
> +        </para>

We don't read the DTRACE environment variable, but the DTRACE option.

Good catch. Changed.



> +   <para>
> +    <literal>ninja install</literal> should work for most cases,
> +    but if you'd like to use more options, you could also use
> +    <literal>meson install</literal> instead. You can learn more about
> +    <ulink url="https://mesonbuild.com/Commands.html#install">meson install</ulink>
> +    and its options in the meson documentation.
> +   </para>

Maybe we should mention meson --quiet here? The verbosity of ninja install is
a bit annoying.

Done
 


> +# Run the main pg_regress and isolation tests
> +<userinput>meson test --suite setup --suite main</userinput>

Since yesterday the main suite is no more. There's 'regress' and 'isolation'
now.

Changed 

Regards,
Samay


Greetings,

Andres Freund
Attachment

Re: Documentation for building with meson

From
Nazir Bilal Yavuz
Date:

Hi,

I did some tests on windows. I used 'ninja' as a backend.

On 11/8/2022 9:23 PM, samay sharma wrote:
Hi,

On Sat, Nov 5, 2022 at 2:39 PM Andres Freund <andres@anarazel.de> wrote:
Hi,

On 2022-10-30 20:51:59 -0700, samay sharma wrote:
> +# setup and enter build directory (done only first time)
> +meson setup build src --prefix=$PWD/install

This command won't work on windows, I think.


Yes, $PWD isn't recognized on windows, %CD% could be alternative.

> +      <varlistentry>
> +       <term><option>--sysconfdir=<replaceable>DIRECTORY</replaceable></option></term>
> +       <listitem>
> +        <para>
> +         Sets the directory for various configuration files,
> +         <filename><replaceable>PREFIX</replaceable>/etc</filename> by default.
> +        </para>
> +       </listitem>
> +      </varlistentry>

Need to check what windows does here.

It is same on windows: 'PREFIX/etc'.

I also checked other dirs(bindir, sysconfdir, libdir, includedir, datadir, localedir, mandir), default path is correct for all of them.

Regards,
Nazir Bilal Yavuz



Re: Documentation for building with meson

From
samay sharma
Date:
Hi,

On Thu, Nov 10, 2022 at 4:46 AM Nazir Bilal Yavuz <byavuz81@gmail.com> wrote:

Hi,

I did some tests on windows. I used 'ninja' as a backend.

On 11/8/2022 9:23 PM, samay sharma wrote:
Hi,

On Sat, Nov 5, 2022 at 2:39 PM Andres Freund <andres@anarazel.de> wrote:
Hi,

On 2022-10-30 20:51:59 -0700, samay sharma wrote:
> +# setup and enter build directory (done only first time)
> +meson setup build src --prefix=$PWD/install

This command won't work on windows, I think.


Yes, $PWD isn't recognized on windows, %CD% could be alternative.

Added.


> +      <varlistentry>
> +       <term><option>--sysconfdir=<replaceable>DIRECTORY</replaceable></option></term>
> +       <listitem>
> +        <para>
> +         Sets the directory for various configuration files,
> +         <filename><replaceable>PREFIX</replaceable>/etc</filename> by default.
> +        </para>
> +       </listitem>
> +      </varlistentry>

Need to check what windows does here.

It is same on windows: 'PREFIX/etc'.

I also checked other dirs(bindir, sysconfdir, libdir, includedir, datadir, localedir, mandir), default path is correct for all of them.

Thanks. I've made a few windows specific fixes in the latest version. Attached v5.

Regards,
Samay
 

Regards,
Nazir Bilal Yavuz



Attachment

Re: Documentation for building with meson

From
Ian Lawrence Barwick
Date:
2022年10月20日(木) 11:43 Justin Pryzby <pryzby@telsasoft.com>:
>
> On Wed, Oct 19, 2022 at 11:35:10AM -0700, samay sharma wrote:
> > Creating a new thread focussed on adding docs for building Postgres with
> > meson. This is a spinoff from the original thread [1] and I've attempted to
> > address all the feedback provided there in the attached patch.
> >
> > Please let me know your thoughts.
>
> It's easier to review rendered documentation.
> I made a rendered copy available here:
> https://api.cirrus-ci.com/v1/artifact/task/6084297781149696/html_docs/html_docs/install-meson.html

For reference, are there any instructions anywhere on how to do this?  It'd be
useful to be able to provide a link to the latest version of this documentation
(and also document the process for patch authors in general).

Regards

Ian Barwick



Re: Documentation for building with meson

From
Justin Pryzby
Date:
On Wed, Nov 16, 2022 at 10:52:35AM +0900, Ian Lawrence Barwick wrote:
> 2022年10月20日(木) 11:43 Justin Pryzby <pryzby@telsasoft.com>:
> >
> > On Wed, Oct 19, 2022 at 11:35:10AM -0700, samay sharma wrote:
> > > Creating a new thread focussed on adding docs for building Postgres with
> > > meson. This is a spinoff from the original thread [1] and I've attempted to
> > > address all the feedback provided there in the attached patch.
> > >
> > > Please let me know your thoughts.
> >
> > It's easier to review rendered documentation.
> > I made a rendered copy available here:
> > https://api.cirrus-ci.com/v1/artifact/task/6084297781149696/html_docs/html_docs/install-meson.html
> 
> For reference, are there any instructions anywhere on how to do this?  It'd be
> useful to be able to provide a link to the latest version of this documentation
> (and also document the process for patch authors in general).

I've submitted patches which would do that for every patch (ideally,
excluding patches that don't touch the docs, although it looks like the
"exclusion" isn't working).
https://commitfest.postgresql.org/40/3709/

The most recent patches on that thread don't include the "docs as
artifacts" patch, but only the preparatory "build docs as a separate
task".  I think the other part is stalled waiting for some updates to
cfbot to allow knowing how many commits are in the patchset.

FYI, you can navigate from the cirrus task's URL to the git commit (and
its parents)

https://cirrus-ci.com/task/6084297781149696 =>
https://github.com/justinpryzby/postgres/commit/7b57f3323fc77e9b04ef2e76976776090eb8b5b5

-- 
Justin



Re: Documentation for building with meson

From
Justin Pryzby
Date:
On Mon, Nov 14, 2022 at 10:41:21AM -0800, samay sharma wrote:

> You need LZ4, if you want to support compression of data with that
> method; see default_toast_compression and wal_compression. 

=> The first comma is odd.  Maybe it should say "LZ4 is needed to
support .."

> You need Zstandard, if you want to support compression of data or
> backups with that method; see wal_compression. The minimum required
> version is 1.4.0. 

Same.

Also, since v15, LZ4 and zstd can both be used by basebackup.

>Some commonly used ones are mentioned in the subsequent sections

=> Some commonly used options ...

>  Most of these require additional software, as described in Section
>  17.3.2, and are set to be auto features.

=> "Are set to be auto features" sounds odd.  I think it should say
something like " .. and are automatically enabled if the required
software is detected.".

> You can change this behavior by manually setting the auto features to
> enabled to require them or disabled to not build with them. 

remove "auto".  Maybe "enabled" and "disabled" need markup.

> On Windows, the default WinLDAP library is used. It defults to auto

typo: defults

> It defaults to auto and libsystemd and the associated header files need
> to be installed to use this option.

=> write this as two separate sentences.  Same for libxml.

> bsd to use the UUID functions found in FreeBSD, NetBSD, and some other
> BSD-derived systems 

=> should remove mention of netbsd, like c4b6d218e

>     Enables use of the Zlib library. It defaults to auto and enables
>     support for compressed archives in pg_dump ,pg_restore and
>     pg_basebackup and is recommended. 

=> The comma is mis-placed.

>     The default backend meson uses is ninja and that should suffice for
>     most use cases. However, if you'd like to fully integrate with
>     visual studio, you can set the BACKEND to vs. 

=> BACKEND is missing markup.

> This option can be used to specify the buildtype to use; defaults to
> release

=> release is missing markup

>      Specify the optimization level. LEVEL can be set to any of
>      {0,g,1,2,3,s}. 

=> LEVEL is missing markup

Thanks,
-- 
Justin



Re: Documentation for building with meson

From
samay sharma
Date:
Hi,

On Tue, Nov 22, 2022 at 10:36 PM Justin Pryzby <pryzby@telsasoft.com> wrote:
On Mon, Nov 14, 2022 at 10:41:21AM -0800, samay sharma wrote:

> You need LZ4, if you want to support compression of data with that
> method; see default_toast_compression and wal_compression.

=> The first comma is odd.  Maybe it should say "LZ4 is needed to
support .."

> You need Zstandard, if you want to support compression of data or
> backups with that method; see wal_compression. The minimum required
> version is 1.4.0.

Same.

Also, since v15, LZ4 and zstd can both be used by basebackup.

>Some commonly used ones are mentioned in the subsequent sections

=> Some commonly used options ...

>  Most of these require additional software, as described in Section
>  17.3.2, and are set to be auto features.

=> "Are set to be auto features" sounds odd.  I think it should say
something like " .. and are automatically enabled if the required
software is detected.".

> You can change this behavior by manually setting the auto features to
> enabled to require them or disabled to not build with them.

remove "auto".  Maybe "enabled" and "disabled" need markup.

> On Windows, the default WinLDAP library is used. It defults to auto

typo: defults

> It defaults to auto and libsystemd and the associated header files need
> to be installed to use this option.

=> write this as two separate sentences.  Same for libxml.

> bsd to use the UUID functions found in FreeBSD, NetBSD, and some other
> BSD-derived systems

=> should remove mention of netbsd, like c4b6d218e

>     Enables use of the Zlib library. It defaults to auto and enables
>     support for compressed archives in pg_dump ,pg_restore and
>     pg_basebackup and is recommended.

=> The comma is mis-placed.

>     The default backend meson uses is ninja and that should suffice for
>     most use cases. However, if you'd like to fully integrate with
>     visual studio, you can set the BACKEND to vs.

=> BACKEND is missing markup.

> This option can be used to specify the buildtype to use; defaults to
> release

=> release is missing markup

>      Specify the optimization level. LEVEL can be set to any of
>      {0,g,1,2,3,s}.

=> LEVEL is missing markup

Thanks for the feedback. Addressed all and added markup at a few more places in v6 (attached).

Regards,
Samay 

Thanks,
--
Justin
Attachment

Re: Documentation for building with meson

From
Justin Pryzby
Date:
On Wed, Nov 23, 2022 at 11:30:54AM -0800, samay sharma wrote:
> Thanks for the feedback. Addressed all and added markup at a few more
> places in v6 (attached).

Thanks.  It looks good to me.  A couple thoughts, maybe they're not
important.

 - LZ4 and Zstd refer to wal_compression and default_toast_compression,
   but not to any corresponding option to basebackup.

 - There's no space after the hash mark here; but above, there was:
   #Setup build directory with a different installation prefix

 - You use slash to show enumerated options, but it's more typical to
   use braces: {a | b | c}:
   -Dnls=auto/enabled/disabled

 - There's no earlier description/definition of an "auto" feature, but
   still says this:
   "Setting this option allows you to override value of all 'auto' features"

 - Currently the documentation always refers to "PostgreSQL", but you
   added two references to "Postgres":
   + If a program required to build Postgres...
   + Once Postgres is built...

-- 
Justin



Re: Documentation for building with meson

From
samay sharma
Date:
Hi,

On Wed, Nov 23, 2022 at 12:16 PM Justin Pryzby <pryzby@telsasoft.com> wrote:
On Wed, Nov 23, 2022 at 11:30:54AM -0800, samay sharma wrote:
> Thanks for the feedback. Addressed all and added markup at a few more
> places in v6 (attached).

Thanks.  It looks good to me.  A couple thoughts, maybe they're not
important.

Thank you. Attaching v7 addressing most of the points below.
 

 - LZ4 and Zstd refer to wal_compression and default_toast_compression,
   but not to any corresponding option to basebackup.

 - There's no space after the hash mark here; but above, there was:
   #Setup build directory with a different installation prefix

Added a space as that looks better.

 - You use slash to show enumerated options, but it's more typical to
   use braces: {a | b | c}:
   -Dnls=auto/enabled/disabled

Changed. 

 - There's no earlier description/definition of an "auto" feature, but
   still says this:
   "Setting this option allows you to override value of all 'auto' features"

Described what an "auto" feature is in (). 

 - Currently the documentation always refers to "PostgreSQL", but you
   added two references to "Postgres":
   + If a program required to build Postgres...
   + Once Postgres is built...

Good catch. Changed to PostgreSQL.

Regards,
Samay 

--
Justin
Attachment

Re: Documentation for building with meson

From
Justin Pryzby
Date:
Thanks; two more things I saw:

 - In other docs, <replacable> isn't used around { a | b } lists:
   git grep '<replaceable>[^<]*|' doc
 - I think this is(was) missing a word;
   Setting this option allows you to override THE value of all 'auto' features

-- 
Justin



Re: Documentation for building with meson

From
Peter Eisentraut
Date:
On 23.11.22 22:24, samay sharma wrote:
> Thank you. Attaching v7 addressing most of the points below.

I have committed this, after some editing and making some structural 
changes.  I moved the "Requirements" section back to the top level.  It 
did not look appealing to have to maintain two copies of this that have 
almost no substantial difference (but for some reason were written with 
separate structure and wording).  Also, I rearranged the Building with 
Meson section to use the same internal structure as the Building with 
Autoconf and Make section.  This will make it easier to maintain going 
forward.  For example if someone adds a new option, it will be easier to 
find the corresponding places in the lists where to add them.

We will likely keep iterating on the contents for the next little while, 
but I'm glad we now have a structure in place that we should be able to 
live with.




Re: Documentation for building with meson

From
Andres Freund
Date:
Hi,

On 2022-12-01 15:58:39 +0100, Peter Eisentraut wrote:
> On 23.11.22 22:24, samay sharma wrote:
> > Thank you. Attaching v7 addressing most of the points below.
> 
> I have committed this, after some editing and making some structural
> changes.

Thanks. I was working on that too, but somehow felt a bit stuck...

I'll try if I can adapt my pending changes.


> I moved the "Requirements" section back to the top level.  It did
> not look appealing to have to maintain two copies of this that have almost
> no substantial difference (but for some reason were written with separate
> structure and wording).

I don't think this is good. The whole "The following software packages are
required for building PostgreSQL" section is wrong now.  "They are not
required in the default configuration, but they are needed when certain build
options are enabled, as explained below:" section is misleading as well.

By the time we fix all of those we'll end up with a different section again.


> Also, I rearranged the Building with Meson section to use the same internal
> structure as the Building with Autoconf and Make section.  This will make it
> easier to maintain going forward.  For example if someone adds a new option,
> it will be easier to find the corresponding places in the lists where to add
> them.

I don't know. The existing list order makes very little sense to me. The
E.g. --enable-nls is before the rest in configure, presumably because it sorts
there alphabetically. But that's not the case for meson.

Copying "Anti-Features" as a structure element to the meson docs seems bogus
(also the name is bogus, but that's a pre-existing issue). There's no
difference in -Dreadline= to the other options meson-options-features list.

Nor does -Dspinlocks= -Datomics= make sense in the "anti features" section. It
made some sense for autoconf because of the --without- prefix, but that's not
at play in meson.  Their placement in the "Developer Options" made a whole lot
more sense.


I don't like "Miscellaneous" bit containing minor stuff like krb_srvnam and
data layout changing options like blocksize,segsize,wal_blocksize. But it
makes sense to change that for both at the same time.


> We will likely keep iterating on the contents for the next little while, but
> I'm glad we now have a structure in place that we should be able to live
> with.

I agree that it's good to have something we can work from more
iteratively. But I don't think this is a structure that we can live with.


I'm not particularly happy about this level of structural change made without
discussing it prior.

Greetings,

Andres Freund



Re: Documentation for building with meson

From
samay sharma
Date:
Hi,

On Thu, Dec 1, 2022 at 9:21 AM Andres Freund <andres@anarazel.de> wrote:
Hi,

On 2022-12-01 15:58:39 +0100, Peter Eisentraut wrote:
> On 23.11.22 22:24, samay sharma wrote:
> > Thank you. Attaching v7 addressing most of the points below.
>
> I have committed this, after some editing and making some structural
> changes.

Thanks. I was working on that too, but somehow felt a bit stuck...

I'll try if I can adapt my pending changes.

I got back to working on the meson docs. I'm attaching a new patch set proposing some improvements to the current installation docs. I've tried to make some corrections and improvements suggested in this thread while trying to maintain consistency across make and meson docs as per Peter's ask. There are 5 patches in the patch-set:

v8-0001: Makes minor corrections, adds instructions to build docs and adds a few links to meson docs.
v8-0002, v8-0003 and v8-0004 make changes to restructure the configure options based on reasoning discussed below. To maintain consistency, I've made those changes on both the make and meson side.
v8-0005 Reproposes the Short Version I had proposed in v7 as I feel we should discuss that proposal. I think it improves things in terms of installation docs. More details below.



> I moved the "Requirements" section back to the top level.  It did
> not look appealing to have to maintain two copies of this that have almost
> no substantial difference (but for some reason were written with separate
> structure and wording).
 
There are a few reasons why I had done this. Some reasons Andres has described in his previous email and I'll add a few specific examples on why having the same section for both might not be a good idea.

* Having readline and zlib as required for building PostgreSQL is now wrong because they are not required for meson builds. Also, the name of the configs are different for make and meson and the current section only lists the make ones.
* There are many references to configure in that section which don't apply to meson.
* Last I checked Flex and Bison were always required to build via meson but not for make and the current section doesn't explain those differences.

I spent a good amount of time thinking if we could have a single section, clarify these differences to make it correct and not confuse the users. I couldn't find a way to do all three. Therefore, I think we should move to a different requirements section for both. I'm happy to re-propose the previous version which separates them but wanted to see if anybody has better ideas.
 

I don't think this is good. The whole "The following software packages are
required for building PostgreSQL" section is wrong now.  "They are not
required in the default configuration, but they are needed when certain build
options are enabled, as explained below:" section is misleading as well.

By the time we fix all of those we'll end up with a different section again.


> Also, I rearranged the Building with Meson section to use the same internal
> structure as the Building with Autoconf and Make section.  This will make it
> easier to maintain going forward.  For example if someone adds a new option,
> it will be easier to find the corresponding places in the lists where to add
> them. 

I don't know. The existing list order makes very little sense to me. The
E.g. --enable-nls is before the rest in configure, presumably because it sorts
there alphabetically. But that's not the case for meson.

Copying "Anti-Features" as a structure element to the meson docs seems bogus
(also the name is bogus, but that's a pre-existing issue). There's no
difference in -Dreadline= to the other options meson-options-features list.

Nor does -Dspinlocks= -Datomics= make sense in the "anti features" section. It
made some sense for autoconf because of the --without- prefix, but that's not
at play in meson.  Their placement in the "Developer Options" made a whole lot
more sense.

I agree "Anti-Features" desn't make sense in the meson context. One of my patches removes that section and moves some options into the "Postgres Features" section and others into the "Developer Options" section. I've proposed to make those changes on both sides to make it easier to maintain.



I don't like "Miscellaneous" bit containing minor stuff like krb_srvnam and
data layout changing options like blocksize,segsize,wal_blocksize. But it
makes sense to change that for both at the same time.

I've proposed a patch to add a new "Data Layout Options" section which includes: blocksize, segsize and wal_blocksize. I've created that section on both sides.


> We will likely keep iterating on the contents for the next little while, but
> I'm glad we now have a structure in place that we should be able to live
> with.

I feel that there are a few shortcomings of the current "Short Version". I tried to address them in my previous proposal but I noticed that a version similar to the make version was committed. So, I thought I'd describe why I proposed a new structure.

1) The current version has OS specific commands (eg. adduser). They don't work across all platforms.
2) The installation instructions use paths which require sudo and which are also OS specific. Not every developer who wants to build and try out Postgres might have sudo permissions.
3) Most developers have a separate installation path where they store their dev binaries while the current instructions install them in standard paths.
4) I wanted to have a separate directory which can nicely be cleaned once you're done with building and testing the packages. That's easier to do at a local path.
5) There's no description of what each instruction does so that developers can modify the commands if they want to change something.
 
Due to these reasons, I feel it's worth considering the newer version of the "Short version". I've proposed to change it only in the meson docs for now but if there's interest I can modify the make instructions to be the same. I left them as it is as people might be used to those instructions.

Regards,
Samay

I agree that it's good to have something we can work from more
iteratively. But I don't think this is a structure that we can live with.


I'm not particularly happy about this level of structural change made without
discussing it prior.

Greetings,

Andres Freund
Attachment

Re: Documentation for building with meson

From
Peter Eisentraut
Date:
 > [PATCH v8 1/5] Make minor additions and corrections to meson docs

The last hunk revealed that there is some mixing up between meson setup 
and meson configure.  This goes a bit further.  For example, earlier it 
says that to get a list of meson setup options, call meson configure 
--help and look at https://mesonbuild.com/Commands.html#configure, which 
are both wrong.  Also later throughout the text it uses one or the 
other.  I think this has the potential to be very confusing, and we 
should clean this up carefully.

The text about additional meson test options maybe should go into the 
regress.sgml chapter?


 > [PATCH v8 2/5] Add data layout options sub-section in installation
  docs

This makes sense.  Please double check your patch for correct title 
casing, vertical spacing of XML, to keep everything looking consistent.

This text isn't yours, but since your patch emphasizes it, I wonder if 
it could use some clarification:

+     These options affect how PostgreSQL lays out data on disk.
+     Note that changing these breaks on-disk database compatibility,
+     meaning you cannot use <command>pg_upgrade</command> to upgrade to
+     a build with different values of these options.

This isn't really correct.  What breaking on-disk compatibility means is 
that you can't use a server compiled one way with a data directory 
initialized by binaries compiled another way.  pg_upgrade may well have 
the ability to upgrade between one or the other; that's up to pg_upgrade 
to figure out but not an intrinsic property.  (I wonder why pg_upgrade 
cares about the WAL block size.)


 > [PATCH v8 3/5] Remove Anti-Features section from Installation from
  source docs

Makes sense.  But is "--disable-thread-safety" really a developer 
feature?  I think not.


 > [PATCH v8 4/5] Re-organize Miscellaneous section

This moves the Miscellaneous section after Developer Features.  I think 
Developer Features should be last.

Maybe should remove this section and add the options to the regular 
PostgreSQL Features section.

Also consider the grouping in meson_options.txt, which is slightly 
different yet.


 > [PATCH v8 5/5] Change Short Version for meson installation guide

+# create working directory
+mkdir postgres
+cd postgres
+
+# fetch source code
+git clone https://git.postgresql.org/git/postgresql.git src

This comes after the "Getting the Source" section, so at this point they 
already have the source and don't need to do "git clone" etc. again.

+# setup and enter build directory (done only first time)
+## Unix based platforms
+meson setup build src --prefix=$PWD/install
+
+## Windows
+meson setup build src --prefix=%cd%/install

Maybe some people work this way, but to me the directory structures you 
create here are completely weird.

+# Initialize a new database
+../install/bin/initdb -D ../data
+
+# Start database
+../install/bin/pg_ctl -D ../data/ -l logfile start
+
+# Connect to the database
+../install/bin/psql -d postgres

The terminology here needs to be tightened up.  You are using "database" 
here to mean three different things.




Re: Documentation for building with meson

From
samay sharma
Date:
Hi,

On Wed, Mar 15, 2023 at 4:28 AM Peter Eisentraut <peter.eisentraut@enterprisedb.com> wrote:
 > [PATCH v8 1/5] Make minor additions and corrections to meson docs

The last hunk revealed that there is some mixing up between meson setup
and meson configure.  This goes a bit further.  For example, earlier it
says that to get a list of meson setup options, call meson configure
--help and look at https://mesonbuild.com/Commands.html#configure, which
are both wrong.  Also later throughout the text it uses one or the
other.  I think this has the potential to be very confusing, and we
should clean this up carefully.

The text about additional meson test options maybe should go into the
regress.sgml chapter?

I tried to make the meson setup and meson configure usage consistent. I've removed the text for the test options.


 > [PATCH v8 2/5] Add data layout options sub-section in installation
  docs

This makes sense.  Please double check your patch for correct title
casing, vertical spacing of XML, to keep everything looking consistent.

Thanks for noticing. Made it consistent on both sides. 

This text isn't yours, but since your patch emphasizes it, I wonder if
it could use some clarification:

+     These options affect how PostgreSQL lays out data on disk.
+     Note that changing these breaks on-disk database compatibility,
+     meaning you cannot use <command>pg_upgrade</command> to upgrade to
+     a build with different values of these options.

This isn't really correct.  What breaking on-disk compatibility means is
that you can't use a server compiled one way with a data directory
initialized by binaries compiled another way.  pg_upgrade may well have
the ability to upgrade between one or the other; that's up to pg_upgrade
to figure out but not an intrinsic property.  (I wonder why pg_upgrade
cares about the WAL block size.)

 Fixed.


 > [PATCH v8 3/5] Remove Anti-Features section from Installation from
  source docs

Makes sense.  But is "--disable-thread-safety" really a developer
feature?  I think not.


Moved to PostgreSQL features. Do you think there's a better place for it?
 

 > [PATCH v8 4/5] Re-organize Miscellaneous section

This moves the Miscellaneous section after Developer Features.  I think
Developer Features should be last.

Maybe should remove this section and add the options to the regular
PostgreSQL Features section.

Yes, that makes sense. Made this change. 

Also consider the grouping in meson_options.txt, which is slightly
different yet. 

Removed Misc options section from meson_options.txt too. 


 > [PATCH v8 5/5] Change Short Version for meson installation guide

+# create working directory
+mkdir postgres
+cd postgres
+
+# fetch source code
+git clone https://git.postgresql.org/git/postgresql.git src

This comes after the "Getting the Source" section, so at this point they
already have the source and don't need to do "git clone" etc. again.

+# setup and enter build directory (done only first time)
+## Unix based platforms
+meson setup build src --prefix=$PWD/install
+
+## Windows
+meson setup build src --prefix=%cd%/install

Maybe some people work this way, but to me the directory structures you
create here are completely weird.

I'd like to discuss what you think is a good directory structure to work with. I've mentioned some of the drawbacks I see with the current structure for the short version. I know this structure can feel different but it feeling weird is not ideal. Do you have a directory structure in mind which is different but doesn't feel odd to you?
 

+# Initialize a new database
+../install/bin/initdb -D ../data
+
+# Start database
+../install/bin/pg_ctl -D ../data/ -l logfile start
+
+# Connect to the database
+../install/bin/psql -d postgres

The terminology here needs to be tightened up.  You are using "database"
here to mean three different things.

I'll address this together once we are aligned on the overall directory structure etc. 

There are a few reasons why I had done this. Some reasons Andres has described in his previous email and I'll add a few specific examples on why having the same section for both might not be a good idea.
 
 * Having readline and zlib as required for building PostgreSQL is now wrong because they are not required for meson builds. Also, the name of the configs are different for make and meson and the current section only lists the make ones.
 * There are many references to configure in that section which don't apply to meson.
 * Last I checked Flex and Bison were always required to build via meson but not for make and the current section doesn't explain those differences.
 
I spent a good amount of time thinking if we could have a single section, clarify these differences to make it correct and not confuse the users. I couldn't find a way to do all three. Therefore, I think we should move to a  different requirements section for both. I'm happy to re-propose the previous version which separates them but wanted to see if anybody has better ideas.

Do you have thoughts on the requirements section and the motivation to have two different versions I had mentioned upthread? 

Regards,
Samay
Attachment

Re: Documentation for building with meson

From
Andres Freund
Date:
Hi,

On 2023-03-28 12:27:26 -0700, samay sharma wrote:
> Subject: [PATCH v9 1/5] Make minor additions and corrections to meson docs
> 
> This commit makes a few corrections to the meson docs
> and adds a few instructions and links for better clarity.
> ---
>  doc/src/sgml/installation.sgml | 24 +++++++++++++++---------
>  1 file changed, 15 insertions(+), 9 deletions(-)
> 
> diff --git a/doc/src/sgml/installation.sgml b/doc/src/sgml/installation.sgml
> index 70ab5b77d8..e3b9b6c0d0 100644
> --- a/doc/src/sgml/installation.sgml
> +++ b/doc/src/sgml/installation.sgml
> @@ -2057,8 +2057,7 @@ meson setup build -Dssl=openssl
>  <screen>
>  meson configure -Dcassert=true
>  </screen>
> -    <command>meson configure</command>'s commonly used command-line options
> -    are explained in <xref linkend="meson-options"/>.
> +    Commonly used build options for <command>meson configure</command> (and <command>meson setup</command>) are
explainedin <xref linkend="meson-options"/>.
 
>     </para>
>    </step>
>  
> @@ -2078,6 +2077,13 @@ ninja
>      processes used with the command line argument <literal>-j</literal>.
>     </para>
>  
> +   <para>
> +    If you want to build the docs, you can type:
> +<screen>
> +ninja docs
> +</screen>
> +   </para>

"type" sounds a bit too, IDK, process oriented. "To build the docs, use"?


> Subject: [PATCH v9 2/5] Add data layout options sub-section in installation
>  docs
> 
> This commit separates out blocksize, segsize and wal_blocksize
> options into a separate Data layout options sub-section in both
> the make and meson docs. They were earlier in a miscellaneous
> section which included several unrelated options. This change
> also helps reduce the repetition of the warnings that changing
> these parameters breaks on-disk compatibility.

Makes sense. I'm planning to apply this unless Peter or somebody else has
further feedback.


> From 11d82aa49efb3d1cbc08f14562a757f115053c8b Mon Sep 17 00:00:00 2001
> From: Samay Sharma <smilingsamay@gmail.com>
> Date: Mon, 13 Feb 2023 16:23:52 -0800
> Subject: [PATCH v9 3/5] Remove Anti-Features section from Installation from
>  source docs
> 
> Currently, several meson setup options are listed in anti-features.
> However, they are similar to most other options in the postgres
> features list as they are 'auto' features themselves. Also, other
> options are likely better suited to the developer options section.
> This commit, therefore, moves the options listed in the anti-features
> section into other sections and removes that section.
> 
> For consistency, this reorganization has been done on the make section
> of the docs as well.

Makes sense to me. "Anti-Features" is confusing as a name to start with.

Greetings,

Andres Freund



Re: Documentation for building with meson

From
samay sharma
Date:
Hi,

On Tue, Apr 11, 2023 at 10:18 AM Andres Freund <andres@anarazel.de> wrote:
Hi,

On 2023-03-28 12:27:26 -0700, samay sharma wrote:
> Subject: [PATCH v9 1/5] Make minor additions and corrections to meson docs
>
> This commit makes a few corrections to the meson docs
> and adds a few instructions and links for better clarity.
> ---
>  doc/src/sgml/installation.sgml | 24 +++++++++++++++---------
>  1 file changed, 15 insertions(+), 9 deletions(-)
>
> diff --git a/doc/src/sgml/installation.sgml b/doc/src/sgml/installation.sgml
> index 70ab5b77d8..e3b9b6c0d0 100644
> --- a/doc/src/sgml/installation.sgml
> +++ b/doc/src/sgml/installation.sgml
> @@ -2057,8 +2057,7 @@ meson setup build -Dssl=openssl
>  <screen>
>  meson configure -Dcassert=true
>  </screen>
> -    <command>meson configure</command>'s commonly used command-line options
> -    are explained in <xref linkend="meson-options"/>.
> +    Commonly used build options for <command>meson configure</command> (and <command>meson setup</command>) are explained in <xref linkend="meson-options"/>.
>     </para>
>    </step>

> @@ -2078,6 +2077,13 @@ ninja
>      processes used with the command line argument <literal>-j</literal>.
>     </para>

> +   <para>
> +    If you want to build the docs, you can type:
> +<screen>
> +ninja docs
> +</screen>
> +   </para>

"type" sounds a bit too, IDK, process oriented. "To build the docs, use"?

Sure, that works. 


> Subject: [PATCH v9 2/5] Add data layout options sub-section in installation
>  docs
>
> This commit separates out blocksize, segsize and wal_blocksize
> options into a separate Data layout options sub-section in both
> the make and meson docs. They were earlier in a miscellaneous
> section which included several unrelated options. This change
> also helps reduce the repetition of the warnings that changing
> these parameters breaks on-disk compatibility.

Makes sense. I'm planning to apply this unless Peter or somebody else has
further feedback.

Cool. 


> From 11d82aa49efb3d1cbc08f14562a757f115053c8b Mon Sep 17 00:00:00 2001
> From: Samay Sharma <smilingsamay@gmail.com>
> Date: Mon, 13 Feb 2023 16:23:52 -0800
> Subject: [PATCH v9 3/5] Remove Anti-Features section from Installation from
>  source docs
>
> Currently, several meson setup options are listed in anti-features.
> However, they are similar to most other options in the postgres
> features list as they are 'auto' features themselves. Also, other
> options are likely better suited to the developer options section.
> This commit, therefore, moves the options listed in the anti-features
> section into other sections and removes that section.
>
> For consistency, this reorganization has been done on the make section
> of the docs as well.

Makes sense to me. "Anti-Features" is confusing as a name to start with.

Greetings,

Andres Freund

Re: Documentation for building with meson

From
Peter Eisentraut
Date:
On 11.04.23 19:18, Andres Freund wrote:
>> Subject: [PATCH v9 2/5] Add data layout options sub-section in installation
>>   docs
>>
>> This commit separates out blocksize, segsize and wal_blocksize
>> options into a separate Data layout options sub-section in both
>> the make and meson docs. They were earlier in a miscellaneous
>> section which included several unrelated options. This change
>> also helps reduce the repetition of the warnings that changing
>> these parameters breaks on-disk compatibility.
> 
> Makes sense. I'm planning to apply this unless Peter or somebody else has
> further feedback.

I'm okay with patches 0001 through 0004.

I don't like 0005.  I think we should drop that for now and maybe have a 
separate discussion under a separate heading about that.

> 
> 
>>  From 11d82aa49efb3d1cbc08f14562a757f115053c8b Mon Sep 17 00:00:00 2001
>> From: Samay Sharma <smilingsamay@gmail.com>
>> Date: Mon, 13 Feb 2023 16:23:52 -0800
>> Subject: [PATCH v9 3/5] Remove Anti-Features section from Installation from
>>   source docs
>>
>> Currently, several meson setup options are listed in anti-features.
>> However, they are similar to most other options in the postgres
>> features list as they are 'auto' features themselves. Also, other
>> options are likely better suited to the developer options section.
>> This commit, therefore, moves the options listed in the anti-features
>> section into other sections and removes that section.
>>
>> For consistency, this reorganization has been done on the make section
>> of the docs as well.
> 
> Makes sense to me. "Anti-Features" is confusing as a name to start with.




Re: Documentation for building with meson

From
"Tristan Partin"
Date:
Hey!

Nice work organizing the docs. Looks pretty good. I had a single comment
regarding the Meson docs in general.

It seems like Postgres cares a bit about Windows/Mac development. Is
ninja the blessed way to build Postgres on Windows and Mac? Otherwise, I
would suggest just being more generic and replace instances of `ninja
xxx` in the docs with `meson compile xxx`, making the command
backend-agnostic to accomodate XCode and VS project users.

--
Tristan Partin
Neon (https://neon.tech)



Re: Documentation for building with meson

From
"Tristan Partin"
Date:
On Wed Apr 12, 2023 at 4:19 PM CDT, Peter Eisentraut wrote:
> On 11.04.23 19:18, Andres Freund wrote:
> >> Subject: [PATCH v9 2/5] Add data layout options sub-section in installation
> >>   docs
> >>
> >> This commit separates out blocksize, segsize and wal_blocksize
> >> options into a separate Data layout options sub-section in both
> >> the make and meson docs. They were earlier in a miscellaneous
> >> section which included several unrelated options. This change
> >> also helps reduce the repetition of the warnings that changing
> >> these parameters breaks on-disk compatibility.
> >
> > Makes sense. I'm planning to apply this unless Peter or somebody else has
> > further feedback.
>
> I'm okay with patches 0001 through 0004.
>
> I don't like 0005.  I think we should drop that for now and maybe have a
> separate discussion under a separate heading about that.

With regard to 0005, perhaps it makes the most sense to use the XDG
Directory Specification as an example. Install into ~/.local or
~/.local/postgres (as the Meson default prefix is). Put the data into
~/.var/lib/postgres or something similar.

--
Tristan Partin
Neon (https://neon.tech)



Re: Documentation for building with meson

From
Andres Freund
Date:
Hi,

On 2023-03-28 12:27:26 -0700, samay sharma wrote:
> +   <para>
> +    If you want to build the docs, you can type:
> +<screen>
> +ninja docs
> +</screen>
> +   </para>

To me 'you can type' sounds odd. To me even just "To build the docs:" would
sound better. But the make docs do it "your" way, so I'll just go along with
it for now.


> From c5e637a54c2b83e5bd8c4155784d97e82937eb51 Mon Sep 17 00:00:00 2001
> From: Samay Sharma <smilingsamay@gmail.com>
> Date: Mon, 6 Feb 2023 16:09:42 -0800
> Subject: [PATCH v9 2/5] Add data layout options sub-section in installation
>  docs
>
> This commit separates out blocksize, segsize and wal_blocksize
> options into a separate Data layout options sub-section in both
> the make and meson docs. They were earlier in a miscellaneous
> section which included several unrelated options. This change
> also helps reduce the repetition of the warnings that changing
> these parameters breaks on-disk compatibility.

I still like this change, but ISTM that the "Data Layout" section should
follow the "PostgreSQL Features" section, rather than follow "Anti Features",
"Build Process Details" and "Miscellaneous". I realize some of these are
reorganized later on, but even then "Build Process Details"

Would anybody mind if I swapped these around?


> +      <varlistentry id="meson-option-with-blocksize">

I don't quite understand the "-with" added to the ids?


> From 11d82aa49efb3d1cbc08f14562a757f115053c8b Mon Sep 17 00:00:00 2001
> From: Samay Sharma <smilingsamay@gmail.com>
> Date: Mon, 13 Feb 2023 16:23:52 -0800
> Subject: [PATCH v9 3/5] Remove Anti-Features section from Installation from
>  source docs
>
> Currently, several meson setup options are listed in anti-features.
> However, they are similar to most other options in the postgres
> features list as they are 'auto' features themselves. Also, other
> options are likely better suited to the developer options section.
> This commit, therefore, moves the options listed in the anti-features
> section into other sections and removes that section.
> For consistency, this reorganization has been done on the make section
> of the docs as well.
> ---
>  doc/src/sgml/installation.sgml | 140 ++++++++++++++-------------------
>  1 file changed, 57 insertions(+), 83 deletions(-)
>
> diff --git a/doc/src/sgml/installation.sgml b/doc/src/sgml/installation.sgml
> index 7e65cdd72e..d7ab0c205e 100644
> --- a/doc/src/sgml/installation.sgml
> +++ b/doc/src/sgml/installation.sgml
> @@ -1214,23 +1214,6 @@ build-postgresql:
>         </listitem>
>        </varlistentry>
>
> -     </variablelist>
> -
> -   </sect3>
> -
> -   <sect3 id="configure-options-anti-features">
> -    <title>Anti-Features</title>
> -
> -    <para>
> -     The options described in this section allow disabling
> -     certain <productname>PostgreSQL</productname> features that are built
> -     by default, but which might need to be turned off if the required
> -     software or system features are not available.  Using these options is
> -     not recommended unless really necessary.
> -    </para>
> -
> -     <variablelist>
> -
>        <varlistentry id="configure-option-without-readline">
>         <term><option>--without-readline</option></term>
>         <listitem>

I don't think this is quite right. The section above the list says

"The options described in this section enable building of various PostgreSQL
features that are not built by default. Most of these are non-default only
because they require additional software, as described in Section 17.1."

So just merging --without-icu, --without-readline, --without-zlib,
--disable-thread-safety, in with the rest doesn't quite seem right.

I suspect that the easiest way for that is to just move --disable-atomics,
--disable-spinlocks to the developer section and then to leave the
anti-features section around for autoconf.

Any better idea?

Greetings,

Andres Freund



Re: Documentation for building with meson

From
Peter Eisentraut
Date:
On 10.06.23 06:00, Andres Freund wrote:
>>  From c5e637a54c2b83e5bd8c4155784d97e82937eb51 Mon Sep 17 00:00:00 2001
>> From: Samay Sharma<smilingsamay@gmail.com>
>> Date: Mon, 6 Feb 2023 16:09:42 -0800
>> Subject: [PATCH v9 2/5] Add data layout options sub-section in installation
>>   docs
>>
>> This commit separates out blocksize, segsize and wal_blocksize
>> options into a separate Data layout options sub-section in both
>> the make and meson docs. They were earlier in a miscellaneous
>> section which included several unrelated options. This change
>> also helps reduce the repetition of the warnings that changing
>> these parameters breaks on-disk compatibility.
> I still like this change, but ISTM that the "Data Layout" section should
> follow the "PostgreSQL Features" section, rather than follow "Anti Features",
> "Build Process Details" and "Miscellaneous". I realize some of these are
> reorganized later on, but even then "Build Process Details"
> 
> Would anybody mind if I swapped these around?

I don't mind a Data Layout section in principle, but I wonder whether 
it's worth changing now.  The segsize option is proposed to be turned 
into a run-time option (and/or removed).  For the WAL block size, I had 
previously mentioned, I don't think it is correct that pg_upgrade should 
actually care about it.  So I wouldn't spend too much time trying to 
carefully refactor the notes on the data layout options if we're going 
to have to change them around before long again anyway.





Re: Documentation for building with meson

From
vignesh C
Date:
On Wed, 29 Mar 2023 at 00:57, samay sharma <smilingsamay@gmail.com> wrote:
>
> Hi,
>
> On Wed, Mar 15, 2023 at 4:28 AM Peter Eisentraut <peter.eisentraut@enterprisedb.com> wrote:
>>
>>  > [PATCH v8 1/5] Make minor additions and corrections to meson docs
>>
>> The last hunk revealed that there is some mixing up between meson setup
>> and meson configure.  This goes a bit further.  For example, earlier it
>> says that to get a list of meson setup options, call meson configure
>> --help and look at https://mesonbuild.com/Commands.html#configure, which
>> are both wrong.  Also later throughout the text it uses one or the
>> other.  I think this has the potential to be very confusing, and we
>> should clean this up carefully.
>>
>> The text about additional meson test options maybe should go into the
>> regress.sgml chapter?
>
>
> I tried to make the meson setup and meson configure usage consistent. I've removed the text for the test options.
>>
>>
>>
>>  > [PATCH v8 2/5] Add data layout options sub-section in installation
>>   docs
>>
>> This makes sense.  Please double check your patch for correct title
>> casing, vertical spacing of XML, to keep everything looking consistent.
>
>
> Thanks for noticing. Made it consistent on both sides.
>>
>>
>> This text isn't yours, but since your patch emphasizes it, I wonder if
>> it could use some clarification:
>>
>> +     These options affect how PostgreSQL lays out data on disk.
>> +     Note that changing these breaks on-disk database compatibility,
>> +     meaning you cannot use <command>pg_upgrade</command> to upgrade to
>> +     a build with different values of these options.
>>
>> This isn't really correct.  What breaking on-disk compatibility means is
>> that you can't use a server compiled one way with a data directory
>> initialized by binaries compiled another way.  pg_upgrade may well have
>> the ability to upgrade between one or the other; that's up to pg_upgrade
>> to figure out but not an intrinsic property.  (I wonder why pg_upgrade
>> cares about the WAL block size.)
>
>
>  Fixed.
>>
>>
>>
>>  > [PATCH v8 3/5] Remove Anti-Features section from Installation from
>>   source docs
>>
>> Makes sense.  But is "--disable-thread-safety" really a developer
>> feature?  I think not.
>>
>
> Moved to PostgreSQL features. Do you think there's a better place for it?
>
>>
>>
>>  > [PATCH v8 4/5] Re-organize Miscellaneous section
>>
>> This moves the Miscellaneous section after Developer Features.  I think
>> Developer Features should be last.
>>
>> Maybe should remove this section and add the options to the regular
>> PostgreSQL Features section.
>
>
> Yes, that makes sense. Made this change.
>>
>>
>> Also consider the grouping in meson_options.txt, which is slightly
>> different yet.
>
>
> Removed Misc options section from meson_options.txt too.
>>
>>
>>
>>  > [PATCH v8 5/5] Change Short Version for meson installation guide
>>
>> +# create working directory
>> +mkdir postgres
>> +cd postgres
>> +
>> +# fetch source code
>> +git clone https://git.postgresql.org/git/postgresql.git src
>>
>> This comes after the "Getting the Source" section, so at this point they
>> already have the source and don't need to do "git clone" etc. again.
>>
>> +# setup and enter build directory (done only first time)
>> +## Unix based platforms
>> +meson setup build src --prefix=$PWD/install
>> +
>> +## Windows
>> +meson setup build src --prefix=%cd%/install
>>
>> Maybe some people work this way, but to me the directory structures you
>> create here are completely weird.
>
>
> I'd like to discuss what you think is a good directory structure to work with. I've mentioned some of the drawbacks I
seewith the current structure for the short version. I know this structure can feel different but it feeling weird is
notideal. Do you have a directory structure in mind which is different but doesn't feel odd to you? 
>
>>
>>
>> +# Initialize a new database
>> +../install/bin/initdb -D ../data
>> +
>> +# Start database
>> +../install/bin/pg_ctl -D ../data/ -l logfile start
>> +
>> +# Connect to the database
>> +../install/bin/psql -d postgres
>>
>> The terminology here needs to be tightened up.  You are using "database"
>> here to mean three different things.
>
>
> I'll address this together once we are aligned on the overall directory structure etc.
>
>> There are a few reasons why I had done this. Some reasons Andres has described in his previous email and I'll add a
fewspecific examples on why having the same section for both might not be a good idea. 
>>
>>  * Having readline and zlib as required for building PostgreSQL is now wrong because they are not required for meson
builds.Also, the name of the configs are different for make and meson and the current section only lists the make ones. 
>>  * There are many references to configure in that section which don't apply to meson.
>>  * Last I checked Flex and Bison were always required to build via meson but not for make and the current section
doesn'texplain those differences. 
>>
>> I spent a good amount of time thinking if we could have a single section, clarify these differences to make it
correctand not confuse the users. I couldn't find a way to do all three. Therefore, I think we should move to a
differentrequirements section for both. I'm happy to re-propose the previous version which separates them but wanted to
seeif anybody has better ideas. 
>
>
> Do you have thoughts on the requirements section and the motivation to have two different versions I had mentioned
upthread?

I have changed the status of commitfest entry to "Returned with
Feedback" as there was no followup on Tristan Partin and Andres's
comments from many months. Please handle the comments and add a new
commitfest entry if required for any pending tasks left.

Regards,
Vignesh