Thread: Service not starting on Ubuntu 15.04

On Mon, 26 Oct 2015 11:21:23 +0000
Lasse Westh-Nielsen <lassewesth@gmail.com> wrote:
>
> I posted to stackoverflow about my problem upgrading from Ubuntu 14.04 to
> Ubuntu 15.04:
> http://stackoverflow.com/questions/33306475/ubuntu-15-04-postgresql-doesnt-start
>
> Tl;dr: postgresql service does not start properly when installed as a
> package using cloud-init.
>
> And I can't figure out if I am doing something wrong, if the AMI is no
> good, if the package has problems, ... I reckon I cannot be the first
> person to use Postgres on Ubuntu Vivid, but I have been hammering my head
> against the wall with this for hours.
>
> Any help greatly appreciated!

I'm taking a shot in the dark here, but ...

The symptoms you describe seem to suggest that the script is starting
PostgreSQL asynchronously (i.e. in the background) which means that
the CREATE command runs too quickly and the server isn't started yet.

A quick way to _test_ this theory would be to put a sleep between the
install and the CREATE commands and see if the problem goes away.

If that does seem to be the problem, then a good _fix_ would be to
find a way to foreground the startup of the server, or have some command
that tests to ensure the server is started and blocks until it is
before running the create command.

The only point I'm unclear on is whether you've confirmed that
Postgres actually _is_ started once the server is up (albiet without
the CREATE statement having succeeded).

--
Bill Moran

On 10/26/2015 04:21 AM, Lasse Westh-Nielsen wrote:

Hey,

I posted to stackoverflow about my problem upgrading from Ubuntu 14.04
to Ubuntu 15.04:
http://stackoverflow.com/questions/33306475/ubuntu-15-04-postgresql-doesnt-start

Pretty sure it is no coincidence that the 14.04 --> 15.04 upgrade changes the init process from Upstart to systemd. Might want to take a look at this article:

http://linux.softpedia.com/blog/Ubuntu-15-04-Users-Can-Switch-Off-Systemd-and-Use-Upstart-479373.shtml

and this:

https://wiki.ubuntu.com/SystemdForUpstartUsers

Tl;dr: postgresql service does not start properly when installed as a
package using cloud-init.

What does the /var/log/bootstrap.log show?

And I can't figure out if I am doing something wrong, if the AMI is no
good, if the package has problems, ... I reckon I cannot be the first
person to use Postgres on Ubuntu Vivid, but I have been hammering my
head against the wall with this for hours.

Any help greatly appreciated!

Regards,

Lasse

--
Adrian Klaver
adrian.klaver@aklaver.com

On 10/26/2015 07:08 AM, Lasse Westh-Nielsen wrote:

Adrian,

Thanks. I know about the systemd change, and indeed the postgres package
I end up with _has_ systemd integration already:

    $ cat /etc/systemd/system/multi-user.target.wants/postgresql.service
    # systemd service for managing all PostgreSQL clusters on the
    system. This
    # service is actually a systemd target, but we are using a service since
    # targets cannot be reloaded.

    [Unit]
    Description=PostgreSQL RDBMS

    [Service]
    Type=oneshot
    ExecStart=/bin/true
    ExecReload=/bin/true
    RemainAfterExit=on

    [Install]
    WantedBy=multi-user.target

The thing above the above is I do not see anything that starts Postgres.

Are you sure the same script is being called in cloud-init and outside of it?

Or more to the point, what script actually starts Postgres?




--
Adrian Klaver
adrian.klaver@aklaver.com

On 26 October 2015 at 18:21, Lasse Westh-Nielsen <lassewesth@gmail.com> wrote:
> Hey,
>
> I posted to stackoverflow about my problem upgrading from Ubuntu 14.04 to
> Ubuntu 15.04:
> http://stackoverflow.com/questions/33306475/ubuntu-15-04-postgresql-doesnt-start
>
> Tl;dr: postgresql service does not start properly when installed as a
> package using cloud-init.
>
> And I can't figure out if I am doing something wrong, if the AMI is no good,
> if the package has problems, ... I reckon I cannot be the first person to
> use Postgres on Ubuntu Vivid, but I have been hammering my head against the
> wall with this for hours.
>
> Any help greatly appreciated!

I'd be trying 'pg_ctlcluster --force 9.4 main stop'  and
'pg_ctlcluster 9.4 main start' instead of service start/stop. If it
works, you have narrowed down the problem to systemd and/or the
startup scripts. If it doesn't work, the problem is elsewhere (eg.
some required path like /var/run or /tmp or /var/log/postgresql
doesn't exist or has peculiar permissions). If nothing else, you
should get more informative output rather than systemd hiding it away
somewhere.

--
Stuart Bishop <stuart@stuartbishop.net>
http://www.stuartbishop.net/

On 10/26/2015 08:04 AM, Lasse Westh-Nielsen wrote:

Adrian,

The service starts once the package is installed.

  - It did that on Ubuntu14.04
  - On Ubuntu 15.04, `sudo service postgresql status` _claims_ it has
been started.

Yes, it starts but it is a go nowhere, do nothing service:

    $ cat /etc/systemd/system/multi-user.target.wants/postgresql.service
    # systemd service for managing all PostgreSQL clusters on the system. This
    # service is actually a systemd target, but we are using a service since
    # targets cannot be reloaded.

    [Unit]
    Description=PostgreSQL RDBMS

    [Service]
    Type=oneshot
    ExecStart=/bin/true
    ExecReload=/bin/true
    RemainAfterExit=on

    [Install]
    WantedBy=multi-user.target

All it does is start /bin/true which is why the status shows it started, but Postgres does not run. My guess is the answer is in here:

http://askubuntu.com/questions/632882/does-upstart-really-handle-all-startup-scripts

Second answer

" PostgreSQL comes with two systemd service unit files in version 15, /lib/systemd/system/postgresql@.service and /lib/systemd/system/postgresql.service. System 5 rc stuff is thus completely ignored."


Best guess is /lib/systemd/system/postgresql@.service is the one that actually starts Postgres. Sort of confirmed by the comments in the service script above:

"# systemd service for managing all PostgreSQL clusters on the system. This
# service is actually a systemd target, but we are using a service since
# targets cannot be reloaded."

Pretty sure /lib/systemd/system/postgresql.service is just a placeholder script.

At any rate Stuarts answer and your application of it confirms the service script is the issue.

On Mon, Oct 26, 2015 at 2:57 PM, Adrian Klaver

<adrian.klaver@aklaver.com <mailto:adrian.klaver@aklaver.com>> wrote:

    On 10/26/2015 07:08 AM, Lasse Westh-Nielsen wrote:

        Adrian,

        Thanks. I know about the systemd change, and indeed the postgres
        package
        I end up with _has_ systemd integration already:

             $ cat
        /etc/systemd/system/multi-user.target.wants/postgresql.service
             # systemd service for managing all PostgreSQL clusters on the
             system. This
             # service is actually a systemd target, but we are using a
        service since
             # targets cannot be reloaded.

             [Unit]
             Description=PostgreSQL RDBMS

             [Service]
             Type=oneshot
             ExecStart=/bin/true
             ExecReload=/bin/true
             RemainAfterExit=on

             [Install]
             WantedBy=multi-user.target


    The thing above the above is I do not see anything that starts Postgres.

    Are you sure the same script is being called in cloud-init and
    outside of it?

    Or more to the point, what script actually starts Postgres?




    --
    Adrian Klaver

    adrian.klaver@aklaver.com <mailto:adrian.klaver@aklaver.com>

--
Adrian Klaver
adrian.klaver@aklaver.com


--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

Just out of curiosity, is there anything in the postgresql.log which gives an indication of a problem?

On Mon, Oct 26, 2015 at 11:18 AM, Adrian Klaver <adrian.klaver@aklaver.com> wrote:

On 10/26/2015 08:04 AM, Lasse Westh-Nielsen wrote:

Adrian,

The service starts once the package is installed.

  - It did that on Ubuntu14.04
  - On Ubuntu 15.04, `sudo service postgresql status` _claims_ it has
been started.

Yes, it starts but it is a go nowhere, do nothing service:

    $ cat /etc/systemd/system/multi-user.target.wants/postgresql.service
    # systemd service for managing all PostgreSQL clusters on the system. This
    # service is actually a systemd target, but we are using a service since
    # targets cannot be reloaded.

    [Unit]
    Description=PostgreSQL RDBMS

    [Service]
    Type=oneshot
    ExecStart=/bin/true
    ExecReload=/bin/true
    RemainAfterExit=on

    [Install]
    WantedBy=multi-user.target

All it does is start /bin/true which is why the status shows it started, but Postgres does not run. My guess is the answer is in here:

http://askubuntu.com/questions/632882/does-upstart-really-handle-all-startup-scripts

Second answer

" PostgreSQL comes with two systemd service unit files in version 15, /lib/systemd/system/postgresql@.service and /lib/systemd/system/postgresql.service. System 5 rc stuff is thus completely ignored."


Best guess is /lib/systemd/system/postgresql@.service is the one that actually starts Postgres. Sort of confirmed by the comments in the service script above:

"# systemd service for managing all PostgreSQL clusters on the system. This
# service is actually a systemd target, but we are using a service since
# targets cannot be reloaded."

Pretty sure /lib/systemd/system/postgresql.service is just a placeholder script.

At any rate Stuarts answer and your application of it confirms the service script is the issue.

On Mon, Oct 26, 2015 at 2:57 PM, Adrian Klaver

<adrian.klaver@aklaver.com <mailto:adrian.klaver@aklaver.com>> wrote:

    On 10/26/2015 07:08 AM, Lasse Westh-Nielsen wrote:

        Adrian,

        Thanks. I know about the systemd change, and indeed the postgres
        package
        I end up with _has_ systemd integration already:

             $ cat
        /etc/systemd/system/multi-user.target.wants/postgresql.service
             # systemd service for managing all PostgreSQL clusters on the
             system. This
             # service is actually a systemd target, but we are using a
        service since
             # targets cannot be reloaded.

             [Unit]
             Description=PostgreSQL RDBMS

             [Service]
             Type=oneshot
             ExecStart=/bin/true
             ExecReload=/bin/true
             RemainAfterExit=on

             [Install]
             WantedBy=multi-user.target


    The thing above the above is I do not see anything that starts Postgres.

    Are you sure the same script is being called in cloud-init and
    outside of it?

    Or more to the point, what script actually starts Postgres?




    --
    Adrian Klaver

    adrian.klaver@aklaver.com <mailto:adrian.klaver@aklaver.com>

--
Adrian Klaver
adrian.klaver@aklaver.com


--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

--

Melvin Davidson
I reserve the right to fantasize.  Whether or not you
wish to share my fantasy is entirely up to you.