Some more notes on databses.
Although the pg_upgrade failed, I've decided to check the databases in the new cluster (10.4). There is no database with oid 0 either. Also to be noted that some system databases changed the oids while others retained them.
And of my databases - sslentry. It had a very big oid (can that seem strange?) and its oid has changed.
select oid, datname from pg_database;
oid | datname
-------+-----------
13011 | template0
16400 | bof
13012 | postgres
16401 | sslentry
1 | template1
(5 rows)