Thread: Containerized development environment for pgweb

Hello pgweb team,

Attaching a patch to simplify pgweb dev env setup.

Dockerfile - Main container with Django and PostgreSQL client
docker-compose.yml - to create proper dependency between python and 
postgres container
settings_local.py - Development setting, mounted inside container to 
override pgweb/settings.py
django_setup_local.sh - this script is executed at start of container 
and to setup pgweb specific env

Updated README.rst with commands.

Tested on Fedora 40 with podman as rootless containers. No custom setup 
was required.
start - `podman compose --file docker-compose.yml up --detach`
stop - `podman compose --file docker-compose.yml down`

-- 
Kind Regards,
Yogesh Sharma
PostgreSQL, Linux, and Networking Expert
Open Source Enthusiast and Advocate
PostgreSQL Contributors Team @ RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com

On 6/21/24 9:08 AM, Yogesh Sharma wrote:
> Hello pgweb team,
> 
> Attaching a patch to simplify pgweb dev env setup.
> 
> Dockerfile - Main container with Django and PostgreSQL client
> docker-compose.yml - to create proper dependency between python and 
> postgres container
> settings_local.py - Development setting, mounted inside container to 
> override pgweb/settings.py
> django_setup_local.sh - this script is executed at start of container 
> and to setup pgweb specific env
> 
> Updated README.rst with commands.
> 
> Tested on Fedora 40 with podman as rootless containers. No custom setup 
> was required.
> start - `podman compose --file docker-compose.yml up --detach`
> stop - `podman compose --file docker-compose.yml down`

This generally looks good to me. A few comments:

* Authentication - are we able to use "peer" auth instead of requiring a 
password? Though IIRC that may be challenging if we're using composable 
containers.
   * Otherwise, do we have a way to dynamically/securely generate the 
password?

Thanks,

Jonathan

On 2024-06-21 23:08, Yogesh Sharma wrote:
<snip>
> docker-compose.yml - to create proper dependency between python and
> postgres container

This one's using PG 16:

   image: postgres:16

Does anyone know if the PG infrastructure uses PG 16 too?

Asking because it'd make sense for them to match, just in case. :)

And if it doesn't matter what version runs, then using the
auto-upgrade container might be worth considering:

   https://hub.docker.com/repository/docker/pgautoupgrade/pgautoupgrade

---

Bear in mind that I'm not personally involved in the PostgreSQL WWW
side of things, so none of the above should be seen as anything more
than just ideas. :)

Regards and best wishes,

Justin Clift

On 2024-06-26 02:21, Jonathan S. Katz wrote:
<snip>
>   * Otherwise, do we have a way to dynamically/securely generate the 
> password?

Maybe having something call out to pwgen (or similar) would be feasible?

Regards and best wishes,

Justin Clift

On 6/26/24 04:23, Magnus Hagander wrote:
> On Wed, Jun 26, 2024 at 10:07 AM Justin Clift <justin@postgresql.org>
> wrote:
>
>     On 2024-06-21 23:08, Yogesh Sharma wrote:
>     <snip>
>     > docker-compose.yml - to create proper dependency between python and
>     > postgres container
>
>     This one's using PG 16:
>
>        image: postgres:16
>
>     Does anyone know if the PG infrastructure uses PG 16 too?
>
>     Asking because it'd make sense for them to match, just in case. :)
>
>
> It also is on python 3.7 which should be 3.9 to match what's deployed.
> In general, I'd recommend starting off the debian bullseye container
> instead of the python specific one -- that's what's used in the
> production deployments, so it makes sense to keep to the same.
> That way it could also be using the same versions of the debian
> packages for all requirements (basically everything except Django
> itself), to ensure it's exactly the same.
>
> //Magnus

Which Debian release is used by PG infrastructure?

--
Kind Regards,
Yogesh Sharma
PostgreSQL, Linux, and Networking Expert
Open Source Enthusiast and Advocate
PostgreSQL Contributors Team @ RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com

On 6/27/24 09:34, Magnus Hagander wrote:
>
>     > It also is on python 3.7 which should be 3.9 to match what's
>     deployed.
>     > In general, I'd recommend starting off the debian bullseye
>     container
>     > instead of the python specific one -- that's what's used in the
>     > production deployments, so it makes sense to keep to the same.
>     > That way it could also be using the same versions of the debian
>     > packages for all requirements (basically everything except Django
>     > itself), to ensure it's exactly the same.
>     >
>     > //Magnus
>
>     Which Debian release is used by PG infrastructure?
>
>
> Currently, bullseye.

When using python 3.9 PyYAML 3.13 (requirements.txt) fails to compile.
---
       ext/_yaml.c: In function ‘__Pyx_modinit_type_init_code’:
       ext/_yaml.c:25698:25: error: ‘PyTypeObject’ {aka ‘struct
_typeobject’} has no member named ‘tp_print’
       25698 |   __pyx_type_5_yaml_Mark.tp_print = 0;
             |                         ^
       ext/_yaml.c:25718:28: error: ‘PyTypeObject’ {aka ‘struct
_typeobject’} has no member named ‘tp_print’
       25718 |   __pyx_type_5_yaml_CParser.tp_print = 0;
             |                            ^
       ext/_yaml.c:25732:29: error: ‘PyTypeObject’ {aka ‘struct
_typeobject’} has no member named ‘tp_print’
       25732 |   __pyx_type_5_yaml_CEmitter.tp_print = 0;
             |                             ^
---

I assume, PyYAML was updated on system but not requirements.txt.
Possible to verify following package versions from the PG Infra server?

psycopg2==2.8.5
pycryptodomex>=3.4.7,<3.5
Markdown==3.0.1
requests-oauthlib==1.0.0
cvss==2.2
pytidylib==0.3.2
pycodestyle==2.4.0
pynliner==0.8.0
Babel==2.6.0
bleach==3.1.4
PyYAML==3.13



--
Kind Regards,
Yogesh Sharma
PostgreSQL, Linux, and Networking Expert
Open Source Enthusiast and Advocate
PostgreSQL Contributors Team @ RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com