Simple repo browsers for the website (PoC) - Mailing list pgsql-www

From Dave Page
Subject Simple repo browsers for the website (PoC)
Date
Msg-id CA+OCxoxyXqwaVUbwtEJ_nHbRV3jPiujt1CFeTpG8rLF+PEvnTg@mail.gmail.com
Whole thread Raw
Responses Re: Simple repo browsers for the website (PoC)  (Ray O'Donnell <ray@rodonnell.ie>)
List pgsql-www
I've been working on a simple browser for the apt/yum/zypp repos to add to the website, for which a PoC is attached.

There has been past discussion of a similar patch from Christoph Berg (https://www.postgresql.org/message-id/flat/CA%2BOCxoxK1KR7cyn_wn-VwVfYOvpUXhooNC0VAuOysPp2uoM8%3DQ%40mail.gmail.com#de0886d11ab262a5dddfcffb94eaa051), however, I believe that patch is a) way more complex than we'd want on the main website, both to maintain and for the user experience for the vast majority of visitors, and b) it only supports apt, not yum and zypp.

This patch implements a pretty simple browser, allowing users to search, list and view details of all the packages in any of the repos. A separate script (which would run on ftpmaster) would scan the repositories periodically, and load the data into the database behind the website. 

Current limitations:

- Search is pretty basic, allowing substring matching on the package name and selection of the repository (e.g. "fedora-31 testing"). That can be enhanced to allow drill-down selections (distro, arch, repo etc), but that would really require some JS scripting which whilst easy enough, we typically try to avoid if there's no pure-HTML alternative.

- I can't figure out a sane way to get the licence for .deb packages. That seems to be the one thing that's not in the repo metadata, and would require reading the package itself (and there are thousands of them, so...).

- Loading the data is currently a transactional delete/reload process. That's somewhat icky, but doing a better job requires figuring out what packages have been deleted from the repos which I suspect may be more expensive in the long run.

I've also attached a patch that will sync the required repo meta data from the PostgreSQL servers for local testing.

The loader script (scan-repos.py) needs the psycopg2 and repomd packages.

I've also attached a couple of screenshots.

Comments/suggestions/objections please!

--
Attachment

pgsql-www by date:

Previous
From: Rocio Avila
Date:
Subject: Wiki editor request
Next
From: Ray O'Donnell
Date:
Subject: Re: Simple repo browsers for the website (PoC)