Thread: Development environment setup for pgarchives and pgweb

Development environment setup for pgarchives and pgweb

From
Sahil Harpal
Date:
Hello, 
As a part of GSoC 2022, I am working on the project pgarchives. I have cloned both pgarchives & pgweb and installed all the dependencies. But I am facing a problem in connecting pgarchives to pgweb. If I open the pgarchives in the browser, it first redirects me to login screen of pgweb. Then after giving credentials and submitting the form, it remains on the same screen. Just the URL is changing. Also I am getting 404 on accessing mailing lists from the mailing lists option present in the quick links section of pgweb.
Could you please help me resolve these issues and to set this up?

Thanks,
Sahil Harpal
Attachment

Re: Development environment setup for pgarchives and pgweb

From
Célestin Matte
Date:
Hello,

I wrote a document a while ago to install pglister, pgweb and pgarchives that might help you, although your setup is
notexactly the same: 
 
https://gitlab.com/cmatte/pglister/-/blob/master/INSTALL.md
I guess you're missing the part where you have to create a community auth site in pgweb's
/admin/account/communityauthorg/add/and /admin/account/communityauthsite/add/
 

Good luck,
-- 
Célestin Matte



Re: Development environment setup for pgarchives and pgweb

From
Sahil Harpal
Date:
Thanks for such a quick response and installation document.
I am not aware of this pglister. Do I need to set up a pglister as well? 
Also, I have already added community org and auth in pgweb.

Thanks,
Sahil Harpal

On Thu, 2 Jun 2022 at 13:30, Célestin Matte <celestin.matte@cmatte.me> wrote:
Hello,

I wrote a document a while ago to install pglister, pgweb and pgarchives that might help you, although your setup is not exactly the same:
https://gitlab.com/cmatte/pglister/-/blob/master/INSTALL.md
I guess you're missing the part where you have to create a community auth site in pgweb's /admin/account/communityauthorg/add/ and /admin/account/communityauthsite/add/

Good luck,
--
Célestin Matte
Attachment

Re: Development environment setup for pgarchives and pgweb

From
Alvaro Herrera
Date:
On 2022-Jun-02, Sahil Harpal wrote:

> Thanks for such a quick response and installation document.
> I am not aware of this pglister. Do I need to set up a pglister as well?
> Also, I have already added community org and auth in pgweb.

pglister is the mailing list distribution software.  You probably don't
need that in order to work with pgweb.

-- 
Álvaro Herrera         PostgreSQL Developer  —  https://www.EnterpriseDB.com/
"Uno puede defenderse de los ataques; contra los elogios se esta indefenso"



Re: Development environment setup for pgarchives and pgweb

From
Magnus Hagander
Date:


On Thu, Jun 2, 2022, 09:38 Sahil Harpal <sahilharpal1234@gmail.com> wrote:
Hello, 
As a part of GSoC 2022, I am working on the project pgarchives. I have cloned both pgarchives & pgweb and installed all the dependencies. But I am facing a problem in connecting pgarchives to pgweb. If I open the pgarchives in the browser, it first redirects me to login screen of pgweb. Then after giving credentials and submitting the form, it remains on the same screen. Just the URL is changing. Also I am getting 404 on accessing mailing lists from the mailing lists option present in the quick links section of pgweb.
Could you please help me resolve these issues and to set this up?


It is normal that the link to /list/ directly on the site does not work. For that to work you need to set up a reverse proxy that forwards /list/ on the pgweb port to whatever port you run pgarchives on. But that's not needed to test and use pgarchives. 

The majority of pgarchives doesn't need or use any authentication at all, but I assume you're planning to do something that needs it. It's hard to figure out from your screenshot what is actually failing though. Which port is pgarchives on? And what url is actrually triggering the initial redirect? Everything in your screenshots seems to be from pgweb and nothing from pgarchives?

/Magnus 

Re: Development environment setup for pgarchives and pgweb

From
Sahil Harpal
Date:
Hello Magnus,
Yeah you are right /list/ will not work directly in pgweb.
I also think authentication doesn't need to access most of the pgarchives' data. It will be required only to get private data. In the current setup, I am running pgarchives on port 9000 and pgweb on 8000. In the community auth of pgweb, I added the URL of pgarchives in the redirect URL field. http://127.0.0.1:9000 redirects me to the login screen of pgweb.
I think I don't even need pgweb. In the current configuration, I changed PUBLIC_ARCHIVES = True & ALLOW_RESEND = False and it worked. I guess making these changes skips the authentication step, which is not required to access public data.
I also made a few changes to access /admin/. Since in the code present on the github this part is disabled. Now I can add sample/dummy data from the admin panel.
Since my work will involve mostly frontend parts, I think for now I don't even need to think about the reverse proxy thing. I can directly work on the templates directory of pgarchives.
Thanks for giving me the right direction!
---
Sahil Harpal


On Fri, 3 Jun 2022 at 21:49, Magnus Hagander <magnus@hagander.net> wrote:


On Thu, Jun 2, 2022, 09:38 Sahil Harpal <sahilharpal1234@gmail.com> wrote:
Hello, 
As a part of GSoC 2022, I am working on the project pgarchives. I have cloned both pgarchives & pgweb and installed all the dependencies. But I am facing a problem in connecting pgarchives to pgweb. If I open the pgarchives in the browser, it first redirects me to login screen of pgweb. Then after giving credentials and submitting the form, it remains on the same screen. Just the URL is changing. Also I am getting 404 on accessing mailing lists from the mailing lists option present in the quick links section of pgweb.
Could you please help me resolve these issues and to set this up?


It is normal that the link to /list/ directly on the site does not work. For that to work you need to set up a reverse proxy that forwards /list/ on the pgweb port to whatever port you run pgarchives on. But that's not needed to test and use pgarchives. 

The majority of pgarchives doesn't need or use any authentication at all, but I assume you're planning to do something that needs it. It's hard to figure out from your screenshot what is actually failing though. Which port is pgarchives on? And what url is actrually triggering the initial redirect? Everything in your screenshots seems to be from pgweb and nothing from pgarchives?

/Magnus 

Re: Development environment setup for pgarchives and pgweb

From
Magnus Hagander
Date:


On Fri, Jun 3, 2022 at 8:52 PM Sahil Harpal <sahilharpal1234@gmail.com> wrote:
Hello Magnus,
Yeah you are right /list/ will not work directly in pgweb.

For those types of pages, you shouldn't need pgweb at all, since they do not require authentication.

 
I also think authentication doesn't need to access most of the pgarchives' data. It will be required only to get private data. In the current setup, I am running pgarchives on port 9000 and pgweb on 8000. In the community auth of pgweb, I added the URL of pgarchives in the redirect URL field. http://127.0.0.1:9000 redirects me to the login screen of pgweb.

The redirect url should be http://127.0.0.1:9000/list/_auth/auth_receive/ for pgarchives on port 9000.

 
I think I don't even need pgweb. In the current configuration, I changed PUBLIC_ARCHIVES = True & ALLOW_RESEND = False and it worked. I guess making these changes skips the authentication step, which is not required to access public data.

Correct.

 
I also made a few changes to access /admin/. Since in the code present on the github this part is disabled. Now I can add sample/dummy data from the admin panel.
Since my work will involve mostly frontend parts, I think for now I don't even need to think about the reverse proxy thing. I can directly work on the templates directory of pgarchives.


Yes, there is no /admin/ part for the archives, as in normal usage the data is all fed from other systems. But I can see how it helps to add dummy data in a dev situation.

//Magnus

Re: Development environment setup for pgarchives and pgweb

From
Sahil Harpal
Date:
Can I get some sample data (.json files)? So that the dev environment will also simulate the actual project and will cover different possibilities/edge cases. It will be easy to set up db by directly loading json files instead of manually adding data from the admin dashboard.
---
Sahil Harpal


On Sun, 5 Jun 2022 at 01:01, Magnus Hagander <magnus@hagander.net> wrote:


On Fri, Jun 3, 2022 at 8:52 PM Sahil Harpal <sahilharpal1234@gmail.com> wrote:
Hello Magnus,
Yeah you are right /list/ will not work directly in pgweb.

For those types of pages, you shouldn't need pgweb at all, since they do not require authentication.

 
I also think authentication doesn't need to access most of the pgarchives' data. It will be required only to get private data. In the current setup, I am running pgarchives on port 9000 and pgweb on 8000. In the community auth of pgweb, I added the URL of pgarchives in the redirect URL field. http://127.0.0.1:9000 redirects me to the login screen of pgweb.

The redirect url should be http://127.0.0.1:9000/list/_auth/auth_receive/ for pgarchives on port 9000.

 
I think I don't even need pgweb. In the current configuration, I changed PUBLIC_ARCHIVES = True & ALLOW_RESEND = False and it worked. I guess making these changes skips the authentication step, which is not required to access public data.

Correct.

 
I also made a few changes to access /admin/. Since in the code present on the github this part is disabled. Now I can add sample/dummy data from the admin panel.
Since my work will involve mostly frontend parts, I think for now I don't even need to think about the reverse proxy thing. I can directly work on the templates directory of pgarchives.


Yes, there is no /admin/ part for the archives, as in normal usage the data is all fed from other systems. But I can see how it helps to add dummy data in a dev situation.

//Magnus

Re: Development environment setup for pgarchives and pgweb

From
Magnus Hagander
Date:

On Mon, Jun 6, 2022 at 8:43 PM Sahil Harpal <sahilharpal1234@gmail.com> wrote:


(please don't top-post)


Can I get some sample data (.json files)? So that the dev environment will also simulate the actual project and will cover different possibilities/edge cases. It will be easy to set up db by directly loading json files instead of manually adding data from the admin dashboard.

We don't have that data readily available. But the easiest way to load up some data is to download an mbox from the existing archives and use the loading tool to reload it into your database.

//Magnus