Thread: Large scale reliable software system
Dear all,
After greeting,
I taught PostgreSQL myself and developed a small scale experimental software system using PostgreSQL in the back-end.
I would like to know your advices to develop a large scale reliable software system using PostgreSQL in the back-end, through which i can share the storage with the different system users where they login to the system through the web application front-end with different passwords and usernames , save the privacy of each user data, improve overall system security and performance, achieve fast response, make backups and save the stored data from loss. The system will be hosted on a cloud.
Thank you in advance.
On 6/26/23 16:48, B M wrote: > Dear all, > > After greeting, > > I taught PostgreSQL myself and developed a small scale > experimentalsoftware system using PostgreSQL in the back-end. > > I would like to know your advices to develop a large scale reliable > software system using PostgreSQL in the back-end, through which i can > share the storage with the different system users where they login to > the system through the web application front-end with different > passwords and usernames , save the privacy of each user data, improve > overall system security and performance, achieve fast response, make > backups and save the stored data from loss. The system will be hosted on > a cloud. https://www.djangoproject.com/ > > Thank you in advance. > -- Adrian Klaver adrian.klaver@aklaver.com
Likely because of the influence of the originator of Ruby on Rails, it is close to holy writ in the web development community that the database must be treated as a dumb data bucket and all business logic must be implemented in the Ruby or Python or whatever back end code.
This heuristic is nearly always mostly wrong.
Guyren G Howe
On 6/26/23 16:48, B M wrote:Dear all,
After greeting,
I taught PostgreSQL myself and developed a small scale
experimentalsoftware system using PostgreSQL in the back-end.
I would like to know your advices to develop a large scale reliable
software system using PostgreSQL in the back-end, through which i can
share the storage with the different system users where they login to
the system through the web application front-end with different
passwords and usernames , save the privacy of each user data, improve
overall system security and performance, achieve fast response, make
backups and save the stored data from loss. The system will be hosted on
a cloud.
https://www.djangoproject.com/
Thank you in advance.
--
Adrian Klaver
adrian.klaver@aklaver.com
It's not just Ruby, dumb databases are preferred in projects like WordPress, Drupal and Joomla, too. Now, if it's because they're used to using MySQL, well maybe that's not so hard to understand. :-) On Mon, Jun 26, 2023 at 8:05 PM Guyren Howe <guyren@gmail.com> wrote: > > This is a reasonable answer, but I want to offer a caveat. > > Likely because of the influence of the originator of Ruby on Rails, it is close to holy writ in the web development communitythat the database must be treated as a dumb data bucket and all business logic must be implemented in the Ruby orPython or whatever back end code. > > This heuristic is nearly always mostly wrong. > > Guyren G Howe > On Jun 26, 2023 at 17:48 -0700, Adrian Klaver <adrian.klaver@aklaver.com>, wrote: > > On 6/26/23 16:48, B M wrote: > > Dear all, > > After greeting, > > I taught PostgreSQL myself and developed a small scale > experimentalsoftware system using PostgreSQL in the back-end. > > I would like to know your advices to develop a large scale reliable > software system using PostgreSQL in the back-end, through which i can > share the storage with the different system users where they login to > the system through the web application front-end with different > passwords and usernames , save the privacy of each user data, improve > overall system security and performance, achieve fast response, make > backups and save the stored data from loss. The system will be hosted on > a cloud. > > > https://www.djangoproject.com/ > > > Thank you in advance. > > > -- > Adrian Klaver > adrian.klaver@aklaver.com > > >
On Jun 26, 2023, at 21:32, Michael Nolan <htfoot@gmail.com> wrote:It's not just Ruby, dumb databases are preferred in projects like
WordPress, Drupal and Joomla, too.
Now, if it's because they're used to using MySQL, well maybe that's
not so hard to understand. :-)
On Mon, Jun 26, 2023 at 8:05 PM Guyren Howe <guyren@gmail.com> wrote:
This is a reasonable answer, but I want to offer a caveat.
Likely because of the influence of the originator of Ruby on Rails, it is close to holy writ in the web development community that the database must be treated as a dumb data bucket and all business logic must be implemented in the Ruby or Python or whatever back end code.
This heuristic is nearly always mostly wrong.
Guyren G Howe
On Jun 26, 2023 at 17:48 -0700, Adrian Klaver <adrian.klaver@aklaver.com>, wrote:
On 6/26/23 16:48, B M wrote:
Dear all,
After greeting,
I taught PostgreSQL myself and developed a small scale
experimentalsoftware system using PostgreSQL in the back-end.
I would like to know your advices to develop a large scale reliable
software system using PostgreSQL in the back-end, through which i can
share the storage with the different system users where they login to
the system through the web application front-end with different
passwords and usernames , save the privacy of each user data, improve
overall system security and performance, achieve fast response, make
backups and save the stored data from loss. The system will be hosted on
a cloud.
https://www.djangoproject.com/
Thank you in advance.
--
Adrian Klaver
adrian.klaver@aklaver.com
So, what's your advice, and is there some book / resource to get upto speed?
Correct. It’s a tragically wrong piece of folk wisdom that’s pretty general across web development communities.On Jun 26, 2023, at 21:32, Michael Nolan <htfoot@gmail.com> wrote:It's not just Ruby, dumb databases are preferred in projects like
WordPress, Drupal and Joomla, too.
Now, if it's because they're used to using MySQL, well maybe that's
not so hard to understand. :-)
On Mon, Jun 26, 2023 at 8:05 PM Guyren Howe <guyren@gmail.com> wrote:
This is a reasonable answer, but I want to offer a caveat.
Likely because of the influence of the originator of Ruby on Rails, it is close to holy writ in the web development community that the database must be treated as a dumb data bucket and all business logic must be implemented in the Ruby or Python or whatever back end code.
This heuristic is nearly always mostly wrong.
Guyren G Howe
On Jun 26, 2023 at 17:48 -0700, Adrian Klaver <adrian.klaver@aklaver.com>, wrote:
On 6/26/23 16:48, B M wrote:
Dear all,
After greeting,
I taught PostgreSQL myself and developed a small scale
experimentalsoftware system using PostgreSQL in the back-end.
I would like to know your advices to develop a large scale reliable
software system using PostgreSQL in the back-end, through which i can
share the storage with the different system users where they login to
the system through the web application front-end with different
passwords and usernames , save the privacy of each user data, improve
overall system security and performance, achieve fast response, make
backups and save the stored data from loss. The system will be hosted on
a cloud.
https://www.djangoproject.com/
Thank you in advance.
--
Adrian Klaver
adrian.klaver@aklaver.com
Correct. It’s a tragically wrong piece of folk wisdom that’s pretty general across web development communities.On Jun 26, 2023, at 21:32, Michael Nolan <htfoot@gmail.com> wrote:It's not just Ruby, dumb databases are preferred in projects like
WordPress, Drupal and Joomla, too.
Now, if it's because they're used to using MySQL, well maybe that's
not so hard to understand. :-)
On Mon, Jun 26, 2023 at 8:05 PM Guyren Howe <guyren@gmail.com> wrote:
This is a reasonable answer, but I want to offer a caveat.
Likely because of the influence of the originator of Ruby on Rails, it is close to holy writ in the web development community that the database must be treated as a dumb data bucket and all business logic must be implemented in the Ruby or Python or whatever back end code.
This heuristic is nearly always mostly wrong.
Guyren G Howe
On Jun 26, 2023 at 17:48 -0700, Adrian Klaver <adrian.klaver@aklaver.com>, wrote:
On 6/26/23 16:48, B M wrote:
--
Adrian Klaver
adrian.klaver@aklaver.com
On Tue, 27 Jun 2023 at 07:08, Guyren Howe <guyren@gmail.com> wrote:Correct. It’s a tragically wrong piece of folk wisdom that’s pretty general across web development communities.On Jun 26, 2023, at 21:32, Michael Nolan <htfoot@gmail.com> wrote:It's not just Ruby, dumb databases are preferred in projects like
WordPress, Drupal and Joomla, too.
Now, if it's because they're used to using MySQL, well maybe that's
not so hard to understand. :-)
On Mon, Jun 26, 2023 at 8:05 PM Guyren Howe <guyren@gmail.com> wrote:
This is a reasonable answer, but I want to offer a caveat.
Likely because of the influence of the originator of Ruby on Rails, it is close to holy writ in the web development community that the database must be treated as a dumb data bucket and all business logic must be implemented in the Ruby or Python or whatever back end code.
This heuristic is nearly always mostly wrong.
Guyren G Howe
On Jun 26, 2023 at 17:48 -0700, Adrian Klaver <adrian.klaver@aklaver.com>, wrote:
On 6/26/23 16:48, B M wrote:
--
Adrian Klaver
adrian.klaver@aklaver.comThe accepted front-end developer wisdom of treating the DB as a dumb data store works under conditions, for example the DB will never be accessed from a different ORM / framework, and where the performance attributes of using an ORM with 'standard' datastructures are acceptable.The moment you need to plug in something like reporting tools, or access from a different system / API / framework / language / ORM or whatever, the approach not having rules / views / procedures / whatever built into the database falls apart.Other things to consider are performance / load / overhead: we have one system that involves processing through large amounts of data for reports / queries. Shipping all that back through the ORM / db interface (ODBC / JDBC / psycopg2 / whatever for resolution / filtering on the front end application where SQL / procedures / views could do that in the DB and just ship back the required data seems counterproductive.Tony Shelver
Dear all,
After greeting,
I taught PostgreSQL myself and developed a small scale experimental software system using PostgreSQL in the back-end.
I would like to know your advices to develop a large scale reliable software system using PostgreSQL in the back-end, through which i can share the storage with the different system users where they login to the system through the web application front-end with different passwords and usernames , save the privacy of each user data, improve overall system security and performance, achieve fast response, make backups and save the stored data from loss. The system will be hosted on a cloud.
Thank you in advance.
On 6/27/23 07:58, Merlin Moncure wrote: > > > On Mon, Jun 26, 2023 at 6:49 PM B M <bmmasprep@gmail.com > <mailto:bmmasprep@gmail.com>> wrote: > > Dear all, > > After greeting, > > I taught PostgreSQL myself and developed a small scale > experimentalsoftware system using PostgreSQL in the back-end. > > I would like to know your advices to develop a large scale reliable > software system using PostgreSQL in the back-end, through which i > can share the storage with the different system users where they > login to the system through the web application front-end with > different passwords and usernames , save the privacy of each user > data, improve overall system security and performance, achieve fast > response, make backups and save the stored data from loss. The > system will be hosted on a cloud. > > Thank you in advance. > > > * your sql is code, and treat it as such, check it into git etc > * robust deployment strategy is essential to scaling team > * write a lot of tests > * become intimate with pg_stat_statements > * keep your transactions as short as possible while preserving safety > * avoid developers who advocate for keeping business logic out of the > database religiously (controversial) > * try to avoid assumptions that only one technical stack interacts with > your database > * do not waste time making ERDs use a tool that generates them (i like > schemaspy) > * test your DR strategy before disaster strikes > * think about security model up front +1 > > merlin > -- Adrian Klaver adrian.klaver@aklaver.com