Thread: Postgres Pain Points 2 ruby / node language drivers
På torsdag 11. august 2016 kl. 19:13:08, skrev support-tiger <support@tigernassau.com>:
A database is only as good as the ability to use it. Using ORM's with
Postgres seems dumb, like having a Ferrari but only allowed to drive 25
kmh. Really, for getting data in and out with Postgres, SQL is pretty
simple. The ORM abstractions take as much effort as simply writing
direct SQL statements and convenience functions with straight sql.
Ruby: The "pg" driver seems to be tweaked to work okay with active
record, and maybe Sequel, but we are having problems with with straight
ruby/sinatra. (see comments above re ORM's). Since we changed to
Fedora and Ruby 2.3, we are now having breaking connections and JSONB
query problems. This could be from lack of clear docs and examples - we
cant' find any good examples or tutorials with JSONB. It seems with this
driver it is Rails or the highway.
Nodejs: The docs and tutorials for "node-postgres" are a mess. The
github readme, examples, and wikis are all different. Then there is
"native" and "pooling" and prepared statement alternatives. Callback
hell is bad enough, then to have lots of different approaches.
Async/Await should help this issue (whenever it arrives in Node) but
meanwhile, it's a painful struggle. Whatever examples exist are for
such trivial use, they fall short of helping with real-life examples,
like sequential results.
Python: The Postgres team supported driver psycopg2 seems to work great
(but we really don't want to rewrite lots of code into python
It would be great if the Ruby and Node drivers can be brought under the
Postgres team umbrella and make them as reliable and clearly documented
as the Python or jdbc drivers.
I cannot not comment on this. Saying that ORM seems dumb, and working with PG using ORM does not fly, is a very good recipe for not being taken seriously. Of course you can make an ORM hose your DB, but any developer could just as easily write SQL which does the same. Also, remember that PG is based on volunteer effort and if drivers for some languages are more mature and better supported than others, that's because core-pg-developers or someone in "the inner circle", or someone else, has use for them. Just because language X or framework Y popus up doesn't mean the PG-project should maintain drivers for them or support them.
So - yes, it would be great if all the drivers for all languages would be made as reliable and clearly documented as the most supported ones.
-- 
Andreas Joseph Krogh
CTO / Partner - Visena AS
Mobile: +47 909 56 963
Attachment
On Thu, Aug 11, 2016 at 4:20 PM, Andreas Joseph Krogh <andreas@visena.com> wrote:
>Nodejs: The docs and tutorials for "node-postgres" are a mess.På torsdag 11. august 2016 kl. 19:13:08, skrev support-tiger <support@tigernassau.com>:A database is only as good as the ability to use it. Using ORM's with
Postgres seems dumb, like having a Ferrari but only allowed to drive 25
kmh. Really, for getting data in and out with Postgres, SQL is pretty
simple. The ORM abstractions take as much effort as simply writing
direct SQL statements and convenience functions with straight sql.
Ruby: The "pg" driver seems to be tweaked to work okay with active
record, and maybe Sequel, but we are having problems with with straight
ruby/sinatra. (see comments above re ORM's). Since we changed to
Fedora and Ruby 2.3, we are now having breaking connections and JSONB
query problems. This could be from lack of clear docs and examples - we
cant' find any good examples or tutorials with JSONB. It seems with this
driver it is Rails or the highway.
Nodejs: The docs and tutorials for "node-postgres" are a mess. The
github readme, examples, and wikis are all different. Then there is
"native" and "pooling" and prepared statement alternatives. Callback
hell is bad enough, then to have lots of different approaches.
Async/Await should help this issue (whenever it arrives in Node) but
meanwhile, it's a painful struggle. Whatever examples exist are for
such trivial use, they fall short of helping with real-life examples,
like sequential results.
Python: The Postgres team supported driver psycopg2 seems to work great
(but we really don't want to rewrite lots of code into python
It would be great if the Ruby and Node drivers can be brought under the
Postgres team umbrella and make them as reliable and clearly documented
as the Python or jdbc drivers.I cannot not comment on this. Saying that ORM seems dumb, and working with PG using ORM does not fly, is a very good recipe for not being taken seriously. Of course you can make an ORM hose your DB, but any developer could just as easily write SQL which does the same. Also, remember that PG is based on volunteer effort and if drivers for some languages are more mature and better supported than others, that's because core-pg-developers or someone in "the inner circle", or someone else, has use for them. Just because language X or framework Y popus up doesn't mean the PG-project should maintain drivers for them or support them.So - yes, it would be great if all the drivers for all languages would be made as reliable and clearly documented as the most supported ones.
I blatantly disagree with your comment above. It sounds as if you are just making an excuse for not reading them. There are also many fine books that have been written
that cover PostgreSQL for novices and advanced users. No one is going to spoon feed you and there is no such thing as a "Vulcan Mind Meld" to give you instant knowledge.
Like any fine tool, you have to spend some time to learn how to use it properly.
--
Melvin Davidson
I reserve the right to fantasize. Whether or not you
wish to share my fantasy is entirely up to you.
I reserve the right to fantasize. Whether or not you
wish to share my fantasy is entirely up to you.
Attachment
On Thu, Aug 11, 2016 at 10:20 PM, Andreas Joseph Krogh <andreas@visena.com> wrote:
På torsdag 11. august 2016 kl. 19:13:08, skrev support-tiger <support@tigernassau.com>:I cannot not comment on this. Saying that ORM seems dumb, and working with PG using ORM does not fly, is a very good recipe for not being taken seriously.
And yet everyone I have talked to understands that ORMs are pretty problematic as a concept. They make some things easy but they have some pretty massive downsides. ORMs, it is true, do solve some problems, but they usually create many more in the process. The reason is that as much as relations look like collections of objects, they are best organized along very different principles. While we break down our tables based on functional dependencies between data values, we break down our object models based on how we can encapsulate state changes behind consistent interfaces. The latter is dependent on use, while the former far less so.
Of course you *can* use them well. I remember talking about this with one author or a major ORM and he said that on thing he often does is create views with triggers and then use the ORM against those. This solves the problem above very well. But it still leaves the fact that the database and the application have to share an implicit understanding of an object model and keeping that in sync as the project grows can be troublesome.
But once you have a non-trivial project, the promise that ORMs are often sold on ('you don't have to know SQL') evaporates and you find that you have to know SQL and the ORM well to get half-way decent performance.
Of course you can make an ORM hose your DB, but any developer could just as easily write SQL which does the same. Also, remember that PG is based on volunteer effort and if drivers for some languages are more mature and better supported than others, that's because core-pg-developers or someone in "the inner circle", or someone else, has use for them. Just because language X or framework Y popus up doesn't mean the PG-project should maintain drivers for them or support them.
Agreed on this.
So - yes, it would be great if all the drivers for all languages would be made as reliable and clearly documented as the most supported ones.
--
Best Wishes,
Chris Travers
Efficito: Hosted Accounting and ERP. Robust and Flexible. No vendor lock-in.
Attachment
På fredag 12. august 2016 kl. 05:27:42, skrev Chris Travers <chris.travers@gmail.com>:
On Thu, Aug 11, 2016 at 10:20 PM, Andreas Joseph Krogh <andreas@visena.com> wrote:På torsdag 11. august 2016 kl. 19:13:08, skrev support-tiger <support@tigernassau.com>:I cannot not comment on this. Saying that ORM seems dumb, and working with PG using ORM does not fly, is a very good recipe for not being taken seriously.And yet everyone I have talked to understands that ORMs are pretty problematic as a concept. They make some things easy but they have some pretty massive downsides. ORMs, it is true, do solve some problems, but they usually create many more in the process. The reason is that as much as relations look like collections of objects, they are best organized along very different principles. While we break down our tables based on functional dependencies between data values, we break down our object models based on how we can encapsulate state changes behind consistent interfaces. The latter is dependent on use, while the former far less so.
Of course you *can* use them well. I remember talking about this with one author or a major ORM and he said that on thing he often does is create views with triggers and then use the ORM against those. This solves the problem above very well. But it still leaves the fact that the database and the application have to share an implicit understanding of an object model and keeping that in sync as the project grows can be troublesome.
I don't understand why people bashing ORMs seem to think that once you have an ORM in you project you have to use it for everything. Of course, the ORM doesn't free you from using SQL directly where appropriate. IMO ORMs are best using for CRUD, but not for reporting or batch-processing. In a large project you have both, so combining is, IMO, the best.
But once you have a non-trivial project, the promise that ORMs are often sold on ('you don't have to know SQL') evaporates and you find that you have to know SQL and the ORM well to get half-way decent performance.
I don't believe for a second that having an ORM in a project relieves the developers from knowing SQL.
-- 
Andreas Joseph Krogh
CTO / Partner - Visena AS
Mobile: +47 909 56 963
Attachment
On Fri, Aug 12, 2016 at 9:59 AM, Andreas Joseph Krogh <andreas@visena.com> wrote:
På fredag 12. august 2016 kl. 05:27:42, skrev Chris Travers <chris.travers@gmail.com>:On Thu, Aug 11, 2016 at 10:20 PM, Andreas Joseph Krogh <andreas@visena.com>wrote: På torsdag 11. august 2016 kl. 19:13:08, skrev support-tiger <support@tigernassau.com>:I cannot not comment on this. Saying that ORM seems dumb, and working with PG using ORM does not fly, is a very good recipe for not being taken seriously.And yet everyone I have talked to understands that ORMs are pretty problematic as a concept. They make some things easy but they have some pretty massive downsides. ORMs, it is true, do solve some problems, but they usually create many more in the process. The reason is that as much as relations look like collections of objects, they are best organized along very different principles. While we break down our tables based on functional dependencies between data values, we break down our object models based on how we can encapsulate state changes behind consistent interfaces. The latter is dependent on use, while the former far less so.
Of course you *can* use them well. I remember talking about this with one author or a major ORM and he said that on thing he often does is create views with triggers and then use the ORM against those. This solves the problem above very well. But it still leaves the fact that the database and the application have to share an implicit understanding of an object model and keeping that in sync as the project grows can be troublesome.I don't understand why people bashing ORMs seem to think that once you have an ORM in you project you have to use it for everything. Of course, the ORM doesn't free you from using SQL directly where appropriate. IMO ORMs are best using for CRUD, but not for reporting or batch-processing. In a large project you have both, so combining is, IMO, the best.
The problems I mention above occur when doing CRUD via an ORM (at least all ORMs I have worked with). The fundamental problem is that ORMs make for bad data management in most cases because *either* you structure your database table structures around your object model of your application *or* you add complexity of a logical level structured for your application.
But either way, the database has to have intimate familiarity with the internals of the applications using it. And there isn't an easy way around this.
But once you have a non-trivial project, the promise that ORMs are often sold on ('you don't have to know SQL') evaporates and you find that you have to know SQL and the ORM well to get half-way decent performance.I don't believe for a second that having an ORM in a project relieves the developers from knowing SQL.
--
Best Wishes,
Chris Travers
Efficito: Hosted Accounting and ERP. Robust and Flexible. No vendor lock-in.
Attachment
På fredag 12. august 2016 kl. 10:33:19, skrev Chris Travers <chris.travers@gmail.com>:
On Fri, Aug 12, 2016 at 9:59 AM, Andreas Joseph Krogh <andreas@visena.com> wrote:På fredag 12. august 2016 kl. 05:27:42, skrev Chris Travers <chris.travers@gmail.com>:On Thu, Aug 11, 2016 at 10:20 PM, Andreas Joseph Krogh <andreas@visena.com>wrote: På torsdag 11. august 2016 kl. 19:13:08, skrev support-tiger <support@tigernassau.com>:I cannot not comment on this. Saying that ORM seems dumb, and working with PG using ORM does not fly, is a very good recipe for not being taken seriously.And yet everyone I have talked to understands that ORMs are pretty problematic as a concept. They make some things easy but they have some pretty massive downsides. ORMs, it is true, do solve some problems, but they usually create many more in the process. The reason is that as much as relations look like collections of objects, they are best organized along very different principles. While we break down our tables based on functional dependencies between data values, we break down our object models based on how we can encapsulate state changes behind consistent interfaces. The latter is dependent on use, while the former far less so.
Of course you *can* use them well. I remember talking about this with one author or a major ORM and he said that on thing he often does is create views with triggers and then use the ORM against those. This solves the problem above very well. But it still leaves the fact that the database and the application have to share an implicit understanding of an object model and keeping that in sync as the project grows can be troublesome.I don't understand why people bashing ORMs seem to think that once you have an ORM in you project you have to use it for everything. Of course, the ORM doesn't free you from using SQL directly where appropriate. IMO ORMs are best using for CRUD, but not for reporting or batch-processing. In a large project you have both, so combining is, IMO, the best.The problems I mention above occur when doing CRUD via an ORM (at least all ORMs I have worked with). The fundamental problem is that ORMs make for bad data management in most cases because *either* you structure your database table structures around your object model of your application *or* you add complexity of a logical level structured for your application.
But either way, the database has to have intimate familiarity with the internals of the applications using it. And there isn't an easy way around this.
If you don't like your domain-model to be very close to your physical DB-model, there's nothing preventing you from having a persistence-model, using the ORM, and map that to/from your domain-model. However, I don't see any of these challenges getting easier by throwing the ORM out and having the developers handling everything themselves using SQL directly.
-- 
Andreas Joseph Krogh
CTO / Partner - Visena AS
Mobile: +47 909 56 963
Attachment
On Fri, Aug 12, 2016 at 10:58 AM, Andreas Joseph Krogh <andreas@visena.com> wrote:
På fredag 12. august 2016 kl. 10:33:19, skrev Chris Travers <chris.travers@gmail.com>:Of course you *can* use them well. I remember talking about this with one author or a major ORM and he said that on thing he often does is create views with triggers and then use the ORM against those. This solves the problem above very well. But it still leaves the fact that the database and the application have to share an implicit understanding of an object model and keeping that in sync as the project grows can be troublesome.I don't understand why people bashing ORMs seem to think that once you have an ORM in you project you have to use it for everything. Of course, the ORM doesn't free you from using SQL directly where appropriate. IMO ORMs are best using for CRUD, but not for reporting or batch-processing. In a large project you have both, so combining is, IMO, the best.The problems I mention above occur when doing CRUD via an ORM (at least all ORMs I have worked with). The fundamental problem is that ORMs make for bad data management in most cases because *either* you structure your database table structures around your object model of your application *or* you add complexity of a logical level structured for your application.
But either way, the database has to have intimate familiarity with the internals of the applications using it. And there isn't an easy way around this.If you don't like your domain-model to be very close to your physical DB-model, there's nothing preventing you from having a persistence-model, using the ORM, and map that to/from your domain-model. However, I don't see any of these challenges getting easier by throwing the ORM out and having the developers handling everything themselves using SQL directly.
My preference is stored procedures plus service locators, to be honest. It enables a degree of loose coupling and even dynamic discovery that ORMs are generally not well suited to.
But I think the difference may be bigger. ORMs make sense when you want a database for your application. They break down badly when you want an application for your database. Usually I tend to want the latter.
Best Wishes,
Chris Travers
Efficito: Hosted Accounting and ERP. Robust and Flexible. No vendor lock-in.
Attachment
På fredag 12. august 2016 kl. 11:07:08, skrev Chris Travers <chris.travers@gmail.com>:
[snip]My preference is stored procedures plus service locators, to be honest. It enables a degree of loose coupling and even dynamic discovery that ORMs are generally not well suited to.But I think the difference may be bigger. ORMs make sense when you want a database for your application. They break down badly when you want an application for your database. Usually I tend to want the latter.
I'm in the first group, and I think many are when evaluating ORMs or having an opinion about them. Being in the latter, I more understand you concerns.
-- 
Andreas Joseph Krogh
CTO / Partner - Visena AS
Mobile: +47 909 56 963
Attachment
På fredag 12. august 2016 kl. 11:24:16, skrev Daevor The Devoted <dollien@gmail.com>:
[snip]If you don't like your domain-model to be very close to your physical DB-model, there's nothing preventing you from having a persistence-model, using the ORM, and map that to/from your domain-model. However, I don't see any of these challenges getting easier by throwing the ORM out and having the developers handling everything themselves using SQL directly.Could you elaborate on this method? Or direct me to docs describing it in more detail? I'm quite interested in this approach, but would like to see what amount of overhead (development and maintenance) this might result in.
I'm not in favor of this method myself as, IMO, it adds an unnecessary mapping-layer which just slows down development. But it might make sense depending on how much control you have on your DB, domain-model, code, developers etc.
I don't have any pointers describing this separation tho, try asking Mr. Google, that's what I'd do.
-- 
Andreas Joseph Krogh
CTO / Partner - Visena AS
Mobile: +47 909 56 963