Re: [ADMIN] Processing very large TEXT columns (300MB+) using C/libpq - Mailing list pgsql-admin

From Bear Giles
Subject Re: [ADMIN] Processing very large TEXT columns (300MB+) using C/libpq
Date
Msg-id CALBNtw7TzuoJ91hjQHqVQzjXqQg8US-hF17BH+LbLNo_+ayGEg@mail.gmail.com
Whole thread Raw
In response to Re: [ADMIN] Processing very large TEXT columns (300MB+) using C/libpq  (Geoff Winkless <pgsqladmin@geoff.dj>)
Responses Re: [ADMIN] Processing very large TEXT columns (300MB+) using C/libpq  (Geoff Winkless <pgsqladmin@geoff.dj>)
List pgsql-admin
​In that case you must put a read lock on the string that covers the loop. If you're in a multi-threaded environment and not using locks when appropriate then all bets are off.


On Fri, Oct 20, 2017 at 12:05 PM, Geoff Winkless <pgsqladmin@geoff.dj> wrote:
On 20 October 2017 at 18:52, Bear Giles <bgiles@coyotesong.com> wrote:
As an aside any halfway decent optimizer would realize that the results of strlen() are unchanging as long as the contents of what it's passed isn't modified. That's a common enough pattern that it should be checked.

​IME this is a myth perpetuated by bad computer science lecturers who haven't thought through the consequences of what they're saying.​ strlen() can change because of changes inside the loop but also because of also threads in the system; I've not yet seen a compiler optimise that away, and neither should it, IMO.

G

pgsql-admin by date:

Previous
From: Geoff Winkless
Date:
Subject: Re: [ADMIN] Processing very large TEXT columns (300MB+) using C/libpq
Next
From: Ervin Weber
Date:
Subject: [ADMIN] confusing .pgpass behaviour for undocumented replication=trueconnection parameter