Re: How different is AWS-RDS postgres? - Mailing list pgsql-general
From | Gmail |
---|---|
Subject | Re: How different is AWS-RDS postgres? |
Date | |
Msg-id | 6FE46A7E-FE5E-4F1C-ADE6-A393B572002A@gmail.com Whole thread Raw |
In response to | Re: How different is AWS-RDS postgres? (Sam Gendler <sgendler@ideasculptor.com>) |
List | pgsql-general |
Not important who does what with whom re Catalina/Tomcat ;)
I will indeed re-examine timeouts and such as inserting 100sK record is not instantaneous by any stretch. Tomcat is the new kid on my block as prior to this release I managed a naked port with a Selector and that had no trouble with this same test data. My biggest concern is the lack of server side indications. I also need to confirm the table create/bulk-copy and update to target table (segment) are in separate transactions. Doesn’t look like it, but they’re supposed to be. I want the data written to the db in bulk, then come back round and load to the final table in chunks.
Unless something has changed in recent years, the core servlet engine of tomcat IS catalina. Embedded tomcat is embedded catalina. It looks like the I/O error is a result of attempting to send a query on an already dead connection. I'd look for something that is limiting how long a connection can be open - either an explicit or default value for a timeout in the connection pool or on the server side. If you don't get the same behaviour when running against a database locally, I'd definitely look at the default settings in RDS. It may be automatically closing connections if they are idle for even a brief period.On Thu, May 27, 2021 at 3:35 PM Rob Sargent <robjsargent@gmail.com> wrote:On 5/27/21 4:25 PM, Sam Gendler wrote:That sure looks like something is causing your connection to have a transaction rollback. I haven't worked in Java in far too long, but it seems like your connection pool is under the impression your connection was abandoned so it reclaims it and rollback the transaction, which would explain why you aren't seeing the table when all is said and done - all of the work is being undone at the end.
One possibility, based on the catalina log you provided - if you have either end of the connection set up to automatically close idle connections after a period of time, then you might receive a closed connection from the pool, which will just error out when you attempt to run a query. In which case, you need to set up your connection pool to test a connection before it returns it to the requester. Usually something as simple as "select 2" will be sufficient to determine if the database connection is open. I can just about guarantee that your connection pool has a parameter which allows you to specify a query to execute when a connection is requested.
Well I /was/ doingcontextResource.setProperty("validationQuery", "SELECT 1");but I see that I lost that when I switched to using a properties file. Thanks for point me there.
The loop of 16 insert statement is in a single transaction, single connection so I'm not sure who's choking first. Is the connection idle after the I/O error or is the I/O error from a dead connection? (Small disclaimer: there is no catalina involved here, just an embedded tomcat instance.)
pgsql-general by date: