Thread: PostgreSQL 10.20 crashes / Antivirus

PostgreSQL 10.20 crashes / Antivirus

From
"Thomas, Richard"
Date:

I’ve been having problems with several PostgreSQL crashes over the last month (originally with 10.16, then more recently with 10.20, all on a Windows Server 2012R2 platform), always with a low level permissions error:

"server process (PID 5664) was terminated by exception 0xC0000022"

(This NT error translates as "Access Denied: A process has requested access to an object but has not been granted those access rights").

 

I was certainly having issues earlier with McAfee Antivirus causing some of these problems but this has been reduced after following the advice here to prevent scanning of PGDATA folders and postgres.exe (if someone can confirm that reference to “postgresql.exe” is a typo on that page I will correct it, but don’t want to in case it refers to a different version of PostgreSQL):

https://wiki.postgresql.org/wiki/Running_%26_Installing_PostgreSQL_On_Native_Windows#Antivirus_software

I have also got our IT to disable antivirus on our tablespace folder and on the folder running our backup PowerShell script (which calls pg_dump).

 

The last few crashes have occurred during the backup process, with the most recent one apparently after one database pg_dump completes (successfully) but before the next database pgdump can begin – although the new pg_dump log file was created (is empty) it does not get as far as creating the .backup file and the PostgreSQL log file shows no new operations (I have logging set quite verbose including log_min_duration_statement = 0    to log every statement). I have copied out the bit running up to the failure at the end of this email where you can see the final part (dumping of sde_xml_indexes) of the pg_dump on database a9sde, then it disconnecting from that database.

 

Any suggestions for how to fix/diagnose? Some things I have tried already:

  • Ran “sfc” (System File Checker) to check for/fix any Windows OS corruptions (none found that were relevant)
  • Ran “amcheck” on several of the databases
  • Unfortunately checksums are not turned on with the database (not an option with the EDB PostgreSQL Windows Installer) and can’t be turned on after initdb on version 10
  • I hoped to run pg_catcheck but cannot find a binary version of this online and I think it would take me quite a lot of work to build it myself (I don’t have Visual Studio, only VSC).

 

Richard

 

2022-04-02 02:14:53 BST: [7480] user=scde_suad,db=a9sde,app=pg_dump,host=a9dualling.atkinsgeospatial.com LOG:  00000: duration: 0.816 ms  statement: COPY sde.sde_xml_indexes (index_id, index_name, owner, index_type, description) TO stdout;

2022-04-02 02:14:53 BST: [7480] user=scde_suad,db=a9sde,app=pg_dump,host=a9dualling.atkinsgeospatial.com LOCATION:  exec_simple_query, postgres.c:1200

2022-04-02 02:14:54 BST: [7480] user=scde_suad,db=a9sde,app=pg_dump,host=a9dualling.atkinsgeospatial.com LOG:  00000: disconnection: session time: 0:36:52.430 user=scde_suad database=a9sde host=a9dualling.atkinsgeospatial.com port=59467

2022-04-02 02:14:54 BST: [7480] user=scde_suad,db=a9sde,app=pg_dump,host=a9dualling.atkinsgeospatial.com LOCATION:  log_disconnections, postgres.c:4577

2022-04-02 02:14:55 BST: [5016] LOG:  00000: server process (PID 100) was terminated by exception 0xC0000022

At Atkins - member of the SNC-Lavalin Group, we work flexible hours around the world. Although I have sent this email at a time convenient for me, I don't expect you to respond until it works for you.

NOTICE – This email message and any attachments may contain information or material that is confidential, privileged, and/or subject to copyright or other rights. Any unauthorized viewing, disclosure, retransmission, dissemination, or other use of or reliance on this message or anything contained therein is strictly prohibited and may be unlawful. If you believe you may have received this message in error, kindly inform the sender by return email and delete this message from your system. Thank you.

Re: PostgreSQL 10.20 crashes / Antivirus

From
Adrian Klaver
Date:
On 4/6/22 03:59, Thomas, Richard wrote:
> I’ve been having problems with several PostgreSQL crashes over the last 
> month (originally with 10.16, then more recently with 10.20, all on a 
> Windows Server 2012R2 platform), always with a low level permissions error:
> 
> "server process (PID 5664) was terminated by exception 0xC0000022"
> 
> (This NT error translates as "Access Denied: A process has requested 
> access to an object but has not been granted those access rights").
> 

> I have also got our IT to disable antivirus on our tablespace folder and 
> on the folder running our backup PowerShell script (which calls pg_dump).
> 

Is either one of those directories where the dump file is being output to?

If not is the directory that file is being created in have AV checks 
disabled?



-- 
Adrian Klaver
adrian.klaver@aklaver.com



RE: PostgreSQL 10.20 crashes / Antivirus

From
"Thomas, Richard"
Date:
Ah... no - the AV is still allowed to run riot in the folder where the dump file is written to. Do you think that could
bethe issue?
 

Richard

-----Original Message-----
From: Adrian Klaver <adrian.klaver@aklaver.com>
Sent: 06 April 2022 15:42
To: Thomas, Richard <Richard.Thomas@atkinsglobal.com>; pgsql-general@lists.postgresql.org
Subject: Re: PostgreSQL 10.20 crashes / Antivirus

On 4/6/22 03:59, Thomas, Richard wrote:
> I’ve been having problems with several PostgreSQL crashes over the
> last month (originally with 10.16, then more recently with 10.20, all
> on a Windows Server 2012R2 platform), always with a low level permissions error:
>
> "server process (PID 5664) was terminated by exception 0xC0000022"
>
> (This NT error translates as "Access Denied: A process has requested
> access to an object but has not been granted those access rights").
>

> I have also got our IT to disable antivirus on our tablespace folder
> and on the folder running our backup PowerShell script (which calls pg_dump).
>

Is either one of those directories where the dump file is being output to?

If not is the directory that file is being created in have AV checks disabled?



--
Adrian Klaver
adrian.klaver@aklaver.com
At Atkins - member of the SNC-Lavalin Group, we work flexible hours around the world. Although I have sent this email
ata time convenient for me, I don't expect you to respond until it works for you.
 
NOTICE – This email message and any attachments may contain information or material that is confidential, privileged,
and/orsubject to copyright or other rights. Any unauthorized viewing, disclosure, retransmission, dissemination, or
otheruse of or reliance on this message or anything contained therein is strictly prohibited and may be unlawful. If
youbelieve you may have received this message in error, kindly inform the sender by return email and delete this
messagefrom your system. Thank you.
 

Re: PostgreSQL 10.20 crashes / Antivirus

From
Adrian Klaver
Date:
On 4/6/22 07:55, Thomas, Richard wrote:
> Ah... no - the AV is still allowed to run riot in the folder where the dump file is written to. Do you think that
couldbe the issue?
 

I would say that would be the most likely cause. Mainly because I can 
not think of another for this, "Access Denied: A process has requested
access to an object but has not been granted those access rights", 
happening after a previous dump file had been successfully saved in the 
same location.

> 
> Richard


-- 
Adrian Klaver
adrian.klaver@aklaver.com



Re: PostgreSQL 10.20 crashes / Antivirus

From
Adrian Klaver
Date:
On 4/6/22 07:55, Thomas, Richard wrote:
> Ah... no - the AV is still allowed to run riot in the folder where the dump file is written to. Do you think that
couldbe the issue?
 

Should have added to previous post:

Try directing the pg_dump to one of the directories that have AV 
scanning disabled(if that is possible) and see if it completes.

> 
> Richard
> 


-- 
Adrian Klaver
adrian.klaver@aklaver.com



RE: PostgreSQL 10.20 crashes / Antivirus

From
"Thomas, Richard"
Date:
A request for further suggestions for how to fix/diagnose this PG crash (having just got back from holiday I discover
thatPG is still crashing even though I have changed the pgdump target output location to a folder excluded from the
McAfeeAntivirus scanning - but thanks for the suggestion Adrian). The last few crashes have occurred only during the
backupprocess, when database pg_dump completes (successfully - including the disconnection message in the log) but
beforethe next database pgdump can begin (that pg_dump command gets the error that it cannot connect to the database as
theserver is no longer responding). This will happen after dumping several databases - sometimes it manages the
completemultiple database dump fine. One new thing I've noticed is that when it fails the file access 0xC0000022 error
occursa second time when PG is trying to restart itself after shutting down all the other processes, suggesting
whatevercaused the crash is still around (a later manual restart works fine):
 

2022-04-15 03:54:06 BST: [3720] WARNING:  57P02: terminating connection because of crash of another server process
2022-04-15 03:54:06 BST: [3720] DETAIL:  The postmaster has commanded this server process to roll back the current
transactionand exit, because another server process exited abnormally and possibly corrupted shared memory.
 
2022-04-15 03:54:06 BST: [3720] HINT:  In a moment you should be able to reconnect to the database and repeat your
command.
2022-04-15 03:54:06 BST: [3720] LOCATION:  quickdie, postgres.c:2622
2022-04-15 03:54:06 BST: [7900] LOG:  00000: all server processes terminated; reinitializing
2022-04-15 03:54:06 BST: [7900] LOCATION:  PostmasterStateMachine, postmaster.c:3924
2022-04-15 03:54:08 BST: [7900] LOG:  00000: startup process (PID 3652) was terminated by exception 0xC0000022
2022-04-15 03:54:08 BST: [7900] HINT:  See C include file "ntstatus.h" for a description of the hexadecimal value.
2022-04-15 03:54:08 BST: [7900] LOCATION:  LogChildExit, postmaster.c:3650
2022-04-15 03:54:08 BST: [7900] LOG:  00000: aborting startup due to startup process failure
2022-04-15 03:54:08 BST: [7900] LOCATION:  reaper, postmaster.c:2907
2022-04-15 03:54:09 BST: [7900] LOG:  00000: database system is shut down
2022-04-15 03:54:09 BST: [7900] LOCATION:  UnlinkLockFiles, miscinit.c:765

Some details from my original email (as it was 2 weeks ago):

I’ve been having problems with several PostgreSQL crashes over the last month (originally with 10.16, then more
recentlywith 10.20, all on a Windows Server 2012R2 platform), always with a low level permissions error:
 
"server process (PID 5664) was terminated by exception 0xC0000022"
(This NT error translates as "Access Denied: A process has requested access to an object but has not been granted those
accessrights").
 

I was certainly having issues earlier with McAfee Antivirus causing some of these problems but this has been reduced
afterfollowing the advice here to prevent scanning of PGDATA folders and postgres.exe (if someone can confirm that
referenceto “postgresql.exe” is a typo on that page I will correct it, but don’t want to in case it refers to a
differentversion of PostgreSQL):
 
https://wiki.postgresql.org/wiki/Running_%26_Installing_PostgreSQL_On_Native_Windows#Antivirus_software
I have also got our IT to disable antivirus on our tablespace folder and on the folder running our backup PowerShell
script(which calls pg_dump).
 

Any suggestions for how to fix/diagnose? Some things I have tried already:
- Ran “sfc” (System File Checker) to check for/fix any Windows OS corruptions (none found that were relevant)
- Ran “amcheck” on several of the databases
- Unfortunately checksums are not turned on with the database (not an option with the EDB PostgreSQL Windows Installer)
andcan’t be turned on after initdb on version 10
 
- I hoped to run pg_catcheck but cannot find a binary version of this online and I think it would take me quite a lot
ofwork to build it myself (I don’t have Visual Studio, only VSC).
 

Richard

2022-04-02 02:14:53 BST: [7480] user=scde_suad,db=a9sde,app=pg_dump,host=a9dualling.atkinsgeospatial.com LOG:  00000:
duration:0.816 ms  statement: COPY sde.sde_xml_indexes (index_id, index_name, owner, index_type, description) TO
stdout;
2022-04-02 02:14:53 BST: [7480] user=scde_suad,db=a9sde,app=pg_dump,host=a9dualling.atkinsgeospatial.com LOCATION:
exec_simple_query,postgres.c:1200
 
2022-04-02 02:14:54 BST: [7480] user=scde_suad,db=a9sde,app=pg_dump,host=a9dualling.atkinsgeospatial.com LOG:  00000:
disconnection:session time: 0:36:52.430 user=scde_suad database=a9sde host=a9dualling.atkinsgeospatial.com port=59467
 
2022-04-02 02:14:54 BST: [7480] user=scde_suad,db=a9sde,app=pg_dump,host=a9dualling.atkinsgeospatial.com LOCATION:
log_disconnections,postgres.c:4577
 
2022-04-02 02:14:55 BST: [5016] LOG:  00000: server process (PID 100) was terminated by exception 0xC0000022

At Atkins - member of the SNC-Lavalin Group, we work flexible hours around the world. Although I have sent this email
ata time convenient for me, I don't expect you to respond until it works for you.
 
NOTICE – This email message and any attachments may contain information or material that is confidential, privileged,
and/orsubject to copyright or other rights. Any unauthorized viewing, disclosure, retransmission, dissemination, or
otheruse of or reliance on this message or anything contained therein is strictly prohibited and may be unlawful. If
youbelieve you may have received this message in error, kindly inform the sender by return email and delete this
messagefrom your system. Thank you.
 

Re: PostgreSQL 10.20 crashes / Antivirus

From
Adrian Klaver
Date:
On 4/19/22 04:59, Thomas, Richard wrote:
> A request for further suggestions for how to fix/diagnose this PG crash (having just got back from holiday I discover
thatPG is still crashing even though I have changed the pgdump target output location to a folder excluded from the
McAfeeAntivirus scanning - but thanks for the suggestion Adrian). The last few crashes have occurred only during the
backupprocess, when database pg_dump completes (successfully - including the disconnection message in the log) but
beforethe next database pgdump can begin (that pg_dump command gets the error that it cannot connect to the database as
theserver is no longer responding). This will happen after dumping several databases - sometimes it manages the
completemultiple database dump fine. One new thing I've noticed is that when it fails the file access 0xC0000022 error
occursa second time when PG is trying to restart itself after shutting down all the other processes, suggesting
whatevercaused the crash is still around (a later manual restart works fine):
 

 From you original post:

"... although the new pg_dump log file was created (is empty) it does 
not get as far as creating the .backup file ..."

What are the actual commands you are using to do the above?



> 
> Richard


-- 
Adrian Klaver
adrian.klaver@aklaver.com

RE: PostgreSQL 10.20 crashes / Antivirus

From
"Thomas, Richard"
Date:
Adrian Klaver wrote:
> What are the actual commands you are using to do the above?

The command used in a PowerShell script (run with Windows task scheduler) to dump each database should evaluate to:

"C:\Program Files\PostgreSQL\10\bin\pg_dump.exe" -b -v -F c -d $dbName -h localhost -p 6488 -U backup_su -f $backupFile
2`>`&1| Out-File $pgdumpLogFile
 

where:
- "backup_su" is a superuser role (with password stored in the user's pgpass.conf file)
- $backupFile and $pgdumpLogFile are in folders excluded from McAfee scanning
- pg_dump.exe executable is not excluded from McAfee on-access scanning (although as recommended postgres.exe is)

The actual script segment building the command is:

        $pgdumpCmd = "C:\Program Files\PostgreSQL\10\bin\pg_dump.exe"
        $pgdumpArgs = @("-b", "-v",
            "-F", "c",
            "-d", $dbName,
            "-h", "localhost",
            "-p", "6488",
            "-U", " backup_su",
            "-f", $backupFile)
        cmd /c $pgdumpCmd $pgdumpArgs 2`>`&1 | Out-File $pgdumpLogFile

Note that the .backup file for the first failing pg_dump (after several successful ones) is now being produced but is
oflength 0, with the associated pg_dump log file simply reading:
 

pg_dump: [archiver (db)] connection to database "dbexample" failed: could not connect to server: Connection refused
(0x0000274D/10061)
        Is the server running on host "localhost" (127.0.0.1) and accepting
        TCP/IP connections on port 6488?
could not connect to server: Connection refused (0x0000274D/10061)
        Is the server running on host "localhost" (::1) and accepting
        TCP/IP connections on port 6488?

Richard

-----Original Message-----
From: Adrian Klaver <adrian.klaver@aklaver.com>
Sent: 19 April 2022 18:11
To: Thomas, Richard <Richard.Thomas@atkinsglobal.com>; pgsql-general@lists.postgresql.org
Subject: Re: PostgreSQL 10.20 crashes / Antivirus

On 4/19/22 04:59, Thomas, Richard wrote:
> A request for further suggestions for how to fix/diagnose this PG crash (having just got back from holiday I discover
thatPG is still crashing even though I have changed the pgdump target output location to a folder excluded from the
McAfeeAntivirus scanning - but thanks for the suggestion Adrian). The last few crashes have occurred only during the
backupprocess, when database pg_dump completes (successfully - including the disconnection message in the log) but
beforethe next database pgdump can begin (that pg_dump command gets the error that it cannot connect to the database as
theserver is no longer responding). This will happen after dumping several databases - sometimes it manages the
completemultiple database dump fine. One new thing I've noticed is that when it fails the file access 0xC0000022 error
occursa second time when PG is trying to restart itself after shutting down all the other processes, suggesting
whatevercaused the crash is still around (a later manual restart works fine):
 

 From you original post:

"... although the new pg_dump log file was created (is empty) it does not get as far as creating the .backup file ..."

What are the actual commands you are using to do the above?
At Atkins - member of the SNC-Lavalin Group, we work flexible hours around the world. Although I have sent this email
ata time convenient for me, I don't expect you to respond until it works for you.
 
NOTICE – This email message and any attachments may contain information or material that is confidential, privileged,
and/orsubject to copyright or other rights. Any unauthorized viewing, disclosure, retransmission, dissemination, or
otheruse of or reliance on this message or anything contained therein is strictly prohibited and may be unlawful. If
youbelieve you may have received this message in error, kindly inform the sender by return email and delete this
messagefrom your system. Thank you.
 

Re: PostgreSQL 10.20 crashes / Antivirus

From
Adrian Klaver
Date:
On 4/20/22 01:06, Thomas, Richard wrote:
> Adrian Klaver wrote:
>> What are the actual commands you are using to do the above?
> 
> The command used in a PowerShell script (run with Windows task scheduler) to dump each database should evaluate to:
> 
> "C:\Program Files\PostgreSQL\10\bin\pg_dump.exe" -b -v -F c -d $dbName -h localhost -p 6488 -U backup_su -f
$backupFile2`>`&1 | Out-File $pgdumpLogFile
 

FYI, -b is the default unless you are restricting the dump to a schema 
or a table .

Do you have large objects(blobs) in the database?

> 
> where:
> - "backup_su" is a superuser role (with password stored in the user's pgpass.conf file)
> - $backupFile and $pgdumpLogFile are in folders excluded from McAfee scanning
> - pg_dump.exe executable is not excluded from McAfee on-access scanning (although as recommended postgres.exe is)

Why not?

I would think the whole C:\Program Files\PostgreSQL\10\bin\ would be 
excluded.

> 
> The actual script segment building the command is:
> 
>          $pgdumpCmd = "C:\Program Files\PostgreSQL\10\bin\pg_dump.exe"
>          $pgdumpArgs = @("-b", "-v",
>              "-F", "c",
>              "-d", $dbName,
>              "-h", "localhost",
>              "-p", "6488",
>              "-U", " backup_su",
>              "-f", $backupFile)
>          cmd /c $pgdumpCmd $pgdumpArgs 2`>`&1 | Out-File $pgdumpLogFile
> 
> Note that the .backup file for the first failing pg_dump (after several successful ones) is now being produced but is
oflength 0, with the associated pg_dump log file simply reading:
 

What does the Windows event log show?

Same for the A/V software log.

> 
> pg_dump: [archiver (db)] connection to database "dbexample" failed: could not connect to server: Connection refused
(0x0000274D/10061)
>          Is the server running on host "localhost" (127.0.0.1) and accepting
>          TCP/IP connections on port 6488?
> could not connect to server: Connection refused (0x0000274D/10061)
>          Is the server running on host "localhost" (::1) and accepting
>          TCP/IP connections on port 6488?
> 
> Richard
> 



-- 
Adrian Klaver
adrian.klaver@aklaver.com



RE: PostgreSQL 10.20 crashes / Antivirus

From
"Thomas, Richard"
Date:
Adrian Klaver wrote:
> On 4/20/22 01:06, Thomas, Richard wrote:
> > The command used in a PowerShell script (run with Windows task scheduler)
> to dump each database should evaluate to:
> >
> > "C:\Program Files\PostgreSQL\10\bin\pg_dump.exe" -b -v -F c -d $dbName
> > -h localhost -p 6488 -U backup_su -f $backupFile 2`>`&1 | Out-File
> > $pgdumpLogFile
>
> Do you have large objects(blobs) in the database?

No, but do have PostGIS geometries in almost all tables.

> > - pg_dump.exe executable is not excluded from McAfee on-access
> > scanning (although as recommended postgres.exe is)
>
> Why not?
>
> I would think the whole C:\Program Files\PostgreSQL\10\bin\ would be
> excluded.

I was following the instructions here:
https://wiki.postgresql.org/wiki/Running_%26_Installing_PostgreSQL_On_Native_Windows#Antivirus_software
I am not particularly familiar with how our McAfee is configured or operates (all under control of our IT, including it
seemsaccess to many of the logs). With the executable postgres.exe, they have specified that in McAfee as an executable
notto include in "on-access scanning" (wherever the executable lives). This differs from the file read/write scanning
wherethe folders to exclude are specified. I have put in a request earlier today to add exclusions for pg_dump.exe;
maybeI'll hear back from IT in a day or so ;-(
 

> What does the Windows event log show?

Only events:
- Info: "The Windows Error Reporting Service service entered the running state" (goes into a stopped state 2 minutes
later)
- (Error event messages generated by my PowerShell script)

Not sure if WER is actually writing data somewhere (none of the "*\AppData\Local\Microsoft\Windows\WER\ReportArchive"
folderson the server have any data from the last 2 years)
 

> Same for the A/V software log.

I can't find much separate McAfee log information beyond the last hour in "Endpoint Security" app. Previously (before I
gotthe PostgreSQL backup PowerShell script removed from scanning, McAfee used to put messages in the Windows Event
Viewerreporting an error that it was a violating one of its rules, but it would allow the operation to continue (no
longerget that).
 

Richard
At Atkins - member of the SNC-Lavalin Group, we work flexible hours around the world. Although I have sent this email
ata time convenient for me, I don't expect you to respond until it works for you.
 
NOTICE – This email message and any attachments may contain information or material that is confidential, privileged,
and/orsubject to copyright or other rights. Any unauthorized viewing, disclosure, retransmission, dissemination, or
otheruse of or reliance on this message or anything contained therein is strictly prohibited and may be unlawful. If
youbelieve you may have received this message in error, kindly inform the sender by return email and delete this
messagefrom your system. Thank you.
 

Re: PostgreSQL 10.20 crashes / Antivirus

From
Adrian Klaver
Date:
On 4/20/22 10:23 AM, Thomas, Richard wrote:
> Adrian Klaver wrote:
>> On 4/20/22 01:06, Thomas, Richard wrote:

>>> - pg_dump.exe executable is not excluded from McAfee on-access
>>> scanning (although as recommended postgres.exe is)
>>
>> Why not?
>>
>> I would think the whole C:\Program Files\PostgreSQL\10\bin\ would be
>> excluded.
> 
> I was following the instructions here:
> 

> I am not particularly familiar with how our McAfee is configured or operates (all under control of our IT, including
itseems access to many of the logs). With the executable postgres.exe, they have specified that in McAfee as an
executablenot to include in "on-access scanning" (wherever the executable lives). This differs from the file read/write
scanningwhere the folders to exclude are specified. I have put in a request earlier today to add exclusions for
pg_dump.exe;maybe I'll hear back from IT in a day or so ;-(
 

That link also has:

"Specific issues have also been reported with McAfee and Panda 
anti-virus software and NetLimiter network monitoring software. While 
some people do have PostgreSQL working with these software packages, 
there is no specific or even recommend solutions that have not worked in 
some cases, so the issues would appear to be installation specific, 
sometimes even requiring uninstallation. "

Now the last edit to the page was 2013 so not sure how relevant the 
above is anymore. Also, I don't use Windows anymore(other then 
occasionally working on peoples machines) so I am not the best person to 
comment on current reality. Still the Postgres process seems to be 
killed by an external program and my suspicion is that the AV software 
is involved. Looks like this is something the Sys admins are going to 
have to look into via the tools and files they have access to.



> Richard



-- 
Adrian Klaver
adrian.klaver@aklaver.com



Re: PostgreSQL 10.20 crashes / Antivirus

From
Adrian Klaver
Date:
On 4/20/22 10:23 AM, Thomas, Richard wrote:
> Adrian Klaver wrote:
>> On 4/20/22 01:06, Thomas, Richard wrote:
>>> The command used in a PowerShell script (run with Windows task scheduler)
>> to dump each database should evaluate to:
>>>
>>> "C:\Program Files\PostgreSQL\10\bin\pg_dump.exe" -b -v -F c -d $dbName
>>> -h localhost -p 6488 -U backup_su -f $backupFile 2`>`&1 | Out-File
>>> $pgdumpLogFile
>>

Should have asked earlier:

Have you tried doing the pg_dumps as independent actions manually for 
each database instead of iterating over a list of databases in a script?


-- 
Adrian Klaver
adrian.klaver@aklaver.com



RE: PostgreSQL 10.20 crashes / Antivirus

From
"Thomas, Richard"
Date:
Adrian (all),

Thanks for your suggestions. Running the equivalent pg_dump commands at the command line instead of in a script works
fineand generates no errors/warnings/McAfee messages in the Windows Event Viewer or on stdout/stderr. (Most days the
backupscript itself completes running multiple pg_dump without problem). I too suspect McAfee of causing the problem -
evenif it isn't consciously taking any action, it must be processing all file I/O accesses in order to be able to
filterwhich ones it should do virus scanning on. Unfortunately it is a very painful process for me to get our IT to
modifyvirus scanning or report back on AV messages (even if not errors), so any diagnostics I can perform help speed
theprocess. (And unfortunately, my hands are currently tied on having to use an existing AWS EC2 Windows Server, much
asI would prefer to use a Linux system or a cloud-based PaaS.)
 

Richard

> > Adrian Klaver wrote:
> >> On 4/20/22 01:06, Thomas, Richard wrote:
>
> >>> - pg_dump.exe executable is not excluded from McAfee on-access
> >>> scanning (although as recommended postgres.exe is)
> >>
> >> Why not?
> >>
> >> I would think the whole C:\Program Files\PostgreSQL\10\bin\ would be
> >> excluded.
> >
> > I was following the instructions here:
> >
>
> > I am not particularly familiar with how our McAfee is configured or
> > operates (all under control of our IT, including it seems access to
> > many of the logs). With the executable postgres.exe, they have
> > specified that in McAfee as an executable not to include in "on-access
> > scanning" (wherever the executable lives). This differs from the file
> > read/write scanning where the folders to exclude are specified. I have
> > put in a request earlier today to add exclusions for pg_dump.exe;
> > maybe I'll hear back from IT in a day or so ;-(
>
> That link also has:
>
> "Specific issues have also been reported with McAfee and Panda anti-virus
> software and NetLimiter network monitoring software. While some people do
> have PostgreSQL working with these software packages, there is no specific or
> even recommend solutions that have not worked in some cases, so the issues
> would appear to be installation specific, sometimes even requiring
> uninstallation. "
>
> Now the last edit to the page was 2013 so not sure how relevant the above is
> anymore. Also, I don't use Windows anymore(other then occasionally working
> on peoples machines) so I am not the best person to comment on current
> reality. Still the Postgres process seems to be killed by an external program and
> my suspicion is that the AV software is involved. Looks like this is something the
> Sys admins are going to have to look into via the tools and files they have
> access to.
At Atkins - member of the SNC-Lavalin Group, we work flexible hours around the world. Although I have sent this email
ata time convenient for me, I don't expect you to respond until it works for you.
 
NOTICE – This email message and any attachments may contain information or material that is confidential, privileged,
and/orsubject to copyright or other rights. Any unauthorized viewing, disclosure, retransmission, dissemination, or
otheruse of or reliance on this message or anything contained therein is strictly prohibited and may be unlawful. If
youbelieve you may have received this message in error, kindly inform the sender by return email and delete this
messagefrom your system. Thank you.
 

Re: PostgreSQL 10.20 crashes / Antivirus

From
Adrian Klaver
Date:
On 4/21/22 02:13, Thomas, Richard wrote:
> Adrian (all),
> 
> Thanks for your suggestions. Running the equivalent pg_dump commands at the command line instead of in a script works
fineand generates no errors/warnings/McAfee messages in the Windows Event Viewer or on stdout/stderr. (Most days the
backupscript itself completes running multiple pg_dump without problem). 
 

For the time being a possible solution would be to insert a time out 
between iterations of:

$pgdumpCmd = "C:\Program Files\PostgreSQL\10\bin\pg_dump.exe"
         $pgdumpArgs = @("-b", "-v",
             "-F", "c",
             "-d", $dbName,
             "-h", "localhost",
             "-p", "6488",
             "-U", " backup_su",
             "-f", $backupFile)
         cmd /c $pgdumpCmd $pgdumpArgs 2`>`&1 | Out-File $pgdumpLogFile

> 
> Richard
> 



-- 
Adrian Klaver
adrian.klaver@aklaver.com