Thread: No. Of wal files generated
Hi,
Please could you help me by sharing any redhat linux command through which I can count the no. of wal files and no. of ".ready" files generated in last 10 minutes.
Regards,
Atul
On Fri, Mar 7, 2025 at 10:59 AM Atul Kumar <akumar14871@gmail.com> wrote:
Hi,Please could you help me by sharing any redhat linux command through which I can count the no. of wal files and no. of ".ready" files generated in last 10 minutes.
What problem are you trying to solve?
Death to <Redacted>, and butter sauce.
Don't boil me, I'm still alive.
<Redacted> lobster!
I want to check and compare the speed wal file generation process and archival process.
So I want to check how many wal file got created in x minutes and how many .ready files got created in those x minutes.
Regards,
Atul
On Fri, 7 Mar 2025, 22:45 Ron Johnson, <ronljohnsonjr@gmail.com> wrote:
On Fri, Mar 7, 2025 at 10:59 AM Atul Kumar <akumar14871@gmail.com> wrote:Hi,Please could you help me by sharing any redhat linux command through which I can count the no. of wal files and no. of ".ready" files generated in last 10 minutes.What problem are you trying to solve?--Death to <Redacted>, and butter sauce.Don't boil me, I'm still alive.<Redacted> lobster!
inotifywait can log every file creation and deletion in a directory.
Honestly, though, I'd strongly think about using PgBackRest instead of something you wrote yourself.
On Fri, Mar 7, 2025 at 2:47 PM Atul Kumar <akumar14871@gmail.com> wrote:
I want to check and compare the speed wal file generation process and archival process.So I want to check how many wal file got created in x minutes and how many .ready files got created in those x minutes.Regards,AtulOn Fri, 7 Mar 2025, 22:45 Ron Johnson, <ronljohnsonjr@gmail.com> wrote:On Fri, Mar 7, 2025 at 10:59 AM Atul Kumar <akumar14871@gmail.com> wrote:Hi,Please could you help me by sharing any redhat linux command through which I can count the no. of wal files and no. of ".ready" files generated in last 10 minutes.What problem are you trying to solve?--Death to <Redacted>, and butter sauce.Don't boil me, I'm still alive.<Redacted> lobster!
Death to <Redacted>, and butter sauce.
Don't boil me, I'm still alive.
<Redacted> lobster!
Please can you elaborate more about the meaning of inotifywait.
And how pgbackrest will solve the issue ?
Regards.
On Sat, 8 Mar 2025, 01:37 Ron Johnson, <ronljohnsonjr@gmail.com> wrote:
inotifywait can log every file creation and deletion in a directory.Honestly, though, I'd strongly think about using PgBackRest instead of something you wrote yourself.On Fri, Mar 7, 2025 at 2:47 PM Atul Kumar <akumar14871@gmail.com> wrote:I want to check and compare the speed wal file generation process and archival process.So I want to check how many wal file got created in x minutes and how many .ready files got created in those x minutes.Regards,AtulOn Fri, 7 Mar 2025, 22:45 Ron Johnson, <ronljohnsonjr@gmail.com> wrote:On Fri, Mar 7, 2025 at 10:59 AM Atul Kumar <akumar14871@gmail.com> wrote:Hi,Please could you help me by sharing any redhat linux command through which I can count the no. of wal files and no. of ".ready" files generated in last 10 minutes.What problem are you trying to solve?--Death to <Redacted>, and butter sauce.Don't boil me, I'm still alive.<Redacted> lobster!--Death to <Redacted>, and butter sauce.Don't boil me, I'm still alive.<Redacted> lobster!
On Fri, Mar 7, 2025 at 7:53 PM Atul Kumar <akumar14871@gmail.com> wrote:
Please can you elaborate more about the meaning of inotifywait.
Google is your friend.
And how pgbackrest will solve the issue ?
PgBackRest does all the WAL archiving for you, keeps track of them, makes sure they are not overwritten, purges old ones when not needed etc. Even compresses and encrypts them if you want.
Regards.On Sat, 8 Mar 2025, 01:37 Ron Johnson, <ronljohnsonjr@gmail.com> wrote:inotifywait can log every file creation and deletion in a directory.Honestly, though, I'd strongly think about using PgBackRest instead of something you wrote yourself.On Fri, Mar 7, 2025 at 2:47 PM Atul Kumar <akumar14871@gmail.com> wrote:I want to check and compare the speed wal file generation process and archival process.So I want to check how many wal file got created in x minutes and how many .ready files got created in those x minutes.Regards,AtulOn Fri, 7 Mar 2025, 22:45 Ron Johnson, <ronljohnsonjr@gmail.com> wrote:On Fri, Mar 7, 2025 at 10:59 AM Atul Kumar <akumar14871@gmail.com> wrote:Hi,Please could you help me by sharing any redhat linux command through which I can count the no. of wal files and no. of ".ready" files generated in last 10 minutes.What problem are you trying to solve?--Death to <Redacted>, and butter sauce.Don't boil me, I'm still alive.<Redacted> lobster!--Death to <Redacted>, and butter sauce.Don't boil me, I'm still alive.<Redacted> lobster!
Death to <Redacted>, and butter sauce.
Don't boil me, I'm still alive.
<Redacted> lobster!
Take a look at the pg_stat_archiver view, if you have not already:
So I want to check how many wal file got created in x minutes and how many .ready files got created in those x minutes.
It's not clear why you would want to track .ready files, they should be quite transient, and not a concern unless you are rolling your own archiving system (which you should not be). But Ron's suggestion of inotifywait is a good one for this. Probably listed as inotify or inotify-tools in your packaging system. Basic usage would be something like this, from the data directory:
timeout 10m inotifywait pg_wal/ -r -m -e close_write -o walinfo.log
Cheers,
Greg
--
Crunchy Data - https://www.crunchydata.com
Enterprise Postgres Software Products & Tech Support
On 3/7/25 17:59, Atul Kumar wrote:
Hi,Please could you help me by sharing any redhat linux command through which I can count the no. of wal files and no. of ".ready" files generated in last 10 minutes.
Do you have wal archiving enabled? If yes, then simply go to the archive dir and suit your self with
find , ls, grep, awk, sort , unic -c , etc .
Find files created in the last 10 mins :
find -cmin 10
to get statistics over time :
***CLOUD*** jboss@sma:/smadb/smadb/pgsql/pitr$ ls -lt | head -10
total 629962796
-rw------- 1 postgres postgres 16777216 Mar 10 09:32 000000010000472D00000078
-rw------- 1 postgres postgres 16777216 Mar 10 09:32 000000010000472D00000077
-rw------- 1 postgres postgres 16777216 Mar 10 09:32 000000010000472D00000076
-rw------- 1 postgres postgres 16777216 Mar 10 09:32 000000010000472D00000075
...
you may use :
ls -lt | awk '{print $6""$7"_"substr($8,1,2) }' | sort | uniq -c | less
to get the distribution over (group by) 1 hour intervals
Regards,Atul