Re: Counting the occurences of a substring within a very large text - Mailing list pgsql-general

From Marc Mamin
Subject Re: Counting the occurences of a substring within a very large text
Date
Msg-id B6F6FD62F2624C4C9916AC0175D56D8828BEA742@jenmbs01.ad.intershop.net
Whole thread Raw
In response to Re: Counting the occurences of a substring within a very large text  (Marc Mamin <M.Mamin@intershop.de>)
Responses Re: Counting the occurences of a substring within a very large text
List pgsql-general
> > > I'd like to count the number  linebreaks within a string, but I get
> > > a memory allocation error when using regexp_matches or
> > regexp_split_to_table.
> > >
> > > Any idea for an alternative to this problem  ?
> > >
> > > select count(*)-1 from
> > > (  select regexp_split_to_table(full_message,'(\n)', 'g')
> > >    from mytable
> > >    where id =-2146999703
> > > )foo;
> > >
> > > ERROR:  invalid memory alloc request size 1447215584
> >
> > Does any of these two work:
> >
> > SELECT length(regexp_replace(full_message, '[^\n]', '', 'g')) FROM
> > mytable WHERE id = -2146999703;
> >
> > or
> >
> > SELECT length(full_message) - length(replace(full_message, E'\n',
> ''))
> > FROM mytable WHERE id = -2146999703;
> 
> 
> no, they both yeld the same error.
> 


and this fails too, which is more annoying as it looks like a bug:
SELECT replace(full_message, E'\n', '') FROM stadium_rprod.aserrorfull_20150623 WHERE id = -2146999703;

note that the 345MB text only contains 635 lines. This might be the issue...

Marc Mamin


> a new string functions for this would be nice, as it could certainly be
> implemented in a more efficient way...
> 
>     BTW: the text to check is a single 350 MB error message from a
> log file :)
> 
> 
> 
> 
> 
> >
> > Yours,
> > Laurenz Albe

pgsql-general by date:

Previous
From: "Gauthier, Dave"
Date:
Subject: DB access speeds, App(linux)<->PG(linux) vs App(linux) <->MSSql(Windows)
Next
From: John Bleichert
Date:
Subject: mirroring a server and/or hot standby