Re: PG vs ElasticSearch for Logs - Mailing list pgsql-general

From Andreas Kretschmer
Subject Re: PG vs ElasticSearch for Logs
Date
Msg-id 20160819104402.GA23325@tux
Whole thread Raw
In response to Re: PG vs ElasticSearch for Logs  (Thomas Güttler <guettliml@thomas-guettler.de>)
Responses Re: PG vs ElasticSearch for Logs  (Francisco Olarte <folarte@peoplecall.com>)
Re: PG vs ElasticSearch for Logs  (Thomas Güttler <guettliml@thomas-guettler.de>)
Re: PG vs ElasticSearch for Logs  (John R Pierce <pierce@hogranch.com>)
List pgsql-general
Thomas Güttler <guettliml@thomas-guettler.de> wrote:

>> How will you be using the logs? What kind of queries? What kind of searches?
>> Correlating events and logs from various sources could be really easy with joins, count and summary operations.
>
> Wishes raise with possibilities. First I want to do simple queries about
> hosts and timestamps. Then some simple substring matches.

for append-only tables like this consider 9.5 and BRIN-Indexes for
timestamp-searches. But if you deletes after N weeks BRIN shouldn't work
properly because of vacuum and re-use of space within the table.
Do you know BRIN?

So, in your case, consider partitioning, maybe per month. So you can
also avoid mess with table and index bloat.


Greetings from Dresden to Chemnitz (is this still valid?)



Regards, Andreas Kretschmer
--
Andreas Kretschmer
http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services


pgsql-general by date:

Previous
From: Rafal Pietrak
Date:
Subject: Re: PG vs ElasticSearch for Logs
Next
From: Francisco Olarte
Date:
Subject: Re: PG vs ElasticSearch for Logs