Postgres Pro
Downloads
Home   >   mailing lists

Thread: [GENERAL] londiste3: removing a node that does not exist

[GENERAL] londiste3: removing a node that does not exist

From
Willy-Bas Loos
Date:
Hi,

I'm using londiste3 for replication in postgres 9.3
I made a typo when registering a consumer and I changed the name of that consumer during the creation process.

I now have a non-functional node in the producer database, that is mongering historical data.
I have tried to use londiste <config> drop-node <node>, but i get this error:
ERROR get_node_database: cannot resolve <node>

That is correct, but i still want to remove it.

I also tried tag-dead, which gives me no error but also has no noticeable effect.

The node is mentioned in the output of the db function pgq.get_consumer_info(), but not in the output of londiste <config-file> status nor londiste <config-file> members.

There seems to be surplus data in these tables, in descending volume: pgq.event_1_2, pgq.event_1_2_txid_idx, pgq.tick, pgq.event_1_1, pgq.tick_pkey, pgq.event_1_0

Does anyone know of a method to get rid of the bad node and the data that it is gathering?

Also, it seems like londiste is slowly dying. At least the documentation is still very scarce and the mailing list is pretty much dead. I know that microsoft bought skype, and it seems like londiste has been receiving less love since then. Is everyone abandoning londiste for a new thing?

Cheers,
--
Willy-Bas Loos

Re: [GENERAL] londiste3: removing a node that does not exist

From
Victor Yegorov
Date:
2017-06-22 18:21 GMT+03:00 Willy-Bas Loos <willybas@gmail.com>:
Does anyone know of a method to get rid of the bad node and the data that it is gathering?

I also do not know the correct way to achieve this.


But once I needed to rename one of the queues to follow the internal naming standard.
And I used the following queries on the provider node to prune _old_ queue:

SET search_path TO pgq_node;
DELETE FROM local_state WHERE queue_name='q-2rm';
DELETE FROM subscriber_info WHERE queue_name='q-2rm';
DELETE FROM node_info WHERE queue_name='q-2rm';
DELETE FROM node_location WHERE queue_name='q-2rm';

And restarted worker after that.


--
Victor Yegorov

Re: [GENERAL] londiste3: removing a node that does not exist

From
Willy-Bas Loos
Date:
On Thu, Jun 22, 2017 at 6:03 PM, Victor Yegorov <vyegorov@gmail.com> wrote:
2017-06-22 18:21 GMT+03:00 Willy-Bas Loos <willybas@gmail.com>:
Does anyone know of a method to get rid of the bad node and the data that it is gathering?

I also do not know the correct way to achieve this.


But once I needed to rename one of the queues to follow the internal naming standard.
And I used the following queries on the provider node to prune _old_ queue:

SET search_path TO pgq_node;
DELETE FROM local_state WHERE queue_name='q-2rm';
DELETE FROM subscriber_info WHERE queue_name='q-2rm';
DELETE FROM node_info WHERE queue_name='q-2rm';
DELETE FROM node_location WHERE queue_name='q-2rm';

And restarted worker after that.


--
Victor Yegorov

Hi thanks for the answer. I've left this alone for a while and i've been looking  into it in more detail today.
All the data is in the schema pgq. The other schema's are super small.
Also, the non-functional node is not in any of the tables in pgq_node.

But it is mentioned in pgq.consumer and linked with pgq.subscription.
When i remove the subscription records, nothing happens (worker and pgqd are running)
I was hoping hat some maintenance routine would remove the surplus data, but it doesn't.

I would delete the data myself but i can't figure out how to link that data to a consumer.
The event tables that contain all the data have no foreign keys. And i've tried to find mutual id's in those and other tables like pgq.tick but i haven't found anything of substance.
There are transaction id's in the event tables, i just hope that that is not the link because that would probably get seriously complicated to handle.

Can anyone give me some advice?

--
Willy-Bas Loos