Re: Clock-skew management in logical replication - Mailing list pgsql-hackers

From Joe Conway
Subject Re: Clock-skew management in logical replication
Date
Msg-id c6b3b8ef-8909-47ef-afb5-bf5012fdaf0d@joeconway.com
Whole thread Raw
In response to Re: Clock-skew management in logical replication  (shihao zhong <zhong950419@gmail.com>)
Responses Re: Clock-skew management in logical replication
Re: Clock-skew management in logical replication
List pgsql-hackers
On 9/21/24 01:31, shihao zhong wrote:
> Nisha Moond <nisha.moond412@gmail.com> writes:
>> Thoughts? Looking forward to hearing others' opinions!
> 
> Had a productive conversation with Amit Kaplia today about time skew
> in distributed systems, and wanted to share some thoughts.
> Essentially, we're grappling with the classic distributed snapshot
> problem. In a multi-active environment, where multiple nodes can
> independently process transactions,  it becomes crucial to determine
> the visibility of these transactions across the system.  Time skew,
> where different machines have different timestamps make it a hard
> problem. How can we ensure consistent transaction ordering and
> visibility when time itself is unreliable?
> 
> As you mentioned, there are several ways to tackle the time skew
> problem in distributed systems. These approaches generally fall into
> three main categories:
> 
> 1. Centralized Timestamps (Timestamp Oracle)
> 2. Atomic Clocks (True Time)
> 3. Hybrid Logical Clocks
> 4 Local Clocks

> I recommend ...<snip>... implement a pluggable time access method. This
> allows users to integrate with different time services as needed.

Huge +1

> In the mid-term, implementing the HLC approach would provide highly
> consistent snapshot reads. This offers a significant advantage for
> many use cases.

agreed

> Long-term, we should consider integrating with a distributed time
> service like AWS Time Sync Service. This ensures high accuracy and
> scalability for demanding applications.

I think the pluggable access method should make this possible, no?

-- 
Joe Conway
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com



pgsql-hackers by date:

Previous
From: Tomas Vondra
Date:
Subject: Re: scalability bottlenecks with (many) partitions (and more)
Next
From: Marcos Pegoraro
Date:
Subject: Re: Why mention to Oracle ?