Thread: Support for Rust
Hello,
Are there any plans or thoughts about adding support for other languages than C into Postgres, namely Rust? I would love to hack on some features but I worry somewhat that the C compiler won't give me enough hints that I'm doing something wrong, and the Rust compiler has been excellent at preventing bugs.
Best,
Lev
Hi! > On 10 Sep 2022, at 07:38, Lev Kokotov <lev@hyperparam.ai> wrote: > > Are there any plans or thoughts about adding support for other languages than C into Postgres, namely Rust? I would loveto hack on some features but I worry somewhat that the C compiler won't give me enough hints that I'm doing somethingwrong, and the Rust compiler has been excellent at preventing bugs. You can write Postgres extensions in Rust. And Postgres extensions are really powerful. What kind of features are you interestedin? Undoubtedly, attracting Rust folks to contribute Postgres could be a good things. Yet some very simple questions arise. 1. Is Rust compatible with Memory Contexts and shared memory constructs of Postgres? With elog error reporting, PG_TRY()and his friends? 2. Does Rust support same set of platforms as Postgres? Quick glance at Build Farm can give an impression of what is supportedby Postgres[0]. 3. Do we gain anything besides compiler hints? Postgres development is hard due to interference of complex subsystems. Itwill be even harder if those systems will be implemented in different languages. Probably, answers to all these questions are obvious to Rust pros. I just think this can be of interest to someone new toRust (like me). Best regards, Andrey Borodin. [0] https://buildfarm.postgresql.org/cgi-bin/show_members.pl
On Fri, Sep 09, 2022 at 07:38:14PM -0700, Lev Kokotov wrote: > Are there any plans or thoughts about adding support for other languages > than C into Postgres, namely Rust? I would love to hack on some features > but I worry somewhat that the C compiler won't give me enough hints that > I'm doing something wrong, and the Rust compiler has been excellent at > preventing bugs. There was some discussion about Rust back in 2017 [0] that you might be interested in. [0] https://www.postgresql.org/message-id/flat/CAASwCXdQUiuUnhycdRvrUmHuzk5PsaGxr54U4t34teQjcjb%3DAQ%40mail.gmail.com -- Nathan Bossart Amazon Web Services: https://aws.amazon.com
> You can write Postgres extensions in Rust. And Postgres extensions are really powerful. What kind of features are you interested in?
Agreed, I've been writing one in Rust using tcdi/pgx [0]. Some features can't be done there though, e.g. adding ON CONFLICT support to COPY.
> 1. Is Rust compatible with Memory Contexts and shared memory constructs of Postgres? With elog error reporting, PG_TRY() and his friends?
Not to my knowledge. Just by reading the implementation, jumping to arbitrary positions in the code is against safe programming that Rust guarantees.
> 2. Does Rust support the same set of platforms as Postgres? Quick glance at Build Farm can give an impression of what is supported by Postgres.
Rust compiles to LLVM, so the platforms supported by Rust are roughly the same as LLVM. The list is large, but smaller than GCC. There is a Rust GCC frontend in the works and the number of Rust targets is also increasing including embedded [1].
> 3. Do we gain anything besides compiler hints? Postgres development is hard due to interference of complex subsystems. It will be even harder if those systems will be implemented in different languages.
Rust gives many things we wanted for decades:
1. No undefined behavior
2. No memory leaks, guaranteed at compile time
3. RAII - no surprise NULLs, no segfaults
4. "Zero-cost abstractions" - simple looking and otherwise expensive code (e.g. generics, collections) is optimized at compile time and has no runtime cost
5. Find bugs at compile time instead of runtime
Rust also has a large and ever growing community, and a great cross-platform package manager and build system (cargo).
> There was some discussion about Rust back in 2017 [0] that you might be
interested in.
interested in.
That was a good discussion. My takeaways are:
- Suggesting that someone else should do the work is a bad idea (I agree)
- This should be done by both C and Rust experts
- It's hard to find a good starting point and build momentum
- C++ is the favorite at the moment
Well... at the risk of embarrassing myself on the internet, I tried it this weekend: https://github.com/postgresml/postgres/pull/1
I took a small part of Postgres to get started, so just as a PoC; it compiles and runs though. Larger parts will take more work (deleting code, not just swapping object files), and more fancy things like PG_TRY() and friends will have to be rewritten, so not a short and easy migration.
On Sat, Sep 10, 2022 at 10:15 AM Nathan Bossart <nathandbossart@gmail.com> wrote:
On Fri, Sep 09, 2022 at 07:38:14PM -0700, Lev Kokotov wrote:
> Are there any plans or thoughts about adding support for other languages
> than C into Postgres, namely Rust? I would love to hack on some features
> but I worry somewhat that the C compiler won't give me enough hints that
> I'm doing something wrong, and the Rust compiler has been excellent at
> preventing bugs.
There was some discussion about Rust back in 2017 [0] that you might be
interested in.
[0] https://www.postgresql.org/message-id/flat/CAASwCXdQUiuUnhycdRvrUmHuzk5PsaGxr54U4t34teQjcjb%3DAQ%40mail.gmail.com
--
Nathan Bossart
Amazon Web Services: https://aws.amazon.com
Lev Kokotov <lev@hyperparam.ai> writes:
>> 3. Do we gain anything besides compiler hints? Postgres development is
>> hard due to interference of complex subsystems. It will be even harder if
>> those systems will be implemented in different languages.
> Rust gives many things we wanted for decades:
> 1. No undefined behavior
> 2. No memory leaks, guaranteed at compile time
Really? It seems impossible to me that a language that even thinks
it can guarantee that could interoperate with the backend's memory
management. And that's not something we are interested in replacing.
> I took a small part of Postgres to get started, so just as a PoC; it
> compiles and runs though. Larger parts will take more work (deleting code,
> not just swapping object files), and more fancy things like PG_TRY() and
> friends will have to be rewritten, so not a short and easy migration.
Yeah, that's what I thought. "Allow some parts to be written in
language X" soon turns into "Rewrite the entire system in language X,
including fundamental rethinking of memory management, error handling,
and some other things". That's pretty much a non-starter.
regards, tom lane
On Mon, Sep 12, 2022 at 11:29:12AM -0400, Tom Lane wrote: > Lev Kokotov <lev@hyperparam.ai> writes: > >> 3. Do we gain anything besides compiler hints? Postgres development is > >> hard due to interference of complex subsystems. It will be even harder if > >> those systems will be implemented in different languages. > > > Rust gives many things we wanted for decades: > > > 1. No undefined behavior > > 2. No memory leaks, guaranteed at compile time > > Really? It seems impossible to me that a language that even thinks > it can guarantee that could interoperate with the backend's memory > management. And that's not something we are interested in replacing. Indeed, it can only guarantee that if you rely on regular safe rust where lifetimes can be checked. To use the memory context infrastructure you would need to use unsafe rust, and when you do that you're back with the same problems. > > I took a small part of Postgres to get started, so just as a PoC; it > > compiles and runs though. Larger parts will take more work (deleting code, > > not just swapping object files), and more fancy things like PG_TRY() and > > friends will have to be rewritten, so not a short and easy migration. > > Yeah, that's what I thought. "Allow some parts to be written in > language X" soon turns into "Rewrite the entire system in language X, > including fundamental rethinking of memory management, error handling, > and some other things". That's pretty much a non-starter. Also, unless I'm missing something the modified code will only work for frontend programs, where palloc / pfree are really malloc / free calls. The rewritten BuildRestoreCommand won't return a palloc'd string on the backend, so the recovery TAP tests should crash when using it.
On Mon, Sep 12, 2022 at 10:29 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
>
> Lev Kokotov <lev@hyperparam.ai> writes:
> > I took a small part of Postgres to get started, so just as a PoC; it
> > compiles and runs though. Larger parts will take more work (deleting code,
> > not just swapping object files), and more fancy things like PG_TRY() and
> > friends will have to be rewritten, so not a short and easy migration.
>
> Yeah, that's what I thought. "Allow some parts to be written in
> language X" soon turns into "Rewrite the entire system in language X,
> including fundamental rethinking of memory management, error handling,
> and some other things". That's pretty much a non-starter.
Added "Rewrite the code in a different language" to "Features We Do Not Want" section of Wiki, referencing the two threads that came up:
https://wiki.postgresql.org/wiki/Todo#Features_We_Do_Not_Want
--
John Naylor
EDB: http://www.enterprisedb.com
On Mon, 2022-09-12 at 11:29 -0400, Tom Lane wrote:
> > Rust gives many things we wanted for decades:
>
> > 1. No undefined behavior
> > 2. No memory leaks, guaranteed at compile time
>
> Really? It seems impossible to me that a language that even thinks
> it can guarantee that could interoperate with the backend's memory
> management. And that's not something we are interested in replacing.
It's a distraction to talk about rust's safety "guarantees" in the
context of this thread. #1 is partially true, and #2 is outright
false[1].
C interoperability is the most compelling rust feature, in my opinion.
C memory representations are explicitly supported, and high-level
language features don't impose on your struct layouts. For instance,
rust does dynamic dispatch using trait objects[2], which hold the
vtable along with the reference, rather than in the struct itself. And
a "Foo *" from C has the same memory representation as an Option<&Foo>
in rust, so that you get the type safety.
Of course, rewriting Postgres would be terrible idea regardless of the
merits of rust for all kinds of reasons. But writing *extensions* in
rust is very promising because of this C interoperability.
>
> Yeah, that's what I thought. "Allow some parts to be written in
> language X" soon turns into "Rewrite the entire system in language X,
> including fundamental rethinking of memory management, error
> handling,
> and some other things". That's pretty much a non-starter.
You may be surprised how much you can do with rust extensions without
changing any of those things[3].
Regards,
Jeff Davis
[1] https://doc.rust-lang.org/std/mem/fn.forget.html
[2] https://doc.rust-lang.org/book/ch17-02-trait-objects.html
[3] https://www.pgcon.org/2019/schedule/attachments/532_RustTalk.pdf