Thread: File system operations.
Hi, hackers!
Do we have any standard set of functions to operate with files?
I mean link(), copy(), rename(), unlink(), ftruncate() and so on.
Mostly I worry about the first two since they have different
implementation on Windows.
I found a couple of functions in src/backend/storage/file/copydir.c,
some more in src/bin/pg_upgrade/file.c along with a set of defines
in src/bin/pg_upgrade/pg_upgrade.h. Obviously md.c and fd.c contain
some functions.
Do we have any policy of using them? There is a comment in fd.c
* For this scheme to work, most (if not all) routines throughout the * server should use these interfaces instead of
callingthe C library * routines (e.g., open(2) and fopen(3)) themselves. Otherwise, we * may find ourselves short of
realfile descriptors anyway.
and even more in fd.h
* File {Close, Read, Write, Seek, Tell, Sync} * {Path Name Open, Allocate, Free} File * * These are NOT JUST
RENAMINGSOF THE UNIX ROUTINES. * Use them for all file activity...
but I see that, for example, pg_open_tzfile() and get_controlfile(),
logfile_open()
call open()/fopen() directly. Same behavior you can find for any C
library function.
Am I missing something important or it is simply a legasy/overlooked code?
What do you think about moving stuff from pg_upgrade/file.c to storage/file/
to allow reuse of this code? I think it'll be really helpful for
developers of contrib modules
like backup managers.
--
Anastasia Lubennikova
Postgres Professional:http://www.postgrespro.com
The Russian Postgres Company
On Thu, Sep 15, 2016 at 11:01 AM, Anastasia Lubennikova <a.lubennikova@postgrespro.ru> wrote: > What do you think about moving stuff from pg_upgrade/file.c to storage/file/ > to allow reuse of this code? I think it'll be really helpful for developers > of contrib modules > like backup managers. Well, storage/file is backend and pg_upgrade is frontend. If you want to reuse the same code for both it's got to go someplace else. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company
Robert Haas <robertmhaas@gmail.com> writes:
> On Thu, Sep 15, 2016 at 11:01 AM, Anastasia Lubennikova
> <a.lubennikova@postgrespro.ru> wrote:
>> What do you think about moving stuff from pg_upgrade/file.c to storage/file/
>> to allow reuse of this code? I think it'll be really helpful for developers
>> of contrib modules
>> like backup managers.
> Well, storage/file is backend and pg_upgrade is frontend. If you want
> to reuse the same code for both it's got to go someplace else.
Also, to the extent that it's important to use those wrappers rather
than libc directly, it's because the wrappers are tied into backend
resource management and error handling conventions. I don't see how
you convert that into code that also works in a frontend program,
or what the point would be exactly.
regards, tom lane
On четверг, 15 сентября 2016 г. 19:09:16 MSK, Tom Lane wrote: > Robert Haas <robertmhaas@gmail.com> writes: >> On Thu, Sep 15, 2016 at 11:01 AM, Anastasia Lubennikova >> <a.lubennikova@postgrespro.ru> wrote: >>> What do you think about moving stuff from pg_upgrade/file.c >>> to storage/file/ >>> to allow reuse of this code? I think it'll be really helpful >>> for developers >>> of contrib modules >>> like backup managers. > >> Well, storage/file is backend and pg_upgrade is frontend. If you want >> to reuse the same code for both it's got to go someplace else. > > Also, to the extent that it's important to use those wrappers rather > than libc directly, it's because the wrappers are tied into backend > resource management and error handling conventions. I don't see how > you convert that into code that also works in a frontend program, > or what the point would be exactly. Maybe we should towards to framework ecosystem. I mean, developers of third party tools must use maximum same api what in backend. -- Yury Zhuravlev Postgres Professional: http://www.postgrespro.com The Russian Postgres Company
On 22 September 2016 at 20:02, Yury Zhuravlev <u.zhuravlev@postgrespro.ru> wrote: > On четверг, 15 сентября 2016 г. 19:09:16 MSK, Tom Lane wrote: >> >> Robert Haas <robertmhaas@gmail.com> writes: >>> >>> On Thu, Sep 15, 2016 at 11:01 AM, Anastasia Lubennikova >>> <a.lubennikova@postgrespro.ru> wrote: >>>> >>>> What do you think about moving stuff from pg_upgrade/file.c to >>>> storage/file/ >>>> to allow reuse of this code? I think it'll be really helpful for >>>> developers >>>> of contrib modules >>>> like backup managers. >> >> >>> Well, storage/file is backend and pg_upgrade is frontend. If you want >>> to reuse the same code for both it's got to go someplace else. >> >> >> Also, to the extent that it's important to use those wrappers rather >> than libc directly, it's because the wrappers are tied into backend >> resource management and error handling conventions. I don't see how >> you convert that into code that also works in a frontend program, >> or what the point would be exactly. > > > Maybe we should towards to framework ecosystem. I mean, developers of third > party tools must use maximum same api what in backend. Well, there's a very gradual shift in that direction with libpgcommon etc. But it's minimalist. Most of what's done in the backend makes no sense in frontend code. It'd make much more sense to adopt an existing portable runtime (nspr, apr, whatever) than write our own if we wanted to go full framework. But I don't think we're likely to. Much more likely to cause pain than prevent it, esp since we're multiprocessing and shmem based. There are a few areas we could use abstractions for though, like fork()/exec() vs CreateProcessEx(...). -- Craig Ringer http://www.2ndQuadrant.com/PostgreSQL Development, 24x7 Support, Training & Services