Thread: Status of new relation file naming
Where are we in subj? Oid.version or UniqueId? If noone is going to implement it soon then I'll have to change code to OID file naming for WAL. Vadim
[ Charset KOI8-R unsupported, converting... ] > Where are we in subj? > Oid.version or UniqueId? > If noone is going to implement it soon then I'll have to > change code to OID file naming for WAL. Well, we can move to one of these, but we need tools so people can find the real table names that go with the files, and even then, it will never be as good as the system we currently have. My idea was to append a version number or oid on to the end of the file name, and use that somehow. -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 853-3000+ If your life is a hard drive, | 830 Blythe Avenue + Christ can be your backup. | Drexel Hill, Pennsylvania19026
On Tue, 12 Sep 2000, Bruce Momjian wrote: > [ Charset KOI8-R unsupported, converting... ] > > Where are we in subj? > > Oid.version or UniqueId? > > If noone is going to implement it soon then I'll have to > > change code to OID file naming for WAL. > > Well, we can move to one of these, but we need tools so people can find > the real table names that go with the files, and even then, it will > never be as good as the system we currently have. I thought it was generally agreed that this wasn't a requirement, and that is someone felt it was required in the future, like any open source project, they could ante up the time to build it ...
> On Tue, 12 Sep 2000, Bruce Momjian wrote: > > > [ Charset KOI8-R unsupported, converting... ] > > > Where are we in subj? > > > Oid.version or UniqueId? > > > If noone is going to implement it soon then I'll have to > > > change code to OID file naming for WAL. > > > > Well, we can move to one of these, but we need tools so people can find > > the real table names that go with the files, and even then, it will > > never be as good as the system we currently have. > > I thought it was generally agreed that this wasn't a requirement, and that > is someone felt it was required in the future, like any open source > project, they could ante up the time to build it ... Well, if we release 7.1 without those tools, we can expect lots of complaints. -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 853-3000+ If your life is a hard drive, | 830 Blythe Avenue + Christ can be your backup. | Drexel Hill, Pennsylvania19026
On Tue, 12 Sep 2000, Bruce Momjian wrote: > > On Tue, 12 Sep 2000, Bruce Momjian wrote: > > > > > [ Charset KOI8-R unsupported, converting... ] > > > > Where are we in subj? > > > > Oid.version or UniqueId? > > > > If noone is going to implement it soon then I'll have to > > > > change code to OID file naming for WAL. > > > > > > Well, we can move to one of these, but we need tools so people can find > > > the real table names that go with the files, and even then, it will > > > never be as good as the system we currently have. > > > > I thought it was generally agreed that this wasn't a requirement, and that > > is someone felt it was required in the future, like any open source > > project, they could ante up the time to build it ... > > Well, if we release 7.1 without those tools, we can expect lots of > complaints. stock answer "we look forward to seeing patches to correct this problem" :)
> > > I thought it was generally agreed that this wasn't a requirement, and that > > > is someone felt it was required in the future, like any open source > > > project, they could ante up the time to build it ... > > > > Well, if we release 7.1 without those tools, we can expect lots of > > complaints. > > stock answer "we look forward to seeing patches to correct this > problem" :) The problem is that I have no idea how to suggest writing such a tool that fits all needs. -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 853-3000+ If your life is a hard drive, | 830 Blythe Avenue + Christ can be your backup. | Drexel Hill, Pennsylvania19026
On Tue, 12 Sep 2000, Bruce Momjian wrote: > > > > I thought it was generally agreed that this wasn't a requirement, and that > > > > is someone felt it was required in the future, like any open source > > > > project, they could ante up the time to build it ... > > > > > > Well, if we release 7.1 without those tools, we can expect lots of > > > complaints. > > > > stock answer "we look forward to seeing patches to correct this > > problem" :) > > The problem is that I have no idea how to suggest writing such a tool > that fits all needs. IMHO, we have a choice ... we either move forward with a change that I *believe* everyone agrees has to happen, which will prompt someone to come up with a solution to (again, what I believe) is the only drawback ... or, we don't implement while waiting for someone to come up with a solution ... if we wait, again, IMHO, it will never happen, since there is no impetus for someone to "fix the problem" ... depending on what it takes to implement, if we implement it now, that leaves ~1.5mos for someone to come up with a solution before release ...
> > > stock answer "we look forward to seeing patches to correct this > > > problem" :) > > > > The problem is that I have no idea how to suggest writing such a tool > > that fits all needs. > > IMHO, we have a choice ... we either move forward with a change that I > *believe* everyone agrees has to happen, which will prompt someone to come > up with a solution to (again, what I believe) is the only drawback ... or, > we don't implement while waiting for someone to come up with a solution > ... > > if we wait, again, IMHO, it will never happen, since there is no impetus > for someone to "fix the problem" ... > > depending on what it takes to implement, if we implement it now, that > leaves ~1.5mos for someone to come up with a solution before release ... Well, we are 18 days from beta. Do we think we can handle that at this time? If so, let's do it. I guess my hope was that WAL could record the file/version# name in its logs, and use those to find the files, rather than making the file names themselves just numbers, but I know some thought that there was a chick-and-egg problem in doing that. -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 853-3000+ If your life is a hard drive, | 830 Blythe Avenue + Christ can be your backup. | Drexel Hill, Pennsylvania19026
On Tue, 12 Sep 2000, Bruce Momjian wrote: > > > > stock answer "we look forward to seeing patches to correct this > > > > problem" :) > > > > > > The problem is that I have no idea how to suggest writing such a tool > > > that fits all needs. > > > > IMHO, we have a choice ... we either move forward with a change that I > > *believe* everyone agrees has to happen, which will prompt someone to come > > up with a solution to (again, what I believe) is the only drawback ... or, > > we don't implement while waiting for someone to come up with a solution > > ... > > > > if we wait, again, IMHO, it will never happen, since there is no impetus > > for someone to "fix the problem" ... > > > > depending on what it takes to implement, if we implement it now, that > > leaves ~1.5mos for someone to come up with a solution before release ... > > Well, we are 18 days from beta. Do we think we can handle that at this > time? My feeling was that the implementation was pretty much decided upon, it was just a matter of saying "go ahead, do it" ... so, "go ahead, do it" ... we have 18 days before beta, and then another 30 or so until release, which should have us releasing, what, around Jan 1, 2001? :)
> > > depending on what it takes to implement, if we implement it now, that > > > leaves ~1.5mos for someone to come up with a solution before release ... > > > > Well, we are 18 days from beta. Do we think we can handle that at this > > time? > > My feeling was that the implementation was pretty much decided upon, it > was just a matter of saying "go ahead, do it" ... so, "go ahead, do > it" ... we have 18 days before beta, and then another 30 or so until > release, which should have us releasing, what, around Jan 1, 2001? :) Yes, Jan 1 would be my guess. My only question is whether we are making things harder for administrators and whether putting some smarts in WAL to keep readable file names would be easier. -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 853-3000+ If your life is a hard drive, | 830 Blythe Avenue + Christ can be your backup. | Drexel Hill, Pennsylvania19026
On Tue, 12 Sep 2000, Bruce Momjian wrote: > > > > depending on what it takes to implement, if we implement it now, that > > > > leaves ~1.5mos for someone to come up with a solution before release ... > > > > > > Well, we are 18 days from beta. Do we think we can handle that at this > > > time? > > > > My feeling was that the implementation was pretty much decided upon, it > > was just a matter of saying "go ahead, do it" ... so, "go ahead, do > > it" ... we have 18 days before beta, and then another 30 or so until > > release, which should have us releasing, what, around Jan 1, 2001? :) > > Yes, Jan 1 would be my guess. My only question is whether we are making > things harder for administrators and whether putting some smarts in WAL > to keep readable file names would be easier. guess we'll find out ... considering what I've seen of Oracle and its layouts, I doubt we're making anything harder then most are used to :) now, guess the next question is ... who is the one implementing this and how soon can we get it in place?
> My idea was to append a version number or oid on to the end > of the file name, and use that somehow. You'll lose all you would buy as soon as we'll begin to store many relations in single file... and I would like to implement this in 7.1 Vadim
[ Charset ISO-8859-1 unsupported, converting... ] > > My idea was to append a version number or oid on to the end > > of the file name, and use that somehow. > > You'll lose all you would buy as soon as we'll begin to store many > relations in single file... and I would like to implement this in 7.1 Yes, that is the key. You want to implement a new storage manager, so if that is coming, we may as well take the hit now. -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 853-3000+ If your life is a hard drive, | 830 Blythe Avenue + Christ can be your backup. | Drexel Hill, Pennsylvania19026
"Mikheev, Vadim" wrote: > > > My idea was to append a version number or oid on to the end > > of the file name, and use that somehow. > > You'll lose all you would buy as soon as we'll begin to store many > relations in single file... Perhaps we could then use the name of DATASPACE = filename ? Or will the fact that some relations are stored in the same file be completely invisible to the user ? > and I would like to implement this in 7.1 Will this new storage manager replace the current one or will one be able to choose which storage manager to use (at compile time, at startup, for each table)? PostgreSQL started as an extensible ORDBMS, but IIRC at some stage all other SMs were thrown out. I don't think it would be a good idea to completely abandon the notion of storage manager as a replacable component. OTOH, the idea of storing single-inheritance hierarchies (SQL3 CREATE UNDER) in one file would almost automatically get us many benefits, like shared primary keys and automatic index inheriting. -------------- Hannu
At 09:44 13/09/00 +0300, Hannu Krosing wrote: >"Mikheev, Vadim" wrote: >> >> > My idea was to append a version number or oid on to the end >> > of the file name, and use that somehow. >> >> You'll lose all you would buy as soon as we'll begin to store many >> relations in single file... > >Perhaps we could then use the name of DATASPACE = filename ? > I don't want to (re)^n ignite the the debate, but file naming has been discussed many times before and my recollection of the result of the last debate was that the names should either be random or OID based. It seems that adding a different meaning to filenames at this stage would be a bad idea, and doom us to repeat the file naming debate again in a year or two. My vote is for a random number, and then someone can write the tools to display the file info. I'll even volunteer to work on them... ---------------------------------------------------------------- Philip Warner | __---_____ Albatross Consulting Pty. Ltd. |----/ - \ (A.B.N. 75 008 659 498) | /(@) ______---_ Tel: (+61) 0500 83 82 81 | _________ \ Fax: (+61) 0500 83 82 82 | ___________ | Http://www.rhyme.com.au | / \| | --________-- PGP key available upon request, | / and from pgp5.ai.mit.edu:11371 |/
> You have to tell us whether you plan to implement > a safe file rename in WAL ? If yes a simple filename > without version would be possible and better. What do you mean? Vadim
> Will this new storage manager replace the current one or will one be > able to choose which storage manager to use (at compile time, at > startup, for each table)? This would be possible, but no way if new smgr will be overwriting one (smgr nature affects access methods). > PostgreSQL started as an extensible ORDBMS, but IIRC at some stage > all other SMs were thrown out. There was just one additional smgr for stable memory. If someone has this feature in comp then he could try to resurrect it. > I don't think it would be a good idea to completely abandon the > notion of storage manager as a replacable component. Smgr wrapper is still in place. Vadim
> My vote is for a random number, and then someone can write > the tools to display the file info. I'll even volunteer to > work on them... Ok. If someone will decide to implement this please try to use RelFileNode structure defined in storage/relfilenode.h. It's just place holder for the moment - I needed in *some* structure as "file pointer" in log records - so feel free to change context. Vadim
At 10:48 13/09/00 -0700, Mikheev, Vadim wrote: >> My vote is for a random number, and then someone can write >> the tools to display the file info. I'll even volunteer to >> work on them... > >Ok. If someone will decide to implement this please try to use >RelFileNode structure defined in storage/relfilenode.h. Just to clarify, I was offering to write the file info tools, not the changes to the file handling in PG. Although I would be willing to help in the latter for obvious reasons. ---------------------------------------------------------------- Philip Warner | __---_____ Albatross Consulting Pty. Ltd. |----/ - \ (A.B.N. 75 008 659 498) | /(@) ______---_ Tel: (+61) 0500 83 82 81 | _________ \ Fax: (+61) 0500 83 82 82 | ___________ | Http://www.rhyme.com.au | / \| | --________-- PGP key available upon request, | / and from pgp5.ai.mit.edu:11371 |/
Rename... Why would we need in rename with OID filenames? Ok, let's start with OID (*without tablename prefix|suffix*) filenames and we'll see later how it will work. So, could someone implement OID filenames? (Please use RelFileNode structure). Vadim > > > You have to tell us whether you plan to implement > > > a safe file rename in WAL ? If yes a simple filename > > > without version would be possible and better. > > > > What do you mean? > > The previous discussion we had where we concluded, that > an os rename of a file cannot be done without risc. > But that risc was imho acceptable to avoid the extra version > in the filename > > (a rename back to the old name could fail when the tx is supposed > to be rolled back). > > Search the archive for "file rename sync". > > My conlusion would be an oid only filename, or a mixture of > oid and tablename, where tablename can be wildcarded on a > directory search, > since oid is already unique. No version in the name, we would > do renames in > that case. > > If I remember correctly a patch exists that does oid only filenames. > > Andreas > >
"Mikheev, Vadim" wrote: > Rename... Why would we need in rename with OID filenames? Andreas seems to refer to in place replacement of OID files e.g. using your *relink*. Regards. Hiroshi Inoue
> > Rename... Why would we need in rename with OID filenames? > > Andreas seems to refer to in place replacement of OID files e.g. > using your *relink*. Sorry, I've messed things for myself. Ok. In short, I vote for UNIQUE_ID (unrelated to pg_class.oid) file names. I think that it's better to implement this (but neither OID nor OID.VERSION) right now because of this is like what we'll have in new smgr - tablespace_id.relation_file_node. Pg_class' OID is kind of logical things, totaly unrelated to the issue how/where to store relation file. Please comment ASAP. Vadim
> -----Original Message----- > From: Mikheev, Vadim [mailto:vmikheev@SECTORBASE.COM] > > > > Rename... Why would we need in rename with OID filenames? > > > > Andreas seems to refer to in place replacement of OID files e.g. > > using your *relink*. > > Sorry, I've messed things for myself. > > Ok. In short, I vote for UNIQUE_ID (unrelated to pg_class.oid) file names. > I think that it's better to implement this (but neither OID nor > OID.VERSION) > right now > because of this is like what we'll have in new smgr - > tablespace_id.relation_file_node. > Pg_class' OID is kind of logical things, totaly unrelated to the issue > how/where to > store relation file. > > Please comment ASAP. > Philip Warner mentioned about the advantage of random number. It's exactly what I've wanted to say. >> it removes the temptation to write utilities that rely on >> the internal representation of our data. It is preferable that file naming rule is encapsulated so that we can change it without notice. Regards. Hiroshi Inoue
On Thu, 14 Sep 2000, Mikheev, Vadim wrote: > > > > Rename... Why would we need in rename with OID filenames? > > > > Andreas seems to refer to in place replacement of OID files e.g. > > using your *relink*. > > Sorry, I've messed things for myself. > > Ok. In short, I vote for UNIQUE_ID (unrelated to pg_class.oid) file names. > I think that it's better to implement this (but neither OID nor OID.VERSION) > right now > because of this is like what we'll have in new smgr - > tablespace_id.relation_file_node. > Pg_class' OID is kind of logical things, totaly unrelated to the issue > how/where to > store relation file. > > Please comment ASAP. sounds perfect to me
> Philip Warner mentioned about the advantage of random number. > It's exactly what I've wanted to say. > > >> it removes the temptation to write utilities that rely on > >> the internal representation of our data. > > It is preferable that file naming rule is encapsulated so that we > can change it without notice. So, I assume that you vote YES on this subject? -:) (As far as I remember, it was your idea). Vadim
> -----Original Message----- > From: Mikheev, Vadim [mailto:vmikheev@SECTORBASE.COM] > > > Philip Warner mentioned about the advantage of random number. > > It's exactly what I've wanted to say. > > > > >> it removes the temptation to write utilities that rely on > > >> the internal representation of our data. > > > > It is preferable that file naming rule is encapsulated so that we > > can change it without notice. > > So, I assume that you vote YES on this subject? -:) > (As far as I remember, it was your idea). > Yes. Hiroshi Inoue
> > So, I assume that you vote YES on this subject? -:) > > (As far as I remember, it was your idea). > > > > Yes. UNIQUE_ID file names: Hiroshi, Marc, Vadim We can use oids as unique ids, but these were another oids -:) Vadim