Thread: contrib promotion?
There has been action to clean up and remove some contrib modules, and this is good. I would like to suggest that we should try to move one or two the other way, namely right into the core proper, on the ground that they have widespread applicability and should have maximum visibility. I'm talking particularly about pgcrypto and tsearch2, and the main thing lacking that I see with these is documentation. Thoughts? cheers andrew
Andrew Dunstan <andrew@dunslane.net> writes:
> There has been action to clean up and remove some contrib modules, and
> this is good. I would like to suggest that we should try to move one or
> two the other way, namely right into the core proper, on the ground that
> they have widespread applicability and should have maximum visibility.
> I'm talking particularly about pgcrypto and tsearch2, and the main thing
> lacking that I see with these is documentation.
I don't see a strong need for moving pgcrypto into core, and there's at
least one argument against it: if someone needs a crypto-free version of
postgres for use someplace with benighted laws, they would be screwed.
tsearch2 is functionality that definitely should be in core eventually,
but even Oleg still says it's not done. Aside from the documentation
issue, it's not clear that we've got a stable API for it.
regards, tom lane
Tom Lane wrote: >Andrew Dunstan <andrew@dunslane.net> writes: > > >>There has been action to clean up and remove some contrib modules, and >>this is good. I would like to suggest that we should try to move one or >>two the other way, namely right into the core proper, on the ground that >>they have widespread applicability and should have maximum visibility. >>I'm talking particularly about pgcrypto and tsearch2, and the main thing >>lacking that I see with these is documentation. >> >> > >I don't see a strong need for moving pgcrypto into core, and there's at >least one argument against it: if someone needs a crypto-free version of >postgres for use someplace with benighted laws, they would be screwed. > > Could that be handled with a configure option? >tsearch2 is functionality that definitely should be in core eventually, >but even Oleg still says it's not done. Aside from the documentation >issue, it's not clear that we've got a stable API for it. > > > Well, that's a pity. This will be the 4th release with it in contrib, IIRC. I know it's advanced stuff, but surely it has to settle down sometime. Quite apart from anything else, it's important that we do get better docco on these modules. cheers andrew
Tom Lane wrote: > Andrew Dunstan <andrew@dunslane.net> writes: >> There has been action to clean up and remove some contrib modules, and >> this is good. I would like to suggest that we should try to move one or >> two the other way, namely right into the core proper, on the ground that >> they have widespread applicability and should have maximum visibility. >> I'm talking particularly about pgcrypto and tsearch2, and the main thing >> lacking that I see with these is documentation. > > I don't see a strong need for moving pgcrypto into core, and there's at > least one argument against it: if someone needs a crypto-free version of > postgres for use someplace with benighted laws, they would be screwed. Doesn't our inclusion of md5() pretty much blow that argument away? (Just asking). Sincerely, Joshua D. Drake -- === The PostgreSQL Company: Command Prompt, Inc. === Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240 Providing the most comprehensive PostgreSQL solutionssince 1997 http://www.commandprompt.com/
"Joshua D. Drake" <jd@commandprompt.com> writes:
> Tom Lane wrote:
>> I don't see a strong need for moving pgcrypto into core, and there's at
>> least one argument against it: if someone needs a crypto-free version of
>> postgres for use someplace with benighted laws, they would be screwed.
> Doesn't our inclusion of md5() pretty much blow that argument away?
No: md5 is hashing, not encryption. The difference is that you can't
retrieve the original plaintext from a hash. That is a very large
difference in the eyes of most munitions laws --- encryption is useful
for spies, hashing not so much.
regards, tom lane
On Jul 14, 2006, at 12:32 PM, Joshua D. Drake wrote: > Doesn't our inclusion of md5() pretty much blow that argument away? > (Just asking). I don't think so because md5 is just a one way hash function. There is no method to decrypt anything :). John DeSoi, Ph.D. http://pgedit.com/ Power Tools for PostgreSQL
Andrew Dunstan wrote: > >I don't see a strong need for moving pgcrypto into core, and there's > > at least one argument against it: if someone needs a crypto-free > > version of postgres for use someplace with benighted laws, they > > would be screwed. > > Could that be handled with a configure option? In such a jurisdiction you would possibly have to distribute rebuilt tarballs with all the source code removed. The contrib option makes that much easier. > Quite apart from anything else, it's important that we do get better > docco on these modules. I'm willing to help with DocBook options. What do you have in mind? -- Peter Eisentraut http://developer.postgresql.org/~petere/
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 >> Doesn't our inclusion of md5() pretty much blow that argument away? >> (Just asking). > > I don't think so because md5 is just a one way hash function. There > is no method to decrypt anything :). Actually, I've had to install pgcrypto on more than one occasion for clients who needed to have sha1 instead of md5. I've had to install pgcrypto for other functions as well, so +1 for me on coring it, but at the least please consider adding in sha1. - -- Greg Sabino Mullane greg@turnstep.com PGP Key: 0x14964AC8 200607141355 http://biglumber.com/x/web?pk=2529DF6AB8F79407E94445B4BC9B906714964AC8 -----BEGIN PGP SIGNATURE----- iD8DBQFEt9sfvJuQZxSWSsgRAhrCAJ9VZneUN6+3pvq+vJf+Y6lhjamj/QCfUDYc +rYM+ITWvhw2dvo2B1hta6g= =h4aD -----END PGP SIGNATURE-----
Greg Sabino Mullane wrote: > >>>> Doesn't our inclusion of md5() pretty much blow that argument away? >>>> (Just asking). >>> I don't think so because md5 is just a one way hash function. There >>> is no method to decrypt anything :). > > Actually, I've had to install pgcrypto on more than one occasion for > clients who needed to have sha1 instead of md5. I've had to install > pgcrypto for other functions as well, so +1 for me on coring it, but > at the least please consider adding in sha1. I don't have a very strong opinion on that but sha1() is something I need on a regular base too from pgcrypto. Stefan
Peter Eisentraut wrote: >>Quite apart from anything else, it's important that we do get better >>docco on these modules. >> >> > >I'm willing to help with DocBook options. What do you have in mind? > > > Well, if we could make provision for sucking in a chapter per contrib module if it exists that would be a good start. (Speaking of DocBook, did we ever make progress on switching to XML?) cheers andrew
> tsearch2 is functionality that definitely should be in core eventually, > but even Oleg still says it's not done. Aside from the documentation > issue, it's not clear that we've got a stable API for it. Issues/TODO to move tsearch2 into core (by fast look) * memory management. Dictionaries and tsearch2 itself cache a lot of data in memory allocated by malloc/palloc(TopContext)and not all can be correctly freed by reset_tsearch() function. * tsearch2 doesn't automatically reinit dictionary/configuration after configuration changes * Also, dictionary doesn't automatically recompile its files if they were changed * It will be good to store shared information (such as dictionary's structure) in shared memory. For example, ispell dicttakes for compile its files about 1-5 seconds, sharing should help * I suppose, at least snowball stemmers should be compiled in postgresql or compiled them into separate *.so libs. Sources of all snowball's files take ~1Mb on disk. The goal is easier configuration for end-user. Ispell dict's files take1-3 Mb per language. * Now database can be created with another encoding then cluster was initialized, but locale can't be changed. Tsearch2depends on locale because of lower/upper casing and isalpha(and somr another is*). So, it's possible to get unworkingtsearch2 configuration. I'm going to vacation tomorrow for two week, so I havn't time to resolve that issues before feature freeze, sorry.
We estimate 1 month for both us to complete these issues/todos and 1 month extra to write documentation, so we certainly are not ready for 8.2, most probable 8.3. We're looking for sponsorship of our work. Oleg On Sat, 15 Jul 2006, Teodor Sigaev wrote: >> tsearch2 is functionality that definitely should be in core eventually, >> but even Oleg still says it's not done. Aside from the documentation >> issue, it's not clear that we've got a stable API for it. > > Issues/TODO to move tsearch2 into core (by fast look) > * memory management. Dictionaries and tsearch2 itself cache a lot > of data in memory allocated by malloc/palloc(TopContext) and > not all can be correctly freed by reset_tsearch() function. > * tsearch2 doesn't automatically reinit dictionary/configuration > after configuration changes > * Also, dictionary doesn't automatically recompile its files if they > were changed > * It will be good to store shared information (such as > dictionary's structure) in shared memory. For example, > ispell dict takes for compile its files about 1-5 seconds, sharing > should help > * I suppose, at least snowball stemmers should be compiled in > postgresql or compiled them into separate *.so libs. Sources > of all snowball's files take ~1Mb on disk. The goal is easier > configuration for end-user. Ispell dict's files take 1-3 Mb per > language. > * Now database can be created with another encoding then cluster was > initialized, but locale can't be changed. Tsearch2 depends on locale > because of lower/upper casing and isalpha(and somr another is*). So, > it's possible to get unworking tsearch2 configuration. > > > I'm going to vacation tomorrow for two week, so I havn't time to resolve that > issues before feature freeze, sorry. > > > > ---------------------------(end of broadcast)--------------------------- > TIP 3: Have you checked our extensive FAQ? > > http://www.postgresql.org/docs/faq > Regards, Oleg _____________________________________________________________ Oleg Bartunov, Research Scientist, Head of AstroNet (www.astronet.ru), Sternberg Astronomical Institute, Moscow University, Russia Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/ phone: +007(495)939-16-83, +007(495)939-23-83
On 7/14/06, Tom Lane <tgl@sss.pgh.pa.us> wrote: > I don't see a strong need for moving pgcrypto into core, and there's at > least one argument against it: if someone needs a crypto-free version of > postgres for use someplace with benighted laws, they would be screwed. Image of hypothetical evil government is not a thing to base decisions on :) Although I've tried to develop pgcrypto to be easily mergable into core, I don't want to push it myself, the push should come from users. That said, there is one situation that is badly handled in current setup - storing passwords in database. There is md5() function in core and everything in /contrib in basically invisible in website and official docs. So even PG core devs suggest using md5() for this task. But this is inadequate - bruteforcing md5 hash can be done pretty easily on todays desktop computers. PostgreSQL itself can get away with it only because it regular users cant see the hash. But that is not so for ordinary apps. So I would like either some mention of the more useful/stable modules in core docs or a way for contrib modules to become 'official' add-on modules (like PL-s are). Full merge into core would fix this also, but indeed there is not many techical reasons for it. (And editing pg_proc.h is PITA - I'd consider it technical reason against it ;) -- marko
On 7/14/2006 12:01 PM, Tom Lane wrote: > tsearch2 is functionality that definitely should be in core eventually, > but even Oleg still says it's not done. Aside from the documentation > issue, it's not clear that we've got a stable API for it. Would moving it in its current state into core help it to get there and what would the risk be if it is added and remains fragile for a release? If the problems are wrong positives/negatives on search results, then it is IMHO okay. If the problems are process crashes or the like, then it is not. Jan -- #======================================================================# # It's easier to get forgiveness for being wrong than for being right. # # Let's break this rule - forgive me. # #================================================== JanWieck@Yahoo.com #
On Fri, Jul 14, 2006 at 08:08:11PM +0200, Stefan Kaltenbrunner wrote: > Greg Sabino Mullane wrote: > > > >>>> Doesn't our inclusion of md5() pretty much blow that argument away? > >>>> (Just asking). > >>> I don't think so because md5 is just a one way hash function. There > >>> is no method to decrypt anything :). > > > > Actually, I've had to install pgcrypto on more than one occasion for > > clients who needed to have sha1 instead of md5. I've had to install > > pgcrypto for other functions as well, so +1 for me on coring it, but > > at the least please consider adding in sha1. > > I don't have a very strong opinion on that but sha1() is something I > need on a regular base too from pgcrypto. sha1 would be nice, as would actual datatypes for them (though the datatypes are probably better left to pgFoundry). -- Jim C. Nasby, Sr. Engineering Consultant jnasby@pervasive.com Pervasive Software http://pervasive.com work: 512-231-6117 vcard: http://jim.nasby.net/pervasive.vcf cell: 512-569-9461
On Tue, Jul 18, 2006 at 03:37:52PM +0300, Marko Kreen wrote: > On 7/14/06, Tom Lane <tgl@sss.pgh.pa.us> wrote: > >I don't see a strong need for moving pgcrypto into core, and there's at > >least one argument against it: if someone needs a crypto-free version of > >postgres for use someplace with benighted laws, they would be screwed. > > Image of hypothetical evil government is not a thing to base decisions on :) > > Although I've tried to develop pgcrypto to be easily mergable into core, > I don't want to push it myself, the push should come from users. > > That said, there is one situation that is badly handled in current > setup - storing passwords in database. There is md5() function in > core and everything in /contrib in basically invisible in website > and official docs. So even PG core devs suggest using md5() for > this task. But this is inadequate - bruteforcing md5 hash can be > done pretty easily on todays desktop computers. PostgreSQL itself > can get away with it only because it regular users cant see the hash. > But that is not so for ordinary apps. > > So I would like either some mention of the more useful/stable modules > in core docs or a way for contrib modules to become 'official' add-on > modules (like PL-s are). This is actually an issue that goes way beyond pgcrypto. I think the manual should formally mention both /contrib and pgFoundry.org as someplace to get add-on features. An even better long-term solution would be something akin to CPAN, but I'm not holding my breath for that... > Full merge into core would fix this also, but indeed there is not many > techical reasons for it. (And editing pg_proc.h is PITA - I'd consider > it technical reason against it ;) > > -- > marko > > ---------------------------(end of broadcast)--------------------------- > TIP 6: explain analyze is your friend > -- Jim C. Nasby, Sr. Engineering Consultant jnasby@pervasive.com Pervasive Software http://pervasive.com work: 512-231-6117 vcard: http://jim.nasby.net/pervasive.vcf cell: 512-569-9461
>> So I would like either some mention of the more useful/stable modules >> in core docs or a way for contrib modules to become 'official' add-on >> modules (like PL-s are). > > This is actually an issue that goes way beyond pgcrypto. I think the > manual should formally mention both /contrib and pgFoundry.org as > someplace to get add-on features. An even better long-term solution > would be something akin to CPAN, but I'm not holding my breath for > that... The manual does talk about pgFoundry, especially in 8.2 (that was my patch ;)). It talks about it in 8.1 as well but it is not as apparent. Joshua D. Drake > >> Full merge into core would fix this also, but indeed there is not many >> techical reasons for it. (And editing pg_proc.h is PITA - I'd consider >> it technical reason against it ;) >> >> -- >> marko >> >> ---------------------------(end of broadcast)--------------------------- >> TIP 6: explain analyze is your friend >> > -- === The PostgreSQL Company: Command Prompt, Inc. === Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240 Providing the most comprehensive PostgreSQL solutionssince 1997 http://www.commandprompt.com/