Thread: PostgreSQL in the press again
I've just been contacted by ZDnet again for a few more details relating to a story they are about to run (today!...) about PostgreSQL reference sites. They have quotes from a number of them.... one of which is, I believe, the Oxford University Computing Centre. I have been asked about why there is no single preferred Replication Solution etc...; I said it was the nature of the development community that many ideas are allowed out at once. Many people ask this, so let's chew it out now ahead of the 8.0 release... I've got better things to do too, but I keep getting asked, so I want to have a full steer... we probably need a press FAQ for the release: this is No.1 "So which one would you choose?" My personal answer was that IMHO Slony seems to have the most community support and is the one I would choose to use. Flame on... -- Best Regards, Simon Riggs
At the risk of making dire enemies I vote for SLONY :-) On November 9, 2004 10:47 am, Simon Riggs wrote: > I've just been contacted by ZDnet again for a few more details relating > to a story they are about to run (today!...) about PostgreSQL reference > sites. They have quotes from a number of them.... one of which is, I > believe, the Oxford University Computing Centre. > > I have been asked about why there is no single preferred Replication > Solution etc...; I said it was the nature of the development community > that many ideas are allowed out at once. > > Many people ask this, so let's chew it out now ahead of the 8.0 > release... I've got better things to do too, but I keep getting asked, > so I want to have a full steer... we probably need a press FAQ for the > release: this is No.1 > > "So which one would you choose?" My personal answer was that IMHO Slony > seems to have the most community support and is the one I would choose > to use. > > Flame on...
Am Dienstag, 9. November 2004 16:47 schrieb Simon Riggs: > I have been asked about why there is no single preferred Replication > Solution etc...; I said it was the nature of the development community > that many ideas are allowed out at once. What are the other solutions? -- Peter Eisentraut http://developer.postgresql.org/~petere/
On Tue, Nov 09, 2004 at 03:47:48PM +0000, Simon Riggs wrote:
> I have been asked about why there is no single preferred Replication
> Solution etc...; I said it was the nature of the development community
> that many ideas are allowed out at once.
I don't know that this is the only reason. Both Oracle and IBM will
sell you more than one thing when you go to them asking for
replication. That's because there are different needs, and one has
to have systems which support those needs. I doubt that a single
system would really be able to accommodate every scenario.
A
--
Andrew Sullivan | ajs@crankycanuck.ca
A certain description of men are for getting out of debt, yet are
against all taxes for raising money to pay it off.
--Alexander Hamilton
On Tue, 2004-11-09 at 17:04, Andrew Sullivan wrote: > On Tue, Nov 09, 2004 at 03:47:48PM +0000, Simon Riggs wrote: > > I have been asked about why there is no single preferred Replication > > Solution etc...; I said it was the nature of the development community > > that many ideas are allowed out at once. > > I don't know that this is the only reason. Completely agree, but knowing I would be quoted, I had to say something and didn't want to upset too many applecarts. This post is all about: "so what do we say then?" -- Best Regards, Simon Riggs
A long time ago, in a galaxy far, far away, peter_e@gmx.net (Peter Eisentraut) wrote: > Am Dienstag, 9. November 2004 16:47 schrieb Simon Riggs: >> I have been asked about why there is no single preferred Replication >> Solution etc...; I said it was the nature of the development community >> that many ideas are allowed out at once. > > What are the other solutions? Mammoth Replicator, and whatever is happening with eRServer these days... -- "cbbrowne","@","acm.org" http://www.ntlug.org/~cbbrowne/advocacy.html "The day Microsoft makes something that doesn't suck is probably the day they start making vacuum cleaners" - Ernst Jan Plugge
On Tue, 2004-11-09 at 16:33, Peter Eisentraut wrote:
> Am Dienstag, 9. November 2004 16:47 schrieb Simon Riggs:
> > I have been asked about why there is no single preferred Replication
> > Solution etc...; I said it was the nature of the development community
> > that many ideas are allowed out at once.
>
> What are the other solutions?
The press release additional information says
Replication:
Slony-I
pgReplicator
eRServer
...though there are I think 2? other web pages/codesets out there that
claim to work with PostgreSQL also..
--
Best Regards, Simon Riggs
Guys, > > What are the other solutions? > > Mammoth Replicator, and whatever is happening with eRServer these > days... dbMirror is still quite popular. This is partly because it is better suited for "very slow replication", e.g. replication via FTP server once per day, a la MusicBrainz. Both pgPool and C-JDBC offer synchronous query distribution based MM replication, although at the present time neither is transaction-safe. When we get XA, C-JDBC will become a very viable alternative. The issue talking with the press is that you need to communicate to them that "Replication" is a general programming topic, and NOT a single task, just like "database" is. Nobody in the industry would expect to use the same database for all purposes; neither would anyone expect to use the same replication tool for all purposes. The reason you get this question all the time is: 1) Many DBMSs (SQL Server, MySQL) support only one replication tool; 2) reporters have no clear idea what "replication" is. Personally, I'd answer: "Slony-I is undoubtedly our most popular replication tool. It supports Master-Slave High Availability Replication. However, there are a number of other solutions, such as dbMirror, eRServer, pgPool, C-JDBC, and the proprietary Mammoth Replicator, all of which are in wide use because they solve different replication problems than Slony-I does. Replication is not a single solution for a single problem; it is several solutions for a wide array of different problems. That's why no one replication tool will ever be the "default" replication for PostgreSQL." -- --Josh Josh Berkus Aglio Database Solutions San Francisco
Andrew, > ... That's because there are different needs, and one has > to have systems which support those needs. I doubt that a single > system would really be able to accommodate every scenario. > I've seen this argument several times and it always prevents Slony and other replication solutions from being mentioned. My suggestion is that the scenarios are outlined (there can't be that many, 2-4 perhaps?) together with the corresponding preferred solution. Regards, Thomas Hallgren
On Tue, 9 Nov 2004, Josh Berkus wrote: > Guys, > >>> What are the other solutions? >> >> Mammoth Replicator, and whatever is happening with eRServer these >> days... > > dbMirror is still quite popular. This is partly because it is better suited > for "very slow replication", e.g. replication via FTP server once per day, a > la MusicBrainz. > > Both pgPool and C-JDBC offer synchronous query distribution based MM > replication, although at the present time neither is transaction-safe. When > we get XA, C-JDBC will become a very viable alternative. > > The issue talking with the press is that you need to communicate to them that > "Replication" is a general programming topic, and NOT a single task, just > like "database" is. Nobody in the industry would expect to use the same > database for all purposes; neither would anyone expect to use the same > replication tool for all purposes. The reason you get this question all the > time is: > 1) Many DBMSs (SQL Server, MySQL) support only one replication tool; > 2) reporters have no clear idea what "replication" is. > > Personally, I'd answer: > > "Slony-I is undoubtedly our most popular replication tool. It supports > Master-Slave High Availability Replication. However, there are a number of > other solutions, such as dbMirror, eRServer, pgPool, C-JDBC, and the > proprietary Mammoth Replicator, all of which are in wide use because they > solve different replication problems than Slony-I does. Replication is not a > single solution for a single problem; it is several solutions for a wide > array of different problems. That's why no one replication tool will ever be > the "default" replication for PostgreSQL." This answer almost sounds perfect for inclusion into the FAQ itself ... ---- Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email: scrappy@hub.org Yahoo!: yscrappy ICQ: 7615664
Marc, > This answer almost sounds perfect for inclusion into the FAQ itself ... > Yes, this was excellent. The "I'd answer" part is great for the less technical audience and a reduced version (retaining the very brief explanation of each solution) of the full text for the FAQ. Regards, Thomas Hallgren
On Tue, Nov 09, 2004 at 07:54:14PM +0100, Thomas Hallgren wrote:
> I've seen this argument several times and it always prevents Slony and
> other replication solutions from being mentioned. My suggestion is that
> the scenarios are outlined (there can't be that many, 2-4 perhaps?)
> together with the corresponding preferred solution.
Well, see Josh's excellent response in this thread, also. I can
think of at least three scenarios:
1. Only one read-write database (or, more precisely, one per
table) with (possibly multiple) read-only copy(ies). Most of the
systems meet this need. Mammoth's Replicator, erserver, Slony-I,
dbmirror will all do this. Some of them do it with greater
flexibility than others: the BSD version of erserver is a fair pain
in the neck to administer, for instance. Each has some features
which may work for particular installations. Slony-I, for intstance,
currently requires superuser access on all nodes, and that all nodes
talk to one another. For connections to untrusted sites, then,
Slony-I would be an exceptionally bad choice. It also has pretty
awkward support for DDL, on the grounds that your code shouldn't add
and remove tables as a matter of course (because, as Jan argues,
that's akin to self-modifying code). If you have an application
which depends on the adding and removing of tables as part of regular
operation, some other approach will be better for you.
2. One read-write database (per table) with untrusted target
replicas. This is a nifty extension of dbmirror that came from the
bright folks at MusicBrainz.
3. Multi-master, with possible loss of transactions in one node
or another. Some applications can tolerate the odd lost transaction,
as long as there is _some_ system that has the data. Current
offerings of C-JDBC and pg_pool can provide this.
4. True multi-master, with transactional semantics across all
nodes. This one is currently planned, but not available anywhere.
Jan is working on an implementation; the Postgres-R project has a
proof of concept which currently doesn't offer all the features.
Does that help?
A
--
Andrew Sullivan | ajs@crankycanuck.ca
The plural of anecdote is not data.
--Roger Brinner
>> >>"So which one would you choose?" My personal answer was that IMHO Slony >>seems to have the most community support and is the one I would choose >>to use. >> >>Flame on... I don't think it really matters. We should just mention the ones that are known to be heavily supported and active in development. Slony-I Mammoth Replicator DbMirror? possibly cjdbc, pgpool? Sincerely, Joshua D. Drake > > > ---------------------------(end of broadcast)--------------------------- > TIP 4: Don't 'kill -9' the postmaster -- Command Prompt, Inc., home of PostgreSQL Replication, and plPHP. Postgresql support, programming shared hosting and dedicated hosting. +1-503-667-4564 - jd@commandprompt.com - http://www.commandprompt.com Mammoth PostgreSQL Replicator. Integrated Replication for PostgreSQL
Attachment
On Tue, 9 Nov 2004, Simon Riggs wrote:
> I have been asked about why there is no single preferred Replication
> Solution etc...; I said it was the nature of the development community
> that many ideas are allowed out at once.
>
> Many people ask this, so let's chew it out now ahead of the 8.0
> release... I've got better things to do too, but I keep getting asked,
> so I want to have a full steer... we probably need a press FAQ for the
> release: this is No.1
Well up here in the Inland Northwest (USA) we suffer a quandry about
pickup trucks. Full/mid/mini size, 2/4 wheel drive, Chevy/Dodge/Ford/
Toyota/etc.
> "So which one would you choose?" My personal answer was that IMHO Slony
> seems to have the most community support and is the one I would choose
> to use.
I prefer a full size, 2 wheel drive, Chevy ... Oh darn wrong question.
:-)
Rod
--
"Open Source Software - You usually get more than you pay for..."
"Build A Brighter Lamp :: Linux Apache {middleware} PostgreSQL"
> > > I prefer a full size, 2 wheel drive, Chevy ... Oh darn wrong question. > :-) OT but I can't resist... BAH full size 2 wheel Chevy? I got a F250 with a 460 in it buddy. Get a real truck ;) Of course it only gets 14 miles to the gallon and I only drive it about once every 6 weeks to take stuff to the dump :) Sincerely, Joshua D. Drake > > > Rod -- Command Prompt, Inc., home of PostgreSQL Replication, and plPHP. Postgresql support, programming shared hosting and dedicated hosting. +1-503-667-4564 - jd@commandprompt.com - http://www.commandprompt.com Mammoth PostgreSQL Replicator. Integrated Replication for PostgreSQL
Attachment
On Tue, 9 Nov 2004, Joshua D. Drake wrote: >>> >>> "So which one would you choose?" My personal answer was that IMHO Slony >>> seems to have the most community support and is the one I would choose >>> to use. >>> >>> Flame on... > > I don't think it really matters. We should just mention the ones that > are known to be heavily supported and active in development. > > Slony-I > Mammoth Replicator > DbMirror? eRServer ---- Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email: scrappy@hub.org Yahoo!: yscrappy ICQ: 7615664
On Tue, 9 Nov 2004, Christopher Browne wrote: > A long time ago, in a galaxy far, far away, peter_e@gmx.net (Peter Eisentraut) wrote: >> Am Dienstag, 9. November 2004 16:47 schrieb Simon Riggs: >>> I have been asked about why there is no single preferred Replication >>> Solution etc...; I said it was the nature of the development community >>> that many ideas are allowed out at once. >> >> What are the other solutions? > > Mammoth Replicator, and whatever is happening with eRServer these > days... eRServer is in the middle of a re-write and code review ... the old version works well for our clients, but the system requirements grew to be a *wee* bit larger then we wanted ... ---- Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email: scrappy@hub.org Yahoo!: yscrappy ICQ: 7615664
Andrew Sullivan wrote: > > ... does that help? > Ok, so to abbreviate what you and Josh writes into what I had in mind (??? denotes my own lack of further knowledge): Master + read-only slaves: - Slony-I when all sites are trusted - dbMirror for untrusted slaves and/or table based master slave assignment - Mammoth Replicator, proprietary ??? - erServer ??? Multi-master: - C-JDBC, Will be transaction safe once PostgreSQL has XA - pgPool, not transaction safe ??? (- future, the stuff Jan is working on and Postgres-R) Perhaps some info about which solutions that are PostgreSQL specific and which are database agnostic should be added. Regards, Thomas Hallgren
On Tue, Nov 09, 2004 at 09:13:00PM +0100, Thomas Hallgren wrote:
>
> Master + read-only slaves:
> - Slony-I when all sites are trusted
> - dbMirror for untrusted slaves and/or table based master slave assignment
You can do the table-based trick with Slony, too. In fact, AFAIK,
you can do it with every async system.
> - erServer ???
I don't know if anyone is stepping up to take over the BSD version,
but as far as I'm concerned, it's a dead letter. I know there is
some development going on with it at PgSQL, Inc, though.
> - C-JDBC, Will be transaction safe once PostgreSQL has XA
> - pgPool, not transaction safe ???
I think the latter would be, too, if Pg had XA.
A
--
Andrew Sullivan | ajs@crankycanuck.ca
The fact that technology doesn't work is no bar to success in the marketplace.
--Philip Greenspun
Thomas Hallgren wrote:
> Andrew Sullivan wrote:
>
>>
>> ... does that help?
>>
> Ok, so to abbreviate what you and Josh writes into what I had in mind
> (??? denotes my own lack of further knowledge):
>
> Master + read-only slaves:
> - Slony-I when all sites are trusted
> - dbMirror for untrusted slaves and/or table based master slave assignment
> - Mammoth Replicator, proprietary ???
proprietary, transaction safe and simple to
administer.
OR
propreitary, transaction log based.
The are really only four significant differences between Slony and
Mammoth Replicator:
1. Slony is trigger based, Mammoth is transaction log based
2. Slony is batch replication (Jan what do you call this?), Mammoth is
transaction based.
3. Slony requires modification to your database schema, Mammoth does not
(outside of the mutual requirement of a unique key (Mammoth requires a
primary key).
4. Slony is Open Source, Mammoth is not.
Both support:
1. 1 master, multiple slaves
2. failover, promotion capabilities
3. Neither lock the database during a full sync
4. Both are transaction safe
Sincerely,
Joshua D. Drake
> - erServer ???
>
> Multi-master:
> - C-JDBC, Will be transaction safe once PostgreSQL has XA
> - pgPool, not transaction safe ???
> (- future, the stuff Jan is working on and Postgres-R)
>
> Perhaps some info about which solutions that are PostgreSQL specific and
> which are database agnostic should be added.
>
> Regards,
> Thomas Hallgren
>
>
> ---------------------------(end of broadcast)---------------------------
> TIP 6: Have you searched our list archives?
>
> http://archives.postgresql.org
--
Command Prompt, Inc., home of PostgreSQL Replication, and plPHP.
Postgresql support, programming shared hosting and dedicated hosting.
+1-503-667-4564 - jd@commandprompt.com - http://www.commandprompt.com
Mammoth PostgreSQL Replicator. Integrated Replication for PostgreSQL
Attachment
Thomas Hallgren wrote: > Master + read-only slaves: > - Slony-I when all sites are trusted > - dbMirror for untrusted slaves and/or table based master slave > assignment - Mammoth Replicator, proprietary ??? > - erServer ??? That begs the question in turn why there are so many master/slave replication solutions. I mean, I don't care, but this categorization doesn't really answer the original question. > Multi-master: > - C-JDBC, Will be transaction safe once PostgreSQL has XA > - pgPool, not transaction safe ??? These are not multimaster solutions in the sense that you can write to any one of multiple hosts. In a sense, they are really master/slave solutions with the program components distributed differently. To write, you always have to go through one host. -- Peter Eisentraut http://developer.postgresql.org/~petere/
On Tue, 2004-11-09 at 19:00, Marc G. Fournier wrote: > On Tue, 9 Nov 2004, Josh Berkus wrote: > > > Guys, > > > >>> What are the other solutions? > >> > >> Mammoth Replicator, and whatever is happening with eRServer these > >> days... > > > > dbMirror is still quite popular. This is partly because it is better suited > > for "very slow replication", e.g. replication via FTP server once per day, a > > la MusicBrainz. > > > > Both pgPool and C-JDBC offer synchronous query distribution based MM > > replication, although at the present time neither is transaction-safe. When > > we get XA, C-JDBC will become a very viable alternative. > > > > The issue talking with the press is that you need to communicate to them that > > "Replication" is a general programming topic, and NOT a single task, just > > like "database" is. Nobody in the industry would expect to use the same > > database for all purposes; neither would anyone expect to use the same > > replication tool for all purposes. The reason you get this question all the > > time is: > > 1) Many DBMSs (SQL Server, MySQL) support only one replication tool; > > 2) reporters have no clear idea what "replication" is. > > > > Personally, I'd answer: > > > > "Slony-I is undoubtedly our most popular replication tool. It supports > > Master-Slave High Availability Replication. However, there are a number of > > other solutions, such as dbMirror, eRServer, pgPool, C-JDBC, and the > > proprietary Mammoth Replicator, all of which are in wide use because they > > solve different replication problems than Slony-I does. Replication is not a > > single solution for a single problem; it is several solutions for a wide > > array of different problems. That's why no one replication tool will ever be > > the "default" replication for PostgreSQL." > > This answer almost sounds perfect for inclusion into the FAQ itself ... > Agreed. > That's why no one replication tool will ever be > > the "default" replication for PostgreSQL." > Externally, everybody thinks that there should be just one, just like there is for other databases. That was the feedback from various PostgreSQL reference sites and that's why I was asked the question. Anyway, trying to summarise these things for PR soundbites is hard and I'll just have to put up with everybody thinking I know jack. :-) Here's the link just now... http://news.zdnet.co.uk/software/applications/0,39020384,39173013,00.htm -- Best Regards, Simon Riggs
On Tue, Nov 09, 2004 at 09:28:12PM +0000, Simon Riggs wrote:
> Externally, everybody thinks that there should be just one, just like
> there is for other databases.
I guess it's this thing that I want to understand. Why do people
believe that? Because other databases, where "other" are "the ones
I'd actually run important systems on" _don't_ have just one.
A
--
Andrew Sullivan | ajs@crankycanuck.ca
I remember when computers were frustrating because they *did* exactly what
you told them to. That actually seems sort of quaint now.
--J.D. Baldwin
On Tuesday 09 November 2004 16:35, Andrew Sullivan wrote:
> On Tue, Nov 09, 2004 at 09:28:12PM +0000, Simon Riggs wrote:
> > Externally, everybody thinks that there should be just one, just like
> > there is for other databases.
>
> I guess it's this thing that I want to understand. Why do people
> believe that? Because other databases, where "other" are "the ones
> I'd actually run important systems on" _don't_ have just one.
>
1) Many of the databases that you _wouldn't_ run important systems on have
only one type of replication and they will tell you that is all you need.
2) explaining what your replication solution(s) can do invariably leads to
what it can't do, and its safer even for vendors with multiple solutions to
just drill home the message that they have replication so they can make the
sale.
--
Robert Treat
Build A Brighter Lamp :: Linux Apache {middleware} PostgreSQL
Martha Stewart called it a Good Thing when peter_e@gmx.net (Peter Eisentraut) wrote:
> Thomas Hallgren wrote:
>> Master + read-only slaves:
>> - Slony-I when all sites are trusted
>> - dbMirror for untrusted slaves and/or table based master slave
>> assignment - Mammoth Replicator, proprietary ???
>> - erServer ???
>
> That begs the question in turn why there are so many master/slave
> replication solutions. I mean, I don't care, but this
> categorization doesn't really answer the original question.
I think there are multiple answers because of the combination of:
a) Coding before thinking, where some of the systems have been
"hacked together" without too much forethought;
b) Greatly varying implementation strategies.
For instance, one of the big problems we encountered with eRServer was
in its use of memory. The "snapshot" notion it uses tends to lead to
fairly spectacular RAM consumption. Had it gotten more design effort,
earlier on, perhaps it could have been more modest in memory usage.
That became one of the design requirements for Slony-I...
Part of the history has been that people in a rush to get some form of
replication working looked at different "parts of the elephant," and,
seeing different things, implemented different things.
If a system can get a bit more "thoughtfulness" applied to it, it may
well become forcibly preferable to the other options.
>> Multi-master:
>> - C-JDBC, Will be transaction safe once PostgreSQL has XA
>> - pgPool, not transaction safe ???
>
> These are not multimaster solutions in the sense that you can write
> to any one of multiple hosts. In a sense, they are really
> master/slave solutions with the program components distributed
> differently. To write, you always have to go through one host.
The pgpool approach is somewhat ambiguous, but you're probably right.
My expectation of a "multimaster" system is that I should be able to
fire an update at any of the 'masters' and expect it to propagate to
the rest of the databases more or less automatically. That's not what
either XA or pgpool do.
--
(format nil "~S@~S" "cbbrowne" "acm.org")
http://www.ntlug.org/~cbbrowne/linuxdistributions.html
Rules of the Evil Overlord #158. "I will exchange the labels on my
folder of top-secret plans and my folder of family recipes. Imagine
the hero's surprise when he decodes the stolen plans and finds
instructions for Grandma's Potato Salad."
<http://www.eviloverlord.com/>
On Wed, Nov 10, 2004 at 12:02:35AM -0500, Christopher Browne wrote:
> For instance, one of the big problems we encountered with eRServer was
> in its use of memory. The "snapshot" notion it uses tends to lead to
> fairly spectacular RAM consumption. Had it gotten more design effort,
> earlier on, perhaps it could have been more modest in memory usage.
Actually, to be fair to the original designers, that was precisely
one of their objections; but they were over-ruled by someone who
ought to have known better, but had a fixation with Java. The folks
actually responsible for the mistake in the design did not, of
course, end up continuing work on it; but the damage was already
done.
Indeed, one of the really valuable things I learned from my
experiences with erserver was that it is even harder to revisit
version 1's poor decisions in version 2 than I'd previously believed.
It's interesting that sometimes people argue in the mainline
PostgreSQL project that getting something which delivers most of what
poeple want is better than doing nothing; and yet those arguments
often don't win, unless they come with a plan of how the "real" goal
will be achieved. Tablespaces is a good example: there were a lot of
half-way proposals floating about, but until someone showed up with
plans that at least indicated the path to full-blown support, nothing
really happened. I think this is a great strength of the project,
and my experience with erserver (not to mention Marc's recent comment
that they're finding the requirements for erserver a little onerous,
so they're fixing it) is an example of the reason why.
A
--
Andrew Sullivan | ajs@crankycanuck.ca
When my information changes, I alter my conclusions. What do you do sir?
--attr. John Maynard Keynes
Andrew Sullivan wrote: >On Tue, Nov 09, 2004 at 09:28:12PM +0000, Simon Riggs wrote: > > >>Externally, everybody thinks that there should be just one, just like >>there is for other databases. >> >> > >I guess it's this thing that I want to understand. Why do people >believe that? Because other databases, where "other" are "the ones >I'd actually run important systems on" _don't_ have just one. > >A > > > > I don;t think it is really an issue of having just one supported replication system. It is a different issue that people don't really internalize well enough to talk about. It is, IMO, the fact that these other systems (MS SQL, Oracle, etc) have replication systems through the same vendor as the software itself. Now I realize that Mommoth PostgreSQL is available from the same vendor that offros Mommoth Replicator. But in the eyes of the customer Mammoth and PostgreSQL are not identical. Therefore they are afraid of having one vendor tell them that the problem exists with another third-party product. This reaction that add-ons are somewhat inferior to solutions offered from the same source then leads people to say "PostgreSQL doesn't have replication!" MS SQL has at least 2 forms of replication, PostgreSQL doesn't have any in this view. What we need to do to counter this perception is advertise the fact that many of these replication systems are developed by core community members and/or developers, and some of them are quite mature. Some, like dbmirror, have a history of being included in the source release, even. We need to advertise these things. Yes, the following projects had involvement by members of our core team.... The following projects/products have great commercial support.... The following products/projects are widely used in the community.... As a community, obviously our focus will be on open source solutions, but I have no problem with such a document pointing to commercial products from time to time in order to show that we as a community endorse solutions to this problem. (Regarding commercial Win32 ports, I am not opposed to having a "commercial ports" list available either.) Yes, this opens a can of worms, but it solves another bigger problem (perception by potential customers and the press). It might not be a bad idea for a collaborative effort to be used to write a faq solely on replication options available. Best Wishes, Chris Travers Metatron Technology Consulting
> Now I realize that Mommoth PostgreSQL is available from the same vendor > that offros Mommoth Replicator. > But in the eyes of the customer Mammoth > and PostgreSQL are not identical. This is actually something we are working on. Command Prompt over the last 6 months or so has been trying to reidentify Mammoth PostgreSQL as: PostgreSQL + Replication Because, except for the compression features the only difference between Mammoth and PostgreSQL proper is the replication. The compression features are being prepped to be push to patches for review for 8.1. Therefore they are afraid of having > one vendor tell them that the problem exists with another third-party > product. This reaction that add-ons are somewhat inferior to solutions > offered from the same source then leads people to say "PostgreSQL > doesn't have replication!" MS SQL has at least 2 forms of replication, > PostgreSQL doesn't have any in this view. Well I think that part of that is that PostgreSQL doesn't say it has replication, when it clearly does. A look at the features list: http://www.postgresql.org/users-lounge/features.html Shows no such mention of replication. It should plain as day say: Replication: Available via Slony-I, and Replicator (or any others we see fit). Considering that google brings that page up first if you type in postgresql features... > What we need to do to counter this perception is advertise the fact that > many of these replication systems are developed by core community > members and/or developers, and some of them are quite mature. Some, > like dbmirror, have a history of being included in the source release, > even. We need to advertise these things. Yes, the following projects > had involvement by members of our core team.... The following > projects/products have great commercial support.... The following > products/projects are widely used in the community.... Errr... doh to me, read above ;) Sincerely, Joshua D. Drake > > As a community, obviously our focus will be on open source solutions, > but I have no problem with such a document pointing to commercial > products from time to time in order to show that we as a community > endorse solutions to this problem. (Regarding commercial Win32 ports, I > am not opposed to having a "commercial ports" list available either.) > Yes, this opens a can of worms, but it solves another bigger problem > (perception by potential customers and the press). > > It might not be a bad idea for a collaborative effort to be used to > write a faq solely on replication options available. > > Best Wishes, > Chris Travers > Metatron Technology Consulting > > ---------------------------(end of broadcast)--------------------------- > TIP 7: don't forget to increase your free space map settings -- Command Prompt, Inc., home of PostgreSQL Replication, and plPHP. Postgresql support, programming shared hosting and dedicated hosting. +1-503-667-4564 - jd@commandprompt.com - http://www.commandprompt.com Mammoth PostgreSQL Replicator. Integrated Replication for PostgreSQL
Attachment
On Wed, Nov 10, 2004 at 10:34:57AM -0800, Chris Travers wrote:
> internalize well enough to talk about. It is, IMO, the fact that these
> other systems (MS SQL, Oracle, etc) have replication systems through the
> same vendor as the software itself.
I have two, inconsistent responses to this line.
1. Free software just _isn't_ commercial software. Companies
that are going to approach free software like commercial software are
going to fail in their use of it. (In my opinion, most cases of
using commercial software according to the commercial
license-and-support model are _also_ failures, but I'll spare y'all
that rant today.) You simply need to understand how the community
works if you're going to get any sort of reasonable experience with
Postgres, or *BSD, or Linux, or even gnumeric. If you expect the
sort of experience you have when running Windows or Oracle, then
you're doomed. This is an advantage, but also a disadvantage. That
I can ask the developer who wrote a bit of code why it works the way
it does is great, but it also imposes on me a requirement that I have
something approaching a clue when I ask, lest I get a pretty frosty
reception. The point of customer service in a tech company is, after
all, not only to avoid giving the customer the frosty reception, but
also to protect the actual techies from endless series of
didn't-read-the-fine-manual questions.
2. If it's that important, then there's money to be made in the
ISV role, and someone should (continue to -- some people already are)
do it.
A
--
Andrew Sullivan | ajs@crankycanuck.ca
In the future this spectacle of the middle classes shocking the avant-
garde will probably become the textbook definition of Postmodernism.
--Brad Holland
> I have two, inconsistent responses to this line.
>
> 1. Free software just _isn't_ commercial software.
I don't agree with this. Free software is an ideology and has nothing
to do with wether or not it is commercial. You can have commercial
Free Software. Red Hat is commercial Free Software as is SuSE or JBoss.
Commerical:
Of or relating to Commerce.
It has nothing to do with wether or not their is a fix product cost or
not. With free software, the product cost is wrapped into the support
and services.
Sincerely,
Joshua D. Drake
Companies
> that are going to approach free software like commercial software are
> going to fail in their use of it. (In my opinion, most cases of
> using commercial software according to the commercial
> license-and-support model are _also_ failures, but I'll spare y'all
> that rant today.) You simply need to understand how the community
> works if you're going to get any sort of reasonable experience with
> Postgres, or *BSD, or Linux, or even gnumeric. If you expect the
> sort of experience you have when running Windows or Oracle, then
> you're doomed. This is an advantage, but also a disadvantage. That
> I can ask the developer who wrote a bit of code why it works the way
> it does is great, but it also imposes on me a requirement that I have
> something approaching a clue when I ask, lest I get a pretty frosty
> reception. The point of customer service in a tech company is, after
> all, not only to avoid giving the customer the frosty reception, but
> also to protect the actual techies from endless series of
> didn't-read-the-fine-manual questions.
>
> 2. If it's that important, then there's money to be made in the
> ISV role, and someone should (continue to -- some people already are)
> do it.
>
> A
>
--
Command Prompt, Inc., home of PostgreSQL Replication, and plPHP.
Postgresql support, programming shared hosting and dedicated hosting.
+1-503-667-4564 - jd@commandprompt.com - http://www.commandprompt.com
Mammoth PostgreSQL Replicator. Integrated Replication for PostgreSQL
Attachment
On Wed, Nov 10, 2004 at 11:03:10AM -0800, Joshua D. Drake wrote:
> I don't agree with this. Free software is an ideology and has nothing
> to do with wether or not it is commercial. You can have commercial
> Free Software. Red Hat is commercial Free Software as is SuSE or JBoss.
But my experience as a consumer of all of those very products has
not been what my experience of real commercial products has been.
And as I say, that cuts both ways. (I agree that the terminology is
probably considerably less good. Something more neutral would be
helpful. I don't find "Open Source" any more neutral, BTW, mostly
because of the bombs that get thrown over these terms.)
A
--
Andrew Sullivan | ajs@crankycanuck.ca
The fact that technology doesn't work is no bar to success in the marketplace.
--Philip Greenspun
On 11/10/2004 12:02 AM, Christopher Browne wrote: > Martha Stewart called it a Good Thing when peter_e@gmx.net (Peter Eisentraut) wrote: >> Thomas Hallgren wrote: >>> Master + read-only slaves: >>> - Slony-I when all sites are trusted >>> - dbMirror for untrusted slaves and/or table based master slave >>> assignment - Mammoth Replicator, proprietary ??? >>> - erServer ??? >> >> That begs the question in turn why there are so many master/slave >> replication solutions. I mean, I don't care, but this >> categorization doesn't really answer the original question. > > I think there are multiple answers because of the combination of: > > a) Coding before thinking, where some of the systems have been > "hacked together" without too much forethought; > > b) Greatly varying implementation strategies. > > For instance, one of the big problems we encountered with eRServer was > in its use of memory. The "snapshot" notion it uses tends to lead to > fairly spectacular RAM consumption. Had it gotten more design effort, > earlier on, perhaps it could have been more modest in memory usage. > That became one of the design requirements for Slony-I... To be fair, that isn't really a fault of the initial design. The way it was redone in JAVA moved the temporary files used by Vadim's Perl stuff into the RAM. If the JAVA version would be able to spill to disk it would be no issue at all. > > Part of the history has been that people in a rush to get some form of > replication working looked at different "parts of the elephant," and, > seeing different things, implemented different things. > > If a system can get a bit more "thoughtfulness" applied to it, it may > well become forcibly preferable to the other options. > >>> Multi-master: >>> - C-JDBC, Will be transaction safe once PostgreSQL has XA >>> - pgPool, not transaction safe ??? >> >> These are not multimaster solutions in the sense that you can write >> to any one of multiple hosts. In a sense, they are really >> master/slave solutions with the program components distributed >> differently. To write, you always have to go through one host. > > The pgpool approach is somewhat ambiguous, but you're probably right. Pgpool is no multimaster. It is an approach of query duplication done by a connection proxy, which has next to no tolerance for a single server aborting a transaction during commit. You would lose that node and have to rebuild it from scratch during a downtime of the whole cluster. The only "working" multimaster system I have seen so far is pgcluster. I don't know if Mitani would be happy with us advertising it at this time though. > > My expectation of a "multimaster" system is that I should be able to > fire an update at any of the 'masters' and expect it to propagate to > the rest of the databases more or less automatically. That's not what > either XA or pgpool do. ... that among other things. I also expect that I can get a crashed or otherwise lost node back into the group without interruption. And that I can add new nodes from scratch without that as well. And, and, and. The reason why we have so many replication solutions is that most of them look like they have been developed like "let's not worry about all the problems now ... get some data replicating first". And that is almost a guarantee never to get all those "other features" implemented at all, since the choices made early on don't inherit the capabilities if you haven't considered them. Jan -- #======================================================================# # It's easier to get forgiveness for being wrong than for being right. # # Let's break this rule - forgive me. # #================================================== JanWieck@Yahoo.com #
On Tue, 2004-11-09 at 21:35, Andrew Sullivan wrote: > On Tue, Nov 09, 2004 at 09:28:12PM +0000, Simon Riggs wrote: > > Externally, everybody thinks that there should be just one, just like > > there is for other databases. > > I guess it's this thing that I want to understand. Why do people > believe that? I try not to ask "why" people believe what they do. It's usually makes no sense. > Because other databases, where "other" are "the ones > I'd actually run important systems on" _don't_ have just one. OK. I guess the big O have RAC, Data Guard, ASD, Advanced Replication, Streams... The perception is there though: O Replication Perhaps we should refer to PostgreSQL methods like this... HA Slony Replication eRserver Log Shipping PITR Load Balancing pgpool That would do it-ish -- Best Regards, Simon Riggs
On 11/9/2004 2:39 PM, Roderick A. Anderson wrote: > I prefer a full size, 2 wheel drive, Chevy ... Oh darn wrong question. > :-) Having a 4 wheel drive let's you get stuck in more inaccessable places. :-) Jan -- #======================================================================# # It's easier to get forgiveness for being wrong than for being right. # # Let's break this rule - forgive me. # #================================================== JanWieck@Yahoo.com #
Jan Wieck wrote: > On 11/9/2004 2:39 PM, Roderick A. Anderson wrote: > >> I prefer a full size, 2 wheel drive, Chevy ... Oh darn wrong >> question. :-) > > > Having a 4 wheel drive let's you get stuck in more inaccessable places. That is definately the truth :) > > :-) > > > Jan > -- Command Prompt, Inc., home of PostgreSQL Replication, and plPHP. Postgresql support, programming shared hosting and dedicated hosting. +1-503-667-4564 - jd@commandprompt.com - http://www.commandprompt.com Mammoth PostgreSQL Replicator. Integrated Replication for PostgreSQL
Attachment
> Perhaps we should refer to PostgreSQL methods like this... > HA Slony > Replication eRserver > Log Shipping PITR > Load Balancing pgpool > > That would do it-ish Where would clusgres by LinuxLabs fit in? http://www.linuxlabs.com/clusgres.html It's commercial and only runs on their specially designed hardware, but it might be worth a mention. Merlin
Merlin, > Where would clusgres by LinuxLabs fit in? In "Vaporware", unless you know something I don't. -- Josh Berkus Aglio Database Solutions San Francisco
Jan, > To be fair, that isn't really a fault of the initial design. The way it > was redone in JAVA moved the temporary files used by Vadim's Perl stuff > into the RAM. If the JAVA version would be able to spill to disk it > would be no issue at all. > I'm sure you didn't capitalize Java to indicate that it in some way incured this design flaw ;-) Just kidding... Regards, Thomas Hallgren
On 11/11/2004 12:14 PM, Joshua D. Drake wrote: > Jan Wieck wrote: >> On 11/9/2004 2:39 PM, Roderick A. Anderson wrote: >> >>> I prefer a full size, 2 wheel drive, Chevy ... Oh darn wrong >>> question. :-) >> >> >> Having a 4 wheel drive let's you get stuck in more inaccessable places. > > That is definately the truth :) The best one I had (so far) was around Xmas time, in a remote State Park in Maine, couple miles from the next village and out of cellphone reception. Nobody knew we were out there and the sun was approaching the horizon. Took a scary moment to get the Jeep rolling on snow again. Real Jeepers allways have a shovel in the trunk. Jan -- #======================================================================# # It's easier to get forgiveness for being wrong than for being right. # # Let's break this rule - forgive me. # #================================================== JanWieck@Yahoo.com #
On November 10, 2004 05:55 pm, Jan Wieck wrote: > On 11/9/2004 2:39 PM, Roderick A. Anderson wrote: > > I prefer a full size, 2 wheel drive, Chevy ... Oh darn wrong question. > > > > :-) > > Having a 4 wheel drive let's you get stuck in more inaccessable places. Actually I've had a lot of fun watching trendy tourists with their 4 wheel drive SUV's spin out of control on icy mountain roads because they overestimate what their $50,000 vehicles can do. -- Robert Bernier PostgreSQL Business Intelligence Analyst, SRA AMERICA (Formerly of One WTC) Consulting, PostgreSQL Services & PowerGres on Windows One Penn Plaza, Suite 1910 New York, NY 10119 Tel: 212.244.8833
On November 10, 2004 05:55 pm, Jan Wieck wrote: > On 11/9/2004 2:39 PM, Roderick A. Anderson wrote: > > I prefer a full size, 2 wheel drive, Chevy ... Oh darn wrong question. > > > > :-) > > Having a 4 wheel drive let's you get stuck in more inaccessable places. Actually I've had a lot of fun watching trendy tourists with their 4 wheel drive SUV's spin out of control on icy mountainroads because they overestimate what their $50,000 vehicles can do. -- Robert Bernier PostgreSQL Business Intelligence Analyst, SRA AMERICA (Formerly of One WTC) Consulting, PostgreSQL Services & PowerGres on Windows One Penn Plaza, Suite 1910 New York, NY 10119 Tel: 212.244.8833
On 11/11/2004 2:22 PM, Thomas Hallgren wrote: > Jan, >> To be fair, that isn't really a fault of the initial design. The way it >> was redone in JAVA moved the temporary files used by Vadim's Perl stuff >> into the RAM. If the JAVA version would be able to spill to disk it >> would be no issue at all. >> > I'm sure you didn't capitalize Java to indicate that it in some way > incured this design flaw ;-) > > Just kidding... It is never the fault of a programming language per se. People with a good understanding of object design will write "object oriented code" in every language, even assembler. People who just don't know what they are doing will write bad code, and the best Pascal compiler in the world won't be able to prevent that. Jan -- #======================================================================# # It's easier to get forgiveness for being wrong than for being right. # # Let's break this rule - forgive me. # #================================================== JanWieck@Yahoo.com #
> > It is never the fault of a programming language per se. People with a > good understanding of object design will write "object oriented code" > in every language, even assembler. People who just don't know what > they are doing will write bad code, and the best Pascal compiler in > the world won't be able to prevent that. Yes but I believe even you would agree that their are programming languages that are better for certain tasks than others. The use of java as a replication engine for PostgreSQL seems, well... incorrect. Sincerely, Joshua D. Drake > > > Jan > -- Command Prompt, Inc., home of Mammoth PostgreSQL - S/ODBC and S/JDBC Postgresql support, programming shared hosting and dedicated hosting. +1-503-667-4564 - jd@commandprompt.com - http://www.commandprompt.com PostgreSQL Replicator -- production quality replication for PostgreSQL
Attachment
On Sat, 13 Nov 2004, Joshua D. Drake wrote: > >> >> It is never the fault of a programming language per se. People with a good >> understanding of object design will write "object oriented code" in every >> language, even assembler. People who just don't know what they are doing >> will write bad code, and the best Pascal compiler in the world won't be >> able to prevent that. > > Yes but I believe even you would agree that their are programming languages > that are better for certain tasks than others. The use of java as a > replication engine for PostgreSQL seems, well... incorrect. We definitely concur with that, which is why we are re-writing it ... going to Java, as Andrew has mentioned, was *not* a design decision that we made, but was made for us :( ---- Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email: scrappy@hub.org Yahoo!: yscrappy ICQ: 7615664
Joshua D. Drake wrote: > Yes but I believe even you would agree that their are programming > languages that are better for certain tasks than others. The use of > java as a replication engine for PostgreSQL seems, well... incorrect. Marc G. Fournier wrote: > We definitely concur with that, which is why we are re-writing it ... > going to Java, as Andrew has mentioned, was *not* a design decision that > we made, but was made for us :( > Now I get really curious. Why would Java be a bad choice for a replication engine? I would consider it an excellent choice, provided of course that the people tasked with the implementation had the right skills. C-JDBC for instance, is written in Java. Regards, Thomas Hallgren
On Sat, 13 Nov 2004, Thomas Hallgren wrote: > Joshua D. Drake wrote: >> Yes but I believe even you would agree that their are programming >> languages that are better for certain tasks than others. The use of >> java as a replication engine for PostgreSQL seems, well... incorrect. > > Marc G. Fournier wrote: >> We definitely concur with that, which is why we are re-writing it ... >> going to Java, as Andrew has mentioned, was *not* a design decision that >> we made, but was made for us :( >> > Now I get really curious. Why would Java be a bad choice for a replication > engine? I would consider it an excellent choice, provided of course that the > people tasked with the implementation had the right skills. C-JDBC for > instance, is written in Java. Everyone obviously has their opinion, but in mine, Java just has toooooo large of a memory foot print ... I don't know enough about Java to know if this is something that is restricted to how eRServer/Java was coded or not, but by default, the damn thing takes something like 300Mb of RAM for just the engine :( ---- Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email: scrappy@hub.org Yahoo!: yscrappy ICQ: 7615664
On 11/13/2004 12:06 PM, Joshua D. Drake wrote: >> >> It is never the fault of a programming language per se. People with a >> good understanding of object design will write "object oriented code" >> in every language, even assembler. People who just don't know what >> they are doing will write bad code, and the best Pascal compiler in >> the world won't be able to prevent that. > > Yes but I believe even you would agree that their are programming > languages that are better for certain tasks than others. The use of java > as a replication engine for PostgreSQL seems, well... incorrect. Mammoth is written in C, the followup for eRServer will be C (++?) and Slony is C ... I guess disagreeing would be, well ... ignorant. Jan -- #======================================================================# # It's easier to get forgiveness for being wrong than for being right. # # Let's break this rule - forgive me. # #================================================== JanWieck@Yahoo.com #
Jan Wieck wrote: > On 11/13/2004 12:06 PM, Joshua D. Drake wrote: > >>> >>> It is never the fault of a programming language per se. People with >>> a good understanding of object design will write "object oriented >>> code" in every language, even assembler. People who just don't know >>> what they are doing will write bad code, and the best Pascal >>> compiler in the world won't be able to prevent that. >> >> >> Yes but I believe even you would agree that their are programming >> languages that are better for certain tasks than others. The use of >> java as a replication engine for PostgreSQL seems, well... incorrect. > > > Mammoth is written in C, the followup for eRServer will be C (++?) and > Slony is C ... I guess disagreeing would be, well ... ignorant. Heh :) > > > Jan > -- Command Prompt, Inc., home of Mammoth PostgreSQL - S/ODBC and S/JDBC Postgresql support, programming shared hosting and dedicated hosting. +1-503-667-4564 - jd@commandprompt.com - http://www.commandprompt.com PostgreSQL Replicator -- production quality replication for PostgreSQL
Attachment
Oops! scrappy@postgresql.org ("Marc G. Fournier") was seen spray-painting on a wall:
> On Sat, 13 Nov 2004, Thomas Hallgren wrote:
>
>> Joshua D. Drake wrote:
>>> Yes but I believe even you would agree that their are programming
>>> languages that are better for certain tasks than others. The use
>>> of java as a replication engine for PostgreSQL seems,
>>> well... incorrect.
>>
>> Marc G. Fournier wrote:
>>> We definitely concur with that, which is why we are re-writing it
>>> ... going to Java, as Andrew has mentioned, was *not* a design
>>> decision that we made, but was made for us :(
>>>
>> Now I get really curious. Why would Java be a bad choice for a
>> replication engine? I would consider it an excellent choice,
>> provided of course that the people tasked with the implementation
>> had the right skills. C-JDBC for instance, is written in Java.
>
> Everyone obviously has their opinion, but in mine, Java just has
> toooooo large of a memory foot print ... I don't know enough about
> Java to know if this is something that is restricted to how
> eRServer/Java was coded or not, but by default, the damn thing takes
> something like 300Mb of RAM for just the engine :(
The problem with Java is twofold:
1. Naive system implementations wind up gratuitously using a lot of
memory.
2. The garbage collection system makes it particularly difficult to
be aware of how the "memory life cycle" works. Which helps keep
developers naive for somewhat longer...
In the case of eRServer, the way the snapshot system was constructed
led to "gratuitous memory use," and that's not an obvious result of
either 1. or 2.
Someone could have made a C-based version of ERS that, by using
similar implementation strategies, would also use "gratuitously large"
amounts of memory.
In contrast, Slony-I happens to be _immensely_ more frugal in its use
of memory. That is a matter of design, not of the language used. The
"strategy" involves loading into memory only the "buffering" (more or
less) of the data that is being loaded. If there's a replication set
consisting of 80GB of data, you don't need to hold it all in RAM; you
just need to buffer a few hundred KB of it so that you're streaming
large enough blocks across the network to let the network connections
be used efficiently. If the strategies of Slony-I had been
implemented in Java, the memory footprint would still be relatively
small. The fact that Java has heftier libraries than C means that
Java apps will be somewhat bigger than C ones.
But I wouldn't raise any "red flags" if a "Slony-Java" process
consumed 25MB whilst the C version only consumed 8MB. Those are both
small enough sizes that they're not going to challenge JVM maximum
memory sizes. On a couple occasions, I saw eRServer "blow up" due to
the JVM not being configured to have enough memory space, and could
foresee situations where you couldn't set memory space high enough
:-(.
I'd expect a C++-based system to fall somewhere in between. Between
exception handling, templates, and such, C++ adds a bit of "gratuitous
bloat," but not quite so much as in Java. (Unless you use STL Way
Lots, but that's another story :-).)
But in all of this, the things that cause the _real_ bloat are
pessimal algorithmic design choices. The things to _fix_ bloat are
algorithmic changes, not changes of language.
The things to "hate" about Java aren't about any of this. It's more
like:
- Java runs, in a "supportable" manner, on way fewer platforms than
PostgreSQL
- If you pick libraries that are functional enough to be useful,
then you likely have to get a Sun JDK with pretty proprietary
licensing
- Due to licensing complexities, it's WAY more complex to deploy
Java-based apps than C-based apps. The average Linux or BSD
distribution contains hundreds if not thousands of apps
deployed in C; doing the same for Java has proved more than
troublesome.
--
output = reverse("gro.mca" "@" "enworbbc")
http://www.ntlug.org/~cbbrowne/linux.html
"Using Java as a general purpose application development language is
like going big game hunting armed with Nerf weapons."
-- Author Unknown
Quoth JanWieck@Yahoo.com (Jan Wieck): > On 11/13/2004 12:06 PM, Joshua D. Drake wrote: > >>> >>> It is never the fault of a programming language per se. People with >>> a good understanding of object design will write "object oriented >>> code" in every language, even assembler. People who just don't know >>> what they are doing will write bad code, and the best Pascal >>> compiler in the world won't be able to prevent that. >> Yes but I believe even you would agree that their are programming >> languages that are better for certain tasks than others. The use of >> java as a replication engine for PostgreSQL seems, well... incorrect. > > Mammoth is written in C, the followup for eRServer will be C (++?) and > Slony is C ... I guess disagreeing would be, well ... ignorant. Sure, but I seem to recall that your Slony-I prototype was initially in Tcl. There may be a Perl-based prototype of one of the new bits, and if bottlenecks aren't evident, I'm not convinced everything has to stay in C. I don't think I'd propose Ada (Andrew Sullivan would be aghast! ;-)), but I could see Perl or Python being reasonable languages for handling processes where the _real_ bottlenecks lie in database access. -- "cbbrowne","@","linuxfinances.info" http://www3.sympatico.ca/cbbrowne/x.html "If Ada became the hot, in-language you would see a lot more bad code in Ada." -- Thaddeus L. Olczyk <olczyk@interaccess.com>, comp.lang.C++
Christopher, Thanks for clearing a few things up. I think what you wrote made a lot o sense. Not surprisingly, I have some opinions concerning the things to "hate" :-) > The problem with Java is twofold: > > 1. Naive system implementations wind up gratuitously using a lot of > memory. > > 2. The garbage collection system makes it particularly difficult to > be aware of how the "memory life cycle" works. Which helps keep > developers naive for somewhat longer... > > In the case of eRServer, the way the snapshot system was constructed > led to "gratuitous memory use," and that's not an obvious result of > either 1. or 2. > I agree with this. That's why I included "the right skills" in my original claim. It's a known fact that Java get's bashed a lot because it gives you freedom under responsibility and people tend to forget the latter. > The things to "hate" about Java aren't about any of this. It's more > like: > > - Java runs, in a "supportable" manner, on way fewer platforms than > PostgreSQL > An argument that holds true in theory. I wonder what percentage of potential replication users that would be lost in real life due to portability issues when moving to Java. My guess is zero or perhaps fragments of a percent. I seriously belive that the loss contributed to "religion" would be greater. On the Java plus side, you can distribute one runnable binary for all platforms where it *does* run as opposed to source that requires the user to have a complete build environment. So perhaps this actually works in Java's favor. > - If you pick libraries that are functional enough to be useful, > then you likely have to get a Sun JDK with pretty proprietary > licensing > Here I disagree with some emphasis. The JRE in itself contains far more useful libraries than any C/C++ compiler package that I'm aware of. And if you want to complement what you have, go to Sourceforge, Apache, or any other Open Source site where a lot of very useful packages can be found. Many of them with production quality. The JDK/JRE licensing in itself has never been a problem in any projects where I have been involved, nor any other Java project that I'm aware of. You just don't bundle the JRE, you assume that the customer has it installed. > - Due to licensing complexities, it's WAY more complex to deploy > Java-based apps than C-based apps. The average Linux or BSD > distribution contains hundreds if not thousands of apps > deployed in C; doing the same for Java has proved more than > troublesome. > Funny, I've been writing Java apps for the better part of 6 years now. I've *never* experienced any licensing complexities *what so ever*. Many thousand users use Java on Linux and FreeBSD and they are not violating any licenses. Can you please explain where and how you see license problems pop up? All and all, I think the licensing question is non existent for people who want to provide utilities written in Java. Sourceforge today counts 13192 Java projects which is very close to the number of C (13762) and C++ (14066) projects. A vast majority (95%) of those projects are OSI-Approved Open Source. I have no doubt that Java will be #1 by this time next year. On the Apache site, you'll find other really useful (and free) Java utilities. In other words, if licensing was a problem you wouldn't see the Java community expanding the way it does today, using free Open Source as a primary vehicle. Regards, Thomas Hallgren
On 11/14/2004 2:11 AM, Christopher Browne wrote: > Quoth JanWieck@Yahoo.com (Jan Wieck): >> On 11/13/2004 12:06 PM, Joshua D. Drake wrote: >> >>>> >>>> It is never the fault of a programming language per se. People with >>>> a good understanding of object design will write "object oriented >>>> code" in every language, even assembler. People who just don't know >>>> what they are doing will write bad code, and the best Pascal >>>> compiler in the world won't be able to prevent that. >>> Yes but I believe even you would agree that their are programming >>> languages that are better for certain tasks than others. The use of >>> java as a replication engine for PostgreSQL seems, well... incorrect. >> >> Mammoth is written in C, the followup for eRServer will be C (++?) and >> Slony is C ... I guess disagreeing would be, well ... ignorant. > > Sure, but I seem to recall that your Slony-I prototype was initially > in Tcl. There may be a Perl-based prototype of one of the new bits, > and if bottlenecks aren't evident, I'm not convinced everything has to > stay in C. I am sure with the right skills one could have done Slony-I in Java and get pretty much the same results. And note that Slony-I isn't a 100% C solution anyway, as we use PL/pgSQL extensively - although not in any time critical path. The problem really is that it is rather difficult to find Java programmers with the skillset required here. I have met a good deal of Java "fast-thinkers", and except for one (and you know who I mean) ALL of them are usually nice, but completely incompetent guy's who copy binary jar's all over their projects, create classes by the dozen but literally zero code that actually "does" something, and simply fail to understand why the timestamp they take in their Java code has absolutely zero to do with the serializable transaction order. IMNSVHO Java is a nice language for GUIs and maybe some WEB stuff ... and that's it. Jan -- #======================================================================# # It's easier to get forgiveness for being wrong than for being right. # # Let's break this rule - forgive me. # #================================================== JanWieck@Yahoo.com #
After a long battle with technology, thhal@mailblocks.com (Thomas Hallgren), an earthling, wrote:
> The JDK/JRE licensing in itself has never been a problem in any
> projects where I have been involved, nor any other Java project that
> I'm aware of. You just don't bundle the JRE, you assume that the
> customer has it installed.
Right, you have to "assume that the customer has it installed."
>> - Due to licensing complexities, it's WAY more complex to deploy
>> Java-based apps than C-based apps. The average Linux or BSD
>> distribution contains hundreds if not thousands of apps
>> deployed in C; doing the same for Java has proved more than
>> troublesome.
> Funny, I've been writing Java apps for the better part of 6 years
> now. I've *never* experienced any licensing complexities *what so
> ever*. Many thousand users use Java on Linux and FreeBSD and they
> are not violating any licenses.
Tell me how many Linux distributions come with Java(tm) and Java(tm)
applications installed by default. :-)
The answer is pretty clear; the licensing of Blackdown mandates that
installation of Java(tm) be done separately, which _severely_ limits
the incidence of this. I see, on Debian, a few dependancies on some
form of JDK/JRE; it's generally staying with Java 1.1 where there are
feasible if limited "free software" options...
Which means that systems cannot and do not include either Java(tm)
(or, as a result, Java(tm) applications). For them to do so would
indeed require violating Sun's licenses.
This doesn't prevent adding Java(tm) as an "aftermarket add-on." But
it is clearly that, an "aftermarket add-on." Java just isn't a "first
class citizen" that can be considered ubiquitous the way C and C++
are.
--
wm(X,Y):-write(X),write('@'),write(Y). wm('cbbrowne','linuxfinances.info').
http://www3.sympatico.ca/cbbrowne/languages.html
The cigars in Los Angeles that were Duchamp-signed and then smoked.
On Sat, Nov 13, 2004 at 01:45:50PM -0400, Marc G. Fournier wrote:
> We definitely concur with that, which is why we are re-writing it ...
> going to Java, as Andrew has mentioned, was *not* a design decision that
> we made, but was made for us :(
I didn't, however, intend that to be a proof that Java was the wrong
tool for replication. It was, however, the wrong tool used the wrong
way for this job.
There was a thread on the erserver list about why the implementation
sucked as much as it did in certain places. But to give folks a
flavour of it, the original Java porter ("OJP") was one of those
big-design guys. So he had an apparent plan for an error-reporting
abstraction layer. Problem was, he never wrote it. So every raised
exception in the program caused the error handling stub to be called,
which simply spit out an exception. The net result was that you
_always_ got the error-handling stub line number in every exception.
Bugs? Hey, grovel through the code a line at a time and see if you
can figure out why it died. It's edumacashunal.
I'm with Jan on this: poor programming can be done in any language at
all.
A
--
Andrew Sullivan | ajs@crankycanuck.ca
The plural of anecdote is not data.
--Roger Brinner
On Sun, Nov 14, 2004 at 02:05:21AM -0500, Christopher Browne wrote:
>
> In the case of eRServer, the way the snapshot system was constructed
> led to "gratuitous memory use," and that's not an obvious result of
> either 1. or 2.
Something else that's worth noting: erserver was designed against
PostgreSQL 7.1. So it had a number of limitations it was working
with that we, in happier later days, don't have to worry about any
more.
A
--
Andrew Sullivan | ajs@crankycanuck.ca
In the future this spectacle of the middle classes shocking the avant-
garde will probably become the textbook definition of Postmodernism.
--Brad Holland
On Sun, Nov 14, 2004 at 02:11:48AM -0500, Christopher Browne wrote:
> I don't think I'd propose Ada (Andrew Sullivan would be aghast! ;-)),
> but I could see Perl or Python being reasonable languages for handling
> processes where the _real_ bottlenecks lie in database access.
Having been around at the time of the erserver Java decision, and
having lost the argument that translating it to Java was a waste of
time, I can tell you that erserver _was_ originally written in Perl
and, frankly, worked better as a prototype than as a final product.
But this brings us back to Jan's earlier point, which is generally
summed up in the maxim, "It's a poor workman who blames his tools."
The decision in favour of Java, BTW, is another bit of evidence that
threads are not some sort of magic seasoning you can sprinkle on
something and make it better. The reason for picking Java was the
ease with which it could be multi-threaded. Unfortunately, the
actual implementation essentially used multiple threads to build up a
big-giant snapshot which had to be applied serially to the slave.
This approach is also responsible for the snapshot-size limitations
built into the program.
I should note, for the record, that most of these problems were
subsequently addressed, although they mostly did exist in the
original BSD erserver. (I'll also note that we're pretty far off
-advocacy charter, and so this is the last on this topic I'll say.)
A
--
Andrew Sullivan | ajs@crankycanuck.ca
The fact that technology doesn't work is no bar to success in the marketplace.
--Philip Greenspun
Christopher Browne wrote: > This doesn't prevent adding Java(tm) as an "aftermarket add-on." But > it is clearly that, an "aftermarket add-on." Java just isn't a "first > class citizen" that can be considered ubiquitous the way C and C++ > are. > Ah, so all licensing issues that you had, boils down to this one single thing. A JRE needs to be installed on the customer site. Yeah. That sounds WAY to complex. Regards, Thomas Hallgren
Jan Wieck wrote: > IMNSVHO Java is a nice language for GUIs and maybe some WEB stuff ... > and that's it. Right. I want enforce this inviting you all to made an experiment. Ask your programmers why is insane in java produce code like this: Vector v = new Vector(); v.add( new MyClass() ); <==== Thousand entries most of programmers reply: nothing. Unfortunately Java like other language "Garbage collector driven" are producing around lot of programmers producing garbage code. Regards Gaetano Mendola
Christopher Browne wrote: > I'd expect a C++-based system to fall somewhere in between. Between > exception handling, templates, and such, C++ adds a bit of "gratuitous > bloat," but not quite so much as in Java. (Unless you use STL Way > Lots, but that's another story :-).) The only think that STL bloat is the "code" and not always ( like all templates ). Having say that I largelly accept a code bloath of few hundreads bytes instead of reinvent the well each time you need a list, or you shall implement a sort. I'm sad to see however people talking or writing about C++ performances or memory requirements without even have write a single line of C++ code. Alway in situation like this I to read a couple of book or article about templates: "Modern C++ Design: Generic Programming and Design Patterns Applied " and most of all the capabilities of this library: http://boost-consulting.com/boost/libs/multi_index/doc/index.html May be I had an over reaction ( excuse me ) but this week is not the first ( also out of this list ) talking against C++. Regards Gaetano Mendola
Jan Wieck wrote: > I am sure with the right skills one could have done Slony-I in Java and > get pretty much the same results... > > The problem really is that it is rather difficult to find Java > programmers with the skillset required here... > > IMNSVHO Java is a nice language for GUIs and maybe some WEB stuff ... > and that's it... > Isn't the last claim a bit contradicory? If you where to say, "Java is a nice language, but we can't use it for advanced server side development here since we are unable to find people with the right skills", then it would make a lot of sense. And I would agree, that's often a very valid argument to use C/C++. Regards, Thomas Hallgren
> Isn't the last claim a bit contradicory? If you where to say, "Java is a > nice language, but we can't use it for advanced server side development > here since we are unable to find people with the right skills", then it > would make a lot of sense. And I would agree, that's often a very valid > argument to use C/C++. Everyone does understand here, that we aren't going to agree right? IMHO Java is worthless but does it matter? People code in what they are either forced to, or are comfortable with. Sincerely, Joshua D. Drake > > Regards, > Thomas Hallgren > > > ---------------------------(end of broadcast)--------------------------- > TIP 4: Don't 'kill -9' the postmaster -- Command Prompt, Inc., home of PostgreSQL Replication, and plPHP. Postgresql support, programming shared hosting and dedicated hosting. +1-503-667-4564 - jd@commandprompt.com - http://www.commandprompt.com Mammoth PostgreSQL Replicator. Integrated Replication for PostgreSQL
Attachment
Joshua, > Everyone does understand here, that we aren't going to agree right? > > IMHO Java is worthless but does it matter? People code in what they > are either forced to, or are comfortable with. Agree. Everyone has their preferences and opinions. And perhaps you'd agree that it's better to find the true cause of a problem then to put the blame on a language, no matter what preferences one may have. If you do, well, then we do agree after all :-) Regards, Thomas Hallgren
On Monday 15 November 2004 01:29, Thomas Hallgren wrote:
> Christopher Browne wrote:
> > This doesn't prevent adding Java(tm) as an "aftermarket add-on." But
> > it is clearly that, an "aftermarket add-on." Java just isn't a "first
> > class citizen" that can be considered ubiquitous the way C and C++
> > are.
>
> Ah, so all licensing issues that you had, boils down to this one single
> thing.
>
> A JRE needs to be installed on the customer site.
>
> Yeah. That sounds WAY to complex.
>
It's funny how I can build umpteen packages from C code on my system, but if I
want to play around with runescape.com my lack of java support (which I would
think I have in a straightofrward fc2 machine) turns out to be a showstopper.
--
Robert Treat
Build A Brighter Lamp :: Linux Apache {middleware} PostgreSQL
Folks, Can we kill this thread, please? Or does someone want to start a "vi vs. emacs" flamewar? -- Josh Berkus Aglio Database Solutions San Francisco
On Wednesday 17 November 2004 00:40, Josh Berkus wrote:
> Folks,
>
> Can we kill this thread, please?
>
> Or does someone want to start a "vi vs. emacs" flamewar?
No one here is silly enough to use emacs ;-)
--
Robert Treat
Build A Brighter Lamp :: Linux Apache {middleware} PostgreSQL
On Wed, 17 Nov 2004, Robert Treat wrote: > On Wednesday 17 November 2004 00:40, Josh Berkus wrote: >> Folks, >> >> Can we kill this thread, please? >> >> Or does someone want to start a "vi vs. emacs" flamewar? > > No one here is silly enough to use emacs ;-) I used to ... fantastic little editor that is, when you have all the right keybindings :) ---- Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email: scrappy@hub.org Yahoo!: yscrappy ICQ: 7615664
>> >> No one here is silly enough to use emacs ;-) > > > I used to ... fantastic little editor that is, when you have all the > right keybindings :) Yep, nothing like running an operating system on an operating system just so you can edit a text file ;) > > ---- > Marc G. Fournier Hub.Org Networking Services > (http://www.hub.org) > Email: scrappy@hub.org Yahoo!: yscrappy ICQ: > 7615664 > > ---------------------------(end of broadcast)--------------------------- > TIP 5: Have you checked our extensive FAQ? > > http://www.postgresql.org/docs/faqs/FAQ.html -- Command Prompt, Inc., home of Mammoth PostgreSQL - S/ODBC and S/JDBC Postgresql support, programming shared hosting and dedicated hosting. +1-503-667-4564 - jd@commandprompt.com - http://www.commandprompt.com PostgreSQL Replicator -- production quality replication for PostgreSQL
Attachment
Since the history on this was cut off, I would just like to remind everyone that it was Josh Berkus' idea to start a thread on emacs vs vi (altho, of course, he did suggest it "tongue in cheek" ... the rest of us just had to jump onto the bandwagon *grin*) ... On Wed, 17 Nov 2004, Joshua D. Drake wrote: > >>> >>> No one here is silly enough to use emacs ;-) >> >> >> I used to ... fantastic little editor that is, when you have all the right >> keybindings :) > > Yep, nothing like running an operating system on an operating system just so > you can edit > a text file ;) Well, in my case, it was for C programming ... but you are right, it is "an operating system ontop of an operating system" ... it was good pre stuff like 'screen', since you could do multi-windows, cut-n-paste between them, compiling without exiting the editor ... one thing that it did do nicely was that if you compiled inside of the editor, the 'errors' linked to the actual code itself, so you could jump right to it ... Between screen and X though, vi uses less resources, and other then the 'jumping to the buggy code' feature, the rest can be easily duplicated ... ---- Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email: scrappy@hub.org Yahoo!: yscrappy ICQ: 7615664
Marc G. Fournier wrote: > > Since the history on this was cut off, I would just like to remind > everyone that it was Josh Berkus' idea to start a thread on emacs vs vi > (altho, of course, he did suggest it "tongue in cheek" ... the rest of > us just had to jump onto the bandwagon *grin*) ... Noted ;) > between them, compiling without exiting the editor ... one thing that it > did do nicely was that if you compiled inside of the editor, the > 'errors' linked to the actual code itself, so you could jump right to it > ... > > Between screen and X though, vi uses less resources, and other then the > 'jumping to the buggy code' feature, the rest can be easily duplicated ... I like joe which has wordstar type key bindings and is modeless. You can do the whole multi-screen,buffer deal as well. I also know vi but will use joe anytime I can :) Sincerely, Joshua D. Drake > > > ---- > Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) > Email: scrappy@hub.org Yahoo!: yscrappy ICQ: 7615664 > > ---------------------------(end of broadcast)--------------------------- > TIP 2: you can get off all lists at once with the unregister command > (send "unregister YourEmailAddressHere" to majordomo@postgresql.org) -- Command Prompt, Inc., home of PostgreSQL Replication, and plPHP. Postgresql support, programming shared hosting and dedicated hosting. +1-503-667-4564 - jd@commandprompt.com - http://www.commandprompt.com Mammoth PostgreSQL Replicator. Integrated Replication for PostgreSQL