Thread: How can we expand PostgreSQL ecosystem?
Hello,
As I said in the previous greeting mail, I'd like to discuss how to expand PostgreSQL ecosystem. Here, ecosystem means "interoperability" -- the software products and cloud services which use/support PostgreSQL. If pgsql-advocacy or somewhere else is better for this topic, just tell me so.
THE BACKGROUND
==================================================
Thanks to the long and hard efforts by the community, PostgreSQL has been evolving to be a really great software comparable to existing strong commercial products. Open source databases are gaining more popularity to influence the database market.
Open source threatens to eat the database market
"Though the proprietary RDBMS market grew at a sluggish 5.4 percent in 2014, the open source database market grew 31 percent to hit $562 million."
"As Gartner highlights in a recent research report, open source databases now consume 25 percent of relational database usage."
Perhaps related to this is that the revenues of Oracle, IBM and Microsoft have been declining (but I read in an article that SQL Server is gaining more revenue).
On the other hand, there is a gulf between the two top popular databases -- Oracle and MySQL -- and PostgreSQL. They are nearly five times more popular than PostgreSQL.
DB-Engines Ranking
Yes, I understand this ranking doesn't necessarily reflect the actual use, but I also don't think the ranking is far from the real popularity. In fact, some surveys show that MySQL has been in more widespread use even here in Japan than PostgreSQL since around 2010 (IIRC).
What should we do to boost the popularity of PostgreSQL? One challenge is to increase the number of software which supports PostgreSQL. To take advantage of the trend of shift from commercial products to open source, PostgreSQL needs to interoperate with many software that are used together with the commercial databases.
The easily understandable target is Oracle, because it is anticipated that more users of Oracle will seek another database to avoid the expensive Oracle Standard Edition 2 and increasing maintenance costs. In addition, PostgreSQL has affinity for Oracle.
However, there is a problem. The number of software is very small that the users can know to interoperate with PostgreSQL. That is, when the users want to migrate from commercial databases to PostgreSQL, they can't get information on whether they can continue to use their assets with PostgreSQL. Many applications might be interoperable through standard interfaces like JDBC/ODBC, but the case is unknown. For example:
* Only 24 open source projects are listed as interoperable.
Open Source Projects Using PostgreSQL
* Even EnterpriseDB has only 12 certified application vendors.
* PostgreSQL Enterprise Consortium lists only about30 related products (Japanese only).
* MySQL touts more than 2,000 ISV/OEM/VARs.
Besides, in practice, we probably should increase the number of software interoperable with PostgreSQL. e.g. one customer asked us whether Arcserve can be used to back up PostgreSQL databases, but unfortunately we had to answer no. They are using Arcserve to back up Oracle databases and other resources. "Then, you can use NetVault instead" is not the best answer; they just want to replace the database.
PROPOSAL
==================================================
Last month, I attended the steering committee of PostgreSQL Enterprise Consortium (PGECons) for the first time and proposed starting the following activity. PGECons is a Japanese non-profit organization to promote PostgreSQL for enterprise use. The members include NTT, SRA OSS (Tatsuo Ishii runs), NEC, Hitachi, HP, Fujitsu, etc. We concluded that we need to consult the PostgreSQL community on how to proceed the activity and work in cooperation with the community.
* Attract and ask product/service vendors to support/use PostgreSQL.
Provide technical assistance to those vendors as an organization so that they can support PostgreSQL smoothly.
If the vendors aren't proactive, we verify the interoperability with their software by executing it.
* Make a directory of software/services that can be used with PostgreSQL on the community web site (wiki.postgresql.org or www.postgresql.org).
Software/services vendors and PostgreSQL developers/users can edit this directory.
This list not only has the names of software and its vendors, but also other information such as the level of interoperability (certified by the vendor, or verified by the community/users) and remarks about configuration, tuning, and whatever useful for users.
PostgreSQL Enterprise Consortium (PGECons)
CONSULTATION AND DISCUSSION
==================================================
I'd like to discuss and hear opinions on how can we expand the ecosystem of PostgreSQL. Example points are:
* How/Where can we get the knowledge of expanding the software ecosystem? Is there any OSS project that we can learn from?
How can we attract software vendors to support PostgreSQL? What words are convincing to appeal the increasing potential of PostgreSQL as a good replacement for commercial databases?
* How can we gain momentum for the activity?
Can we involve globally influential companies like Dell, HPE, and Red Hat?
* Do we need some kind of assistance from a foundation or establish a foundation?
There should be many, many software to address, so the ecosystem activity has to be long-lasting. Plus, money and expertise is necessary.
Would we benefit if we join the Linux Foundation Collaborative Projects?
Linux Foundation Collaborative Projects
* Which software/services in what category should we address preferentially?
What software would many users desire to be interoperable when migrating from commercial databases?
What is the effective way to absorb user requests for this? Is it enough to make a questionnaire like the following? What is the popular questionnaire site which can catch many users (SurveyMonkey?)
Regards
MauMau
On 06/03/16 18:29, MauMau wrote: > As I said in the previous greeting mail, I'd like to discuss how to > expand PostgreSQL ecosystem. Here, ecosystem means "interoperability" > -- the software products and cloud services which use/support > PostgreSQL. If pgsql-advocacy or somewhere else is better for this > topic, just tell me so. For cloud - in particular Openstack (which I am working with ATM), the biggest thing would be: - multi-master replication or failing that: - self managing single master failover (voting/quorum etc) so that operators can essentially 'set and forget'. We currently use Mysql+ Galera (multi master) and Mongodb (self managing single master) and the convenience and simplicity is just so important (Openstack is a huge complex collection of services - hand holding of any one service is pretty much a non starter). regards Mark
On 6 March 2016 at 13:29, MauMau <maumau307@gmail.com> wrote:
However, there is a problem. The number of software is very small that the users can know to interoperate with PostgreSQL.
Yep, even among many big and popular OSS apps.
Many applications might be interoperable through standard interfaces like JDBC/ODBC, but the case is unknown.
Not tons, in my experience, unless they're doing only trivial SQL. Many *are* PostgreSQL-compatible via higher level abstractions like Hibernate/JPA, ActiveRecord, etc, but those are also the systems that are least interesting from a database PoV since they generally use only the lowest common denominator of DB features, with the DB treated as not much more than a dumb row store.
There *is* some cool stuff we could do with ORMs if we'd stop sneering at them for long enough. I'd love to see the ability to omit repeated values on a LEFT JOIN for example, where we project the left tuple completely the first time we see it, i.e. when the key changes, then we emit only the key when we project it again for subsequent right-hand rows. This would save ORMs an immense amount of computational effort and network bandwidth... and you only have to patch each ORM to use it, not each app.
Even more interesting would be to emit JSON as a left-join product, so a typical chained left join to fetch 'customer' with 'account' and 'contact' for each 'account' would emit
[
{
customer_id: '42',
...
accounts: [
{
account_id = 1,
...,
contacts: [
{
contact_id = 4,
...
},
{
contact_id = 5,
...
}
]
},
{
account_id = 2,
...
}
]
},
{
customer_id = 43,
...
}
]
i.e. emit the requested data in the form the application actually wants, without needing to produce a giant relational projection and then process it in the application.
We could help ORMs solve the N+1 SELECTs problem and help them avoid transferring vast join projections unnecessarily. That'd make PostgreSQL pretty compelling for exactly the users we're mostly too busy dismissing to consider.
When you consider the adoption of PostgreSQL in the Rails/ActiveRecord crowd I think there's some interesting potential there. What there isn't is funding AFAIK.
I'd be interested in reaching out to some Hibernate/JPA and ActiveRecord folks about how the DB could help the ORM and have been meaning to explore this area for a while, but -ENOTIME. If anyone pursues it I'll be really interested in hearing how things go.
Besides, in practice, we probably should increase the number of software interoperable with PostgreSQL. e.g. one customer asked us whether Arcserve can be used to back up PostgreSQL databases, but unfortunately we had to answer no. They are using Arcserve to back up Oracle databases and other resources. "Then, you can use NetVault instead" is not the best answer; they just want to replace the database.
The "we" here is the problem. It's not likely to be folks focused on PostgreSQL core dev, but ... who, exactly?
Provide technical assistance to those vendors as an organization so that they can support PostgreSQL smoothly.
This one is a help. That said, pgsql-general is pretty helpful already...
* Make a directory of software/services that can be used with PostgreSQL on the community web site (wiki.postgresql.org or www.postgresql.org).Software/services vendors and PostgreSQL developers/users can edit this directory.
I thought we had that? Yep.
It's pretty invisible though, partly due to the postgresql.org landing page's need for a trim-down and tidy. (Don't even get me started on http://www.postgresql.org/about/ )
* How/Where can we get the knowledge of expanding the software ecosystem? Is there any OSS project that we can learn from?
Mongo.
Despite the real world capabilities being ... patchy ... and the magic autoscaling claims etc being a bit dubious, they've successfully marketed it as the magic solution for everything.
This leads to a fair bit of disillusionment, but also massive adoption. I think they take it too far, but try opening these two pages side by side:
... and tell me which you'd look over first if you were evaluating things.
How can we attract software vendors to support PostgreSQL? What words are convincing to appeal the increasing potential of PostgreSQL as a good replacement for commercial databases?
Change the name :p
<dives for fireproof clothing>
> From: pgsql-hackers-owner@postgresql.org > [mailto:pgsql-hackers-owner@postgresql.org] On Behalf Of Mark Kirkwood > For cloud - in particular Openstack (which I am working with ATM), the > biggest thing would be: > > - multi-master replication > > or failing that: > > - self managing single master failover (voting/quorum etc) > > so that operators can essentially 'set and forget'. We currently use > Mysql+ Galera (multi master) and Mongodb (self managing single master) > and the convenience and simplicity is just so important (Openstack is a > huge complex collection of services - hand holding of any one service is > pretty much a non starter). Yes, I was also asked whether PostgreSQL has any optional functionality like Galera Cluster for MySQL. He was planning ascalable PaaS service which performs heavy reads and writes. Demand exists. Regards Takayuki Tsunakawa
> * Only 24 open source projects are listed as interoperable. > Open Source Projects Using PostgreSQL > https://wiki.postgresql.org/wiki/OpenSource_Projects_Using_PostgreSQL It's pity that some very popular OSS projects do not care about PostgreSQL. WordPress is one of such OSS projects. Long time ago someone came up with "PostgreSQL plugin" but it didn't catch up WordPress releases and now cannot be used with the latest version of WordPress. In my understanding WordPress fixes security issues only with the latest version; as a result, we cannot use safe version of WordPress if we want to use PostgreSQL for its database engine. Best regards, -- Tatsuo Ishii SRA OSS, Inc. Japan English: http://www.sraoss.co.jp/index_en.php Japanese:http://www.sraoss.co.jp
On 03/07/2016 07:30 AM, Tsunakawa, Takayuki wrote: >> From: pgsql-hackers-owner@postgresql.org >> [mailto:pgsql-hackers-owner@postgresql.org] On Behalf Of Mark Kirkwood >> For cloud - in particular Openstack (which I am working with ATM), the >> biggest thing would be: >> >> - multi-master replication >> >> or failing that: >> >> - self managing single master failover (voting/quorum etc) Hmm.... consul (optionally in a dockerized setup) ? https://www.consul.io/ >> so that operators can essentially 'set and forget'. We currently use >> Mysql+ Galera (multi master) and Mongodb (self managing single master) >> and the convenience and simplicity is just so important (Openstack is a >> huge complex collection of services - hand holding of any one service is >> pretty much a non starter). > Yes, I was also asked whether PostgreSQL has any optional functionality like Galera Cluster for MySQL. He was planninga scalable PaaS service which performs heavy reads and writes. Demand exists. AFAIK, Galera has its own set of drawbacks.... HTH, / JL
On 7 March 2016 at 17:06, Tatsuo Ishii <ishii@postgresql.org> wrote:
> * Only 24 open source projects are listed as interoperable.
> Open Source Projects Using PostgreSQL
> https://wiki.postgresql.org/wiki/OpenSource_Projects_Using_PostgreSQL
It's pity that some very popular OSS projects do not care about
PostgreSQL. WordPress is one of such OSS projects. Long time ago
someone came up with "PostgreSQL plugin" but it didn't catch up
WordPress releases and now cannot be used with the latest version of
WordPress. In my understanding WordPress fixes security issues only
with the latest version; as a result, we cannot use safe version of
WordPress if we want to use PostgreSQL for its database engine.
The killer there was that the plugin could only alter queries used by Wordpress core. Nobody just uses Wordpress core. The whole reason Wordpress became popular is the vast collection of plugins, themes, etc. 90% of which are written by three stoned monkeys who once saw a PHP 4 manual in a mirror, which is part of why it has such an appalling security history. Plugins can just talk straight to the DB, and are written by people who have never heard of parametrized queries or, half the time, transactions.
What makes it popular and successful is also what makes supporting Pg in a way that'll actually see useful adoption hard.
On 6 March 2016 at 13:46, Mark Kirkwood <mark.kirkwood@catalyst.net.nz> wrote:
For cloud - in particular Openstack (which I am working with ATM), the
biggest thing would be:
- multi-master replication
Working on it ;)
or failing that:
- self managing single master failover (voting/quorum etc)
I don't think that's either/or, frankly, in that you can have multimaster that needs a lot of babying or multimaster that has magic autoscaling and autosharding and autofailover (or some approximation of).
That said, I firmly believe that autofailover is a giant foot-cannon unless *regularly tested in the live environment*. The same is true of most of the other autoFOOs. Products that claim to offer them often do some creative bending of the guarantees they claim to offer elsewhere, so I also tend to take such claims with a grain or ten of salt. Witness the recent discussions of Galleria.
Pg currently makes testing autofailover pretty painful, since failover is a major and disruptive event that reduces your redundancy until the old master has resynced. pg_rewind will help a lot there if it proves robust enough - hopefully enough that we can start adopting a ChaosMonkey-flavoured approach to HA.
Eventually, a long way down the track, I'd like to see logical replication based failover be part of that picture too.
so that operators can essentially 'set and forget'. We currently use
Mysql+ Galera (multi master) and Mongodb (self managing single master)
and the convenience and simplicity is just so important (Openstack is a
huge complex collection of services - hand holding of any one service is
pretty much a non starter).
I rarely see a service where "self managing" doesn't turn into "needs lots of tuning, tweaking and fixing" at significant scales, but it's nice to have it be automagic and correct enough of the time for small to medium deployments.
That's one area where Pg core's focus on getting it right can hold us back. Not necessarily for the worse. We're not usually big on saying "oh, it probably won't corrupt your data if you use it at reasonable transaction rates with reasonable inter-node latencies". Having had a few interesting surprises with set-and-forget-until-it-brings-the-whole-network-down-and-you-have-no-idea-how-to-fix-it systems I tend to favour doing it right myself. Which is why some of the compromises currently necessary in the logical replication space on Pg irk me so....
--
From: Craig Ringer
--------------------------------------------------
We could help ORMs solve the N+1 SELECTs problem and help them avoid transferring vast join projections unnecessarily. That'd make PostgreSQL pretty compelling for exactly the users we're mostly too busy dismissing to consider.
I'd be interested in reaching out to some Hibernate/JPA and ActiveRecord folks about how the DB could help the ORM and have been meaning to explore this area for a while, but -ENOTIME. If anyone pursues it I'll be really interested in hearing how things go.
--------------------------------------------------
You have various ideas and experience, don't you?
Are those ideas on ORMs beneficial exclusively to PostgreSQL or to all DBMSs? I don't know the structure of ORMs allows for improvements to be advantageous to a specific DBMS.
From: Craig Ringer
--------------------------------------------------
When you consider the adoption of PostgreSQL in the Rails/ActiveRecord crowd I think there's some interesting potential there. What there isn't is funding AFAIK.
--------------------------------------------------
Yes, this needs investigation. But I'm not sure Ruby ecosystem is the world where PostgreSQL should go. Do many users of commercial databases have assets based on Ruby which they want to continue to use?
Recently, my colleague, a Ruby committer, told me that Ruby may be losing momentum because developers are moving toward JavaScript+Node.js. Addressing JavaScript+Node.js ecosystem may be more beneficial to PostgreSQL popularity.
From: Craig Ringer
--------------------------------------------------
Besides, in practice, we probably should increase the number of software interoperable with PostgreSQL. e.g. one customer asked us whether Arcserve can be used to back up PostgreSQL databases, but unfortunately we had to answer no. They are using Arcserve to back up Oracle databases and other resources. "Then, you can use NetVault instead" is not the best answer; they just want to replace the database.
The "we" here is the problem. It's not likely to be folks focused on PostgreSQL core dev, but ... who, exactly?
--------------------------------------------------
I simply meant PostgreSQL developers and users (of course, including me) who wish PostgreSQL to become more popular and thrive for a long time. Most users use databases combined with applications (not just the DBMS alone), so increasing the number of applications supporting PostgreSQL will attract more users. PostgreSQL developers will gain wider market and business/job oppotunities.
From: Craig Ringer
--------------------------------------------------
Provide technical assistance to those vendors as an organization so that they can support PostgreSQL smoothly.
This one is a help. That said, pgsql-general is pretty helpful already...
--------------------------------------------------
Certainly.
From: Craig Ringer
--------------------------------------------------
* Make a directory of software/services that can be used with PostgreSQL on the community web site (wiki.postgresql.org or www.postgresql.org).
Software/services vendors and PostgreSQL developers/users can edit this directory.
I thought we had that? Yep.
It's pretty invisible though, partly due to the postgresql.org landing page's need for a trim-down and tidy. (Don't even get me started on http://www.postgresql.org/about/ )
--------------------------------------------------
Thanks, I know the page. It totally slipped off my head. Yes, I'm afraid the path to the page (Home -> Download -> Software Catalog) is not good.
From: Craig Ringer
--------------------------------------------------
* How/Where can we get the knowledge of expanding the software ecosystem? Is there any OSS project that we can learn from?
Mongo.
... and tell me which you'd look over first if you were evaluating things.
--------------------------------------------------
Yes, I want something like a site map. But I couldn't find information about software supporting MongoDB within a few minutes, and I gave up. MongoDB website may not be necessarily nice.
In that respect, MySQL website is good, because "Partners" link is on the home page. PostgreSQL website can also have "Related software" or something like that on the home page.
From: Craig Ringer
--------------------------------------------------
How can we attract software vendors to support PostgreSQL? What words are convincing to appeal the increasing potential of PostgreSQL as a good replacement for commercial databases?
Change the name :p
--------------------------------------------------
Sorry, I couldn't catch the implication. Do you mean changing the name PostgreSQL to something else, or just a joke?
Regards
MauMau
> The killer there was that the plugin could only alter queries used by > Wordpress core. Nobody just uses Wordpress core. The whole reason > Wordpress became popular is the vast collection of plugins, themes, etc. > 90% of which are written by three stoned monkeys who once saw a PHP 4 > manual in a mirror, which is part of why it has such an appalling security > history. Plugins can just talk straight to the DB, and are written by > people who have never heard of parametrized queries or, half the time, > transactions. Well, I think one of the reasons is they only know MySQL. MySQL had been lack of transactions and parametrized queries. > What makes it popular and successful is also what makes supporting Pg in a > way that'll actually see useful adoption hard. Let's enlighten WordPress users by supporting PostgreSQL. Best regards, -- Tatsuo Ishii SRA OSS, Inc. Japan English: http://www.sraoss.co.jp/index_en.php Japanese:http://www.sraoss.co.jp
From: Craig Ringer
--------------------------------------------------
* Make a directory of software/services that can be used with PostgreSQL on the community web site (wiki.postgresql.org or www.postgresql.org).
Software/services vendors and PostgreSQL developers/users can edit this directory.
I thought we had that? Yep.
--------------------------------------------------
There are 172 software listed here in total. Excluding the following categories which can be considered part of a DBMS, the number is 121. Certainly not much.
•Drivers and interfaces
•PostgreSQL-derived servers
•PostgreSQL extensions
•Procedural languages
Speaking without fear of misunderstanding, there are few software products listed that migrators from commercial databases would want to know... Even famous software like Pentaho, Microstrategy, Tableau, Eclipse and Visual Studio are not listed, which are known to work or should work with PostgreSQL. This is a shame.
Why don't we enrich the catalog? I'd like to hear ideas on how to enrich the catalog efficiently. It's ideal for software vendors and users to voluntarily add to the catalog.
Regards
MauMau
On 03/07/2016 11:31 AM, MauMau wrote: > Why don't we enrich the catalog? I'd like to hear ideas on how to > enrich the catalog efficiently. It's ideal for software vendors and > users to voluntarily add to the catalog. I think the product/software directory has vastly outlived its purpose. We are not longer a, "I wonder if Pg works with X". It is more, "Why doesn't X work with Pg" nowadays. I could see a page that discusses various tools in general that are supported but by no means do we need a "list" anymore. Sincerely, JD -- Command Prompt, Inc. http://the.postgres.company/ +1-503-667-4564 PostgreSQL Centered full stack support, consulting and development. Everyone appreciates your honesty, until you are honest with them.
On 07/03/16 11:54, José Luis Tallón wrote: > On 03/07/2016 07:30 AM, Tsunakawa, Takayuki wrote: >>> From: pgsql-hackers-owner@postgresql.org >>> [mailto:pgsql-hackers-owner@postgresql.org] On Behalf Of Mark Kirkwood >>> For cloud - in particular Openstack (which I am working with ATM), the >>> biggest thing would be: >>> >>> - multi-master replication >>> >>> or failing that: >>> >>> - self managing single master failover (voting/quorum etc) > > Hmm.... consul (optionally in a dockerized setup) ? > https://www.consul.io/ There are already HA solutions based on consensus, like Patroni: https://github.com/zalando/patroni > >>> so that operators can essentially 'set and forget'. We currently use >>> Mysql+ Galera (multi master) and Mongodb (self managing single master) >>> and the convenience and simplicity is just so important (Openstack is a >>> huge complex collection of services - hand holding of any one >>> service is >>> pretty much a non starter). >> Yes, I was also asked whether PostgreSQL has any optional >> functionality like Galera Cluster for MySQL. He was planning a >> scalable PaaS service which performs heavy reads and writes. Demand >> exists. > > AFAIK, Galera has its own set of drawbacks.... Right, some of them are explained here: https://aphyr.com/posts/327-jepsen-mariadb-galera-cluster Regards, Álvaro -- Álvaro Hernández Tortosa ----------- 8Kdata
MauMau, Crossing this over to pgsql-advocacy list where it really belongs. That's what that list is *for*. Especially since the discussion on -hackers has focused on new PostgreSQL Features, which while also good don't address the general question. On 03/05/2016 09:29 PM, MauMau wrote: > Hello, > As I said in the previous greeting mail, I'd like to discuss how to > expand PostgreSQL ecosystem. Here, ecosystem means "interoperability" > -- the software products and cloud services which use/support > PostgreSQL. If pgsql-advocacy or somewhere else is better for this > topic, just tell me so. > THE BACKGROUND > ================================================== > Thanks to the long and hard efforts by the community, PostgreSQL has > been evolving to be a really great software comparable to existing > strong commercial products. Open source databases are gaining more > popularity to influence the database market. > Open source threatens to eat the database market > http://www.infoworld.com/article/2916057/open-source-software/open-source-threatens-to-eat-the-database-market.html > "Though the proprietary RDBMS market grew at a sluggish 5.4 percent in > 2014, the open source database market grew 31 percent to hit $562 million." > "As Gartner highlights in a recent research report, open source > databases now consume 25 percent of relational database usage." > Perhaps related to this is that the revenues of Oracle, IBM and > Microsoft have been declining (but I read in an article that SQL Server > is gaining more revenue). > On the other hand, there is a gulf between the two top popular databases > -- Oracle and MySQL -- and PostgreSQL. They are nearly five times more > popular than PostgreSQL. > DB-Engines Ranking > http://db-engines.com/en/ranking > Yes, I understand this ranking doesn't necessarily reflect the actual > use, but I also don't think the ranking is far from the real > popularity. In fact, some surveys show that MySQL has been in more > widespread use even here in Japan than PostgreSQL since around 2010 (IIRC). > What should we do to boost the popularity of PostgreSQL? One challenge > is to increase the number of software which supports PostgreSQL. To take > advantage of the trend of shift from commercial products to open source, > PostgreSQL needs to interoperate with many software that are used > together with the commercial databases. > The easily understandable target is Oracle, because it is anticipated > that more users of Oracle will seek another database to avoid the > expensive Oracle Standard Edition 2 and increasing maintenance costs. > In addition, PostgreSQL has affinity for Oracle. > However, there is a problem. The number of software is very small that > the users can know to interoperate with PostgreSQL. That is, when the > users want to migrate from commercial databases to PostgreSQL, they > can't get information on whether they can continue to use their assets > with PostgreSQL. Many applications might be interoperable through > standard interfaces like JDBC/ODBC, but the case is unknown. For example: > * Only 24 open source projects are listed as interoperable. > Open Source Projects Using PostgreSQL > https://wiki.postgresql.org/wiki/OpenSource_Projects_Using_PostgreSQL > * Even EnterpriseDB has only 12 certified application vendors. > http://www.enterprisedb.com/partner-programs/enterprisedb-certified-application-vendors > * PostgreSQL Enterprise Consortium lists only about30 related products > (Japanese only). > https://www.pgecons.org/postgresql-info/business_sw/ > * MySQL touts more than 2,000 ISV/OEM/VARs. > http://www.mysql.com/oem/ > Besides, in practice, we probably should increase the number of software > interoperable with PostgreSQL. e.g. one customer asked us whether > Arcserve can be used to back up PostgreSQL databases, but unfortunately > we had to answer no. They are using Arcserve to back up Oracle > databases and other resources. "Then, you can use NetVault instead" is > not the best answer; they just want to replace the database. > PROPOSAL > ================================================== > Last month, I attended the steering committee of PostgreSQL Enterprise > Consortium (PGECons) for the first time and proposed starting the > following activity. PGECons is a Japanese non-profit organization to > promote PostgreSQL for enterprise use. The members include NTT, SRA OSS > (Tatsuo Ishii runs), NEC, Hitachi, HP, Fujitsu, etc. We concluded that > we need to consult the PostgreSQL community on how to proceed the > activity and work in cooperation with the community. > * Attract and ask product/service vendors to support/use PostgreSQL. > Provide technical assistance to those vendors as an organization so that > they can support PostgreSQL smoothly. > If the vendors aren't proactive, we verify the interoperability with > their software by executing it. > * Make a directory of software/services that can be used with PostgreSQL > on the community web site (wiki.postgresql.org or www.postgresql.org). > Software/services vendors and PostgreSQL developers/users can edit this > directory. > This list not only has the names of software and its vendors, but also > other information such as the level of interoperability (certified by > the vendor, or verified by the community/users) and remarks about > configuration, tuning, and whatever useful for users. > PostgreSQL Enterprise Consortium (PGECons) > https://www.pgecons.org/en > CONSULTATION AND DISCUSSION > ================================================== > I'd like to discuss and hear opinions on how can we expand the ecosystem > of PostgreSQL. Example points are: > * How/Where can we get the knowledge of expanding the software > ecosystem? Is there any OSS project that we can learn from? > How can we attract software vendors to support PostgreSQL? What words > are convincing to appeal the increasing potential of PostgreSQL as a > good replacement for commercial databases? > * How can we gain momentum for the activity? > Can we involve globally influential companies like Dell, HPE, and Red Hat? > * Do we need some kind of assistance from a foundation or establish a > foundation? > There should be many, many software to address, so the ecosystem > activity has to be long-lasting. Plus, money and expertise is necessary. > Would we benefit if we join the Linux Foundation Collaborative Projects? > Linux Foundation Collaborative Projects > http://collabprojects.linuxfoundation.org/ > * Which software/services in what category should we address preferentially? > What software would many users desire to be interoperable when migrating > from commercial databases? > What is the effective way to absorb user requests for this? Is it > enough to make a questionnaire like the following? What is the popular > questionnaire site which can catch many users (SurveyMonkey?) > https://postgresql.uservoice.com/forums/21853-general > Regards > MauMau -- -- Josh Berkus Red Hat OSAS (any opinions are my own)
On 03/07/2016 01:37 PM, Josh berkus wrote: > MauMau, > > Crossing this over to pgsql-advocacy list where it really belongs. > That's what that list is *for*. > > Especially since the discussion on -hackers has focused on new > PostgreSQL Features, which while also good don't address the general > question. > > On 03/05/2016 09:29 PM, MauMau wrote: >> Hello, >> As I said in the previous greeting mail, I'd like to discuss how to >> expand PostgreSQL ecosystem. Here, ecosystem means "interoperability" >> -- the software products and cloud services which use/support >> PostgreSQL. If pgsql-advocacy or somewhere else is better for this >> topic, just tell me so. >> THE BACKGROUND >> ================================================== >> Thanks to the long and hard efforts by the community, PostgreSQL has >> been evolving to be a really great software comparable to existing >> strong commercial products. Open source databases are gaining more >> popularity to influence the database market. >> Open source threatens to eat the database market >> http://www.infoworld.com/article/2916057/open-source-software/open-source-threatens-to-eat-the-database-market.html >> "Though the proprietary RDBMS market grew at a sluggish 5.4 percent in >> 2014, the open source database market grew 31 percent to hit $562 million." >> "As Gartner highlights in a recent research report, open source >> databases now consume 25 percent of relational database usage." >> Perhaps related to this is that the revenues of Oracle, IBM and >> Microsoft have been declining (but I read in an article that SQL Server >> is gaining more revenue). >> On the other hand, there is a gulf between the two top popular databases >> -- Oracle and MySQL -- and PostgreSQL. They are nearly five times more >> popular than PostgreSQL. >> DB-Engines Ranking >> http://db-engines.com/en/ranking >> Yes, I understand this ranking doesn't necessarily reflect the actual >> use, but I also don't think the ranking is far from the real >> popularity. In fact, some surveys show that MySQL has been in more >> widespread use even here in Japan than PostgreSQL since around 2010 (IIRC). >> What should we do to boost the popularity of PostgreSQL? One challenge >> is to increase the number of software which supports PostgreSQL. To take >> advantage of the trend of shift from commercial products to open source, >> PostgreSQL needs to interoperate with many software that are used >> together with the commercial databases. >> The easily understandable target is Oracle, because it is anticipated >> that more users of Oracle will seek another database to avoid the >> expensive Oracle Standard Edition 2 and increasing maintenance costs. >> In addition, PostgreSQL has affinity for Oracle. >> However, there is a problem. The number of software is very small that >> the users can know to interoperate with PostgreSQL. That is, when the >> users want to migrate from commercial databases to PostgreSQL, they >> can't get information on whether they can continue to use their assets >> with PostgreSQL. Many applications might be interoperable through >> standard interfaces like JDBC/ODBC, but the case is unknown. For example: >> * Only 24 open source projects are listed as interoperable. >> Open Source Projects Using PostgreSQL >> https://wiki.postgresql.org/wiki/OpenSource_Projects_Using_PostgreSQL >> * Even EnterpriseDB has only 12 certified application vendors. >> http://www.enterprisedb.com/partner-programs/enterprisedb-certified-application-vendors >> * PostgreSQL Enterprise Consortium lists only about30 related products >> (Japanese only). >> https://www.pgecons.org/postgresql-info/business_sw/ >> * MySQL touts more than 2,000 ISV/OEM/VARs. >> http://www.mysql.com/oem/ Though if you go to the actual list: http://www.mysql.com/customers/embedded/ There is at a rough count, 300. This points to the central dilemma, some of which is addressed below, separating fact from fiction. >> Besides, in practice, we probably should increase the number of software >> interoperable with PostgreSQL. e.g. one customer asked us whether >> Arcserve can be used to back up PostgreSQL databases, but unfortunately >> we had to answer no. They are using Arcserve to back up Oracle >> databases and other resources. "Then, you can use NetVault instead" is >> not the best answer; they just want to replace the database. >> PROPOSAL >> ================================================== >> Last month, I attended the steering committee of PostgreSQL Enterprise >> Consortium (PGECons) for the first time and proposed starting the >> following activity. PGECons is a Japanese non-profit organization to >> promote PostgreSQL for enterprise use. The members include NTT, SRA OSS >> (Tatsuo Ishii runs), NEC, Hitachi, HP, Fujitsu, etc. We concluded that >> we need to consult the PostgreSQL community on how to proceed the >> activity and work in cooperation with the community. >> * Attract and ask product/service vendors to support/use PostgreSQL. >> Provide technical assistance to those vendors as an organization so that >> they can support PostgreSQL smoothly. >> If the vendors aren't proactive, we verify the interoperability with >> their software by executing it. >> * Make a directory of software/services that can be used with PostgreSQL >> on the community web site (wiki.postgresql.org or www.postgresql.org). >> Software/services vendors and PostgreSQL developers/users can edit this >> directory. >> This list not only has the names of software and its vendors, but also >> other information such as the level of interoperability (certified by >> the vendor, or verified by the community/users) and remarks about >> configuration, tuning, and whatever useful for users. >> PostgreSQL Enterprise Consortium (PGECons) >> https://www.pgecons.org/en >> CONSULTATION AND DISCUSSION >> ================================================== >> I'd like to discuss and hear opinions on how can we expand the ecosystem >> of PostgreSQL. Example points are: >> * How/Where can we get the knowledge of expanding the software >> ecosystem? Is there any OSS project that we can learn from? >> How can we attract software vendors to support PostgreSQL? What words >> are convincing to appeal the increasing potential of PostgreSQL as a >> good replacement for commercial databases? >> * How can we gain momentum for the activity? >> Can we involve globally influential companies like Dell, HPE, and Red Hat? >> * Do we need some kind of assistance from a foundation or establish a >> foundation? >> There should be many, many software to address, so the ecosystem >> activity has to be long-lasting. Plus, money and expertise is necessary. >> Would we benefit if we join the Linux Foundation Collaborative Projects? >> Linux Foundation Collaborative Projects >> http://collabprojects.linuxfoundation.org/ >> * Which software/services in what category should we address preferentially? >> What software would many users desire to be interoperable when migrating >> from commercial databases? >> What is the effective way to absorb user requests for this? Is it >> enough to make a questionnaire like the following? What is the popular >> questionnaire site which can catch many users (SurveyMonkey?) >> https://postgresql.uservoice.com/forums/21853-general >> Regards >> MauMau > > -- Adrian Klaver adrian.klaver@aklaver.com
On Mon, Mar 7, 2016 at 9:25 PM, Craig Ringer wrote: > pg_rewind will help a lot there if it proves robust enough - FWIW, some of my colleagues are doing a lot of QE/QA on a HA solution based on pg_rewind, and it is proving to be quite stable for the moment, they are having a hard time breaking it (that's the 9.4 version, though it is intentionally close enough to what is in core, I'm careful about that). -- Michael
On 7 March 2016 at 20:55, MauMau <maumau307@gmail.com> wrote:
From: Craig Ringer--------------------------------------------------We could help ORMs solve the N+1 SELECTs problem and help them avoid transferring vast join projections unnecessarily. That'd make PostgreSQL pretty compelling for exactly the users we're mostly too busy dismissing to consider.I'd be interested in reaching out to some Hibernate/JPA and ActiveRecord folks about how the DB could help the ORM and have been meaning to explore this area for a while, but -ENOTIME. If anyone pursues it I'll be really interested in hearing how things go.--------------------------------------------------You have various ideas and experience, don't you?Are those ideas on ORMs beneficial exclusively to PostgreSQL or to all DBMSs? I don't know the structure of ORMs allows for improvements to be advantageous to a specific DBMS.
Most ORMs have dialect layers for query generation, DB-specific customisations, etc.
Whether they're flexible enough to handle this sort of change - I don't know. That's part of why I'd like to explore the ideas with ActiveRecord, Hibernate, JPA WG, etc folks.
Hello, Josh, > From: pgsql-hackers-owner@postgresql.org > [mailto:pgsql-hackers-owner@postgresql.org] On Behalf Of Josh berkus> Crossing this over to pgsql-advocacy list where itreally belongs. > That's what that list is *for*. > > Especially since the discussion on -hackers has focused on new PostgreSQL > Features, which while also good don't address the general question. > Thank you for pointing me to the correct place. I wondered which list is better, because I thought this topic whould bebetter discussed among hackers. I'll post subsequent mails only to pgsql-advocacy. Regards Takayuki Tsunakawa