Thread: Working toward a JTA 1.0.1 Compliant XADataSource
I'd like to start this thread by thanking everyone for their input on the questions I had regarding the existing XA implementationthis week. You have all been very supportive and very encouraging. Special thanks to Heikki and the rest ofthe community for providing the existing XA support and helping us understand it. For our application and middleware stack we've not had any luck making minor adjustments to improve the issues we've beenhaving. My early panic about serialized methods did help to eliminate one class of errors in our test environment (oncewe were finally able to duplicate the issue), but culminated in exposing the lack of interleaving or shareable XAResourcesfor 2pc operations with the GlassFish transaction manager as the ultimate culprit for our issues. It seems thatwhen GlassFish gets busy (and your resource pool gets busy) it starts expecting some aspects of interleaving -- in theform of commit(xid1, false) after start(xid2, NOFLAGS) has been issued to an XAResource. As has been discussed on this list earlier in the week an implementation that truly implements JTA 1.0.1, or at least supportsinterleaving will bring a new set of trade-offs. Such an implementation would also make more complex JEE deploymentsatop PostgreSQL possible (and more reliable, as in our case) for a wider set of containers and transaction managers.We love PostgreSQL. It's done very well for us in the past and we'd like to give back by making it a better productfor everyone to use. This looks like a pretty good place to start doing that. It is my intent to contribute to an effort to expand and improve the XA support for PostgreSQL. I believe this can be done.I believe a good balance of configurable trade-offs to meet more (if not all) of the JTA 1.0.1 specification can befound. I believe a more compliant XA implementation can be robust and preform respectably. I also like to think this can happen fairly fast. On behalf of the development team I've been working with on this the last few days, I'd like to invite any interested partyto take a peek at what we're doing on github. https://github.com/polarislabs/pgjdbc/tree/POLARIS_XA I've waited to announce our work on this until I was convinced we had the ability to pull it off and that it was in a stateworth asking for input on. Right now, the XADataSource implementation in our POLARIS_XA branch passes the existing XADataSource test suite -- even theinterleaving tests that were commented out for the old implementation. The new implementation already has code to support: * resource sharing * interleaving * suspend / resume / join * Achieves functional parity (according to the junit tests) with the existing driver. There are several of us still working on filling in the implementation. We know we have quite a ways to go before this is'production ready'. That said our situation is such that we are working to get our immediate needs met while trying hard to do so in a mannerthat will not degrade or break existing uses of the XADataSource. We're going to need help in making sure we don'thave horrible regressions. We'd like to invite critiques, comments, or pull requests from the rest of the community. Especially those of you who'vehelped us reach this point (weather you knew it or not)! What can we do better? What things have we done wrong? Whatproblems do you foresee? Now that there's what I'd consider a functional prototype let's get the conversation started. What's the next step to getting this pushed upstream? What barriers do you foresee, and what can I (or anyone else) do inthe meantime? Is this the appropriate communication channel for such things? Regards, -Bryan Varner Senior Developer POLARIS Laboratories The statements, views, and opinions expressed in this email are my own personal remarks, and do no represent or imply anyobligation on behalf of my employer, POLARIS Laboratories. P.S. I'm scheduled to be on vacation next week, but I have a sneaking suspicion I'll be keeping tabs on this mailing list.The other developers from our team who subscribe to this group, (Kirk Winters : kwinters at polarislabs.com, and AdamGray : agray at polarislabs.com) will also be keeping tabs.
On 16.02.2013 09:16, Bryan Viarner wrote: > For our application and middleware stack we've not had any luck making minor adjustments to improve the issues we've beenhaving. My early panic about serialized methods did help to eliminate one class of errors in our test environment (oncewe were finally able to duplicate the issue), but culminated in exposing the lack of interleaving or shareable XAResourcesfor 2pc operations with the GlassFish transaction manager as the ultimate culprit for our issues. It seems thatwhen GlassFish gets busy (and your resource pool gets busy) it starts expecting some aspects of interleaving -- in theform of commit(xid1, false) after start(xid2, NOFLAGS) has been issued to an XAResource. Sorry to hear that. I'd be interested to have an isolated, stand-alone test application that reproduces the problem, so that I could also play with it. > As has been discussed on this list earlier in the week an implementation that truly implements JTA 1.0.1, or at least supportsinterleaving will bring a new set of trade-offs. Such an implementation would also make more complex JEE deploymentsatop PostgreSQL possible (and more reliable, as in our case) for a wider set of containers and transaction managers. Frankly, I don't think this is worth the trouble, or worth complicating the driver for. It would be nice to be fully compliant, just so we could say that we are, but in practice, meh. The PostgreSQL driver is not alone with the limitations. I believe Oracle has similar limitations, as does MySQL, and DB2. Which is why all application servers I've bumped into this far have options to work around them. I'm very surprised if Glassfish doesn't. In practice, a transaction manager that insists on working support for transaction interleaving and suspend/resume, is pretty useless. For example, in JBoss the option is called "track-connection-by-tx". Since JBoss 5, the default is "true", so you don't need to remember to set it manually anymore. I'm guessing that the JBoss developers decided that it's better to set it by default, because in practice most connector implementations require it to be set. On Atomikos, the setting is called "com.atomikos.icatch.threaded_2pc", and they also seem to have switched the default to false (ie. the right option for PostgreSQL") in version 3.7. IMHO it was a serious mistake from the JTA spec authors to require those extra features. They are not required for robust two-phase commit, and are difficult to implement in many JDBC drivers, including PostgreSQL's. > We love PostgreSQL. It's done very well for us in the past and we'd like to give back by making it a better product foreveryone to use. This looks like a pretty good place to start doing that. I think the best action here would be to expand the FAQ with specific information on how to set up the data source on various app servers (http://jdbc.postgresql.org/documentation/faq.html). And if Glassfish doesn't yet provide a suitable work-around option, petition them to implement one. > On behalf of the development team I've been working with on this the last few days, I'd like to invite any interested partyto take a peek at what we're doing on github. > https://github.com/polarislabs/pgjdbc/tree/POLARIS_XA There seems to be some unrelated refactoring in this branch, moving files around, so it's a bit hard to see what the real changes are. One random thought just occurred to me: while I don't think this is worth the complexity in the PostgreSQL driver, would it be possible to implement this as a generic wrapper that could be used with any XADataSource implementation? The concept of implementing the transaction interleaving and suspend/resume by adding yet another layer of pooling seems pretty universal. It could then be used with any application server which doesn't already provide a work-around option, and with any JDBC implementation that has these limitations. - Heikki
> Sorry to hear that. I'd be interested to have an isolated, stand-alone > test application that reproduces the problem, so that I could also play > with it. Now that we've been able to reproduce it in our test environments, it's far more likely we can do that. I'd be happy to send you some debug logs from the pgjdbc driver, where it documented what was going on. > Frankly, I don't think this is worth the trouble, or worth complicating > the driver for. It would be nice to be fully compliant, just so we could > say that we are, but in practice, meh. If the existing implementation actually worked for my case, I'd agree with you. It's not working for me, so at this point, I disagree. > The PostgreSQL driver is not alone with the limitations. I believe > Oracle has similar limitations, as does MySQL, and DB2. Which is why all > application servers I've bumped into this far have options to work > around them. I'm very surprised if Glassfish doesn't. In practice, a > transaction manager that insists on working support for transaction > interleaving and suspend/resume, is pretty useless. While we are not seeing join / suspend / resume, we are seeing the expectation that you can commit a previously prepared TX while already working on a new TX. I've read the JTA specification through several times at this point, and I'm not sure if that should be considered interleaving, or properly implementing the resource sharing aspect of XAResource. Here's the configuration options for the GlassFish version we're using. http://docs.oracle.com/cd/E18930_01/html/821-2416/beanp.html#scrolltoc The closest thing to interleaving I see is ALLOW_MULTIPLE_ENLISTS_DELISTS, which is disabled (false) by default. > IMHO it was a serious mistake from the JTA spec authors to require those > extra features. They are not required for robust two-phase commit, and > are difficult to implement in many JDBC drivers, including PostgreSQL's. I too, hope this gets fixed in the 2.0 spec. >> We love PostgreSQL. It's done very well for us in the past and we'd >> like to give back by making it a better product for everyone to use. >> This looks like a pretty good place to start doing that. > > I think the best action here would be to expand the FAQ with specific > information on how to set up the data source on various app servers > (http://jdbc.postgresql.org/documentation/faq.html). And if Glassfish > doesn't yet provide a suitable work-around option, petition them to > implement one. I have been in contact with a few of the GlassFish developers. I will try to see if there is something they could do in a future version. GlassFish is a bigger, slower moving project at this point, and I don't have the time to wait for them to change their JTA implementation. >> https://github.com/polarislabs/pgjdbc/tree/POLARIS_XA > > There seems to be some unrelated refactoring in this branch, moving > files around, so it's a bit hard to see what the real changes are. To get better support for code completion in a few IDE's we moved things into a 'src' directory, and updated the build.xml to handle it. We added a protected method to the PGPoolingConnection so that subclasses can obtain the underlying Connection without having to cache it as a private member variable. The XADataSource now implements most of the XA control invocation, and extends AbstractXADataSource, which is the file filtered to extend the proper spec version implementation by the ant build. > One random thought just occurred to me: while I don't think this is > worth the complexity in the PostgreSQL driver, would it be possible to > implement this as a generic wrapper that could be used with any > XADataSource implementation? That's essentially what we're doing with this implementation... except that we're tied to the pgjdbc build system to make sure we build for the right version of the jdbc spec, and the PostgreSQL dialect. It certainly would be 'possible' to wrap a normal DataSource into an XA compliant pool and provide different implementations for different spec versions and dialects. > The concept of implementing the transaction > interleaving and suspend/resume by adding yet another layer of pooling > seems pretty universal. It could then be used with any application > server which doesn't already provide a work-around option, and with any > JDBC implementation that has these limitations. Having it work for 'any' database is a lofty goal, would take a lot more time, and won't help address the problem I have any faster. From a practical engineering standpoint, what you're proposing is a very large undertaking, and one that won't help me resolve my situation any faster than sticking to one database, one driver, and two jdbc version specs. Regards, -Bryan Varner
For what it's worth Bryan I thing what you're doing is a good thing, this makes the implementation more robust and I'll be testing what you have as soon as I find the time. Thanks, Florent On Fri, Feb 15, 2013 at 11:16 PM, Bryan Varner <bvarner@polarislabs.com> wrote: > I'd like to start this thread by thanking everyone for their input on the questions I had regarding the existing XA implementationthis week. You have all been very supportive and very encouraging. Special thanks to Heikki and the rest ofthe community for providing the existing XA support and helping us understand it. > > > For our application and middleware stack we've not had any luck making minor adjustments to improve the issues we've beenhaving. My early panic about serialized methods did help to eliminate one class of errors in our test environment (oncewe were finally able to duplicate the issue), but culminated in exposing the lack of interleaving or shareable XAResourcesfor 2pc operations with the GlassFish transaction manager as the ultimate culprit for our issues. It seems thatwhen GlassFish gets busy (and your resource pool gets busy) it starts expecting some aspects of interleaving -- in theform of commit(xid1, false) after start(xid2, NOFLAGS) has been issued to an XAResource. > > > As has been discussed on this list earlier in the week an implementation that truly implements JTA 1.0.1, or at least supportsinterleaving will bring a new set of trade-offs. Such an implementation would also make more complex JEE deploymentsatop PostgreSQL possible (and more reliable, as in our case) for a wider set of containers and transaction managers.We love PostgreSQL. It's done very well for us in the past and we'd like to give back by making it a better productfor everyone to use. This looks like a pretty good place to start doing that. > > It is my intent to contribute to an effort to expand and improve the XA support for PostgreSQL. I believe this can be done.I believe a good balance of configurable trade-offs to meet more (if not all) of the JTA 1.0.1 specification can befound. I believe a more compliant XA implementation can be robust and preform respectably. > > I also like to think this can happen fairly fast. > > On behalf of the development team I've been working with on this the last few days, I'd like to invite any interested partyto take a peek at what we're doing on github. > https://github.com/polarislabs/pgjdbc/tree/POLARIS_XA > > I've waited to announce our work on this until I was convinced we had the ability to pull it off and that it was in a stateworth asking for input on. > > Right now, the XADataSource implementation in our POLARIS_XA branch passes the existing XADataSource test suite -- eventhe interleaving tests that were commented out for the old implementation. > The new implementation already has code to support: > * resource sharing > * interleaving > * suspend / resume / join > * Achieves functional parity (according to the junit tests) with the existing driver. > > There are several of us still working on filling in the implementation. We know we have quite a ways to go before thisis 'production ready'. > That said our situation is such that we are working to get our immediate needs met while trying hard to do so in a mannerthat will not degrade or break existing uses of the XADataSource. We're going to need help in making sure we don'thave horrible regressions. > > We'd like to invite critiques, comments, or pull requests from the rest of the community. Especially those of you who'vehelped us reach this point (weather you knew it or not)! What can we do better? What things have we done wrong? Whatproblems do you foresee? > Now that there's what I'd consider a functional prototype let's get the conversation started. > > What's the next step to getting this pushed upstream? What barriers do you foresee, and what can I (or anyone else) doin the meantime? Is this the appropriate communication channel for such things? > > Regards, > -Bryan Varner > > Senior Developer > POLARIS Laboratories > > The statements, views, and opinions expressed in this email are my own personal remarks, and do no represent or imply anyobligation on behalf of my employer, POLARIS Laboratories. > > P.S. I'm scheduled to be on vacation next week, but I have a sneaking suspicion I'll be keeping tabs on this mailing list.The other developers from our team who subscribe to this group, (Kirk Winters : kwinters at polarislabs.com, and AdamGray : agray at polarislabs.com) will also be keeping tabs. -- Florent Guillaume, Director of R&D, Nuxeo Open Source, Java EE based, Enterprise Content Management (ECM) http://www.nuxeo.com http://www.nuxeo.org +33 1 40 33 79 87
Since it's been nearly two weeks since my first message, I thought I'd send an update on where we are with our XA reimplementation. Our production environment (GlassFish) is standing on it's own two feet with our XA implementation. https://github.com/polarislabs/pgjdbc/tree/POLARIS_XA We're seeing decent performance and (mostly) reliable operation. We're hammering out a few obscure race condition bugs with regards to threaded invocations of shared XAResources & the connection handle usage. I expect we'll have these fixed and in our github repo by Monday. We are passing all the existing XA pgjdbc tests, as well as the expanded ones we've produced. The driver operates with and without interleaving (you can disable the extra physical connection allocation with a property). Our XA implementation also passes the JBoss TM (version 4.17) JDBC (XA mode!) test suite. https://github.com/jbosstm/narayana/tree/4.17 I know there were some complaints earlier in the thread that we moved all the source code into a 'src' directory. We did this to support IDE environments which don't like having the source folder be the same as the root of the project (NetBeans comes to mind). We also fixed the build.xml to _completely_ respect the property it already had for the source directory location. If the src directory and build changes are going to be an obstacle to getting this work accepted upstream, now would be a good time to tell me. If there's no hope of having this accepted upstream, now would be a good time to tell me. Again, I invite anyone to suggest changes, test it out, and report back what you find. Regards, -Bryan Varner ---------------- Senior Developer POLARIS Laboratories On 02/20/2013 05:44 PM, Florent Guillaume wrote: > For what it's worth Bryan I thing what you're doing is a good thing, > this makes the implementation more robust and I'll be testing what you > have as soon as I find the time. > > Thanks, > Florent
Yes, moving everything to src will not work for the project.
Dave
On Fri, Mar 1, 2013 at 3:48 PM, Bryan Varner <bvarner@polarislabs.com> wrote:
Since it's been nearly two weeks since my first message, I thought I'd send an update on where we are with our XA reimplementation.
Our production environment (GlassFish) is standing on it's own two feet with our XA implementation.
https://github.com/polarislabs/pgjdbc/tree/POLARIS_XA
We're seeing decent performance and (mostly) reliable operation.
We're hammering out a few obscure race condition bugs with regards to threaded invocations of shared XAResources & the connection handle usage. I expect we'll have these fixed and in our github repo by Monday.
We are passing all the existing XA pgjdbc tests, as well as the expanded ones we've produced. The driver operates with and without interleaving (you can disable the extra physical connection allocation with a property).
Our XA implementation also passes the JBoss TM (version 4.17) JDBC (XA mode!) test suite.
https://github.com/jbosstm/narayana/tree/4.17
I know there were some complaints earlier in the thread that we moved all the source code into a 'src' directory. We did this to support IDE environments which don't like having the source folder be the same as the root of the project (NetBeans comes to mind). We also fixed the build.xml to _completely_ respect the property it already had for the source directory location.
If the src directory and build changes are going to be an obstacle to getting this work accepted upstream, now would be a good time to tell me.
If there's no hope of having this accepted upstream, now would be a good time to tell me.
Again, I invite anyone to suggest changes, test it out, and report back what you find.
Regards,
-Bryan Varner
----------------
Senior Developer
POLARIS Laboratories
On 02/20/2013 05:44 PM, Florent Guillaume wrote:For what it's worth Bryan I thing what you're doing is a good thing,
this makes the implementation more robust and I'll be testing what you
have as soon as I find the time.
Thanks,
Florent--
Sent via pgsql-jdbc mailing list (pgsql-jdbc@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-jdbc
On 03/01/2013 10:16 PM, Dave Cramer wrote:
Could you also tell _why_ it would not work ?Yes, moving everything to src will not work for the project.
Will it break some other IDEs expectations ?
Hannu
DaveOn Fri, Mar 1, 2013 at 3:48 PM, Bryan Varner <bvarner@polarislabs.com> wrote:Since it's been nearly two weeks since my first message, I thought I'd send an update on where we are with our XA reimplementation.
Our production environment (GlassFish) is standing on it's own two feet with our XA implementation.
https://github.com/polarislabs/pgjdbc/tree/POLARIS_XA
We're seeing decent performance and (mostly) reliable operation.
We're hammering out a few obscure race condition bugs with regards to threaded invocations of shared XAResources & the connection handle usage. I expect we'll have these fixed and in our github repo by Monday.
We are passing all the existing XA pgjdbc tests, as well as the expanded ones we've produced. The driver operates with and without interleaving (you can disable the extra physical connection allocation with a property).
Our XA implementation also passes the JBoss TM (version 4.17) JDBC (XA mode!) test suite.
https://github.com/jbosstm/narayana/tree/4.17
I know there were some complaints earlier in the thread that we moved all the source code into a 'src' directory. We did this to support IDE environments which don't like having the source folder be the same as the root of the project (NetBeans comes to mind). We also fixed the build.xml to _completely_ respect the property it already had for the source directory location.
If the src directory and build changes are going to be an obstacle to getting this work accepted upstream, now would be a good time to tell me.
If there's no hope of having this accepted upstream, now would be a good time to tell me.
Again, I invite anyone to suggest changes, test it out, and report back what you find.
Regards,
-Bryan Varner
----------------
Senior Developer
POLARIS Laboratories
On 02/20/2013 05:44 PM, Florent Guillaume wrote:For what it's worth Bryan I thing what you're doing is a good thing,
this makes the implementation more robust and I'll be testing what you
have as soon as I find the time.
Thanks,
Florent--
Sent via pgsql-jdbc mailing list (pgsql-jdbc@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-jdbc
There's no technical reason as to why it would not work.
The project has used this layout for as long as I can remember. If we were to change it for this patch why not change it for every patch ?
I can get the project to build in eclipse and IDEA, I haven't tried netbeans, but I see no reason why it wouldn't work.
Dave
On Fri, Mar 1, 2013 at 6:48 PM, Hannu Krosing <hannu@2ndquadrant.com> wrote:
Could you also tell _why_ it would not work ?On 03/01/2013 10:16 PM, Dave Cramer wrote:Yes, moving everything to src will not work for the project.
Will it break some other IDEs expectations ?
HannuDaveOn Fri, Mar 1, 2013 at 3:48 PM, Bryan Varner <bvarner@polarislabs.com> wrote:Since it's been nearly two weeks since my first message, I thought I'd send an update on where we are with our XA reimplementation.
Our production environment (GlassFish) is standing on it's own two feet with our XA implementation.
https://github.com/polarislabs/pgjdbc/tree/POLARIS_XA
We're seeing decent performance and (mostly) reliable operation.
We're hammering out a few obscure race condition bugs with regards to threaded invocations of shared XAResources & the connection handle usage. I expect we'll have these fixed and in our github repo by Monday.
We are passing all the existing XA pgjdbc tests, as well as the expanded ones we've produced. The driver operates with and without interleaving (you can disable the extra physical connection allocation with a property).
Our XA implementation also passes the JBoss TM (version 4.17) JDBC (XA mode!) test suite.
https://github.com/jbosstm/narayana/tree/4.17
I know there were some complaints earlier in the thread that we moved all the source code into a 'src' directory. We did this to support IDE environments which don't like having the source folder be the same as the root of the project (NetBeans comes to mind). We also fixed the build.xml to _completely_ respect the property it already had for the source directory location.
If the src directory and build changes are going to be an obstacle to getting this work accepted upstream, now would be a good time to tell me.
If there's no hope of having this accepted upstream, now would be a good time to tell me.
Again, I invite anyone to suggest changes, test it out, and report back what you find.
Regards,
-Bryan Varner
----------------
Senior Developer
POLARIS Laboratories
On 02/20/2013 05:44 PM, Florent Guillaume wrote:For what it's worth Bryan I thing what you're doing is a good thing,
this makes the implementation more robust and I'll be testing what you
have as soon as I find the time.
Thanks,
Florent--
Sent via pgsql-jdbc mailing list (pgsql-jdbc@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-jdbc
Dave Cramer <pg@fastcrypt.com> wrote: > Hannu Krosing <hannu@2ndquadrant.com> wrote: >> Dave Cramer wrote: >>> Bryan Varner <bvarner@polarislabs.com> wrote: >>>> If the src directory and build changes are going to be an >>>> obstacle to getting this work accepted upstream, now would be >>>> a good time to tell me. >>> Yes, moving everything to src will not work for the project. >> Could you also tell _why_ it would not work ? > There's no technical reason as to why it would not work. > > The project has used this layout for as long as I can remember. > If we were to change it for this patch why not change it for > every patch ? My opinions, FWIW: (1) A change like this should be done as a separate patch, so that it doesn't make it harder to find substantive logical changes to the code. I recommend that you offer the improvements to XADataSource without the source file relocation, and if you want to convince people to reorganize the directory structure work on that at a different time on a new thread. (2) I strongly prefer having a src directory below the top level. (3) I don't think anyone is advocating that each patch be allowed to reorganize the directory structure; it's a question of which is the best way to organize it long-term. -- Kevin Grittner EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company
For what it's worth, there isn't any belligerence intended. We're just merely asking in order to determine how much morewe have left to do before it can be considered. Reverting the src directory change at this point is rather minor comparedto the work that's been put into it so far. There's still a little work to be done chasing down the corner cases,but it's good to have the conversation now so the patch can be in an acceptable form when submitted rather than gothrough a more extensive revision process after-the-fact. -----Original Message----- From: pgsql-jdbc-owner@postgresql.org [mailto:pgsql-jdbc-owner@postgresql.org] On Behalf Of Kevin Grittner Sent: Saturday, March 02, 2013 9:20 AM To: Dave Cramer; Hannu Krosing Cc: Bryan Varner; Florent Guillaume; pgsql-jdbc@postgresql.org Subject: Re: [JDBC] Working toward a JTA 1.0.1 Compliant XADataSource Dave Cramer <pg@fastcrypt.com> wrote: > Hannu Krosing <hannu@2ndquadrant.com> wrote: >> Dave Cramer wrote: >>> Bryan Varner <bvarner@polarislabs.com> wrote: >>>> If the src directory and build changes are going to be an obstacle >>>> to getting this work accepted upstream, now would be a good time to >>>> tell me. >>> Yes, moving everything to src will not work for the project. >> Could you also tell _why_ it would not work ? > There's no technical reason as to why it would not work. > > The project has used this layout for as long as I can remember. > If we were to change it for this patch why not change it for every > patch ? My opinions, FWIW: (1) A change like this should be done as a separate patch, so that it doesn't make it harder to find substantive logicalchanges to the code. I recommend that you offer the improvements to XADataSource without the source file relocation,and if you want to convince people to reorganize the directory structure work on that at a different time on anew thread. (2) I strongly prefer having a src directory below the top level. (3) I don't think anyone is advocating that each patch be allowed to reorganize the directory structure; it's a questionof which is the best way to organize it long-term. -- Kevin Grittner EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-jdbc mailing list (pgsql-jdbc@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-jdbc
I think it would be better to adopt the Maven-style directory layout e.g src/main/java and src/test/java. It would allow the flexibility to move to other build systems in the future, if desired. Plus it seems quite a logical arrangement.
I also agree, that such a big change should form a separate patch that only performs the directory modifications, to confirm all permutations of the existing build creates the same artifact after the re-arrangement.
Thanks
Stephen
+1 on using the src/main/java and src/main/test as this is now used by maven, gradle, and is widely understood convention. I think it might even help in making it easier to create a maven build script for the driver, which might make it easierto get the driver into maven central. On 2013-03-03, at 3:52 PM, Stephen Nelson <stephen@eccostudio.com> wrote: > I think it would be better to adopt the Maven-style directory layout e.g src/main/java and src/test/java. It would allowthe flexibility to move to other build systems in the future, if desired. Plus it seems quite a logical arrangement. > > I also agree, that such a big change should form a separate patch that only performs the directory modifications, to confirmall permutations of the existing build creates the same artifact after the re-arrangement. > > Thanks > > Stephen
> I also agree, that such a big change should form a separate patch that > only performs the directory modifications, to confirm all permutations > of the existing build creates the same artifact after the re-arrangement. > Thanks > Stephen I too, agree with this approach. The current layout 'works' in netbeans, it just looks weird. There's no way (that we can find) to filter out the 'build' or non-source directories. In the future, I'd really like to see the build system moved to something that allows us to easily push maven artifacts. I can't think of any reason maven central wouldn't accept this project for inclusion. Our POLARIS_XA branch currently has everything moved back to '.'. However, I've retained the changes to build.xml which made it respect the 'srcdir' property. These are pretty minor changes, and they're tied up with our change to what java.in classes get filtered during the build. I don't think this should cause too much of an issue, and it will likely help make it clear what needs to be done if the build system is altered in the future. At this point, we're actively trying to prep for an upstream patch. https://github.com/polarislabs/pgjdbc/compare/POLARIS_XA What is the preferred method for doing this? Perhaps a github pull request? ;-) In our branch, the XA junit tests have been expanded quite a bit to exercise all the corner cases we've seen crop up. We've also been running the PGXADataSource against the jboss tm (narayana) QA test suite as an additional smoke-test. I will be getting in contact with the glassfish project (again) this week to determine what it takes to get the PGXADataSource listed as a supported configuration. http://docs.oracle.com/cd/E18930_01/html/821-2416/beamw.html#gjksj Interleaving is currently enabled by default, but can be disabled (set the xaAcquireTimeout parameter to 0). We do have test coverage for both situations. If a TM tries to do interleaving when it's disabled, you'll get an appropriate exception message telling you what to do about it. Due to the nature of interleaving and isolation (and the way we've implemented it so far) there is not a 1-to-1 relationship between connections returned from the PGXADataSource and the number of physical DB connections to the back-end. The actual number of physical connections needed to service the XA invocations is totally application, workload, and TM dependent. If your TM never attempts interleaving, and your application doesn't do anything in local TX scopes, it's likely that you'll see 1-to-1 connection count ratios. We've added a high-water mark count that logs when the logger is set to 'info' so you can get a fairly decent idea of what the driver is doing. Because the physical connection count is so dependent upon application and TM, I don't feel comfortable giving a rule-of-thumb for configuring things here. What I can say is that, we're seeing that in cases of high-volume, we have about 12 physical connections per JVM more than the configured 'max pool size' in GlassFish. That's all I can think of for now. If anyone wants to start testing builds, it should be reasonably safe to do so now. Regards, -Bryan Varner
On Tue, Mar 5, 2013 at 12:45 PM, Bryan Varner <bvarner@polarislabs.com> wrote:
I too, agree with this approach.I also agree, that such a big change should form a separate patch that
only performs the directory modifications, to confirm all permutations
of the existing build creates the same artifact after the re-arrangement.
Thanks
Stephen
The current layout 'works' in netbeans, it just looks weird. There's no way (that we can find) to filter out the 'build' or non-source directories.
In the future, I'd really like to see the build system moved to something that allows us to easily push maven artifacts. I can't think of any reason maven central wouldn't accept this project for inclusion.
there is already a maven patch in the works which I haven't finished applying yet and it works with the current layout.
Our POLARIS_XA branch currently has everything moved back to '.'. However, I've retained the changes to build.xml which made it respect the 'srcdir' property. These are pretty minor changes, and they're tied up with our change to what java.in classes get filtered during the build. I don't think this should cause too much of an issue, and it will likely help make it clear what needs to be done if the build system is altered in the future.
At this point, we're actively trying to prep for an upstream patch.
https://github.com/polarislabs/pgjdbc/compare/POLARIS_XA
What is the preferred method for doing this?
Perhaps a github pull request? ;-)
Yes a github pull request is preferred
In our branch, the XA junit tests have been expanded quite a bit to exercise all the corner cases we've seen crop up. We've also been running the PGXADataSource against the jboss tm (narayana) QA test suite as an additional smoke-test. I will be getting in contact with the glassfish project (again) this week to determine what it takes to get the PGXADataSource listed as a supported configuration.
http://docs.oracle.com/cd/E18930_01/html/821-2416/beamw.html#gjksj
Interleaving is currently enabled by default, but can be disabled (set the xaAcquireTimeout parameter to 0). We do have test coverage for both situations. If a TM tries to do interleaving when it's disabled, you'll get an appropriate exception message telling you what to do about it.
Due to the nature of interleaving and isolation (and the way we've implemented it so far) there is not a 1-to-1 relationship between connections returned from the PGXADataSource and the number of physical DB connections to the back-end. The actual number of physical connections needed to service the XA invocations is totally application, workload, and TM dependent. If your TM never attempts interleaving, and your application doesn't do anything in local TX scopes, it's likely that you'll see 1-to-1 connection count ratios.
We've added a high-water mark count that logs when the logger is set to 'info' so you can get a fairly decent idea of what the driver is doing.
Because the physical connection count is so dependent upon application and TM, I don't feel comfortable giving a rule-of-thumb for configuring things here. What I can say is that, we're seeing that in cases of high-volume, we have about 12 physical connections per JVM more than the configured 'max pool size' in GlassFish.
That's all I can think of for now. If anyone wants to start testing builds, it should be reasonably safe to do so now.
Regards,
-Bryan Varner
Thanks for the work!