Thread: Testing
I noticed, that there is not any story of how PostgreSQL is tested. I could find a FAQ, and some stuff about regression tests, but what I am looking for, is that is there a small group, that is responsible for testing, what tools they are using, if bugs are found, what happens then etc. Is there a location, where I could find something, or is there someone I could ask, how it is done? (So I can write something about it...) Rgs, Jussi
Jussi Mikkola wrote: > I noticed, that there is not any story of how PostgreSQL is tested. I > could find a FAQ, and some stuff about regression tests, but what I am > looking for, is that is there a small group, that is responsible for > testing, what tools they are using, if bugs are found, what happens then > etc. When you compile postgresql from sources, after make is done, you can run make check which runs lots of tests to make sure that it works according to design. After that, you can use it for development and production. If you find a bug, submit a bug report to pgsql-bugs@postgresql.org. There is no special testing team. The user base is the testing team..:-) HTH Shridhar
Okay. The tests. Is it so, that if I make a new feature, I also make a test for it? What if that new feature runs without errors, but is very slow, uses plenty of memory etc. ? Can it leak memory? Is there a test for all new features? What happens, if there is no test for a feature? Is there a link from a bug to change tests? Well, when I look at the release history, and for example 7.3, there was 7.3.4 that came out before 7.4. So there are not very many releases. That means, that there can't be very many serious bugs left around. Many bugs would mean that there would need to be minor releases quite often. Usually it is not just luck, that a product works. But I would like to tell, that there is a reason, and that also the future releases are very reliable ;-) Certainly, I would not like to tell, that as a user, you are the tester... But of course, it is different, if we say, that as a user, you have the ability to test the software, and verify that it works properly ;-) Jussi Shridhar Daithankar wrote: > Jussi Mikkola wrote: > >> I noticed, that there is not any story of how PostgreSQL is tested. I >> could find a FAQ, and some stuff about regression tests, but what I >> am looking for, is that is there a small group, that is responsible >> for testing, what tools they are using, if bugs are found, what >> happens then etc. > > > When you compile postgresql from sources, after make is done, you can > run make check which runs lots of tests to make sure that it works > according to design. > > After that, you can use it for development and production. If you find > a bug, submit a bug report to pgsql-bugs@postgresql.org. > > There is no special testing team. The user base is the testing team..:-) > > HTH > > Shridhar > > -- Jussi Mikkola Partner, Project Manager Bonware Oy gsm +358 40 830 7561 Tekniikantie 21 tel +358 9 2517 5570 02150 Espoo fax +358 9 2517 5571 Finland www.bonware.com
On Friday 28 November 2003 13:41, Jussi Mikkola wrote: > Okay. The tests. Is it so, that if I make a new feature, I also make a > test for it? Depends on the feature really. > What if that new feature runs without errors, but is very > slow, uses plenty of memory etc. ? Can it leak memory? Is there a test > for all new features? What happens, if there is no test for a feature? > Is there a link from a bug to change tests? If the feature is slow/incomplete etc then it will only be used by those that really need that feature. Badly written code wont make it into the system because the core developers will refuse to commit it. It is not easy to define a test for all features - do you test to see if it works or it won't break? What about testing how all features interact? How do you define what is/is not a feature? Having said that, you've seen the regression tests and they check the standard operations work as they should. > Well, when I look at the release history, and for example 7.3, there was > 7.3.4 that came out before 7.4. So there are not very many releases. > That means, that there can't be very many serious bugs left around. Many > bugs would mean that there would need to be minor releases quite often. If there were many bugs, or bugs that were common then PostgreSQL wouldn't be in use. If you want to see the process of releasing a new version check the mailing list archives for pgsql-hackers and -bugs. A new release goes through a number of beta-tests and release candidates before being finalised. > Usually it is not just luck, that a product works. But I would like to > tell, that there is a reason, and that also the future releases are very > reliable ;-) Certainly, I would not like to tell, that as a user, you > are the tester... But of course, it is different, if we say, that as a > user, you have the ability to test the software, and verify that it > works properly ;-) With PG, some of the users are also the testers. Some of the users are the developers, some are advocates, some build add-on tools and some help out on the mailing lists. Of course, you don't have to do any of this, you could just run 7.3.x and wait until 7.4.1 or 7.4.2 is out, by which time any bugs should be pretty obscure. What I try to do is test the development version of my applications against new releases while sticking with the last stable release on my customers' machines. -- Richard Huxton Archonet Ltd
On Friday 28 November 2003 19:11, Jussi Mikkola wrote: > Okay. The tests. Is it so, that if I make a new feature, I also make a > test for it? What if that new feature runs without errors, but is very Yes. New tests are to be developed. > slow, uses plenty of memory etc. ? Can it leak memory? Is there a test > for all new features? What happens, if there is no test for a feature? > Is there a link from a bug to change tests? No. Postgresql does not have any bug tracking system and that kind of tracability. It is a TODO ATM. > Usually it is not just luck, that a product works. But I would like to > tell, that there is a reason, and that also the future releases are very > reliable ;-) Certainly, I would not like to tell, that as a user, you > are the tester... But of course, it is different, if we say, that as a > user, you have the ability to test the software, and verify that it > works properly ;-) Well, that's the way open source works. User is tester, user is developer if he offers work and user is support person as well because he knows by experience what works and why etc. Shridhar