Thread: What's the state of postgresql on ext4 now?
I know there were a lot of performance issues with ext4, but i don't know the state of it now.
I have a private openstreetmap server installed on a ubuntu 11.10 64bit pc with both partitions (/ and /home) formated with ext4. My problem is that the server works very slow.
> My problem is that the server works very slow. Someone may chime in with general advice, but for more details, can you be more specific? E.g., http://wiki.postgresql.org/wiki/Slow_Query_Questions --- Maciek Sakrejda | System Architect | Truviso 1065 E. Hillsdale Blvd., Suite 215 Foster City, CA 94404 (650) 242-3500 Main www.truviso.com
On 11/14/11 2:00 AM, Alexandru wrote: > I know there were a lot of performance issues with ext4, but i don't know the state of it now. > I have a private openstreetmap server installed on a ubuntu 11.10 64bit pc with both partitions (/ and /home) formatedwith ext4. My problem is that the server works very slow. Why would you assume this has anything to do with Ext4? Other configuration issues seem more likely. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com
On Tue, Nov 15, 2011 at 1:48 PM, Josh Berkus <josh@agliodbs.com> wrote: > On 11/14/11 2:00 AM, Alexandru wrote: >> I know there were a lot of performance issues with ext4, but i don't know the state of it now. >> I have a private openstreetmap server installed on a ubuntu 11.10 64bit pc with both partitions (/ and /home) formatedwith ext4. My problem is that the server works very slow. > > Why would you assume this has anything to do with Ext4? Other > configuration issues seem more likely. I think that about a year or so ago the linux kernel finally started making ext4 obey barriers and / or fsync properly, and the performance fell back into line with other file systems that also obeyed fsync. Combined with a rather histrionic report from phoronix many people were led to believe that ext4 suddenly became very slow when it fact it just stopped being artificially fast. Just a guess tho.
On 11/14/2011 05:00 AM, Alexandru wrote:
The only performance issue with ext4 is that it is careful to flush data to disk when the database asks it to. That is slow on most hard drives, and it wasn't done correctly by ext3 on older Linux kernels. So to many people this looked like a performance problem, when it was actually a reliability improvement as far as PostgreSQL is concerned. The reliable behavior just takes longer. See http://wiki.postgresql.org/wiki/Reliable_Writes for more information about why this is important. Reports about ext4 being slower by sources like Phoronix were very misinformed about what was going on.
When you turn off the synchronous_commit parameter in the postgresql.conf, the database will stop asking the filesystem to ensure things are on disk this way. You can lose some data in the event of a crash, but things will be faster. Try your system with that configuration change. If it gets rid of what you see as a performance problem, then the reliability changes made going from ext3 to ext4 are your problem. If instead the server is still slow to you even with synchronous_commit disabled, whatever is happening is unlikely to be caused by the ext4 changes. In just about every other way but commit performance, ext4 is faster than most other filesystems.
I know there were a lot of performance issues with ext4, but i don't know the state of it now.I have a private openstreetmap server installed on a ubuntu 11.10 64bit pc with both partitions (/ and /home) formated with ext4. My problem is that the server works very slow.
The only performance issue with ext4 is that it is careful to flush data to disk when the database asks it to. That is slow on most hard drives, and it wasn't done correctly by ext3 on older Linux kernels. So to many people this looked like a performance problem, when it was actually a reliability improvement as far as PostgreSQL is concerned. The reliable behavior just takes longer. See http://wiki.postgresql.org/wiki/Reliable_Writes for more information about why this is important. Reports about ext4 being slower by sources like Phoronix were very misinformed about what was going on.
When you turn off the synchronous_commit parameter in the postgresql.conf, the database will stop asking the filesystem to ensure things are on disk this way. You can lose some data in the event of a crash, but things will be faster. Try your system with that configuration change. If it gets rid of what you see as a performance problem, then the reliability changes made going from ext3 to ext4 are your problem. If instead the server is still slow to you even with synchronous_commit disabled, whatever is happening is unlikely to be caused by the ext4 changes. In just about every other way but commit performance, ext4 is faster than most other filesystems.
-- Greg Smith 2ndQuadrant US greg@2ndQuadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.us
On Tue, Nov 15, 2011 at 8:48 PM, Greg Smith <greg@2ndquadrant.com> wrote: > When you turn off the synchronous_commit parameter in the postgresql.conf, > the database will stop asking the filesystem to ensure things are on disk > this way. You can lose some data in the event of a crash, but things will > be faster. An important bit here is that unlike turning fsync off, your filesystem and database will still be coherent after a power loss event. so it's semi-safe, in that you won't be recovering your whole db in the event of a power loss / crash. > unlikely to be caused by the ext4 changes. In just about every other way > but commit performance, ext4 is faster than most other filesystems. On fast hardware, ext4 is a good performer overall and comes within a pretty close reach of the other fast file systems. And since it's in the mainline kernel and used by lots of distros, it gets a lot of real world testing and bug fixing to boot. I'm with you, if there's a real performance problem I'd suspect something other than ext4.
On Tue, Nov 15, 2011 at 10:48 PM, Greg Smith <greg@2ndquadrant.com> wrote: > In just about every other way > but commit performance, ext4 is faster than most other filesystems. As someone who is looked at as an expert and knowledgable my many of us, are you getting to the point of migrating large XFS filesystems to ext4 for production databases yet? Or at least using ext4 in new large-scale filesystems for PG? a. -- Aidan Van Dyk Create like a god, aidan@highrise.ca command like a king, http://www.highrise.ca/ work like a slave.
On 11/15/2011 10:54 PM, Aidan Van Dyk wrote: > are you getting to the point of migrating large XFS filesystems to > ext4 for production databases yet? Or at least using ext4 in new > large-scale filesystems for PG? > Not really. Last time I checked (a few months ago), there were still some scary looking bugs in ext4 that concerned me, and I've seen one difficult to explain bit of corruption on it. Lots of these bugs had been fixed in the mainline Linux kernel, but I wasn't sure they'd all been backported to the sort of distributions my customers run. I expect to re-evaluate RHEL6 to see how it's doing soon, just haven't had time/requests for it. -- Greg Smith 2ndQuadrant US greg@2ndQuadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.us