Thread: Pre-Allocate tablespace on disk
Hello, I was wondering if there is a way to pre-allocate tablespace on disk before adding data and indexes. My understanding is: PG writes data into files sequentially. If more space is needed, disk space is requested from the OS and if there is space,the OS will give PG a file system block and PG will write data/indexes until that block is full and so on. Other database that I have worked with before and that I'm still working with, allow you to pre-allocate disk space so youget huge chunks of contiguous space at one, which has major impacts on database performance. Can this be done with PG at all? If not, is this planned for any time in the future? best regards, chris -- chris ruprecht database grunt and bit pusher extraordinaíre
Le 15/11/2010 20:42, Chris Ruprecht a écrit : > Hello, > > I was wondering if there is a way to pre-allocate tablespace on disk before adding data and indexes. > My understanding is: > PG writes data into files sequentially. If more space is needed, disk space is requested from the OS and if there is space,the OS will give PG a file system block and PG will write data/indexes until that block is full and so on. > Other database that I have worked with before and that I'm still working with, allow you to pre-allocate disk space soyou get huge chunks of contiguous space at one, which has major impacts on database performance. > > Can this be done with PG at all? No. A tablespace is a directory on PostgreSQL. And you can't allocate space for a directory. > If not, is this planned for any time in the future? AFAICT, nope. -- Guillaume http://www.postgresql.fr http://dalibo.com
On 11/15/10 11:42 AM, Chris Ruprecht wrote: > I was wondering if there is a way to pre-allocate tablespace on disk before > adding data and indexes. > My understanding is: > PG writes data into files sequentially. If more space is needed, disk space > is requested from the OS and if there is space, the OS will give PG a file > system block and PG will write data/indexes until that block is full and so on. > Other database that I have worked with before and that I'm still working with, > allow you to pre-allocate disk space so you get huge chunks of contiguous > space at one, which has major impacts on database performance. This shouldn't really be a performance issue on modern Unix-type systems. Most file systems have fairly smart block-allocationalgorithms that tend to allocate large blocks of contiguous space to files even when lots of processes areasking for little chunks in interleaved requests. It doesn't just allocate a block to the next guy who happens to ask. It's a lot smarter than that. In most cases, you'll barely be able to tell the difference between a file that was allocatedall at once and one that was allocated on an as-needed basis. The only time you get bad fragmentation is when yourdisk gets nearly full, which takes away the file system's ability to be clever. If you really want to preallocate a file, build a separate partition on your disk and only put one database (or even justone table) on that file system.
Hello I have migrate a application from a server with postgres 8.1 to another with 8.3 in the new server there are some page that doesn't work because seems to want to compare different data type does anyone know if it is a version issue or there is something Im missing? Thanks Octavio the error in spanish 2010-11-15 19:03:21 PET UBICACIÃN: op_error, parse_oper.c:896 2010-11-15 19:03:21 PET SENTENCIA: update clamcartga set codusuario=$1,fechapro=$2,horapro=$3,fechaenv=$4,flenvmail=$5 wh ere corrabon=$6 and tipoafil=$7 and numcaso=$8 and corpacien=$9 and tipcarta=$10 and corrcart=$11 2010-11-15 19:06:16 PET ERROR: 42883: el operador no existe: character = integer en carácter 383 2010-11-15 19:06:16 PET HINT: Ningún operador coincide con el nombre y el tipo de los argumentos. Puede desear agregar convesiones explÃcitas de tipos. |
What does a \d on the table ‘clamcartga’ in both environments say? Can you spot any differences between the 8.1 and 8.3 environments?
Sam
From: pgsql-admin-owner@postgresql.org [mailto:pgsql-admin-owner@postgresql.org] On Behalf Of Octavio
Sent: Tuesday, 16 November 2010 3:58 PM
To: pgsql-admin@postgresql.org
Subject: [ADMIN] ignoring data type
Hello |
On Mon, Nov 15, 2010 at 10:28 PM, Octavio <octaviomaiden@yahoo.com> wrote: > > Hello > > I have migrate a application from a server with postgres 8.1 to another with 8.3 > in the new server there are some page that doesn't work because seems to want to compare different data type > > does anyone know if it is a version issue or there is something Im missing? Starting in 8.3 many automatic conversions from one type to another were removed because it wasn't always obvious if this was the right thing to do. So, you might need to either change some column types, or cast them explicitly to the type you want to be used for comparison.
Thanks Scott In fact checking the application there are a lots of queries that doesn't make the conversion, in order to temporally skip this issue,I like to try to import those conversions from the 8.1 version to the actual postgres, how can I do that? Octavio --- El mar, 11/16/10, Scott Marlowe <scott.marlowe@gmail.com> escribió:
|
* Chris Ruprecht: > Other database that I have worked with before and that I'm still > working with, allow you to pre-allocate disk space so you get huge > chunks of contiguous space at one, which has major impacts on > database performance. PostgreSQL's write patterns do not trigger significant fragmentation with most file systems, even when other database systems (Oracle Berkeley DB comes to my mind) would create heavily fragmented files. -- Florian Weimer <fweimer@bfk.de> BFK edv-consulting GmbH http://www.bfk.de/ Kriegsstraße 100 tel: +49-721-96201-1 D-76133 Karlsruhe fax: +49-721-96201-99
Not only that, but if your "disk" is really a bunch of disks behind a hardware RAID controller, or SAN, or iSCSI or NFSserver - what is contiguous for the OS may not be contiguous on the physical drives (and vice versa). <br /> Which iswhy performance can (but may not) improve if you disable linux kernel IO schedulers that attempt to optimize blocks tobe sequential.<br /><br /><br /><br /> On 11/19/2010 5:01 AM, Florian Weimer wrote: <blockquote cite="mid:82zkt55ux4.fsf@mid.bfk.de"type="cite"><pre wrap="">* Chris Ruprecht: </pre><blockquote type="cite"><pre wrap="">Other database that I have worked with before and that I'm still working with, allow you to pre-allocate disk space so you get huge chunks of contiguous space at one, which has major impacts on database performance. </pre></blockquote><pre wrap=""> PostgreSQL's write patterns do not trigger significant fragmentation with most file systems, even when other database systems (Oracle Berkeley DB comes to my mind) would create heavily fragmented files. </pre></blockquote><br /><br /><div class="moz-signature">-- <br /><style type="text/css"> .sig, .sig a, .sig td { font:12px Arial; color: DarkBlue; white-space: nowrap; vertical-align: top; text-align: left; } .sig img { border: 0; } .siga { font: 11px Arial; } .sig .back { color: #ffffff; background-color: DarkBlue; text-decoration: none; } .sig .right{ text-align: right; } .sig .border { border: 1px solid DarkBlue; margin:5px; border-collapse: collapse; } .sig .bottom{ vertical-align:bottom } .sig p { clear:both; margin:2px; } </style><div class="sig"><table class="border"><tbody><tr><tdclass="back"> <b>Steve Francis</b></td><td class="back" style="width: 30px;"><br /></td><tdclass="back" style="text-align: right;"><b>LogicMonitor LLC</b></td></tr><tr><td><br /></td><td><br /></td><tdclass="right"><br /></td></tr><tr><td class="bottom"><a class="moz-txt-link-abbreviated" href="mailto:sfrancis@logicmonitor.com">sfrancis@logicmonitor.com</a><br/><a href="http://www.logicmonitor.com" target="_blank">www.logicmonitor.com</a></td><td><br/></td><td class="bottom" style="text-align: right;"> Ph: 1 888 41 LOGICx500<br /> Ph: 1 805 698 0770</td></tr></tbody></table><p> <br /></div></div>
Chris Ruprecht <chris@ruprecht.org> writes: > I was wondering if there is a way to pre-allocate tablespace on disk > before adding data and indexes. As others have already said, it's not clear there's a point in doing that with PostgreSQL. Now, you can achieve it and see for yourself by relying on OS tools, such as LVM of ZFS, or some other tricks. For example, create a big enough file, loop mount it as a device, and create your tablespace in there. Of course what to do exactly when the file is full is left as an exercise to the reader. Bonus points for any solution not involving downtime. Baseline: it seems that PostgreSQL works differently enough from your usual software than you may want to reconsider your habits :) Regards, -- Dimitri Fontaine http://2ndQuadrant.fr PostgreSQL : Expertise, Formation et Support