Thread: 10GbE / iSCSI storage for postgresql.
Hi , Can PostgreSQL run fast ( within 80% of DAS) with iSCSI sotrage connected via 10GbE ? regds mallah.
On 09/22/2011 03:49 AM, Rajesh Kumar Mallah wrote: > Hi , > > Can PostgreSQL run fast ( within 80% of DAS) with iSCSI sotrage > connected via 10GbE ? "Maybe". What's that 80% of? Sequential read throughput? Random IOPS? Individual read latency? What's the expected workload? Read-heavy, write-heavy, or middle-ground? Data warehouse/OLAP or OLTP? Lots of small simple transactions, or fewer big complex transactions? Does the system on the other end of the iSCSI link have battery-backed write caching, flash-logged write cache, or some other way to guarantee writes are persistent without having to wait for data to flush out to spinning disks? You'll need something like this for decent write performance especially if you're doing lots of small transactions. If the SAN doesn't have a safe way to cache writes you can partly work around the issue by doing fewer bigger transactions and/or by using a commit_delay. What kind of read cache does the SAN have? How much contention with other users will there be? How big is its write-back cache (if it has one)? Does it have any kind of QoS to prevent something like someone disk-imaging a server from starving your Pg instance of read bandwidth? -- Craig Ringer
Dear Craig , The other end of the iSCSI shall have all the goodies like the raid controller with a WBC with BBU. There can even be multiple raid cards for multiple servers and disksets. I am even planning for NICs having TOE features . The doubt is will it work withing a acceptable performance range as compared to the situation of DAS (Direct Attached Storage). Has anyone tried like this before ? regds mallah. On Thu, Sep 22, 2011 at 9:44 AM, Craig Ringer <ringerc@ringerc.id.au> wrote: > On 09/22/2011 03:49 AM, Rajesh Kumar Mallah wrote: >> >> Hi , >> >> Can PostgreSQL run fast ( within 80% of DAS) with iSCSI sotrage >> connected via 10GbE ? > > "Maybe". > > What's that 80% of? Sequential read throughput? Random IOPS? Individual read > latency? > > What's the expected workload? Read-heavy, write-heavy, or middle-ground? > Data warehouse/OLAP or OLTP? Lots of small simple transactions, or fewer big > complex transactions? > > Does the system on the other end of the iSCSI link have battery-backed write > caching, flash-logged write cache, or some other way to guarantee writes are > persistent without having to wait for data to flush out to spinning disks? > You'll need something like this for decent write performance especially if > you're doing lots of small transactions. If the SAN doesn't have a safe way > to cache writes you can partly work around the issue by doing fewer bigger > transactions and/or by using a commit_delay. > > What kind of read cache does the SAN have? How much contention with other > users will there be? How big is its write-back cache (if it has one)? Does > it have any kind of QoS to prevent something like someone disk-imaging a > server from starving your Pg instance of read bandwidth? > > -- > Craig Ringer >
On 22/09/2011 5:47 PM, Rajesh Kumar Mallah wrote: > Dear Craig , > > The other end of the iSCSI shall have all the goodies like the raid controller > with a WBC with BBU. There can even be multiple raid cards for multiple > servers and disksets. I am even planning for NICs having TOE features . > > The doubt is will it work withing a acceptable performance range as > compared to the situation of DAS (Direct Attached Storage). Has anyone tried like this before ? Sure, people use iSCSI and similar relatively frequently, and as I said it depends a lot on the controller (client- and server-side), the workload, and the details of the implementation. If the iSCSI storage is fast, PostgreSQL will be fast. If the iSCSI storage has slow writes, PostgreSQL will have slow writes. And so on. -- Craig Ringer