Thread: Testing in AWS, EBS

We are starting some testing in AWS, with EC2, EBS backed setups.

What I found interesting today, was a single EBS 1TB volume, gave me
something like 108MB/s throughput, however a RAID10 (4 250GB EBS
volumes), gave me something like 31MB/s (test after test after test).

I'm wondering what you folks are using inside of Amazon (not
interested in RDS at the moment).

Thanks
Tory

Indeed, old-style disk EBS vs new-style SSd EBS.

Be aware that EBS traffic is considered as part of the total "network"
traffic, and each type of instance has different limits on maximum network
throughput. Those difference are very significant, do tests on the same volume
between two different type of instances, both with enough cpu and memory for
the I/O to be the bottleneck, you will be surprised!


On 2016-05-25 17:02, Rayson Ho wrote:
> There are many factors that can affect EBS performance. For example, the type
> of EBS volume, the instance type, whether EBS-optimized is turned on or not, etc.
>
> Without the details, then there is no apples to apples comparsion...
>
> Rayson
>
> ==================================================
> Open Grid Scheduler - The Official Open Source Grid Engine
> http://gridscheduler.sourceforge.net/
> http://gridscheduler.sourceforge.net/GridEngine/GridEngineCloud.html
>
>
>
> On Wed, May 25, 2016 at 6:34 PM, Tory M Blue <tmblue@gmail.com
> <mailto:tmblue@gmail.com>> wrote:
>>
>> We are starting some testing in AWS, with EC2, EBS backed setups.
>>
>> What I found interesting today, was a single EBS 1TB volume, gave me
>> something like 108MB/s throughput, however a RAID10 (4 250GB EBS
>> volumes), gave me something like 31MB/s (test after test after test).
>>
>> I'm wondering what you folks are using inside of Amazon (not
>> interested in RDS at the moment).
>>
>> Thanks
>> Tory
>>
>>
>> --
>> Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org
> <mailto:pgsql-performance@postgresql.org>)
>> To make changes to your subscription:
>> http://www.postgresql.org/mailpref/pgsql-performance


--
http://yves.zioup.com
gpg: 4096R/32B0F416

On 2016-05-25 19:08, Rayson Ho wrote:
> Actually, when "EBS-Optimized" is on, then the instance gets dedicated
> bandwidth to EBS.

Hadn't realised that, thanks.
Is the EBS bandwidth then somewhat limited depending on the type of instance too?

--
http://yves.zioup.com
gpg: 4096R/32B0F416

Thanks Artem.

So no EBS pre-warming does not apply to EBS volumes created from snapshots.

Rayson

==================================================
Open Grid Scheduler - The Official Open Source Grid Engine
http://gridscheduler.sourceforge.net/
http://gridscheduler.sourceforge.net/GridEngine/GridEngineCloud.html


On Thu, May 26, 2016 at 10:52 AM, Artem Tomyuk <admin@leboutique.com> wrote:
> Please look at the official doc.
>
> "New EBS volumes receive their maximum performance the moment that they are
> available and do not require initialization (formerly known as pre-warming).
> However, storage blocks on volumes that were restored from snapshots must be
> initialized (pulled down from Amazon S3 and written to the volume) before
> you can access the block"
>
> Quotation from:
> http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ebs-initialize.html
>
> 2016-05-26 17:47 GMT+03:00 Rayson Ho <raysonlogin@gmail.com>:
>>
>> On Thu, May 26, 2016 at 10:00 AM, Artem Tomyuk <admin@leboutique.com>
>> wrote:
>>>
>>>
>>> 2016-05-26 16:50 GMT+03:00 Rayson Ho <raysonlogin@gmail.com>:
>>>>
>>>> Amazon engineers said that EBS pre-warming is not needed anymore.
>>>
>>>
>>> but still if you will skip this step you wont get much performance on ebs
>>> created from snapshot.
>>
>>
>>
>> IIRC, that's not what Amazon engineers said. Is that from your personal
>> experience, and if so, when did you do the test??
>>
>> Rayson
>>
>> ==================================================
>> Open Grid Scheduler - The Official Open Source Grid Engine
>> http://gridscheduler.sourceforge.net/
>> http://gridscheduler.sourceforge.net/GridEngine/GridEngineCloud.html
>>
>>
>>
>

On 2016-05-26 09:03, Artem Tomyuk wrote:
> Why no? Or you missed something?

I think Rayson is correct, but the double negative makes it hard to read:

"So no EBS pre-warming does not apply to EBS volumes created from snapshots."

Which I interpret as:
So, "no EBS pre-warming", does not apply to EBS volumes created from snapshots.

Which is correct, you sitll have to warm your EBS when created from sanpshots (to get the data from S3 to the
filesystem).


--
http://yves.zioup.com
gpg: 4096R/32B0F416

Thanks Yves for the clarification!

It used to be very important to pre-warm EBS before running benchmarks
in order to get consistent results.

Then at re:Invent 2015, the AWS engineers said that it is not needed
anymore, which IMO is a lot less work for us to do benchmarking in
AWS, because pre-warming a multi-TB EBS vol is very time consuming,
and the I/Os were not free.

Rayson

==================================================
Open Grid Scheduler - The Official Open Source Grid Engine
http://gridscheduler.sourceforge.net/
http://gridscheduler.sourceforge.net/GridEngine/GridEngineCloud.html


On Thu, May 26, 2016 at 11:41 AM, Yves Dorfsman <yves@zioup.com> wrote:
> On 2016-05-26 09:03, Artem Tomyuk wrote:
>> Why no? Or you missed something?
>
> I think Rayson is correct, but the double negative makes it hard to read:
>
> "So no EBS pre-warming does not apply to EBS volumes created from snapshots."
>
> Which I interpret as:
> So, "no EBS pre-warming", does not apply to EBS volumes created from snapshots.
>
> Which is correct, you sitll have to warm your EBS when created from sanpshots (to get the data from S3 to the
filesystem).
>
>
> --
> http://yves.zioup.com
> gpg: 4096R/32B0F416
>
>
>
> --
> Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-performance