Trekintech

Beginners Guide to AWS EBS Storage

Earlier this year I wrote a blog post covering an introduction to AWS S3 storage so felt it was only apt to do the same for EBS. Whilst I appreciate block storage is more commonly understood I feel it's still worth covering off the basics for those that are new to EBS.

What EBS Is

EBS (Elastic Block Storage) is AWS's Block based storage, it is provisioned against an EC2 instance (Elastic Cloud Compute which you can think of as a virtual server) and is the ideal storage platform for your legacy applications given that it is highly likely these applications are not written for S3 object storage.

If you're not entirely sure what block storage is just think SAN - effectively block storage is a raw volume presented directly to an application or O/S, from there each block of data can be formatted. Transfer of data to block storage is typically provided by ISCSI or FC to transport SCSI commands.

AWS provides resiliency of an EBS volume by replicating data amongst multiple servers within the same availability zone. EBS is SLA'd to 99.95% uptime and has an annual failure rate of 0.1% - 0.2% meaning you should expect to lose 1 to 2 volumes for every 1000 that you provision.

You can also use an O/S's software RAID on top of the native EBS resiliency which has the added benefit of increasing IOPs.

What EBS Is Not

EBS does not allow for the sharing of data across multiple EC2 instances, an EBS volume can be mounted to only one EC2 instance at a time. If you need shared storage then you should be looking at Amazon EFS (Elastic File System) which is currently in preview or something like NetApp Cloud ONTAP which provides enterprise storage capabilities such as storage efficiency, multiple protocols and efficient replication.

EBS does not have much in the way of data management features that you have most likely become accustomed to with an enterprise storage array. Features such as thin provisioning and other storage efficiency techniques do not exist - if you provision 100gb , you pay for 100gb.

Flavours of EBS

Amazon have recently expanded out their EBS offerings in order to meet a wider range of use cases. Understanding which is correct for your application is very important in order to avoid unnecessary charges or a format that doesn't provide the performance required. Below is a comparison of the current offerings

Max IOPs is based on 16 Kib block size for gp2 and io1 and 1 Mib for st1 and sc1.

It is worth noting that the performance of 10,000 IOPS on gp2 is available to any volume larger than 3333 Gib , anything below that has a baseline performance number of 3 IOPS per Gib measured in single digit latency. Volumes below 3333 Gib have burst capability upto 3,000 IOPs which works on a credit system detailed here.

Also available is magnetic storage which Amazon appear to be pushing to the back of any marketing information calling it previous generation. Magnetic volumes have a maximum size of 1024GB and offer 40-200 IOPS and 40-90MB/s.

It's worth understanding that you will not be able to achieve those maximum performance numbers on any EC2 instance. EBS optimised instances exist for a reason - they have the compute and network available to achieve the quoted performance. Sign up for a t2.micro and you are not going to be getting anywhere near the quoted numbers. Some instances are EBS optimised by default however for those that aren't there is an additional hourly charge. Amazon state that with EBS optimised instances gp2 (General Purpose SSD) is designed to deliver within 10% of their provisioned performance 99.9% of the time. Optimised HDD guarantee performance consistency of 90% burst throughput 99% of the time.

Encryption

Data encryption works by encrypting at the server level meaning that data is encrypted both at rest and whilst in transit from EBS to the EC2 instance. All snapshots that are created on an encrypted volume are also by nature encrypted.

Both encrypted and unencrypted volumes can reside together on a single EC2 instance however there is no simple way to change a volume from encrypted to unencrypted or vice versa. If you wish to remove or add encryption data needs to be moved at the host level using rsync or robocopy.

AWS has their own key management for EBS encryption uses the AES 256 algorithm and is FIPS 140-2 approved.

SnapShots

Snapshots work on an incremental forever basis, the first is a full and from there on only data that is new or changed will be copied. Snapshots are automatically copied into S3 which has the added benefit of providing a separate fault domain with 11 9's of durability.

When a snapshot is taken of an EBS root volume the EC2 instance will need to be stopped. For additional volumes it is recommended that IO to the volume is paused or the volume is unmounted to ensure that data is consistent.

Snapshots can be used to move data between regions, selecting a snapshot and clicking copy under actions shows just how simple this is.

Snapshots can also be publicly shared , kept private or shared to a specific AWS account number.

Restoring to a snapshot is straight forward, volumes can be created from a snapshot and attached to an EC2 instance. Snapshots are made available to the instance immediately whilst data is loaded "lazily" in the background from S3. If data that is being requested is not yet available it is immediately requested. Because of this performance of a volume when restoring from a snapshot is impacted , latency can spike. If you need to avoid this impact there is an initialisation process which forces all the blocks on the volume to be read before putting this volume into production. This process uses dd or fio and is documented here.

Snapshots can also be used to create images for quick provisioning

Pricing

Pricing is based on the amount of storage you provision each month, prorated to the hour. This is true for all flavours except for Provisioned IOPS (io1) which charges based on the amount of IOPs you provision during this time aswell.

You are also charged for snapshots at the standard S3 rate aswell as the transfer charges for cross region copies. Also remember, if your EC2 instance isn't EBS optimised by default there will be a small additional charge for that also.

All charges can be found here and there is also the AWS Simple Monthly Calculator.

Author image
About Mike Andrews
Manchester, England Website
Senior System Engineer currently working at Cohesity focussed on Data Management. I have a keen interest in cloud and have worked in the IT industry for 13 years. My views are my own.