mirror of
https://github.com/nickpoida/og-aws.git
synced 2025-02-13 18:32:01 +00:00
Joshua Levy
GitHub
commit
b568f75b7b
1 changed files with 33 additions and 10 deletions
43
README.md
43
README.md
|
|
@ -31,7 +31,7 @@ Table of Contents
|
|||
| [AMIs](#amis) | [📗](#ami-basics) | [📘](#ami-tips) | [📙](#ami-gotchas-and-limitations) |
|
||||
| [Auto Scaling](#auto-scaling) | [📗](#auto-scaling-basics) | [📘](#auto-scaling-tips) | [📙](#auto-scaling-gotchas-and-limitations) |
|
||||
| [EBS](#ebs) | [📗](#ebs-basics) | [📘](#ebs-tips) | [📙](#ebs-gotchas-and-limitations) |
|
||||
| [EFS](#efs) | [📗](#efs-basics) | | |
|
||||
| [EFS](#efs) | [📗](#efs-basics) | [📘](#efs-tips) | [📙](#efs-gotchas-and-limitations) |
|
||||
| [Load Balancers](#load-balancers) | [📗](#load-balancer-basics) | [📘](#load-balancer-tips) | [📙](#load-balancer-gotchas-and-limitations) |
|
||||
| [CLB (ELB)](#clb) | [📗](#clb-basics) | [📘](#clb-tips) | [📙](#clb-gotchas-and-limitations) |
|
||||
| [ALB](#alb) | [📗](#alb-basics) | [📘](#alb-tips) | [📙](#alb-gotchas-and-limitations) |
|
||||
|
|
@ -202,7 +202,7 @@ General Information
|
|||
- [CloudTrail](https://aws.amazon.com/cloudtrail/): AWS API logging and audit (often neglected but important)
|
||||
- ⛓🕍[CloudFormation](#cloudformation): Templatized configuration of collections of AWS resources
|
||||
- 🕍[Elastic Beanstalk](https://aws.amazon.com/elasticbeanstalk/): Fully managed (PaaS) deployment of packaged Java, .NET, PHP, Node.js, Python, Ruby, Go, and Docker applications
|
||||
- 🐥⛓[EFS](https://aws.amazon.com/efs/): Network filesystem
|
||||
- 🐥[EFS](#efs): Network filesystem compatible with NFSv4.1
|
||||
- ⛓🕍[ECS](#ecs): Docker container/cluster management (note Docker can also be used directly, without ECS)
|
||||
- ⛓[ECR](https://aws.amazon.com/ecr/): Hosted private Docker registry
|
||||
- 🐥[Config](https://aws.amazon.com/config/): AWS configuration inventory, history, change notifications
|
||||
|
|
@ -858,20 +858,43 @@ EBS
|
|||
EFS
|
||||
---
|
||||
|
||||
- 📒 [Homepage](https://aws.amazon.com/efs/) ∙ [User guide](http://docs.aws.amazon.com/efs/latest/ug) ∙ [FAQ](https://aws.amazon.com/efs/faq/) ∙ [Pricing](https://aws.amazon.com/efs/pricing/)
|
||||
|
||||
### EFS Basics
|
||||
|
||||
- 📒 [Homepage](https://aws.amazon.com/efs/) ∙ [User guide](http://docs.aws.amazon.com/efs/latest/ug) ∙ [FAQ](https://aws.amazon.com/efs/faq/) ∙ [Pricing](https://aws.amazon.com/efs/pricing/)
|
||||
- 🐥**EFS** is Amazon’s new (general release 2016) network filesystem.
|
||||
- It is designed to be highly available and durable and each EFS file system object is redundantly stored across multiple availability zones.
|
||||
- It is similar to [EBS](#ebs) in that it is a network-attached drive, but it [differs in important ways](https://aws.amazon.com/efs/details/#When_to_Use_Amazon_EFS_vs._Amazon_EBS):
|
||||
- EFS file systems can automatically scale to petabytes of data without needing to provision storage.
|
||||
- EFS can be attached to many instances (up to thousands), while EBS can only be attached to one drive. It does this via [NFSv4.1](https://en.wikipedia.org/wiki/Network_File_System).
|
||||
- EFS can offer [higher throughput](http://docs.aws.amazon.com/efs/latest/ug/performance.html) (multiple gigabytes per second) and better durability and availability than EBS (see [the comparison table](#storage-durability-availability-and-price)), but with higher latency.
|
||||
- EFS cannot be used as a boot volume or in certain other ways EBS can.
|
||||
- EFS costs much more than EBS (up to [three times as much](#storage-durability-availability-and-price)).
|
||||
- EFS is designed to be used as a shared network drive and it can automatically scale up to petabytes of stored data and thousands of instances attached to it.
|
||||
- It's presented as an [NFSv4.1](https://en.wikipedia.org/wiki/Network_File_System#NFSv4) server, so any compatible NFS client can mount it.
|
||||
- EFS can offer [higher throughput](http://docs.aws.amazon.com/efs/latest/ug/performance.html) (multiple gigabytes per second) and better durability and availability than EBS (see [the comparison table](#storage-durability-availability-and-price)), but with higher latency.
|
||||
- EFS is priced based on the amount of data stored and it costs [much more than EBS](#storage-durability-availability-and-price), about three times as much compared to general purpose gp2 EBS volumes.
|
||||
- ⏱ [Performance](http://docs.aws.amazon.com/efs/latest/ug/performance.html) depends on the amount of data stored on it, which also determines the price:
|
||||
- Like EBS, EFS uses a credit based system. Credits are earned at a rate of 50 KiB/s per GiB of storage and consumed in bursts during reading/writing files or metadata. Unlike EBS, operations on metadata (file size, owner, date, etc.) also consume credits. The [BurstCreditBalance metric](http://docs.aws.amazon.com/efs/latest/ug/monitoring-cloudwatch.html#efs-metrics) in CloudWatch should be monitored to make sure the file system doesn't run out of credits.
|
||||
- Throughput capacity during bursts is also dependant on size. Under 1 TiB, throughput can go up to 100 MiB/s. Above that, 100 MiB/s is added for each stored TiB. So a file system storing 5 TiB would be able to burst at a rate of 500 MiB/s. Maximum throughput per EC2 instance is 250 MiB/s.
|
||||
- EFS has two performance modes that can only be set when a file system is created. One is "General Purpose", the other is "Max I/O". Max I/O scales higher, but at the cost of higher latency. When in doubt, use General Purpose, which is also the default. If the [PercentIOLimit metric](http://docs.aws.amazon.com/efs/latest/ug/monitoring-cloudwatch.html#efs-metrics) in CloudWatch hovers around 100%, Max I/O is recommended. Changing performance mode means creating a new EFS and migrating data.
|
||||
- High availability is achieved by having [mount targets in different subnets / availability zones](http://docs.aws.amazon.com/efs/latest/ug/images/overview-flow.png).
|
||||
|
||||
### EFS Tips
|
||||
- ⏱ Carefully consider using one EFS for multiple applications. Any directory on the EFS can be mounted, it doesn't have to be the root directory. One application could mount fs-12345678:/prog1, another fs-12345678:/prog2. [User and group level permissions](https://docs.aws.amazon.com/efs/latest/ug/accessing-fs-nfs-permissions.html) can be used to limit access to some extent.
|
||||
|
||||
Pros:
|
||||
- Because performance is based on total size of stored files, having everything on one drive will increase performance for everyone and might offset one application consuming credits faster than it can accumulate by another application that just stores files on EFS and rarely accesses them.
|
||||
|
||||
Cons:
|
||||
- Since credits are shared, if one application gets out of control it will affect the others.
|
||||
- Security of the drive is compromised. All clients will have to have network access to the drive and any of the clients can mount another application's folder. Someone with root access on one client instance can easily read and write to any application's mount.
|
||||
|
||||
### EFS Gotchas and Limitations
|
||||
|
||||
- 🔸 A number of NFSv4.1 features are [not supported](http://docs.aws.amazon.com/efs/latest/ug/nfs4-unsupported-features.html) and there are some [limits](http://docs.aws.amazon.com/efs/latest/ug/limits.html) to the service.
|
||||
- 🔸 As of 2016-11, EFS does not offer disk level encryption, though it is on the roadmap.
|
||||
- ❗ Some applications, like SQLite and IPython, [might not work properly](https://sqlite.org/faq.html#q5) on EFS when accessed from multiple clients. This is because lock upgrades and downgrades are [not supported](http://docs.aws.amazon.com/efs/latest/ug/nfs4-unsupported-features.html). There might be [workarounds](https://github.com/ipython/ipython/issues/882) for some issues.
|
||||
- 🔸 Mounting EFS over VPN connection, VPC peering, or AWS Direct Connect is not supported.
|
||||
- 🔸 Using an EFS volume on Windows is not supported, apparently due to Microsoft implementing NFS differently.
|
||||
- ⏱ When a file is uploaded to EFS, it can take hours for EFS to update the details for billing and burst credit purposes.
|
||||
- 🔸 ⏱ Metadata operations can be costly in terms of burst credit consumption. Recursively traversing a tree containing thousands of files can easily ramp up to tens or even hundreds of megabytes of burst credits being consumed, even if no file is being touched. Commands like ```find``` or ```chown -R``` can have an adverse impact on performace if run periodically.
|
||||
- 🔸 Mount points are AZ-based. In an Auto scaling group spread across zones, you can end up with instances in one zone mounting EFS from a different zone. That might decrease performance and would create an unintended single point of failure. One way to fix it would be [a shell script](https://gist.github.com/bgdnlp/9fd326dc4a23f46bab93a1eade023fe4) that runs before network drives are mounted and edits /etc/fstab with the proper AZ.
|
||||
|
||||
🚧 [*Please help expand this incomplete section.*](CONTRIBUTING.md)
|
||||
|
||||
Load Balancers
|
||||
--------------
|
||||
|
|
|
|||
Loading…
Reference in a new issue