From 82523efc0dc319a26c2e95c2be76eb34ce68f1ee Mon Sep 17 00:00:00 2001 From: Joshua Levy Date: Tue, 11 Oct 2016 10:18:51 -0700 Subject: [PATCH 01/10] More on credentials. Start at #94. --- README.md | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/README.md b/README.md index 0873645..7f867b9 100644 --- a/README.md +++ b/README.md @@ -17,6 +17,7 @@ Table of Contents **AWS in General** - [General Information](#general-information) +- [Learning and Career Development](#learning-aws) - [Managing AWS](#managing-aws) - [Managing Servers and Applications](#managing-servers-and-applications) @@ -245,6 +246,7 @@ We’ve assembled a landscape of a few of the services. This is far from complet 🚧 *Suggestions to improve this figure? Please [file an issue](CONTRIBUTING.md).* + ### Common Concepts - 📒 The AWS [**General Reference**](https://docs.aws.amazon.com/general/latest/gr/Welcome.html) covers a bunch of common concepts that are relevant for multiple services. @@ -383,6 +385,19 @@ It’s important to know the maturity of each AWS product. Here is a mostly comp - [OpenStack](https://www.openstack.org/) is a private cloud alternative to AWS used by large companies that wish to avoid public cloud offerings. +Learning and Career Development +------------------------------- + +### Certifications + +- **Certifications:** AWS offers [**certifications**](https://aws.amazon.com/certification/) for IT professionals who want to demonstrate their knowledge. They are: + - Certified Solutions Architect [Associate](https://aws.amazon.com/certification/certified-solutions-architect-associate/) and [Professional](https://aws.amazon.com/certification/certified-solutions-architect-professional/), + [Certified Developer Associate](https://aws.amazon.com/certification/certified-developer-associate/) + - [Certified SysOps Administrator Associate](https://aws.amazon.com/certification/certified-sysops-admin-associate/) + - [Certified Certified DevOps Engineer Professional](https://aws.amazon.com/certification/certified-devops-engineer-professional/) +- **Getting certified:** If you’re interested in studying for and getting certifications, [this practical overview](https://gist.github.com/leonardofed/bbf6459ad154ad5215d354f3825435dc) tells you a lot of what you need to know. The official page is [here](https://aws.amazon.com/training/) and there is an [FAQ](https://aws.amazon.com/certification/faqs/). +- **Do you need a certification?** Especially in consulting companies or when working in key tech roles in large non-tech companies, certifications are important credentials. In others, including in many tech companies and startups, certifications are not common or considered necessary. (In fact, fairly or not, some Silicon Valley hiring managers and engineers see them as a “negative” signal on a resume.) + Managing AWS ------------ From 69dbfc022c78d7ea6b5ee574c5b419304e0736a7 Mon Sep 17 00:00:00 2001 From: danhermann Date: Tue, 11 Oct 2016 16:46:56 -0500 Subject: [PATCH 02/10] added data migration tips for RDS Aurora --- README.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index b87746e..c267489 100644 --- a/README.md +++ b/README.md @@ -1,4 +1,4 @@ -![An Open Guide](figures/signpost-horiz1-1600.jpg) +![An Open Guide](figures/signpost-horiz1-1600.jpg) The Open Guide to Amazon Web Services ===================================== @@ -935,6 +935,7 @@ RDS - If you’re looking for the managed convenience of RDS for MongoDB, this isn’t offered by AWS directly, but you may wish to consider a provider such as [**mLab**](https://mlab.com/). - MySQL RDS allows access to [binary logs](http://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_LogAccess.Concepts.MySQL.html#USER_LogAccess.MySQL.BinaryFormat). - 🔸**MySQL vs MariaDB vs Aurora:** If you prefer a MySQL-style database but are starting something new, you probably should consider Aurora and MariaDB as well. **Aurora** has increased availability and is the next-generation solution. That said, Aurora [may not be](http://blog.takipi.com/benchmarking-aurora-vs-mysql-is-amazons-new-db-really-5x-faster/) as fast relative to MySQL as is sometimes reported, and is more complex to administer. **MariaDB**, the modern [community fork](https://en.wikipedia.org/wiki/MariaDB) of MySQL, [likely now has the edge over MySQL](http://cloudacademy.com/blog/mariadb-vs-mysql-aws-rds/) for many purposes and is supported by RDS. +- 🔸**Aurora:** Aurora is based on MySQL 5.6. If you are planning to migrate to Aurora from an existing MySQL database, avoiding any MySQL features from 5.7 or later will ease the transition. The easiest migration path to Aurora is restoring a database snapshot from MySQL 5.6. The next easiest method is restoring a dump from a MySQL-compatible database such as MariaDB. If neither of those methods are options, Amazon offers a [fee-based data migration service](http://docs.aws.amazon.com/dms/latest/userguide/Welcome.html). ### RDS Gotchas and Limitations From 5edafa05ca3a3771b2adc7277f43d8af956b1dfb Mon Sep 17 00:00:00 2001 From: Magnus Kulke Date: Wed, 12 Oct 2016 01:00:19 +0200 Subject: [PATCH 03/10] Added ALB Gotcha about unhealthy targets --- README.md | 1 + 1 file changed, 1 insertion(+) diff --git a/README.md b/README.md index 94dbace..70349ad 100644 --- a/README.md +++ b/README.md @@ -879,6 +879,7 @@ ALB - ALBs do not (yet) support routing based on HTTP “Host” header or HTTP verb. - Instances in the ALB's target groups have to either have a single, fixed healthcheck port (“EC2 instance”-level healthcheck) or the healthcheck port for a target has to be the same as its application port (“Application instance”-level healthcheck) - you can't configure a per-target healthcheck port that is different than the application port. - ALBs are VPC-only (they are not available in EC2 Classic) +- In a target group, if there is no healthy target, all requests are routed to all targets. An example: you add a target group containing a single service (with a long init phase maybe) as target to a listener, it does not pass health checks (yet), requests will still reach your (initializing) service. Elastic IPs ----------- From 80d80b3fb332e1ff17dc767d929194e98dd0beb6 Mon Sep 17 00:00:00 2001 From: Magnus Kulke Date: Wed, 12 Oct 2016 01:11:55 +0200 Subject: [PATCH 04/10] Addressed review comments. --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 70349ad..fd6dfcb 100644 --- a/README.md +++ b/README.md @@ -879,7 +879,7 @@ ALB - ALBs do not (yet) support routing based on HTTP “Host” header or HTTP verb. - Instances in the ALB's target groups have to either have a single, fixed healthcheck port (“EC2 instance”-level healthcheck) or the healthcheck port for a target has to be the same as its application port (“Application instance”-level healthcheck) - you can't configure a per-target healthcheck port that is different than the application port. - ALBs are VPC-only (they are not available in EC2 Classic) -- In a target group, if there is no healthy target, all requests are routed to all targets. An example: you add a target group containing a single service (with a long init phase maybe) as target to a listener, it does not pass health checks (yet), requests will still reach your (initializing) service. +- In a target group, if there is no healthy target, all requests are routed to all targets. For example, if you point a listener at a target group containing a single service that has a long initialization phase (during which the health checks would fail), requests will reach the service while it is still starting up. Elastic IPs ----------- From fc4c6185e6c5746cdf99bc74e28f33c24d0b758e Mon Sep 17 00:00:00 2001 From: chris-griffin Date: Tue, 11 Oct 2016 20:24:43 -0400 Subject: [PATCH 05/10] Update SSL/TLS info for static website hosting --- README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 115f843..e437658 100644 --- a/README.md +++ b/README.md @@ -609,9 +609,9 @@ S3 - S3 has a [static website hosting option](http://docs.aws.amazon.com/AmazonS3/latest/dev/WebsiteHosting.html) that is simply a setting that enables configurable HTTP index and error pages and [HTTP redirect support](http://docs.aws.amazon.com/AmazonS3/latest/dev/how-to-page-redirect.html) to [public content](http://docs.aws.amazon.com/AmazonS3/latest/dev/WebsiteAccessPermissionsReqd.html) in S3. It’s a simple way to host static assets or a fully static website. - Consider using CloudFront in front of most or all assets: - Like any CDN, CloudFront improves performance significantly. - - 🔸SSL is only supported on the built-in amazonaws.com domain. S3 does support serving these sites through a [custom domain](http://docs.aws.amazon.com/AmazonS3/latest/dev/website-hosting-custom-domain-walkthrough.html), but [not over SSL on a custom domain](http://stackoverflow.com/questions/11201316/how-to-configure-ssl-for-amazon-s3-bucket). + - 🔸SSL is only supported on the built-in amazonaws.com domain for S3. S3 supports serving these sites through a [custom domain](http://docs.aws.amazon.com/AmazonS3/latest/dev/website-hosting-custom-domain-walkthrough.html), but [not over SSL on a custom domain](http://stackoverflow.com/questions/11201316/how-to-configure-ssl-for-amazon-s3-bucket). However, [CloudFront allows you to serve a custom domain over https](http://docs.aws.amazon.com/acm/latest/userguide/gs-cf.html). Amazon provides free SNI SSL/TLS certificates via Amazon Certificate Manager. [SNI does not work on very outdated browsers/operating systems](https://en.wikipedia.org/wiki/Server_Name_Indication#Support). Alternatively, you can provide your own certificate to use on CloudFront to support all browsers/operating systems. - 🔸If you are including resources across domains, such as fonts inside CSS files, you may need to [configure CORS](https://docs.aws.amazon.com/AmazonS3/latest/dev/cors.html) for the bucket serving those resources. - - Since pretty much everything is moving to SSL nowadays, and you likely want control over the domain, you probably want to set up CloudFront your own certificate in front of S3 (and to ignore the [AWS example on this](http://docs.aws.amazon.com/AmazonS3/latest/dev/website-hosting-custom-domain-walkthrough.html) as it is non-SSL only). + - Since pretty much everything is moving to SSL nowadays, and you likely want control over the domain, you probably want to set up CloudFront with your own certificate in front of S3 (and to ignore the [AWS example on this](http://docs.aws.amazon.com/AmazonS3/latest/dev/website-hosting-custom-domain-walkthrough.html) as it is non-SSL only). - That said, if you do, you’ll need to think through invalidation or updates on CloudFront. You may wish to [include versions or hashes in filenames](https://abhishek-tiwari.com/post/CloudFront-design-patterns-and-best-practices) so invalidation is not necessary. - **Permissions:** - 🔸It’s important to manage permissions sensibly on S3 if you have data sensitivities, as fixing this later can be a difficult task if you have a lot of assets and internal users. From b373c80080dee9a9a52a583718075ab8fae8bea1 Mon Sep 17 00:00:00 2001 From: David Schott Date: Tue, 11 Oct 2016 20:30:49 -0400 Subject: [PATCH 06/10] EFS is GA, not Preview Per: https://aws.amazon.com/about-aws/whats-new/2016/06/amazon-elastic-file-system-efs-is-now-generally-available/ --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 115f843..f6f7d1a 100644 --- a/README.md +++ b/README.md @@ -310,7 +310,7 @@ It’s important to know the maturity of each AWS product. Here is a mostly comp | 🐥[CodeCommit](https://aws.amazon.com/releasenotes/AWS-CodeCommit?browse=1) | 2015-07 | General | | 🐥[API Gateway](https://aws.amazon.com/releasenotes/Amazon-API-Gateway?browse=1) | 2015-07 | General | | 🐥[Config](https://aws.amazon.com/releasenotes/AWS-Config?browse=1) | 2015-06 | General | -| 🐥[EFS](https://aws.amazon.com/releasenotes/Amazon-EFS?browse=1) | 2015-05 | Preview | +| 🐥[EFS](https://aws.amazon.com/releasenotes/Amazon-EFS?browse=1) | 2015-05 | General | | 🐥[Machine Learning](https://aws.amazon.com/releasenotes/AmazonML?browse=1) | 2015-04 | General | | [Lambda](https://aws.amazon.com/releasenotes/AWS-Lambda?browse=1) | 2014-11 | General | | [ECS](https://aws.amazon.com/ecs/release-notes/) | 2014-11 | General | From 825d435f9c3761b1c1ddf16ef6385838a9cdeefc Mon Sep 17 00:00:00 2001 From: Michael Ortali Date: Tue, 11 Oct 2016 21:56:27 -0400 Subject: [PATCH 07/10] Update README.md --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index ce607f0..6c89ea8 100644 --- a/README.md +++ b/README.md @@ -210,7 +210,7 @@ General Information - **Compound services:** These are similarly specific, but are full-blown services that tackle complex problems and may tie you in. Usefulness depends on your requirements. If you have large or significant need, you may have these already managed by in-house systems and engineering teams. - [Machine Learning](https://aws.amazon.com/machine-learning/): Machine learning model training and classification - ⛓🕍[Data Pipeline](https://aws.amazon.com/datapipeline/): Managed ETL service - - ⛓🕍[SWF](https://aws.amazon.com/swf/): Managed background job workflow + - ⛓🕍[SWF](https://aws.amazon.com/swf/): Managed state tracker for distributed polyglot job workflow - ⛓🕍[Lumberyard](https://aws.amazon.com/lumberyard/): 3D game engine - **Mobile/app development:** - [SNS](https://aws.amazon.com/sns/): Manage app push notifications and other end-user notifications From 2df10ec4fd20fb6be0030e31eb2b881e23e883dc Mon Sep 17 00:00:00 2001 From: Patrick McDavid Date: Tue, 11 Oct 2016 20:58:36 -0600 Subject: [PATCH 08/10] grammatical improvement --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 0dc31e7..2e543d9 100644 --- a/README.md +++ b/README.md @@ -803,7 +803,7 @@ EBS ### EBS Gotchas and Limitations -- ❗EBS durability is reasonably good for a regular hardware drive (annual failure rate of [between 0.1% - 0.2%](http://aws.amazon.com/ebs/details/#availabilityanddurability)). On the other hand, that is very poor if you don’t have backups! By contrast, S3 durability is extremely high. *If you care about your data, back it up S3 with snapshots.* +- ❗EBS durability is reasonably good for a regular hardware drive (annual failure rate of [between 0.1% - 0.2%](http://aws.amazon.com/ebs/details/#availabilityanddurability)). On the other hand, that is very poor if you don’t have backups! By contrast, S3 durability is extremely high. *If you care about your data, back it up to S3 with snapshots.* - 🔸EBS has an [**SLA**](http://aws.amazon.com/ec2/sla/) with **99.95%** uptime. See notes on high availability below. - ❗EBS volumes have a [**volume type**](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/EBSVolumeTypes.html) indicating the physical storage type. The types called “standard” (**st1** or **sc1**) are actually old spinning-platter disks, which deliver only hundreds of IOPS — not what you want unless you’re really trying to cut costs. Modern SSD-based **gp2** or **io1** are typically the options you want. From 4ece93728c23d8b30991d73c82006c6ceb4f1e6c Mon Sep 17 00:00:00 2001 From: maiki Date: Tue, 11 Oct 2016 20:09:08 -0700 Subject: [PATCH 09/10] Remove extra word It said, "shift shift". That's 100% too many shifts! --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 2e543d9..eb74fa6 100644 --- a/README.md +++ b/README.md @@ -500,7 +500,7 @@ This guide is about AWS, not DevOps or server configuration management in genera ### Tips -- **NTP and accurate time:** If you are not using Amazon Linux (which comes preconfigured), you should confirm your servers [configure NTP correctly](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/set-time.html#configure_ntp), to avoid insidious time drift (which can then cause all sorts of issues, from breaking API calls to misleading logs). This should be part of your automatic configuration for every server. If time has already drifted substantially (generally >1000 seconds), remember NTP won’t shift shift it back, so you may need to remediate manually (for example, [like this](http://askubuntu.com/questions/254826/how-to-force-a-clock-update-using-ntp) on Ubuntu). +- **NTP and accurate time:** If you are not using Amazon Linux (which comes preconfigured), you should confirm your servers [configure NTP correctly](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/set-time.html#configure_ntp), to avoid insidious time drift (which can then cause all sorts of issues, from breaking API calls to misleading logs). This should be part of your automatic configuration for every server. If time has already drifted substantially (generally >1000 seconds), remember NTP won’t shift it back, so you may need to remediate manually (for example, [like this](http://askubuntu.com/questions/254826/how-to-force-a-clock-update-using-ntp) on Ubuntu). Security and IAM ---------------- From b174d01ffc241f78d9d4426bfc02b98e7fb80947 Mon Sep 17 00:00:00 2001 From: Thanos Baskous Date: Tue, 11 Oct 2016 20:36:02 -0700 Subject: [PATCH 10/10] Additional note about less common abbreviations --- CONTRIBUTING.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index aa43265..13b34a8 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -28,7 +28,7 @@ Finally, if an issue isn’t appropriate, or if you might have skills and inclin When you contribute, keep in mind these conventions: -- **Abbreviations:** For AWS service names, we use the abbreviation throughout the guide if it is more common, e.g. EC2 and not Elastic Compute Cloud. We also omit “Amazon” at the front of product names, e.g. EMR and not Amazon EMR. +- **Abbreviations:** For AWS service names, we use the abbreviation throughout the guide if it is more common, e.g. EC2 and not Elastic Compute Cloud. We also omit “Amazon” at the front of product names, e.g. EMR and not Amazon EMR. If an abbreviation is convenient but not always used, e.g. AZ instead of Availability Zone, either use the full term once per section/paragraph and abbreviate subsequent usages or do not abbreviate it at all. - Terms that appear for the first time in **boldface** are defined there in a brief summary, with a link if possible to what is probably the best page for that concept. It’s also fine to boldface **key statements** that guide the eye. - **Boldfaced headings:** When possible and appropriate, begin bulleted items with a boldfaced summary, as illustrated here. This helps the reader skim the contents. - **Links:**