mirror of
https://github.com/nickpoida/og-aws.git
synced 2025-02-13 10:21:57 +00:00
Merge master
This commit is contained in:
ustroetz
commit
4fc2e75e9c
4 changed files with 175 additions and 121 deletions
44
AUTHORS.md
44
AUTHORS.md
|
|
@ -1,47 +1,59 @@
|
|||
# Authors
|
||||
|
||||
This work is a collaborative effort.
|
||||
It was begun and is edited by [@jlevy](https://github.com/jlevy) and [@ThanosBaskous](https://github.com/ThanosBaskous).
|
||||
The following people (in alphabetical order) have contributed to or reviewed this or earlier versions of the guide.
|
||||
This “Open Guide” is a collaborative effort.
|
||||
It was begun and is led by [@jlevy](https://github.com/jlevy) and [@ThanosBaskous](https://github.com/ThanosBaskous),
|
||||
but the content is the result of a community of contributors, editors, and experts.
|
||||
Please help if you can, and see the [contribution guidelines](CONTRIBUTING.md) for notes on roles and editorial process.
|
||||
Alphabetically by username:
|
||||
|
||||
|
||||
* [Alexander Atallah (alexanderatallah)](https://github.com/alexanderatallah)
|
||||
* [Artem Nikitin (artemnikitin)](https://github.com/artemnikitin) — [2+](https://github.com/open-guides/og-aws/commits?author=artemnikitin)/[1+](https://github.com/open-guides/og-aws/issues?q=author%3Aartemnikitin)
|
||||
* [Andrew Lane (AndrewLane)](https://github.com/AndrewLane) — [1+](https://github.com/open-guides/og-aws/commits?author=AndrewLane)/[1+](https://github.com/open-guides/og-aws/issues?q=author%3AAndrewLane)
|
||||
* [Artem Nikitin (artemnikitin)](https://github.com/artemnikitin) — [3+](https://github.com/open-guides/og-aws/commits?author=artemnikitin)/[3+](https://github.com/open-guides/og-aws/issues?q=author%3Aartemnikitin)
|
||||
* [Benjamin Bunk (benbunk)](https://github.com/benbunk) — [1+](https://github.com/open-guides/og-aws/commits?author=benbunk)/[1+](https://github.com/open-guides/og-aws/issues?q=author%3Abenbunk)
|
||||
* [Ben Kehoe (benkehoe)](https://github.com/benkehoe) — [4+](https://github.com/open-guides/og-aws/commits?author=benkehoe)/[1+](https://github.com/open-guides/og-aws/issues?q=author%3Abenkehoe)
|
||||
* [Ben Kehoe (benkehoe)](https://github.com/benkehoe) — [4+](https://github.com/open-guides/og-aws/commits?author=benkehoe)/[1+](https://github.com/open-guides/og-aws/issues?q=author%3Abenkehoe) — _expert (IoT)_
|
||||
* [Adam Mathias Bittlingmayer (bittlingmayer)](https://github.com/bittlingmayer)
|
||||
* [Bradly Feeley (bradly)](https://github.com/bradly) — [2+](https://github.com/open-guides/og-aws/commits?author=bradly)/[2+](https://github.com/open-guides/og-aws/issues?q=author%3Abradly)
|
||||
* [Brady Dowling (bradydowling)](https://github.com/bradydowling) — [1+](https://github.com/open-guides/og-aws/commits?author=bradydowling)/[1+](https://github.com/open-guides/og-aws/issues?q=author%3Abradydowling)
|
||||
* [chris-griffin](https://github.com/chris-griffin) — [1+](https://github.com/open-guides/og-aws/commits?author=chris-griffin)/[2+](https://github.com/open-guides/og-aws/issues?q=author%3Achris-griffin)
|
||||
* [danhermann](https://github.com/danhermann) — [1+](https://github.com/open-guides/og-aws/commits?author=danhermann)/[1+](https://github.com/open-guides/og-aws/issues?q=author%3Adanhermann)
|
||||
* [Donne Martin (donnemartin)](https://github.com/donnemartin)
|
||||
* [Chris Leyva (chrisleyva)](https://github.com/chrisleyva) — [1+](https://github.com/open-guides/og-aws/commits?author=chrisleyva)/[1+](https://github.com/open-guides/og-aws/issues?q=author%3Achrisleyva)
|
||||
* [Dan Hermann (danhermann)](https://github.com/danhermann) — [1+](https://github.com/open-guides/og-aws/commits?author=danhermann)/[1+](https://github.com/open-guides/og-aws/issues?q=author%3Adanhermann)
|
||||
* [Donne Martin (donnemartin)](https://github.com/donnemartin) — _expert (tools)_
|
||||
* [Dmitry Guyvoronsky (dreamiurg)](https://github.com/dreamiurg) — [1+](https://github.com/open-guides/og-aws/commits?author=dreamiurg)/[1+](https://github.com/open-guides/og-aws/issues?q=author%3Adreamiurg)
|
||||
* [Patrick McDavid (ehippy)](https://github.com/ehippy) — [1+](https://github.com/open-guides/og-aws/commits?author=ehippy)/[1+](https://github.com/open-guides/og-aws/issues?q=author%3Aehippy)
|
||||
* [Max Grigorev (forwidur)](https://github.com/forwidur)
|
||||
* [Max Grigorev (forwidur)](https://github.com/forwidur) — _editor (EBS, RDS)_
|
||||
* [Glynn Forrest (glynnforrest)](https://github.com/glynnforrest) — [1+](https://github.com/open-guides/og-aws/commits?author=glynnforrest)/[1+](https://github.com/open-guides/og-aws/issues?q=author%3Aglynnforrest)
|
||||
* [Dmitry Golyshev (golyshev)](https://github.com/golyshev)
|
||||
* [Gulam Shakir (gshakir)](https://github.com/gshakir) — [2+](https://github.com/open-guides/og-aws/commits?author=gshakir)/[1+](https://github.com/open-guides/og-aws/issues?q=author%3Agshakir)
|
||||
* [Itay Shakury (itaysk)](https://github.com/itaysk) — [1+](https://github.com/open-guides/og-aws/commits?author=itaysk)/[2+](https://github.com/open-guides/og-aws/issues?q=author%3Aitaysk)
|
||||
* [jbao](https://github.com/jbao) — [1+](https://github.com/open-guides/og-aws/commits?author=jbao)/[1+](https://github.com/open-guides/og-aws/issues?q=author%3Ajbao)
|
||||
* [Joshua Levy (jlevy)](https://github.com/jlevy) — [86+](https://github.com/open-guides/og-aws/commits?author=jlevy)/[76+](https://github.com/open-guides/og-aws/issues?q=author%3Ajlevy) — _general editor_
|
||||
* [Joshua Levy (jlevy)](https://github.com/jlevy) — [87+](https://github.com/open-guides/og-aws/commits?author=jlevy)/[80+](https://github.com/open-guides/og-aws/issues?q=author%3Ajlevy) — _project lead, editor (topics not otherwise assigned)_
|
||||
* Jurgen Philippaerts
|
||||
* [KAZUYUKI TANIMURA (kazuyukitanimura)](https://github.com/kazuyukitanimura)
|
||||
* [Lynn Langit (lynnlangit)](https://github.com/lynnlangit) — [3+](https://github.com/open-guides/og-aws/commits?author=lynnlangit)/[2+](https://github.com/open-guides/og-aws/issues?q=author%3Alynnlangit)
|
||||
* [KAZUYUKI TANIMURA (kazuyukitanimura)](https://github.com/kazuyukitanimura) — [0+](https://github.com/open-guides/og-aws/commits?author=kazuyukitanimura)/[1+](https://github.com/open-guides/og-aws/issues?q=author%3Akazuyukitanimura)
|
||||
* [Lynn Langit (lynnlangit)](https://github.com/lynnlangit) — [3+](https://github.com/open-guides/og-aws/commits?author=lynnlangit)/[4+](https://github.com/open-guides/og-aws/issues?q=author%3Alynnlangit) — _editor (IoT)_
|
||||
* [maiki](https://github.com/maiki) — [1+](https://github.com/open-guides/og-aws/commits?author=maiki)/[1+](https://github.com/open-guides/og-aws/issues?q=author%3Amaiki)
|
||||
* [Manoj M J (manojmj92)](https://github.com/manojmj92) — [1+](https://github.com/open-guides/og-aws/commits?author=manojmj92)/[1+](https://github.com/open-guides/og-aws/issues?q=author%3Amanojmj92)
|
||||
* [Marcello Bastéa-Forte (marcello3d)](https://github.com/marcello3d)
|
||||
* [Max Zanko (max-zanko)](https://github.com/max-zanko) — [3+](https://github.com/open-guides/og-aws/commits?author=max-zanko)/[1+](https://github.com/open-guides/og-aws/issues?q=author%3Amax-zanko) — _editor (S3, EMR, Redshift)_
|
||||
* [John Merrells (merrells)](https://github.com/merrells)
|
||||
* [Max Zanko (max-zanko)](https://github.com/max-zanko) — [10+](https://github.com/open-guides/og-aws/commits?author=max-zanko)/[6+](https://github.com/open-guides/og-aws/issues?q=author%3Amax-zanko) — _editor (EC2, S3, Glacier, EMR, Redshift)_
|
||||
* [John Merrells (merrells)](https://github.com/merrells) — _expert (cloud infrastructure, when to use AWS)_
|
||||
* [Magnus Kulke (mkulke)](https://github.com/mkulke) — [4+](https://github.com/open-guides/og-aws/commits?author=mkulke)/[3+](https://github.com/open-guides/og-aws/issues?q=author%3Amkulke)
|
||||
* [Nitin S (nitingithub)](https://github.com/nitingithub) — [6+](https://github.com/open-guides/og-aws/commits?author=nitingithub)/[4+](https://github.com/open-guides/og-aws/issues?q=author%3Anitingithub) — _editor (cost management)_
|
||||
* [Ola Wiberg (olawiberg)](https://github.com/olawiberg)
|
||||
* [Phillip Calvin (pnc)](https://github.com/pnc) — [1+](https://github.com/open-guides/og-aws/commits?author=pnc)/[1+](https://github.com/open-guides/og-aws/issues?q=author%3Apnc)
|
||||
* Praveen Patnala
|
||||
* [Rich Adams (richadams)](https://github.com/richadams) — [1+](https://github.com/open-guides/og-aws/commits?author=richadams)/[1+](https://github.com/open-guides/og-aws/issues?q=author%3Arichadams) — _editor (VPC)_
|
||||
* [Russell Power (rjpower)](https://github.com/rjpower)
|
||||
* [David Schott (shott85)](https://github.com/shott85) — [1+](https://github.com/open-guides/og-aws/commits?author=shott85)/[1+](https://github.com/open-guides/og-aws/issues?q=author%3Ashott85)
|
||||
* [Thanos Baskous (ThanosBaskous)](https://github.com/ThanosBaskous) — [12+](https://github.com/open-guides/og-aws/commits?author=ThanosBaskous)/[12+](https://github.com/open-guides/og-aws/issues?q=author%3AThanosBaskous) — _general editor_
|
||||
* [Adam Nelson (spudstuff)](https://github.com/spudstuff) — [4+](https://github.com/open-guides/og-aws/commits?author=spudstuff)/[2+](https://github.com/open-guides/og-aws/issues?q=author%3Aspudstuff)
|
||||
* [Steven Maude (StevenMaude)](https://github.com/StevenMaude) — [1+](https://github.com/open-guides/og-aws/commits?author=StevenMaude)/[1+](https://github.com/open-guides/og-aws/issues?q=author%3AStevenMaude)
|
||||
* [Thanos Baskous (ThanosBaskous)](https://github.com/ThanosBaskous) — [15+](https://github.com/open-guides/og-aws/commits?author=ThanosBaskous)/[15+](https://github.com/open-guides/og-aws/issues?q=author%3AThanosBaskous) — _project lead, editor (topics not otherwise assigned)_
|
||||
* [Tom Schlick (tomschlick)](https://github.com/tomschlick) — [3+](https://github.com/open-guides/og-aws/commits?author=tomschlick)/[1+](https://github.com/open-guides/og-aws/issues?q=author%3Atomschlick)
|
||||
* [Trayton White (traytonwhite)](https://github.com/traytonwhite) — [1+](https://github.com/open-guides/og-aws/commits?author=traytonwhite)/[1+](https://github.com/open-guides/og-aws/issues?q=author%3Atraytonwhite)
|
||||
* [Stefan Zier (weirded)](https://github.com/weirded)
|
||||
* [Michael Ortali (xethorn)](https://github.com/xethorn) — [1+](https://github.com/open-guides/og-aws/commits?author=xethorn)/[1+](https://github.com/open-guides/og-aws/issues?q=author%3Axethorn)
|
||||
|
||||
Additional authors are welcome; see the [contribution guidelines](CONTRIBUTING.md).
|
||||
Please let the editors know of any errors or omissions on this list.
|
||||
Please don't PR the AUTHORS.md or authors-info.yml files! It is auto-generated regularly by the project leads.
|
||||
Please let the project leads know of any errors or omissions on this list.
|
||||
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -1,30 +1,34 @@
|
|||
Contributing
|
||||
------------
|
||||
|
||||
Contributions of all kinds, including discussion, corrections, additions, and improvements, are welcome! We hope you'll join and help, in small ways or large. We gladly credit all contributors and authors. Here are few notes before you jump in.
|
||||
|
||||
[](https://og-aws.slack.lexikon.io/) [](https://airtable.com/shrXZ61VrovWfXYBg)
|
||||
|
||||
The simplest thing you can do to contribute is [**join the Slack channel**](https://og-aws.slack.lexikon.io/) or [**add to our list of common questions**](https://airtable.com/shrXZ61VrovWfXYBg), which help guide us in ways to improve the guide.
|
||||
Contributions of all kinds, including discussion, corrections, additions, and improvements, are welcome! We hope you'll join and help, in small ways or large. We gladly [credit](/AUTHORS.md) all contributors. Here are few notes before you jump in.
|
||||
|
||||
[**File issues**](https://github.com/open-guides/og-aws/issues) if it’s clear what needs to be improved. (For less well-defined issues, discussing first on Slack may be helpful.)
|
||||
### Please Help
|
||||
|
||||
[**Pull requests**](https://github.com/open-guides/og-aws/pulls) with changes are even better. Please keep them small and focused, so we can add items individually, and review the conventions below. (Again, Slack discussion can help.)
|
||||
If you’ve found this guide useful, you have many ways to help:
|
||||
|
||||
Please review current issues and pull requests to avoid duplication.
|
||||
- The simplest thing you can do to contribute is [**join the Slack channel**](https://og-aws.slack.lexikon.io/) or [**add to our list of common questions**](https://airtable.com/shrXZ61VrovWfXYBg), which helps the community and guides what contributors can focus on. We encourage you to ask AWS questions and help others!
|
||||
- [**File issues**](https://github.com/open-guides/og-aws/issues) if it’s clear something needs to be improved and you’re not able to make a pull request.
|
||||
- [**Pull requests**](https://github.com/open-guides/og-aws/pulls) with changes are always welcome. Please keep them small and focused, so we can add items individually, and review the conventions below. If you want to make a larger change, try to discuss it in Slack.
|
||||
- **Review** or **comment** on existing issues and pull requests if you have expertise.
|
||||
- If you have deep expertise, we may ask you to be an **editor** or **expert**. Editors and experts are assigned roles that [help us review](#editorial-process) the Guide. Join Slack to discuss this.
|
||||
|
||||
We gladly [credit](/AUTHORS.md) all contributors.
|
||||
### Pull Request Etiquette
|
||||
|
||||
Finally, if an issue isn’t appropriate, or if you might have skills and inclination to help improve the guide in a more substantial way, please shoot an e-mail to [@jlevy](https://github.com/jlevy).
|
||||
- Keep changes as small as is practical. Do not make changes to multiple sections at once, alter whitespace in broad ways, etc.
|
||||
- Neutrality: If you have an affiliation related to what you are changing, please disclose it.
|
||||
- Please do your best to review current issues and pull requests to avoid duplication.
|
||||
|
||||
### Style
|
||||
### Writing Style
|
||||
|
||||
- **Use references:** If you are adding an item, whenever possible, try to add a link or reference to relevant discussion or reference pages.
|
||||
- **Be brief:** Avoid long expository paragraphs; it’s better to link to a blog. (We are open to linking to your own blog, if it’s the best source.)
|
||||
- **Include opinions and common practice:** Thoughtful opinion is helpful. If there are multiple conventions or ideas on something held by experts, mention the different ones.
|
||||
- **Clarity:** Strive for consistency with conventions listed here, but clarity is most important.
|
||||
- **Link to references:** If you are adding an item, whenever possible, try to add a link or reference to relevant discussion or reference pages.
|
||||
- **Be brief:** Avoid long expository paragraphs; it’s better to link to a blog. (We are open to linking to your own blog, if it’s the best source.)
|
||||
- **Include opinions and common practice:** Thoughtful opinion is helpful. If there are multiple conventions or ideas on something held by experts, mention the different ones.
|
||||
- **Clarity:** Strive for consistency with conventions listed here, but clarity is most important.
|
||||
|
||||
### Conventions
|
||||
### Writing Conventions
|
||||
|
||||
When you contribute, keep in mind these conventions:
|
||||
|
||||
|
|
@ -44,7 +48,23 @@ When you contribute, keep in mind these conventions:
|
|||
- *X Alternatives and Lock-In*: Should you be using this service or something else? Is the decision an important one that locks you in?
|
||||
- *X Tips*: Everything you should know about the service, from big stuff to details.
|
||||
- *X Gotchas and Limitations*: Common problems, large and small, as well as misconceptions and quirks.
|
||||
- Not all sections need to follow the above conventions exactly.
|
||||
- Note we try to make sections uniquely titled, so GitHub links to Markdown section anchors don’t collide and are stable.
|
||||
- Not all sections need to follow the above conventions exactly.
|
||||
- Note we try to make sections uniquely titled, so GitHub links to Markdown section anchors don’t collide and are stable.
|
||||
|
||||
Note we keep consistent formatting in Markdown via [markdownfmt](https://github.com/shurcooL/markdownfmt). We run **admin/reformat.sh** to do this, but you don’t have to worry about it unless you really want to.
|
||||
|
||||
### Editorial Process
|
||||
|
||||
- Roles:
|
||||
- **Project leads:** Own overall quality of the Guide, direction, and process.
|
||||
- **Editors:** Contributors own specific sections or aspects of the Guide, reviewing PRs and/or writing. requires expert knowledge.
|
||||
- **Experts:** People with expert knowledge in various areas, who have agreed to review or help on demand with tougher questions or PRs.
|
||||
- **Contributors:** Everyone who contributes content or helps one way or another.
|
||||
- All PRs are reviewed by an **editor** and for non-trivial changes, a **project lead**, usually in that order, but it can be reversed for expediency.
|
||||
- In addition, anyone with relevant knowledge is encouraged to review/comment on PRs.
|
||||
- Both editors and project leads are responsible for checking for style or problems.
|
||||
- Trivial changes (including copy editing) may be merged in directly by project leads or editors.
|
||||
|
||||
### Contact
|
||||
|
||||
If you have concerns or additional ideas of ways to help, e-mail **og-aws@lexikon.io** or use Slack to contact the [project leads](AUTHORS.md).
|
||||
|
|
|
|||
171
README.md
171
README.md
|
|
@ -5,6 +5,8 @@ The Open Guide to Amazon Web Services
|
|||
|
||||
[](https://og-aws.slack.lexikon.io/) [](https://airtable.com/shrXZ61VrovWfXYBg)
|
||||
|
||||
[Credits](AUTHORS.md) ∙ [Contributing guidelines](CONTRIBUTING.md)
|
||||
|
||||
Table of Contents
|
||||
-----------------
|
||||
|
||||
|
|
@ -17,7 +19,7 @@ Table of Contents
|
|||
**AWS in General**
|
||||
|
||||
- [General Information](#general-information)
|
||||
- [Learning and Career Development](#learning-aws)
|
||||
- [Learning and Career Development](#learning-and-career-development)
|
||||
- [Managing AWS](#managing-aws)
|
||||
- [Managing Servers and Applications](#managing-servers-and-applications)
|
||||
|
||||
|
|
@ -47,7 +49,7 @@ Table of Contents
|
|||
| [CloudFront](#cloudfront) | [📗](#cloudfront-basics) | [📘](#cloudfront-tips) | [📙](#cloudfront-gotchas-and-limitations) |
|
||||
| [DirectConnect](#directconnect) | [📗](#directconnect-basics) | [📘](#directconnect-tips) | |
|
||||
| [Redshift](#redshift) | [📗](#redshift-basics) | [📘](#redshift-tips) | [📙](#redshift-gotchas-and-limitations) |
|
||||
| [EMR](#emr) | [📗](#emr-basics) | [📘](#emr-tips) | |
|
||||
| [EMR](#emr) | [📗](#emr-basics) | [📘](#emr-tips) | [📙](#emr-gotchas-and-limitations) |
|
||||
|
||||
**Special Topics**
|
||||
|
||||
|
|
@ -139,9 +141,9 @@ General Information
|
|||
- Each of those reasons above might point to situations where other services are preferable. In practice, many, if not most, tech startups as well as a number of modern large companies can or already do benefit from using AWS. Many large enterprises are partly migrating internal infrastructure to Azure, Google Cloud, and AWS.
|
||||
- **Costs:** Billing and cost management are such big topics that we have [an entire section on this](#billing-and-cost-management).
|
||||
- 🔹**EC2 vs. other services:** Most users of AWS are most familiar with [EC2](#ec2), AWS’ flagship virtual server product, and possibly a few others like S3 and CLBs. But AWS products now extend far beyond basic IaaS, and often companies do not properly understand or appreciate all the many AWS services and how they can be applied, due to the [sharply growing](#which-services-to-use) number of services, their novelty and complexity, branding confusion, and fear of ⛓lock-in to proprietary AWS technology. Although a bit daunting, it’s important for technical decision-makers in companies to understand the breadth of the AWS services and make informed decisions. (We hope this guide will help.)
|
||||
- 🚪**AWS vs. other cloud providers:** While AWS is the dominant IaaS provider (31% market share in [this 2016 estimate](https://www.srgresearch.com/articles/aws-remains-dominant-despite-microsoft-and-google-growth-surges)), there is significant competition and alternatives that are better suited to some companies:
|
||||
- The most significant direct competitor is [**Google Cloud**](https://cloud.google.com/). It arrived later to market than AWS, but has vast resources and is now used widely by many companies, including a few large ones. It is gaining market share. Not all AWS services have similar or analogous services in Google Cloud. And vice versa: In particular Google offers some more advanced machine learning-based services like the [Vision](https://cloud.google.com/vision/), [Speech](https://cloud.google.com/speech/), and [Natural Language](https://cloud.google.com/natural-language/) APIs. It’s not common to switch once you’re up and running, but it does happen: [Spotify migrated](http://www.wsj.com/articles/google-cloud-lures-amazon-web-services-customer-spotify-1456270951) from AWS to Google Cloud. There is more discussion [on Quora](https://www.quora.com/What-are-the-reasons-to-choose-AWS-over-Google-Cloud-or-vice-versa-for-a-high-traffic-web-application) about relative benefits.
|
||||
- [**Microsoft Azure**](https://azure.microsoft.com/en) is the de facto choice for companies and teams that are focused on a Microsoft stack.
|
||||
- 🚪**AWS vs. other cloud providers:** While AWS is the dominant IaaS provider (31% market share in [this 2016 estimate](https://www.srgresearch.com/articles/aws-remains-dominant-despite-microsoft-and-google-growth-surges)), there is significant competition and alternatives that are better suited to some companies. [This Gartner report](https://www.gartner.com/doc/reprints?id=1-2G2O5FC&ct=150519&st=sb) has a good overview of the major cloud players :
|
||||
- [**Google Cloud**](https://cloud.google.com/). It arrived later to market than AWS, but has vast resources and is now used widely by many companies, including a few large ones. It is gaining market share. Not all AWS services have similar or analogous services in Google Cloud. And vice versa: In particular Google offers some more advanced machine learning-based services like the [Vision](https://cloud.google.com/vision/), [Speech](https://cloud.google.com/speech/), and [Natural Language](https://cloud.google.com/natural-language/) APIs. It’s not common to switch once you’re up and running, but it does happen: [Spotify migrated](http://www.wsj.com/articles/google-cloud-lures-amazon-web-services-customer-spotify-1456270951) from AWS to Google Cloud. There is more discussion [on Quora](https://www.quora.com/What-are-the-reasons-to-choose-AWS-over-Google-Cloud-or-vice-versa-for-a-high-traffic-web-application) about relative benefits.
|
||||
- [**Microsoft Azure**](https://azure.microsoft.com/en) is the de facto choice for companies and teams that are focused on a Microsoft stack, and it has now placed significant emphasis on Linux as well
|
||||
- In **China**, AWS’ footprint is relatively small. The market is dominated by Alibaba’s [Aliyun](https://intl.aliyun.com/).
|
||||
- Companies at (very) large scale may want to reduce costs by managing their own infrastructure. For example, [Dropbox migrated](https://news.ycombinator.com/item?id=11282948) to their own infrastructure.
|
||||
- Other cloud providers such as [Digital Ocean](https://www.digitalocean.com/) offer similar services, sometimes with greater ease of use, more personalized support, or lower cost. However, none of these match the breadth of products, mind-share, and market domination AWS now enjoys.
|
||||
|
|
@ -161,6 +163,7 @@ General Information
|
|||
- **Hybrid cloud:** In larger enterprises, it is common to have [hybrid deployments](https://aws.amazon.com/enterprise/hybrid/) encompassing private cloud or on-premises servers and AWS — or other enterprise cloud providers like [IBM](https://www.ibm.com/cloud-computing/solutions/hybrid-cloud)/[Bluemix](http://www.ibm.com/cloud-computing/bluemix/hybrid/), [Microsoft](https://www.microsoft.com/en-us/cloud-platform/hybrid-cloud)/[Azure](https://azure.microsoft.com/en-us/overview/azure-stack/), [NetApp](http://www.netapp.com/us/solutions/cloud/hybrid-cloud/), or [EMC](http://www.emc.com/en-us/cloud/hybrid-cloud-computing/index.htm).
|
||||
- **Major customers:** Who uses AWS and Google Cloud?
|
||||
- AWS’s [list of customers](https://aws.amazon.com/solutions/case-studies/) includes large numbers of mainstream online properties and major brands, such as Netflix, Pinterest, Spotify (moving to Google Cloud), Airbnb, Expedia, Yelp, Zynga, Comcast, Nokia, and Bristol-Myers Squibb.
|
||||
- Azure's [list of customers](https://azure.microsoft.com/en-us/case-studies/) includes companies such as NBC Universal, 3M and Honeywell Inc.
|
||||
- Google Cloud’s [list of customers](https://cloud.google.com/customers/) is large as well, and includes a few mainstream sites, such as [Snapchat](http://www.businessinsider.com/snapchat-is-built-on-googles-cloud-2014-1), Best Buy, Domino’s, and Sony Music.
|
||||
|
||||
### Which Services to Use
|
||||
|
|
@ -268,12 +271,12 @@ Many services within AWS can at least be compared with Google Cloud offerings or
|
|||
| File storage | S3 | Cloud Storage | GFS | Storage Account | | Swift, HDFS |
|
||||
| Block storage | EBS | Persistent Disk | | Storage Account | | NFS |
|
||||
| SQL datastore | RDS | Cloud SQL | | SQL Database | | MySQL, PostgreSQL |
|
||||
| Sharded RDBMS | | Cloud SQL | F1, Spanner | | | Crate.io |
|
||||
| Bigtable | | Cloud Bigtable | Bigtable | | | CockroachDB |
|
||||
| Sharded RDBMS | | | F1, Spanner | | | Crate.io, CockroachDB |
|
||||
| Bigtable | | Cloud Bigtable | Bigtable | | | HBase |
|
||||
| Key-value store, column store | DynamoDB | Cloud Datastore | Megastore | Tables, DocumentDB | | Cassandra, CouchDB, RethinkDB, Redis |
|
||||
| Memory cache | ElastiCache | App Engine Memcache | | Redis Cache | | Memcached, Redis |
|
||||
| Search | CloudSearch, Elasticsearch (managed) | | | Search | Algolia, QBox | Elasticsearch, Solr |
|
||||
| Data warehouse | Redshift | BigQuery | | SQL Data Warehouse | Oracle, IBM, SAP, HP, many others | Greenplum |
|
||||
| Data warehouse | Redshift | BigQuery | Dremel | SQL Data Warehouse | Oracle, IBM, SAP, HP, many others | Greenplum |
|
||||
| Business intelligence | QuickSight | | | Power BI | Tableau | |
|
||||
| Lock manager | [DynamoDB (weak)](https://gist.github.com/ryandotsmith/c95fd21fab91b0823328) | | Chubby | Lease blobs in Storage Account | | ZooKeeper, Etcd, Consul |
|
||||
| Message broker | SQS, SNS, IoT | Pub/Sub | PubSub2 | Service Bus | | RabbitMQ, Kafka, 0MQ |
|
||||
|
|
@ -281,14 +284,14 @@ Many services within AWS can at least be compared with Google Cloud offerings or
|
|||
| MapReduce | EMR | Dataproc | MapReduce | HDInsight, DataLake Analytics | Qubole | Hadoop |
|
||||
| Monitoring | CloudWatch | Monitoring | Borgmon | Monitor | | Prometheus(?) |
|
||||
| Metric management | | | Borgmon, TSDB | Application Insights | | Graphite, InfluxDB, OpenTSDB, Grafana, Riemann, Prometheus |
|
||||
| CDN | CloudFront | | | CDN | | Apache Traffic Server |
|
||||
| CDN | CloudFront | Cloud CDN | | CDN | | Apache Traffic Server |
|
||||
| Load balancer | CLB/ALB | Load Balancing | GFE | Load Balancer, Application Gateway | | nginx, HAProxy, Apache Traffic Server |
|
||||
| DNS | Route53 | DNS | | DNS | | bind |
|
||||
| Email | SES | | | | Sendgrid, Mandrill, Postmark | |
|
||||
| Git hosting | CodeCommit | | | Visual Studio Team Services | GitHub, BitBucket | GitLab |
|
||||
| Git hosting | CodeCommit | Cloud Source Repositories | | Visual Studio Team Services | GitHub, BitBucket | GitLab |
|
||||
| User authentication | Cognito | | | Azure Active Directory | | oauth.io |
|
||||
| Mobile app analytics | Mobile Analytics | | | HockeyApp | Mixpanel | |
|
||||
| Mobile app testing | Device Farm |Cloud Test Lab | | Xamarin Test Cloud | BrowserStack, Sauce Labs, Testdroid |
|
||||
| Mobile app analytics | Mobile Analytics | Firebase Analytics | | HockeyApp | Mixpanel | |
|
||||
| Mobile app testing | Device Farm | Firebase Test Lab | | Xamarin Test Cloud | BrowserStack, Sauce Labs, Testdroid |
|
||||
|
||||
|
||||
🚧 [*Please help fill this table in.*](CONTRIBUTING.md)
|
||||
|
|
@ -301,58 +304,61 @@ Selected resources with more detail on this chart:
|
|||
|
||||
It’s important to know the maturity of each AWS product. Here is a mostly complete list of first release date, with links to the [release notes](https://aws.amazon.com/releasenotes/). Most recently released services are first. Not all services are available in all regions; see [this table](https://aws.amazon.com/about-aws/global-infrastructure/regional-product-services/).
|
||||
|
||||
| Service | Original release | Availability |
|
||||
|------------------------------------------------------------------------------------------------------------|------------------|-------------------------------------------------------------------------------|
|
||||
| 🐥[Database Migration Service](https://aws.amazon.com/releasenotes/AWS-Database-Migration-Service?browse=1) | 2016-03 | General |
|
||||
| 🐥[IoT](https://aws.amazon.com/blogs/aws/aws-iot-now-generally-available/) | 2015-08 | General |
|
||||
| 🐥[WAF](https://aws.amazon.com/releasenotes/AWS-WAF?browse=1) | 2015-10 | General |
|
||||
| 🐥[Data Pipeline](https://aws.amazon.com/releasenotes/AWS-Data-Pipeline?browse=1) | 2015-10 | General |
|
||||
| 🐥[Elasticsearch](https://aws.amazon.com/releasenotes/Amazon-Elasticsearch-Service?browse=1) | 2015-10 | General |
|
||||
| 🐥[Service Catalog](https://aws.amazon.com/releasenotes/AWS-Service-Catalog?browse=1) | 2015-07 | General |
|
||||
| 🐥[CodePipeline](https://aws.amazon.com/releasenotes/AWS-CodePipeline?browse=1) | 2015-07 | General |
|
||||
| 🐥[CodeCommit](https://aws.amazon.com/releasenotes/AWS-CodeCommit?browse=1) | 2015-07 | General |
|
||||
| 🐥[API Gateway](https://aws.amazon.com/releasenotes/Amazon-API-Gateway?browse=1) | 2015-07 | General |
|
||||
| 🐥[Config](https://aws.amazon.com/releasenotes/AWS-Config?browse=1) | 2015-06 | General |
|
||||
| 🐥[EFS](https://aws.amazon.com/releasenotes/Amazon-EFS?browse=1) | 2015-05 | General |
|
||||
| 🐥[Machine Learning](https://aws.amazon.com/releasenotes/AmazonML?browse=1) | 2015-04 | General |
|
||||
| [Lambda](https://aws.amazon.com/releasenotes/AWS-Lambda?browse=1) | 2014-11 | General |
|
||||
| [ECS](https://aws.amazon.com/ecs/release-notes/) | 2014-11 | General |
|
||||
| [KMS](https://aws.amazon.com/releasenotes/AWS-KMS?browse=1) | 2014-11 | General |
|
||||
| [CodeDeploy](https://aws.amazon.com/releasenotes/AWS-CodeDeploy?browse=1) | 2014-11 | General |
|
||||
| [Kinesis](https://aws.amazon.com/releasenotes/Amazon-Kinesis?browse=1) | 2013-12 | General |
|
||||
| [CloudTrail](https://aws.amazon.com/releasenotes/AWS-CloudTrail?browse=1) | 2013-11 | General |
|
||||
| [AppStream](https://aws.amazon.com/releasenotes/Amazon-AppStream?browse=1) | 2013-11 | Preview |
|
||||
| [CloudHSM](https://aws.amazon.com/releasenotes/AWS-CloudHSM?browse=1) | 2013-03 | General |
|
||||
| [Silk](https://aws.amazon.com/releasenotes/Amazon-Silk?browse=1) | 2013-03 | Obsolete? |
|
||||
| [OpsWorks](https://aws.amazon.com/releasenotes/AWS-OpsWorks?browse=1) | 2013-02 | General |
|
||||
| [Redshift](https://aws.amazon.com/releasenotes/Amazon-Redshift?browse=1) | 2013-02 | General |
|
||||
| [Elastic Transcoder](https://aws.amazon.com/releasenotes/Amazon-Elastic-Transcoder?browse=1) | 2013-01 | General |
|
||||
| [Glacier](https://aws.amazon.com/releasenotes/Amazon-Glacier?browse=1) | 2012-08 | General |
|
||||
| [CloudSearch](https://aws.amazon.com/releasenotes/Amazon-CloudSearch?browse=1) | 2012-04 | General |
|
||||
| [SWF](https://aws.amazon.com/releasenotes/Amazon-SWF?browse=1) | 2012-02 | General |
|
||||
| [Storage Gateway](https://aws.amazon.com/releasenotes/AWS-Storage-Gateway?browse=1) | 2012-01 | General |
|
||||
| [DynamoDB](https://aws.amazon.com/releasenotes/Amazon-DynamoDB?browse=1) | 2012-01 | General |
|
||||
| [DirectConnect](https://aws.amazon.com/releasenotes/AWS-Direct-Connect?browse=1) | 2011-08 | General |
|
||||
| [ElastiCache](https://aws.amazon.com/releasenotes/Amazon-ElastiCache?browse=1) | 2011-08 | General |
|
||||
| [CloudFormation](https://aws.amazon.com/releasenotes/AWS-CloudFormation?browse=1) | 2011-04 | General |
|
||||
| [SES](https://aws.amazon.com/releasenotes/Amazon-SES?browse=1) | 2011-01 | General |
|
||||
| [Elastic Beanstalk](https://aws.amazon.com/releasenotes/AWS-Elastic-Beanstalk?browse=1) | 2010-12 | General |
|
||||
| [Route 53](https://aws.amazon.com/releasenotes/Amazon-Route-53?browse=1) | 2010-10 | General |
|
||||
| [IAM](https://aws.amazon.com/releasenotes/AWS-Identity-and-Access-Management?browse=1) | 2010-09 | General |
|
||||
| [SNS](https://aws.amazon.com/releasenotes/Amazon-SNS?browse=1) | 2010-04 | General |
|
||||
| [EMR](https://aws.amazon.com/releasenotes/Elastic-MapReduce?browse=1) | 2010-04 | General |
|
||||
| [RDS](https://aws.amazon.com/releasenotes/Amazon-RDS?browse=1) | 2009-12 | General |
|
||||
| [VPC](https://aws.amazon.com/releasenotes/Amazon-VPC?browse=1) | 2009-08 | General |
|
||||
| [Snowball](https://aws.amazon.com/releasenotes/AWS-ImportExport?browse=1) | 2009-05 | General |
|
||||
| [CloudWatch](https://aws.amazon.com/releasenotes/CloudWatch?browse=1) | 2009-05 | General |
|
||||
| [CloudFront](https://aws.amazon.com/releasenotes/CloudFront?browse=1) | 2008-11 | General |
|
||||
| [Fulfillment Web Service](https://aws.amazon.com/releasenotes/Amazon-FWS?browse=1) | 2008-03 | Obsolete? |
|
||||
| [SimpleDB](https://aws.amazon.com/releasenotes/Amazon-SimpleDB?browse=1) | 2007-12 | ❗[Nearly obsolete](https://forums.aws.amazon.com/thread.jspa?threadID=121711) |
|
||||
| [DevPay](https://aws.amazon.com/releasenotes/DevPay?browse=1) | 2007-12 | General |
|
||||
| [Flexible Payments Service](https://aws.amazon.com/releasenotes/Amazon-FPS?browse=1) | 2007-08 | Retired |
|
||||
| [EC2](https://aws.amazon.com/releasenotes/Amazon-EC2?browse=1) | 2006-08 | General |
|
||||
| [SQS](https://aws.amazon.com/releasenotes/Amazon-SQS?browse=1) | 2006-07 | General |
|
||||
| [S3](https://aws.amazon.com/releasenotes/Amazon-S3?browse=1) | 2006-03 | General |
|
||||
| Service | Original release | Availability | CLI Support |
|
||||
|------------------------------------------------------------------------------------------------------------|------------------|-------------------------------------------------------------------------------|:-----------:|
|
||||
| 🐥[Database Migration Service](https://aws.amazon.com/releasenotes/AWS-Database-Migration-Service?browse=1) | 2016-03 | General | |
|
||||
| 🐥[IoT](https://aws.amazon.com/blogs/aws/aws-iot-now-generally-available/) | 2015-08 | General | ✓ |
|
||||
| 🐥[WAF](https://aws.amazon.com/releasenotes/AWS-WAF?browse=1) | 2015-10 | General | ✓ |
|
||||
| 🐥[Data Pipeline](https://aws.amazon.com/releasenotes/AWS-Data-Pipeline?browse=1) | 2015-10 | General | ✓ |
|
||||
| 🐥[Elasticsearch](https://aws.amazon.com/releasenotes/Amazon-Elasticsearch-Service?browse=1) | 2015-10 | General | ✓ |
|
||||
| 🐥[Service Catalog](https://aws.amazon.com/releasenotes/AWS-Service-Catalog?browse=1) | 2015-07 | General | ✓ |
|
||||
| 🐥[Device Farm](https://aws.amazon.com/releasenotes/AWS-Device-Farm?browse=1) | 2015-07 | General | ✓ |
|
||||
| 🐥[CodePipeline](https://aws.amazon.com/releasenotes/AWS-CodePipeline?browse=1) | 2015-07 | General | ✓ |
|
||||
| 🐥[CodeCommit](https://aws.amazon.com/releasenotes/AWS-CodeCommit?browse=1) | 2015-07 | General | ✓ |
|
||||
| 🐥[API Gateway](https://aws.amazon.com/releasenotes/Amazon-API-Gateway?browse=1) | 2015-07 | General | ✓ |
|
||||
| 🐥[Config](https://aws.amazon.com/releasenotes/AWS-Config?browse=1) | 2015-06 | General | ✓ |
|
||||
| 🐥[EFS](https://aws.amazon.com/releasenotes/Amazon-EFS?browse=1) | 2015-05 | General | ✓ |
|
||||
| 🐥[Machine Learning](https://aws.amazon.com/releasenotes/AmazonML?browse=1) | 2015-04 | General | ✓ |
|
||||
| [Lambda](https://aws.amazon.com/releasenotes/AWS-Lambda?browse=1) | 2014-11 | General | ✓ |
|
||||
| [ECS](https://aws.amazon.com/ecs/release-notes/) | 2014-11 | General | ✓ |
|
||||
| [KMS](https://aws.amazon.com/releasenotes/AWS-KMS?browse=1) | 2014-11 | General | ✓ |
|
||||
| [CodeDeploy](https://aws.amazon.com/releasenotes/AWS-CodeDeploy?browse=1) | 2014-11 | General | ✓ |
|
||||
| [Kinesis](https://aws.amazon.com/releasenotes/Amazon-Kinesis?browse=1) | 2013-12 | General | ✓ |
|
||||
| [CloudTrail](https://aws.amazon.com/releasenotes/AWS-CloudTrail?browse=1) | 2013-11 | General | ✓ |
|
||||
| [AppStream](https://aws.amazon.com/releasenotes/Amazon-AppStream?browse=1) | 2013-11 | Preview | |
|
||||
| [CloudHSM](https://aws.amazon.com/releasenotes/AWS-CloudHSM?browse=1) | 2013-03 | General | ✓ |
|
||||
| [Silk](https://aws.amazon.com/releasenotes/Amazon-Silk?browse=1) | 2013-03 | Obsolete? | |
|
||||
| [OpsWorks](https://aws.amazon.com/releasenotes/AWS-OpsWorks?browse=1) | 2013-02 | General | ✓ |
|
||||
| [Redshift](https://aws.amazon.com/releasenotes/Amazon-Redshift?browse=1) | 2013-02 | General | ✓ |
|
||||
| [Elastic Transcoder](https://aws.amazon.com/releasenotes/Amazon-Elastic-Transcoder?browse=1) | 2013-01 | General | ✓ |
|
||||
| [Glacier](https://aws.amazon.com/releasenotes/Amazon-Glacier?browse=1) | 2012-08 | General | ✓ |
|
||||
| [CloudSearch](https://aws.amazon.com/releasenotes/Amazon-CloudSearch?browse=1) | 2012-04 | General | ✓ |
|
||||
| [SWF](https://aws.amazon.com/releasenotes/Amazon-SWF?browse=1) | 2012-02 | General | ✓ |
|
||||
| [Storage Gateway](https://aws.amazon.com/releasenotes/AWS-Storage-Gateway?browse=1) | 2012-01 | General | ✓ |
|
||||
| [DynamoDB](https://aws.amazon.com/releasenotes/Amazon-DynamoDB?browse=1) | 2012-01 | General | ✓ |
|
||||
| [DirectConnect](https://aws.amazon.com/releasenotes/AWS-Direct-Connect?browse=1) | 2011-08 | General | ✓ |
|
||||
| [ElastiCache](https://aws.amazon.com/releasenotes/Amazon-ElastiCache?browse=1) | 2011-08 | General | ✓ |
|
||||
| [CloudFormation](https://aws.amazon.com/releasenotes/AWS-CloudFormation?browse=1) | 2011-04 | General | ✓ |
|
||||
| [SES](https://aws.amazon.com/releasenotes/Amazon-SES?browse=1) | 2011-01 | General | ✓ |
|
||||
| [Elastic Beanstalk](https://aws.amazon.com/releasenotes/AWS-Elastic-Beanstalk?browse=1) | 2010-12 | General | ✓ |
|
||||
| [Route 53](https://aws.amazon.com/releasenotes/Amazon-Route-53?browse=1) | 2010-10 | General | ✓ |
|
||||
| [IAM](https://aws.amazon.com/releasenotes/AWS-Identity-and-Access-Management?browse=1) | 2010-09 | General | ✓ |
|
||||
| [SNS](https://aws.amazon.com/releasenotes/Amazon-SNS?browse=1) | 2010-04 | General | ✓ |
|
||||
| [EMR](https://aws.amazon.com/releasenotes/Elastic-MapReduce?browse=1) | 2010-04 | General | ✓ |
|
||||
| [RDS](https://aws.amazon.com/releasenotes/Amazon-RDS?browse=1) | 2009-12 | General | ✓ |
|
||||
| [VPC](https://aws.amazon.com/releasenotes/Amazon-VPC?browse=1) | 2009-08 | General | ✓ |
|
||||
| [Snowball](https://aws.amazon.com/releasenotes/AWS-ImportExport?browse=1) | 2009-05 | General | ✓ |
|
||||
| [CloudWatch](https://aws.amazon.com/releasenotes/CloudWatch?browse=1) | 2009-05 | General | ✓ |
|
||||
| [CloudFront](https://aws.amazon.com/releasenotes/CloudFront?browse=1) | 2008-11 | General | ✓ |
|
||||
| [Fulfillment Web Service](https://aws.amazon.com/releasenotes/Amazon-FWS?browse=1) | 2008-03 | Obsolete? | |
|
||||
| [SimpleDB](https://aws.amazon.com/releasenotes/Amazon-SimpleDB?browse=1) | 2007-12 | ❗[Nearly obsolete](https://forums.aws.amazon.com/thread.jspa?threadID=121711) | ✓ |
|
||||
| [DevPay](https://aws.amazon.com/releasenotes/DevPay?browse=1) | 2007-12 | General | |
|
||||
| [Flexible Payments Service](https://aws.amazon.com/releasenotes/Amazon-FPS?browse=1) | 2007-08 | Retired | |
|
||||
| [EC2](https://aws.amazon.com/releasenotes/Amazon-EC2?browse=1) | 2006-08 | General | ✓ |
|
||||
| [SQS](https://aws.amazon.com/releasenotes/Amazon-SQS?browse=1) | 2006-07 | General | ✓ |
|
||||
| [S3](https://aws.amazon.com/releasenotes/Amazon-S3?browse=1) | 2006-03 | General | ✓ |
|
||||
| [Alexa Top Sites](https://aws.amazon.com/alexa-top-sites/) | 2006-01 | General ❗HTTP-only | |
|
||||
| [Alexa Web Information Service](https://aws.amazon.com/awis/) | 2005-10 | General ❗HTTP-only | |
|
||||
|
||||
### Compliance
|
||||
|
||||
|
|
@ -391,11 +397,12 @@ Learning and Career Development
|
|||
|
||||
### Certifications
|
||||
|
||||
- **Certifications:** AWS offers [**certifications**](https://aws.amazon.com/certification/) for IT professionals who want to demonstrate their knowledge. They are:
|
||||
- Certified Solutions Architect [Associate](https://aws.amazon.com/certification/certified-solutions-architect-associate/) and [Professional](https://aws.amazon.com/certification/certified-solutions-architect-professional/),
|
||||
[Certified Developer Associate](https://aws.amazon.com/certification/certified-developer-associate/)
|
||||
- [Certified SysOps Administrator Associate](https://aws.amazon.com/certification/certified-sysops-admin-associate/)
|
||||
- [Certified Certified DevOps Engineer Professional](https://aws.amazon.com/certification/certified-devops-engineer-professional/)
|
||||
- **Certifications:** AWS offers [**certifications**](https://aws.amazon.com/certification/) for IT professionals who want to demonstrate their knowledge.
|
||||
- [Certified Solutions Architect Associate](https://aws.amazon.com/certification/certified-solutions-architect-associate/)
|
||||
- [Certified Developer Associate](https://aws.amazon.com/certification/certified-developer-associate/)
|
||||
- [Certified SysOps Administrator Associate](https://aws.amazon.com/certification/certified-sysops-admin-associate/)
|
||||
- [Certified Solutions Architect Professional](https://aws.amazon.com/certification/certified-solutions-architect-professional/)
|
||||
- [Certified DevOps Engineer Professional](https://aws.amazon.com/certification/certified-devops-engineer-professional/)
|
||||
- **Getting certified:** If you’re interested in studying for and getting certifications, [this practical overview](https://gist.github.com/leonardofed/bbf6459ad154ad5215d354f3825435dc) tells you a lot of what you need to know. The official page is [here](https://aws.amazon.com/training/) and there is an [FAQ](https://aws.amazon.com/certification/faqs/).
|
||||
- **Do you need a certification?** Especially in consulting companies or when working in key tech roles in large non-tech companies, certifications are important credentials. In others, including in many tech companies and startups, certifications are not common or considered necessary. (In fact, fairly or not, some Silicon Valley hiring managers and engineers see them as a “negative” signal on a resume.)
|
||||
|
||||
|
|
@ -517,6 +524,7 @@ We cover security basics first, since configuring user accounts is something you
|
|||
- IAM manages various kinds of authentication, for both users and for software services that may need to authenticate with AWS, including:
|
||||
- [**Passwords**](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_passwords.html) to log into the console. These are a username and password for real users.
|
||||
- [**Access keys**](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_access-keys.html), which you may use with command-line tools. These are two strings, one the “id”, which is an upper-case alphabetic string of the form 'AXXXXXXXXXXXXXXXXXXX', and the other is the secret, which is a 40-character mixed-case base64-style string. These are often set up for services, not just users.
|
||||
- 📜 Access keys that start with AKIA are normal keys. Access keys that start with ASIA are session/temporary keys from STS, and will require an additional "SessionToken" parameter to be sent along with the id and secret.
|
||||
- [**Multi-factor authentication (MFA)**](https://aws.amazon.com/iam/details/mfa/), which is the highly recommended practice of using a keychain fob or smartphone app as a second layer of protection for user authentication.
|
||||
- IAM allows complex and fine-grained control of permissions, dividing users into groups, assigning permissions to roles, and so on. There is a [policy language](https://docs.aws.amazon.com/IAM/latest/UserGuide/access_policies.html) that can be used to customize security policies in a fine-grained way.
|
||||
- 🔸The policy language has a complex and error-prone JSON syntax that’s quite confusing, so unless you are an expert, it is wise to base yours off trusted examples or AWS’ own pre-defined [managed policies](https://docs.aws.amazon.com/IAM/latest/UserGuide/access_policies_managed-vs-inline.html).
|
||||
|
|
@ -665,7 +673,7 @@ S3
|
|||
- 🔸After uploading, any change that you make to the object causes a full rewrite of the object, so avoid appending-like behavior with regular files.
|
||||
- 🔸Eventual data consistency, as discussed above, can be surprising sometimes. If S3 suffers from internal replication issues, an object may be visible from a subset of the machines, depending on which S3 endpoint they hit. Those usually resolve within seconds; however, we’ve seen isolated cases when the issue lingered for 20-30 hours.
|
||||
- 🔸**MD5s and multi-part uploads:** In S3, the [ETag header in S3](http://docs.aws.amazon.com/AmazonS3/latest/API/RESTCommonResponseHeaders.html) is a hash on the object. And in many cases, it is the MD5 hash. However, this [is not the case in general](http://stackoverflow.com/questions/12186993/what-is-the-algorithm-to-compute-the-amazon-s3-etag-for-a-file-larger-than-5gb) when you use multi-part uploads. One workaround is to compute MD5s yourself and put them in a custom header (such as is done by [s4cmd](https://github.com/bloomreach/s4cmd)).
|
||||
- 🔸**US Standard region:** Most S3 endpoints match the region they’re in, with the exception of the us-east-1 region, which is called 'us-standard' in S3 terminology. This region is also the only region that is replicated across coasts. As a result, latency varies more in this region than in others. You can minimize latency from us-east-1 by using *[s3-external-1.amazonaws.com](http://s3-external-1.amazonaws.com/)*.
|
||||
- 🔸**US Standard region:** Previously, the us-east-1 region (also known as the US Standard region) was replicated across coasts, which led to greater variability of latency. Effective Jun 19, 2015 this is [no longer the case](https://forums.aws.amazon.com/ann.jspa?annID=3112). All Amazon S3 Regions now support read-after-write consistency. Amazon S3 also renamed the US Standard Region to the US East (N. Virginia) Region to be consistent with AWS regional naming conventions.
|
||||
|
||||
### Storage Durability, Availability, and Price
|
||||
|
||||
|
|
@ -705,7 +713,7 @@ EC2
|
|||
- Selecting instance types is complex since there are so many types. Additionally, there are different generations, released [over the years](https://aws.amazon.com/blogs/aws/ec2-instance-history/).
|
||||
- 🔹Use the list at [**ec2instances.info**](http://www.ec2instances.info/) to review costs and features. [Amazon’s own list](https://aws.amazon.com/ec2/instance-types/) of instance types is hard to use, and doesn’t list features and price together, which makes it doubly difficult.
|
||||
- Prices vary a lot, so use [**ec2instances.info**](http://www.ec2instances.info/) to determine the set of machines that meet your needs and [**ec2price.com**](http://ec2price.com/) to find the cheapest type in the region you’re working in. Depending on the timing and region, it might be much cheaper to rent an instance with *more* memory or CPU than the bare minimum.
|
||||
- [**Dedicated instances**](https://aws.amazon.com/ec2/purchasing-options/dedicated-instances/) and [**dedicated hosts**](https://aws.amazon.com/ec2/dedicated-hosts/) are assigned hardware, instead of usual virtual instances. They more expensive than virtual instances but [can be preferable](https://aws.amazon.com/ec2/dedicated-hosts/) for performance, compliance, or licensing reasons.
|
||||
- [**Dedicated instances**](https://aws.amazon.com/ec2/purchasing-options/dedicated-instances/) and [**dedicated hosts**](https://aws.amazon.com/ec2/dedicated-hosts/) are assigned hardware, instead of usual virtual instances. They are more expensive than virtual instances but [can be preferable](https://aws.amazon.com/ec2/dedicated-hosts/) for performance, compliance, or licensing reasons.
|
||||
- **32 bit vs 64 bit:** A few micro, small, and medium instances are still available to use as 32-bit architecture. You’ll be using 64-bit EC2 (“amd64”) instances nowadays, though smaller instances still support 32 bit (“i386”). Use 64 bit unless you have legacy constraints or other good reasons to use 32.
|
||||
- **HVM vs PV:** There are two kinds of virtualization technology used by EC2, [hardware virtual machine (HVM) and paravirtual (PV)](http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/virtualization_types.html). Historically, PV was the usual type, but [now HVM is becoming the standard](https://www.opswat.com/blog/aws-2015-why-you-need-switch-pv-hvm). If you want to use the newest instance types, you must use HVM. See the [instance type matrix](https://aws.amazon.com/amazon-linux-ami/instance-type-matrix/) for details.
|
||||
- **Operating system:** To use EC2, you’ll need to pick a base operating system. It can be Windows or Linux, such as Ubuntu or [Amazon Linux](https://aws.amazon.com/amazon-linux-ami/). You do this with AMIs, which are covered in more detail in their own section below.
|
||||
|
|
@ -723,7 +731,9 @@ EC2
|
|||
- ❗Never use ssh passwords. Just don’t do it; they are too insecure, and consequences of compromise too severe. Use keys instead. [Read up on this](https://www.digitalocean.com/community/tutorials/how-to-set-up-ssh-keys--2) and fully disable ssh password access to your ssh server by making sure 'PasswordAuthentication no' is in your /etc/ssh/sshd_config file. If you’re careful about managing ssh private keys everywhere they are stored, it is a major improvement on security over password-based authentication.
|
||||
- 🔸For all [newer instance types](https://aws.amazon.com/amazon-linux-ami/instance-type-matrix/), when selecting the AMI to use, be sure you select the HVM AMI, or it just won’t work.
|
||||
- ❗When creating an instance and using a new ssh key pair, [make sure the ssh key permissions are correct](http://stackoverflow.com/questions/1454629/aws-ssh-access-permission-denied-publickey-issue).
|
||||
- 🔸Sometimes certain EC2 instances can get scheduled for retirement by AWS due to “detected degradation of the underlying hardware,” in which case you are given a couple of weeks to migrate to a new instance.
|
||||
- 🔸Sometimes certain EC2 instances can get scheduled for retirement by AWS due to “detected degradation of the underlying hardware,” in which case you are given a couple of weeks to migrate to a new instance
|
||||
- If your instance root device is an EBS volume, you can typically stop and then start the instance which moves it to heathly host hardware, giving you control over timing of this event. Note however that you will lose any instance store volume data ([ephemeral drives](http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/InstanceStorage.html)) if your instance type has instance store volumes.
|
||||
- The instance public IP (if it has one) will likely change unless you're using Elastic IPs. This could be a problem if other systems depend on the IP address.
|
||||
- 🔸Periodically you may find that your server or load balancer is receiving traffic for (presumably) a previous EC2 server that was running at the same IP address that you are handed out now (this may not matter, or it can be fixed by migrating to another new instance).
|
||||
- ❗If the EC2 API itself is a critical dependency of your infrastructure (e.g. for automated server replacement, custom scaling algorithms, etc.) and you are running at a large scale or making many EC2 API calls, make sure that you understand when they might fail (calls to it are [rate limited](http://docs.aws.amazon.com/AWSEC2/latest/APIReference/query-api-troubleshooting.html#api-request-rate) and the limits are not published and subject to change) and code and test against that possibility.
|
||||
- ❗Many newer EC2 instance types are EBS-only. Make sure to factor in EBS performance and costs when planning to use them.
|
||||
|
|
@ -960,6 +970,8 @@ RDS
|
|||
|
||||
- ⏱RDS instances run on EBS volumes, and hence are constrained by the EBS performance.
|
||||
- 🔸Verify what database features you need, as not everything you might want is available on RDS. For example, if you are using Postgres, check the list of [supported features and extensions](http://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/CHAP_PostgreSQL.html#SQLServer.Concepts.General.FeatureSupport). If the features you need aren’t supported by RDS, you’ll have to deploy your database yourself.
|
||||
- 🔸**DB migration to RDS:** While importing your database into RDS ensure you take into consideration the maintenance window settings. If a backup is running at the same time, your import can take a considerable longer time than you would have expected.
|
||||
|
||||
|
||||
DynamoDB
|
||||
--------
|
||||
|
|
@ -1163,14 +1175,14 @@ CloudFront
|
|||
|
||||
- 📒 [Homepage](https://aws.amazon.com/cloudfront/) ∙ [Developer guide](http://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/) ∙ [FAQ](https://aws.amazon.com/cloudfront/faqs/) ∙ [Pricing](https://aws.amazon.com/cloudfront/pricing/)
|
||||
- **CloudFront** is AWS’ [content delivery network (CDN)](https://en.wikipedia.org/wiki/Content_delivery_network).
|
||||
- Its primary use is improving latency for end users in to accessing cacheable content by hosting it at [about 40 global edge locations](http://aws.amazon.com/cloudfront/details/).
|
||||
- Its primary use is improving latency for end users in to accessing cacheable content by hosting it at [over 60 global edge locations](http://aws.amazon.com/cloudfront/details/).
|
||||
|
||||
### CloudFront Alternatives and Lock-in
|
||||
|
||||
- 🚪CDNs are [a highly fragmented market](https://www.datanyze.com/market-share/cdn/). CloudFront has grown to be a leader, but many alternatives that might better suit specific needs.
|
||||
|
||||
### CloudFront Tips
|
||||
|
||||
- 🐥**IPv6** is [now supported](https://aws.amazon.com/about-aws/whats-new/2016/10/ipv6-support-for-cloudfront-waf-and-s3-transfer-acceleration/)!
|
||||
- 🐥**HTTP/2** is [now supported](https://aws.amazon.com/about-aws/whats-new/2016/09/amazon-cloudfront-now-supports-http2/)! Clients [must support TLS 1.2 and SNI](https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/distribution-web-values-specify.html#DownloadDistValuesSupportedHTTPVersions).
|
||||
- While the most common use is for users to browse and download content (GET or HEAD methods) requests, CloudFront also supports ([since 2013](https://aws.amazon.com/blogs/aws/amazon-cloudfront-content-uploads-post-put-other-methods/)) uploaded data (POST, PUT, DELETE, OPTIONS, and PATCH).
|
||||
- You must enable this by specifying the [allowed HTTP methods](https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/distribution-web-values-specify.html#DownloadDistValuesAllowedHTTPMethods) when you create the distribution.
|
||||
|
|
@ -1184,6 +1196,7 @@ CloudFront
|
|||
### CloudFront Gotchas and Limitations
|
||||
|
||||
- If using S3 as a backing store, remember that the endpoints for website hosting and for general S3 are different. Example: “bucketname.s3.amazonaws.com” is a standard S3 serving endpoint, but to have redirect and error page support, you need to use the website hosting endpoint listed for that bucket, e.g. “bucketname.s3-website-us-east-1.amazonaws.com” (or the appropriate region).
|
||||
- 🔸By default, CloudFront will not forward HTTP Host: headers through to your origin servers. This can be problematic for your origin if you run multiple sites switched with host headers. You can [enable host header forwarding](http://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/RequestAndResponseBehaviorCustomOrigin.html#request-custom-headers-behavior) in the default cache behavior settings.
|
||||
|
||||
DirectConnect
|
||||
-------------
|
||||
|
|
@ -1219,10 +1232,11 @@ Redshift
|
|||
- Although Redshift is mostly Postgres-compatible, its SQL dialect and performance profile are different.
|
||||
- Redshift supports only [12 primitive data types](https://docs.aws.amazon.com/redshift/latest/dg/c_Supported_data_types.html). ([List of unsupported Postgres types](https://docs.aws.amazon.com/redshift/latest/dg/c_unsupported-postgresql-datatypes.html)\)
|
||||
- It has a leader node and computation nodes (the leader node distributes queries to the computation ones). Note that some functions [can be executed only on the lead node.](https://docs.aws.amazon.com/redshift/latest/dg/c_SQL_functions_leader_node_only.html)
|
||||
- Major 3rd-party BI tools support Redshift integration (see [Quora](https://www.quora.com/Which-BI-visualisation-solution-goes-best-with-Redshift)).
|
||||
- Major third-party BI tools support Redshift integration (see [Quora](https://www.quora.com/Which-BI-visualisation-solution-goes-best-with-Redshift)).
|
||||
- [Top 10 Performance Tuning Techniques for Amazon Redshift](https://blogs.aws.amazon.com/bigdata/post/Tx31034QG0G3ED1/Top-10-Performance-Tuning-Techniques-for-Amazon-Redshift) provides an excellent list of performance tuning techniques.
|
||||
- [Amazon Redshift Utils](https://github.com/awslabs/amazon-redshift-utils) contains useful utilities, scripts and views to simplify Redshift ops.
|
||||
- [VACUUM](http://docs.aws.amazon.com/redshift/latest/dg/t_Reclaiming_storage_space202.html) regularly following a significant number of deletes or updates to reclaim space and improve query performance.
|
||||
- Redshift provides various [column compression](http://docs.aws.amazon.com/redshift/latest/dg/t_Compressing_data_on_disk.html) options to optimize the stored data size. AWS strongly encourages users to use [automatic compression](http://docs.aws.amazon.com/redshift/latest/dg/c_Loading_tables_auto_compress.html) at the COPY stage, when Redshift uses a sample of the data being ingested to analyze the column compression options. However, automatic compression can only be applied to an empty table with no data. Therefore, make sure the initial load batch is big enough to provide Redshift with a representative sample of the data (the default sample size is 100000 rows).
|
||||
|
||||
### Redshift Gotchas and Limitations
|
||||
|
||||
|
|
@ -1252,11 +1266,16 @@ EMR
|
|||
### EMR Tips
|
||||
|
||||
- EMR relies on many versions of Hadoop and other supporting software. Be sure to check [which versions are in use](https://docs.aws.amazon.com/ElasticMapReduce/latest/ReleaseGuide/emr-release-components.html).
|
||||
- 💸❗**EMR costs** can pile up quickly since it involves lots of instances, efficiency can be poor depending on cluster configuration and choice of workload, and accidents like hung jobs are costly. See the [section on EC2 cost management](#ec2-cost-management), especially the tips there about Spot instances and avoiding hourly billing. [This blog post](http://engineering.bloomreach.com/strategies-for-reducing-your-amazon-emr-costs/) has additional tips.
|
||||
- ⏱Off-the-shelf EMR and Hadoop can have significant overhead when compared with efficient processing on a single machine. If your data is small and performance matters, you may wish to consider alternatives, as [this post](http://aadrake.com/command-line-tools-can-be-235x-faster-than-your-hadoop-cluster.html) illustrates.
|
||||
- Python programmers may want to take a look at Yelp’s [mrjob](https://github.com/Yelp/mrjob).
|
||||
- It takes time to tune performance of EMR jobs, which is why third-party services such as [Qubole’s data service](https://www.qubole.com/mapreduce-as-a-service/) are gaining popularity as ways to improve performance or reduce costs.
|
||||
|
||||
### EMR Gotchas and Limitations
|
||||
- 💸❗**EMR costs** can pile up quickly since it involves lots of instances, efficiency can be poor depending on cluster configuration and choice of workload, and accidents like hung jobs are costly. See the [section on EC2 cost management](#ec2-cost-management), especially the tips there about Spot instances and avoiding hourly billing. [This blog post](http://engineering.bloomreach.com/strategies-for-reducing-your-amazon-emr-costs/) has additional tips.
|
||||
- 💸 Beware of “double-dipping”. With EMR, you pay for the EC2 capacity and the service fees. In addition, EMR syncs task logs to S3, which means you pay for the storage and **PUT requests** at [S3 standard rates](https://aws.amazon.com/s3/pricing/#Request_Pricing). While the log files tend to be relatively small, every Hadoop job, depending on the size, generates thousands of log files that can quickly add up to thousands of dollars on the AWS bill. YARN's [log aggregation](http://hortonworks.com/blog/simplifying-user-logs-management-and-access-in-yarn/) is not available on EMR.
|
||||
|
||||
|
||||
|
||||
High Availability
|
||||
-----------------
|
||||
|
||||
|
|
|
|||
|
|
@ -1,28 +1,33 @@
|
|||
# This file is used to configure the "ghizmo assemble-authors" command.
|
||||
|
||||
header: |
|
||||
This work is a collaborative effort.
|
||||
It was begun and is edited by [@jlevy](https://github.com/jlevy) and [@ThanosBaskous](https://github.com/ThanosBaskous).
|
||||
The following people (in alphabetical order) have contributed to or reviewed this or earlier versions of the guide.
|
||||
|
||||
This “Open Guide” is a collaborative effort.
|
||||
It was begun and is led by [@jlevy](https://github.com/jlevy) and [@ThanosBaskous](https://github.com/ThanosBaskous),
|
||||
but the content is the result of a community of contributors, editors, and experts.
|
||||
Please help if you can, and see the [contribution guidelines](CONTRIBUTING.md) for notes on roles and editorial process.
|
||||
Alphabetically by username:
|
||||
|
||||
footer: |
|
||||
Additional authors are welcome; see the [contribution guidelines](CONTRIBUTING.md).
|
||||
Please let the editors know of any errors or omissions on this list.
|
||||
Please don't PR the AUTHORS.md or authors-info.yml files! It is auto-generated regularly by the project leads.
|
||||
Please let the project leads know of any errors or omissions on this list.
|
||||
|
||||
exclude:
|
||||
gitter-badger
|
||||
ReadmeCritic
|
||||
|
||||
roles:
|
||||
jlevy: general editor
|
||||
ThanosBaskous: general editor
|
||||
max-zanko: editor (S3, EMR, Redshift)
|
||||
jlevy: project lead, editor (topics not otherwise assigned)
|
||||
ThanosBaskous: project lead, editor (topics not otherwise assigned)
|
||||
max-zanko: editor (EC2, S3, Glacier, EMR, Redshift)
|
||||
nitingithub: editor (cost management)
|
||||
forwidur: editor (EBS, RDS)
|
||||
lynnlangit: editor (IoT)
|
||||
richadams: editor (VPC)
|
||||
donnemartin: expert (tools)
|
||||
merrells: expert (cloud infrastructure, when to use AWS)
|
||||
benkehoe: expert (IoT)
|
||||
marcello3d:
|
||||
golyshev:
|
||||
forwidur:
|
||||
merrells:
|
||||
Praveen Patnala:
|
||||
kazuyukitanimura:
|
||||
olawiberg:
|
||||
|
|
@ -30,6 +35,4 @@ roles:
|
|||
bittlingmayer:
|
||||
rjpower:
|
||||
alexanderatallah:
|
||||
donnemartin:
|
||||
benkehoe:
|
||||
Jurgen Philippaerts:
|
||||
|
|
|
|||
Loading…
Reference in a new issue