Awesome
How they AWS
| 
A curated collection of publicly available resources on how technology and tech-savvy organizations around the world use Amazon Web Services (AWS)
Introduction
How they AWS is a curated knowledge repository of engineering blog posts and videos covering architectures, deployment patterns and use of various AWS services by the leading technology or tech-savvy organizations.
Many organizations regularly come forward and share their best practices, architecture, trade-offs, tools, techniques and offer an insight into engineering culture on various public platforms like blogs, conferences & meetups. The content is curated from these avenues and shared in this repository.
Note to readers: This list refers to some of the articles, posts, videos, tools, and techniques published before 2015 (marked with :exclamation:). Please use such material with caution as there may be recent advances in technology and practices which offer better alternatives and perspectives.
Topics
- AWS Compute, Storage, Network, Database services
- Architectures
- Deployment Patterns
- Infrastructure as Code
- SRE, DevOps
- Incidents
- Automation
- Scalability, Performance
- Security
Organizations
<details> <summary>A Cloud Guru</summary>Blog Posts
- How we reduced Lambda cold starts at ACG, 2020
- Optimizing a multimillion-dollar cloud bill
- Streaming Video at Scale: A Tale of Transcoding at ACG, 2020
Videos
- Natasja Laurie: The tale of transcoding at A Cloud Guru, 2020
- Serverless: From one function to 43 microservices
- How A Cloud Guru does Serverless, 2019
- The Story of a Serverless Startup, 2017
Blog Posts
- Our Journey Towards Cloud Efficiency, 2021
- Achieving Insights and Savings with Cost Data, 2021
- BinaryAlert: Real-time Serverless Malware Detection , 2017
- Introducing Syslog to AWS Kinesis via Osquery, 2016
- Unlocking Horizontal Scalability in Our Web Serving Tier, 2016
- When The Cloud Gets Dark — How Amazon’s Outage Affected Airbnb, 2011 :exclamation:
Blog Posts
- Making Search Talk: Connecting Algolia and Alexa, 2017
- Tips for Reducing the Cost of Your Infrastructure, 2017
- DNS fallback for better resilience, 2015
Blog Posts
-
How we modified our infrastructure to deploy an EU data center, 2020
-
How to build stable, accessible data infrastructure at a startup, 2014 :exclamation:
-
Issues Moving to Amazon’s Elastic Load Balancer, 2012 :exclamation:
Blog Posts
- Automated Testing: 5 Lessons from Atlassian’s Kubernetes team on testing infrastructure as code, 2020
- Using an event-driven architecture to improve Jira Software responsiveness, 2019
- Caching in: performance engineering in Jira Cloud, 2019
- Our not-so-magic journey scaling low latency, multi-region services on AWS, 2019
Migrating the Atlassian Identity Platform to AWSScaling React server-side rendering in Jira Cloud
Blog Posts
- AWS S3: You’re out of order, 2020
- How to waste half a day by not reading RFC 1034, 2020
- Running spot instances effectively with Amazon EKS, 2020
- Seamless branch deploys with Kubernetes, 2020
Blog Posts
- 10 lessons learnt scaling Redshift Cluster at Box, 2019
- Snitch: Putting consistency back into S3, 2018
Blog Posts
- Building Intelligent, Elastic Architectures in the Cloud
- Moving to DynamoDB to Increase Application Resiliency, 2021
- Advice on Taking the AWS Machine Learning — Specialty Exam, 2021
- Automate AWS Infrastructure with Boto 3 — AWS Instance Tags, 2020
- Automate AWS Infrastructure with Boto 3: AWS Health Checks, 2020
- Automate Your AWS Infrastructure with Boto 3 — AWS Snapshots, 2020
- Guardrails for AWS Event-Driven Serverless Architectures, 2020
- Starting Out with AWS & DevOps — 10 Tips From an Expert, 2020
- Active-Active Shared-Nothing Database Architecture, 2020
- AWS Lambda Java Tutorial: Best Practices to Lower Cold Starts, 2020
- AWS Glue: An ETL Solution with Huge Potential, 2020
- Terraform Poka-Yokes — Writing Effective, Scalable, Dynamic, and Error-Resistant Terraform, 2020
- A Developer Walks into Amazon SageMaker…, 2029
- A Deep Dive Into Seamless Blue/Green Deployment Using AWS CodeDeploy, 2019
- 4 Serverless Myths to Understand Before Getting Started with AWS, 2019
- Best Practices for AWS Lambda Container Reuse, 2019
- Building Feature Toggles into Terraform, 2019
- Multi-Region Deployments with Terraform, 2018
- Applying Minification and Uglification to AWS Lambda Functions, 2018
- Serverless Transactions Serve Customers, 2017
- Building CelebritySleuth — A Serverless Framework Application, 2017
- Moving One of Capital One’s Largest Customer-Facing Apps to AWS, 2017
Blog Posts
- Using AWS EC2 and ECS to host hundreds of services, 2020
- CloudFormation To Terraform, 2020
- Building a Payments Lambda with Terraform, 2018
Blog Posts
- High Slope Traffic & K8s Shenanigans, 2020
- Journey from EC2 to Containers, 2020
- GoCD on Kubernetes, 2020
- Building Pubsub for 50M concurrent socket connections, 2019
- Scaling for Tsunami traffic, 2019
- Infrastructure Security as a product for Hotstar, 2020
- Saving Millions : Leveraging EC2 Spots at scale, 2018
Videos
</details> <details> <summary>Dropbox</summary>Blog Posts
- Alki, or how we learned to stop worrying and love cold metadata, 2020
- How we designed Dropbox ATF: an async task framework, 2020
- Scaling to exabytes and beyond, 2016
Blog Posts
- DynamoDB: Guidelines for faster reads and writes, 2019
- DynamoDB: Efficient Indexes, 2019
- How to run a Hadoop Application in an ECS Cluster, 2019
- DynamoDB: Data Modeling, 2018
- DynamoDB: Why migrate to DynamoDB from Cassandra?, 2018
- Using API Gateway for Authorization and Authentication, 2018
- re:Invent 2017 — Getting Started with Amazon Aurora, 2017
- AWS Lessons Learned for Data Processing Pipelines, 2016
Blog Posts
</details> <details> <summary>Freshworks</summary>Blog Posts
</details> <details> <summary>GoDaddy</summary>Blog Posts
- GoDaddy's Journey to the Cloud | Part 1, 2021
- GoDaddy and AWS join forces to release a Serverless plugin for use with AWS Service Catalog, 2021
- Securing the Cloud: The GoDaddy Way, 2019
- Connecting an On-Premises Data Center to AWS with HA Software VPN Tunnels, 2019
- GoDaddy and Amazon EKS, 2018
Blog Posts
- Trident - Real-time event processing at scale, 2021
- Optimally scaling Kafka consumer applications, 2020
- How We Simplified Our Data Ingestion & Transformation Process, 2019
- A Lean and Scalable Data Pipeline to Capture Large Scale Events and Support Experimentation Platform, 2019
- Querying Big Data in Real-Time with Presto & Grab's TalariaDB, 2019
- Troubleshooting Unusual AWS ELB 5XX Error, 2017
Videos
</details> <details> <summary>Grammarly</summary>Blog Posts
- Security Operations in an AWS Environment, 2020
- Perfecting Smooth Rolling Updates in Amazon Elastic Container Service, 2020
- Lessons We Learned From Moving Our On-Premise macOS Build Environment to AWS
Blog Posts
- Introducing arn, a library for working with AWS ARNs, 2020
- Terraforming RDS — Part 1, 2020
- Terraforming RDS — Part 2, 2020
- Terraforming RDS — Part 3, 2020
- Terraforming RDS — Bonus Anecdote, 2020
- Creating a Logical Replica from a Snapshot in RDS Postgres, 2020
Blog Posts
</details> <details> <summary>Lyft</summary>Blog Posts
- IAM whatever you say IAM, 2020
- Announcing cni-ipvlan-vpc-k8s: IPvlan overlay-free Kubernetes Networking in AWS, 2017
- SaltStack as an Alternative to Terraform for AWS Orchestration, 2017
- Overcoming AWS Complexity with SaltStack patterns, 2017
- Extending IAM Policy and AWS APIs Using KMS and Lambda, 2016
- Scoping AWS IAM roles to Docker containers, 2016
Blog Posts
- Scaling Email Infrastructure for Medium Digest, 2020
- Starting FARGATE, 2017
- Medium’s DynamoDB Data Source for Apache Spark, 2016
- How Medium Detects Hotspots in DynamoDB using ElasticSearch, Logstash and Kibana, 2015
Blog Posts
- ConsoleMe: A Central Control Plane for AWS Permissions and Access, 2021
- Unbundling Data Science Workflows with Metaflow and AWS Step Functions, 2020
- Building Netflix’s Distributed Tracing Infrastructure, 2020
- How Netflix is able to enrich VPC Flow Logs at Hyper Scale to provide Network Insight, 2020
- Byte Down: Making Netflix’s Data Infrastructure Cost-Effective, 2020
- How Netflix microservices tackle dataset pub-sub, 2019
- Cache warming: Agility for a stateful service, 2018
- Netflix Information Security: Preventing Credential Compromise in AWS, 2018
- Netflix Cloud Security: Detecting Credential Compromise in AWS, 2018
- Auto Scaling Production Services on Titus, 2018
- Netflix Billing Migration to AWS, 2016
- Netflix Billing Migration to AWS — Part II, 2016
- Netflix Billing Migration to AWS — Part III, 2016
- Creating Your Own EC2 Spot Market, 2015
- Using Presto in our Big Data Platform on AWS, 2014 :exclamation:
- Lessons Netflix Learned from the AWS Outage, 2011 :exclamation:
- 5 Lessons We’ve Learned Using AWS, 2010 :exclamation:
Videos
- AWS re:Invent 2019: Capacity management made easy with Amazon EC2 Auto Scaling (CMP326-R1)
- AWS re:Invent 2019: A day in the life of a Netflix engineer (NFX202)
- AWS re:Invent 2019: Monitoring anomalous application behavior (NFX205)
Blog Posts
</details> <details> <summary>Postman</summary>Blog Posts
- How Postman Engineering handles a million concurrent connections, 2020
- Kubernetes Tutorial: Your Complete Guide to Deploying an App on AWS with Postman, 2020
- Auditing AWS IAM for better security practices, 2019
- Automate monitoring of inactive cache clusters, 2019
Blog Posts
- Launching Worker Pod Autoscaler — Solving specific problems with worker scaling in Kubernetes, 2019
- Serverless flows with Step Functions, 2019
- Container Logging @ Practo, 2018
- Blue Green Deployment, 2017
Blog Posts
- Adopting Kubernetes at Quora, 2019
- Qmessage: Handling Billions of Tasks Per Day, 2018
- Automated Infrastructure Cost Optimization at Scale with AWS EC2 Reserved Instances, 2018
- Ensuring Quora's Resilience to Disaster, 2016
Blog Posts
</details> <details> <summary>Scribd</summary>Blog Posts
- Automatically recycling EKS worker nodes, 2020
- Using Panther to monitor AWS infrastructure, 2020
- Using Terraform to integrate Datadog and AWS, 2020
- Easy read-only ECR access for the entire AWS Organization, 2020
Blog Posts
</details> <details> <summary>Slalom Build</summary>Blog Posts
- Crafting AWS Accounts: Applications Accounts, 2020
- Container Image Support in AWS Lambda, 2020
- The Anatomy of a CDK App, 2020
- Solving cold starts on AWS Lambda when using .NET Core, 2020
- Crafting Secure AWS Environments: Using an AWS Multi-Account Environment, 2020
- Creating Isolated Serverless Environments Using AWS, Terraform, and Jenkins, 2020
- AWS Multi-Account Architecture with Terraform, Yeoman, and Jenkins, 2018
- How to build a secure by default Kubernetes cluster with a basic CI/CD pipeline on AWS, 2018
- Event Sourcing with AWS Lambda, 2018
- Serverless: The New Cloud Trend, 2017
Blog Posts
- The secret life of DNS packets: investigating complex networks, 2020
- Effectively using AWS Reserved Instances, 2018
- AWS Case Study: Stripe
Blog Posts
- Real-life AWS infrastructure cost optimization strategy, 2017
- Reducing AWS EMR data processing costs, 2020
Blog Posts
</details> <details> <summary>Udemy</summary>Blog Posts
- Delivering AI/ML Products Efficiently: The Single-Node Machine Learning Workflow, 2021
- Supporting Multiple Time Zones on Hive with Single Data Source, 2020
- Improving Amazon Redshift Performance: Our Data Warehouse Story, 2018
Blog Posts
- Orchestrating Cassandra on Kubernetes with Operators, 2020
- Autoscaling AWS Step Functions Activities, 2019
- Breaking down the monolith with AWS Step Functions, 2017
Resources
- You can also find over 500+ videos in This is My Architecture Series presented by various customers of AWS.
Credits
- Inspired by Howtheytest from Abhijeet Vaikar
- The list of organizations is referred from my other repo awesome-engineering.
- Banner image Cartoon vector created by vectorjuice - www.freepik.com
Other How They... repos
Contribute
Contributions welcome! Read the contribution guidelines first.
License
To the extent possible under law, Unmesh Gundecha has waived all copyright and related or neighboring rights to this work.
If you decide to use this anywhere please give a credit to @upgundecha on twitter, also If you like my work, check out other projects on my Github.