Awesome
System Design Resources
These are the best resources for System Design on the Internet.
Table of Contents
- Video Processing
- Cluster and Workflow Management
- Intra-Service Messaging
- Message Queue Antipattern
- Service Mesh
- Practical System Design
- Distributed File System
- Time Series Databases
- Rate Limiting
- In Memory Database - Redis
- Network Protocols
- Chess Engine Design
- Subscription Management System
- Google Docs
- API Design
- NoSQL Database Internals
- NoSQL Database Algorithms
- Database Replication
- Containers and Docker
- Capacity Estimation
- Publisher Subscriber
- Event Driven Architectures
- Software Architectures
- Microservices
- Distributed Transactions consistency Patterns
- Load Balancing
- Alerts and Anomaly Detection
- Distributed Logging
- Metrics and Text Search Engine
- Single Point of Failure
- Location Based Services
- Batch Processing
- Real Time Stream Processing
- Caching
- Distributed Consensus
- Authorization
- Content Delivery Network
- Testing Distributed Systems
- System Design Resources
Video Processing
- Transcoding Videos at Scale
- Facebook Video Broadcasting
- Netflix Video Encoding at Scale
- Netflix Shot based encoding
Cluster and Workflow Management
- Facebook Cluster Management
- Google Autopilot - Autoscaling
- Netflix Workflow Orchestration
- Opensource Workflow Management
- Meta Hardware Management
- Meta Capacity Assignment
- Amazon EC2
Intra-Service Messaging
Message Queue Antipattern
- DB as queue Antipattern
- Using a database as a message queue
- Anti-pattern of DB as a queue
- Drawbacks of DB as a queue
Service Mesh
- Kubernetes Service Mesh
- Kubernetes Sidecar
- Service Mesh
- NginX Service Mesh
- Data Plane and Control Plane
Practical System Design
- Facebook Messenger Optimisations
- YouTube Architecture
- YouTube scalability 2012
- Distributed Design Patterns
- Monolith to Microservice
- Zerodha Tech Stack
Distributed File System
Time Series Databases
- Pinterest Time Series Database
- Uber Time Series DB
- TimeSeries Relational DB
- Facebook Gorilla Time Series DB
Rate Limiting
In Memory Database - Redis
- Redis Official Documentation
- Learn Redis through Redis University
- Redis Open Source Repo
- Redis Architecture
Network Protocols
- What is HTTP
- QUIC Protocol
- TCP Protocol algorithms (First 10 pages are important)
- WebRTC
- WebSockets
- Dynamic Source Routing using QUIC
Chess Engine Design
Subscription Management System
Google Docs
API Design
NoSQL Database Internals
- Cassandra Architecture
- Google BigTable Architecture
- Amazon Dynamo DB Internals
- Design Patterns in Amazon Dynamo DB
- Internals of Amazon Dynamo DB
NoSQL Database Algorithms
- Hyperloglog Algorithm
- Log Structured Merge Tree
- Sorted String Tables and Compaction Strategies
- Leveled Compaction Cassandra
- Scylla DB Compaction
- Indexing in Cassandra
Database Replication
- Database replication
- Netflix Data replication - Change Data Capture
- LinkedIn Logging Usecases
- Uber Trillions of indexes in LedgerStore
Containers and Docker
Capacity Estimation
- Google Capacity Estimation
- Scalability at YouTube 2012
- Back of envelope Calculations at AWS
- Capacity Estimation
Publisher Subscriber
Event Driven Architectures
Software Architectures
- Hexagonal Architecture
- Hexagonal architecture (Alistair Cockburn)
- The Clean Code by Robert C. Martin (Uncle Bob)
- CQRS
- DomainDrivenDesign
Microservices
- Monolith Architecture
- Monoliths vs Microservices
- Microservices
- Uber Nanoservices antipattern
- Uber Domain oriented microservice
Distributed Transactions consistency Patterns
Load Balancing
- Load Balancer with Sticky Sessions
- NetScaler what is load balancing
- Nginx Load Balancing
- Consistent hashing
- Minimizing connection churn
- Google Subsetting Algorithm
Alerts and Anomaly Detection
- Outlier Detection
- Anomaly Detection
- Uber Real Time Monitoring and Root Cause Analysis Argos
- Microsoft Anomaly Detection
- Facebook Data Engineering
- LinkedIn Real Time Alerting
- LinkedIn Isolation Forests
Distributed Logging
Metrics and Text Search Engine
Single Point of Failure
- Avoiding Single Points of Failure
- Netflix Multi-Region Availability
- Oracle Single Points of failure
- DNS single point of failure 2004
- DNS traffic management by Shopify
- Sharding
Location Based Services
Batch Processing
Real Time Stream Processing
- LinkedIn Brooklin- Real-time data streaming
- Netflix Real Time Stream Processing
- KSQLDB for Kafka
- Netflix Psyberg
Caching
- Google Guava Cache
- Caching (See the README)
- Caching
- Microsoft Caching Guide
- Caching patterns
- Uber's Integrated Cache for 40M RPS
Distributed consensus
Authorization
- Designing an Authorization Model for an Enterprise
- The Architectural Patterns of Cloud-native Authorization Systems