System Design for Absolute Beginners: The Complete 11-Week Roadmap
Yotek - Innovations That Drive Tomorrow
If you’re an absolute beginner in System Design, this roadmap will guide you step-by-step through all the key concepts using one consistent real-world example: Instagram.
Each week includes foundational theory, architectural trade-offs, and the best visual resources to help you master these concepts practically.
Week 1 – Foundations
What is System Design?
The process of building software systems that can scale (handle hundreds of millions of users), stay reliable (no catastrophic crashes), and remain maintainable (easy to update, inspect, and extend).
The 4 Core Components
- Client: The Instagram mobile app or web client running on your device.
- Server: The compute layer that receives your HTTP/RPC request, executes business logic, and prepares the response.
- API: The contract defining how the client talks to Instagram’s backend infrastructure (e.g.,
GET /feedto fetch your home timeline). - Database: The persistent storage layer storing account profiles, media metadata, social graphs, and reels.
Types of APIs
- REST: The industry standard. Instagram uses REST endpoints to fetch posts, profiles, and comments.
- GraphQL: Developed internally by Meta. Allows clients to declare and request only the exact fields needed, preventing over-fetching over mobile networks.
- gRPC: High-performance, binary protocol used for internal microservice-to-microservice communication.
Functional vs. Non-Functional Requirements
Every system design interview begins by bifurcating requirements:
- Functional Requirements (What the system does): Upload photos/videos, render chronological or ranked feeds, like/comment on posts, follow accounts.
- Non-Functional Requirements (How well it performs): Sub-200ms p99 feed latency, support for 500M Daily Active Users (DAU), 99.99% system availability.
Back-of-Envelope Estimation
Before architecting, estimate throughput and capacity requirements:
- Calculation: Instagram handles ~500M DAU. If each user opens the app 5 times per day and each session queries 20 posts, that totals 50 billion feed reads per day (~580,000 queries per second).
- Architectural Takeaway: Direct relational database queries will instantly fail under this load; multi-tier in-memory caching and horizontal scaling are mandatory from day one.
Visual Resources:
Week 2 – Scaling Basics & Load Balancing
Vertical vs. Horizontal Scaling
- Vertical Scaling (Scale Up): Adding more CPU cores, RAM, or NVMe storage to a single server. Simple, but reaches an unavoidable physical ceiling and introduces a single point of failure (SPOF).
- Horizontal Scaling (Scale Out): Deploying hundreds or thousands of commodity compute instances in parallel. Incoming traffic is distributed across the fleet.
Load Balancer Routing Algorithms
Load balancers evenly distribute user requests across healthy upstream servers:
- Round Robin: Sequential request routing (Server 1 → Server 2 → Server 3).
- Least Connections: Routes new requests to the instance with the fewest active socket connections.
- L4 vs. L7 Load Balancing:
- L4 (Transport Layer): Routes strictly by IP address and TCP port (ultra-fast, protocol-agnostic).
- L7 (Application Layer): Inspects HTTP headers, cookies, and URL paths. Instagram uses L7 routing to direct
/feedrequests to Feed rendering pools and/uploadtraffic to asynchronous media processing workers.
Stateless Architectures
In a horizontally scaled environment, application servers must remain stateless. Authentication tokens (such as signed JWTs) accompany every request, allowing any server in the fleet to validate identity without shared in-memory session state.
Health Checks & Automated Failover
The load balancer continuously pings upstream server endpoints. If an instance fails consecutive heartbeats, traffic is automatically removed from the active routing table without client disruption.
Visual Resources:
Week 3 – Databases & Data Partitioning
SQL vs. NoSQL
- Relational / SQL (PostgreSQL, MySQL): Structured tables with ACID guarantees. Instagram utilizes relational databases for core user identity, follow graphs, and transactional relationships where strong consistency is non-negotiable.
- NoSQL (Apache Cassandra, ScyllaDB, DynamoDB): High-write throughput, horizontally partitionable key-value and wide-column stores. Utilized for write-heavy, append-only data like like counts, comments, and real-time activity feeds where eventual consistency is acceptable.
Database Replication
Data is replicated across multiple availability zones and regions:
- Primary-Replica Setup: The primary instance handles all write mutations and replicates the Write-Ahead Log (WAL) to secondary read replicas.
- Disaster Recovery: If an entire data center region goes offline, DNS failover redirects traffic to replica regions.
Database Sharding (Horizontal Partitioning)
Splitting large datasets across distinct database instances:
- Hash-Based Sharding: Shard ID =
hash(user_id) % num_shards. Ensures all data for a specific user (profile, posts, metadata) resides on the same physical shard for query locality. - The Hotspot (Celebrity) Problem: Viral accounts (e.g., Cristiano Ronaldo) generate disproportionate read/write spikes that can overwhelm an assigned shard. Mitigated through dedicated caching layers and fan-out-on-read architectures.
CAP Theorem & Indexing
Distributed systems must choose between Consistency (C) and Availability (A) under network Partitions (P). High-scale social platforms like Instagram prioritize Availability + Partition Tolerance (AP)—allowing slight propagation delays on like counters rather than rejecting incoming interactions.
-- Database Indexing: Avoid full table scans across billions of records
CREATE INDEX idx_posts_user_created ON posts(user_id, created_at DESC);
Visual Resources:
Week 4 – Caching & Content Delivery Networks (CDN)
In-Memory Caching (Redis / Memcached)
In-memory key-value stores holding precomputed query results and active user feeds. Serving timelines directly from RAM drops read latencies from 50ms down to sub-5ms.
Cache Strategies
- Cache-Aside (Lazy Loading): Application checks cache first. On a cache miss, it reads from the database, writes the record into cache, and returns it. Ideal for read-heavy workloads like user profiles.
- Write-Through: Updates are written to the cache and database synchronously. Prevents stale reads at the expense of higher write latency.
- Write-Back (Write-Behind): Writes are committed to cache immediately; database persistence occurs asynchronously in micro-batches. Ultra-fast writes with risk of data loss during cache failure.
Eviction Policies & TTL
- LRU (Least Recently Used): Purges keys that have remained idle for the longest duration.
- LFU (Least Frequently Used): Purges keys with the lowest access frequency.
- Time-to-Live (TTL): Hard expiration timestamp ensuring stale cached data is periodically invalidated.
Edge Delivery via CDNs
Static media assets (high-resolution photos, short-form reels, avatar thumbnails) are stored in object storage (AWS S3) and cached across a globally distributed Content Delivery Network (Cloudflare, CloudFront, Fastly). When a client in Tokyo requests a post uploaded in Virginia, the media is served directly from the Tokyo edge node.
Visual Resources:
Week 5 – Distributed Systems Building Blocks
Consistent Hashing
Standard modulus hashing (hash(key) % N) causes massive cache thrashing (~(N-1)/N keys remapped) whenever nodes join or leave the cluster.
Consistent Hashing Solution: Maps both servers and data keys to a 360° virtual ring. Keys are assigned to the nearest server clockwise. When a node is added or removed, only K/N keys need migration. Virtual nodes ensure uniform distribution across uneven hardware.
Rate Limiting Algorithms
Protects upstream microservices against DoS spikes and abusive API scraping:
- Token Bucket: Buckets refill at a constant rate ; each request consumes 1 token. Handles bursty traffic up to bucket capacity .
- Sliding Window Log: Stores timestamps of user requests to calculate precise sliding rates, eliminating edge-boundary vulnerabilities of fixed windows.
Blob & Object Storage (AWS S3, Google Cloud Storage)
Relational and NoSQL databases store metadata (post IDs, timestamps, user associations); binary image and video payloads are stored in durable, infinitely scalable object storage buckets with presigned upload URLs.
Visual Resources:
Week 6 – Event-Driven Architecture & Message Queues
Asynchronous Decoupling
When a user uploads a video to Instagram:
- The Upload Service accepts the payload, generates an event
video.uploaded, and responds with202 Acceptedimmediately. - Multiple downstream worker services independently consume the event to transcode resolutions (1080p, 720p, 480p), run ML safety moderation, extract thumbnail frames, and fan out notifications.
Kafka vs. Traditional Message Queues (RabbitMQ / SQS)
- RabbitMQ / SQS (Task Queue): Point-to-point delivery where messages are deleted upon consumer acknowledgment.
- Apache Kafka (Distributed Append-Only Commit Log): High-throughput partition logs where events are retained across configurable time windows. Multiple consumer groups read from the same topic at independent offsets and can replay historical events.
Dead Letter Queues (DLQ)
Unparseable or corrupted messages that fail maximum retry thresholds are routed to an isolated DLQ for telemetry analysis, preventing pipeline blockage.
Visual Resources:
Week 7 – Networking, APIs & Microservices
Protocols: REST vs. WebSockets
- HTTP/HTTPS (Stateless Request-Response): Standard mechanism for feed scrolling, post publishing, and profile lookups.
- WebSockets (Full-Duplex Persistent Sockets): Enables real-time bidirectional communication for Instagram Direct Messaging and live stream comments without continuous client polling.
API Design Best Practices
- HTTP Status Code Precision:
200 OK,201 Created,400 Bad Request,401 Unauthorized,429 Too Many Requests,500 Internal Server Error. - Cursor-Based Pagination: Using timestamps or unique record IDs (
?cursor=eyJpZCI6MTIzfQ==) instead of offset-based pagination (OFFSET 10000) to prevent performance degradation on large datasets. - Idempotency Keys: Unique UUIDs passed in request headers ensuring retried network requests (e.g., payment charges or post creations) are executed exactly once.
API Gateway Pattern
Acts as the single front door for all mobile and web traffic. Handles TLS termination, token authentication, rate limiting, and dynamic path routing to internal microservices.
Visual Resources:
Week 8 – Security, Authentication & Authorization
Authentication vs. Authorization
- Authentication (AuthN): Validating the user's identity (Username/Password + MFA).
- Authorization (AuthZ): Enforcing access control (e.g., ensuring User A cannot delete User B's post).
Session Cookies vs. JSON Web Tokens (JWT)
- Session Identifiers: Requires a centralized session lookup in Redis on every request; introduces storage bottlenecks at massive scale.
- Stateless JWTs: Cryptographically signed tokens containing embedded claims (user ID, expiration, roles). Any upstream microservice can verify validity locally via public key cryptography without database lookups.
Defense-in-Depth Security
- Transport Encryption: TLS 1.3 encryption across all public and internal service meshes.
- Password Hashing: Storing non-reversible adaptive hashes using
bcryptorArgon2idwith cryptographic salts. - CORS & CSRF Defenses: Enforcing origin whitelisting and Anti-CSRF token verification.
Visual Resources:
Week 9 – High-Level Design (HLD) Interview Framework
When tackling end-to-end HLD questions, follow this repeatable 5-step blueprint:
- Clarify Requirements (5 mins): Scope core features, expected scale (DAU/MAU), read/write ratios, and latency/availability targets.
- Back-of-Envelope Estimation (5 mins): Calculate requests per second (QPS), storage growth per year, and network bandwidth.
- High-Level Diagram (15 mins): Lay out client entry, API Gateway, primary microservices, message queues, caches, and storage engines.
- Component Deep Dive (15 mins): Address bottleneck resolution (fan-out write vs. fan-out read, consistent hashing, caching topology).
- Trade-Off Analysis (5 mins): Articulate CAP theorem choices, single points of failure, and operational monitoring.
Week 10 – Low-Level Design (LLD) & Object-Oriented Principles
Core Design Patterns
- Creational: Singleton, Factory Method, Builder Pattern.
- Structural: Adapter, Decorator, Facade, Composite Pattern.
- Behavioral: Observer, Strategy, Command, Chain of Responsibility.
SOLID Principles
- S – Single Responsibility Principle: A class should have only one reason to change.
- O – Open/Closed Principle: Open for extension, closed for modification.
- L – Liskov Substitution Principle: Subtypes must be substitutable for their base types.
- I – Interface Segregation Principle: Favor small, client-specific interfaces over bloated ones.
- D – Dependency Inversion Principle: Depend on abstractions, not concrete implementations.
Visual Resources:
Week 11 – Interview Prep & Mock Whiteboarding
Comprehensive Preparation Playlists
- System Design Full Course & Mock Interview (Scaler)
- Design LeetCode System Design Mock (Anubhav Sethi)
- Design TikTok System Design Mock (Exponent)
This roadmap serves as the companion deep-dive for the System Design series. Bookmark it and follow through sequentially as you prepare for technical interviews.