Section 5

Data Storage & Processing

Database scalability, ACID vs BASE, isolation levels, message brokers, exactly-once semantics, and big-data pipelines.

8 lessons ยท ~1.2 h reading

Database Design

5.1

Scalability and Data Modeling in Databases

10 min โ†’

How to think about database scalability โ€” workload patterns, data models, and the trade-offs between relational, document, wide-column, and NoSQL choices.

5.2

ACID vs BASE Trade-offs

7 min โ†’

Two paradigms for data consistency. What ACID and BASE actually guarantee, when each applies, and how modern systems blend them.

5.3

Transaction Isolation Levels

8 min โ†’

The four standard isolation levels, the anomalies each one allows, and how to pick the right level for each operation.

Messaging & Processing

5.4

Distributed Search

8 min โ†’

How distributed search engines work โ€” inverted indexes, sharding, scoring, and the trade-offs of Elasticsearch, OpenSearch, and the modern alternatives.

5.5

Message Queue Systems

9 min โ†’

The major message broker categories and when each fits. Queues vs. streams, RabbitMQ vs. Kafka vs. SQS, and the architectural patterns they support.

5.6

Exactly-Once Semantics

8 min โ†’

The myth, the reality, and the patterns that approximate "exactly-once" in production. Idempotency, deduplication, and the practical tools.

5.7

Big Data Processing

8 min โ†’

Batch and streaming at scale โ€” Spark, Flink, Beam, and the modern data stack. The architectural patterns that move TB-scale data through pipelines.

5.8

Kafka Deep Dive

12 min โ†’

How Kafka actually works โ€” partitions, replication, producers, consumers, transactions. The mental model for using it well in production SaaS.

Start with 5.1 โ†’