← All posts

CAP Is Incomplete: PACELC and the Consistency Choices You Make on Every Query

December 4, 2025

CAP is often misread as "pick two of three forever and put it in the architecture doc." The consistency-availability trade-off fires during partitions. The trade-off you live with every day, on every query path in a healthy cluster, is latency versus consistency. That everyday axis is what PACELC makes explicit.

If you already know ACID vs BASE and sharding vs replication, this post is the decision frame those pieces sit inside: what C/A/P actually mean, why partitions force CP vs AP, how the consistency spectrum maps to real operations, and how to choose per query instead of per system.

CAP, defined without the folklore

Brewer's theorem says a distributed system cannot simultaneously guarantee all three:

Network partitions are not a rare academic event. Links flap, AZs degrade, GC pauses look like death, DNS lies. In practice you assume P is required. The real choice under a partition is CP or AP.

CAP triangle: Consistency, Availability, Partition tolerance

Important: CAP "consistency" is not ACID's C (constraints). Mixing them up produces meetings that go nowhere. CAP C is about replica agreement under concurrent operations; ACID C is about invariants inside a transaction.

CP vs AP in production language

CP (consistency + partition tolerance). During a split, some clients get errors or waits rather than a wrong answer. Bank ledger debit, inventory reservation that must not oversell, auth token revocation: better down than wrong. After recovery, you do not reconcile conflicting balances; you never created them.

AP (availability + partition tolerance). During a split, reads and writes keep succeeding. Reads may be stale; writes may not be visible everywhere yet. Shopping cart contents, activity feeds, product recommendations: better available with a brief lie than a 503 on Black Friday.

Neither is "more distributed." They encode different business failure modes.

The consistency spectrum (pick per operation)

Linearizability is the top of a ladder. Production systems climb down deliberately:

Consistency spectrum from linearizable to eventual

Most mature SaaS codebases do not pick one rung for the whole system. Payments and auth use strong guarantees. Search indexing, analytics rollups, and "users also bought" tolerate eventual. The architecture doc should list operations, not a single CAP letter.

PACELC: the rest of the time

CAP is silent about healthy clusters. PACELC fills that gap:

If Partition → Availability vs Consistency. Else → Latency vs Consistency.

PACELC decision tree: partition then A vs C, else L vs C

Why latency vs consistency even when the network is fine? Resilience requires replicas, often across AZs or regions. Strong consistency waits for synchronous replication acknowledgements. Asynchronous replication returns fast and converges later.

Three replicas, synchronous credit of $100:

Steps of synchronous replication before client ack

Latency grows with replica count and RTT. Skip the wait and you have eventual consistency: PACELC's EL (prefer latency) rather than EC (prefer consistency).

How common systems land on the map

CAP and PACELC classification of popular systems

The asterisk on MongoDB matters: defaults are not destiny. Quorum and read/write concern knobs move you along the spectrum per call.

Navigation patterns that ship

Tunable consistency. Cassandra/Dynamo-style ONE / QUORUM / ALL (and the dual on writes). Use quorum when the answer must be recent; use ONE when a slightly stale read is fine and latency budgets are tight. With replication factor N, choosing R and W such that R+W > N gives overlap that detects the latest write, without needing every replica on every call.

Async writes with sticky read-your-writes. Acknowledge the write quickly; for a few seconds (or until replication catches up), steer that user's reads to the leader or a session-sticky replica. Most users never notice the underlying eventual model.

Strong only when needed. A useful planning heuristic: roughly 20% of operations genuinely need CP (payments, auth, ledger, entitlement checks); roughly 80% are happier on AP/EL. Check which bucket an endpoint is in before you pay Spanner prices for a feed query.

Multi-tenant SaaS lens

Which SaaS workloads prefer strong vs eventual consistency

Within a tenant, the domain is often small enough that strong consistency is cheap, especially with DB-per-tenant. Cross-tenant aggregates (billing rollups, platform analytics) almost always accept eventual. Shared-schema multi-tenancy makes CP harder: one hot tenant's quorum traffic shares fate with everyone else. Isolation model and consistency model are coupled decisions.

What belongs in the architecture doc

Not "we are a CP system." Write the operation:

That list drives database choice, quorum settings, retry policy, and what you tell support when an AZ dies.

A worked partition scenario

Two AZs, a ledger service, replication factor three. AZ-B loses connectivity to AZ-A for ninety seconds. Under CP, the minority side refuses writes to the account balance: clients get timeouts or 503s, support sees errors, balances stay correct. Under AP, both sides accept transfers; when the link heals you reconcile conflicting histories (last-write-wins, merge functions, or human ops). Banks choose the first pain. Collaborative carts often choose the second.

Same cluster, healthy network, cross-region replica: every strongly consistent read now waits on the far RTT. That is PACELC's Else branch. Product managers feel it as "the page got slow after we added DR," not as a theorem.

Anti-patterns

Key takeaways

  1. CAP binds under partition; partitions are inevitable, so the live choice is CP vs AP.
  2. CAP consistency means linearizability, which is stronger than most product language of "consistent."
  3. PACELC adds the everyday axis: latency vs consistency when the network is fine.
  4. Real systems pick per operation along a spectrum, not one letter for the fleet.
  5. Tunable quorums, sticky read-your-writes, and "strong only for money/auth" are how you navigate without dogma.
  6. For SaaS: strong inside the tenant where it matters; eventual for feeds, search, and cross-tenant analytics.

Stop arguing about whether the company is CP or AP. Argue about the query.