<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0"><channel><title><![CDATA[The Byte Mechanics]]></title><description><![CDATA[The Byte Mechanics]]></description><link>https://thebytemechanics.hashnode.dev</link><generator>RSS for Node</generator><lastBuildDate>Wed, 18 Nov 2026 15:13:08 GMT</lastBuildDate><atom:link href="https://thebytemechanics.hashnode.dev/rss.xml" rel="self" type="application/rss+xml"/><language><![CDATA[en]]></language><ttl>60</ttl><item><title><![CDATA[Why One Discord Channel Could Slow Down Everyone]]></title><description><![CDATA[Hot partitions, shared reads, a live database migration, and the deleted messages that froze it at 99.9999%]]></description><link>https://thebytemechanics.hashnode.dev/why-one-discord-channel-could-slow-down-everyone</link><guid isPermaLink="true">https://thebytemechanics.hashnode.dev/why-one-discord-channel-could-slow-down-everyone</guid><dc:creator><![CDATA[The Byte Mechanics]]></dc:creator><pubDate>Wed, 18 Nov 2026 14:47:46 GMT</pubDate><enclosure url="https://cdn.hashnode.com/uploads/covers/6ae732d54803f0be26b37004/c4d334c2-c229-47ca-b5fb-dc32ccacb8bc.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>One busy Discord channel could slow down conversations that had nothing to do with it.</p>
<p>Discord was storing trillions of messages across 177 database servers. Adding more servers didn't fix the real problem. So the engineers changed how requests reached the database, and then moved every message to a new one. When that move was 99.9999% complete, it stopped. The reason: messages people had already deleted.</p>
<p>This article follows a single message through all of it. If you'd rather watch, here's the animated version:</p>
<p><a href="https://youtu.be/N5KyGD7fRTQ">Watch Here</a></p>
<p><em>This is a historical case study. The migration finished in May 2022 and Discord described it in March 2023; the background comes from Discord's January 2017 post. It isn't Discord's architecture today. Diagrams of servers and traffic are illustrative.</em></p>
<img src="https://cdn.hashnode.com/uploads/covers/6ae732d54803f0be26b37004/2b6ec00f-bcf3-4114-8631-787a46a3594a.png" alt="fig02" style="display:block;margin:0 auto" />

<h2>Where a message lives</h2>
<p>Start with what the app actually asks for. Open a channel and you need its newest messages. Scroll up and you need the ones just before. That's an ordered slice of one channel's history.</p>
<p>So Discord stored messages by an <strong>address</strong> and a <strong>position</strong>. The 2017 post describes a schema like this (simplified):</p>
<pre><code class="language-text">PRIMARY KEY ((channel_id, bucket), message_id)
</code></pre>
<p>The part in the inner parentheses is the <strong>partition key</strong>: the channel plus a time window called a bucket. It decides which machines hold the data. The <code>message_id</code> is a Snowflake, an ID that sorts roughly by creation time, so it gives each message its position inside that group.</p>
<img src="https://cdn.hashnode.com/uploads/covers/6ae732d54803f0be26b37004/cbc9a181-a702-4766-b431-02ebd8532c0c.png" alt="fig03" style="display:block;margin:0 auto" />

<p>Think of a shelf. The address takes you to the right shelf. The position tells you where to start reading.</p>
<p>Why the bucket? A busy channel keeps growing for years, and buckets cut its history into pieces of manageable size (the 2017 post describes 10-day buckets; that's historical, not a current setting). The trade-off is that a quiet channel may need to check several buckets to fill one screen.</p>
<img src="https://cdn.hashnode.com/uploads/covers/6ae732d54803f0be26b37004/6953788f-072e-4105-bb29-e8a682b6693b.png" alt="fig04" style="display:block;margin:0 auto" />

<p>But notice what a bucket limits. It limits how much history sits in one place. It does not limit how many people ask for it at once.</p>
<h2>Spreading data is not spreading demand</h2>
<p>Cassandra, the database Discord used, spreads partitions across machines by hashing their keys, and keeps three copies of each.</p>
<p>Now make one channel wildly popular. Every request for it still goes to the machines that hold its copies.</p>
<p>Would adding more servers help? It helps the cluster, but not this channel. New servers take over some ranges of data from the old ones, which frees up capacity overall. A single partition is never split, though. It moves as one piece, and all of its traffic moves with it.</p>
<img src="https://cdn.hashnode.com/uploads/covers/6ae732d54803f0be26b37004/fa5dc0b4-a59b-415c-803b-bf46c2ab77ad.png" alt="fig05" style="display:block;margin:0 auto" />

<blockquote>
<p>Every machine can hold the same amount of data and still get wildly different traffic.</p>
</blockquote>
<h2>Why the quiet channel waits</h2>
<p>Inside one of those machines, two jobs compete. Users are reading. And the database is constantly tidying its own files, merging them and throwing away old versions. That's <strong>compaction</strong>, and it uses the same CPU and disk the users need.</p>
<img src="https://cdn.hashnode.com/uploads/covers/6ae732d54803f0be26b37004/b93362a1-122b-45d0-88a5-7e234c02ed3d.png" alt="fig06" style="display:block;margin:0 auto" />

<p>Discord read and wrote with <strong>quorum consistency</strong>: with three copies, a request waits for two of them to answer. If one copy is slow, that's usually fine, because the other two answer. But traffic for a hot partition concentrates on the replicas that hold it. When enough of those are overloaded, a quorum request can't avoid them, and it waits.</p>
<p>Those same machines also hold lots of quiet channels. So queries that touch them see higher latency too, including a quiet channel that just happened to share them. Discord saw latency cascade from node to node as machines struggled to compact, and the team had to step in by hand to keep nodes healthy.</p>
<img src="https://cdn.hashnode.com/uploads/covers/6ae732d54803f0be26b37004/8f3baf3b-5b74-4120-ab39-bb5bf46fc2fa.png" alt="fig07" style="display:block;margin:0 auto" />

<h2>Stop asking the same question twice</h2>
<p>Before buying bigger databases, Discord asked a better question: does every request need its own database read?</p>
<p>Picture a big announcement. Hundreds of people open the channel in the same second, and they all ask for the same messages. A naive system sends hundreds of identical reads.</p>
<p>Discord built a layer of small services in Rust that sit between the app and the database. When the first request for some data arrives, the service starts a read. When an identical request arrives while that read is still running, it doesn't start another. It joins the first one and waits for the same result. That's <strong>request coalescing</strong>.</p>
<img src="https://cdn.hashnode.com/uploads/covers/6ae732d54803f0be26b37004/03a7a792-ec7c-422b-91e3-06cda5daa4e7.png" alt="fig08" style="display:block;margin:0 auto" />

<p>It isn't caching. A cache reuses an answer that already finished. Coalescing shares a read that's still in progress.</p>
<img src="https://cdn.hashnode.com/uploads/covers/6ae732d54803f0be26b37004/41efcffb-8011-42ec-9ac1-c397a87da5c0.png" alt="fig09" style="display:block;margin:0 auto" />

<p>The clever part is deciding what counts as the same request. Same channel isn't enough: the newest page and last week's history are different questions. And a shared read shares its failures too. If it errors, everyone waiting gets the error. So coalescing only works on truly identical work.</p>
<img src="https://cdn.hashnode.com/uploads/covers/6ae732d54803f0be26b37004/87415e69-38d2-48fe-9241-7912a87d0cf9.png" alt="fig10" style="display:block;margin:0 auto" />

<h2>Making the requests meet</h2>
<p>There's a catch. Discord runs many copies of these services. If identical requests land on different copies, each copy starts its own read, because it can't see what the others are doing. You've built a meeting point, then sent everyone to different rooms.</p>
<p>So Discord routed by channel, using a <strong>consistent hash on the channel ID</strong>. Every request for the same channel goes to the same service instance, and identical requests actually meet.</p>
<img src="https://cdn.hashnode.com/uploads/covers/6ae732d54803f0be26b37004/f6e8f4f1-b64b-4e3e-9951-d04a701a1a39.png" alt="fig11" style="display:block;margin:0 auto" />

<p>Two separate ideas, working together: routing decides where requests go; coalescing decides which of them can share one read. And this routing is separate from the database's own placement. One picks a service instance. The other picks where the data lives.</p>
<h2>A new engine</h2>
<p>Fewer duplicate reads helped, but Discord also wanted a database that behaved more predictably under load. They picked <strong>ScyllaDB</strong>. It speaks the same query language as Cassandra, so the data model could stay the same. Underneath it works very differently:</p>
<ul>
<li>It's written in C++ instead of Java, so there's no garbage collector pausing the database to clean up memory.</li>
<li>It splits each machine by CPU core. Every core owns its own slice of the data and memory and does its own work, and cores message each other only when they have to.</li>
</ul>
<img src="https://cdn.hashnode.com/uploads/covers/6ae732d54803f0be26b37004/8f17696c-2070-4410-b2f6-5431f9d88ff4.png" alt="fig12" style="display:block;margin:0 auto" />

<p>A new engine doesn't make hot partitions impossible, and Discord says they can still happen on ScyllaDB. The fix was the combination: less duplicate work, and a more predictable engine underneath.</p>
<p>Before switching, Discord tested their real queries. One failed: reverse queries, which scan messages in the opposite direction to how they're stored, were too slow. The ScyllaDB team improved it, and that removed the last blocker.</p>
<img src="https://cdn.hashnode.com/uploads/covers/6ae732d54803f0be26b37004/3df0eb95-2794-435a-90fc-744934076db2.png" alt="fig13" style="display:block;margin:0 auto" />

<blockquote>
<p>Benchmark the awkward query, not just the easy one.</p>
</blockquote>
<h2>Moving the floor while people stand on it</h2>
<p>Replacing the engine while trillions of messages were still changing was a different challenge. Discord started by writing every new message to both databases, then had to copy everything that already existed.</p>
<p>That sounds simple until things change mid-copy. Someone edits a message you're copying right now. If the old version lands after the new one, the old one must not win. And writing to two databases isn't one step: if one write succeeds and the other fails, "we wrote to both" no longer means "both agree".</p>
<img src="https://cdn.hashnode.com/uploads/covers/6ae732d54803f0be26b37004/62223afe-cdf2-40e8-b833-7e3e4bc41cfc.png" alt="fig14" style="display:block;margin:0 auto" />

<p>Discord's first plan used Spark, and the estimate was three months. So they wrote their own migrator in Rust. It reads the old database in chunks, saves its progress (in SQLite) so a crash doesn't restart everything, and pushes the data across. The new estimate was nine days, and at its peak it moved up to 3.2 million messages per second. Both "three months" and "nine days" were estimates, not measured run times.</p>
<img src="https://cdn.hashnode.com/uploads/covers/6ae732d54803f0be26b37004/00b7cd92-f549-4753-8f3f-f746687428e0.png" alt="fig15" style="display:block;margin:0 auto" />

<h2>Copied is not the same as correct</h2>
<p>A new database can hold every row and still give the wrong answer: messages in the wrong order, an old edit, a deleted message that came back, or an empty page.</p>
<p>So Discord checked behaviour, not just row counts. They sent a small share of real reads to both databases and compared the answers. That's strong evidence, but it isn't proof. Treat <strong>copy progress</strong>, <strong>validation</strong> and <strong>readiness</strong> as three separate signals. A green progress bar answers only one of them.</p>
<img src="https://cdn.hashnode.com/uploads/covers/6ae732d54803f0be26b37004/7dd6ecbb-931e-42ab-a179-9c02e7889381.png" alt="fig16" style="display:block;margin:0 auto" />

<h2>The deleted message problem</h2>
<p>When you delete a message in a database with three copies, you can't just erase it. If one copy was offline at that moment, it comes back later still holding the message, and the deleted message rises from the dead.</p>
<p>So the database writes a marker instead: a <strong>tombstone</strong>, which says "this was deleted". Reads have to step over every tombstone, and tombstones hang around until compaction is allowed to clean them up.</p>
<img src="https://cdn.hashnode.com/uploads/covers/6ae732d54803f0be26b37004/90a35537-3212-4482-885b-20ace983c392.png" alt="fig17" style="display:block;margin:0 auto" />

<p>Discord already knew this could hurt. In 2017, one channel with millions of deleted messages took twenty seconds to load. Deleted, from the user's point of view, does not mean gone from the database.</p>
<h2>Stuck at 99.9999 percent</h2>
<p>Several days into the migration, the progress bar hit 99.9999% and stopped. The last few ranges kept timing out. Inside them were gigantic runs of tombstones the old database had never cleaned up.</p>
<img src="https://cdn.hashnode.com/uploads/covers/6ae732d54803f0be26b37004/768a6182-50ef-4c0b-be8e-3c6d3ac0be7a.png" alt="fig18" style="display:block;margin:0 auto" />

<p>So the team compacted that range. Seconds later, the migration was complete.</p>
<blockquote>
<p>Never guess the time left from the percentage left.</p>
</blockquote>
<h2>What changed</h2>
<p>From Discord's March 2023 post:</p>
<ul>
<li><strong>Database nodes:</strong> 177 Cassandra nodes became 72 ScyllaDB nodes.</li>
<li><strong>Reading message history, p99:</strong> 40–125 ms before, 15 ms after.</li>
<li><strong>Inserting a message, p99:</strong> 5–70 ms before, a steady 5 ms after.</li>
</ul>
<p>p99 is the time that 99% of requests finish within. These aren't a controlled benchmark: the hardware changed too (about 4 TB per old node against 9 TB per new one), and so did the application path.</p>
<img src="https://cdn.hashnode.com/uploads/covers/6ae732d54803f0be26b37004/eb3897fc-d0bd-4641-a04a-323a9bfada08.png" alt="fig19" style="display:block;margin:0 auto" />

<p>Then came the 2022 World Cup final. Every time Messi or Mbappé scored, a flood of people reacted in the same few seconds, and you can see the whole match in Discord's message graph. It was the kind of crowd that used to overload the database. This time, by Discord's account, it didn't even break a sweat.</p>
<img src="https://cdn.hashnode.com/uploads/covers/6ae732d54803f0be26b37004/f2c0169a-54cd-4c03-abbf-31c581e5d5eb.png" alt="fig20" style="display:block;margin:0 auto" />

<h2>Three lessons</h2>
<img src="https://cdn.hashnode.com/uploads/covers/6ae732d54803f0be26b37004/be87a7f3-7357-4e13-a890-5209b6336c3f.png" alt="fig21" style="display:block;margin:0 auto" />

<ul>
<li><strong>A good partition key doesn't stop a hotspot.</strong> It bounds how much data sits in one place, not how many people ask for it.</li>
<li><strong>A faster engine doesn't stop duplicate work.</strong> Route identical requests together, then let them share one read.</li>
<li><strong>A finished copy doesn't prove the data is correct.</strong> Compare real answers, and watch out for what deletes leave behind.</li>
</ul>
<p>Each layer solves a different problem.</p>
<p>Watch the full animated version: <a href="https://youtu.be/N5KyGD7fRTQ">Why One Discord Channel Could Slow Down Everyone (YouTube)</a></p>
<p>Have you ever had a progress bar stall at 99%? What was hiding in the last 1%? 👇</p>
<h2>Sources</h2>
<ul>
<li>Discord, <a href="https://discord.com/blog/how-discord-stores-trillions-of-messages">How Discord Stores Trillions of Messages</a> (March 2023)</li>
<li>Discord, <a href="https://discord.com/blog/how-discord-stores-billions-of-messages">How Discord Stores Billions of Messages</a> (January 2017)</li>
<li>Apache Cassandra docs: <a href="https://cassandra.apache.org/doc/latest/cassandra/architecture/dynamo.html">Dynamo architecture</a> and <a href="https://cassandra.apache.org/doc/latest/cassandra/managing/operating/compaction/tombstones.html">Tombstones</a></li>
<li><a href="https://docs.scylladb.com/">ScyllaDB documentation</a> (shard-per-core design)</li>
</ul>
<hr />
<h3>Follow The Byte Mechanics</h3>
<p>How software works, under the hood ⚙️</p>
<ul>
<li>🌐 <a href="https://www.thebytemechanics.com/">Website</a></li>
<li>▶️ <a href="https://www.youtube.com/@thebytemechanics">YouTube</a></li>
<li>📸 <a href="https://www.instagram.com/thebytemechanics/">Instagram</a></li>
<li>🧵 <a href="https://www.threads.com/@thebytemechanics">Threads</a></li>
<li>📘 <a href="https://www.facebook.com/thebytemechanics">Facebook</a></li>
<li>𝕏 <a href="https://x.com/thebytemechanic">X</a></li>
<li>✍️ <a href="https://medium.com/@thebytemechanics">Medium</a></li>
<li>💻 <a href="https://dev.to/thebytemechanics">DEV Community</a></li>
<li>📝 <a href="https://thebytemechanics.hashnode.dev/">Hashnode</a></li>
</ul>
]]></content:encoded></item><item><title><![CDATA[What Happens When 1 Million People Click Buy at Once?]]></title><description><![CDATA[Load balancers, through one tap in a big sale: round robin, health checks, the server that dies mid-sale, and the cart that vanishes]]></description><link>https://thebytemechanics.hashnode.dev/what-happens-when-1-million-people-click-buy-at-once</link><guid isPermaLink="true">https://thebytemechanics.hashnode.dev/what-happens-when-1-million-people-click-buy-at-once</guid><dc:creator><![CDATA[The Byte Mechanics]]></dc:creator><pubDate>Fri, 13 Nov 2026 15:12:42 GMT</pubDate><enclosure url="https://cdn.hashnode.com/uploads/covers/6ae732d54803f0be26b37004/50973236-f810-4ddc-8ff9-16e1cc3c7ba0.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>It's 8 PM. The biggest sale of the year just opened, and you and a million other people tap Buy in the same minute.</p>
<p>Behind that one button there are fifty servers. Something has to decide which one gets your tap. And one of those servers is about to die, with an order still on it.</p>
<p>This article follows one tap through five steps: <strong>Choose → Balance → Detect → Survive → Scale</strong>. If you'd rather watch, here's the 9-minute animated version:</p>
<p><a href="https://youtu.be/YmdPwxz6TCU">Watch here</a></p>
<p><em>We'll follow Ananya as she shops the sale on a store at</em> <a href="http://shop.example.com"><em>shop.example.com</em></a><em>. The numbers are an illustrative model, not a benchmark.</em></p>
<h2>Why one server isn't enough</h2>
<p>One tap can trigger a few requests behind the scenes, but let's keep it simple: one request per tap. A million taps in a minute is about <strong>16,667 requests every second</strong>.</p>
<p>In our example, one server handles about 500 a second. Ask it for 17,000 and it doesn't slow down politely. It falls over.</p>
<img src="https://cdn.hashnode.com/uploads/covers/6ae732d54803f0be26b37004/c83a27f9-1dec-400c-9454-327213d2450d.png" alt="fig02" style="display:block;margin:0 auto" />

<p>You could buy a bigger machine. That's <strong>scaling up</strong>. It hits a ceiling fast, and when that one machine dies, everything dies.</p>
<p>So you run many ordinary machines side by side. That's <strong>scaling out</strong>. 16,667 ÷ 500 ≈ 34 servers. You run 50, for spare room.</p>
<p>Which creates a new problem: with fifty servers, how does your phone know which one to talk to?</p>
<h2>Step 1, Choose: the load balancer</h2>
<p>It doesn't. Your phone knows one address, and that address belongs to a <strong>load balancer</strong>.</p>
<img src="https://cdn.hashnode.com/uploads/covers/6ae732d54803f0be26b37004/ebf080cb-64b4-41f8-ae7c-e809910cde69.png" alt="fig03" style="display:block;margin:0 auto" />

<p>Every request arrives at the load balancer first. It picks a healthy server from its list, forwards the request, and the reply comes back to the phone. To Ananya it looks like one service. Behind it, fifty servers share the work, about 333 requests each, every second.</p>
<p>So how does it pick?</p>
<h2>Step 2, Balance: round robin</h2>
<p>The simplest rule is <strong>round robin</strong>: server 1, then 2, then 3, all the way to 50, then back to 1. When requests take roughly the same time, that works well.</p>
<img src="https://cdn.hashnode.com/uploads/covers/6ae732d54803f0be26b37004/939931f4-6462-469d-b08a-fb739354bb4c.png" alt="fig04" style="display:block;margin:0 auto" />

<p>But one shopper is buying a phone case, and another has forty items, three coupons and gift wrap. That request takes thirty times longer. Round robin hands out <em>turns</em>, not <em>work</em>, so server 9 is stuck on the heavy order while new requests keep landing in its queue, and servers 8 and 10 sit nearly idle.</p>
<blockquote>
<p>Equal turns aren't equal work.</p>
</blockquote>
<h2>Least connections and weights</h2>
<p><strong>Least connections</strong> asks a better question: which server has the fewest active connections right now? The next request goes there, and server 9 gets a break until it catches up.</p>
<img src="https://cdn.hashnode.com/uploads/covers/6ae732d54803f0be26b37004/c9aa8054-fc03-4a73-9b46-9b3db785596a.png" alt="fig05" style="display:block;margin:0 auto" />

<p>It doesn't measure CPU. Open connections are a rough signal of how busy a server is, and a cheap one to track.</p>
<p>If some machines are bigger, give them <strong>weights</strong>. A server with weight 2 can receive twice the share of a server with weight 1.</p>
<p>And some apps add one more rule: send the same user back to the same server every time. That's a <strong>sticky session</strong>. Remember it. It's going to hurt.</p>
<h2>Step 3, Detect: health checks</h2>
<p>A load balancer doesn't automatically know whether a server is alive, so it checks. Some checks just try to open a connection; others call a health endpoint in the app that says whether the service is ready.</p>
<img src="https://cdn.hashnode.com/uploads/covers/6ae732d54803f0be26b37004/cdc17783-2869-44ec-8309-8375ac912183.png" alt="fig06" style="display:block;margin:0 auto" />

<p>In our example it checks every 2 seconds, and three misses in a row mark a server unhealthy. Real systems tune these numbers, but here that's <strong>about 6 seconds to notice</strong>.</p>
<p>At 8:00:04 PM, server 7 stops answering.</p>
<img src="https://cdn.hashnode.com/uploads/covers/6ae732d54803f0be26b37004/ece50750-2760-436f-a456-0f72aecd2fda.png" alt="fig07" style="display:block;margin:0 auto" />

<h2>Step 4, Survive: the six seconds</h2>
<p>For those six seconds, the balancer still thinks server 7 is fine. At about 333 requests a second, roughly <strong>2,000 requests</strong> may be sent its way before it's removed.</p>
<p>Ananya's payment is one of them. She taps Pay. The spinner turns, and turns. Then: <em>something went wrong</em>.</p>
<img src="https://cdn.hashnode.com/uploads/covers/6ae732d54803f0be26b37004/18e008b8-ad62-4f64-ad71-5416f558e109.png" alt="fig08" style="display:block;margin:0 auto" />

<p>Did she pay or not? Here's what actually happened. Server 7 saved her order to the database, then died before it could reply. The order was saved; the reply was lost.</p>
<p>So her app tries again, and the balancer sends the retry to server 23, which is alive. If nothing stops it, that's a second order. <strong>Charged twice.</strong></p>
<img src="https://cdn.hashnode.com/uploads/covers/6ae732d54803f0be26b37004/e559ab8a-ece9-4c45-bd84-930e491e87dd.png" alt="fig09" style="display:block;margin:0 auto" />

<p>Payment systems prevent this with an <strong>idempotency key</strong>: one unique ID for this one payment, sent with every retry. The key and its result are stored reliably in a place every server can reach. Server 23 looks up the key, sees the payment is already done, and returns the original result instead of charging her again.</p>
<img src="https://cdn.hashnode.com/uploads/covers/6ae732d54803f0be26b37004/2ae7d96b-095f-4194-845b-ff86fef90a55.png" alt="fig10" style="display:block;margin:0 auto" />

<pre><code class="language-text">POST /payments
Idempotency-Key: A7F3
→ first time: process the payment, store (A7F3 → result)
→ any retry:  find A7F3, return the stored result
</code></pre>
<h2>Pause and guess</h2>
<p>Remember the sticky session? Ananya's cart was kept in server 7's memory: three headphones, a charger and a coupon.</p>
<p>Server 7 is gone. She refreshes the page. What does she see?</p>
<img src="https://cdn.hashnode.com/uploads/covers/6ae732d54803f0be26b37004/db7fe1ee-a9c2-45bf-b87c-3d552676b51e.png" alt="fig11" style="display:block;margin:0 auto" />

<p>Her cart as it was? An empty cart? An error? Make your guess before you read on.</p>
<h2>The cart that vanished</h2>
<p>An empty cart. The balancer sent her to a healthy server, but that server had never heard of her. Her cart died with server 7.</p>
<p>That's the trap. When something a user needs lives inside one server's memory, that server can't be replaced. That something is called <strong>state</strong>.</p>
<p>The fix is to take the state out of the servers. Keep carts and sessions in a <strong>shared session store</strong>, such as a fast cache like Redis, set up so the data survives the failures your app needs to handle. Every server can read it, so the servers become <strong>stateless</strong>: any healthy server can handle any request.</p>
<img src="https://cdn.hashnode.com/uploads/covers/6ae732d54803f0be26b37004/42c9bdcc-9fd7-4273-aa8d-6880b7a685a5.png" alt="fig12" style="display:block;margin:0 auto" />

<p>Ananya refreshes again, lands on server 41, and her cart is right there.</p>
<h2>Layer 4 vs Layer 7</h2>
<p>Some load balancers work like a postman who only reads the envelope. A <strong>Layer 4</strong> load balancer looks at the IP address, the port and the transport protocol, and forwards the packet without opening it. Very fast, very simple.</p>
<p>A <strong>Layer 7</strong> load balancer opens the envelope. It reads the request itself: the path, the headers, even which website name was asked for. So <code>/checkout</code> goes to the checkout servers and <code>/images</code> goes to the image servers.</p>
<img src="https://cdn.hashnode.com/uploads/covers/6ae732d54803f0be26b37004/b21bd6b3-ef7b-406b-90cb-0936e611329d.png" alt="fig13" style="display:block;margin:0 auto" />

<table>
<thead>
<tr>
<th></th>
<th>Layer 4</th>
<th>Layer 7</th>
</tr>
</thead>
<tbody><tr>
<td>Reads</td>
<td>IP, port, protocol</td>
<td>Path, headers, host</td>
</tr>
<tr>
<td>Speed</td>
<td>Very fast</td>
<td>More work per request</td>
</tr>
<tr>
<td>Can route by</td>
<td>Connection</td>
<td>What the request asks for</td>
</tr>
</tbody></table>
<p>Many big systems use both: <strong>a fast Layer 4 balancer in front, and Layer 7 routing behind it</strong>.</p>
<h2>Step 5, Scale: who balances the balancer?</h2>
<p>If every request goes through one load balancer, that balancer is the new single point of failure. So you run more than one.</p>
<img src="https://cdn.hashnode.com/uploads/covers/6ae732d54803f0be26b37004/8257f772-e9a2-4993-b9f9-30242e317a0e.png" alt="fig14" style="display:block;margin:0 auto" />

<ul>
<li><strong>Active-passive:</strong> one handles the traffic while a standby waits, ready to take over the address if the first one fails.</li>
<li><strong>Active-active:</strong> they share the traffic, and if one fails the others carry on, provided they have spare capacity.</li>
</ul>
<p>Either way, failover is automatic, usually within seconds, depending on how it's set up.</p>
<p>For users all over the world there's one more layer. <strong>DNS-based global traffic management</strong> directs users towards a suitable region based on location, availability and routing policies. Ananya might be sent to Mumbai while someone in Jakarta goes to Singapore, and each region has its own load balancers and servers. That's a different job from a CDN, which serves cached files like images from the edge.</p>
<img src="https://cdn.hashnode.com/uploads/covers/6ae732d54803f0be26b37004/3431321b-fcf8-4ef7-8ec5-422bf2a05fec.png" alt="fig15" style="display:block;margin:0 auto" />

<h2>Every piece on one screen</h2>
<img src="https://cdn.hashnode.com/uploads/covers/6ae732d54803f0be26b37004/520a2361-6805-45f7-ae60-a4cb1e07ff39.png" alt="fig16" style="display:block;margin:0 auto" />

<ul>
<li><strong>DNS</strong> sends you to a suitable region.</li>
<li>A <strong>pair of load balancers</strong> share one address, so either can fail.</li>
<li>They pick a healthy server <strong>by turns, by fewest connections, or by weight</strong>.</li>
<li><strong>Health checks</strong> take dead servers out within seconds.</li>
<li>An <strong>idempotency key</strong> makes a retry return the original result instead of a second charge.</li>
<li>A <strong>shared session store</strong> keeps the cart, so any server can pick up where another left off.</li>
</ul>
<p>One address on the outside. Fifty servers on the inside. And even when one fails, the service keeps going.</p>
<p>Watch the full animated version: [Click here] (<a href="https://youtu.be/YmdPwxz6TCU">https://youtu.be/YmdPwxz6TCU</a>)</p>
<p>What was your guess for Ananya's cart? And has a retry ever charged you twice? 👇</p>
<p><em>Before her tap even reaches the load balancer, it crosses DNS, routers, TCP and TLS. That part of the trip is in the Networking video:</em> <a href="https://youtu.be/sFySVuvONWA"><em>https://youtu.be/sFySVuvONWA</em></a></p>
<hr />
<h3>Follow The Byte Mechanics</h3>
<p>How software works, under the hood ⚙️</p>
<ul>
<li>🌐 <a href="https://www.thebytemechanics.com/">Website</a></li>
<li>▶️ <a href="https://www.youtube.com/@thebytemechanics">YouTube</a></li>
<li>📸 <a href="https://www.instagram.com/thebytemechanics/">Instagram</a></li>
<li>🧵 <a href="https://www.threads.com/@thebytemechanics">Threads</a></li>
<li>📘 <a href="https://www.facebook.com/thebytemechanics">Facebook</a></li>
<li>𝕏 <a href="https://x.com/thebytemechanic">X</a></li>
<li>✍️ <a href="https://medium.com/@thebytemechanics">Medium</a></li>
<li>💻 <a href="https://dev.to/thebytemechanics">DEV Community</a></li>
<li>📝 <a href="https://thebytemechanics.hashnode.dev/">Hashnode</a></li>
</ul>
]]></content:encoded></item><item><title><![CDATA[What Happens When You Hit Search? Every Stop in 190 Milliseconds]]></title><description><![CDATA[DNS, routers, TCP, TLS, the edge and the search itself, and why the same app can be four times slower]]></description><link>https://thebytemechanics.hashnode.dev/what-happens-when-you-hit-search-every-stop-in-190-milliseconds</link><guid isPermaLink="true">https://thebytemechanics.hashnode.dev/what-happens-when-you-hit-search-every-stop-in-190-milliseconds</guid><dc:creator><![CDATA[The Byte Mechanics]]></dc:creator><pubDate>Fri, 13 Nov 2026 15:07:44 GMT</pubDate><enclosure url="https://cdn.hashnode.com/uploads/covers/6ae732d54803f0be26b37004/3b3c712d-8d56-41ed-8ab6-5fda70c6243e.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>Hit search in an e-commerce app. Your request leaves your phone, crosses about a thousand kilometres and comes back with results, all in about 190 milliseconds. A blink takes about 300.</p>
<p>In between, it makes seven stops: <strong>DNS → Routers → Handshake → Lock → Edge → Search → Back</strong>.</p>
<p>And here's the twist: the exact same app, running the exact same code, can be four times slower just because its servers are in the wrong place. No engineer in the world can fix that part. By the end of this article you'll know why. If you'd rather watch, here's the 8-minute animated version:</p>
<p><em>We'll follow Ananya, in Bengaluru, as she searches for "wireless headphones" on a store at</em> <a href="http://shop.example.com"><em>shop.example.com</em></a><em>. The numbers are an illustrative model, not a live benchmark: a fresh connection, a full TLS 1.3 handshake and a simplified request path.</em></p>
<p><a href="https://youtu.be/sFySVuvONWA">Watch</a></p>
<hr />
<h2>Stop 1: DNS (30 ms)</h2>
<p>Your phone knows the name <code>shop.example.com</code>, but computers don't route by names. They use IP addresses, like phone numbers for machines.</p>
<p>So your phone asks a <strong>DNS resolver</strong>: where is this? If the answer isn't cached, the resolver asks a root server, then the <code>.com</code> servers, then the store's own authoritative name server, which finally returns the IP.</p>
<img src="https://cdn.hashnode.com/uploads/covers/6ae732d54803f0be26b37004/339b1d50-c543-43ad-a6c1-4c885970b193.png" alt="fig02" style="display:block;margin:0 auto" />

<p>The clever part is that the answer can depend on where you are. Ananya is in Bengaluru, so she gets the address of a server near her, in Mumbai, not one on the other side of the world.</p>
<img src="https://cdn.hashnode.com/uploads/covers/6ae732d54803f0be26b37004/ed162fa6-a25c-42d2-81d0-0f2936987a90.png" alt="fig03" style="display:block;margin:0 auto" />

<p>About 30 milliseconds, and the answer is cached for next time.</p>
<h2>Stop 2: The routers</h2>
<p>Your search travels as <strong>IP packets</strong>, each stamped with where it's from and where it's going. From your Wi-Fi or mobile network to your internet provider, then router after router.</p>
<p>Each router looks only at the destination address and forwards the packet one hop closer. No single router knows the whole journey. To Mumbai, that's about 10 hops and 1,000 km.</p>
<img src="https://cdn.hashnode.com/uploads/covers/6ae732d54803f0be26b37004/313eb023-6bd3-4ba7-bb0b-f45f729698b0.png" alt="fig04" style="display:block;margin:0 auto" />

<h2>Stop 3: The TCP handshake (25 ms)</h2>
<p>Before any data moves, your phone and the server shake hands. Your phone sends SYN: "Can we talk?" The server replies SYN-ACK: "Yes, I hear you." Your phone sends ACK: "Great." That's one full round trip, about 25 ms, before your search is even sent.</p>
<img src="https://cdn.hashnode.com/uploads/covers/6ae732d54803f0be26b37004/b8d82327-7b01-4547-b349-f416bc93ddcf.png" alt="fig05" style="display:block;margin:0 auto" />

<p>Why bother? Because <strong>TCP</strong> guarantees every byte arrives complete and in order. It numbers the data, waits for acknowledgements and resends anything that's lost. That's what you want for a search or a payment.</p>
<p>A video call or a game is different. It often uses <strong>UDP</strong>, which skips the handshake and never resends, because fresh data matters more than an old packet. Hold on to that: UDP comes back later.</p>
<img src="https://cdn.hashnode.com/uploads/covers/6ae732d54803f0be26b37004/7a526e4f-e0ad-42ee-b22e-9f96a0e7539f.png" alt="fig06" style="display:block;margin:0 auto" />

<h2>Stop 4: The lock (25 ms)</h2>
<p>Without encryption, every router on the way could read what you searched, and later your card number. So HTTPS adds <strong>TLS</strong>.</p>
<p>In a full TLS 1.3 handshake, your phone and the server agree on secret keys, and the server shows a <strong>certificate</strong> proving it really is the store. One more round trip, 25 ms. After that, every router only sees scrambled data. On a return visit, session resumption can skip most of this.</p>
<img src="https://cdn.hashnode.com/uploads/covers/6ae732d54803f0be26b37004/d1281d79-071c-48f5-84b0-1475eb4ed8a0.png" alt="fig07" style="display:block;margin:0 auto" />

<p>Notice that every stop so far has been close by. Remember that.</p>
<h2>Stop 5: The edge and the load balancer (30 ms)</h2>
<p>The search reaches an <strong>edge server</strong> in Mumbai, one of many edge locations around the world. The edge blocks bad traffic with a web application firewall and already holds the images and scripts, so they never travel far.</p>
<p>Live prices and stock can't be cached there, so the search is forwarded to the data centre. A <strong>load balancer</strong> picks one of many identical servers, using simple rules like round robin or least connections, so no single machine gets crushed.</p>
<img src="https://cdn.hashnode.com/uploads/covers/6ae732d54803f0be26b37004/7a56e8e7-1393-4274-bd94-60c96fa76c3b.png" alt="fig08" style="display:block;margin:0 auto" />

<p>About 30 ms. And your search still hasn't been searched.</p>
<h2>Stop 6: The real work (80 ms)</h2>
<p>An <strong>API gateway</strong> checks the request: is this a real user, are they sending too many requests, and which service should handle it? It goes to the search service.</p>
<p>"Headphones" is a popular search, so the answer may already be in a <strong>cache</strong> in memory. If not, the service asks a <strong>search index</strong>, which jumps straight to the matches instead of reading every product, just like a database index (<a href="https://medium.com/@thebytemechanics/from-9-seconds-to-2-milliseconds-database-indexes-explained-b62e250ac9e4?sharedUserId=thebytemechanics">Database Indexes</a>).</p>
<p>At the same time, a price service and an inventory service fetch live prices and stock. They run in parallel, and the result is assembled when they're all done. 80 ms.</p>
<img src="https://cdn.hashnode.com/uploads/covers/6ae732d54803f0be26b37004/9424ab0d-0291-4c12-90e8-b4b1d4bb9657.png" alt="fig09" style="display:block;margin:0 auto" />

<h2>Stop 7: The way back</h2>
<p>The results go back over the connection that's already open, so there's no new handshake and no new lock. Back through the load balancer, the edge and the routers to the phone, where the images were already waiting.</p>
<p>Total: <strong>190 ms</strong>. Ananya didn't even blink.</p>
<img src="https://cdn.hashnode.com/uploads/covers/6ae732d54803f0be26b37004/b935bbeb-023d-4348-9983-064a4553f018.png" alt="fig10" style="display:block;margin:0 auto" />

<table>
<thead>
<tr>
<th>Stop</th>
<th>Time</th>
</tr>
</thead>
<tbody><tr>
<td>DNS</td>
<td>30 ms</td>
</tr>
<tr>
<td>TCP handshake</td>
<td>25 ms</td>
</tr>
<tr>
<td>TLS (the lock)</td>
<td>25 ms</td>
</tr>
<tr>
<td>Edge + load balancer</td>
<td>30 ms</td>
</tr>
<tr>
<td>The real work</td>
<td>80 ms</td>
</tr>
<tr>
<td><strong>Total</strong></td>
<td><strong>about 190 ms</strong></td>
</tr>
</tbody></table>
<h2>Now move the servers to Virginia</h2>
<p>Same store, same servers, same code, but it all runs from one data centre in Virginia, in the United States. Same search, from Bengaluru. Before you read on, guess: about 200 ms, 400 ms, or close to a full second?</p>
<img src="https://cdn.hashnode.com/uploads/covers/6ae732d54803f0be26b37004/2a3af337-5d74-446b-a46d-278b75f16557.png" alt="fig11" style="display:block;margin:0 auto" />

<p>Close to a full second: <strong>770 ms</strong>, four times slower with the exact same code.</p>
<p>The handshake, the lock and the request are each a round trip. To Mumbai a round trip is 25 ms. To Virginia and back it's 220 ms. Three of those, plus 30 ms of DNS and the same 80 ms of work, makes 770.</p>
<img src="https://cdn.hashnode.com/uploads/covers/6ae732d54803f0be26b37004/71cb8365-b4a0-425f-b330-e63676253f67.png" alt="fig12" style="display:block;margin:0 auto" />

<h2>The stop no one can fix</h2>
<p>The problem is distance. Light in fibre travels about 200,000 km per second. Bengaluru to Virginia is about 14,000 km, so one way takes 70 ms and a round trip at least 140 ms. That's the floor for every round trip, even on a perfect network. Real networks add routing and congestion on top.</p>
<img src="https://cdn.hashnode.com/uploads/covers/6ae732d54803f0be26b37004/69b0cb65-66d1-4b80-9545-50b80527eb8f.png" alt="fig13" style="display:block;margin:0 auto" />

<blockquote>
<p>You can buy faster servers. You can tune your database. You can rewrite your code. You can't make light faster.</p>
</blockquote>
<h2>How big stores win: fewer trips, shorter trips</h2>
<p><strong>Fewer round trips.</strong> Keep the connection open so the next search skips the handshake and the lock. HTTP/2 lets many requests share that one connection. HTTP/3 is where UDP comes back: it runs on QUIC, which builds reliability on top of UDP and sets up the connection and encryption together in a single round trip.</p>
<img src="https://cdn.hashnode.com/uploads/covers/6ae732d54803f0be26b37004/07701372-7fdd-44f7-b596-3d3894221074.png" alt="fig14" style="display:block;margin:0 auto" />

<p><strong>Shorter round trips.</strong> Put edges and data centres close to the people using them. That's why Ananya's search went to Mumbai, not Virginia.</p>
<p>But running in many regions isn't free. You have to keep data in sync, handle failover and pay for all of it. So the goal isn't servers everywhere. It's the right servers in the right places.</p>
<h2>The whole trip on one screen</h2>
<img src="https://cdn.hashnode.com/uploads/covers/6ae732d54803f0be26b37004/0d022569-0444-49e1-86a6-be94f1e310ef.png" alt="fig15" style="display:block;margin:0 auto" />

<ul>
<li><strong>DNS</strong> turns a name into a nearby address.</li>
<li><strong>Routers</strong> forward packets hop by hop.</li>
<li><strong>TCP</strong> sets up a reliable connection, and <strong>TLS</strong> locks it.</li>
<li>The <strong>edge</strong> and the <strong>load balancer</strong> get you to a healthy server close by.</li>
<li>The <strong>gateway, cache and search index</strong> find the answer.</li>
<li>Every round trip pays for distance, so <strong>fewer trips and shorter trips</strong> win.</li>
</ul>
<p>Watch the full animated version: <a href="https://youtu.be/sFySVuvONWA">Link</a></p>
<p>What was your guess for Virginia? And which stop surprised you most?</p>
<hr />
<h3>Follow The Byte Mechanics</h3>
<p>How software works, under the hood ⚙️</p>
<ul>
<li>🌐 <a href="https://www.thebytemechanics.com/">Website</a></li>
<li>▶️ <a href="https://www.youtube.com/@thebytemechanics">YouTube</a></li>
<li>📸 <a href="https://www.instagram.com/thebytemechanics/">Instagram</a></li>
<li>🧵 <a href="https://www.threads.com/@thebytemechanics">Threads</a></li>
<li>📘 <a href="https://www.facebook.com/thebytemechanics">Facebook</a></li>
<li>𝕏 <a href="https://x.com/thebytemechanic">X</a></li>
<li>✍️ <a href="https://medium.com/@thebytemechanics">Medium</a></li>
<li>💻 <a href="https://dev.to/thebytemechanics">DEV Community</a></li>
<li>📝 <a href="https://thebytemechanics.hashnode.dev/">Hashnode</a></li>
</ul>
]]></content:encoded></item><item><title><![CDATA[From 9 Seconds to 2 Milliseconds: Database Indexes Explained]]></title><description><![CDATA[This query took nine seconds: One line of SQL made it two milliseconds. That's four thousand times faster, with no new hardware and no code changes. That…]]></description><link>https://thebytemechanics.hashnode.dev/from-9-seconds-to-2-milliseconds-database-indexes-explained</link><guid isPermaLink="true">https://thebytemechanics.hashnode.dev/from-9-seconds-to-2-milliseconds-database-indexes-explained</guid><dc:creator><![CDATA[The Byte Mechanics]]></dc:creator><pubDate>Fri, 13 Nov 2026 15:01:36 GMT</pubDate><content:encoded><![CDATA[<h2>How an index works, why it's a B-tree, and the one mistake that makes the database ignore it</h2>
<img src="https://cdn.hashnode.com/uploads/covers/6ae732d54803f0be26b37004/7a092bc4-b43e-4da6-ba76-d0d24ec476f2.png" alt="fig01" style="display:block;margin:0 auto" />

<p>This query took nine seconds:</p>
<pre><code class="language-text">SELECT * FROM users WHERE email = 'priya@example.com';
</code></pre>
<p>One line of SQL made it two milliseconds. That's four thousand times faster, with no new hardware and no code changes. That line creates a database index.</p>
<p>But here's the twist: you can have exactly the right index, and the database will still ignore it, because of one tiny thing in your query. By the end of this article you'll know how indexes work, and you'll spot that mistake in your own code. If you'd rather watch, here's the 7-minute animated version:</p>
<p>{% embed <a href="https://youtu.be/qCSdO5mVKrg">https://youtu.be/qCSdO5mVKrg</a> %}</p>
<hr />
<h2>1. Why it took nine seconds</h2>
<p>The <code>users</code> table has ten million rows. You asked for one email. With no index, the database has no idea where that row is, so it does the only thing it can: it reads every row and checks each one. That's called a <strong>full table scan</strong>.</p>
<p>And the data isn't read row by row from memory. It's read in pages from disk, thousands of them. Ten million checks to find one person.</p>
<img src="https://cdn.hashnode.com/uploads/covers/6ae732d54803f0be26b37004/dfdb6ff3-eebf-4871-ad63-06856b97b71a.png" alt="fig02" style="display:block;margin:0 auto" />

<p>You'd never search a phone book like that. So why does your database?</p>
<h2>2. The phone book trick</h2>
<p>Imagine finding "Priya" in a phone book by reading every name from page one. Nobody does that. You open near the middle. "M"? Priya comes after M, so you jump right. "R"? Too far, jump back. Within a few jumps, you're on the right page.</p>
<p>That only works because the names are sorted. Sorting is what lets you skip almost everything.</p>
<img src="https://cdn.hashnode.com/uploads/covers/6ae732d54803f0be26b37004/ffa95e31-4286-41c2-9e6d-2e677331da57.png" alt="fig03" style="display:block;margin:0 auto" />

<p>An index is exactly that: <strong>a sorted copy of one column, kept separately, that points back to the full rows.</strong></p>
<pre><code class="language-text">CREATE INDEX idx_email ON users(email);
</code></pre>
<p>But a database doesn't store it as a list. It uses a tree.</p>
<h2>3. The B-tree</h2>
<p>Most databases store indexes as a B-tree. At the top is a root page holding a few hundred keys that split the range: everything before "F" is down this branch, everything from "F" to "M" down that one. At the bottom, the leaf pages hold the actual emails, in order, each with a pointer to its row.</p>
<img src="https://cdn.hashnode.com/uploads/covers/6ae732d54803f0be26b37004/07398a7f-da57-4244-b1d5-09e2df80560a.png" alt="fig04" style="display:block;margin:0 auto" />

<p>Because every page holds hundreds of keys, the tree stays incredibly short. Ten million rows fit in about three or four levels. So instead of ten million checks, the database reads three or four pages. That's the jump from nine seconds to two milliseconds.</p>
<h2>4. Prove it with EXPLAIN</h2>
<p>You don't have to guess. Put the word <code>EXPLAIN</code> in front of your query, and the database shows you its plan.</p>
<pre><code class="language-text">EXPLAIN SELECT * FROM users WHERE email = 'priya@example.com';
</code></pre>
<p>Before the index, it says <strong>Seq Scan</strong>: read everything. After the index, it says <strong>Index Scan using idx_email</strong>. Same query, completely different plan.</p>
<img src="https://cdn.hashnode.com/uploads/covers/6ae732d54803f0be26b37004/19bef9d0-abe4-4735-baf0-aed1bacbd345.png" alt="fig05" style="display:block;margin:0 auto" />

<p>Make this a habit: any query that feels slow, run <code>EXPLAIN</code> first. Because sometimes you'll add an index, run <code>EXPLAIN</code>, and still see "Seq Scan". And you'll wonder why.</p>
<h2>5. Column order matters</h2>
<p>Indexes can cover more than one column. An index on <code>(last_name, first_name)</code> is like a phone book sorted by surname, and then by first name within each surname. That's perfect for finding "Smith, Priya". It also works for every "Smith".</p>
<p>But try to find everyone named "Priya" with any surname, and the sort order is useless. They're scattered through the whole book.</p>
<img src="https://cdn.hashnode.com/uploads/covers/6ae732d54803f0be26b37004/d1b5bb0e-3e41-46c9-adfa-096ecce32e22.png" alt="fig06-kumar" style="display:block;margin:0 auto" />

<p>This is the <strong>leftmost-prefix rule</strong>: a multi-column index helps only when your query uses the columns from the left. And that's just one of the ways an index gets skipped. The most common one is far sneakier.</p>
<h2>6. Where indexes quietly stop working</h2>
<p>Remember, the index is a sorted list of the exact values in that column. The database can only jump through it if your query asks about those exact values.</p>
<p>The moment your query transforms the column first, the sorted order no longer helps. The database can't jump anymore. It goes back to reading every row, and it doesn't warn you.</p>
<img src="https://cdn.hashnode.com/uploads/covers/6ae732d54803f0be26b37004/50a0f4cf-d0a3-4c58-b69d-8ac968224b37.png" alt="fig07" style="display:block;margin:0 auto" />

<h2>7. Test yourself</h2>
<p>You have an index on the <code>email</code> column. Users sometimes type their email with capital letters, so the code does this:</p>
<pre><code class="language-text">SELECT * FROM users WHERE LOWER(email) = 'priya@example.com';
</code></pre>
<img src="https://cdn.hashnode.com/uploads/covers/6ae732d54803f0be26b37004/99226022-2cd9-4810-807d-864fd5cb2d1d.png" alt="fig08" style="display:block;margin:0 auto" />

<p>Does the database use the email index for this query? Decide before you read on.</p>
<h2>8. The answer is no</h2>
<p>The index stores emails exactly as written: some with capitals, some without. <code>LOWER(email)</code> is a different value that isn't in the index at all, so the database has to compute it for every row. Full scan, nine seconds again.</p>
<img src="https://cdn.hashnode.com/uploads/covers/6ae732d54803f0be26b37004/29404e34-2cfa-4b3f-8c55-e188c719545b.png" alt="fig09" style="display:block;margin:0 auto" />

<p>The same thing happens with a leading wildcard, like <code>LIKE '%@gmail.com'</code>, because the start of the value is unknown. And it happens when you compare a text column to a number and the database has to convert types.</p>
<p>The fix has two options. Store the value already normalised, lowercase on the way in. Or create an index on the expression itself:</p>
<pre><code class="language-text">CREATE INDEX idx_email_lower ON users (LOWER(email));
</code></pre>
<img src="https://cdn.hashnode.com/uploads/covers/6ae732d54803f0be26b37004/45ef0fd0-558b-4050-a65c-48c1e5c9bc1d.png" alt="fig10" style="display:block;margin:0 auto" />

<p>Then run <code>EXPLAIN</code> again and confirm.</p>
<h2>9. Indexes aren't free</h2>
<p>So, if indexes are this powerful, why not index every column? Because every index is another sorted structure the database must keep up to date. Insert a row, and every index on that table gets updated too. Ten indexes means eleven writes for every insert. They also take disk space and memory.</p>
<img src="https://cdn.hashnode.com/uploads/covers/6ae732d54803f0be26b37004/8c7c958d-ab0f-4a2f-8081-d06fa2ecf523.png" alt="fig11" style="display:block;margin:0 auto" />

<p>And some indexes barely help. Index a column with only two values, like true or false, and the database often decides scanning is cheaper anyway. So index the columns you actually filter by, join on and sort by. Check with <code>EXPLAIN</code>. And remove indexes nobody uses.</p>
<h2>10. How real databases build on this</h2>
<p>In PostgreSQL and MySQL, the default index is a B-tree, exactly what we drew. In MySQL's InnoDB, the table itself is stored in primary key order, so looking up by primary key goes straight to the row.</p>
<p>If an index contains every column a query needs, the database can answer from the index alone, without touching the table. That's a <strong>covering index</strong>.</p>
<img src="https://cdn.hashnode.com/uploads/covers/6ae732d54803f0be26b37004/a02ccca1-3c7c-4a83-851e-6c083a964d1c.png" alt="fig12" style="display:block;margin:0 auto" />

<p>And for special data there are special indexes: hash indexes for exact matches, and in PostgreSQL, GIN indexes for things like full-text search and JSON.</p>
<h2>11. Database indexes in one screen</h2>
<img src="https://cdn.hashnode.com/uploads/covers/6ae732d54803f0be26b37004/a29b2da4-c342-4710-a401-63baf2fc53db.png" alt="fig13" style="display:block;margin:0 auto" />

<ul>
<li>Without an index, the database scans every row.</li>
<li>An index is a sorted B-tree that finds a row in a few page reads.</li>
<li>Use <code>EXPLAIN</code> to check the plan.</li>
<li>Multi-column indexes work from the left.</li>
<li>Don't wrap the indexed column in a function, a leading wildcard or a type conversion.</li>
<li>Every index costs you on writes, so add the ones you need.</li>
</ul>
<p>Now the query is fast. But between your user and your database is a network, and every request crosses it.</p>
<hr />
<p><strong>▶️ Watch the animated version:</strong> <a href="https://youtu.be/qCSdO5mVKrg">Database Indexes Explained (YouTube)</a></p>
<p><strong>Previous:</strong> <a href="https://medium.com/@thebytemechanics/from-3-seconds-to-3-milliseconds-caching-explained-b451fd740e54?sharedUserId=thebytemechanics">From 3 Seconds to 3 Milliseconds: Caching Explained</a></p>
<hr />
<h3>Follow The Byte Mechanics</h3>
<p>How software works, under the hood ⚙️</p>
<ul>
<li>🌐 <a href="https://www.thebytemechanics.com/">Website</a></li>
<li>▶️ <a href="https://www.youtube.com/@thebytemechanics">YouTube</a></li>
<li>📸 <a href="https://www.instagram.com/thebytemechanics/">Instagram</a></li>
<li>🧵 <a href="https://www.threads.com/@thebytemechanics">Threads</a></li>
<li>📘 <a href="https://www.facebook.com/thebytemechanics">Facebook</a></li>
<li>𝕏 <a href="https://x.com/thebytemechanic">X</a></li>
<li>✍️ <a href="https://medium.com/@thebytemechanics">Medium</a></li>
<li>💻 <a href="https://dev.to/thebytemechanics">DEV Community</a></li>
<li>📝 <a href="https://thebytemechanics.hashnode.dev/">Hashnode</a></li>
</ul>
]]></content:encoded></item><item><title><![CDATA[From 3 Seconds to 3 Milliseconds: Caching Explained]]></title><description><![CDATA[How a cache makes apps fast, why stale data happens, and the stampede that takes sites down]]></description><link>https://thebytemechanics.hashnode.dev/from-3-seconds-to-3-milliseconds-caching-explained</link><guid isPermaLink="true">https://thebytemechanics.hashnode.dev/from-3-seconds-to-3-milliseconds-caching-explained</guid><dc:creator><![CDATA[The Byte Mechanics]]></dc:creator><pubDate>Thu, 05 Nov 2026 14:00:55 GMT</pubDate><enclosure url="https://cdn.hashnode.com/uploads/covers/6ae732d54803f0be26b37004/046259fc-2823-404a-b2b0-61e89954435e.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>A product page took three seconds to load. Then it took three milliseconds.</p>
<p>Same server. Same database. Same code, except for one idea: caching. Almost every fast app you use runs on it.</p>
<p>But the same cache that makes a site a thousand times faster can also take the whole thing down in seconds. This article walks through how caching works, where it goes wrong, and the three-line fix the big companies use. If you’d rather watch, here’s the 7-minute animated version:</p>
<p><a href="https://youtu.be/Jez2Rg0uUgQ?si=DwWF9Ul-elk2pxvh">Full Video on Youtube</a></p>
<ol>
<li>Why the page was slow Every visit asked the database the same question: what’s the price, the stock and the reviews for this product? Thousands of people, the same question, the same answer, computed again every single time.</li>
</ol>
<p>And asking is expensive. Reading from memory takes around a hundred nanoseconds. A database query over the network takes milliseconds, and a heavy one can take seconds. That’s tens of thousands of times slower.</p>
<img src="https://cdn.hashnode.com/uploads/covers/6ae732d54803f0be26b37004/1f977b27-ddef-4317-bc14-0aa5f425957b.png" alt="" style="display:block;margin:0 auto" />

<p>So why ask again when you already know the answer? That’s the whole trick: keep the answer somewhere much closer.</p>
<ol start="2">
<li>What a cache is A cache is just a copy of data, kept closer to where it’s needed, so the next read is fast.</li>
</ol>
<p>You use caches all day without noticing. Your CPU caches memory. Your browser caches images. A CDN caches whole pages in a city near you. And your backend usually has its own cache, often Redis or Memcached, sitting in memory right next to your app.</p>
<img src="https://cdn.hashnode.com/uploads/covers/6ae732d54803f0be26b37004/a2fa6b6c-46f7-489c-9dd1-57f5b259cf59.png" alt="" style="display:block;margin:0 auto" />

<p>It’s the same idea at every layer: the closer the copy, the faster the answer.</p>
<ol start="3">
<li>Cache-aside: the pattern most apps use The most common pattern is called cache-aside, and it has three steps:</li>
</ol>
<p>Check the cache. If the answer is there, that’s a hit, and you return it in about a millisecond. On a miss, go to the database and get the real answer. Save that answer in the cache with an expiry time (a TTL), so the next person gets a hit.</p>
<img src="https://cdn.hashnode.com/uploads/covers/6ae732d54803f0be26b37004/79bd34da-44e5-475c-953e-a54850d6a1d9.png" alt="" style="display:block;margin:0 auto" />

<p><code>def get_product(id):</code><br /><code>value = cache.get(f"product:{id}")</code><br /><code>if value is None: # miss</code><br /><code>value = db.query_product(id)</code><br /><code>cache.set(f"product:{id}", value, ttl=300)</code><br /><code>return value</code><br /><br />The first visitor pays the full price, and everyone after them gets it almost for free.</p>
<ol start="4">
<li>The number that decides everything: hit ratio The hit ratio is the share of requests the cache answers. Say a hit takes 1 ms and a miss takes 100 ms. At a 50% hit ratio, the average request takes about 50 ms. At 99%, it’s about 2 ms.</li>
</ol>
<img src="https://cdn.hashnode.com/uploads/covers/6ae732d54803f0be26b37004/3c656c22-c30b-41a4-9421-f69e1f87ce2e.png" alt="" style="display:block;margin:0 auto" />

<p>And it’s not just speed. Every hit is one less query, so a 99% hit ratio means your database sees only one request in every hundred. That’s why a small cache can let a small database serve a huge amount of traffic.</p>
<ol start="5">
<li>Stale data But the cache is a copy, and copies go out of date. The price of item 42 changes from ₹999 to ₹799 in the database, but the cache still has ₹999. For the next few minutes, customers see the wrong number.</li>
</ol>
<img src="https://cdn.hashnode.com/uploads/covers/6ae732d54803f0be26b37004/b0d4fa54-0b81-4d71-8f5a-5923599e9005.png" alt="" style="display:block;margin:0 auto" />

<p>There’s a famous line, usually credited to Phil Karlton:</p>
<p>There are only two hard things in Computer Science: cache invalidation and naming things.</p>
<p>Invalidation just means deciding when a cached copy is no longer true, and getting rid of it.</p>
<ol start="6">
<li>The bug most teams ship When data changes, the instinct is to update the cache, then update the database. But if two updates run at the same time, they can finish in different orders. Now the cache says one price and the database says another, possibly for hours.</li>
</ol>
<img src="https://cdn.hashnode.com/uploads/covers/6ae732d54803f0be26b37004/f35e5a4d-51e6-4d23-aa29-e26f6cc93823.png" alt="" style="display:block;margin:0 auto" />

<p>The safer default is simple: write to the database first, then delete the cache key. Don’t update it, delete it. The next read misses, fetches the fresh value and refills the cache.</p>
<img src="https://cdn.hashnode.com/uploads/covers/6ae732d54803f0be26b37004/9aa6b065-0095-4703-b402-d8976a9ab12f.png" alt="" style="display:block;margin:0 auto" />

<p><code>def update_price(id, price):</code><br /><code>db.update_price(id, price) # 1. the source of truth first</code><br /><code>cache.delete(f"product:{id}") # 2. then drop the copy</code><br /><br />And always set a TTL as a safety net. Even if an invalidation gets lost, the stale copy dies on its own.</p>
<ol start="7">
<li>When the cache fills up Caches live in memory, and memory runs out. The most common eviction rule is LRU, least recently used: throw out whatever nobody has asked for in the longest time. Popular items stay, and forgotten ones leave.</li>
<li>The stampede It’s the biggest sale of the year. Your homepage is cached, and a hundred thousand people are refreshing it every minute. Then the homepage’s cache entry hits its TTL and expires.</li>
</ol>
<p>In that moment, every request misses at the same time. Thousands of requests all go to the database at once, all asking for the same homepage. The database slows down, slow queries pile up, requests time out and retry, which adds even more load. And because the database is drowning, nobody manages to refill the cache.</p>
<img src="https://cdn.hashnode.com/uploads/covers/6ae732d54803f0be26b37004/69cb9092-eee6-4069-8150-aa4a62e2b3d3.png" alt="" style="display:block;margin:0 auto" />

<p>That’s a cache stampede, also called a thundering herd. The site goes down at its busiest minute.</p>
<ol start="9">
<li>The three-line fix Let only one request rebuild the value. Everyone else waits for it, or gets the slightly old copy in the meantime. And give your TTLs a little random jitter, so thousands of keys don’t expire in the same second.</li>
</ol>
<img src="https://cdn.hashnode.com/uploads/covers/6ae732d54803f0be26b37004/5460b65b-82e0-4203-832a-08a2fda96582.png" alt="" style="display:block;margin:0 auto" />

<p>if cache.lock(key): # one request rebuilds cache.set(key, db.get(key), ttl + jitter()) else: return cache.stale(key) # everyone else waits or gets stale One request rebuilds, everyone else waits.</p>
<ol start="10">
<li>The rule that ties it together Cache data that’s read a lot and changes rarely: product pages, profiles, settings, the results of expensive queries. Give every entry a TTL. Design for the miss, because misses always happen: on deploys, on restarts, and when a key expires at the worst possible moment. A cache should make your app faster. It should never be the only thing holding it up.</li>
<li>How the biggest sites do it Facebook runs one of the largest Memcached deployments ever built. In their paper Scaling Memcache at Facebook (NSDI 2013), their engineers describe leases: one client gets a lease to refill a missing key, and everyone else waits briefly or uses a slightly stale value. That’s the stampede fix at enormous scale.</li>
</ol>
<p>CDNs like Cloudflare cache pages in data centres around the world, so most visitors never reach the origin server at all. And your browser respects cache headers like Cache-Control, which is why a site loads faster the second time you visit.</p>
<img src="https://cdn.hashnode.com/uploads/covers/6ae732d54803f0be26b37004/d182ffd9-145a-432a-bf4d-eb79c2ec9256.png" alt="" style="display:block;margin:0 auto" />

<p>Same idea everywhere: keep a copy close, and protect the source.</p>
<ol start="12">
<li>Caching in one screen A cache is a copy kept close. Cache-aside: check, miss, fetch, save. The hit ratio decides how much you win. On writes, update the database, then delete the key. Always set a TTL. Protect against stampedes with one rebuilder and jittered expiry. But there’s one thing a cache can’t fix: a slow miss. If the query behind it takes nine seconds, every miss still takes nine seconds.</li>
</ol>
<p>▶️ Watch the animated version: <a href="https://youtu.be/Jez2Rg0uUgQ?si=DwWF9Ul-elk2pxvh">Caching Explained (YouTube)</a></p>
<hr />
<h3>Follow The Byte Mechanics</h3>
<p>How software works, under the hood ⚙️</p>
<ul>
<li>🌐 <a href="https://www.thebytemechanics.com/">Website</a></li>
<li>▶️ <a href="https://www.youtube.com/@thebytemechanics">YouTube</a></li>
<li>📸 <a href="https://www.instagram.com/thebytemechanics/">Instagram</a></li>
<li>🧵 <a href="https://www.threads.com/@thebytemechanics">Threads</a></li>
<li>📘 <a href="https://www.facebook.com/thebytemechanics">Facebook</a></li>
<li>𝕏 <a href="https://x.com/thebytemechanic">X</a></li>
<li>✍️ <a href="https://medium.com/@thebytemechanics">Medium</a></li>
<li>💻 <a href="https://dev.to/thebytemechanics">DEV Community</a></li>
<li>📝 <a href="https://thebytemechanics.hashnode.dev/">Hashnode</a></li>
</ul>
]]></content:encoded></item><item><title><![CDATA[One Thread, 10,000 Users: Async and Event Loops Explained]]></title><description><![CDATA[How the event loop works, what await really does, and the one line that freezes everyone]]></description><link>https://thebytemechanics.hashnode.dev/one-thread-10-000-users-async-and-event-loops-explained</link><guid isPermaLink="true">https://thebytemechanics.hashnode.dev/one-thread-10-000-users-async-and-event-loops-explained</guid><dc:creator><![CDATA[The Byte Mechanics]]></dc:creator><pubDate>Sun, 01 Nov 2026 14:34:13 GMT</pubDate><enclosure url="https://cdn.hashnode.com/uploads/covers/6ae732d54803f0be26b37004/484e6700-cc76-4fbd-81c8-f2698e87a59f.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p><em>Originally published on</em> <a href="https://medium.com/@thebytemechanics/one-thread-10-000-users-async-and-event-loops-explained-6771e101b577"><em>Medium</em></a><em>.</em></p>
<p>This server has one thread. Ten thousand people are using it right now, and none of them are waiting.</p>
<p>Give each of those users their own thread instead, and the same server runs out of memory and falls over.</p>
<p>So how does one thread beat ten thousand? This article walks through the event loop, async/await, and the one harmless-looking line of code that can freeze every user at once. If you'd rather watch, here's the 8-minute animated version:</p>
<p><a class="embed-card" href="https://youtu.be/zYGXjMid7HY">https://youtu.be/zYGXjMid7HY</a></p>

<hr />
<h2>1. The secret is waiting</h2>
<p>Take one web request. It reads the request, asks the database for some rows, maybe calls another API, and sends a response.</p>
<p>Now time it. The CPU actually works for about two milliseconds. The other ninety-eight milliseconds are spent waiting: for the database, for the network, for the disk.</p>
<img src="https://cdn.hashnode.com/uploads/covers/6ae732d54803f0be26b37004/a82c0172-c4d8-4ccc-a676-ccd88c177fda.png" alt="fig02" style="display:block;margin:0 auto" />

<p>A 100 ms request spends only 2 ms on the CPU and the rest waiting on the database, network and disk. Your server isn't busy, it's bored</p>
<p>So your server isn't busy. It's bored. Most of every request is just waiting. And in the classic design, a waiting thread does nothing at all while still holding on to everything it owns.</p>
<h2>2. Why one thread per user runs out</h2>
<p>The classic answer is one thread per connection. A user connects, you give them a thread, and that thread waits for them. It's simple, and it works for a few hundred users.</p>
<p>But every thread has its own stack, often around a megabyte reserved. Ten thousand threads means gigabytes of memory, just to wait. And the operating system has to keep switching between all of them, so more and more time goes into switching instead of real work.</p>
<img src="https://cdn.hashnode.com/uploads/covers/6ae732d54803f0be26b37004/ba7eafab-1d83-416f-aeb7-827a3d46f8ec.png" alt="fig03" style="display:block;margin:0 auto" />

<p>Thread per connection: a grid of 5,322 threads using 5.3 GB of RAM just to wait</p>
<p>In 1999, Dan Kegel gave this wall a name: <strong>the C10K problem</strong>. How do you handle ten thousand connections on one machine? The answer didn't come from adding more threads.</p>
<h2>3. The waiter</h2>
<p>Picture a restaurant that hires one waiter per table. The waiter takes your order, walks to the kitchen, and stands there until the food is ready. A hundred tables means a hundred waiters, mostly standing around.</p>
<p>Now picture a restaurant with one waiter. They take an order, hand it to the kitchen, and go straight to the next table. When the kitchen rings the bell, they pick up the food and deliver it. One waiter, a whole room of tables, and nobody waits on the waiter.</p>
<img src="https://cdn.hashnode.com/uploads/covers/6ae732d54803f0be26b37004/d9e14ca6-ae8e-4ac4-9bd3-f926bca7d766.png" alt="fig04" style="display:block;margin:0 auto" />

<p>Restaurant analogy: one waiter per table means 100 waiters mostly standing around, versus one waiter serving every table</p>
<p>That waiter has a name in software: <strong>the event loop</strong>.</p>
<h2>4. The event loop</h2>
<p>An event loop is one thread running one simple cycle, over and over:</p>
<ol>
<li><strong>Check what's ready:</strong> a new connection, a database reply, a timer that just went off.</li>
<li><strong>Run the code</strong> for each ready thing, quickly.</li>
<li><strong>Hand off the waits.</strong> Whenever that code needs to wait, it doesn't. The wait goes to the operating system, and the loop moves on.</li>
</ol>
<p>Then it goes back to step one.</p>
<img src="https://cdn.hashnode.com/uploads/covers/6ae732d54803f0be26b37004/2772d3ac-8bd0-456a-b708-83a47c59df72.png" alt="fig05" style="display:block;margin:0 auto" />

<p>The event loop serving 10,000 connections: check what's ready, run the code quickly, hand off the waits to the OS</p>
<p>Because the thread never stops to wait, it can juggle thousands of connections. It only spends time on the small slice that's real work. But notice the rule hiding in there:</p>
<blockquote>
<p>Every step has to be quick.</p>
</blockquote>
<p>Remember that rule. It's the key to the line that freezes everyone.</p>
<h2>5. The operating system does the watching</h2>
<p>How does the loop know when something is ready? The operating system tells it.</p>
<p>Sockets can be put in <strong>non-blocking mode</strong>. Instead of sleeping until data arrives, a read returns straight away and says: nothing yet. Then the loop hands the operating system its whole list of sockets and says: wake me up when any of these are ready.</p>
<img src="https://cdn.hashnode.com/uploads/covers/6ae732d54803f0be26b37004/a4950282-24d7-4415-8135-b0972a382f2f.png" alt="fig06" style="display:block;margin:0 auto" />

<p>The kernel watches every socket and wakes the event loop only when some of them are ready</p>
<p>On Linux that's <strong>epoll</strong>. On macOS and BSD it's <strong>kqueue</strong>. On Windows, <strong>I/O completion ports</strong>. Older calls like <code>select</code> scanned every socket on every call. epoll just hands back the ready ones, so ten thousand idle connections cost almost nothing.</p>
<h2>6. From callbacks to async/await</h2>
<p>In the early days, you gave the loop a function to call later: a <strong>callback</strong>. Read the file, and when you're done, call this. Chain a few of those and the code drifts to the right, deeper and deeper. People called it callback hell.</p>
<p><strong>Promises</strong> flattened it out. A promise is a placeholder for a value that isn't ready yet. Then <strong>async and await</strong> made it look like normal code again:</p>
<pre><code class="language-javascript">const user = await getUser(id);
const orders = await getOrders(user);
return render(user, orders);
</code></pre>
<img src="https://cdn.hashnode.com/uploads/covers/6ae732d54803f0be26b37004/10fe7dda-cf4f-4ef0-81e3-597b111c2da9.png" alt="fig07" style="display:block;margin:0 auto" />

<p>Blocking code and async code look almost the same, but they are not: only the await version gives the loop back</p>
<p>It reads top to bottom, exactly like blocking code. But it isn't blocking code, and the difference is the whole trick.</p>
<h2>7. What await really does</h2>
<p>When your code hits <code>await</code>, your function doesn't sit there waiting. It pauses, saves its place, and hands control back to the loop. The loop serves other requests. When the database replies, the loop picks your function up exactly where it left off.</p>
<img src="https://cdn.hashnode.com/uploads/covers/6ae732d54803f0be26b37004/5b834b81-7d23-4cb2-b202-9a7a3725bce0.png" alt="fig08" style="display:block;margin:0 auto" />

<p>An async handler awaiting getUser and getOrders while the loop serves other requests in between</p>
<p>So <code>await</code> means: <em>I'm going to wait, so go help someone else.</em> This is called <strong>cooperative multitasking</strong>. Every function promises to give the loop back quickly.</p>
<p>Which raises a question. What happens if one function doesn't?</p>
<h2>8. The trap: one line freezes everyone</h2>
<p>The line that freezes everyone looks completely harmless:</p>
<pre><code class="language-javascript">const thumb = resizeImage(upload); // 40 MB
</code></pre>
<p>No await. Just two seconds of pure CPU work on the only thread you have. The same goes for hashing a password, parsing a huge file, a synchronous file read, a <code>sleep</code>, or an old blocking HTTP library called from async code.</p>
<img src="https://cdn.hashnode.com/uploads/covers/6ae732d54803f0be26b37004/0f2946f9-c4b8-4b3c-bee0-ae3d0568f57c.png" alt="fig09" style="display:block;margin:0 auto" />

<p>The trap: resizeImage on a 40 MB file holds the only thread for 0.7 seconds and freezes every user</p>
<p>For those two seconds, the loop is stuck. Every other request sits in the queue. Health checks time out, and your load balancer decides the server is dead. One request, and ten thousand users frozen.</p>
<p>If you read the last article, this should sound familiar. There, one bad request <em>crashed</em> everyone. Here, one slow request <em>freezes</em> everyone.</p>
<h2>9. The one rule</h2>
<blockquote>
<p>The event loop coordinates. It never does the heavy lifting.</p>
</blockquote>
<ul>
<li><strong>Send CPU-heavy work to a pool</strong> of worker threads or processes, and await the result. In Python that's <code>await asyncio.to_thread(resize, img)</code> (or a process pool for CPU-heavy Python). In Node it's <code>worker_threads</code>.</li>
<li><strong>Node already does some of this for you.</strong> File system calls, DNS lookups and some crypto run on libuv's small hidden thread pool (four threads by default), so they don't touch the loop.</li>
<li><strong>Use async versions of everything that waits:</strong> async database drivers, async HTTP clients. <code>requests.get()</code> or <code>readFileSync()</code> inside async code blocks the loop.</li>
<li><strong>If a job is huge, cut it into chunks</strong> so the loop can breathe in between.</li>
</ul>
<img src="https://cdn.hashnode.com/uploads/covers/6ae732d54803f0be26b37004/c56eb029-78fe-45ac-af47-f1a801615998.png" alt="fig10" style="display:block;margin:0 auto" />

<p>Moving heavy work off the loop: Node's hidden libuv thread pool and a worker pool for CPU-bound tasks</p>
<h2>10. How the giants do it</h2>
<p>In production you almost never see just one loop. You see one per CPU core.</p>
<ul>
<li><strong>Nginx</strong> was built for exactly the C10K problem. It starts one worker process per core, and each worker runs its own event loop.</li>
<li><strong>Node</strong> apps scale the same way, with one process per core using cluster mode or a process manager like PM2.</li>
<li><strong>Python</strong> web apps run several Uvicorn workers, each with its own asyncio loop.</li>
<li><strong>Redis</strong> executes commands on a single thread and still handles massive traffic, because each command is tiny and the loop never waits.</li>
<li><strong>Go</strong> hides all of it. You write plain blocking code in goroutines, and the runtime's network poller runs an event loop underneath for you.</li>
</ul>
<img src="https://cdn.hashnode.com/uploads/covers/6ae732d54803f0be26b37004/e9d2b679-adf2-4188-82f5-ac300c026c07.png" alt="fig11" style="display:block;margin:0 auto" />

<p>Processes for cores, loops for waiting: four processes, one per core, each running its own event loop</p>
<p>The pattern is the one from the last article: <strong>processes for cores and isolation, and inside each one, a loop for the waiting.</strong></p>
<h2>11. The cheat sheet</h2>
<p><strong>Use async</strong> when your work is mostly waiting: web servers, APIs, chat apps, proxies, anything with lots of connections doing a little work each. Its risk is blocking the loop.</p>
<p><strong>Use threads or processes</strong> for CPU-heavy work. Async won't speed up computation; for that you need more cores.</p>
<p><strong>In production, use both:</strong> one loop per core for the waiting, and a worker pool for the heavy lifting.</p>
<p>Ten thousand users, one thread. It works, as long as that thread never waits, and never does the heavy lifting itself.</p>
<hr />
<p><strong>▶️ Watch the animated version:</strong> <a href="https://youtu.be/zYGXjMid7HY">Event Loop Explained: How One Thread Handles 10,000 Users (YouTube)</a></p>
<hr />
<h3>Follow The Byte Mechanics</h3>
<p>How software works, under the hood ⚙️</p>
<ul>
<li>🌐 <a href="https://www.thebytemechanics.com/">Website</a></li>
<li>▶️ <a href="https://www.youtube.com/@thebytemechanics">YouTube</a></li>
<li>📸 <a href="https://www.instagram.com/thebytemechanics/">Instagram</a></li>
<li>🧵 <a href="https://www.threads.com/@thebytemechanics">Threads</a></li>
<li>📘 <a href="https://www.facebook.com/thebytemechanics">Facebook</a></li>
<li>𝕏 <a href="https://x.com/thebytemechanic">X</a></li>
<li>✍️ <a href="https://medium.com/@thebytemechanics">Medium</a></li>
<li>💻 <a href="https://dev.to/thebytemechanics">DEV Community</a></li>
<li>📝 <a href="https://thebytemechanics.hashnode.dev/">Hashnode</a></li>
</ul>
]]></content:encoded></item><item><title><![CDATA[One Bad Request Crashed Everyone: Processes vs Threads]]></title><description><![CDATA[How each one works, where each one bites, and which one to pick]]></description><link>https://thebytemechanics.hashnode.dev/one-bad-request-crashed-everyone-processes-vs-threads</link><guid isPermaLink="true">https://thebytemechanics.hashnode.dev/one-bad-request-crashed-everyone-processes-vs-threads</guid><dc:creator><![CDATA[The Byte Mechanics]]></dc:creator><pubDate>Sun, 01 Nov 2026 14:28:17 GMT</pubDate><enclosure url="https://cdn.hashnode.com/uploads/covers/6ae732d54803f0be26b37004/47aa76a4-2ce8-45a5-9e8e-405fb5f1b237.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p><em>Originally published on</em> <a href="https://medium.com/@thebytemechanics/one-bad-request-crashed-everyone-processes-vs-threads-4803683bf947?sharedUserId=thebytemechanics"><em>Medium</em></a><em>.</em></p>
<p>A user uploads one corrupted image. Your server crashes. And every other user on it goes down too: their uploads, their checkouts, their sessions, all gone.</p>
<p>One bad request shouldn't be able to do that. But on a lot of servers, it can. Whether it does comes down to one choice: <strong>processes or threads</strong>.</p>
<p>This article explains both, the traps that come with each, and how real systems combine them. If you'd rather watch, here's the 8-minute animated version:</p>
<p><a class="embed-card" href="https://youtu.be/QNoIC4uyFh8">https://youtu.be/QNoIC4uyFh8</a></p>

<hr />
<h2>1. The problem: many requests at once</h2>
<p>Hundreds of requests hit a server at the same time: uploads, logins, checkouts. Handle them one by one, and everyone waits, while the CPU sits idle every time a request waits on the disk or the database.</p>
<p>So the operating system gives us two ways to work on many requests at once: <strong>processes</strong> and <strong>threads</strong>.</p>
<h2>2. A process is a house</h2>
<p>A <strong>process</strong> is a running program with its own private world. The operating system gives it its own memory (code, data, heap and stack), its own ID, its own open files and its own permissions.</p>
<p>Think of a process as a house. Everything inside belongs to that house, and the neighbours can't walk in and move your furniture.</p>
<img src="https://cdn.hashnode.com/uploads/covers/6ae732d54803f0be26b37004/9c9dacd2-bce8-41d3-a940-5cbd35bc2515.png" alt="fig02" style="display:block;margin:0 auto" />

<p>A process is a house: its own code, heap, stack, files and PID, walled off from neighbouring processes</p>
<p>This isolation is enforced by hardware. Each process sees its own <strong>virtual addresses</strong>, and the CPU translates them to real memory. Process A literally cannot point at Process B's memory. So one process can corrupt its own memory, freeze or crash, and the house next door doesn't notice.</p>
<h2>3. Threads are the people inside</h2>
<p>A <strong>thread</strong> lives inside a process. It's one path of execution, a sequence of instructions the CPU runs. Every process starts with one thread and can create more.</p>
<p>If the process is the house, threads are the people living in it. Each person has their own to-do list and remembers where they are in it: that's the thread's own <strong>stack and registers</strong>. But they share everything else: the same heap, the same global variables, the same open files.</p>
<blockquote>
<p>That sharing is the whole point of threads. It's also the whole problem.</p>
</blockquote>
<h2>4. Why threads became the default</h2>
<ul>
<li><strong>Cheaper to create.</strong> A new process needs a whole new memory space. Copy-on-write makes that cheaper, but it's still real work. A new thread only needs its own stack.</li>
<li><strong>Cheaper to switch.</strong> Switching between threads of the same process keeps the same memory map. Switching between processes changes the map, and the CPU's address cache goes cold. One switch costs microseconds, and thousands per second add up.</li>
<li><strong>Easier to share data.</strong> Processes must send messages through pipes, sockets or queues, and every message is copied. One thread can simply write a variable that another thread reads.</li>
</ul>
<p>Fast, simple, no copying. But "any thread can change anything, at any time" is a dangerous sentence.</p>
<h2>5. The trap: race conditions</h2>
<p>Two threads each add one to a shared counter that starts at 100. You expect 102. But <code>count = count + 1</code> isn't one step. It's three: read, add, write.</p>
<p>Thread A reads 100. Before it writes, the CPU switches. Thread B also reads 100, adds one and writes 101. Thread A resumes, adds one to its stale 100, and also writes 101. Two increments happened. One got lost.</p>
<img src="https://cdn.hashnode.com/uploads/covers/6ae732d54803f0be26b37004/ea56a078-2a9a-48b7-925c-91ea39fc2a5e.png" alt="fig03" style="display:block;margin:0 auto" />

<p>Race condition: two threads read a shared counter of 100, both add one and write 101, so one update is lost</p>
<p>That's a <strong>race condition</strong>. It only shows up under load, disappears when you add a print statement, and never reproduces on your laptop. Now imagine the counter is an account balance, or the number of concert tickets left.</p>
<h2>6. The fix, and the next trap</h2>
<p>The standard fix is a <strong>lock</strong> (a mutex). A thread takes the lock before touching the counter, and everyone else waits until it's released. Read, add and write now happen as one unit.</p>
<p>But locks bring their own trap. Thread A holds lock 1 and waits for lock 2. Thread B holds lock 2 and waits for lock 1. Neither will ever let go.</p>
<img src="https://cdn.hashnode.com/uploads/covers/6ae732d54803f0be26b37004/217d6ef0-fdde-4b62-9e35-f9528f361777.png" alt="fig04" style="display:block;margin:0 auto" />

<p>Deadlock: thread A holds lock 1 and waits for lock 2 while thread B holds lock 2 and waits for lock 1, so everything stops</p>
<p>That's a <strong>deadlock</strong>. Nothing crashes. Everything just stops. Shared memory is fast, but every shared thing needs rules. Get them wrong, and you get lost updates or a frozen system.</p>
<h2>7. Back to the corrupted image</h2>
<p>Image decoders are often written in C. A broken file can make one touch invalid memory. When a thread does that, the operating system doesn't kill just that thread. It kills the <strong>whole process</strong>, and every other thread dies with it. That's how one upload took down 500 users.</p>
<img src="https://cdn.hashnode.com/uploads/covers/6ae732d54803f0be26b37004/f199f107-0a9f-4347-9ba7-0be640a59e78.png" alt="fig05" style="display:block;margin:0 auto" />

<p>Shared memory, shared fate: one thread touches invalid memory, the OS sends a segfault, and every thread in the server process dies</p>
<p>If each request were handled by a separate <strong>worker process</strong>, only that worker would die. A supervisor starts a fresh one, and nobody else notices.</p>
<img src="https://cdn.hashnode.com/uploads/covers/6ae732d54803f0be26b37004/9e74f391-0d13-41ef-a971-b347e14d6bc4.png" alt="fig06" style="display:block;margin:0 auto" />

<p>Separate worker processes: the bad photo.jpg crashes only worker 3 while workers 1, 2 and 4 keep running</p>
<p>Chrome made this choice on purpose. Each site gets its own renderer process, so a crashing page only kills its own tab. It's a security win too: a malicious site in one process can't read your banking tab's memory. The price is RAM, which is why Chrome uses so much of it. <strong>Isolation costs resources.</strong></p>
<h2>8. Your workload and language matter</h2>
<ul>
<li><strong>I/O-bound work</strong> (mostly waiting on databases, networks or disks): threads are great. While one waits, another runs.</li>
<li><strong>CPU-bound work</strong> (image processing, number crunching): you want real parallel execution on every core.</li>
<li><strong>Python:</strong> in standard CPython, the Global Interpreter Lock lets only one thread run Python code at a time. For CPU-heavy Python, use multiple processes, one per core.</li>
<li><strong>Node.js:</strong> your JavaScript runs on a single thread with an event loop, and heavy work goes to worker threads or separate processes.</li>
</ul>
<h2>9. Real systems use both</h2>
<ul>
<li><strong>PostgreSQL</strong> starts a process per connection. A failed query stays inside its own process, but every backend also shares one block of memory for cached data. So if one backend crashes outright, the postmaster restarts all of them to be safe. Processes are also heavy, which is why people put a connection pooler in front of it.</li>
<li><strong>MySQL</strong> handles each connection with a thread inside one server process. Lighter per connection, with shared memory.</li>
<li><strong>Nginx</strong> runs a few worker processes, usually one per core, each with an event loop juggling thousands of connections.</li>
<li><strong>A typical Python web app</strong> runs several worker processes with threads or async inside each.</li>
</ul>
<img src="https://cdn.hashnode.com/uploads/covers/6ae732d54803f0be26b37004/4e838838-9237-460e-afe8-141cf9afc460.png" alt="fig07" style="display:block;margin:0 auto" />

<p>The pattern used by PostgreSQL, MySQL, Nginx and Python web apps: processes for isolation and every core, threads or async for cheap concurrency</p>
<p>The pattern is the same everywhere: <strong>processes for isolation and to use every core, threads or async inside them for cheap concurrency.</strong></p>
<h2>10. How to choose</h2>
<img src="https://cdn.hashnode.com/uploads/covers/6ae732d54803f0be26b37004/dba690be-d82c-45a1-9734-5948a64764a7.png" alt="fig08" style="display:block;margin:0 auto" />

<p>How to choose: use processes when crashes must stay contained or you need security boundaries; use threads when tasks share lots of data</p>
<p><strong>Reach for processes</strong> when a crash must stay contained, when you need security boundaries, or when your language can't run threads in parallel.</p>
<p><strong>Reach for threads</strong> when tasks share a lot of data, when you need many lightweight workers, and when you're willing to protect shared state with care.</p>
<p>In production, expect both: a few processes, with many threads or async tasks inside. With every request in threads under one roof, one corrupted image took everyone down. With separate worker processes, one worker dies, restarts, and nobody else notices.</p>
<hr />
<p><strong>▶️ Watch the animated version:</strong> <a href="https://youtu.be/QNoIC4uyFh8">One Bad Request Crashed Everyone: Processes vs Threads (YouTube)</a></p>
<p><strong>Next up:</strong> how async and event loops handle thousands of connections with a single thread. <a href="https://youtu.be/zYGXjMid7HY">Watch One Thread, 10,000 Users</a>.</p>
<hr />
<h3>Follow The Byte Mechanics</h3>
<p>How software works, under the hood ⚙️</p>
<ul>
<li>🌐 <a href="https://www.thebytemechanics.com/">Website</a></li>
<li>▶️ <a href="https://www.youtube.com/@thebytemechanics">YouTube</a></li>
<li>📸 <a href="https://www.instagram.com/thebytemechanics/">Instagram</a></li>
<li>🧵 <a href="https://www.threads.com/@thebytemechanics">Threads</a></li>
<li>📘 <a href="https://www.facebook.com/thebytemechanics">Facebook</a></li>
<li>𝕏 <a href="https://x.com/thebytemechanic">X</a></li>
<li>✍️ <a href="https://medium.com/@thebytemechanics">Medium</a></li>
<li>💻 <a href="https://dev.to/thebytemechanics">DEV Community</a></li>
<li>📝 <a href="https://thebytemechanics.hashnode.dev/">Hashnode</a></li>
</ul>
]]></content:encoded></item><item><title><![CDATA[Stop Starting With Microservices: The Path Real Systems Actually Follow]]></title><description><![CDATA[Originally published on Medium. A new project kicks off. Someone opens a blank page and says, "Let's build it with microservices." Everyone nods. It sounds…]]></description><link>https://thebytemechanics.hashnode.dev/stop-starting-with-microservices-the-path-real-systems-actually-follow</link><guid isPermaLink="true">https://thebytemechanics.hashnode.dev/stop-starting-with-microservices-the-path-real-systems-actually-follow</guid><dc:creator><![CDATA[The Byte Mechanics]]></dc:creator><pubDate>Sun, 01 Nov 2026 13:34:39 GMT</pubDate><enclosure url="https://cdn.hashnode.com/uploads/covers/6ae732d54803f0be26b37004/5311184e-a5c5-439a-b017-e978a89809cb.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p><em>Originally published on</em> <a href="https://medium.com/p/d9b73e1233ae"><em>Medium</em></a><em>.</em></p>
<p>A new project kicks off. Someone opens a blank page and says, <em>"Let's build it with microservices."</em> Everyone nods. It sounds modern. It sounds scalable.</p>
<p>But here's the surprise: <strong>most high-scale systems don't start with microservices.</strong> They grow into them, and only when the system demands it.</p>
<p>This article walks through that growth one stage at a time. At each stage you'll see what breaks, why it breaks, and what teams do about it. If you'd rather watch, there's a 10-minute animated version of this article:</p>
<p><a class="embed-card" href="https://www.youtube.com/watch?v=HKnO1UPimhw">https://www.youtube.com/watch?v=HKnO1UPimhw</a></p>

<hr />
<h2>1. Start with a monolith</h2>
<p>Every system starts somewhere, and the first goal isn't scale. It's <strong>correctness, speed of development and clarity</strong>. That's why most successful systems begin as a monolith.</p>
<p>A <strong>monolith</strong> is one application that holds every feature:</p>
<ul>
<li><strong>One backend</strong>, the code that runs on the server.</li>
<li><strong>One deployment</strong>, so you ship it as a single unit.</li>
<li><strong>Usually one database</strong>, where all the data lives.</li>
</ul>
<p>Inside, each feature is a <strong>module</strong>, a separate area of the code. Modules call each other directly, as ordinary function calls with no network in between. That keeps two things easy:</p>
<ul>
<li><strong>Transactions are simple.</strong> A transaction is a group of changes that succeed or fail together, and a single database handles it for you.</li>
<li><strong>Debugging is easy.</strong> One process, one log and one stack trace.</li>
</ul>
<img src="https://cdn.hashnode.com/uploads/covers/6ae732d54803f0be26b37004/3653ab3e-a9df-48a9-a5bf-161e2de081ee.png" alt="fig02" style="display:block;margin:0 auto" />

<p>This leads to the first rule of architecture:</p>
<blockquote>
<p><strong>Don't add distributed complexity, meaning spreading work across many machines, until the system forces you to.</strong></p>
</blockquote>
<hr />
<h2>2. When the monolith feels heavy, fix coupling first</h2>
<p>As features and engineers pile up, the monolith starts to feel heavy:</p>
<ul>
<li>Builds slow down.</li>
<li>Deployments get risky.</li>
<li>A small change breaks something unrelated.</li>
<li>Teams collide in the same repository.</li>
</ul>
<p>Many teams jump straight to microservices here. <strong>That's a mistake</strong>, because the real problem isn't the monolith. It's <strong>coupling</strong>: parts that depend too tightly on each other's insides.</p>
<p>The fix is a <strong>modular monolith</strong>. It still ships as one unit, but inside it's split into clean modules:</p>
<ul>
<li>Each module has a <strong>defined interface</strong>, the official doorway other code must use.</li>
<li>Each module <strong>owns its data</strong>.</li>
<li>No module <strong>reaches into another's internals</strong>.</li>
<li>Each module follows a <strong>domain boundary</strong>, a business area like orders or payments.</li>
</ul>
<img src="https://cdn.hashnode.com/uploads/covers/6ae732d54803f0be26b37004/aae9b954-741d-4fe5-8378-cde17ad2105a.png" alt="fig03" style="display:block;margin:0 auto" />

<p>This step matters more than it looks. <strong>If you split tightly coupled code into services, the coupling doesn't disappear. It just moves onto the network</strong>, where it's slower, harder to debug and can fail in new ways. Clean module boundaries now become clean service boundaries later.</p>
<hr />
<h2>3. When traffic grows, split into microservices</h2>
<p>In a monolith, <strong>scaling means scaling everything</strong>. You add more running copies of the whole app, called <strong>instances</strong>. If payments get heavy traffic, the entire system gets more instances, including all the parts that were fine.</p>
<p>At small scale that's fine. At large scale it's inefficient and expensive, because different parts behave differently:</p>
<ul>
<li><strong>Read-heavy</strong> parts mostly fetch data.</li>
<li><strong>Write-heavy</strong> parts mostly save data.</li>
<li><strong>CPU-heavy</strong> parts need lots of processing power.</li>
<li><strong>I/O-heavy</strong> parts spend their time waiting on disks or networks.</li>
</ul>
<p>This is usually the moment to move to <strong>microservices</strong>: small, separate applications, each owning one domain. Each one can <strong>scale, deploy and fail on its own</strong>.</p>
<img src="https://cdn.hashnode.com/uploads/covers/6ae732d54803f0be26b37004/11c056a8-03ca-4ab1-a4eb-d1b38a917239.png" alt="fig04" style="display:block;margin:0 auto" />

<p>That solves scaling. But there's a catch. <strong>Before, everything lived in one process. Now everything lives on the network.</strong> Every section below is a consequence of that one sentence.</p>
<hr />
<h2>4. Services need to find each other: service discovery and the API gateway</h2>
<p>Services now run on different machines, containers or pods. (A <strong>container</strong> is a packaged, isolated box for running an app, and a <strong>pod</strong> is a common way of grouping containers, for example in Kubernetes.) These instances restart, autoscale and move around, so <strong>hardcoding addresses is impossible</strong>.</p>
<p>The fix is <strong>service discovery</strong>. A <strong>registry</strong> tracks every available instance, and services ask it where to send each request.</p>
<p>The next problem is which service the app on your phone should call. With one backend there was one API. Now there are many. You don't want login checks or traffic limits copied into every service, and you don't want the frontend knowing your internal layout.</p>
<p>So you add an <strong>API gateway</strong>, a single entry point that handles:</p>
<ul>
<li><strong>Routing</strong> each request to the right service</li>
<li><strong>Authentication</strong>, checking who you are</li>
<li><strong>Logging</strong></li>
<li><strong>Throttling</strong>, limiting how fast a client can call</li>
<li>Sometimes <strong>combining responses</strong> from several services</li>
</ul>
<img src="https://cdn.hashnode.com/uploads/covers/6ae732d54803f0be26b37004/9d182c88-77ea-40ef-b5aa-797ffa8ef75f.png" alt="fig05" style="display:block;margin:0 auto" />

<hr />
<h2>5. The database becomes the bottleneck</h2>
<p>As traffic keeps climbing, the first thing to struggle is usually <strong>the database, not the services</strong>. Services scale horizontally: you just add more machines. Databases are much harder.</p>
<p>The symptoms are familiar:</p>
<ul>
<li>Queries slow down.</li>
<li>CPU spikes.</li>
<li><strong>Locks</strong> appear. A lock is a hold on data while it's being changed, which makes everyone else wait.</li>
<li>Write delays grow.</li>
</ul>
<p><strong>Fix 1: read replicas.</strong> These are read-only copies of the database. Heavy queries move there, so reads are separated from writes. That works for a while, until the <strong>primary</strong>, the main copy that accepts writes, becomes the bottleneck itself.</p>
<p><strong>Fix 2: sharding.</strong> The data is split across multiple machines, usually:</p>
<ul>
<li><strong>By user</strong></li>
<li><strong>By tenant</strong>, meaning a customer organization</li>
<li><strong>By hash</strong>, a formula that assigns each record to a bucket</li>
</ul>
<img src="https://cdn.hashnode.com/uploads/covers/6ae732d54803f0be26b37004/0091fe3a-622a-48e9-b39d-f41898d8cffd.png" alt="fig06" style="display:block;margin:0 auto" />

<hr />
<h2>6. Transactions across services: the Saga pattern</h2>
<p>In a monolith, one transaction can safely update many tables. That's an <strong>ACID transaction</strong>: a guarantee that a group of changes happens completely or not at all, and stays correct even when others are working at the same time.</p>
<p>Now one request, like <em>create an order</em>, may touch the <strong>order, payment, inventory and notification</strong> services, each with its own database. There's no easy way to wrap all of them in one ACID transaction. Forcing <strong>strong consistency</strong>, where everyone sees the same correct data at the same moment, usually hurts performance and reliability.</p>
<p>So high-scale systems accept <strong>eventual consistency</strong>: data may be briefly out of sync, but it catches up. They get there with three tools:</p>
<ul>
<li><strong>Events</strong>, to tell other services what happened</li>
<li><strong>Retries</strong>, to recover from temporary failures</li>
<li><strong>Compensation</strong>, which means undoing an earlier step</li>
</ul>
<p>The common pattern that combines them is the <strong>Saga</strong>. Each step succeeds independently. If a later step fails, earlier steps are retried or undone. For example, if stock can't be reserved, the payment is refunded and the order is cancelled.</p>
<img src="https://cdn.hashnode.com/uploads/covers/6ae732d54803f0be26b37004/99819e32-4a09-44ca-ab3c-202bd1344c74.png" alt="fig07" style="display:block;margin:0 auto" />

<p>It's more scalable, but it's also more complex. That trade-off comes up at every stage.</p>
<hr />
<h2>7. Stop waiting: event-driven design</h2>
<p>With a few services, <strong>synchronous calls</strong> work fine. Synchronous means the caller waits for an answer, usually over HTTP. As services multiply, these call chains become slow and fragile:</p>
<ul>
<li>One slow service delays everything.</li>
<li>One failure breaks the whole flow.</li>
<li>Traffic spikes overload the services further down the line.</li>
</ul>
<p>So high-scale systems switch to <strong>messaging</strong>. An <strong>event</strong> is a small message saying <em>something happened</em>. Instead of calling a service directly, you <strong>publish the event to a queue or stream</strong>, an ordered line of messages waiting to be processed. <strong>Consumers</strong>, the services that read events, handle them independently.</p>
<p>What you gain:</p>
<ul>
<li>Services become <strong>loosely coupled</strong>.</li>
<li><strong>Spikes are absorbed</strong> by the queue.</li>
<li><strong>Failures can be retried.</strong></li>
<li>Work happens <strong>asynchronously</strong>, so no one waits.</li>
</ul>
<img src="https://cdn.hashnode.com/uploads/covers/6ae732d54803f0be26b37004/46225cc8-756f-4701-9556-8c059ac1994f.png" alt="fig08" style="display:block;margin:0 auto" />

<p>That's why large systems lean on <strong>Kafka</strong>, a popular event-streaming platform, as well as queues and <strong>pub-sub</strong>, where one sender reaches many listeners.</p>
<hr />
<h2>8. Retries are dangerous without idempotency</h2>
<p>In distributed systems, <strong>failures are normal</strong>. Messages get retried. Requests time out, giving up after waiting too long. A consumer can crash <strong>after doing the work but before acknowledging it</strong>, that is, before telling the queue <em>"I'm done."</em></p>
<p>So <strong>the same request may run more than once</strong>. For payments, orders, wallet updates or inventory changes, that's dangerous. Picture a customer who pays $20 once and gets charged $40.</p>
<p>The answer is <strong>idempotency</strong>: <em>the same request produces the same result, even if it runs multiple times.</em> Systems achieve it with two tools:</p>
<ul>
<li><strong>Idempotency keys</strong> (or unique request IDs), labels attached to each request</li>
<li><strong>Deduplication tables</strong>, a record of which requests have already been handled</li>
</ul>
<p>When a request arrives, the consumer checks its key. If the key isn't in the table, it processes the request and records the key. If the key is already there, it skips the work and returns the earlier result.</p>
<img src="https://cdn.hashnode.com/uploads/covers/6ae732d54803f0be26b37004/1c29f1c6-c2cf-4f7d-8d7a-d1fdf2038556.png" alt="fig09" style="display:block;margin:0 auto" />

<p>At scale, <strong>idempotency isn't an optimization. It's a requirement.</strong></p>
<hr />
<h2>9. Designing to fail safely: rate limits and backpressure</h2>
<p>At high scale, traffic isn't steady. Requests can suddenly spike, and a system that tries to handle everything at once can crash. So it has to <strong>control the flow</strong>:</p>
<ul>
<li><strong>Rate limiting</strong> restricts how much traffic is allowed in.</li>
<li><strong>Backpressure</strong> slows down producers, the senders, when consumers can't keep up.</li>
<li><strong>Queues</strong> buffer short spikes.</li>
<li><strong>Dead-letter queues</strong> capture messages that keep failing, so they can be inspected instead of blocking everything else.</li>
</ul>
<p>Sometimes the right move is to <strong>reject a request</strong> with a polite "try again later" (HTTP 429) instead of crashing the whole service.</p>
<img src="https://cdn.hashnode.com/uploads/covers/6ae732d54803f0be26b37004/f4de938d-6d26-41b4-aa4d-870d50d9d368.png" alt="fig10" style="display:block;margin:0 auto" />

<blockquote>
<p>High-scale systems aren't designed to never fail. <strong>They're designed to fail safely.</strong></p>
</blockquote>
<hr />
<h2>10. Observability and the CAP trade-off</h2>
<p>In a monolith, logs are usually enough. In a distributed system, <strong>one request can pass through many services</strong>, and you need three more tools:</p>
<ul>
<li><strong>Tracing</strong> follows one request's path, so you can see which service is slow.</li>
<li><strong>Metrics</strong> are numbers tracked over time, so you can see where load is growing.</li>
<li><strong>Alerts</strong> tell you about problems <em>before</em> users complain.</li>
</ul>
<p>Together these make up <strong>observability</strong>, the ability to see inside a running system. At this scale it has to be part of the design, not an afterthought.</p>
<p>Then comes the biggest trade-off, known as <strong>CAP</strong>:</p>
<ul>
<li><strong>Consistency:</strong> everyone sees the same latest data.</li>
<li><strong>Availability:</strong> the system always answers.</li>
<li><strong>Partition tolerance:</strong> the system keeps working when the network splits parts apart.</li>
</ul>
<p>Networks <em>do</em> split in distributed systems, so partition tolerance isn't optional. When a split happens, you have to choose: <strong>keep answering, even with slightly stale data (availability), or refuse until the data is correct (consistency).</strong> You can't have all three, and different parts of the same product choose differently:</p>
<ul>
<li><strong>Payments and wallets</strong> favor <strong>consistency</strong>. A wrong balance is worse than a short wait.</li>
<li><strong>Feeds and search</strong> favor <strong>availability</strong>. A slightly stale feed beats an error page.</li>
<li><strong>Analytics</strong> accepts <strong>eventual consistency</strong>. Numbers that catch up in a minute are fine.</li>
</ul>
<img src="https://cdn.hashnode.com/uploads/covers/6ae732d54803f0be26b37004/b92b7015-3ea9-428b-8464-2f79ef556069.png" alt="fig11" style="display:block;margin:0 auto" />

<hr />
<h2>The path, in five steps</h2>
<p>Most successful systems follow the same path:</p>
<ol>
<li><strong>Start simple.</strong></li>
<li><strong>Add structure.</strong></li>
<li><strong>Split when needed.</strong></li>
<li><strong>Distribute when required.</strong></li>
<li><strong>Optimize when scale demands it.</strong></li>
</ol>
<h2>Three takeaways</h2>
<img src="https://cdn.hashnode.com/uploads/covers/6ae732d54803f0be26b37004/40d403b9-b8d9-4554-8b6b-152a6921d8ed.png" alt="fig12" style="display:block;margin:0 auto" />

<p><strong>1. Start with a monolith.</strong> Don't add distributed complexity until the system forces you to.</p>
<p><strong>2. Fix coupling first.</strong> Use a modular monolith before you split into services. Otherwise you'll just move the coupling onto the network.</p>
<p><strong>3. Microservices aren't a starting point.</strong> They're a response to scale, and every stage brings new problems to solve: discovery, data, transactions, retries, traffic and visibility.</p>
<blockquote>
<p>Good architecture isn't the most advanced design. <strong>It's the right design for today, plus knowing when it's time to evolve.</strong></p>
</blockquote>
<hr />
<p><strong>▶️ Watch the animated version:</strong> <a href="https://www.youtube.com/watch?v=HKnO1UPimhw">Stop Starting With Microservices (YouTube)</a></p>
<p><strong>Next up:</strong> why one bad request can crash every user on a server, and how processes and threads decide the blast radius. <a href="https://youtu.be/QNoIC4uyFh8">Watch Processes vs Threads</a>.</p>
<hr />
<h3>Follow The Byte Mechanics</h3>
<p>How software works, under the hood ⚙️</p>
<ul>
<li>🌐 <a href="https://www.thebytemechanics.com/">Website</a></li>
<li>▶️ <a href="https://www.youtube.com/@thebytemechanics">YouTube</a></li>
<li>📸 <a href="https://www.instagram.com/thebytemechanics/">Instagram</a></li>
<li>🧵 <a href="https://www.threads.com/@thebytemechanics">Threads</a></li>
<li>📘 <a href="https://www.facebook.com/thebytemechanics">Facebook</a></li>
<li>𝕏 <a href="https://x.com/thebytemechanic">X</a></li>
<li>✍️ <a href="https://medium.com/@thebytemechanics">Medium</a></li>
<li>💻 <a href="https://dev.to/thebytemechanics">DEV Community</a></li>
<li>📝 <a href="https://thebytemechanics.hashnode.dev/">Hashnode</a></li>
</ul>
]]></content:encoded></item></channel></rss>