Why having more POPs isn’t always better

One of the most interesting parts about working at Fastly is addressing questions about how our offering differs from legacy providers. Answering these questions is pretty easy (tl;dr we can cache event-driven content, we enable real-time interactions with your services, and we seamlessly integrate into your technology stack and existing workflows), but there is one question that comes up more often than others:

"<Insert legacy vendor here> has more points of presence (POPs) than Fastly — doesn’t that mean they’re faster?"

Our team enjoys fielding this question because it lets us talk about the unique approach Fastly took when building out our network. To help illustrate, I’ll use an analogy of convenience stores versus supermarkets.

Convenience stores vs. supermarkets

Think of legacy POPs like convenience stores. You’ll find one on the corner of almost every street. Since they’re close to you, you can reach these stores quickly, but they only have the bare essentials. That means you may need to go online to order the rest of the items on your shopping list and have them shipped overnight (analogous to a cache miss forcing a request back to the origin server). Alternatively, you can go to a large supermarket a few miles away (in this analogy, this would be one of Fastly’s powerful POPs). It may take slightly longer to reach, but the vast selection means that you have a much better chance of finding everything on your shopping list quickly (nearly everything is cached, which results in a higher cache hit ratio).

The legacy vendor dilemma

Legacy CDN vendors are in a tough spot. They’ve all built their architectures based on the conditions that were prevalent 15 to 20 years ago. To their credit, legacy vendors actually made the right design decision at the time — back then the latency (the time taken for data to travel) from your home to an internet backbone, often over dialup, was much, much higher. They went with a larger number of smaller POPs (convenience stores) since deploying larger, higher-capacity POPs (supermarkets) would have meant significantly longer response times for users. However, these vendors are now stuck with hundreds of thousands of small, disparate servers around the world even though the incremental time for a request to get to a strategically placed, high-powered POP is now only a few additional milliseconds. These smaller POPs can’t hold as much content, forcing many more requests to go back to origin and resulting in hundreds of milliseconds of additional latency (which translates to slow load times for end users and additional hardware costs for customers).

Upgrading all of these smaller POPs is practically impossible, so legacy vendors have resorted to positioning this limitation of their networks as a strength.

The Fastly advantage

Fastly was built to take advantage of modern internet architecture. We incorporated the lessons learned from our predecessors and built a fundamentally different network. Instead of deploying many small servers around the world (convenience stores), we decided to build powerful POPs (like Costco-style supermarkets), and large amounts of memory in well-connected locations around the world meaning that we provide a higher cache hit ratio, significantly reducing the need for requests to go back to origin.

You might assume that building a larger cache means that the time to return cached data goes up. Returning to our supermarket metaphor: the larger a store is, the longer it could take to find what you’re looking for.

However, we were able to take advantage of newer solid-state drives (SSDs). Unlike regular hard drives, SSDs have no moving parts, creating constant lookup times for content no matter how big the drives are. To use our analogy one final time (we promise) —  it’s like having all of your items waiting for you at the supermarket checkout stand instead of having to walk around the supermarket with a huge cart hunting for each item on your list. This results in faster user experiences and the ability to eliminate or repurpose more of your hardware, drastically reducing Fastly’s total cost of ownership as compared to legacy solutions.

Don’t take our word for it — check out the results some of our customers have achieved by leveraging our modern network design.

• Drupal’s cache hit ratio went from 70% to over 90%, and now their site loads about a second faster (2.16s to 1.42s) than it did before.

• Lonely Planet’s cache hit ratio improved from 20% to 70%, resulting in a 350% decrease in origin load.

• Mavericks Invitational achieved an average cache hit ratio of 99.9% during their annual contest, allowing them to successfully handle 10,000 requests per second on very little hardware.

• Wanelo improved mobile performance, cut response times, and attained a 98% cache hit ratio while scaling their user base 1,000% in just a year.

• Catch Digital experienced a 98-99% cache hit ratio, allowing them to easily handle a 25x spike in traffic after a successful launch.

That being said, our market is still growing and evolving so we strategically add POPs along well-connected Internet Exchange Points (IXPs) around the world. Deploying our SSD-powered POPs in key regions helps our customers deliver better experiences no matter where their users are located. If you haven’t already done so, I encourage you to sign up for a Fastly account to try our next-generation CDN out for yourself — you can test up to $50 worth of traffic for free. We’re interested to hear what you think!