Cache the uncacheable, and save huge on egress

CHR vs Origin offload

Content delivery networks (CDNs) have long been fixated on cache hit ratio (CHR), but there are two significant issues that you need to understand to get the best performance and save the most money for your organization. 

1. Many people underestimate the impact they can have to reduce traffic to their origin and achieve huge savings on egress with seemingly "minor" CDN caching improvements. 

2. CHR isn’t always the best measure for total offload to a CDN. CHR measures the number of requests handled from cache but doesn’t take into account the amount of data, so a measure that focuses on the actual bytes offloaded, rather than just the number of requests can be much more useful.

Saving on egress with a better CDN

Egress traffic, which refers to the amount of data transferred out of the origin infrastructure, is a significant cost factor for many organizations. Everyone understands its importance for the streaming industry and companies that serve large downloads, but it’s just as important across virtually every industry. Even if a lot of your traffic is highly dynamic or changes often – including API calls and other things you might not think about being served from cache- you could probably be caching a lot more, improving your CHR and origin offload performance.

To understand why a small increase in origin offload can significantly reduce origin load, think about the miss rate. When the offload improves from 90% to 95%, the miss rate drops from 10% to 5%. This means the miss rate is now halved. Since the requests that miss the cache must be fetched from the origin, halving the miss rate effectively halves the load on the origin. If you start with a total traffic of 100 GB, a great CDN will offload 95%, leaving 5 GB to hit your origin. A good CDN will only offload 90%, leaving 10 GB – twice as much – to hit your origin.

A 5% improvement in CDN offload performance can mean a 50% load reduction at origin (see below)! Half the cache misses and half the traffic to origin means you can get rid of 50% of your servers and half of your costs. If you’ve been trying to optimize for performance gains in your infrastructure, getting an extra percent here and a half of a percent there, that all gets blown completely out of the water by the huge gains delivered by caching more with a better CDN.

You can cache more than you think (and it will have a big impact)

There are many reasons why a top-tier edge cloud platform and CDN will always outperform a legacy CDN by a mile, even a pretty good CDN. (If you want to know all of them, get in touch!) One of the easiest ways to improve performance metrics and origin offload is by caching dynamic content. API traffic is a great example of content that many industries assume to be too dynamic to benefit from caching – think about SaaS, Finserv, Travel, eCommerce, Gaming, and others who have tons of APIs and dynamic content outside of what traditional CDNs would tell you to cache. This is a huge missed opportunity for just about everyone operating on the internet.

Fastly’s Instant Purge is a great example of how a better CDN can help you do more. Fastly can purge content globally in 150 ms (as of December 31, 2022), which means that anything updating from your origin less frequently than every 150 ms can be served from cache and it takes a surprisingly low rate of requests per second (RPS) occurring against that data to achieve huge amounts of traffic offload from your origin. 

This chart shows the percentage offload for a given API or other data-requesting endpoint in your infrastructure based on how often it might be refreshed with new data (from 200 ms to once per day) versus the number of requests per second against that endpoint. You can see that even data that updates every 200ms can achieve a whopping 50% offload at only 10 RPS. Most of this chart is green, showing > 90% offload! With this kind of benefit, every API should be served through cache. When we share stats like Fastly customers getting an 189% return on their investment with us, this kind of egress savings is a big component of what makes that possible.

Translated into dollars through some rough back-of-the-envelope calculations, here’s what it means to your bottom line. At a 90% offload rate and egress cost of $0.05 per GB, it only takes an average request size of 1 MB at 5 RPS to save $120,000 per year. Think about that spread across every API and other part of your infrastructure responding to requests and it’s easy to understand how an extra 5% of origin offload may not sound like a huge leap but can be a huge source of savings and infrastructure simplification. And these estimates are pretty conservative! To get $0.05 per GB you must have negotiated a pretty good deal with AWS, and be serving entirely out of the East Coast of North America. Odds are there’s a lot of your traffic that’s currently more expensive than what we’re showing. 

Big reductions in origin offload also let you reduce the size of your infrastructure at origin. This goes beyond enabling your business to pay way less on egress charges and also creates a better experience for your users. The more you serve from the CDN’s cache or from the edge, the faster your sites and apps will respond. This (obviously) serves a better user experience and translates directly into increased revenue, especially if you’re an ecommerce, travel and hospitality, or any other business model where load times can make or break your conversion rates.

Beyond CHR - why we care about origin offload

CHR measures the ratio of requests served from origin vs. the CDN cache, and while that’s one good way to look at how good you’re doing at caching and how well your CDN is performing, it’s not the whole picture. Origin offload can round out the bigger picture when it comes to caching - it’s critical to your origin server efficiency because it reduces the volume of traffic that your origin server must process, and the network capacity required to deliver that traffic to our CDN. For example, if you improved your CHR by moving all of your smallest size requests to be served out of cache, it might make less of an impact to your origin and egress bills than offloading something with few requests, but huge request sizes.

When thinking about the capacity of your origin infrastructure, requests and egress traffic (measured in bytes) are the two key components of capacity planning. A higher cache hit ratio indicates that more requests are being served from our edge caches rather than your origin server, significantly reducing latency and improving your origin’s response times.

A high origin offload reduces the infrastructure and operational costs associated with scaling your origin to handle peak loads. Improving origin offload is essential for maintaining efficient resource utilization and ensuring an uninterrupted experience for your end users. Furthermore, it reduces operational stress on your origin infrastructure, minimizing the opportunities for bottlenecks and server outages.

In the example below, you can see a case for a site where shielding and request collapsing were disabled (at the vertical dashed line). You can see that while CHR recovers to be basically in the same ballpark as it was before the request collapsing was disabled, but the amount of traffic served from origin goes up significantly and stays up! The amount of traffic offloaded has tanked even though the CHR hasn’t changed very much. We’re seeing a 4x increase in bytes fetched from origin without the shielding and request collapsing in place – that will translate into higher egress charges pretty directly.

As you can see in the graph, with shielding, CHR peaks around the low 90% zone, while origin load is capped well below 5GiB/s. Without shielding, after an initial impact to CHR and origin load, CHR recovers to low 90% while the origin load stabilizes in the low 20GiBs range, about 4x the origin load with shielding.

The point is advanced features like better request collapsing or media shielding (which does even more for streamers) can seriously improve your origin offload even if the CHR looks similar between CDN providers ‘on paper’. Behind the scenes, Fastly can cut a ton of egress traffic and bandwidth out, even if it’s a small blip on overall CHR relative to your total requests served.

To sum it up, a good CDN can improve your CHR, but a great CDN should do more to impact your bottom line, your margins, and your organization’s success. 

Only on Fastly

Fastly is uniquely positioned to help you with both CHR and origin offload much more than the competition because we’ve taken a fundamentally different approach.

We’ve focused our efforts on placing fewer, more powerful POPs at strategic locations around the world. Fastly’s high-density POPs enable us to serve more from cache, including static and event-driven content. By leveraging Fastly edge caches to serve dynamic, static, and frequently accessed content. When you add in our superior request collapsing, you start to understand how Fastly can help you achieve better rates of offload.

Fastly customers saw an ROI of 189%^* when moving to us – it may sound too good to be true, like a fake or inflated number, but it’s very real. One of the ways we get you to numbers like that is by saving big on egress in ways that the competition just can’t match.

Fastly’s CDN will always outperform a legacy CDN, get real-time content control, boost performance, and develop modern applications – all faster at the edge and all on our global network. Want to know more? Let’s talk!

^* A commissioned study conducted by Forrester Consulting on behalf of Fastly, July 2023. Results are based on a composite organization.