Our security team’s vision for defending the modern web
Senior Director of Security Research
As the security team at Fastly, our vision is to employ our CDN’s unique position to defend the modern web. This vision, recently articulated by CSO Window Snyder, is of a world where your CDN is able to leverage visibility and team expertise to provide rapid, robust virtual patches to web applications before they can be widely breached. This “air cover” enables web properties to implement patches and mitigations, and should result in fewer website breaches. Web attacks such as SQL injection (SQLi) contribute to the root cause of a significant percentage of reported cyber attacks. Fastly brings the required elements to this situation: a large network, a flexible platform based on Varnish and VCL, and a growing security research team.
Benefits & challenges
A footprint the size of Fastly’s provides a way to shield and defend a great number of web applications against all sorts of attacks. The obvious benefit of such a large network is to guard web application availability against distributed denial of service (DDoS) attacks — it’s simply a bandwidth competition and network control plane advantage you get at this scale.
However, not everything is so straightforward. The challenge to securing the network at scale stems from our position as a third party; we don’t own the origin web applications. If you roll out a defense mechanism, you risk breaking applications at scale and scrambling to revert that change — in short, a miserable situation. It turns out that many web applications don’t behave as you would expect, and a third-party vendor like Fastly has to work within these constraints. Because of this, we try to take a hands-off approach as much as possible, enabling customers to implement Fastly-developed or their own defenses as appropriate (but are always willing to provide as much support as our customers need from us). Keeping pace with the attackers, however, is crucial if we’re to progress towards our vision.
Our response to HTTPoxy
The security team at Fastly works with the global network and security operator community together with the security research community to gather data that enables us to stay abreast of the latest threats. Both formal (e.g. FIRST) and informal (e.g. Ops-Trust) networks built on personal and professional relationships enable those communities to defend the internet. A significant portion of the global internet participates, representing ISPs, security vendors, and major internet application providers. This community works continuously to detect threats and defeat them. As a part of this community, Fastly can prepare for major bugs and implement mitigations, defending customers and the larger internet as a whole.
These communities came into play when we received a pre-release notice about the HTTPoxy issue, giving us time to investigate how we might use Fastly’s infrastructure to defend against the attack. With the details in hand, the security team was able to quickly see the issue and how to use our Varnish-based edge platform to thwart it. By the time information was public, we had a solution (really one line of VCL) drafted and tested, and we then worked on customer communications. Shortly after release of the HTTPoxy advisory, the community embargo was lifted and we were able to share this communication with customers. This enabled customers to quickly go from the vulnerability notice to a solution without forcing them to do their own research.
Going forward
Threats on the web won't be going away any time soon. Modern web applications are more feature rich and complex than ever, and valuable data continues to live in these websites. Fastly’s network position, platform, team, and capabilities therefore enable a new way of tackling thorny security issues. The HTTPoxy example serves as a model for how we’re approaching the vision outlined by Window — work within the broader operator and vendor community to stay on top of emerging security issues, and then leverage the excellent security team at Fastly and our platform to deploy those mitigations to the web at large. The complexities and challenges we face are worth tackling if we’re to progress towards our ultimate goal of a more secure web for all.
Links
