1. Home
2. Solutions
3. Tutorials

Requests passing through Fastly can be transformed in many useful ways, and one of the most common is to add information to a request that was not included by the client by appending additional HTTP headers before sending the request on to the backend.

Fastly exposes a variety of information automatically, such as the geolocation and network related information available in Rust via the Geo interface. This information is simple to add to a client request before passing it to an origin:

5 lines
 
 
  let client_ip = req.get_client_ip_addr().unwrap();
  let geo = geo_lookup(client_ip).unwrap();
  let country_code = geo.country_code();
  req.set_header("Fastly-Geo-Country", country_code);
2 lines
 
 

There are also countless data sources that can provide valuable information and add intelligence to your applications. If these sources expose an API, we can query it at the edge to enrich requests with the data that service provides. This might be one of your own services, like an A/B testing API, or a third-party service.

For the purpose of this tutorial, we will detect requests that contain passwords, send an API request to the Have I Been Pwned (HIBP) API to check whether the password has been leaked, and then add a header to the request before it is forwarded to the origin.

Have I Been "Pwned"?

"Have I been pwned" (HIBP) is a community service that maintains a database of compromised passwords. It provides an API endpoint that allows passwords to be checked against that database in a privacy-preserving way. This works using a k-anonymity principle, which we can invoke like this:

1. Take the original, clear-text credential, such as '123456'.
2. Make a SHA1 hash out of the credential, which results in a 40 character string.
3. Split the hash into two strings: the first 5 characters, and the remaining 35 characters.
4. Send the first 5 characters to the HIBP API.
5. HIBP returns a list of all the SHA1 hashes in its database that begin with those 5 characters.
6. Use the last 35 characters to determine whether the full hash is in the list.

This mechanism allows a precise trade-off to be made between information leakage and functionality. Let's see how this works using command-line tools:

$ printf '123456' | openssl sha1
(stdin)= 7c4a8d09ca3762af61e59520943dc26494f8941b

$ curl https://api.pwnedpasswords.com/range/7c4a8 | grep -i 'd09ca3762af61e59520943dc26494f8941b'
D09CA3762AF61E59520943DC26494F8941B:24230577

Based on the response, we can tell that the password has been reported compromised 24,230,577 times. So '123456' turns out to be a bad idea for a password.

Set up a Rust based Compute project

This tutorial assumes that you have already have the Fastly CLI installed. If you are new to the platform, read our Getting Started guide.

Initialize a project

If you haven't already created a Rust-based Compute project, run fastly compute init in a new directory in your terminal and follow the prompts to provision a new service using the default Rust starter kit and with HTTPBin as a backend:

$ mkdir hibp_enrichment && cd hibp_enrichment
$ fastly compute init

Creating a new Compute project.

Press ^C at any time to quit.

Name: [hibp_enrichment]
Description: Check the HIBP API for pwned passwords and send enriched information to the origin
Author: My Name
Language:
[1] Rust
[2] JavaScript
[4] Other ('bring your own' Wasm binary)
Choose option: [1]
Starter kit:
[1] Default starter for Rust
    A basic starter kit that demonstrates routing, simple synthetic responses and
    overriding caching rules.
    https://github.com/fastly/compute-starter-kit-rust-default
[2] Beacon termination
    Capture beacon data from the browser, divert beacon request payloads to a log
    endpoint, and avoid putting load on your own infrastructure.
    https://github.com/fastly/compute-starter-kit-rust-beacon-termination
[3] Static content
    Apply performance, security and usability upgrades to static bucket services such as
    Google Cloud Storage or AWS S3.
    https://github.com/fastly/compute-starter-kit-rust-static-content
Choose option or paste git URL: [1]

✓ Initializing...
✓ Fetching package template...
✓ Updating package manifest...
✓ Initializing package...

This will create some files for you, which you'll need to edit as you go through the rest of the tutorial:

• fastly.toml describes your project and tells the Fastly CLI where to deploy it to.
• Cargo.toml is the Rust package manifest, where you declare your dependencies.
• src/main.rs is the source code of your project. This will have some example code in it, which you can remove.

Create a Fastly Compute service

Create your new Compute service. Make a note of the service ID that is returned from the following command.

$ fastly service create --name my-enrichment-demo

Add the service ID into the fastly.toml file:

# This file describes a Fastly Compute package. To learn more visit:
# https://www.fastly.com/documentation/reference/compute/fastly-toml/

authors = [...]
description = "Check the HIBP API for compromised passwords and send enriched information to the origin"
language = "rust"
manifest_version = 1
name = "hibp_enrichment"
service_id = "[your_service_id]"

Configure the backends

You'll need two backends for this demo: the HIBP API and your own origin server that will answer the user's request. We'll use httpbin.org as a stand in for our backend, but feel free to substitute your own. You can add these using the fastly backend create command:

$ fastly backend create --name=api --address=api.pwnedpasswords.com --port=443 --use-ssl --service-id=[your_service_id] --version=latest
$ fastly backend create --name=primary --address=httpbin.org --port=443 --use-ssl --service-id=[your_service_id] --version=latest

Write the code

Dependencies

Add these dependencies to your Cargo.toml file:

fastly = "0.10.2"
sha1_smol = "^1"
percent-encoding = "^2.2.0"

Afterwards, import them at the top of src/main.rs. If you haven't already, remove all the existing content of main.rs and replace it with the following:

use fastly::http::{Method, StatusCode};
use fastly::{mime, Error, Request, Response};
use sha1::{Digest, Sha1};
95 lines
 
 

The fastly dependency provides the SDK for the Fastly platform; percent-encoding decodes the percent-encoded password that the user submits in the login form POST; and sha1 to performs the hashing function necessary to be compatible with the HIBP API.

Set up the backends and constants

You created two backends on the Fastly service, called api and primary. Since they are referenced as strings in Rust, assigning them to constants will help to avoid typos later:

4 lines
 
 
// The name of the backend servers associated with this service.
// This must match the backend names you configured using `fastly backend create`.
const BACKEND_APP_SERVER: &str = "primary";
const BACKEND_SECURITY_CHECK: &str = "api";

// Credential prefix length
const PREFIX_LENGTH: usize = 5;

// Login form HTML
static LOGIN_HTML: &str = r#"<!DOCTYPE html>
<html lang="en">
  <head>
    <meta charset="utf-8" />
    <title>Compromised password detection demo</title>
  </head>
  <body>
    <form action="/post" method="post">
      <div class="container">
        <label for="username"><b>Username</b></label>
        <input type="text" placeholder="Enter Username" name="username" required />

        <label for="password"><b>Password</b></label>
        <input type="password" placeholder="Enter Password" name="password" required />

        <button type="submit">Login</button>
      </div>
    </form>
  </body>
</html>"#;
65 lines
 
 

Add helper functions

To find a password in the request, you'll need to be able to parse the request body and extract a field by name. In VCL, we have the subfield function, but there's no equivalent in Rust. You can use a struct to deserialize the password instead:

34 lines
 
 
// Struct to deserialize password field
#[derive(serde::Deserialize)]
struct BodyParams {
    password: Option<String>,
}
59 lines
 
 

You'll also need to be able to compute a SHA1 hash of the password for the HIBP API. The SHA1 crate does that but it's helpful to have an easy way to get it as a string:

40 lines
 
 
/// Generate SHA1 hash from string s
fn hash_sha1(s: &str) -> String {
    let mut hasher = sha1_smol::Sha1::new();
    hasher.update(s.as_bytes());
    hasher.digest().to_string()
}
52 lines
 
 

Fetch enriched data from the API

A Compute program in Rust receives a Request. Since the objective here is to create an improved (enriched) request, a good signature for the enrichment function would be Request -> Result<Request, Error>. This enables the enrichment logic to be nicely encapsulated and can be invoked elegantly as part of processing the incoming request, in conjunction with other similar handlers.

The function will therefore take a Request, add a Fastly-Password-Status header to it, and then return it to the calling scope.

65 lines
 
 
// Process login with threat check
fn process_credential(mut req: Request) -> Result<Request, Error> {
    let params = req.take_body_form::<BodyParams>().unwrap();
    if let Some(plain_cred) = params.password {
        // Generate sha1 hash of credential
        let hashed_cred = hash_sha1(&plain_cred);

        // Split the hash of credential to left and right part at position PREFIX_LENGTH
        let hash_left = &hashed_cred[0..PREFIX_LENGTH];
        let hash_right = &hashed_cred[PREFIX_LENGTH..];

        // Prepare the request for threat check
        // (If you use HIBP in production please use an API key)
        let api_url = format!("https://api.pwnedpasswords.com/range/{hash_left}");
        let api_req = Request::get(api_url);

        // Send threat check request to API with the left-hand-side of the SHA1 hash
        let mut api_res = api_req.send(BACKEND_SECURITY_CHECK)?;
        let api_res_body = api_res.take_body_str();

        // Check if the response body contains the right-hand-side of the sha1 hash
        let result = if api_res_body.contains(hash_right) {
            "compromised-credential"
        } else {
            "safe-credential"
        };

        // Uncomment for debugging. For production use, avoid logging credentials
        // println!("Checked credential {plain_cred}, result is {result}");
        req.set_header("fastly-password-status", result);
    }
    Ok(req)
}

First, take the request body from the request and use the helper function defined earlier to search it for a credential. For the purposes of this tutorial, we'll assume that the body is application/x-www-form-urlencoded and that the field name we want is always password, so a body that would match would be username=Jo&password=123456. Special characters such as "!" and "$" are often used in passwords, but these special characters will be percent-encoded when a user submits a credential for this tutorial. Therefore, we must decode the percent-encoded password before interacting with the HIBP API.

If there's no credential found in the body, it would be very inefficient to make an unnecessary API request, so in this case, you can create a fast path by adding a Fastly-Password-Status: no-credential header and returning immediately.

Where a credential is found, the other helper function defined earlier can be used to compute a SHA1 hash as a 40 character string. The HIBP API takes a 5-character prefix of that as an input, so divide the hash into a 5-character hash_left and a 35-character hash_right. The request to the API will return a list of 'right hand sides' of all hashes in the database that start with the supplied 'left hand side'. It's then easy enough to check whether hash_right is in the list, and if so, conclude that the credential is compromised.

Use the enrichment function

The entry point for a Compute program is the main function. A simple scenario here is to pass every request directly to a backend, and then to return whatever the backend responds with. You need only make a small modification to this - insert a call to the enrichment function, which will modify the request, before you send it to the origin.

47 lines
 
 
#[fastly::main]
fn main(mut req: Request) -> Result<Response, Error> {
8 lines
 
 
    // Pass all requests through the credential detection, which
    // modifies the request to enrich it with new information
    req = process_credential(req)?;

    // Send request to the primary origin
    Ok(req.send(BACKEND_APP_SERVER)?)
}
34 lines
 
 

Commonly, backends require that the Host header sent in the backend request matches the hostname of the backend. Fastly doesn't modify the Host header by default, so you likely also want to do this.

You now have a complete Compute program, which receives a Request, enriches it, forwards it to an origin, and then uses the returned Response to reply to the client.

Add a login page

Normally, your backend (the primary backend here) would serve pages that would invite a user to submit a password somehow. But since HTTPBin (the primary backend we are using in this tutorial) doesn't do that, you could, as a convenient way to test the demo, add a pre-canned login page to the application, and store it in your Compute program. Start by creating a login.html page in the src/ directory:

Then add a section to the main() function to intercept GET requests to the / path and return the login page instead of forwarding the request to origin.

47 lines
 
 
#[fastly::main]
fn main(mut req: Request) -> Result<Response, Error> {

    // For the demo, serve a basic login form on the root path
    if req.get_method() == Method::GET && req.get_path() == "/" {
        return Ok(Response::from_status(StatusCode::OK)
            .with_content_type(mime::TEXT_HTML_UTF_8)
            .with_body(LOGIN_HTML));
    }

    // Pass all requests through the credential detection, which
    // modifies the request to enrich it with new information
    req = process_credential(req)?;

    // Send request to the primary origin
    Ok(req.send(BACKEND_APP_SERVER)?)
}
34 lines
 
 

Build and deploy

Congratulations! You now have a mechanism to see if the submitted credentials are part of the known compromised credentials.

$ fastly compute publish
✓ Initializing...
✓ Verifying package manifest...
✓ Verifying local rust toolchain...
✓ Building package using rust toolchain...
✓ Creating package archive...

SUCCESS: Built rust package hibp-enrichment (pkg/hibp-enrichment.tar.gz)


There is no Fastly service associated with this package. To connect to an existing service
add the Service ID to the fastly.toml file, otherwise follow the prompts to create a
service now.

Press ^C at any time to quit.

Create new service: [y/N] y

✓ Initializing...
✓ Creating service...

Domain: [random-funky-words.edgecompute.app]

Backend (hostname or IP address, or leave blank to stop adding backends): api.pwnedpasswords.com
Backend port number: [80] 443
Backend name: [backend_1] api

Backend (hostname or IP address, or leave blank to stop adding backends): httpbin.org
Backend port number: [80] 443
Backend name: [backend_1] primary

Backend (hostname or IP address, or leave blank to stop adding backends):

✓ Initializing...
✓ Creating domain 'random-funky-words.edgecompute.app'...
✓ Creating backend 'api' (host: api.pwnedpasswords.com, port: 443)...
✓ Creating backend 'primary' (host: httpbin.org, port: 443)...
✓ Uploading package...
✓ Activating version...

Manage this service at:
        https://manage.fastly.com/configure/services/PS1Z4isxPaoZGVKVdv0eY

View this service at:
        https://random-funky-words.edgecompute.app


SUCCESS: Deployed package (service PS1Z4isxPaoZGVKVdv0eY, version 1)

Try it out

Navigate to the URL shown under "View this service at" in the output above, and you should see the login page. When you log in, your request will be forwarded to HTTPBin, which simply echos back to you what it received. This enables you to see that the origin server has received an additional header with the credential in it.

Try it with no password, with the password '123456', and with something strong and random. You should be able to trigger all three of the possible values of Fastly-Password-Status.

Next Steps

Now that you understand how to use an API request to enrich data that is sent to the origin, you could combine this with other Fastly sources such as proxy description to gain visibility into if a given client is coming from a proxy. You could also add API requests to other 3rd party sources or your own sources, and perform them in parallel.