The Guardian: Our day hacking with Compute

Choosing a project:

The key to a successful hackathon is choosing the right project that allows for creative play while still being achievable within a short timeline.

This was my first experience with Compute, so I was able to approach the experience with fresh eyes. Before writing any code, I pondered what to create. 

I decided I wanted to help provoke interest and attract the eye of the Guardian reader. 

The Guardian generates thumbnails dynamically for articles. You’ve probably seen them on article previews when you send a link via social media or private message. In their current form, the Guardian logo is set on top of the article preview image, sometimes with a little extra information.

The resulting idea was to write a program running on Compute that pre-processes the article thumbnail to add richer information that really stands out to people seeing previews for our articles on social media.

Setting up the backend:

The first thing I needed to figure out was how to fetch a thumbnail, title, and additional information about any article on the Guardian upon request. I could then use this to draw my article thumbnails.

At the Guardian, we maintain a service known as CAPI (Content API). This is an open resource that allows anybody to request article content and associated metadata in a handy JSON object. 

Accessing this JSON object involves appending “.json” to any article URL on the Guardian. In return, you receive an object with loads of information! Here’s a small sample of the wealth of information the API returns: 

{
  "config": {
    "isDotcomRendering": false,
    "page": {
      "isSplash": false,
      "isColumn": false,
      "headline": "Shakespeare Trilogy review — Donmar's phenomenal all-female triumph",
      "isImmersive": false,
      "author": "Lyn Gardner",
      "toneIds": "tone/reviews",
      "ipsosTag": "stage",
      "isProd": false,
      "membershipAccess": "",
      "allowUserGeneratedContent": false,
      "commissioningDesks": "uk-culture",
      "webPublicationDate": 1479904472000,
      "forecastsapiurl": "/weatherapi/forecast",
      "commercialBundleUrl": "https://assets-code.guim.co.uk/javascripts/commercial/lb96400cb0c427510fab/graun.standalone.commercial.js",
      "idOAuthUrl": "https://oauth.code.dev-theguardian.com",
      "supportUrI": "https://support.theguardian.com",
      "isNumberedList": false,
      "webTitle": "Shakespeare Trilogy review — Donmar's phenomenal all-female triumph",
      "...": "And lots more!"
    }
  }
}

This API contained more than enough information for my thumbnails. My next step was to write a program to request content from a given article to deploy on Compute. I started with a quick-start Rust template to set up a simple web server in Rust and deploy it to Fastly to begin testing my program. 

I picked a random article and wrote a network request to our Content API to fetch information. To test that it was working correctly, I also returned the result back as the result.

There was a problem though: It worked fine locally, but once deployed to Compute, my program wasn’t able to make any GET requests to fetch the article content from the Guardian. By diving into the Fastly Developer Hub, I learned how to set up a backend to proxy requests to external services and cache the response from the origin.

After spending some time looking through the docs, I discovered that I could register my backends as hosts through the Fastly Compute control panel. Here’s what that configuration looks like for the main website and the Guardian image service: 

After setting up the backend in the control panel, I could then reference it in my fastly.toml configuration file:

Then I needed to specify the “news” backend in my application code when making the fetch request for the article content:

// Tell Fastly that we would like to send the request
// to our "prod" backend defined in fastly.toml
const BACKEND_NAME: &str = "news";

// Run the request and store the JSON body as a string
let mut article_response = page_req.send(BACKEND_NAME)?;
let article_body_string = article_response.take_body().into_string();

Et voila! I was now able to successfully request information for any article I wanted through our API!

Fetching and editing the thumbnail image:

Next up, I had to fetch the thumbnail image. I added a new Fastly backend named “guim” and then used a GET request against the thumbnail, sending it through the newly defined backend.  

// Extract the thumbnail URL from the article JSON.
let mut thumbnail_url = article_json["config"]["page"]["thumbnail"]
    .to_string()
    .replace("\"", "");

// Now we construct a GET request to fetch the thumbnail image
let thumb_req = Request::get(thumbnail_url);

// Send the request using the guim backend and store the binary image data
let mut thumb_resp = thumb_req.send("guim")?;
let thumb_resp_body = thumb_resp.take_body();

I also needed a library to decode the image into a format that I could manipulate to render the new thumbnails. The Rust community provides an image processing library that can decode the JPEG content into an object that I could use in my program.

// Decode the JPEG data into a standardised format.
let mut thumbnail_decoder = JpegDecoder::new(thumb_resp_body)?;

// Instantiate a new DynamicImage using the decoded JPEG
// We are able to freely manipulate this image to draw
// text, images and shapes on our custom thumbnail.
let mut image = DynamicImage::from_decoder(thumbnail_decoder)?.to_rgba8();

A small detail that I got stuck on for a bit when I first imported the library was that WASM doesn’t support multithreading. This meant the standard build did not run successfully when attempting to decode the JPEG image. I found that disabling the “rayon” parallelism library when importing the image package allowed the library to run successfully when compiled to WASM.

image = { version = "0.23.14", default-features = false, features = ["gif", "jpeg", "ico", "png", "pnm", "tga", "tiff", "webp", "bmp", "hdr", "dxt", "dds", "farbfeld"] }

With the thumbnail loaded into memory in a modifiable form, it was time to start thinking about how to programmatically edit the image to fit my thumbnail design ideas.

To do this, I needed to find an image manipulation library that supported text rendering and graphical primitives to match the design. It also needed to compile properly to WASM. imageproc written in Rust fit the bill perfectly. With everything in place, it was time to start the fun stuff!

Time to put it all together:

First, I set out to draw the distinctive Guardian logo on the top-right of the thumbnail.

Because I already loaded the thumbnail image, I had all of the packages and code needed to extend the program to load the png file of the logo and draw it as an overlay on top of the thumbnail image:

let graun_png = include_bytes!("graun_new.png");
let graun_loaded = load_from_memory_with_format(graun_png, ImageFormat::Png)?;

Next, I rendered the article headline and tagline over the top of the thumbnail image. This required loading the Guardian font to achieve our distinctive look.

After getting hold of the font, imageproc made it a breeze to load in the font file and render the article title:

let font = Vec::from(include_bytes!("GuardianTextEgyptian-Regular.ttf") as &[u8]);
let font = Font::try_from_vec(font).unwrap();
 
draw_text_mut(
   &mut image,
   text_colour,
   5, // x
   5, // y
   scale,
   &graun_font,
   &headline,
);

 I also drew a large rectangle underneath to provide better contrast than simply overlaying the text on top of the image:

let title_rect = Rect::at(0, 0).of_size(500, 80);
draw_filled_rect_mut(&mut image, title_rect, graun_blue);

Here’s what my tagline and headline looked like so far:

Next, I needed to fit headlines into the header box. I used the modulo operator to split the headline up into chunks of 5-6 characters, which I could render sequentially to fit most headlines into the space available. 

After implementing this change, the text-wrapping situation was dramatically improved. (comparison below).

Finally, I went to work including the kicker on the thumbnail. 

In the context of a newspaper, kickers are intended to provoke interest and attract the eye of the reader. On The Guardian website, article cards use these to great effect, so I wanted to apply the same to my new thumbnails. 

I shifted the headline down to create space to draw the kicker text in a bright color that stood apart from the headline.

With that, I had a thumbnail that stood out on the page and utilized our Content API to great effect.

Final thoughts on the hack day

The hack day was really great overall. I was happy that I didn’t choose a complex project with a huge scope. It was a simple, yet complex enough concept to really give me a taste of what is possible using the Compute platform. The Fastly engineering team were available during the whole day to help us with anything we needed, which was really helpful whenever I had a problem using the platform.