--- title: Direct Fetch for Fastly Object Storage summary: null url: >- https://www.fastly.com/documentation/guides/platform/object-storage/direct-fetch-for-object-storage --- Direct Fetch for Fastly Object Storage lets you transfer data from a remote source directly into Fastly Object Storage without first downloading it to the client. When a client sends a Direct Fetch request: 1. It sends a `PutObject` or `UploadPart` request with the remote object's URL in the `fastly-object-storage-source-url` header. 2. Fastly Object Storage fetches the data from the provided URL. If the request includes a `Range` header, the same range is requested from the source. 3. Fastly Object Storage writes the fetched data to the destination object or multipart upload part and returns a standard `PutObject` or `UploadPart` response for the requested operation. Direct Fetch reduces transfer time and client bandwidth when copying objects from another storage provider or assembling a multipart upload from ranges of an existing object. ## Before you begin Make sure you review the [prerequisites](https://www.fastly.com/documentation/guides/platform/object-storage/about-object-storage/#prerequisites) for using Fastly Object Storage. ## Limitations and considerations Note the following limitations and considerations when using Direct Fetch for Fastly Object Storage. Direct Fetch requests have the following requirements: - Direct Fetch supports the `PutObject` and `UploadPart` operations. - The `fastly-object-storage-source-url` header must be included in the set of signed headers when generating the [AWS Sigv4 signature](https://docs.aws.amazon.com/AmazonS3/latest/API/sigv4-auth-using-authorization-header.html). If the request includes a `Range` header, it must be included in the signature as well. - A single Direct Fetch source response body cannot be greater than 5 GB. Use the multipart upload API to transfer a larger object in parts. - A maximum of 16 additional headers may be passed to the source. The remote source must meet the following conditions: - The source URL must be reachable by Fastly Object Storage. For an object that isn't publicly accessible, use a pre-signed URL that grants temporary access to the object or [pass authentication headers to the source](https://www.fastly.com/documentation/guides/platform/object-storage/direct-fetch-for-object-storage#passing-headers-to-the-source). - For a request without a `Range` header, the source must return `HTTP 200 OK`. Direct Fetch rejects an unsolicited `HTTP 206 Partial Content` response. - For a ranged request, the source must return either `HTTP 206 Partial Content` with a valid `Content-Range` header or `HTTP 200 OK`. - Multipart ranges are not allowed. - The source must not require more than one redirect. Direct Fetch handles transferred data differently than a standard upload: - The client request must have an empty body and a `Content-Length` header with a value of `0`. Fastly Object Storage obtains the object data from the source URL. - The same metadata restrictions apply to Direct Fetch requests as to standard `PutObject` or `CreateMultipartUpload` requests. For copying and overriding metadata, see [Working with object metadata](https://www.fastly.com/documentation/guides/platform/object-storage/direct-fetch-for-object-storage#working-with-object-metadata). - Object metadata such as `ETag` and `Last-Modified` for data migrated via Direct Fetch will not match the source `ETag` or `Last-Modified`. ## Using Direct Fetch with `PutObject` and `UploadPart` To use Direct Fetch for Fastly Object Storage, set the `fastly-object-storage-source-url` header on a standard `PutObject` or `UploadPart` request. Set the header to the URL of the object you want Fastly Object Storage to fetch, typically from a non-Fastly storage provider like AWS S3. If the object is in a private bucket, use a pre-signed URL to grant temporary, time-limited access without sharing the source account's credentials. > **IMPORTANT:** The `fastly-object-storage-source-url` header must be included in the set of signed headers when generating the [AWS Sigv4 signature](https://docs.aws.amazon.com/AmazonS3/latest/API/sigv4-auth-using-authorization-header.html). The `Range` header must also be signed when requesting a range from the source. Then, create a `PutObject` request to `https://fos-region.object.fastlystorage.app/my-fos-bucket/object-key` with an empty body and `Content-Length: 0`, where: - `fos-region.object.fastlystorage.app` is the [regional Object Storage endpoint](https://www.fastly.com/documentation/guides/platform/object-storage/working-with-object-storage/#managing-object-storage-buckets-and-objects) where you want to send the request. - `my-fos-bucket` is the name of your Object Storage bucket. - `object-key` is the name of the destination object in your Object Storage bucket. To store only part of the source object, add one supported HTTP byte `Range` header to the request. The resulting destination object contains only the selected bytes. ## Using Direct Fetch with multipart uploads To transfer an object through multipart upload, first send a standard `CreateMultipartUpload` request. Set any custom metadata and [HTTP metadata](https://www.fastly.com/documentation/guides/platform/object-storage/direct-fetch-for-object-storage#working-with-object-metadata) you want on the destination object on this request. Direct Fetch doesn't copy metadata from the source for multipart uploads. For each part, send an `UploadPart` request with the upload ID, part number, `fastly-object-storage-source-url` header, an empty body, and `Content-Length: 0`. You can also [pass additional headers to the source](https://www.fastly.com/documentation/guides/platform/object-storage/direct-fetch-for-object-storage#passing-headers-to-the-source) on each `UploadPart` request. > **IMPORTANT:** The `fastly-object-storage-source-url` header must be included in the set of signed headers when generating the [AWS Sigv4 signature](https://docs.aws.amazon.com/AmazonS3/latest/API/sigv4-auth-using-authorization-header.html). The `Range` header must also be signed when requesting a range from the source. You can add a different `Range` header to each `UploadPart` request to fetch specific sections of the source object. After all parts have been uploaded, send a standard `CompleteMultipartUpload` request with the part numbers and ETags returned by the `UploadPart` responses. ## Passing headers to the source To send additional headers to the source URL, include the `fastly-object-storage-source-header` header on your `PutObject` or `UploadPart` request. Use the format `fastly-object-storage-source-header: header-name=header-value`. Repeat this header for each header you want to send to the source, up to 16 per request. For example, if the source requires a bearer token for authentication and a custom `X-Tenant-ID` header to identify the tenant, include the following headers. Replace `YOUR_SOURCE_TOKEN` and `example-tenant` with values for your source: ```http fastly-object-storage-source-url: https://source.example.com/object-key fastly-object-storage-source-header: authorization=Bearer YOUR_SOURCE_TOKEN fastly-object-storage-source-header: x-tenant-id=example-tenant ``` Fastly Object Storage sends `Authorization: Bearer YOUR_SOURCE_TOKEN` and `X-Tenant-ID: example-tenant` to the source URL. > **NOTE:** Send each source header as a separate `fastly-object-storage-source-header: header-name=header-value` field. Do not combine entries into a comma-separated list. Fastly Object Storage treats commas as part of the source header's value, not as separators between entries. Your HTTP client and any intermediaries must preserve these as separate fields. ## Working with object metadata For `PutObject` requests, Direct Fetch copies the following HTTP metadata headers from the source: `Cache-Control`, `Content-Disposition`, `Content-Encoding`, `Content-Language`, `Content-Type`, and `Expires`. If you set any of these headers on the incoming `PutObject` request, that value replaces the corresponding source value on the destination object. To control how Direct Fetch handles custom metadata on a `PutObject` request, set the `x-amz-metadata-directive` header: | Value | Custom metadata written to the destination object | | ----------------- | ----------------------------------------------------------------------------- | | Not set or `COPY` | Copies `x-amz-meta-*` headers from the source. | | `REPLACE` | Uses only the `x-amz-meta-*` headers set on the incoming `PutObject` request. | The `x-amz-metadata-directive` header controls only custom metadata, not the HTTP metadata headers listed above. For example, to replace custom metadata and override the source's `Cache-Control`, include these headers on your Direct Fetch `PutObject` request: ```http x-amz-metadata-directive: REPLACE x-amz-meta-project: example-project Cache-Control: max-age=3600 ``` The destination object uses only `x-amz-meta-project` as custom metadata and has `Cache-Control: max-age=3600`. Direct Fetch copies the other HTTP metadata headers listed above from the source. ## Validating a transfer After a successful Direct Fetch `PutObject` request, send a `HeadObject` or `GetObject` request to the same region, bucket, and object key to confirm that the object exists. For multipart uploads, validate the object after the `CompleteMultipartUpload` request succeeds. ## Error handling Direct Fetch reports feature-specific errors through both an XML response body and a `fastly-object-storage-df-error` header, which contains a description of the error that occurred. For example, if the source server responds with an unsupported `HTTP 503 Service Unavailable` status code, the `fastly-object-storage-df-error` header will report `DirectFetchSourceStatus 503` and the response status itself will be `HTTP 400 Invalid Request`.