S3 and S3-compatible storage: what carries over and what breaks
Using the AWS SDK against Cloudflare R2, Backblaze B2 or another S3-compatible store alongside or instead of Amazon S3 — the client settings that must change (endpoint, region, checksum defaults), the S3 features those stores do not implement (SSE-KMS, ACLs, Object Lock, tagging, IAM roles), and the failures each mismatch produces.
markdownpatternv47058-MKTDstage: buildtrust: officialtrust: fieldservice: s3service: external-storageinterface: sdkinterface: cliS3 compatibleCloudflare R2Backblaze B2S3 alternativerequestChecksumCalculationresponseChecksumValidationWHEN_REQUIREDX-Amz-Checksum-ModeSignatureDoesNotMatch S3-compatibleregion autocustom endpointforcePathStyleSSE-KMS not supportedACL not supportedegressmigrate from S3
S3 and S3-compatible storage
The AWS SDK talks to S3-compatible stores if you change three things: the endpoint, the region value the store expects, and the checksum defaults. What doesn't carry over is everything around the API: IAM roles and policies, KMS encryption, ACLs, Object Lock, event notifications, and the rest of the AWS integrations. Design for the subset you will use on both.
What crosses the boundary
| Concern | Amazon S3 | S3-compatible store |
|---|---|---|
| Credentials | IAM roles, temporary credentials | The store's own access keys (B2 lists IAM roles as unsupported) |
| Endpoint and region | Regional endpoint from the bucket's Region | Custom endpoint. R2: https://<ACCOUNT_ID>.r2.cloudflarestorage.com with region auto (empty or us-east-1 also map to auto) |
| Checksums | SDKs since December 2024 send and validate CRC checksums by default (WHEN_SUPPORTED) | May reject or ignore the new headers. R2: only CRC64NVME supports full-object checksums, and x-amz-checksum-algorithm is unimplemented for CopyObject |
| Encryption | SSE-S3 default, SSE-KMS, SSE-C (blocked by default on new buckets since April 2026) | R2: SSE-C only. B2: its own managed keys (SSE-B2) or SSE-C |
| Access control | Bucket policies, Block Public Access, ownership, access points | R2: no ACL headers or ACL APIs. B2: no object-level ACLs |
| Other features | Object Lock, tagging, website hosting, events → Lambda/SQS/SNS/EventBridge | R2: no Object Lock. B2: no object tagging, no website configuration, no browser POST to presigned URLs |
| Presigned URLs | Yes | B2: GET/PUT yes, POST form uploads no |
Known-good client
import { S3Client, PutObjectCommand, GetObjectCommand } from "@aws-sdk/client-s3";
// One factory, two kinds of endpoint: real S3, or an S3-compatible store.
export function makeS3(target: "aws" | "compatible"): S3Client {
if (target === "aws") return new S3Client({ region: "<bucket region>" });
return new S3Client({
region: "auto", // R2 expects "auto"; other stores document their own value
endpoint: "https://<ACCOUNT_ID>.r2.cloudflarestorage.com",
credentials: { accessKeyId: "<key id>", secretAccessKey: "<secret>" }, // the store's own keys, not AWS ones
requestChecksumCalculation: "WHEN_REQUIRED", // don't send the new default checksum headers
responseChecksumValidation: "WHEN_REQUIRED", // don't ask for checksums the store may not return
forcePathStyle: true, // many compatible stores prefer path-style URLs
});
}
const s3 = makeS3("compatible");
await s3.send(new PutObjectCommand({ Bucket: "<bucket>", Key: "hello.txt", Body: "hi" })); // no SSE-KMS, no ACL headers
const got = await s3.send(new GetObjectCommand({ Bucket: "<bucket>", Key: "hello.txt" }));
console.log(await got.Body?.transformToString());
- The two checksum options exist on the JavaScript v3 client (checked against its types). The same switches are
request_checksum_calculation/response_checksum_validationin~/.aws/config, orAWS_REQUEST_CHECKSUM_CALCULATION/AWS_RESPONSE_CHECKSUM_VALIDATION. Those also cover the CLI and other SDKs. - Whether
forcePathStyleis needed depends on the store (UNVERIFIED for R2 and B2; not stated on their pages). Use what the store documents. - Set
endpointandregiontogether, deliberately. When both are set, the SDK sends requests to the endpoint (SDK maintainer, issue 5081). Whether the region still matters for signing depends on the store (UNVERIFIED). A wrong value usually shows asSignatureDoesNotMatchor anAuthorizationHeaderMalformed … region is wrongerror.
Known-bad
| Configuration | Symptom |
|---|---|
| Current SDK defaults against a store that doesn't know the checksum headers | SignatureDoesNotMatch on downloads (the X-Amz-Checksum-Mode: ENABLED header the default adds), or rejected uploads. Seen in the field with the Go SDK v2 s3 v1.73.0 default change; fixed with WHEN_REQUIRED |
ServerSideEncryption: "aws:kms" or bucket-key headers sent to R2 | Not implemented. Encryption at rest is the store's own; SSE-C is the only request-level option |
ACL: "bucket-owner-full-control" copied from S3 code | Not implemented on R2 or B2 (object ACLs) |
| Code that relies on S3 event notifications to trigger Lambda | Nothing fires: the store has its own event mechanisms, or none |
| Object Lock / legal holds for compliance on R2 | Not implemented |
IAM role credentials (fromNodeProviderChain) used against B2 or R2 | Authentication fails: use the store's keys |
| Browser POST form upload to a B2 presigned URL | Not supported: use presigned PUT |
| Region left as an AWS Region for R2 | Use auto (us-east-1 is also accepted) |
Choosing
- Stay on S3 when you need IAM-native access, KMS keys, Object Lock, or S3 event integrations with Lambda/SQS/EventBridge (
patterns/s3-to-lambda.md), or CloudFront OAC (patterns/cloudfront-to-s3.md). - Consider a compatible store for data mostly served out to the internet, or kept outside AWS. Compare transfer and request pricing on each provider's pricing page; this hub doesn't state prices.
- Using both: keep one client factory as above, and keep S3-only features out of shared code paths.