Storage Transfer Service — S3, Azure Blob, HDFS and POSIX sources, scheduled or event-driven, and the 1 TiB threshold

The data-movement service for object and file storage. The sources — including Amazon S3, Azure Blob Storage and on-premises POSIX and HDFS — the scheduled and event-driven modes, the agents file systems need, and the vendor's own statement about the size of transfer it is optimised for.

markdownreferencev1WZ6Z-Y1K6stage: deploytrust: officialfamily: migrationservice: storage-transfer-serviceinterface: anystorage transfer serviceAmazon S3S3-compatibleAzure Blob StorageData Lake Storage Gen2HTTP listHDFSPOSIX file systemtransfer agentevent-driven transferscheduled transfermetadata preservation1TiB

Created 21 Oct 2026 · Modified 23 Oct 2026 · 1 version

Storage Transfer Service

1. What it is, and the size it is for

"Storage Transfer Service enables seamless data movement across object and file storage systems."

Optimised for transfers exceeding 1 TiB, handling automation, scaling and network optimisation.

The 1 TiB figure is a useful threshold in both directions. Below it, a gcloud storage copy is usually simpler. Above it, the things this service does — retrying, parallelising, validating, resuming — are the things you would otherwise write badly.

2. The sources — and two of them are the other clouds

SourceNote
Amazon S3 and S3-compatible storagethe AWS path, and anything speaking the S3 API
Microsoft Azure Blob Storagethe Azure path
HTTP/HTTPS URLspublicly accessible, from a list
Cloud Storage bucketsbucket-to-bucket, including across locations
on-premises POSIX file systems and HDFSagent-based

HDFS as a first-class source matters for a Hadoop migration: the data moves with this service while the jobs move to Dataproc (reference/data/dataproc.md §5).

3. Scheduled, or event-driven

Both are supported. Event-driven transfers work from Cloud Storage, AWS S3, and Azure Blob Storage / Data Lake Storage Gen2.

That is the facility that turns a migration into a continuous mirror: the source keeps receiving writes and they keep arriving at the destination. Which is how a large object estate cuts over with a short window — the bulk moved on a schedule, the tail arriving by event, and the cut-over a DNS or configuration change rather than a copy.

4. Agents, for file systems

Transfers to and from on-premises file systems need self-hosted transfer agents, which exist to give you network routing control and bandwidth management.

Bandwidth management is the operationally important half. An unthrottled transfer out of a production data centre is an outage with a good explanation. Size and throttle the agent fleet as part of the plan.

5. What is and is not stated

The overview mentions metadata preservation and data integrity validation. It does not detail delete-after-transfer or overwrite behaviour, so neither is described here.

Both of those are destructive-adjacent settings and must be read from the transfer-job configuration reference before a job runs. A mis-set overwrite or delete option on a multi-terabyte transfer is not a recoverable mistake.

6. For an estimate

LineNote
the transfer itselfthis service
source-side egresscharged by the source cloud. On a large estate this is frequently the single largest cost in the project, and it is not on any Google Cloud price list
destination storage classchoose it knowing the minimum storage durations — ARCHIVE commits 365 days (reference/storage/cloud-storage.md §1). Landing a migration directly in a cold class starts that clock
validationbudget time to verify, not just to copy

The egress row is the one to raise first with anyone asking for a number. See guides/parity-and-migration-method.md, which treats egress as a first-class line.

Read this document as markdown — the same bytes an agent receives.